fehzan
March 25, 2023, 11:20pm
1
I submitted for the waitlist on March 14th, but I’m itching to gain access to the API so I can use it for some live projects. I’m not sure if there is any other step I can take to gain access - I am already a ChatGPT Plus member and a paying OpenAI user already utilizing GPT3.5-Turbo for scripts I’m writing to test the waters.
Thanks in advance!
I’ve heard they’re looking for people to help with evals.
# Evals
Evals is a framework for evaluating OpenAI models and an open-source registry of benchmarks.
You can use Evals to create and run evaluations that:
- use datasets to generate prompts,
- measure the quality of completions provided by an OpenAI model, and
- compare performance across different datasets and models.
With Evals, we aim to make it as simple as possible to build an eval while writing as little code as possible. To get started, we recommend that you follow these steps **in order**:
1. Read through this doc and follow the [setup instructions below](README.md#Setup).
2. Learn how to run existing evals: [run-evals.md](docs/run-evals.md).
3. Familiarize yourself with the existing eval templates: [eval-templates.md](docs/eval-templates.md).
4. Walk through the process for building an eval: [build-eval.md](docs/build-eval.md)
5. See an example of implementing custom eval logic: [custom-eval.md](docs/custom-eval.md).
If you think you have an interesting eval, please open a PR with your contribution. OpenAI staff actively review these evals when considering improvements to upcoming models.
____________________
🚨 For a limited time, we will be granting GPT-4 access to those who contribute high quality evals. Please follow the instructions mentioned above and note that spam or low quality submissions will be ignored❗️
show original
Good luck…
2 Likes
fehzan
March 27, 2023, 7:22pm
3
Thanks, I am in! Appreciate the response.
Can you please elaborate on your experience since last you wrote here?
What did you mean by “I am in”? …only, that you were up to help with evals, right?
In the meanwhile, did you already get access to the API?
Me too - been on the waiting list for the GPT4 API since March 16th. I’m an AI dev so you can imagine how excited I am to get access, especially since I have plans to develop dev tools that use AI to work with entire repos. Any powerful people around who can help??
