I have enabled data sharing for it, which is said to be processed at no cost
But I was charged, even if the monthly budget was exceeded
Welcome!
Did you see the limits?
Through the end of 2024, evaluations you share with OpenAI will be processed at no cost (up to 7 per week).
I did see this, but I’m not sure what “7” represents. Is it 7 requests (model call) a week or 7 evaluations?
That was the first evaluation I run.
1 Like
Any update on this? 'cuz this is important for me
I’d reach out to help.openai.com maybe?
I’m not sure. Not heard of anyone else being charged or going over.
Let us know if you find out!
I can’t find any help, that’s why I’m here to feedback
this still charges you
the way it works on the local repository, is that you get charged per api request.
I found the post that announced here in the community about it…
as you can see it says
Through the end of the year, developers can run evaluations for free (up to 7 per week) when they opt in(opens in a new window) to share their Evals with OpenAI. Evals shared with us will be used to help us improve and evaluate our future models.
maybe it has to do with the model used?
also, it seems it says:
2M free training tokens per day on GPT-4o mini and 1M free training tokens per day on GPT-4o until October 31
so that covers a couple of requests, you can check tokenizer to count your requests I guess. I’m sure there’s some python packages for this
you can reply on that community post to see if you get a reply there or search for help.openai.com, like @paulbellow said if you think there is something wrong
1 Like
maybe it has to do with the model used?
1 Like
I was not training a model. I was just evaluating it.
oh, you are right. On the playground evals there is no way to select which model you’ll be using. Did the cost show up on your https://platform.openai.com/usage ?
1 Like
I was at https://platform.openai.com/evaluations, which is not a part of playground, but a part of dashboard.
There is place to select which model to use, when creating new evaluations.
Did the cost show up on your https://platform.openai.com/usage ?
Yes. All requests in that evaluation run showed up, including generation and evaluation. i.e., all evaluations are charged.
1 Like
Thank you. I’m sorry that I don’t know I should leave messages there instead of community.
1 Like
No problem. Thanks for pointing it out. I’m sure someone somewhere in the ever-growing organization that is OpenAI wants to know so they can fix the problem.
Good luck. Please come back to let us know if you get it sorted.
In the meantime, we’ve got a wealth of other information here.
all good, you did it right, its always best to check if there were any previous post and if not create a new one, considering others might have the same issue in the future. you did it just right.
i’ve only interacted with evals on github so far, so its something quite fresh and new. like @paulbellow said, if you can, come back to let us know when its fixed and if some time has passed, feel free to bump this post
1 Like