SOS: ALARMING Situation of Excessive Billing Threatening the Survival of my Company AI Project GPT

API
1

Dear members of the OpenAI community,

I am Yacine, the founder of the AI Project GPT application. I am writing to you today in a desperate and alarming situation, as the very survival of my company is threatened by an issue of excessive billing with the OpenAI API. I urgently need your help.

Our AI Project GPT application, currently serving approximately 300 active users per day, has recently been hit with an uncontrollable surge in costs associated with using the OpenAI API. While we initially had manageable fees, ranging from 1 to 2 dollars per day, these costs have abruptly skyrocketed to over 20 dollars per day, sometimes even more.

Faced with this critical situation, I have taken several actions to try to resolve this issue. I have reached out repeatedly to OpenAIā€™s customer service, but unfortunately, I have received no response. I have also explored different strategies, such as creating multiple OpenAI accounts, frequently changing APIs, and even securing our Firebase database. Unfortunately, none of these measures have been able to stem the tide of rising costs.

The situation is alarming, and the very survival of my company is at stake. Without a swift solution, I am forced to seriously consider migrating to alternatives such as Google Gemini or Meta as soon as they become available in France.

I implore you, members of the OpenAI community, could a member of the team provide urgent assistance to resolve this issue of excessive billing? The future of my company depends on it, and every day that passes worsens our financial situation.

I am deeply disappointed by the lack of responsiveness from OpenAIā€™s customer service, and I am making this plea for help in the hope that a kind-hearted member of the team can provide us with a solution.

Thank you from the bottom of my heart for your attention and support in this difficult time.

With sincere regards,

Yacine, Founder of AI Project GPT

1 Like
2

Capture dā€™eĢcran 2024-03-05 aĢ€ 16.33.14
Capture dā€™eĢcran 2024-03-05 aĢ€ 16.33.141981Ɨ1421 214 KB

1 Like
3

This is really not a decent way to appeal for support. Either you do, or you donā€™t. Thatā€™s up to you and really doesnā€™t make the impact you think it does.

You have provided absolutely no information.

If you want to keep it completely private, thatā€™s fine. You have done what you can - which is contact the support. Hopefully they get back to you.

If you want to post some information maybe we can help you here and others can learn.

This is a community forum. Although staff do appear here, it is almost near-impossibly rare for them to tackle a billing issue.

5 Likes
5

Capture dā€™eĢcran 2024-05-01 aĢ€ 20.33.32
Capture dā€™eĢcran 2024-05-01 aĢ€ 20.33.321920Ɨ984 85.6 KB

6

Based on the variety of models I am guessing you are exposing your API key and have essentially been hacked.

Turn off your billings and figure out whatā€™s going on.

Inb4: ā€œNo itā€™s encryptedā€ :sob:

1 Like
7

I appreciate your candid feedback. Youā€™re right in pointing out that threatening to migrate to alternatives may not be the most respectful way to seek support.

To clarify, my intention was to highlight the urgency of the situation and the severe impact itā€™s having on the viability of my company. I didnā€™t mean to pressure the OpenAI community but rather to seek solutions to a critical issue affecting my business.

Regarding your observation about the lack of information provided, I understand your perspective. In reality, Iā€™ve been attempting to contact OpenAIā€™s customer service for the past three months without success. Iā€™m frustrated by this situation, which led me to share my challenges on this forum in the hopes of finding a solution with the communityā€™s assistance.

Thank you for understanding and for sharing your perspective. If you have any advice or ideas on how I could address this issue differently, Iā€™m all ears.

Best regards,

1 Like
8

I get it. The support is notoriously bad. From long wait times, to completely irrelevant responses. I imagine they are completely overwhelmed.

Unfortunately your best bet is to turn off billing and have your current software audited. A very rough guess is that your API key has been exposed and people are using it freely.

In the meantime Iā€™m sure support will get back to you and provide some assistance.

1 Like
9

I understand your concern, and I appreciate your input. Iā€™ve actually taken steps to address potential security vulnerabilities by changing my API key multiple times. Despite these efforts, the issue of excessive billing persists, which is why Iā€™m reaching out for assistance.

If you have any suggestions or insights on how I can further secure my API key or if there are other potential causes of the problem that I may not have considered, I would greatly appreciate your input.

Thank you for your concern and for taking the time to offer your perspective.

Best regards,

1 Like
10

Are you using the variety of models that your billing page reflects?

Itā€™s really hard to say without knowing what your service is and how you are hosting it. If youā€™re willing to share more information here thereā€™s lots of very smart people that can help out.

Unfortunately the best bet is to just remove your API keys, turn off billing, and temporarily take your service down. (Or just keep it open for a select trusted people)

1 Like
11

Thank you for your understanding and for sharing your insights. I agree that the support situation can be challenging, and I appreciate your suggestion to temporarily turn off billing and conduct a thorough audit of my current software.

Regarding the security measures in place, Iā€™ve implemented rules in Firebase similar to the ones you provided to secure access to the database and API keys. Hereā€™s the rule set Iā€™m currently using:

json

Copy code

{
  "rules": {
    ".read": "auth != null",
    ".write": "auth != null",
    "apiKeys": {
      ".read": false,
      ".write": false
    },
    "users": {
      "$uid": {
        ".read": "$uid === auth.uid",
        ".write": "$uid === auth.uid"
      }
    }
  }
}

Iā€™ve ensured that only authenticated users can read and write to the database, and API keys are restricted from being accessed or modified. However, despite these measures, the issue persists.

Iā€™ll proceed with your suggestion to turn off billing temporarily and conduct a thorough audit of my software. Hopefully, with supportā€™s assistance and your valuable advice, we can identify and resolve the root cause of the problem.

Thank you once again for your support and guidance.

Best regards,

12
  1. Can you identify which users are incurring the usage, by your use of a gateway that handles user authentication and logs and processes their AI queries through to OpenAI, with only your backend making API calls after moderation on the inputs?

orā€¦

  1. Did you place API keys into the application so the application makes direct calls to OpenAI, allowing numerous methods of extracting your API KEY credentials and directly abusing your account.

Logging will allow you to identify the consumption of tokens and the patterns of code or usage that incur high costs.

With use of ā€œAssistantsā€, especially placing files into it for knowledge or with code interpreter, you have little control over usage performed by the backend.

Do not employ assistants unless you want un-billable usage:
I have an assistant with one 315 token file in a vector store. My second chat input to a thread, I type a 86 token question just now. The bill for that is 4827 tokens for one query.

Do not write applications that directly use OpenAI endpoints.

13

Your database rules seem ok. Iā€™m a little skeptical of the apiKeys entry and what it means but seems fine.

Are you running these OpenAI functions in Cloud Functions in Firebase? Or are you having the client run the code?

I would also be very weary of using ChatGPT for these kind of situations. Itā€™s great for a couple chats but once you dig deep with it, it can seriously start hallucinating and be ā€œinfluencedā€ to make strange decisions.

14

To be honest with you, Iā€™m not entirely sure how the OpenAI functions are implemented in our application. I didnā€™t develop the code myself, and my technical knowledge at that level is limited.

Could you please advise me on how I can determine whether the OpenAI functions are running in Firebase Cloud Functions or if the code is executed client-side? Additionally, if you have any suggestions on how to address this issue or strengthen the security of our application, I would greatly appreciate hearing them.

Thank you very much for your assistance and understanding.

15

You can go into your Firebase console and then click Functions to see the current Functions available. There should be a list of them with some sort of relevance to handling OpenAI queries.

Screenshot from 2024-05-02 19-32-34
Screenshot from 2024-05-02 19-32-34821Ɨ730 60 KB

1 Like
16

Maybe, just maybe, when the existing users are using it on a regular basis, their previous sessions are being sent to the api as context.
Which might be an answer to see a sudden rise in costs without actually being hacked or an API leak.
Best thing for you to do first, is actually see whatā€™s going on with your application.
I suggest you to use some service like LangSmith or LangFuse which will give you a thorough understanding of whatā€™s going on with your application and which steps the API actually takes, and where the costs are high.
You could find out how to improve your app too.
Hope youā€™ll find the answer youā€™re looking for.
Cheers.

1 Like
17

Iā€™m definitely not an expert but as Iā€™m exploring OpenAI API usage (with Unreal Engine 5) I noticed that the use of GPT4 is very expensive in comparison to 3.5 that does the same job in my case. For example, a simple question would costs me 0,34 cent in GPT 4 while the same question costs me 0.01 cent in GPT 3.5ā€¦
So Iā€™ve turned off the use of GPT 4 in this specific API. Hope this helps

1 Like
18

If you have control over your backend code you should be able to answer these questions yourself, no? Or is there no programmer available anymore? :S

Check what user is responsible for a large cost and look at the requests being sent.

Possibly you need to limit your usersā€™ daily usage? Perhaps some user is intentionally spamming your service to rack up costs?

Not having a coder available is going to make it difficult to maintain the projectā€¦ :confused:

1 Like
19

With 300 active users per day, daily cost of 40$ is no surprise. I sometimes hit that on my own, using the GPT-4 API to summarize a big number of articles.

You need to log which users are inferring the cost. And you need to enforce per-user usage-limits for the app, so that you can still make profit per user on average.

That being said, the most likely other cause is that you are exposing the API key to the end user.
Never never never put the API key in the client side code or application. There is NO way to secure it there - trust me on that, Iā€™m a professional leading ethical hacker of web-, mobile and IoT applications.

You need to store the API key exclusively on the backend, and never expose it there (like in public Github repositories, etc.).
If you expose the key somewhere in the client (mobile app, website. etc.) it WILL be stolen and abused.

If you need a quick check if the key is exposed in your app, I can do that, hit me via DM, but I need your consent.

PS: consider to stop using GPT-4, GPT-4-Turbo is much cheaper.

20

My advice would be to start experimenting with Sonnet / Haiku models on openrouter (provides you easy api access compatible with openai api so all your existing code will work with them out of the box) to see if they will fit your use case, because these models are better and cheaper than gpt4 for many use cases which are not ā€œmath or coding or rocket scienceā€. So you should split the API keys you have into those that invoke only gpt4 for high cost for most difficult tasks in your platform, and invoke the cheaper models for the easier tasks. Also the models I mentioned have 200k context window input so they can analyze much more at once, than gpt4 or gpt3.5. And of course try other AIs and their APIs to see if they fit your use case - is this even a question? IMHO you should do it EVEN if you were happy with gpt4. Because you need to be aware of the ecosystem and the differences in those AIs and their costs and quality, if you building some startup on top of them. It honestly is surprising that you only used openai exclusively without properly comparing the quality and cost of other available platforms.

1 Like
21

Iā€™ve faced a similar problem. The problem was in the Assistant API usage.
When we reuse the same thread the input token grows with every request causing high bills.

1 Like