Implementing rate limiting in plugins?

Plugins
1

A rate limit is a restriction that an API imposes on the number of times a user or client/IP can access the server within a specified period of time.

Is it recommended to add rate limiting to my API that I use exclusively for ChatGPT? Or does ChatGPT handle this? I’m not sure if requests to my API will have the same IPs as they are coming from ChatGPT and not the actual IP of the user using ChatGPT.

3

It looks like the forwarded IPs are from San Antonio. It seems like we dont see users IPs in order to implement rate limiting based on IPs.

4

You can use the conversation id or the user id - these should both be supplied to you in the request headers.

1 Like
5

Using the API HEADERS, for each user you might be getting a token. Create a api limit for the user tokens.

6

Wouldn’t attackers be able to just put any convo or user id in the headers?

7

You should also use service authentication if this is a concern. With that setup OpenAI will send you a secret key that you can check so you know the calls are coming from them.

1 Like
8

@iamflimflam1 thank you! Service auth would be ideal but to my knowledge OpenAI does not have service auth. They provide only API keys.

9

Docs on service level auth: OpenAI API

This would lock down your API to only ChatGPT requests.

10

Great thank you! I generated a token and it worked! Would everyone who installs my plugin have to generate a token or is this a one time setup?

11

This question inspired this post here: Securing ChatGPT Plugin APIs | HackWithGPT

I wrote a bit about rate limits as well as other steps to take in securing your plugin API. The BLUF is you should implement rate limits, ChatGPT will respect them and back off if your application starts sending 429s. Even if you only implement service-level auth you can rate limit per conversation or per user by using the headers OpenAI passes on each request:

  • openai-ephemeral-user-id
  • openai-conversation-id

Hopefully this helps.

As per your question about all users having to generate a token, if you’re doing service-level auth only you should have access to the secret and you should verify the secret every time a request hits your API. You should also periodically rotate the secret.

1 Like
12

Thank you! I have found the headers openai-ephemeral-user-id and openai-conversation-id but I have not found any official documentation from OpenAI recommending to use them. I am worried the header names may change in the future.

13

Seems they added the headers on a whim: Consistent conversation id in a header?

14

I certainly hope not - my Chess plugin is completely reliant on conversation-id to track game state!

15

Just a quick message to say we added this to our roadmap at www.pluginlab.ai.

It’s not supported yet but if some folks are looking for this kind of effortless protection (rate limiting, error alerting, IP protection) in a few days, I believe it’s worth it to have a look at PluginLab :slight_smile: