[GPT-4] API is not allowing the use of the whole 8196 tokens

Hi everybody, I am facing a strange problem with my GPT-4 based application.

I am using TikToken to gather multiple social media comments up to the limit of 2000 tokens (I have all the necessary code tricks to accumulate, process, and clean the lists, even the last one that was “ripped” in the middle).

After reaching the limit, I call the API to categorize each comment into a predefined category (it has been working properly for months).

My system prompt is 744 tokens. My response is around 3,350 tokens. So, the total amount of tokens consumed per call is approximately: 2,000 + 744 + 3,350 = 6,094.

However, since last weekend, I have encountered a very strange and almost random error in the response; a specific comment tends to disappear from the response message. It is included in the user prompt but then disappears in the response (as a debug method, I have it return not only the category but also the original text like Category | Original text to understand which comment was disappearing).

The comment is:

“@ we have to dodge the darn scythe :joy:

HOWEVER, if I reduce the number of tokens to accumulate before sending them to the API from 2,000 to 900, then it runs perfectly.

Does anybody have a clue what might be happening? Are there weird results coming out of the OpenAI API affecting your applications too?