Hi together,
i am currently working on a system to extract information security knowledge from different standards. In summary i ask chatgpt to formulate vulnerabilities out of a control and a threat. The Prompt i am using does not exceed the context length at all. It is about 2k-3k Token (including the response). And i am formulating each request individually (Do not need any Context from previous messages). However after chatgpt formulates a couple of vulnerabilities i get the error that the max token is reached for the request and i need to shorten the prompt. However this does not make sense at all. All request have roughly the same siz of context. However if i add a sleep timer in my code, then i can do more request. And with every increase of this timer, i can do more and more request.
So i am feeling like chagtp is adding the context up for a certain amount of time. Does anyone face same issues?
Hi and welcome to the Developer Forum!
Can you please provide a code snippet, and the actual error you are receiving.
the error message likely tells more, such as the size of the input and the size of the max tokens you reserved. one of those should be reviewed - they both utilize the same limited space.
I have now figured out the problem. I created an instance of chatGPT, which I then passed as a parameter to the function I was using to make the requests. This was the reason why the context kept adding up. Now I create a new instance of chatGPT before each query in the function and the code works perfectly.
1 Like