Hi, we tried to use davinci model to summarize long texts. There is a problem when I send multiple requests (one after another) with long prompts (e.g. summarizing these texts or answering questions). API seems to be totally unstable then, sometimes responses are fine, but often empty or trashy - like prompt repeated multiple times. Maybe somebody else faced this issue? It’s basically impossible to build working app with good ux. What may cause that? I even tried to put some sleep function before calling api - but it didn’t help. Prompts are fine - sent separately with long breaks between requests.
1 Like
Because you’re using the wonderful Davinci you can use
“echo”: true
Just to see how your prompt is being formatted.
Otherwise, can’t help without seeing your request & response
Thanks, it wasn’t about prompt really (I think), I faced this issue even having “Summarize: {text of long legal document}”
Did you at least try it? There should be no difference in time periods. Most times when people start getting noise, is because they are sending noise. Which, considering you have a {text of a long legal document}, who knows how its being formatted.
At the very least, it rules out the most common issue. There’s no reason not to try it.
Can you post your request?