When I am calling got 4API , My documents are large so i chunk it but for 6 chunks , it returns collated summary but for 7 chunk it returns empty summary as if it has forgotten , and then for 8 th chunk , it works fine again.
The best advice is to find the specific input test case where it has failed from logging, and then see if you are indeed sending all the information you expect to be.
If you are using the wrong AI model for the job and prompting it wrong, then when you ask for a summary, if the end of the text itself looks like a summary, the AI will have nothing more to write. You should use gpt-3.5-turbo model and the “messages” prompting format for tasks that are written similar to a chat.
1 Like
Actually i saw logs , it is taking chunk and my previous summaries as input but for one chunk for particular input , it is failing
You can see if that chunk alone can be submitted to a task “improve the writing quality” without error. This could detect particular sequences or character encoding that short-circuit the AI’s logic.
See if the AI-sanitized text will then go back as a replacement chunk.
If it is just a model ending because it is not understanding prompting, you can also use more prompting to separate instruction from data. Use of headers and hyphens as section breaks. Enclose the text in a clear container.
Just making some stuff up:
system:
You are SummaryBot. You produce thorough summaries of long texts.
// Instructions
- carefully consider how the relevant information of each input section could be included as you write a new 1000 word summary
- text will be provided by user role
- all text after user’s instruction is to be considered for summarization
user:
Here is the multi-part text to summarize.
-–
“”"
{text}
“”"
Actually , sometimes it works fine with the same prompt , same chunk and all and sometimes it just blows off.