Why did my text get cut off? I sent a butch of text for processing and got just about 1/3 or it back in the return. The end fizzled out like this “there in the…”.
My request
// Make the POST request to OpenAI
$response = wp_remote_post($url, array(
'method' => 'POST',
'body' => json_encode($body),
'headers' => [
'Content-Type' => 'application/json',
'Authorization' => 'Bearer ' . $apiKey,
],
'timeout' => 540, // Set to 9 minutes, aligned with PHP execution time
'redirection' => 5,
'blocking' => true,
'httpversion' => '1.1',
'sslverify' => true,
'data_format' => 'body',
));
My return (partial)
{
"id": "chatcmpl-9GyRBI35tpWPFEqiCHbWOC7kE8mQO",
"object": "chat.completion",
"created": 1713832597,
"model": "gpt-4-0125-preview",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "```json\n{lots of text and at the end: there in the...}\n```"
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 6613,
"completion_tokens": 2845,
"total_tokens": 9458
},
"system_fingerprint": "fp_122114e45f"
}
In my request I did not instruct the assistant to cut off the return; but did also not tell it to finish it’s job.
I am using a WordPress proxy page to communicate my request.
Do I need to send a max_tokens with my request to get the full return, or is there a limit I need to adjust in my account settings?
The finish reason being “stop” means the AI ended the output.
2845 response tokens is a large amount compared to what most get from this AI just by asking for a writing task. The training and even automatic token bias may have simply gotten high enough for the AI to emit “stop” or it was one of the random chances of an less-likely token being selected from a wide range of possibilities at default top_p setting.
If that is 1/3, you will never get the task as requested, because that AI model has a hard output limit of 4k response tokens, and you cannot set it higher. So you’ll need to adjust your expectations and task-writing.
The only API model commonly available that does not have the output limit (but still has output training) is gpt-3.5-turbo-16k.
I have found on that particular model, you can indeed supply a 7k input and get 7k back. However, the quality of a rewriting task and following is almost non-existent.
If you do manage to not have your input made into a compressed version by instructing piecemeal operation on sentences or parts, with clear parsing instructions at the start and end of context clearly separated from the data to be acted on, the output may have no improvement at all in the text, even from the very start of output. You can try though.
A bit of expense to get your answering quantity up to 2/3, randomly.
You might compare the quality of the AI revisions to if you now were to overlap some of the ending data. And then just ask for the processing on 1k chunks and compare the quality improvement.
var interject_instructions = `
But before you start making any corrections to the data take special note of these directions. It's essential for the JSON document key "maincontent" value that you use this specific outlined formatting. I will highlight your edits. This formatting is crucial for us to identify the changes you make. Please follow these guidelines closely:
1. **Word Changes (for simple misspellings)**:
- **Original Text**: "The sky is green."
- **Corrected Text**: "The sky is <span class='suggested_removal'>green</span> <span class='suggested_replacement'>blue</span> <span class='remark_on_why'>(Corrected fact)</span>."
2. **Sentence Changes (for restructuring sentences)**:
- **Original Text**: "Dog are not frindly."
- **Corrected Text**: "<span class='suggested_removal sentence'>Dog are not frindly.</span> <span class='suggested_replacement sentence'>Dogs are generally friendly.</span> <span class='remark_on_why sentence'>(Corrected perspective and misspelled word)</span>"
Please ensure that:
- Each type of edit (word change or sentence restructuring) is clearly marked using the provided HTML span tags.
- Do not omit the <span class='remark_on_why'> tag, as it provides necessary context for each correction.
Failure to apply these formatting rules may result in edits being overlooked or not properly credited.
`;
var taskInstructions = `As a film transfer company, we require the document (written in ${targetLang}) to undergo a check for spelling, grammar, and sentence structure.
`;
if (jQuery('input[name="ff_interject_remarks"]:checked').val() == 'y') {
taskInstructions += interject_instructions;
}
taskInstructions += `
Below is the structure of provided JSON file containing the text to be reevaluated:
{
"title": "<The title of the document>",
"maincontent": "<The main document>",
"description": "<Summarized text suitable for a meta description tag>",
}
Data sources within our WordPress setup:
- "Title" from an input field: '<input id="title" value="">'.
- "Maincontent" from a textarea: '<textarea id="content"></textarea>'.
Please make your suggested corrections directly in the document. Do not include any remarks.
${titleInstructions}
Your edits will replace the original text in these WordPress elements, focusing on:
- Preserving WordPress HTML formatting.
- Maintaining the integrity and structure of the HTML and text.
- The title should follow the title case convention (provided that rule applies for the language ${targetLang}) where the first letter of each significant word is capitalized.
`;
if (ff_rework_title_and_first_paragraph == 'y') {
taskInstructions += `As a separate task, reevaluate your completed work for the "title", but this time, provide a second go at it as "engagingTitle". The "engagingTitle" should be more captivating, enticing, maybe fun. Be SEO mined full in terms of length limitation.
In addition, again as a separate task, provide a complete rewrite of the first paragraph. This time pay special attention to the first sentence, trying to make it more captivating, enticing engaging in order to hock the reader.
Lastly reevaluate that the "engagingTitle" and the "engagingFirstParagraph" work well together and are not off topic when considering the document at hand, making adjustment if needed.
`;}
taskInstructions += `Upon completion, return the document in this JSON structured format:
{
"title": "<reworked title>",
"maincontent": "<reworked maincontent>",
"description": "<reworked meta description>",`;
if (ff_rework_title_and_first_paragraph == 'y') {
taskInstructions += `
"engagingTitle": "<engaging title>",
"engagingFirstParagraph": "<first paragraph with engaging start>",`;
}
taskInstructions += `
}
Carefully maintain the original document's integrity and structure while making your improvements.
`;