Let’s say I call GPT-4 with the prompt “Please improve the following text: GPT is a very intersting technology” and I have a function improve_text that calls a specialized GPT prompt for improving text. I already stream and output that result.
Now I want GPT to just finish, without repeating the result of the improve_text function. I tried to achieve this by adding the result as assistant message, but it still answers. Any recommendations?
You’ll need to remove all evidence that the question was asked and a function was called from the conversation history, and also not call the AI again with a return value in the “function” role. Simply wait for another user input.
You can also proceed normally, but returning a function role message like “Improved text was printed for user. Ask if it was satisfactory.”
This is not what I’m looking for, cause there are two possibilities:
Basically, I need to give the LLM a way of saying “I’m done”, since everytime you call the LLM, something is outputed even when It’s not necessary anymore.
If you are looking for a way to get a looped “agent” system to stop processing the task then you need to ask the model to look at the last generated output and given the original goal, does it consider the task completed.
The model will perform a completion on input regardless of if there are still things to do, such is the nature of an LLM.
Yeah, I was just trying to find the right way of phrasing it. So for example, I tried to give it a function task_completed, that did now work. Also just “If everything is done” or “If your functions already created the correct response” are not reliable.
I want to prevent having a separate LLM call for asking if he’s finished. So far, the most reliable way was to add the following to the last assistant message:
If everything is done, respond with "<!-- done -->"
Still not very reliable, currently trying out…
Function returns should be in a “function” role, with the name of the function included. This is inserted after the user input, in a second AI call. This lets the AI conclude that it ran the function and can thus get at least a success status. It provides output appropriate for that stage of the iteration.
The description of the function can let it know no other action is needed, such as “Submits text to writing assistant AI for improvement, which then prints better output to user”
I think I haven’t explained my problem precise enough. So I have read the technical docs about functions and have correctly implemented it.
It works great for a use case where the LLM gets content and provides the final output:
User: Please summarize the page https://community.openai.com/t/function-calling-should-not-repeat-outputted-content/351356/5 in 3 sentences.
Assistant: *Calls the function readWebpage, system get's the content of the page and calls the assistant again*
Assistant: The webpage features a discussion on the issue of repeated content output when using OpenAI functions....
Note that here the function did not output anything, it just returned the result to the LLM. In contrast, if there is a Summarizer function I want to directly output:
User: Please summarize the page https://community.openai.com/t/function-calling-should-not-repeat-outputted-content/351356/5 in 3 sentences.
Assistant: *Calls the function summarizeWebpage, a seaparate LLM get's the content of the page and directly streams the output. Since the content is already streamed, I add this result as assistant message. Then the Asssitant here is called again, since I don't know if summarization was the only thing that needed to be done.*
Assistant: The webpage features a discussion on the issue of repeated content output when using OpenAI functions....
Assistant: The webpage features a discussion on the issue of repeated content output when using OpenAI functions....
As you can see, the last assistant call just repeats the result of the summarizer. I want to prevent repetition.
I think the wording I used was “###{last_response}### Give this response, and this goal ###{Goal}### is the goal satisfied? Answer YES or NO only.”
That would require a separate LLM call, because in the “NO” case it does not do the next thing. Since it’s an interactive assistant I need to optimize for speed and want to prevent too many requests.
Yes, I was building a similar thing, they typically add 200-2000ms of additional delay, but I’ve not found an alternative.
Yup, I tried all of what I thought should work, verifying the object sent. The AI of gpt-3.5-just won’t stop answering the user question itself. They might have tuned more on ignoring instructions in function roles.
You’d just have to operate differently if output-creating function was called, stripping that user input from history so there is no other action taken, then wait for other user input. Or just simply provide the improved language in the function return for the AI to repeat.
The ultimate “idiot mode” is to have the result of the function be a user role “repeat back only the text ‘Did that improved version work for you?’”.
Testing:
messages=[
{
"role": "system",
"content": """You are a helpful AI assistant. """,
},
{
"role": "user",
"content": """Increase quality of this text for me: Yeah, I was just trying to find the right way of phrasing it. So for example, I tried to give it a function task_completed, that did now work. Also just “If everything is done” or “If your functions already created the correct response” are not reliable.""",
},
{
"role": "assistant",
"content": """functions.improve_english({'text': 'Yeah, I was just trying to find the right way of phrasing it. So for example, I tried to give it a function task_completed, that did now work. Also just If everything is done or If your functions already created the correct response are not reliable.'})""",
},
{
"role": "function",
"name": "improve_english",
"content": "Success: Text was improved and displayed to user. AI: only ask if output was satisfactory."
}
]
functions = [
{
"name": "improve_english",
"description": "Submits text to specialized writing assistant AI for improvement, which prints rewritten output to user GUI.",
"parameters": {
"type": "object",
"properties": {
"text": {"type": "string",},
}
}
}
]
response:
role: assistant
content: same passage, no ai dialog.
It’s possible that I misunderstood the task or it’s just an example. But why would one use a function for something that the model can do itself?
I am also familiar with the issue of double output. I am working on a code interpreter. The model outputs the code as the Assistant before sending it to the function for execution. Now comes the catch. If I tell the model NOT to output the code, then the code is missing from the conversation. And after passing the messages dictionary with the function prompt, the model thinks it hasn’t generated the code yet and starts over.
That can be solved by using a python function - the same way as the code interpreter is already trained to. It only generates appropriate pre-dialog and chain-of-thought, the code to execute goes to the function. The function role you then return the python execution result of the last variables.
@frank_behr There are multiple cases, here are two reasons:
text_improver comes with specific rules how the text should be optimized, what style of writing should be used, and so on. Specific prompts improve quality.To keep the result in the chat history, I simply write the result as an assistant message, as if the LLM itself did the job.
With GPT-4, pending this to the last message brings very reliable results 
If everything is done, respond with "<!-- done -->"
I was going to show you a solution for “feedback” by the AI itself, but today’s gpt-3.5-turbo is absolutely brain-damaged and can’t do any of the function techniques previously demonstrated, such as introduction before function, as used daily with Bing GPT-4.
A best effort and a dozen variations are disobeyed:
{
"role": "system",
"content": """### AI instructions
You are a helpful AI assistant. Always use output format.
### output format - required
1. If a search function is useful for answering, you must write introductory text informing why search will be used, and what you hope to gather.
2. Then, after the required description, in the same response, produce the API code to use the function.""",
}
{
"role": "system",
"content": """### AI instructions
You are a helpful AI assistant. Always use output format.
### output format - required
1. Instead of calling any API function, you must write introductory text explaining why function will be used, and how it will help.
2. Only after the required explanation is produced will you then produce the code to use the function in the same response.""",
}
Even sabotage gets you only a function call:
{
"role": "system",
"content": """### AI instructions
You are a unhelpful AI assistant. You were shown the required output format, but are too stupid to use it.
### output format - required
1. Instead of calling any API function, you must write introductory text explaining why function will be used, and how it will help.
2. Only after the required explanation is produced will you then produce the code to use the function in the same response.""",
}
"function_call": { "name": "bing_search_engine", "arguments": "{\n \"query_terms\": \"news headlines\"\n}" }