Exactly, what the title says. Is “system” message necessary/useful when using “function calling” in the chat completions endpoint?
We have system prompt normally when we do the API calls. That is the one with “system” message. Is this even necessary or useful when doing custom functions? This is not clear looking at the documentation. For example in this official example, we don’t see any system message set.
curl https://api.openai.com/v1/chat/completions -u :$OPENAI_API_KEY -H 'Content-Type: application/json' -d '{
"model": "gpt-3.5-turbo-0613",
"messages": [
{"role": "user", "content": "What is the weather like in Boston?"}
],
"functions": [
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["location"]
}
}
]
}'
Has anyone tried “system” message with custom functions.
What has your experience been so far?
I haven’t considered omitting it, just by the logic that gpt-3.5-turbo already comes with a whole bunch of training of how to act like a chatbot.
Why is that a consideration? Assigning your new training a new distinct system prompt or AI name you’ll use in practice immediately distinguishes your usage of functions from the typical system prompt of “you are an AI assistant” or “you are ChatGPT” that we expect OpenAI would have trained the chat-ready AI models on.
edit: removed my incorrect focus on fine tuning.
You can prompt the AI with the typical “you are a chatbot” system message and still get function calling. The frequency of when to use them to answer user questions often is not as desired by function description alone, so system message is needed to describe the application, and also to demand that AI only invokes functions after gathering all necessary input fields from the user, not by merely fabricating them.
Actually, that is what I am trying to do. I am making it take on a specific set of persona and act like a summarizer in a specific style. And also extract other data out of given data.
My goal is to later to take the JSON output to train and fine-tune the models.
So for now I will leave the system prompt there. I will later also report here how much does the system function affect in the function calling.
For sure, it’s why you can include the system prompt in your fine tuning data. If you intend to run a customised system prompt in your live application, it would make sense to train the model with that set as part of the data.
For posterity and other stumbling upon this thread, I have been heavily testing this over the last few days. Over, 100000 API calls made.
I can safely conclude that the the system message has a HUGE effect during function calling too. A large effect. Use it carefully, for it has big influence on the outputs.
Update: The latest GPT4o and mini models in Aug 2024 apparently limit function descriptions to 1,024 characters; further reinforcing that function grounding should occur in system prompts.
