I am using the newest api and have the below code.
When I get tool_calls back in the response, I do not get content. Therefore I have to do a second API call (and force no tools) to get content which will be the answer.
Is this possible to get in one API call or is it a limitation in the models? So one API call → tool_calls & content in response.
Same problem: How to make a function call and get a textual response at the same time? - #21 by dan01962
other related: Getting a function call + textual response in the same call
A “hack” seems to be to add an additional function call “reply_customer” for example. But I don’t think the text inside the function call argument would be as good as in content, but I might be wrong about this.
tool calls should not only be used to give AI extra information, it should also be possible to “do stuff and get an answer”, for me tools is not primarily used to “get more information, then get new answer from AI based on that info”.
private fun processResponseFromAI(
aiResponse: AIAgent.OpenAIResponse,
ticket: Ticket,
messages: MutableList<AIAgent.Message>,
) {
val toolCalls = aiResponse.choices.firstOrNull()?.message?.tool_calls
if (toolCalls != null) {
for (tool in toolCalls) {
// We just tell the model which function we "have" called, to get further answer
// but in reality we have not called the function yet
// The AI DOES NOT respond with both function call and a text answer, therefore
// we MUST do a second call to get the text answer as well as the action to take
// This seems a bit waste, so ideally the model would answer and propose tools to call
val functionCallMessage = AIAgent.Message(
role = "assistant", content = "Function call: $tool"
)
messages.addFirst(functionCallMessage)
val aiResponseWithoutTools = aiAgent.getAnswer(messages, false, department = ticket.department)
val aiResponse = extractAiResponseData(aiResponseWithoutTools)
ticket.aiResponse = aiResponse.message_to_customer
ticket.aiResponseEnglish = aiResponse.message_to_customer_english
ticket.customerLanguage = aiResponse.customer_original_language
try {
val functionCall = FunctionCallWithArgumentsAsMap(
tool.function.name,
mapper.readValue(tool.function.arguments, object : TypeReference<Map<String, String>>() {})
)
ticket.functionCalls = ticket.functionCalls.plus(functionCall)
} catch (e: MismatchedInputException) {
logger.error("Error reading the argument into FunctionCallWithArgumentsAsMap: ${e.message}")
}
}
} else {
val extractedResponse = extractAiResponseData(aiResponse)
ticket.aiResponse = extractedResponse.message_to_customer
ticket.aiResponseEnglish = extractedResponse.message_to_customer_english
ticket.customerLanguage = extractedResponse.customer_original_language
ticket.functionCalls = emptyList()
}
}
fun getAnswer(
messages: List<Message>,
includeFunctionCalling: Boolean = true,
responseFormatJson: Boolean = true,
model: String = "gpt-4o-2024-08-06",
department: String = "customer_service"
): OpenAIResponse? {
val request = ChatRequest(
model = model,
messages = messages,
)
if (!responseFormatJson) {
request.response_format = mapOf("type" to "text")
}
// we want to make functions available to the model only if we have not
// already called a function and gotten a result, because when a result
// is called, we do not get a response, so we need to inject the function
// result into the message and make functions unavilable so the model
// does not call the function again by mistake
if (includeFunctionCalling) {
val toolsExcludedCreateCompany = if (department != "b2b") {
tools.filter { it.function.name != CREATE_COMPANY }
} else {
tools
}
request.tools = toolsExcludedCreateCompany
}
return try {
restClient.post().uri("https://api.openai.com/v1/chat/completions").body(request).retrieve()
.body(OpenAIResponse::class.java)
} catch (e: Exception) {
logger.info("Error making OpenAI request: ${e.message}")
null
}
}