I develop app with livekit multimodal agent and when i change model from gpt-4o-realtime to mini version then occure response generation failing, and if strong set big realtime version then it works good.
At first, I tought that it in cause of long promt instruction, but when i even delete it, the it still not work properly.
Setting model looks like:
model = new openai.realtime.RealtimeModel({
model: 'gpt-4o-mini-realtime-preview-2024-12-17',
instructions: getFormattedInstruction(candidateInfo),
});
And error:
Connecting to OpenAI Realtime API at wss://api.openai.com/v1/realtime?model=gpt-4o-mini-realtime-preview-2024-12-17
[19:50:14.846] ERROR (5364): response generation failed
sessionId: "sess_*******"
code: null
error: "The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the session ID sess_B73HaFxpJAg3KenvvjqDk in your message.)"
I am getting a similar issue, heres the full response:
2025-03-04 11:32:53,694 - INFO - AI_Voice - Full response data: {
“type”: “response.done”,
“event_id”: “event_B7Ko1mbduMcv9nLBLqnHe”,
“response”: {
“object”: “realtime.response”,
“id”: “resp_B7Ko1Ge2Ei3LarXUTzUIU”,
“status”: “failed”,
“status_details”: {
“type”: “failed”,
“error”: {
“type”: “server_error”,
“code”: null,
“message”: “The server had an error while processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the session ID sess_B7KnazsnCarIgWqMxjYNP in your message.)”
}
},
“output”: ,
“conversation_id”: “conv_B7KnaUrlcvs9zpXiiAhwg”,
“modalities”: [
“text”,
“audio”
],
“voice”: “alloy”,
“output_audio_format”: “pcm16”,
“temperature”: 0.8,
“max_output_tokens”: “inf”,
“usage”: {
“total_tokens”: 0,
“input_tokens”: 0,
“output_tokens”: 0,
“input_token_details”: {
“text_tokens”: 0,
“audio_tokens”: 0,
“cached_tokens”: 0,
“cached_tokens_details”: {
“text_tokens”: 0,
“audio_tokens”: 0
}
},
“output_token_details”: {
“text_tokens”: 0,
“audio_tokens”: 0
}
},
“metadata”: null
}
}
I try fix that issue for three days already and i doesn’t resolve it. Livekit On python back same issue if manualy write mini model. I really surprised by that not much people had that issue and can’t understand where is problem