As far as I know, when trying to execute the assistant API, we’ll first create a Thread object along with Assistant. Then we try to append our message on it and triggered a Run action, waiting for the Run and Run Step response.
But when I’m using code_interpreter to generate an image, two problems arose:
- An unexpected empty message appeared in the thread
- The message that really containing content generated by Assistant cannot be retrieved by run API.
For context, I was using the code_interpreter to generate an image. Despite specifying Thread thread_xxxxXBhM and Run run_xxxxsDVm, the List Run Steps API did not return the expected message (msg_xxxxdN5H) that generated the content. Instead, it returned an unrelated empty message (msg_xxxxj4XJ).
This issue affects the reliability of using the Run Steps / Messages API to determine new responses from the Assistant, and the presence of an empty message seems unnecessary and potentially confusing.
Request: GET https://api.openai.com/v1/threads/thread_xxxxXBhM/runs/run_xxxxsDVm/steps,
Response:
{
"object": "list",
"data": [
### MISSING: msg_xxxxdN5H, which really has the file and text content inside the message
{
"id": "step_xxxxhuYJ",
"object": "thread.run.step",
"created_at": 1708941326,
"run_id": "run_xxxxsDVm",
"assistant_id": "asst_xxxxVlwZ",
"thread_id": "thread_xxxxXBhM",
"type": "message_creation",
"status": "completed",
"cancelled_at": null,
"completed_at": 1708941328,
"expires_at": null,
"failed_at": null,
"last_error": null,
"step_details": {
"type": "message_creation",
"message_creation": {
"message_id": "msg_xxxxj4XJ" ### empty message
}
},
"usage": {
"prompt_tokens": 468,
"completion_tokens": 66,
"total_tokens": 534
}
},
{
"id": "step_xxxxhrVJ",
"object": "thread.run.step",
"created_at": 1708941300,
"run_id": "run_xxxxsDVm",
"assistant_id": "asst_xxxxVlwZ",
"thread_id": "thread_xxxxXBhM",
"type": "tool_calls",
"status": "completed",
"cancelled_at": null,
"completed_at": 1708941326,
"expires_at": null,
"failed_at": null,
"last_error": null,
"step_details": {
"type": "tool_calls",
"tool_calls": [
{
"id": "call_xxxx4yzO",
"type": "code_interpreter",
"code_interpreter": {
"input": "import matplotlib.pyplot as plt\n\n# Given data\ncities = [\"Tokyo\", \"Kyoto\", \"Osaka\"]\nvalues = [541, 135, 386]\n\n# Create a bar chart\nplt.figure(figsize=(10, 6))\nplt.bar(cities, values, color=['blue', 'green', 'red'])\nplt.title('Values by City')\nplt.xlabel('City')\nplt.ylabel('Value')\nplt.grid(axis='y')\n\n# Display the graph\nplt.show()",
"outputs": [
{
"type": "image",
"image": {
"file_id": "file-xxxxM4QBg"
}
}
]
}
}
]
},
"usage": {
"prompt_tokens": 338,
"completion_tokens": 104,
"total_tokens": 442
}
}
],
"first_id": "step_xxxxhuYJ",
"last_id": "step_xxxxhrVJ",
"has_more": false
}
Request: GET https://api.openai.com/v1/threads/thread_xxxXBhM/messages
Response:
{
"object": "list",
"data": [
{
"id": "msg_xxxxdN5H",
"object": "thread.message",
"created_at": 1708941326,
"assistant_id": "asst_xxxxVlwZ",
"thread_id": "thread_xxxxXBhM",
"run_id": "run_xxxxsDVm",
"role": "assistant",
"content": [
{
"type": "image_file",
"image_file": {
"file_id": "file-xxxxM4QBg"
}
},
{
"type": "text",
"text": {
"value": "Here is the bar chart visualizing the values you provided for Tokyo, Kyoto, and Osaka. As you can see, Tokyo has the highest value at 541, followed by Osaka with 386, and then Kyoto with 135. If you need any further customization or have more data to visualize, let me know!",
"annotations": []
}
}
],
"file_ids": [],
"metadata": {}
},
{
"id": "msg_xxxxj4XJ",
"object": "thread.message",
"created_at": 1708941326,
"assistant_id": "asst_xxxxVlwZ",
"thread_id": "thread_xxxxXBhM",
"run_id": "run_xxxxsDVm",
"role": "assistant",
"content": [],
"file_ids": [],
"metadata": {}
},
{
"id": "msg_xxxxYVpN",
"object": "thread.message",
"created_at": 1708941299,
"assistant_id": null,
"thread_id": "thread_xxxxXBhM",
"run_id": null,
"role": "user",
"content": [
{
"type": "text",
"text": {
"value": "Tokyo, 541\nKyoto, 135\nOsaka, 386\n\nVisualize them using a bar chart",
"annotations": []
}
}
],
"file_ids": [],
"metadata": {}
},
...