Assistants API Batch - format of results

I was able to get the Batch functionality working quite nicely, using the Assistants API.
What I did find difficult was the lack of documentation for the output file format.
Remember that the Batch produces a text file in .jsonl format, which can be seen in the Storage section of the web interface.
After retrieving the file as a text blob, you must iterate over the lines if you want to display the results to the client application. Here is an example of the jsonl output file:

{"id": "batch_req_HILX2OJI6vgNsgSktV5tbH", "custom_id": "request-1", "response": {"status_code": 200, "request_id": "a57433ff2ef8d05d6321a0446f541fce", "body": {"id": "chatcmpl-9fThFKpv8copwvDO0PLjZEie6TB", "object": "chat.completion", "created": 1719672629, "model": "gpt-3.5-turbo-0125", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Hello! How can I assist you today?"}, "logprobs": null, "finish_reason": "stop"}], "usage": {"prompt_tokens": 20, "completion_tokens": 9, "total_tokens": 29}, "system_fingerprint": null}}, "error": null}
{"id": "batch_req_03Zf0RBDYTvodTKWWWUc2kk", "custom_id": "request-2", "response": {"status_code": 200, "request_id": "59b75aa5e15b4fafbe31c7f255e30c", "body": {"id": "chatcmpl-fThFQoE8RZF7NaVenTLaKS4KAqm", "object": "chat.completion", "created": 1719672629, "model": "gpt-3.5-turbo-0125", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Hello there! What can I not help you with today?"}, "logprobs": null, "finish_reason": "stop"}], "usage": {"prompt_tokens": 22, "completion_tokens": 12, "total_tokens": 34}, "system_fingerprint": null}}, "error": null}

Formatted for readability:

{
  "id": "batch_req_HILX2OJI6vgNsgSktV5tbH",
  "custom_id": "request-1",
  "response": {
    "status_code": 200,
    "request_id": "a57433ff2ef8d05d6321a0446f541fce",
    "body": {
      "id": "chatcmpl-9fThFKpv8copwvDO0PLjZEie6TB",
      "object": "chat.completion",
      "created": 1719672629,
      "model": "gpt-3.5-turbo-0125",
      "choices": [
        {
          "index": 0,
          "message": {
            "role": "assistant",
            "content": "Hello! How can I assist you today?"
          },
          "logprobs": null,
          "finish_reason": "stop"
        }
      ],
      "usage": {
        "prompt_tokens": 20,
        "completion_tokens": 9,
        "total_tokens": 29
      },
      "system_fingerprint": null
    }
  },
  "error": null
}
{
  "id": "batch_req_03Zf0RBDYTvodTKWWWUc2kk",
  "custom_id": "request-2",
  "response": {
    "status_code": 200,
    "request_id": "59b75aa5e15b4fafbe31c7f255e30c",
    "body": {
      "id": "chatcmpl-fThFQoE8RZF7NaVenTLaKS4KAqm",
      "object": "chat.completion",
      "created": 1719672629,
      "model": "gpt-3.5-turbo-0125",
      "choices": [
        {
          "index": 0,
          "message": {
            "role": "assistant",
            "content": "Hello there! What can I not help you with today?"
          },
          "logprobs": null,
          "finish_reason": "stop"
        }
      ],
      "usage": {
        "prompt_tokens": 22,
        "completion_tokens": 12,
        "total_tokens": 34
      },
      "system_fingerprint": null
    }
  },
  "error": null
}

Hope this helps!

1 Like