Usage stats now available when using streaming with the Chat Completions API or Completions API

API
, ,
3

Available in Python openai >= 1.26.0

Worth noting:

  • the finish reason will be in the second to last chunk now
  • the last usage chunk will have no “choices” contents that you might have been parsing
  • stream_options={"include_usage": True} if using library parameters (capital True)

Presentation:

Finish reason chunk

{
  "id": "chatcmpl-9M3...",
  "choices": [
    {
      "delta": {
        "content": null,
        "function_call": null,
        "role": null,
        "tool_calls": null
      },
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null
    }
  ],
  "created": 1715044805,
  "model": "gpt-3.5-turbo-0125",
  "object": "chat.completion.chunk",
  "system_fingerprint": null,
  "usage": null
}

Final chunk

{
  "id": "chatcmpl-9M3...",
  "choices": [],
  "created": 1715044805,
  "model": "gpt-3.5-turbo-0125",
  "object": "chat.completion.chunk",
  "system_fingerprint": null,
  "usage": {
    "completion_tokens": 11,
    "prompt_tokens": 29,
    "total_tokens": 40
  }
}
Some response-scraping logic 
response = client.chat.completions.with_raw_response.create(
    **your_parameter_dict)
content = ""
for chunk in response.parse():
    print(json.dumps(chunk.model_dump(), indent=2))
    if chunk.choices:
        if not chunk.choices[0].finish_reason:
            word = chunk.choices[0].delta.content or ""
            content += word
            print(word, end ="")  # your method
            if chunk.choices[0].delta.function_call:
                function_call += chunk.choices[0].delta.function_call
            if chunk.choices[0].delta.tool_calls:
                tool_calls += chunk.choices[0].delta.tool_calls
        else:
            finish_reason = chunk.choices[0].finish_reason
    if chunk.usage:
        usage_dict = chunk.usage

(the gathered chunks of tools and functions will need to be reassembled)