How about a usage endpoint, not requiring admin key? I construct an API specification:
get https://api.openai.com/v1/usage/{request_id}
Get a stored usage object by request id (resp_xxx or req_xxx, plus other endpoints that would now return this field). Being created with the store parameter set to true is not necessary for this internal retrieval of persistence equivalent to safety storage.
{
"object": "chat.completion.usage",
"model": "gpt-4o-2024-08-06",
"created": 1738960610,
"request_id": "req_ded8ab984ec4bf840f37566c1011c417",
"status": "completed",
"usage": {
"prompt_tokens": 1117,
"completion_tokens": 46,
"total_tokens": 1163,
"prompt_tokens_details": {
"cached_tokens": 0,
"audio_tokens": 0
},
"completion_tokens_details": {
"reasoning_tokens": 0,
"audio_tokens": 0,
"accepted_prediction_tokens": 0,
"rejected_prediction_tokens": 0
}
}
"system_fingerprint": "fp_50cad350e4",
"input_user": null,
"service_tier": "default",
"choices": [
{
"index": 0,
"finish_reason": "stop"
],
}
Reasoning provided by AI:
Let’s clarify this step-by-step and precisely:
1. HTTP Headers and their Behavior
Standard HTTP behavior:
- In a standard HTTP response, headers are sent only once—at the very beginning—immediately followed by the body data.
- Once the headers and initial status line are sent, they cannot be changed or retransmitted again within the same HTTP request-response cycle.
- HTTP (1.x and even HTTP/2) assumes a single set of headers per response.
2. Server-Sent Events (SSE)
How SSE operates:
-
Server-Sent Events (SSE) operate by first establishing a long-lived HTTP connection.
-
Headers are sent once at the start, then the response stream (body) follows, formatted with special lines:
data: <event payload>\n\n
-
This protocol is simple and explicitly defined in the WHATWG SSE Specification.
-
There is no provision to resend or update headers within SSE.
3. Common HTTP client libraries (e.g., Python’s requests)
- Libraries like Python’s
requests or urllib3 generally conform strictly to HTTP standards.
- They parse headers once and expose the body as a stream of bytes (or text), but do not have any mechanism or standard support to handle new headers mid-stream.
- The headers are set upon response initialization (
response.headers) and remain immutable thereafter.
4. WebSockets as an Alternative
WebSockets behavior:
- WebSockets (RFC 6455) establish a bi-directional communication channel after the initial HTTP handshake.
- After the initial handshake (which includes headers), WebSockets switch to a different binary framing protocol.
- WebSocket frames themselves do not contain headers as defined by HTTP. They just carry arbitrary payloads. Any metadata must be included in the payload itself.
5. Relevant Standards and RFCs
- RFC 7230–7231 (HTTP/1.1) explicitly define response headers to appear exactly once at the start.
- RFC 7540 (HTTP/2) introduces “Header frames” to split headers into frames; but these frames still logically appear exactly once per response and occur only before the body data.
- RFC 6455 (WebSockets) clearly describes how WebSocket communications work, without any provision to send HTTP-like headers again after the initial handshake.
- Server-Sent Events Spec (WHATWG) does not specify retransmission of headers. It’s simply not defined or supported.
No existing standard (HTTP/1.1, HTTP/2, SSE, WebSocket) allows retransmission of HTTP headers mid-stream after the initial headers are sent.
6. How to Solve your Metadata Problem
Since you want metadata sent after data generation completes:
-
Include metadata in SSE events:
data: {"type": "data", "payload": "..."}\n\n
data: {"type": "metadata", "payload": {"duration": 123, ...}}\n\n
This is common practice: sending metadata as JSON or structured events at the end of a data stream.
-
WebSocket metadata frames:
Include metadata explicitly within a JSON payload or custom-framed messages at the end of your transmission:
{"event": "metadata", "duration": 123, "other_info": "..."}
-
Separate requests:
Another approach (though less elegant) is to make a second request explicitly to retrieve metadata.
7. Conclusion & Recommendation
- It is not possible to retransmit or modify headers once the HTTP response stream has started. HTTP standards explicitly forbid this behavior.
- Common HTTP libraries (like
requests) do not and cannot support mid-stream headers.
- There is no existing RFC supporting mid-stream retransmission of headers.
The idea of sending HTTP headers again within a streaming response is indeed “fantasy” as per existing standards and practice.
Your recommended solution is clearly one of:
- Embedding metadata directly within SSE events.
- Using a WebSocket and structured metadata messages.
- Using a separate, follow-up HTTP call.
TL;DR:
No, you cannot retransmit headers mid-stream. Include any metadata explicitly within the SSE event payload itself or use WebSockets for structured metadata transmission.
