Is the `gpt-realtime-translate` audio frame size documented incorrectly?

Documentation
, ,
1

The documentation of the real-time translation server events of the gpt-realtime-translate API states that output audio deltas get streamed in frames of 200ms of PCM16 audio encoded in Base64. The sample rate for both input and output audio is fixed at 24kHz. If my math is correct the 200ms of audio at 24kHz should equal 4800 samples per frame.
However, after decoding the Base64 string and converting the bytes to a NumPy array the array contains 9600 samples, which is equal to 400ms of audio at the given sample rate.
I am using the following code, which I adapted from the real-time conversations guide for Python to decode and convert the audio:

def pcm16_to_float(pcm16):
    pcm16_iter = map(lambda x: x[0] / 32767, struct.iter_unpack("<h", pcm16))
    return np.fromiter(pcm16_iter, dtype=np.float32)

def base64_decode_audio(encoded):
    decoded = base64.b64decode(encoded)
    float32_array = pcm16_to_float(decoded)
    return float32_array

Initially my code threw an error as I expected only 4800 samples and allocated a buffer accordingly. After increasing the buffer size to 9600 samples I am able to replay the output audio stream without any distortions.
My questions is whether I am decoding the frames wrong or the documentation states a wrong frame size.

EDIT:
I use the following functions to encode audio, which I have as a NumPy array in float32 format:

def float_to_pcm16(float32_array):
    clipped = map(lambda x: max(-1.0, min(1.0, x)), float32_array)
    pcm16 = b"".join(struct.pack("<h", int(x * 32767)) for x in clipped)
    return pcm16

def base64_encode_audio(float32_array):
    pcm_bytes = float_to_pcm16(float32_array)
    encoded = base64.b64encode(pcm_bytes).decode("ascii")
    return encoded

Also I have run two tests in hope of narrowing down the issue.
In the first test I generated an array of random data, encoded and directly decoded it using the functions from above. Apart from some inaccuracies resulting from type casting the input and output arrays are equal. That rules out my encoding and decoding functions as the source of error.
The second test was about the behavior of the API. I generated an array of 48,000 samples of zeros (which will be interpreted as silence) and sent it as one chunk to the API. According to the documentation this chunk will be chopped into ten frames of 4800 samples each (as per assumption above 200ms equal 4800 samples) and each frame will be pushed into a processing queue. In my naive opinion I should receive ten (or a little bit more than ten, but definitely not less) frames, each with 4800 samples. To my surprise I received 16 frames of 9600 samples each. I received 3.2x more samples than I send to it despite sending only silence.
With each new step the behavior of the API gets more and more strange.

2

Hi and thank you for raising this!

I can confirm that the frame size is larger than expected. I have pinged the team to take a look.

3

This looks like a classic mismatch between documentation assumptions and actual decoded PCM framing. If the server states 200ms @ 24kHz (ā‰ˆ4800 samples) but the decoded buffer consistently yields ~9600 samples, it suggests either a frame aggregation layer (e.g., double-buffering or packet coalescing) or a discrepancy in how Base64 PCM chunks are being split before conversion. It would be useful to confirm whether the stream is delivering raw 200ms deltas or already merged audio frames before decoding, since that would fully explain the 2Ɨ sample inflation.