# How to decrease the latency of Text-To-Speech API?

**URL:** <https://community.openai.com/t/how-to-decrease-the-latency-of-text-to-speech-api/727015>\
**Category:** API\
**Tags:** gpt-4, api\
**Created:** [April 25, 2024, 2:01am UTC](https://community.openai.com/t/how-to-decrease-the-latency-of-text-to-speech-api/727015 "2024-04-25T02:01:46Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![247-Teach](https://avatars.discourse-cdn.com/v4/letter/2/977dab/32.png) [@247-Teach](https://community.openai.com/u/247-Teach)\
**Post date:** [April 25, 2024, 2:01am UTC](https://community.openai.com/t/how-to-decrease-the-latency-of-text-to-speech-api/727015/1 "2024-04-25T02:01:46Z")

</div>

Hello all,

For context, I am using GPT4 API to perform Text-To-Speech and whenever I pass in a large amount of text the latency can reach up to almost a minute. I was wondering if there was a way to decrease the latency?

What I’ve found out so far. In the FAQ ([https://help.openai.com/en/articles/8555505-tts-api](https://help.openai.com/en/articles/8555505-tts-api)), they mention that we can stream the audio in chunks by setting stream=True.

I’ve tried this by using the code below:

response = self.client.audio.speech.create(  
_model_=“tts-1”,  
_voice_=“shimmer”,  
_input_=msg,  
_stream_=True)

But I’m getting this error:  
TypeError: Speech.create() got an unexpected keyword argument ‘stream’

So I’m a bit confused why it says that “stream” doesn’t exist when in their site it says that it does?

Any help is greatly appreciated 🙏

---

<div class="post-metadata">

**Author:** ![VeitB](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/veitb/32/712987_2.png) [@VeitB](https://community.openai.com/u/VeitB)\
**Post date:** [April 25, 2024, 7:31am UTC](https://community.openai.com/t/how-to-decrease-the-latency-of-text-to-speech-api/727015/2 "2024-04-25T07:31:09Z")

</div>

Hi!

Here is the relevant info from the documentation:  
[https://platform.openai.com/docs/guides/text-to-speech/streaming-real-time-audio](https://platform.openai.com/docs/guides/text-to-speech/streaming-real-time-audio)

In general it is preferable to refer the docs on the platform instead of the help files. The documentation is updated more frequently.

---

<div class="post-metadata">

**Author:** ![247-Teach](https://avatars.discourse-cdn.com/v4/letter/2/977dab/32.png) [@247-Teach](https://community.openai.com/u/247-Teach)\
**Post date:** [April 26, 2024, 3:42am UTC](https://community.openai.com/t/how-to-decrease-the-latency-of-text-to-speech-api/727015/3 "2024-04-26T03:42:00Z")

</div>

Hello, and thanks for responding!

I tried doing the method within the link you provided and it doesn’t actually seem like its streaming the audio. When I call on the stream\_to\_file(), the file gets generated but I can’t play it until the whole text has been processed.

---

<div class="post-metadata">

**Author:** ![VeitB](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/veitb/32/712987_2.png) [@VeitB](https://community.openai.com/u/VeitB)\
**Post date:** [April 26, 2024, 7:27am UTC](https://community.openai.com/t/how-to-decrease-the-latency-of-text-to-speech-api/727015/4 "2024-04-26T07:27:27Z")

</div>

You need to handle audio data in chunks as they arrive rather than waiting for the entire file to be completed. In Python there are various libraries to achieve this, such as `pyaudio`, to play audio directly from byte streams.

---

<div class="post-metadata">

**Author:** ![dignity\_for\_all](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/dignity_for_all/32/300058_2.png) [@dignity\_for\_all](https://community.openai.com/u/dignity_for_all)\
**Post date:** [April 26, 2024, 8:51am UTC](https://community.openai.com/t/how-to-decrease-the-latency-of-text-to-speech-api/727015/5 "2024-04-26T08:51:03Z")

</div>

> [@247-Teach](#):
>
> I’m a bit confused why it says that “stream” doesn’t exist when in their site it says that it does?

The comment on the GitHub issue page mentions that the “stream\_to\_speakers” method became available with PyAudio on March 3rd.

> <https://github.com/openai/openai-python/issues/864#issuecomment-1974963204>
>
> \### Confirm this is an issue with the Python library and not an underlying OpenA…I API
> 
> \- \[X\] This is an issue with the Python library
> 
> \### Describe the bug
> 
> When following the documentation on how to use \`client.audio.speech.create()\`, the returned response has a method called \`stream\_to\_file(file\_path)\` which explains that when used, it should stream the content of the audio file as it's being created. This does not seem to work. I've used a rather large text input that generates a 3.5 minute sound file and the file is only created once the whole request is completed.
> 
> \### To Reproduce
> 
> Utilize the following code and replace the text input with a decently large amount of text.
> \`\`\`python
> from pathlib import Path
> from openai import OpenAI
> client = OpenAI()
> 
> speech\_file\_path = Path(\_\_file\_\_).parent / "speech.mp3"
> response = client.audio.speech.create(
> model="tts-1",
> voice="alloy",
> input="""
> \<Decently large bit of text here\>
> """
> )
> 
> response.stream\_to\_file(speech\_file\_path)
> \`\`\`
> 
> Notice that when the script is run that the \`speech.mp3\` file is only ever created \_after\_ the request is fully completed.
> 
> \### Code snippets
> 
> \_No response\_
> 
> \### OS
> 
> macOS
> 
> \### Python version
> 
> Python 3.11.6
> 
> \### Library version
> 
> openai v1.2.4

---

<div class="post-metadata">

**Author:** ![247-Teach](https://avatars.discourse-cdn.com/v4/letter/2/977dab/32.png) [@247-Teach](https://community.openai.com/u/247-Teach)\
**Post date:** [April 26, 2024, 10:10pm UTC](https://community.openai.com/t/how-to-decrease-the-latency-of-text-to-speech-api/727015/6 "2024-04-26T22:10:46Z")

</div>

Gotcha, I’ll try it out. Thanks!

---

<div class="post-metadata">

**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\
**Post date:** [April 26, 2024, 10:36pm UTC](https://community.openai.com/t/how-to-decrease-the-latency-of-text-to-speech-api/727015/7 "2024-04-26T22:36:28Z")

</div>

Here’s receiving a stream of chunks into a buffer (and only afterwards playing WAV.)

```python
import io
import pyaudio
from openai import OpenAI

def byteplay(bytestream):
    pya = pyaudio.PyAudio()
    stream = pya.open(format=pya.get_format_from_width(width=2), channels=1, rate=24000, output=True)
    stream.write(bytestream)
    stream.stop_stream()
    stream.close()

client = OpenAI()

with client.audio.speech.with_streaming_response.create(
    model="tts-1",
    voice="alloy",
    input="hello there, I'm making a WAV file today",
    response_format='wav'
) as response:
    # Initialize an empty bytes buffer
    buffer = io.BytesIO()
    
    # Read audio data from the generator
    for chunk in response.iter_bytes():
        print(len(chunk))
        buffer.write(chunk)

    # Go back to the start of the buffer
    buffer.seek(0)

    # Play the audio from the buffer
    byteplay(buffer.getvalue())
    #print(len(buffer.getvalue()))

```

So you can pull out your http chunks and stream the audio again.

Can WAV be instantly played? Yes, if you want immediate buffer underruns and noise. With a 100k preload threaded playback buffer, I only got a sentence to play before buffer underrun on my WiFi PC: the API doesn’t send uncompressed audio fast enough even at 24k/1ch. You also can’t know the final length from the stream to know how much to buffer even if you measure the stream rate.

> **buffered wav streaming with pyaudio**
>
> ```python
> import pyaudio
> import queue
> import threading
> from openai import OpenAI
> 
> def play_audio_data(pya, audio_queue):
> """ Plays audio chunks from the queue. """
> stream = pya.open(format=pya.get_format_from_width(width=2), channels=1, rate=24000, output=True)
> while True:
> chunk = audio_queue.get()
> if chunk is None: # Sentinel value to stop the playback
> break
> stream.write(chunk)
> stream.stop_stream()
> stream.close()
> 
> def stream_audio(model: str, voice: str, input_text: str, initial_buffer_size: int = 150000):
> pya = pyaudio.PyAudio()
> audio_queue = queue.Queue()
> 
> client = OpenAI()
> 
> with client.audio.speech.with_streaming_response.create(
> model=model,
> voice=voice,
> input=input_text,
> response_format='wav'
> ) as response:
> buffer = b'' # Temporary buffer to accumulate initial chunks
> playback_started = False
> play_thread = None
> 
> # Process each chunk only once
> for chunk in response.iter_bytes():
> if not playback_started:
> buffer += chunk
> # Check if initial buffer is sufficiently filled
> if len(buffer) >= initial_buffer_size:
> # Start the playback thread once the buffer size is reached
> audio_queue.put(buffer) # Send the initial buffer to the queue
> play_thread = threading.Thread(target=play_audio_data, args=(pya, audio_queue))
> play_thread.start()
> playback_started = True
> buffer = b'' # Clear the initial buffer since it's now in the queue
> else:
> audio_queue.put(chunk)
> 
> if not playback_started:
> # If the stream ends before filling the initial buffer, start playback with whatever we have
> audio_queue.put(buffer)
> play_thread = threading.Thread(target=play_audio_data, args=(pya, audio_queue))
> play_thread.start()
> 
> # End signal for the playback thread
> audio_queue.put(None)
> if play_thread:
> play_thread.join()
> 
> # Cleanup
> pya.terminate()
> 
> # Usage
> stream_audio(model="tts-1", voice="alloy", input_text="hello there, I'm making a wav file today")
> 
> ```

Thus we must go compressed (and I choose open source) to make this thing talk some paragraphs before a chat completions response is even done.

Unlike AAC, which is a raw stream and you have to mux it into a mp4 if you want a normal file, when specifying `opus`, the Opus is wrapped in OGG file stream.

An ogg audio is sent in small http chunks by OpenAI, but the actual OGG frames are large. The first two ogg frames have no audio, but lots of null tag space. The internal 20ms latency of Opus as a codec cannot be accessed. One must do a couple rounds of buffering, reassembling OGG packets, and decoding and playing those gapless. That puts at least a sentence delay before you’d be able to start playing without buffer overruns on a Python app. I’ll make it work though…

Just resending OGG packets to a browser, you can let that robust WebRTC client figure out the playing.
