Streaming, and newlines, and empty responses

I am having understanding how to get a formatted response with stream=true. I have prompted asking for markdown and for newlines to be used for readability. I get the markdown, but no newlines.
Also some of the response lines are empty, like the data:‘’
Any help or guidance on how to 1) enforce newlines, 2) whats up with these empty lines in stream response., very much appreciated!

The AI struggles to interpret the language of “newlines” into producing the correct character, I noticed.

Much better is to instruct by “create a separate paragraph”, or “make individual lines” “produce a blank line”…

You can try out that different prompting, even providing examples in the system message, and see if the AI understands better.


There will be null chunks - the very first. Then data: in SSE can be quirky not guaranteed to be complete in one transmission, and need to be reassembled until the completion of a line (if you aren’t using a library).

The AI also may emit tools or functions as chunks that may or may not have content that is null. Best to have a parser that outputs all non-null context while handling others, while also gathering tool-call that could conceivably happen anywhere. So it is good to build a parser that can deal with most anything as chunk and most anything as data: splits.

(then you can just see if the AI is writing nothing…)

Welcome @powellt

Streaming responses can be easily handled with the OpenAI client package for the language you use.

e.g for python:

from openai import OpenAI

client = OpenAI()

stream = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Say this is a test"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

As is evident from the sample code from OpenAI python package the NoneType chunks can simply be handled by replacing them with empty strings.

If any formatting chars like newline, etc. present in the generated response are received in the streaming response, they will reflect as such when the response is printed as it is received.