Streaming completion in Python

General API discussion
1

Hi,

Does anyone have a working code snippet for how to make streaming work in python? All the discussion I’ve seen is about doing this in JavaScript.

Basically, I want the counterpart of the following where stream=True:
r = openai.Completion.create(
model=“code-davinci-002”,
prompt= prompt",
temperature=0,
max_tokens=4096,
top_p=1,
frequency_penalty=0,
presence_penalty=0,
stop=[“I:”, “O:”]
)

I tried using sseclient and urllib3, but can’t get this to work.

Thanks!

1 Like
2

I also looking the answer for this question.
Could you share with me, how they do it with JavaScript?

3

Yea just pass stream=True and handle with a generator

4

could you give example please, or point me to tutorial, thats would be helpful.

1 Like
5

Here’s a very quick example that streams tokens and prints out each token as it comes in:

for resp in openai.Completion.create(model='code-davinci-002', prompt='def hello():', max_tokens=512, stream=True):
    sys.stdout.write(resp.choices[0].text)
    sys.stdout.flush()
2 Likes
6

So, this part works with the content. But what about the token cost? It is sent via server-sent event. Any way to obtain it? Thanks