Does anyone have a working code snippet for how to make streaming work in python? All the discussion I’ve seen is about doing this in JavaScript.
Basically, I want the counterpart of the following where stream=True:
r = openai.Completion.create(
model=“code-davinci-002”,
prompt= prompt",
temperature=0,
max_tokens=4096,
top_p=1,
frequency_penalty=0,
presence_penalty=0,
stop=[“I:”, “O:”]
)
I tried using sseclient and urllib3, but can’t get this to work.
Here’s a very quick example that streams tokens and prints out each token as it comes in:
for resp in openai.Completion.create(model='code-davinci-002', prompt='def hello():', max_tokens=512, stream=True):
sys.stdout.write(resp.choices[0].text)
sys.stdout.flush()