Interrupting completion stream in Python

General API discussion
1

Is it possible to interrupt completion stream and not waste tokens? E.g. when I see that it’s looping or going in the wrong direction.

I know I can use stream option and then use the response object like a generator.

response = openai.Completion.create(
    ...
    stream=True,
)
for line in response:
    print(line)

But is it enough to just jump out of the loop when I decide enough is enough? Will the server then stop generating the rest of the tokens?

2

I think what you are looking for is Stop Sequences here is a short guide on how to use them How do I use Stop Sequences? | OpenAI Help Center

3

I don’t think there’s a way to stop it once the stream starts, but I may be wrong.

4

I’m looking for something like the option to interrupt in the playground. You can just cancel the stream if you see that it’s going in the wrong direction, though I’m not sure if this is really telling the server to stop computing.

I’m not looking for Stop Sequences. I just want the user of my app to have the ability to quickly stop and try something else (and possibly not waste all tokens).

5

@lemi did you figure this out? I have the same question: ability to stop steaming, not via stop sequences, to save token costs when it is clearly going in an unproductive direction (e.g. repetitions).

6

Hey guys,
did you figure out this issue? or any alternative solutions?