Hi, I need to implement the same functionality as in ChatGPT’s “Stop Generating” function. I am using a Completion API call. How can I request this via the API? The Stop Sequences in the initial API call is not what I am looking for. Thanks, and help would be greatly appreciated.

why doesn’t stop sequence do what you need? Can you use streaming and simply close the stream?

Thanks for looking at this. There is no pre-determined reason to stop generation, ie Stop Sequences won’t apply. The reason to stop generating is if a user realises the answer is poor, due to a poor prompt formulation. I.e. need exactly like the “Stop generating” in ChatGPT. I am using streaming…

Sorry, without knowing a lot more about your code I can’t help much. Are you getting the stream in a callback? If the callback is running in a separate thread (most likely), then if you are using python you can probably just call sys.exit(0) python threads (other than the original one) just quietly…

You need to use the streaming API (use server-sent events.) Then you can reset/close/abort the connection when you believe you’ve gotten enough text and want to stop it. There is no way to implement “stop generating” on top of the batch API. There’s a number of resources on the web about how to u…

I generate an instance of AbortController with new AbortController(). This controller possesses a signal attribute, which represents an instance of AbortSignal. During the creation of the fetch request to the server, this signal is introduced as part of the fetch options, thus linking the fetch requ…

@Humboldt Do you have any source code that you don’t mind sharing?? I implemented as you did, but it does not work. I mean when I call abortController.abort() and controller.signal.aborted changed to “false” but the stream did not stop.

Hi, sorry I am very bad at explaining things but here is the key parts of my source code I’m using to this day, let controller; let abortBtn = document.getElementById('abortBtn'); abortBtn.addEventListener('click', function () { if (controller) { controller.abort(); } }); submitButton.add…

I’m curious, if I disconnect, will the AI inference behind ChatGPT stop immediately or not? AI inference means AI create completion progress. Some AI services explain that once the AI service starts generating a completion, it cannot be stopped, so I’m not sure if ChatGPT is like that.

If you are using streaming mode then generation will stop within a few 10’s of tokens (7-30 in testing clustering more at the lower end), if you are not in streaming mode the full reply will be generated irrespective of if you are there to receive it.

ChatGPT's "Stop Generating" function - how to implement?

API

alanhg November 1, 2023, 3:36pm 11

So if it’s in streaming mode, I understand that I only need to pay for the cost of the tokens I received, right?

Topic		Replies	Views
Interrupting completion stream in Python API	12	14761	December 14, 2023
How to abort create chat completion streaming? I use Nodejs + Typescript API	6	8086	September 17, 2023
If we stop streaming output stream before it finishes, do we still get billed for the tokens that weren't ouputted? API	1	818	July 9, 2024
Implementing a 'Stop Generating' Function for OpenAI Streams API api	0	1785	March 29, 2024
Api Billing for streaming if I close connection midway Community gpt-4 , api-billing	1	1353	February 13, 2024

ChatGPT's "Stop Generating" function - how to implement?

Related topics