Streaming GPT chat completions

GPT builders
1

Have not seen this clearly defined on the forum. So I thought I’d share. This script allows for output from GPT to be written directly to terminal as it is being created(streamed). It also allows for very large text inputs so you can copy and paste gigantic pieces of code into the terminal without hitting terminal limits or having to upload any files. Conversation experience similar to Chat GPT on the website. Enjoy.

from openai import OpenAI
import time
client = OpenAI()

while True:
    print("You (type 'END' on a new line when finished):")
    user_input = ""
    while True:
        line = input()
        if line.strip().lower() == 'end':
            break
        user_input += line + "\n"

    if user_input.lower().strip() in ['exit', 'quit']:
        print("Exiting the chat.")
        break
    
    messages = [{"role": "user", "content": user_input}]
    
    try:
        stream = client.chat.completions.create(
            model="gpt-4-0125-preview",
            messages=messages,
            stream=True,
        )
        
        for chunk in stream:
            delta_content = chunk.choices[0].delta.content if chunk.choices[0].delta.content is not None else None
            if delta_content:
                print(delta_content, end="", flush=True,)
                time.sleep(0.1)  # Delay
            else:
                print("AI: End of stream")
    except Exception as e:
        print(f"An error occurred: {e}")
1 Like
2

You can start your own Internet Radio Station in minutes for FREE using the AZ-Streaming Servercom Platform.