Streaming an assistant's response in PHP

API
,
2

This modified version of your code implements a streaming-like behavior using Server-Sent Events (SSE). Here’s a breakdown of the changes:

  1. We set the appropriate headers for SSE at the beginning of the script.
  2. We’ve introduced a new stream_response function that continuously checks for new content and sends it to the client as it becomes available.
  3. The check_run_status function now returns the status of the run, which we use to determine when the assistant has completed its response.
  4. We’ve added a get_new_content function that retrieves new messages from the thread, comparing them with the last message ID we’ve sent to avoid duplicates.
  5. We use the send_sse function to send updates to the client, including new content and status updates.

To use this on the client side, you’d need to implement an EventSource to receive the SSE updates. Here’s a simple example of how you might do that in JavaScript:

<?php

// OpenAI API Key
$OPENAI_API_KEY = [your API key];

// Set headers for SSE
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
header('Connection: keep-alive');

// Flush headers
flush();

// Retrieve POST parameters
$data = json_decode(file_get_contents('php://input'), true);
$thread = isset($data['thread']) ? $data['thread'] : null;
$message = isset($data['message']) ? $data['message'] : null;

// Check the "message" parameter
if ($message === null || trim($message) === '') {
    send_error("'message' parameter is required.");
    exit();
}

// If the thread is null, create a new thread
if (is_null($thread)) {
    $thread = create_new_thread($OPENAI_API_KEY);
    if (!$thread) {
        send_error("Error creating the thread.");
        exit();
    }
}

// Send the message to the OpenAI API
$message_id = send_message_to_thread($OPENAI_API_KEY, $thread, $message);
if (!$message_id) {
    send_error("Error sending the message.");
    exit();
}

// Execute Plato's instructions
$run_id = execute_instructions($OPENAI_API_KEY, $thread);
if (!$run_id) {
    send_error("Error executing the instructions.");
    exit();
}

// Stream the response
stream_response($OPENAI_API_KEY, $thread, $run_id);

// Function to send SSE message
function send_sse($data) {
    echo "data: " . json_encode($data) . "\n\n";
    flush();
}

// Function to send error as SSE
function send_error($message) {
    send_sse(['error' => $message]);
}

// Function to stream the response
function stream_response($api_key, $thread, $run_id) {
    $last_message_id = null;
    $complete = false;
    $retry_count = 0;
    $max_retries = 50;  // Adjust as needed

    while (!$complete && $retry_count < $max_retries) {
        $run_status = check_run_status($api_key, $thread, $run_id);
        
        if ($run_status === 'completed') {
            $complete = true;
        } elseif ($run_status === false) {
            $retry_count++;
            sleep(1);
            continue;
        }

        $new_content = get_new_content($api_key, $thread, $last_message_id);
        
        if ($new_content) {
            send_sse(['content' => $new_content]);
            $last_message_id = $new_content['id'];
        }

        if (!$complete) {
            sleep(1);  // Wait for 1 second before checking again
        }
    }

    if ($complete) {
        send_sse(['status' => 'complete']);
    } else {
        send_error("The run could not be completed within the allotted time.");
    }
}

// Function to check the status of a run
function check_run_status($api_key, $thread, $run_id) {
    $url = "https://api.openai.com/v1/threads/$thread/runs/$run_id";
    $headers = [
        "Authorization: Bearer $api_key",
        "OpenAI-Beta: assistants=v2"
    ];

    $response = make_get_request($url, $headers);
    if ($response && isset($response->status)) {
        return $response->status;
    }
    return false;
}

// Function to get new content
function get_new_content($api_key, $thread, $last_message_id) {
    $url = "https://api.openai.com/v1/threads/$thread/messages";
    $headers = [
        "Authorization: Bearer $api_key",
        "OpenAI-Beta: assistants=v2"
    ];

    $response = make_get_request($url, $headers);
    if ($response && isset($response->data)) {
        foreach ($response->data as $message) {
            if ($message->role === "assistant" && (!$last_message_id || $message->id !== $last_message_id)) {
                return [
                    'id' => $message->id,
                    'text' => $message->content[0]->text->value
                ];
            }
        }
    }
    return null;
}

// ... (keep other functions like create_new_thread, send_message_to_thread, execute_instructions, make_post_request, and make_get_request as they are)

?>

This approach simulates streaming by sending chunks of the assistant’s response as they become available. It’s not a true streaming implementation like the one available for the completion API, but it should provide a similar user experience.

Remember to adjust the max_retries and sleep durations in the stream_response function based on your specific needs and the expected response time of the assistant.

Also, note that this implementation will keep the connection open until the assistant completes its response or the maximum number of retries is reached. Make sure your server configuration allows for long-running PHP scripts if you expect lengthy responses.

3 Likes