Print message token by token

API
1

Hello,

at the moment my tool sends a AJAX request using a PHP file that sends it to https://api.openai.com/v1/chat/completions and generates a JSON message that is rendered via JavaScript.

The problem is that the output is rendered at a strech and not token-via-token.

You could i implement that?

Best regards
Andreas

2

Meanwhile i got it to work. Here’s a small working example:

index.html:

<html>

    <head>
        <title>Stream Demo</title>
    </head>

    <body>

        <div id="content"/>
        <script>
           var eventSource = new EventSource("api.php");

           eventSource.onmessage = function (e) {
              if(e.data == "[DONE]")
              {
                  document.getElementById('content').innerHTML += "<br><br>Finished.";
                  eventSource.close();
              } else {
                  if (JSON.parse(e.data).choices[0]['delta']['content']) {
                    document.getElementById('content').innerHTML += JSON.parse(e.data).choices[0]['delta']['content'];
                  }
              }
           };
           eventSource.onerror = function (e) {
               console.log(e);
           };
        </script>

    </body>
</html>

api.php

<?php

    /* Andreas Koch, abkoch (at) posteo.de */

    $api_key = '<put-your-key-here>';
    $api_url = 'https://api.openai.com/v1/chat/completions';

    $ch = curl_init();

    $messages[] = array("role" => "user", "content" => "Tell me a story about you.");

    $post_fields = array(
        "model" => "gpt-3.5-turbo",
        "stream" => true,
        "temperature" => 0.7,
        "max_tokens" => 512,
        "top_p" => 1,
        "frequency_penalty" => 0,
        "presence_penalty" => 0,
        "stop" => ["\\n"],
        "messages" => $messages
    );

    $header  = [
        'Content-Type: application/json',
        'Authorization: Bearer ' . $api_key
    ];

    header('Content-Type: text/event-stream');
    header('Cache-Control: no-cache');
    header('Connection: keep-alive');
    header('X-Accel-Buffering: no'); 

    curl_setopt($ch, CURLOPT_URL, $api_url);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($post_fields));
    curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
    curl_setopt($ch, CURLOPT_WRITEFUNCTION, function ($ch, $data) {
       echo $data;
       echo PHP_EOL;
       ob_flush();
       flush();
       return strlen($data);
    });

    curl_exec($ch);

?>
1 Like
3

stream (defaults to false…)

If set, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message. See the OpenAI Cookbook for example code.

Here you go. Hope your weekend is going well…

1 Like
4

Thanks. I already found a solution and posted it yesterday. But Askimet blocked it.

5

Did you paste in the entire response and then send it?

As a Discourse admin on another site Askimet will flag post by users with low Discourse trust levels that were not entered by hand. The reason for flagging such post is that this is a key indicator of spam bots as bots are much faster than humans at creating post. If it has been more than 24hrs since posting and if the admins and moderators here were doing what they need to then your post should have been reviewed and a response sent back already.

I was once on another well known site where the all the admins and moderators were not active for months, so I turned the use of the site into a social experiment, I hope the same does not happen here.

6

Yes, i copy&pasted my code out of two files i’ve created.

My trust level was increased some minutes ago. So i give it another try:

api.php:

<?php

    $api_key = '<put-your-key-here>';
    $api_url = 'https://api.openai.com/v1/chat/completions';

    $ch = curl_init();

    $messages[] = array("role" => "user", "content" => "Tell me a story about you.");

    $post_fields = array(
        "model" => "gpt-3.5-turbo",
        "stream" => true,
        "temperature" => 0.7,
        "max_tokens" => 512,
        "top_p" => 1,
        "frequency_penalty" => 0,
        "presence_penalty" => 0,
        "stop" => ["\\n"],
        "messages" => $messages
    );

    $header  = [
        'Content-Type: application/json',
        'Authorization: Bearer ' . $api_key
    ];

    header('Content-Type: text/event-stream');
    header('Cache-Control: no-cache');
    header('Connection: keep-alive');
    header('X-Accel-Buffering: no'); 

    curl_setopt($ch, CURLOPT_URL, $api_url);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($post_fields));
    curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
    curl_setopt($ch, CURLOPT_WRITEFUNCTION, function ($ch, $data) {
       echo $data;
       echo PHP_EOL;
       ob_flush();
       flush();
       return strlen($data);
    });

    curl_exec($ch);

?>

index.html:

<html
    <head>
        <title>Stream Demo</title>
    </head>

    <body>

        <div id="content"/>
        <script>
           var eventSource = new EventSource("api.php");

           eventSource.onmessage = function (e) {
              if(e.data == "[DONE]")
              {
                  document.getElementById('content').innerHTML += "<br><br>Finished.";
                  eventSource.close();
              } else {
                  if (JSON.parse(e.data).choices[0]['delta']['content']) {
                    document.getElementById('content').innerHTML += JSON.parse(e.data).choices[0]['delta']['content'];
                  }
              }
           };
           eventSource.onerror = function (e) {
               console.log(e);
           };
        </script>

    </body>
</html>
2 Likes