Curl api.openai.com is slow, 15-30s/req

API
21

Thank you.

However, the completion time was around 13 seconds (see image) so it was not any faster than your 12s Python test case.

:slight_smile:

1 Like
22

A few conclusions:
davinci’s return time is very fast.
The model of interest is gpt-3.5-turbo
With PHP Curl language, it takes 10-20s per query. To be able to integrate Steam into curl is very difficult.
With JavaScript - Fetch or JavaScript - jQuery triggering the data stream returns feels faster but it will expose the API key
Currently with python it takes 10-12s. (March 15th GMT +7)

1 Like
23

I have see stream

If set, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message. See the OpenAI Cookbook for example code.

My code new:

$url = 'https://api.openai.com/v1/chat/completions';
$model = "gpt-3.5-turbo";

$header = array(
    'Authorization: Bearer '.$API_KEY,
    'Content-type: application/json',
);

$params = json_encode(array(
    'messages'        => $data,
    'model'         => $model,
    'temperature'   => 1,
    'max_tokens'    => 1500,
    'top_p'         => 1,
    'frequency_penalty' => 0,
    'presence_penalty'  => 0,
    'stream'        => True
));

// create a new cURL resource
$curl = curl_init();
// set URL and other appropriate options
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_HEADER, 0);

curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl, CURLOPT_POSTFIELDS, $params);
// enable streaming
curl_setopt($curl, CURLOPT_BINARYTRANSFER, 1);
curl_setopt($curl, CURLOPT_WRITEFUNCTION, function($curl, $chunk) {
    // do something with the streamed data
    echo $chunk;
    return strlen($chunk);
});

// execute the request
curl_exec($curl);

// close cURL resource
curl_close($curl);

With this option I got the first signal after 4s and the contents returned continuously, this made the read backwards feel no delay as they need to read until the last text is printed to the screen.

Demo:

@raymonddavey tks for “stream”

1 Like