Dear Developers,
I am writing to request assistance with an issue that we are currently experiencing with the OpenAI API (GPT 3.5 Turbo) integrated into our PHP based web application. Our web app uses Dialogflow ES to power a conversational chatbot, and we have connected Dialogflow ES with OpenAI, adding a knowledge module for enhanced functionality using a webhook.
Here is the code:
//ini_set('display_errors', 1);
//ini_set('display_startup_errors', 1);
//error_reporting(E_ALL);
header('Content-Type: application/json');
require('../config.php');
include 'vendor/autoload.php';
use Orhanerday\OpenAi\OpenAi;
$data = json_decode(file_get_contents('php://input'), true);
$query = $data['queryResult']['queryText'];
//$query = "what is SPOT";
if($query == "sb-follow-up-form"){
return;
}
$url = "knowledge.txt";
$content = file_get_contents($url);
$messages_array = [];
$messages_array[] = ['role' => 'system', 'content' => 'Following is information about our business, You have to act like you are the a human agent in our support system and do not let the user know about the fact you are an AI language model. This is information of my business:' . $content. 'If the user says or ask anything which is irrelevant with spot, Deny that request or dont respond to that statement by saying that you cannot help them with anything out of SPOT, make sure to not blow your cover by saying you are an AI model. If the user asks anything about you, you are supposed to introduce yourself as "Agent for SPOT Support", If they ask you how are you or anything regarding your or their feelings, make sure to never state that you are an AI Lanugage model.'];
$support_board_data = $data['originalDetectIntentRequest']['payload']['support_board'];
$conversation_id = $support_board_data['conversation_id'];
$user_id = $support_board_data['user_id'];
//$conversation_id = 293;
//$user_id = 334;
$is_last_automated = false;
if($conversation_id){
$conversation_data = sb_get_conversation($user_id, $conversation_id);
foreach ($conversation_data['messages'] as $message) {
$role = ($message['user_type'] === 'agent' || $message['user_type'] === 'bot') ? 'assistant' : 'user';
if (strpos($message['message'], '[email id=') === false ) {
if (!empty($message['payload'])) {
$payload = json_decode($message['payload'], true);
if (isset($payload['rich-messages']['sb-follow-up-form']['result'])) {
$result = $payload['rich-messages']['sb-follow-up-form']['result'];
$first_name = $result['first_name'][0] ?? '';
$last_name = $result['last_name'][0] ?? '';
$email = $result['email'][0] ?? '';
$phone = $result['phone'][0] ?? '';
$content = "Follwing is my information: First Name: $first_name, Last Name: $last_name, Email: $email, Phone: $phone";
$messages_array[] = ['role' => "user", 'content' => $content];
$is_last_automated = true;
} else {
$messages_array[] = ['role' => $role, 'content' => $message['message']];
$is_last_automated = false;
}
}else{
$messages_array[] = ['role' => $role, 'content' => $message['message']];
$is_last_automated = false;
}
}
}
}
$messages_array[] = ['role' => 'user', 'content' => $query];
if($query && $is_last_automated == false)
{
$open_ai = new OpenAi("■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■H7cCwDI8");
$gpt_response = call_gpt($open_ai, $messages_array);
//$gpt_response = "I cant answer that :D";
$dialogflow_response = format_dialogflow_response($gpt_response);
echo json_encode($dialogflow_response);
}
function call_gpt($open_ai, $messages_array) {
$headers = [
'Content-Type: application/json',
'Authorization: Bearer '
];
$chat = $open_ai->chat([
'model' => 'gpt-3.5-turbo',
'messages' => $messages_array,
'max_tokens' => 150,
'temperature' => 0.5,
'top_p' => 0.8,
]);
$response_data = json_decode($chat, true);
$gpt_response = $response_data['choices'][0]['message']['content'];
return $gpt_response;
}
function call_gpt_curl( $messages_array) {
$headers = [
'Content-Type: application/json',
'Authorization: Bearer ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■H7cCwDI8'
];
$data = [
'model' => 'gpt-3.5-turbo',
'messages' => $messages_array,
'max_tokens' => 150,
'temperature' => 0.5,
'top_p' => 0.8,
];
$ch = curl_init('https://api.openai.com/v1/chat/completions');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
$response = curl_exec($ch);
curl_close($ch);
$response_data = json_decode($response, true);
$gpt_response = $response_data['choices'][0]['message']['content'];
return $gpt_response;
}
function format_dialogflow_response($gpt_response) {
$response_data = [
"fulfillmentMessages" => [
[
"text" => [
"text" => [
$gpt_response
]
]
]
]
];
return $response_data;
}
function sb_get_conversation($user_id, $conversation_id) {
// connect to the database
$db = new mysqli(SB_DB_HOST, SB_DB_USER, SB_DB_PASSWORD, SB_DB_NAME);
// check connection
if ($db->connect_error) {
die("Connection failed: " . $db->connect_error);
}
// fetch the messages
$messages_result = $db->query(
"SELECT sb_messages.message, sb_messages.payload, sb_messages.conversation_id, sb_users.user_type
FROM sb_messages
INNER JOIN sb_users ON sb_messages.user_id = sb_users.id
WHERE sb_messages.conversation_id = $conversation_id
ORDER BY sb_messages.id ASC
"
);
if (!$messages_result) {
die("Query failed: " . $db->error);
}
$messages = [];
while ($message = $messages_result->fetch_assoc()) {
$messages[] = $message;
}
return [
'messages' => $messages,
];
}
?>
Here’s a brief description of our workflow:
- The user sends a message through our web app.
- The message is transferred to Dialogflow ES, which identifies the user’s intent.
- The identified intent activates a webhook that fetches the questions along with the available business data.
- This data is then sent to the OpenAI API.
- OpenAI API processes the question, evaluates the business data, generates a response, and sends it back to Dialogflow ES.
- Dialogflow ES then transmits this information back to our web app (support board), where the user can view the response.
The issue we are encountering is related to the response time from the OpenAI API. Dialogflow ES has a time limit of 5 seconds for the webhook to process the request and return a response. During the development with a free account API (which was the developer’s personal account), we observed the requests were being processed within 3-4 seconds. However, when using an API from a different paid account, the same requests are taking 8-10 seconds to process.
This delay poses a significant problem for us as our system is designed to work within the 5 seconds constraint set by Dialogflow ES, and there are no options for us to adjust this timing on our end.
We are seeking your assistance to understand why the response times vary so significantly between different accounts and if there is a way to ensure our API requests are processed and responses are returned within the 5-second window. If possible, could we be allocated to a faster processing cluster, or could there be any other solution to reduce this response time?
Looking forward to your prompt response and assistance in resolving this issue.
This is the Error on dialogflow ES:
Webhook call failed. Error: DEADLINE_EXCEEDED, State: URL_TIMEOUT, Reason: TIMEOUT_WEB.
Best regards,
Team SPOT.