Token limit problems while prompting with lines of cods

Prompting
1

Continuing the discussion from About the Prompting category:

I am facing problems in prompting with lines of codes to be checked by GPT-4o using API refrence. It works good for up to 35 lines of code and for greater than 35 lines of code it gives message Fetch error: Unexpected token ‘d’, "denied by "… is not valid JSON.

1 Like
2

Welcome to the community!

Do you have an example of what you’re sending?

What model are you using?

Sounds like something in the code (probably a stray single quote) is messing it up…

1 Like
3

I am using GPT-4o-2024-08-06. I am sending request to analyze my code comprising of 50 plus lines of code, I am using VS Code.

1 Like
4

Is it a VS Code extension? Problem might be there?

5

no it is not extension I am using proper code files in PHP.

6

Right, but how is the code getting to the API from VS Code. That’s what’s messing up it sounds like…

7

here is send_message.php code

<?php require 'vendor/autoload.php'; use Dotenv\Dotenv; function send_message($message) { // Load environment variables $dotenv = Dotenv::createImmutable(__DIR__); $dotenv->load(); // Retrieve API key from environment variables $api_key = $_ENV['API_KEY']; if (!$api_key) { error_log("API_KEY not found"); return json_encode(['error' => "API_KEY not found"]); } // API endpoint for OpenAI $url = 'https://api.openai.com/v1/chat/completions'; // Prepare the data payload $data = [ "model" => "gpt-4o-2024-08-06", "messages" => [["role" => "user", "content" => $message]], "temperature" => 0.7 ]; // Set up HTTP options $options = [ 'http' => [ 'header' => [ "Content-type: application/json", "Authorization: Bearer $api_key" ], 'method' => 'POST', 'content' => json_encode($data), 'timeout' => 30 ] ]; // Create a stream context with the HTTP options $context = stream_context_create($options); // Send the request and get the response $result = file_get_contents($url, false, $context); // Handle errors if the request fails if ($result === FALSE) { $error = error_get_last(); error_log("file_get_contents failed: " . $error['message']); return json_encode(['error' => "Failed to get response from OpenAI."]); } // Log the raw response for debugging error_log("Raw response: " . $result); // Decode the JSON response $response_data = json_decode($result, true); // Validate the JSON response if (json_last_error() !== JSON_ERROR_NONE) { error_log("Invalid JSON response: " . $result); return json_encode(['error' => "Error: Invalid response from OpenAI."]); } // Extract and return the message content from the response if (isset($response_data['choices'][0]['message']['content'])) { return json_encode(['response' => $response_data['choices'][0]['message']['content']]); } else { error_log("Invalid response: " . json_encode($response_data)); return json_encode(['error' => "Error: Invalid response from OpenAI."]); } }
8

Is it always 35 lines of code exactly? Like I said, I think you’re getting something back that gets stuck here…

/ Validate the JSON response if (json_last_error() !== JSON_ERROR_NONE) { error_log("Invalid JSON response: " . $result); return json_encode([‘error’ => “Error: Invalid response from OpenAI.”]); } // Extract and return the message content from the response if (isset($response_data[‘choices’][0][‘message’][‘content’])) { return json_encode([‘response’ => $response_data[‘choices’][0][‘message’][‘content’]]); } else { error_log("Invalid response: " . json_encode($response_data)); return json_encode([‘error’ => “Error: Invalid response from OpenAI.”]); } }

9

Please suggest the corrected code if possible.

10

I’d have it print out normally + the json and compare…

Do you have an example of the code you’re sending that fails?

11

here is the code <?php
require ‘vendor/autoload.php’;

use Dotenv\Dotenv;

function send_message($message) {
// Load environment variables
$dotenv = Dotenv::createImmutable(DIR);
$dotenv->load();

// Retrieve API key from environment variables
$api_key = $_ENV['API_KEY'] ?? null;
if (!$api_key) {
    error_log("API_KEY not found");
    return json_encode(['error' => "API_KEY not found"]);
}

// API endpoint for OpenAI
$url = 'https://api.openai.com/v1/chat/completions';

// Prepare the data payload
$data = [
    "model" => "gpt-4o-2024-08-06",
    "messages" => [["role" => "user", "content" => $message]],
    "temperature" => 1.0
];

// Set up HTTP options
$options = [
    'http' => [
        'header'  => [
            "Content-type: application/json",
            "Authorization: Bearer $api_key"
        ],
        'method'  => 'POST',
        'content' => json_encode($data),
        'timeout' => 30
    ]
];

// Create a stream context with the HTTP options
$context = stream_context_create($options);

// Send the request and get the response
$result = file_get_contents($url, false, $context);

// Handle errors if the request fails
if ($result === FALSE) {
    $error = error_get_last();
    error_log("file_get_contents failed: " . $error['message']);
    return json_encode(['error' => "Failed to get response from OpenAI."]);
}

// Log the raw response for debugging
error_log("Raw response: " . $result);

// Check if the response is valid JSON
json_decode($result);
if (json_last_error() !== JSON_ERROR_NONE) {
    error_log("Response is not valid JSON: " . $result);
    return json_encode(['error' => "Received invalid response from OpenAI: " . substr($result, 0, 100)]);
}

// Decode the JSON response
$response_data = json_decode($result, true);

// Extract and return the message content from the response
if (isset($response_data['choices'][0]['message']['content'])) {
    return json_encode(['response' => $response_data['choices'][0]['message']['content']]);
} else {
    error_log("Unexpected response structure: " . json_encode($response_data));
    return json_encode(['error' => "Unexpected response structure from OpenAI."]);
12

No, the code you send to the API …

You might try not wrapping response in json to investigate…

ETA: What’s your system prompt? Are you asking for JSON back?

13

I am using three files to make Q&A session 1-index.php // Handle question submission using Fetch API with abort signal for stopping the request
var controller = new AbortController();
document.getElementById(‘questionForm’).onsubmit = function(event) {
event.preventDefault();
controller = new AbortController();
var formData = new FormData(this);
fetch(‘ask.php’, {
method: ‘POST’,
body: formData,
signal: controller.signal
})
.then(response => response.json())
.then(data => {
if (data.error) {
document.getElementById(‘response’).innerText = "Error: " + data.error;
} else {
document.getElementById(‘response’).innerText = data.response;
// Optionally handle balance update internally without displaying it
}
})
.catch(error => {
document.getElementById(‘response’).innerText = "Fetch error: " + error.message;
});
};
2-ask.php <?php
session_start();
require ‘vendor/autoload.php’;
require ‘connect.php’;
require ‘send_message.php’;

header(‘Content-Type: application/json’);
error_reporting(E_ALL);
ini_set(‘display_errors’, 0);
ini_set(‘log_errors’, 1);
ini_set(‘error_log’, ‘/path/to/your/error.log’);

try {
if ($_SERVER[‘REQUEST_METHOD’] === ‘POST’) {
// Retrieve the user message from POST data
user_message = _POST[‘question’] ?? ‘’;

    // Debugging information
    error_log("Received question: " . $user_message);

    // Combine the PDF text with the user's message if available
    $pdf_text = $_SESSION['pdf_text'] ?? '';
    $combined_input = $pdf_text ? ($pdf_text . "\n\nQuestion: " . $user_message) : $user_message;

    // Send the combined input to the send_message function
    $response_json = send_message($combined_input);

    // Log the raw response for debugging
    error_log("Raw API Response: " . $response_json);

    // Parse the JSON response
    $response = json_decode($response_json, true);
    
    if (json_last_error() !== JSON_ERROR_NONE) {
        // Log the JSON error for debugging
        error_log("JSON Error: " . json_last_error_msg());

        // Log the response causing the error
        error_log("Response that failed to parse: " . $response_json);

        echo json_encode(['error' => 'Failed to parse JSON response']);
        exit();
    }

    if (!isset($response['response'])) {
        echo json_encode(['error' => 'An unknown error occurred.']);
        exit();
    }

    // Retrieve the username from the session
    $username = $_SESSION['user'];

   

    // Insert the user message and response into the database
    $stmt = $link->prepare("INSERT INTO MESSAGES2 (USERNAME, user_message, response_message) VALUES (?, ?, ?)");
    if (!$stmt) {
        error_log("Prepare failed: " . $link->error);
        echo json_encode(['error' => 'Database prepare failed: ' . $link->error]);
        exit();
    }
    $stmt->bind_param("sss", $username, $user_message, $response['response']);
    $stmt->execute();
    $query_id = $stmt->insert_id;
    $stmt->close();

   
    // Final response to the client
    $response_array = [
        'response' => $response['response'],
               ];

    // Log the final response for debugging
    error_log("Final JSON response: " . json_encode($response_array));

    // Send the JSON response
    echo json_encode($response_array);

} else {
    echo json_encode(['error' => 'Invalid request method']);
}

} catch (Exception $e) {
error_log("Error: " . $e->getMessage());
echo json_encode([‘error’ => 'An error occurred: ’ . $e->getMessage()]);
}

// Close the database connection
$link->close();
3-send_message.php 4-I also have a function to upload pdf and set Q&A session about it.

1 Like
14

Actual I have created an app to make Q&A session with Language model using API key. It is working with text very well.

15

try setting max_completion_tokens if your code is long, 50 is not very much, depends on how long the lines are I guess.