Incomplete Output of 4o through API

API
1

Works well in Playground, not through API. My complex prompt with several steps often leads to an incomplete output.
How is it possible, is there a difference between API and Playground 4o? What may cause this?

2

image
image771×550 95.2 KB

image
image712×504 19.1 KB

First, you can press the ‘view code’ button to ensure you have the correct parameters. Then despite those, crank the max_tokens value you can receive as output way up. You can see my truncation at 128 tokens.

Then look at the finish_reason in the response. If it is “stop”, the AI just decided to stop writing.

Another oddity about this model is that specifying temperature and top_p values too low is not just a setting to zero, it a setting to “useless”. The parsing of negative exponents is screwed up, perhaps, or math is done with reduced bits that overflow. So a setting like 0.0001 that still allows variance is needed to have moderately replicable outputs.

1 Like
3

Still not sure what was happening but after specifying max output tokens and setting top p from 1 to 0.5, it works reliably.

1 Like
4

May be there is issue in playground api.
I too had the same problem when am doing in the playground its working fine but when am using api am getting some different results.
Exactly what my problem is
Am extracting the information from the image.
Its working fine when am trying with the playground am getting the answers without spelling mistakes, but when an using the API am getting the spelling mistakes.Tweaking with different parameters like temperature and top_p no change in response.

import openai
import base64
import csv
import re
import json
# Set up your OpenAI API key
openai.api_key = 'sk-**************'

# Path to the image file
image_path = '/path_of_the_image.png'

# Function to read image and convert to base64
def image_to_base64(image_path):
    with open(image_path, "rb") as image_file:
        encoded_string = base64.b64encode(image_file.read()).decode('utf-8')
    return encoded_string

# Path to the image


# Convert image to base64
image_base64 = image_to_base64(image_path)

# Define the prompt
prompt = """PCN,Patient ID,Patient Full Name,Claim ID,Paid Amount,Extract the data of all the rows from the image and map with the above columns in json format.
```json
{ 
  "PCN": "", 
  "Patient ID": "32357", 
  "Patient Full Name": "xyz", 
  "Claim ID": "12345678", 
  "Paid Amount": "0.00", 
}
"""

# Make the API call
response = openai.ChatCompletion.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Act as a Data Entry Agent and extract the data from the image."},
        {"role": "user", "content": prompt},
        {"role": "user", "content": [{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}}]}
    ],
    temperature=0,
    max_tokens=4095,
    top_p=1,
    frequency_penalty=0,
    presence_penalty=0
)

# Print the response
json_data = response['choices'][0]['message']['content']
# print(response['choices'][0]['message']['content'])
print(json_data)