I have noticed this issue since early days - but ChatGPT continues to perform much better than APIs. Does anyone have any idea why ? Feels like APIs hide some of the strength on purpose (could be performance considerations?)
2 Likes
This discussion regarding the playground applies to your question:
Hmmm, not sure if it is your issue, but - How do I enable or disable memory in API? - #5 by qrdl
The memory might be providing a pretty huge capability jump.
I don’t think it’s memory or context. I do get a feel the chat continues to be ahead of APIs (pattern from early days)
The API is just part of the solution, but you still have to deal with system prompts, and actions, and architecture, and you can also programmatically, dynamically change the system prompts as it’s solving different tasks.
The way that you’re processing the memory in your application through the API, how are you rolling that memory in chat. There are so many decisions that you can make, that the API is not ever going to be the same as using ChatGPT, because they’re just very different solutions.
1 Like
I do see huge differences. For e.g. go to play ground, upload image and run - the assistant’s quality is much better: see the code: from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model=“gpt-4o”,
messages=[
{
“role”: “system”,
“content”: [
{
“type”: “text”,
“text”: "You are an AI specialized in generating Appian SAIL functions. Your goal is to create accurate and functional SAIL code based on provided design descriptions and images.\r\n \n\r\n \r\n\r\n Guidelines:\r\n 1. Analyze the Input: Carefully analyze the provided the image and its description to understand the requirements. Pay close attention to the image content to accurately capture all visual elements and their functionalities.\r\n 2. Generate SAIL Code: Create the SAIL functions based on your analysis. Use local variables.\n \r\n 3. Format the Answer: Ensure the final SAIL code is well-formatted and meets the requirements."""), \n "
}
]
},
{
“role”: “user”,
“content”: [
{
“type”: “image_url”,
“image_url”: {
“url”: “data:image/jpeg;base64,{image}”
}
},
{
“type”: “text”,
“text”: “The image you see is to add mortgage product. Please generate a Appian SAIL code based on it. This is an Appian SAIL Code”
}
]
}
],
temperature=0,
max_tokens=4095,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)
Use the code to invoke the API, the difference is day and night. From API the quality is bad.
The issue was with the base64 image. when I use proper formatting it worked.f"data:image/jpeg;base64,{base64_image}"
1 Like
@anil.jose is correct. I can reiterate. The API version continues to be a dumber version of ChatGPT (and i suspect they use a much longer context width for chatGPT because it seems to remember a LOT of details from a much older chat or may be they do some smart RAG to auto pull these, I don;t know). But I have never been able to get the same quality of response between API and the console version.
1 Like