Discrepancy in OpenAI API Response Quality Compared to Console Output

API
, ,
1

I’m facing an issue while using the OpenAI API through their npm package. In each API call, I incorporate a prompt, user history, and the current question with parameters like temperature, top_p, and frequency_penalty. I am using ‘createChatCompletion’ function from the openAI package

However, the quality of responses from the API lacks consistency and struggles to retain past conversation context effectively. Interestingly, when I replicate the same prompt in the openAI console, I receive more creative responses that maintain better contextual understanding.

I’m seeking insights for this on how to address this discrepancy and enhance the API’s response quality accordingly. Has anyone faced the similar issues? or can help in this problem?

2

Welcome to the forums. Can you share some sample code and a sample screenshot of the Playground where you are seeing this discrepancy?

1 Like
3

These are queries and answers between the user and openAI console

Query - "Hi"
Answer - "Hello! I'm here to listen and support you. What's on your mind?"

Query - "I lost my job"
Answer - "I'm here to listen and understand. Losing a job can be tough. Can you tell me more about what you're feeling and thinking since this happened?"

Query - "I am feeling stressed"
Answer - "I'm here to support you through this stress. Can you describe the thoughts that are accompanying this feeling of stress? What comes to mind when you think about losing your job?"

Query - "How will I survive from this?"
Answer - "It's natural to have concerns about your future after losing a job. Let's take a closer look at this thought: Are there any specific aspects of your situation that make you worry about survival? Is there any evidence that suggests you might not be able to manage this situation?"

These are queries and answers between the user and my API

Query - "Hi"
Answer - "Hello! What's on your mind?"

Query - "I lost my job"
Answer - "I'm sorry to hear that you lost your job. It can be a difficult and stressful experience. What thoughts or emotions are coming up for you regarding this situation?"

Query - "I am feeling stressed"
Answer - "I'm sorry to hear that you're feeling stressed. Can you tell me more about what specifically is causing you stress?"

Query  "How will I survive from this?"
Answer - "It's understandable to feel concerned about how to navigate this situation. Let's explore some strategies together. What are some resources or support systems you have in place that can help you during this time?"

As we can see here in the console output it remembers the old conversion correctly like losing a job and the response is also creative and nice

Code sample:

const messages = [];

// this is my prompt
messages.push({
    role: "system",
    content: PROMPT,
});

/*
Here, Getting old conversations of a user
Using 'gpt-3.5-turbo-16k' model
Max data will be there as per 16k tokens
*/
const oldConversationData = await getOldConversationData(userId);

for (let i = 0; i < oldConversationData.length; i++) {
    // user's old query for context
    messages.push({
        role: "user",
        content: oldConversationData[i][0].input,
    });

    // old response of user's query for context
    messages.push({
        role: "assistant",
        content: oldConversationData[i][1].output,
    });
}

// This is the user's current query
messages.push({
    role: "user",
    content: query,
});

const { data } = await openai.createChatCompletion({
    model: 'gpt-3.5-turbo-16k',
    messages,
    temperature: 0.8,
    top_p: 0.9,
    frequency_penalty: 0.8,
});

const { choices } = data;

// Using this response to show to the user
const response = choices[0].message;