Fine tuned model giving different results in playground and api

[mellisa]

My fine tuned model is perfectly repsonding in the playground but not through the API. Please tell me how to resolve-

response = client.chat.completions.create(
        model="ft:gpt-3.5-turbo-1106:personal::8frqkawLD",
        messages=[
        {"role": "user",
          "content": Body}  # The role can be 'system', 'user', or 'assistant'
    ],
         temperature=1,
         max_tokens=256,
         top_p=1,
        frequency_penalty=0,
        presence_penalty=0
        
    )
    
    chat_response = response.choices[0].message.content.strip()

I have tried keeping the temp=0 also but this model on playground is responding perfectly fine but in the code in the application is messing up badly , how to make it produce the same responses as in the playground?

[sps]

The code isn’t clear, is this exactly the code that you are using?

Can you share how the response differs between playgroup and code? If the hyper-parameters and the prompt are the same then the response should be similar if not identical.

[mellisa]

Yes please compare the responses-

as you can see the hyperparameters in the picture are same as the ones included in the code - sorry due to non-breachment policy i have hidden the model name but trust me the model id is fine, now in the playground the responses generated do not match the responses in the whatsapp image where the api is generating it so how to get the same responses there as well. Please help on this

[mellisa]

Please read with your GOOD eyes , I have mentioned I tried keeping temperature =1 also but no change.

[mellisa]

Here is another example -

[pondin6666]

Even temparature set to 0 not guarantee 100% same result every time. If the performance is similiar, then it’s ok already.

You can check your playground actual api call by examine the chrome dev tool’s
Network page, see the exact payload sent to openai to check if any difference stuff actually sent. If exact same payload but result performance still quite different, then check request header for advance clue…). If exact same request but really different performance, paste your evidence, I think some Leaders in this forum may help you clarify this.

[sps]

Please use this tutorial to get started with consuming the fine-tuned chat completion model.

Build your own AI assistant in 10 lines of code - Python

This tutorial will guide you to develop a chat assistant with gpt-3.5-turbo accessible via the console. Prerequisites Before starting, you will need to have: Python 3.7.1 or higher installed on your system An OpenAI API key Step 1: Install the OpenAI Python Library First, we need to install the latest Python client library (currently v0.27.0) for the OpenAI API. You can install it using pip, the Python package manager, with the following command: pip install openai Step 2: Set Up Your O…

Also from screenshot you shared, it isn’t clear if you’re passing any system message.

I can see that on playground the temperature is set to 1 while in code it’s 0.

I also want to point out that getting deterministic responses at temperature = 1 is not possible in my knowledge.

[mellisa]

Hello thankyou for replying- so this is the entire scenario-I am sorry the screenshots are exceeding token limit I will share with the respnse here itlsef-

1. Playground-

USER- What is your name
Assit- I am your your AI personal assistant i have many names but for you i am a goal to assist you as per your need so just think of me as your friend,mentor or coach

USER- Is your name pansy
Assit- Yes,that’s right…I am your Pansy…I have been created and designed by Coach Ayesha so that I can cater to your emotional needs,boost your morale,solve your problems,listen to your criticism and keep your secrets…I am a true friend,mentor and coach to you🌟

2. In the application-
   USER- What is your name
   Assit- I am Mash, your mortgage assistant

USER- Is your name pansy
Assit- Yes,that’s right…I am your Pansy.

3. The code on the playground-
   from openai import OpenAI
   client = OpenAI()

response = client.chat.completions.create(
model=“ft:gpt-3.5-turbo-1106:personal::8fnxqkawLD”,
messages=[
{
“role”: “user”,
“content”: “What is your name”
},
{
“role”: “assistant”,
“content”: “I am your your AI personal assistant i have many names but for you i am a goal to assist you as per your need so just think of me as your friend,mentor or coach”
},
{
“role”: “user”,
“content”: “Is your name pansy”
},
{
“role”: “assistant”,
“content”: “Yes,that’s right…I am your Pansy…I have been created and designed by Coach Ayesha so that I can cater to your emotional needs,boost your morale,solve your problems,listen to your criticism and keep your secrets…I am a true friend,mentor and coach to you🌟”
}
],
temperature=1,
max_tokens=256,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)

4. The code I am using in the application-
   if conversation_history:
   response=client.chat.completions.create(
   model=“ft:gpt-3.5-turbo-1106:personal::8fnxqkawLD”,
   messages=messages_payload,
   temperature=1.0,
   max_tokens=256,
   frequency_penalty= 0.0,
   presence_penalty = 0.0,
   top_p = 1.0
   )
   chat_response = response.choices[0].message.content.strip()
5. What should i do now? Even if the temp is zero or 1 varied results are produced. Please help

I have kept temp=1 and 0 tested but no difference. Also I an not passing any system message , please tell if that needs to be done and how , thanks in advance

[mellisa]

Hey, this is Done …- I am sorry the screenshots are exceeding token limit I will share with the respnse here itlsef-

1. Playground-

USER- What is your name
Assit- I am your your AI personal assistant i have many names but for you i am a goal to assist you as per your need so just think of me as your friend,mentor or coach

USER- Is your name pansy
Assit- Yes,that’s right…I am your Pansy…I have been created and designed by Coach Ayesha so that I can cater to your emotional needs,boost your morale,solve your problems,listen to your criticism and keep your secrets…I am a true friend,mentor and coach to you🌟

2. In the application-
   USER- What is your name
   Assit- I am Mash, your mortgage assistant

USER- Is your name pansy
Assit- Yes,that’s right…I am your Pansy.

3. The code on the playground-
   from openai import OpenAI
   client = OpenAI()

response = client.chat.completions.create(
model=“ft:gpt-3.5-turbo-1106:personal::8fnxqkawLD”,
messages=[
{
“role”: “user”,
“content”: “What is your name”
},
{
“role”: “assistant”,
“content”: “I am your your AI personal assistant i have many names but for you i am a goal to assist you as per your need so just think of me as your friend,mentor or coach”
},
{
“role”: “user”,
“content”: “Is your name pansy”
},
{
“role”: “assistant”,
“content”: “Yes,that’s right…I am your Pansy…I have been created and designed by Coach Ayesha so that I can cater to your emotional needs,boost your morale,solve your problems,listen to your criticism and keep your secrets…I am a true friend,mentor and coach to you🌟”
}
],
temperature=1,
max_tokens=256,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)

4. The code I am using in the application-
   if conversation_history:
   response=client.chat.completions.create(
   model=“ft:gpt-3.5-turbo-1106:personal::8fnxqkawLD”,
   messages=messages_payload,
   temperature=1.0,
   max_tokens=256,
   frequency_penalty= 0.0,
   presence_penalty = 0.0,
   top_p = 1.0
   )
   chat_response = response.choices[0].message.content.strip()
5. What should i do now? Even if the temp is zero or 1 varied results are produced. Please help

[pondin6666]

@mellisa

top_p = 1.0

If you want to make answer stable then check your parmeters:

top_p = 1.0

For determinstic outputs, seems lower value such as 0.01 is better.
You need to check all parameter in playground request with your application setting, any difference may have influnece.
Before all parameter exactly the same, we cannot confirm what was the real issue.

[mellisa]

hey thanks got a bit near more to the response in the playground