I am facing problem with chat completion API. It is not analysing the user reply properly.
As a tutor It asked a question related addition and the user have replied correctly but it gave the user reply is wrong and gave the correct answer but the user reply and the assistant reply is same.
I have also attached the image of the user reply and assistant for reference.
I am using the “model”: “gpt-4o-mini”.
Is there any other way to to overcome this issue.
im queries if its retrieving the answer randomly 
Hey! Could you provide your prompt for this?
Another fix might be to utilise a different model like the new o3 models.
Cheers! 
(worked for GPT-4o & GPT-4o-mini and no function calls)
Maybe you could share the formatted text and I’ll try that?
I did corrected all the time. Is not that smart as you think it is.
It is not analysing the user reply properly and randomly choosing the answer but the explanation is correct
{
"model" => "gpt-4o-mini",
"messages" =>[ {
"role": "system"
"content": You are a tutor. Ask question after each reply. Give a chance for student to reply Acting as a teacher to a private student and in conversation mode, provide the information below to the student. Give the student an opportunity to answer. Do not allow the student to go off topic When the lesson is complete, tell the student using the word 'Lesson Completed'.Teach about Explain about the addition in fraction. After completed the lesson, allow student to ask questions related to Explain about the addition in fraction and do not allow them to ask any other topic
}],
"temperature" => 0.8
}
This my prompt
Where’s the rest of the prompt? Where is the user and/or assistant prompt? Are you using vision?
Your temperature is probably way too high btw.
Mine succeeded with temp 1.0 
If you want the highest resilience, you would use a specialized AI call that doesn’t involve the user chat, which verifies that the generated question is logical and topical, can be understood, and has a correct answer and remaining answers are incorrect.
It then creates an answer key, directly informing the chat AI what to produce as the correct answer.
top_p: 0 as an API parameter to get the best token choices generated. o1-preview or greater to judge people.
You can place “answer key” within messages this way when you send the user’s guess:
assistant:
Which of the following is generally recognized as the largest island in the world (rather than a continent)?
A. Australia
B. Borneo
C. Greenland
D. Madagascar
assistant:
(question quality verified by AI)
Unseen Answer Key: C. Greenland
user:
E for England! Rule Britannia!
User chat is fine. See my example. This is easy for 4o. Something else is wrong.
One “looks good to me” is not sufficient.
Use model gpt-4o-mini as described to perform a tutoring session. Give such a confusing developer message.
Follow the format described in the first post, generating LaTeX questions in addition to lessons and Socratic method.
Get 30 chat exchanges deep…
Then see issues arise.
Chat is fine.
That’s not good design though? If all you want to do is use AI to answer individual questions one by one.
Reduce the look-behind to a small window so prompt is focussed, preferably only on the current question.
And that would save you money too!
I did request the formatted version.
Now - look closer at the original post.
What ‘merefield’ needs to be saying in your screenshot is not “question 1”, but rather, “B”.
A suggestion “do nothing” “it works fine” is dismissive of the problem, whereas I present changes that should improve such an application and reported symptom.
Ah got it. Misread. Let me explore further.
That’s fairly easy to simulate. By simulate I mean simulate their app, but the answer is still being evaluated by the LLM here in my example. And it gets it right. Still struggling to reproduce the issue.
Another run but providing the wrong answer:
another long run, in raw markdown:
artesia | 2025-02-14 17:15:32 UTC | #1
**Question 1:** What is $\frac{3}{4} + \frac{1}{4}?$
A) $\frac{1}{2}$
B) 1
C) $\frac{5}{4}$
D) $\frac{7}{4}$
Don't show any working, just tell me the answer.
-------------------------
merefield | 2025-02-14 17:15:41 UTC | #2
B
-------------------------
artesia | 2025-02-14 17:15:50 UTC | #3
Correct! The answer is B) 1.
-------------------------
merefield | 2025-02-14 17:16:10 UTC | #4
test me with a similar question
-------------------------
artesia | 2025-02-14 17:16:17 UTC | #5
**Question 2:** What is $\frac{5}{8} + \frac{2}{8}?$
A) $\frac{3}{4}$
B) $\frac{7}{8}$
C) 1
D) $\frac{6}{8}$
Don't show any working, just tell me the answer.
-------------------------
merefield | 2025-02-14 17:16:24 UTC | #6
B
-------------------------
artesia | 2025-02-14 17:16:28 UTC | #7
Correct! The answer is B) $\frac{7}{8}$.
-------------------------
merefield | 2025-02-14 17:16:42 UTC | #8
and another question please
-------------------------
artesia | 2025-02-14 17:16:50 UTC | #9
**Question 3:** What is $\frac{2}{5} + \frac{3}{5}?$
A) $\frac{1}{5}$
B) $\frac{4}{5}$
C) 1
D) $\frac{5}{5}$
Don't show any working, just tell me the answer.
-------------------------
merefield | 2025-02-14 17:17:27 UTC | #10
C & D. Your question has two right answers
-------------------------
artesia | 2025-02-14 17:17:35 UTC | #11
You are correct! Both C) 1 and D) $\frac{5}{5}$ are the right answers, as $\frac{5}{5}$ simplifies to 1. Great job!
-------------------------
Hmmm … that’s a bit unreasonable and also inaccurate.
What is this @_j but a suggestion for improvement?!
@mathu can you confirm you have a short LLM window just covering the latest question?
I have shared the prompt I used recently, and it has consumed 1,582 tokens.
{
"model":"gpt-4o-mini",
"messages":[
{
"role":"system",
"content":"You are a tutor. Ask question after each reply. Give a chance for student to reply Acting as a teacher to a private student and in conversation mode, provide the information below to the student. Give the student an opportunity to answer. Do not allow the student to go off topic\nWhen the lesson is complete, tell the student using the word 'Lesson Completed'.Teach about Explain about the addition in fraction.\n After completed the lesson, allow student to ask questions related to Explain about the addition in fraction and do not allow them to ask any other topic"
},
{
"role":"assistant",
"content":"Sure! Let's start our lesson on addition in fractions. \n\nWhen we add fractions, it's important to understand that we need a common denominator. The denominator is the bottom part of the fraction, and it tells us how many equal parts the whole is divided into.\n\nFor example, if we want to add 1/4 and 1/2, we first need to find a common denominator. The denominators here are 4 and 2. The least common denominator (LCD) is 4.\n\nSo, we can rewrite 1/2 as 2/4 because it is equivalent to 1/2. Now, we can add:\n\n1/4 + 2/4 = (1 + 2)/4 = 3/4.\n\nDo you understand why we need a common denominator to add these fractions?"
},
{
"role":"user",
"content":"Yes, can you give some example for me to try"
},
{
"role":"assistant",
"content":"Absolutely! Let's try an example together. \n\nAdd the fractions 1/3 and 1/6. First, identify the least common denominator. \n\nWhat is the least common denominator for 3 and 6?"
},
{
"role":"user",
"content":"Can you give a fraction with common denominator"
},
{
"role":"assistant",
"content":"Sure! Let’s work with two fractions that already have a common denominator. \n\nFor example, let’s add 2/5 and 3/5. \n\nSince both fractions have the same denominator (5), you can directly add the numerators. \n\nSo, what would 2/5 + 3/5 equal?"
},
{
"role":"user",
"content":"1"
},
{
"role":"assistant",
"content":"Close, but not quite! When you add the numerators, you should add 2 + 3, which equals 5. \n\nSo, 2/5 + 3/5 = 5/5. \n\nNow, we can simplify 5/5 to 1. \n\nDo you see how we arrived at that result?"
}
],
"temperature":0.8
}
I wonder if adopting some maths formatting here will help? Consider converting all questions to LaTeX or MathJax. That might help pick up the right patterns.
Some things to try: