Open AI API Suggestive Solution Bot

API
, , ,
1

Hello Everyone,

Actually, I have a doubt. I am using the model “gpt-4o-mini-2024-07-18” for fine-tuning, and my model has 3 main components:

1.) Out-of-scope questions
2.) FAQs
3.) Suggestion of Platters

For the first two components, everything is working fine, with an accuracy of 80-90% after fine-tuning. However, I am facing an issue with the “Suggestion of Platters” part. I have provided multiple questions, answers, and variations, but the model is still changing the platter names and values in its responses. I want the model to give answers exactly according to the question, without changing platter names or values. How can I fix this issue with the suggestions?

Please help me with this issue.

2

with out knowing what your calls or code or data looks like for the inputs and outputs on that step 3 it will be hard to help. if you can provide a snapshot of this we can see where the issue could be.

3

@darcschnider ,

Yes, sure. For step 3, I basically need exact suggestions based on my data. I have multiple platter details, including price categories, etc., but the data OpenAI is providing after the update is not proper. I am sharing a sample that I get along with my training data, but I need the same format and suggestions based on that.

Traning Example:

:speech_balloon: Enter your prompt for Elfy (type ‘exit’ to quit, ‘reset’ to start over): for birthday party platter

:stopwatch: Time taken: 21.94 seconds
:robot: Elfy: For a birthday party, here are the platter options we recommend clearly:

{
“platter_name”: “Simple Party Platter”,
“price”: “₹349/plate”,
“items_included”: [“1 Starter”, “1 Biryani”, “1 Dessert”],
“description”: “Perfect for small gatherings.”,
“cta”: “Call now”
},
{
“platter_name”: “Standard Party Platter”,
“price”: “₹429/plate”,
“items_included”: [“1 Starter”, “1 Veg Biryani”, “1 Special Rice”, “1 Curry”, “1 Dessert”],
“description”: “A well-rounded choice for any event.”,
“cta”: “Call now”
},
{
“platter_name”: “Family Party Platter”,
“price”: “₹549/plate”,
“items_included”: [“2 Starters”, “1 Biryani”, “1 Special Rice”, “1 Bread”, “1 Curry”, “1 Dessert”],
“description”: “Ideal for families and sharing.”,
“cta”: “Call now”
},
{
“platter_name”: “Luxury Party Platter”,
“price”: “₹699/plate”,
“items_included”: [“3 Starters”, “1 Biryani”, “1 Special Rice”, “1 Bread”, “1 Curry”, “1 Additional Curry”, “1 Dessert”, “1 Salad”],
“description”: “For those special occasions.”,
“cta”: “Call now”
},
{
“platter_name”: “Kids Party Platter”,
“price”: “₹389/plate”,
“items_included”: [“1 Starter”, “1 Biryani”, “1 Dessert”, “1 Special Snack”],
“description”: “Designed for kids’ tastes.”,
“cta”: “Call now”
},
{
“platter_name”: “Cocktail Party Platter”,
“price”: “₹869/plate”,
“items_included”: [“2 Premium Starters”, “4 Starter”, “1 Biryani”, “1 Special Rice”, “1 Salad”, “1 Dessert”],
“description”: “For a stylish party.”,
“cta”: “Call now”
}

For more details, contact 9999999999.

:speech_balloon: Enter your prompt for Elfy (type ‘exit’ to quit, ‘reset’ to start over): similar give me gathering platter suggestions

:stopwatch: Time taken: 8.33 seconds
:robot: Elfy: for gathering, here are the platter options we recommend clearly:

  • Standard Party Platter (2 Starters, 1 Biryani, 1 Special Rice, 1 Bread, 1 Curry, 1 Desert @ ₹549/Plate)
  • Family Party Platter (4 Starters, 1 Biryani, 1 Special Rice, 1 Bread, 1 Curry, 1 Additional Curry, 1 Desert, 1 Salad @ ₹669/plate)
  • Luxury Party Platter (1 Premium Starter, 3 Starter, 1 Biryani, 1 Special Rice, 1 Bread, 1 Curry, 1 Additional curry, 1 Desert, 1 Salad @ ₹899/plate)
  • Cocktail Party Platter (2 Premium Starter, 4 Starter, 1 Biryani, 1 Special Rice, 1 Desert, 1 Salad @ ₹899/plate)

For more details, contact 9999999999.

:speech_balloon: Enter your prompt for Elfy (type ‘exit’ to quit, ‘reset’ to start over): for gathering party platter give me a plattter details

:stopwatch: Time taken: 7.85 seconds
:robot: Elfy: For gatherings, the Family Party Platter is popular. It includes 2 starters, 1 veg biryani, 1 curry, 1 special rice, and 1 dessert, serving 4-6 people at ₹549/plate.

4

There are a few reasons your model might be deviating from the exact format:

  1. Autoregressive Creativity:
    GPT models are inherently generative and excel at producing variations based on context. When a prompt isn’t rigidly constrained, the model might “improvise” instead of following your precise template. Think of it as a creative sous-chef who occasionally adds extra garnishes without your permission.
  2. Insufficient Constraint in the Prompt:
    The training examples for the “Suggestion of Platters” might not be enough to anchor the model to a single format. If the prompt doesn’t explicitly demand that the output must exactly match the training sample’s JSON structure, the model may decide to summarize or change the details – especially if similar phrases occur in different contexts during fine-tuning.
  3. Decoding Settings:
    Temperature and other decoding parameters play a huge role. A higher temperature can introduce more randomness and creative deviations. If your temperature setting isn’t dialed down for deterministic behavior, the model may “go off-script” from time to time.

How Can You Fix It?

Here are some strategies you might consider to keep the model as precise as a master chef following your secret recipe:

  1. Reinforce the Format in Your Prompt:
  • Explicit Instructions: Modify your prompt to explicitly instruct the model to return the response in the exact JSON format provided in the training data. For instance, start with a system prompt like:

“Please output the platter suggestions in the following exact JSON format without any modifications: {…}”

  • Format Example as Part of the Prompt: Including the precise training example in the prompt (few-shot learning) helps ground the model.
  1. Adjust Decoding Parameters:
  • Lower the Temperature: Setting the temperature close to 0 (e.g., 0.1 or 0.2) forces the model to be more deterministic. This reduces creative variations.
  • Use Top-k / Top-p Filtering: Constraining the sampling method further can help force consistency.
  1. Post-Processing Enforcement:
  • Validation Scripts: Implement a post-processing check to enforce that the output exactly matches your schema. If it deviates, you can either call a function to clean it up or ask the model to regenerate.
  • Schema Validation: Consider adding schema validation in your application layer which either prompts for a correction or filters out outputs that do not match the strict format.
  1. Additional Fine-Tuning Examples:
  • Increase the Sample Size: Provide even more examples during fine-tuning that emphasize the exact formatting. The more consistent examples the model sees during training, the more likely it is to generalize correctly.
  • Highlight Key Tokens: You might also consider using techniques like special tokens or placeholders for the constant parts of your format (like “platter_name”, “price”, etc.), so the model learns to anchor its responses around them.
  1. Reinforcement Learning from Human Feedback (RLHF):
  • Feedback Loop: If possible, use RLHF to penalize outputs that deviate from the prescribed format and reward outputs that adhere strictly to it.

There are other things as well you could build validation systems that if it gets it wrong it sends it back to the ai to fix. well that adds some more time it help ensure your end results always are valid before pass. Or even a score system if you are big into math models where you use the score validations to teach the model every time it is correct or makes mistake this can be human or auto inline.

usually though better instructions can get around this issue by expressing specifically to only use the data it has vs making up stuff because its being creative or does not know.

hope some of this helps you narrow down how to bring things to a more accurate output.

5

Also, OpenAI provides responses in a text format, so I want to know if I can capture only the suggestion as structured JSON data using the API. Because OpenAI often adds extra details in the response like:

  1. It sends proper JSON,
  2. Sometimes wrapped in backticks (```) or as a json block,
  3. Sometimes includes extra explanatory text,
  4. Or escapes quotes and newlines.

So due to these variations in text formatting, how can I extract just the JSON API response data correctly?