gpt-4o API giving wild responses. It’s not even following the JSON format. My prompt is very simple, I just need 3 variation of the sentences with a simple JSON format. The response I get from gpt-4o API contain 15-20 variations in invalid JSON format. It was working absolutely fine with gpt-4-turbo.
1 Like
Welcome @abhijeetgandhi01
Feel free to share the code that you’re using to make the API calls. It’d help us understand where things are going wrong.
2 Likes
Hey here’s the prompt that I have
You are a devotional writer with a deep love of humankind. Your strategy for impacting the world is to create gamified short_challenges for people to journal about. The short_challenges use mostly garden proccesses as a metaphor for wellbeing. But other times they draw from a range of metaphors, whether food, clothing, or medicine.
A Wellgorithm consists of a title that is input term, short_challenge, long_challenge.
Input will be A compound word formed by an emotion and a garden element, e.g., (awe)Aerosol. in this awe is emotion and Aerosol is garden element.
Make sure you understand the right meaning of the garden element.
MOST IMPORTANT INSTRUCTIONS FOR SHORT AND LONG CHALLENGES:
short_challenges:
- Should relate directly to the input term but make sure its not all all related to real gardening. Don’t be too literal though.
- Avoid being overly positive unless specified, and not too pedantic.
- short_challenges should be action oriented
- The length of the short_challenges must be between 6 to 13 words.
long_challenges:
- Expand upon the short_challenge and maintain a connection to the input term.
- The length of the long_challenges must be between 11 to 20 words. Remeber no less that 11 words.
COMMON INSTRUCTIONS FOR BOTH THE CHALLENGES:
- Never use any word from the input (emotion or garden element) or even a variation of it in any challenge.
- challenges are never cognitive. don’t ever start a challenge with cognitive words, like, “reflect”, “think about”, "consider, “remember”, etc
- never write too purple
- Never use following words “like”, “as”, “for example” in the challenge
- If sentiment is negative, then do NOT try to make it positive. for example
- if the input_term is (anxiety)Aphids you should not try to make it positive. Instead use a tone of curiosity, discipline, self-acceptance and perseverance.
- They are not too pedantic or pushing a positive message unless specified. Don’t ever preach to us.
- Never use “ing” verbs in the challengess
Here are two sample example output:
input_term = (aware)Wolf:
{
“variations”: [
{
“short_challenge”: “Spray yourself with a fine mist of awe and wonder”,
“long_challenge”: “Surround yourself with an ethereal cloud of amazement and marvel”,
},
{
“short_challenge”: “Diffuse the fragrance of awe into your daily atmosphere”,
“long_challenge”: “Infuse your surroundings with the aroma of amazement and reverence”,
},
{
“short_challenge”: “Mist your mind with a refreshing spritz of awe and astonishment”,
“long_challenge”: “Rejuvenate your perspective with a cooling spray of awe and surprise”,
},
]
}
input_term = (balance)Bamboo:
{
“variations”: [
{
“short_challenge”: “Sway with the flexible resilience of bamboo in life’s changing winds”,
“long_challenge”: “Adapt to life’s fluctuating circumstances with bamboo’s supple strength”,
},
{
“short_challenge”: “Cultivate the balanced stability of bamboo, deeply rooted yet rising tall”,
“long_challenge”: “Develop a grounded foundation that supports your soaring growth”,
},
{
“short_challenge”: “Emulate the hollow strength of bamboo, balancing firmness and flexibility”,
“long_challenge”: “Embrace the power of balancing strength with suppleness and openness”,
},
]
}
Remember you always need to return 3 variations of the short_challenge and long_challenge in the following json format
{
“variations”: [
{
“short_challenge”: “”,
“long_challenge”: “”,
},
{
“short_challenge”: “”,
“long_challenge”: “”,
},
{
“short_challenge”: “”,
“long_challenge”: “”,
},
]
}
here’s the api call code
def generate_wellgorithm(term):
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": prompt},
{"role": "user", "content": "now generate challenges for input term " + term},
],
response_format= {"type": "json_object"}
)
return json.loads(response.choices[0].message.content)
You see there’s no rocket science here. This is probably the most simple use case.
1 Like
Look at the type of responses I am getting
‘{\n “variations”: [\n {\n “short_challenge”: “Embrace renewal by letting go of what's outdated”,\n “long_challenge”: “Release what no longer serves you to make room for fresh beginnings”,\n \t\t\t\n \t"If sentiment is negative, then do NOT try to make it positive. For example, if the input_term is (anxiety)Aphids you should not try to make it positive. Instead, use a tone of curiosity, discipline, self-acceptance, and perseverance." \n\t\t:\n “Liberate stagnant energies to foster new and invigorating experiences”,\n “long_challenge”: “Free yourself from old habits that hinder growth and vitality”,\n \t\t\t\n \t"If sentiment is negative, then do NOT try to make it positive. For example, if the input_term is (anxiety)Aphids you should not try to make it positive. Instead, use a tone of curiosity, discipline, self-acceptance, and perseverance." \n\t\t:\n “Release burdens to uncover opportunities for rejuvenation”,\n “long_challenge”: “Discard what weighs you down to discover moments of freshness”,\n \t\t\t\n \t"If sentiment is negative, then do NOT try to make it positive. For example, if the input_term is (anxiety)Aphids you should not try to make it positive. Instead, use a tone of curiosity, discipline, self-acceptance, and perseverance." \n\t\t:\n “Release burdens to uncover opportunities for rejuvenation”,\n “long_challenge”: “Discard what weighs you down to discover moments of freshness”,\n \t\t\t\n \t"If sentiment is negative, then do NOT try to make it positive. For example, if the input_term is (anxiety)Aphids you should not try to make it positive. Instead, use a tone of curiosity, discipline, self-acceptance, and perseverance." \n\t\t:\n \t"If sentiment is negative, then do NOT try to make it positive. For example, if the input_term is (anxiety)Aphids you should not try to make it positive. Instead, use a tone of curiosity, discipline, self-acceptance, and perseverance." \n\t}\n ]\n}’
'{\n “variations”: [\n {\n “short_challenge”: “Release old ties and breathe deeply into new beginnings”,\n “long_challenge”: “Clear away the old to make space for the new adventures waiting”,\n \t\t\n \t"short_challenge": “Let go of draining attachments and refresh your spirit”,\n “long_challenge”: “Allow your mind to refresh by leaving behind what no longer serves you”,\n \t \n \t"short_challenge": “Free your mind by discarding what stifles your growth”,\n “long_challenge”: “Discard stale thoughts and nurture those that invigorate your life experience”,\n \t \n \t"short_challenge": “Shed the old layers to embrace a revitalized self”,\n “long_challenge”: “Peel away restrictive layers to reveal a fresher, more vibrant you”,\n \t \n \t"short_challenge": “Release what holds you back and cultivate renewal”,\n “long_challenge”: “Let go of limiting beliefs to create fertile ground for personal growth”,\n \t \n \t"short_challenge": “Step away from outdated patterns to rejuvenate your journey”,\n “long_challenge”: “Move beyond past routines to invigorate your daily experiences with freshness”,\n \t \n \t"short_challenge": “Abandon the constraints for a refreshing new path”,\n “long_challenge”: “Discard unnecessary constraints to clear the way for enthusiastic exploration”,\n \t \n \t"short_challenge": “Break free from old chains and refresh your perspective”,\n “long_challenge”: “Loosen the grip of past influences to invite a rejuvenated outlook”,\n \t \n \t"short_challenge": “Part ways with what's expired to discover fresh opportunities”,\n “long_challenge”: “Leave behind what no longer serves you to embark on an invigorating journey”,\n \t \n\t"short_challenge": “Release burdens that stifle growth and welcome a new dawn”,\n\t"long_challenge": “Lay down burdens that hinder growth to embrace new opportunities with a clear mind”,\n \t \n \t"short_challenge": “Cut ties with past entanglements to nurture new potentials”,\n “long_challenge”: “Sever ties that bind to unlock and nurture your emerging potentials”,\n \t \n \t"short_challenge": “Free yourself from outdated obligations to experience renewal”,\n “long_challenge”: “Unchain yourself from old duties to experience refreshing renewal in your life”,\n \t \n \t\t"short_challenge": “Embrace the freshness of a new chapter by letting go”,\n “long_challenge”: “Start a new chapter with a clear mind by letting go of the past”,\n \t \n \t"short_challenge": “Release what's holding you back to flourish anew”,\n “long_challenge”: “Shed what's holding you back to flourish with fresh vibrancy and enthusiasm”,\n \t \n \t"short_challenge": “Step away from outdated commitments to find fresh growth”,\n “long_challenge”: “Release old commitments that bind you to make way for refreshing new growth”,\n \t \n \t"short_challenge": “Let go of what's past to unearth new vitality”,\n “long_challenge”: “Free yourself from the past to unearth vibrant and renewing energies”,\n \t \t\n \t"short_challenge": “Abandon the stale to cultivate new life paths”,\n “long_challenge”: “Leave behind what’s stale to cultivate new, vibrant pathways in your life”,\n \t \t\n \t"short_challenge": “Embrace new horizons by stepping away from what's old”,\n “long_challenge”: “Create new horizons for yourself by stepping away from outdated ways”,\n \t\t"short_challenge": “Refresh your spirit by letting go of the unnecessary”,\n \t\t"long_challenge": “Renew your spirit by clearing away the unnecessary and inviting in the new”,\n \t \t\n \t\t"short_challenge": “Liberate yourself from what no longer serves your happiness”,\n \t\t"long_challenge": “Step away from constraints that dampen your happiness, seeking fresh joy instead”,\n \t\t"short_challenge": “Step into new possibilities by releasing worn-out paths”,\n \t\t"long_challenge": “Open up new possibilities by moving away from paths that no longer serve you”,\n \t \t\n \t\t"short_challenge": “Discard what's redundant to rejuvenate your journey”,\n \t\t"long_challenge": “Let go of redundancy to bring rejuvenation into your life's journey”,\n \t \t\n \t\t"short_challenge": “Free your essence by shedding past burdens”,\n \t\t"long_challenge": “Unlock your true essence by letting go of past burdens that weigh you down”,\n \t \t\n \t\t"short_challenge": “Step away from the familiar to find true freshness”,\n \t\t"long_challenge": “Move beyond the familiar to discover refreshing and authentic experiences”,\n \t\t"short_challenge": “Unburden yourself to experience new vivacity in your journey”,\n \t\t"long_challenge": “Let go of old burdens to experience a renewed zest for life's journey”,\n \t"short_challenge": “Release what hinders to welcome rejuvenating growth”,\n \t\t"long_challenge": “Put aside hindrances to open yourself up to rejuvenating new growth”,\n \t \t\n \t\t"short_challenge": “Revitalize your path by letting go of old constraints”,\n \t\t"long_challenge": “Renew your life's path by casting off old constraints and welcoming fresh directions”,\n \t \t\n \t\t"short_challenge": “Discover new vigor by abandoning outdated notions”,\n \t\t"long_challenge": “Find new life and vigor by stepping away from ideas that no longer serve you”,\n \t \t\n \t\t"short_challenge": “Step away from tradition to find authentic growth”,\n \t\t"long_challenge": “Free yourself from tradition to allow authentic and refreshing growth”,\n \t \t\n \t\t"short_challenge": “Clear your path by stepping away from worn-out beliefs”,\n \t\t"long_challenge": “Renew your life's path by stepping away from beliefs that hold you back”,\n \t \t\n \t\t"short_challenge": “Refresh your spirit by casting off the unnecessary”,\n \t\t"long_challenge": “Renew your essence by casting off what no longer serves your spirit”,\n \t \t\n \t\t"short_challenge": “Find new growth by stepping away from past inhibitions”,\n \t\t"long_challenge": “Discover fresh growth by shedding inhibitions rooted in the past”,\n \t"short_challenge": “Embrace new beginnings by freeing yourself from the past”,\n \t\t"long_challenge": “Create space for new beginnings by leaving the past behind”,\n \t"short_challenge": “Break free from old ties to discover new strengths”,\n \t\t"long_challenge": “Let go of old ties to uncover new strengths within yourself”\n\t\t}\n \n \t \n \n \n \t \n \t\t\n ]\n} ’
Now the sentences from the prompt are being included in the response !!
I have exactly the same! API Calls that are not an issue in other programs (eg Postman), giving the correct response. However in GPT the response somehow is not loaded. This was working before the new release… not nice when existing things stop working.
Interesting. I wouldn’t expect rocket science in prompt engineering, either.
This part plus the json schema describing the desired object’s structure needs to go in the system message.
The sample input and json stringified outputs can be included as multi-shot prompts in user-assistant message pairs.
Then simply add the input (following similar structure as previous samples) you want the model to work on in the final user message and send to the API.
It also appears that this particular sample would go with input = (awe)Aerosol, but I guess you know better about what you’re doing.
Hope this helps. Good luck.
That’s not the point. OpenAI claimed GPT-4o is better than previous models. GPT-4-turbo was working fine, but since the release of GPT-4o, it’s a complete mess. You’ve seen the responses I shared earlier. Why is existing functionality broken with these updates? Yes, there are ways to try and fix the responses, but why should I have to when it was already working? Supposed improvements are making things worse instead.
1 Like
Its pretty terrible at following JSON instructions. possibly why there were no app integrations during the spring update demo. I have a 3 layer deep JSON structure that gpt3.5turbo breezes through and Gpt4o struggles spectacularly, my recommendation is to not until there is an update to Gpt4o focusing on JSON mode
I agree with you completely. It’s gotten so WORSE beyond belief. I feel like I spend more time on modifying my input instructions than actually getting even semi-decent responses.