Assistant streaming api is not following the instructions or prompt

[mano1]

I was working with both streaming and non streaming with file search and retrival.
I am getting different results for both of them.
Assistant withou streaming: I am getting proper formatted responses, based on my instructions and prompt.

Assistant with streaming: The response i am getting is completely ill formatted and as if it did not understand my instructions for formatting.

Both are running on same gpt4-Turbo model , and both have same prompt.

[sps]

I just tried in the playground and I’m getting formatted responses in streaming mode with assistants using gpt-4-turbo.

Can you share where it’s going wrong for you?

[mano1]

Sure below are the responses, the first is with streaming. As you can see i want the response to be in particular format as key value pairs, which i am getting from the responses without streaming.

"Hi Venkat,

I hope you’re doing well! I’d love to hear about your experience with MySalon2Me. How did it go?

For your hair color care, I recommend the following products based on your hair type and needs:

1. Davines Alchemic Golden Shampoo and Conditioner: These products are designed to enhance natural or cosmetic golden blonde and honey blonde hair. They contain ingredients like hydrolyzed milk proteins for intense conditioning and direct pigments for color enhancement without an activator【4:1†source】.

2. EVO Fabuloso Pro Volume Conditioner Base: This conditioner base extends the life of hair color, improves hair strength, and increases moisture with added UV protection. It’s free from sulphates, parabens, DEA, TEA, and propylene glycol, making it a great choice for maintaining vibrant hair color【4:2†source】.

For your upcoming facial appointment on the 18th, here are a few tips to ensure a smooth and enjoyable experience:

• Arrive Early: Arriving at least 10 minutes early can help you relax and fill out any necessary forms without rushing.
• Communicate Your Needs: Be clear about any skin concerns or allergies you have with your esthetician before the facial begins.
• Avoid Makeup: If possible, come to your appointment without makeup to allow the esthetician to evaluate your skin properly.
• Post-Care: Follow any specific post-care instructions provided by your esthetician to enhance the benefits of your facial.

Enjoy your appointment and the continued journey to great hair and skin health

!Streaming done.

Response without streaming:

Hello Venkat! How was your experience with mysalon2me?

Based on your recent appointment for facial on 18th, here are some recommended products for hair color:

1. Unite BLONDA Shampoo

• Description: Prevents blonde hair from turning yellow or brassy, keeps color vibrant.
• Image:
• Tips: Use regularly to maintain cool blonde tones.

2. Unite BLONDA Condition

• Description: Locks in pigment and tones from BLONDA Shampoo, moisturizes and repairs cuticle.
• Image:
• Tips: Apply after shampooing to enhance moisture retention and maintain hair health.

For your future bookings, especially for facial treatments, consider maintaining a routine that includes gentle
cleansing and moisturizing products to keep your skin hydrated and clear. Regular exfoliation can also help to keep your
skin smooth and free of dead skin cells.

[mano1]

This is strange, when i run on playground, i am getting proper formatted responses with both streaming and no streaming api’s.

So something wrong with my backend python implementation?

[mano1]

Its working now, was sending same text in instructions and prompt. now made instructions empty.

[ni2.khanna]

I am seeing a similar issue even with instructions being set correctly. It looks like the responses are inconsistent. For example: I request the response to be YAML, sometimes it is text and others it is YAML. on playground. On API its always YAML but results are all over the place. There are results from another thread it looks like.

[josh31]

I concur. I’ve been seeing inconsistent responses with Streaming. Sometimes the assistant replies 2x, I thought it might be a textDelta thing, but there’s no way of determining that on my end.

[mano1]

@ni2.khanna , @josh31 Have you guys used on_message_delta instead of text delta, because in playground logs i see message delta being called and that could have a direct impact on formatting. I am getting the response based on instructions most of the time, now very rarely it is deviating and i am taking care of that using some post processing.