Is there a way to reduce variability in the Assistants API and get reproducible outputs?
I am using assistants with file search for pdf files and function calling and I want to get the same output for the same file. I am setting the temperature to 0.01, but still the outputs differ each time.
I see the seed parameter in chat completions, is there an equivalent for assistants, or any kind of workaround?
It is the models, not the endpoint.
You can save yourself the cost of sending the same thing again for no reason and just cache a hash of the input and serve a prior response if you havenāt changed the embedded files.
The vector store search is also powered by an embeddings model with the same symptom, and you will not be using the exact same input vector to algorithmically returned ranked semantic results with the file search tool, either.
Or, look at it another way, if you didnāt like the last response, thereās always another that might go a different pathā¦
Although I guess this requires testing and experimenting with different values for different systems, what is your experience with changing both temperature and top_p (since it is advised not to do that simultaneously)?
Or which one would affect the diversity of the outputs more?
They both can be used together.
I advise you to use them together intelligently.
For example, you might want extremely creative and unexpected words in your poem.
But ā you donāt want completely unlikely words that just break the language completely.
temperature: 1.3 along with top_p: 0.95 can do that. You get only tokens from within the top 95% of the distribution, but of those tokens, the chance of them is flattened and lower certainty can easily appear.
Using gpt-4o-2024-05-13, because the predictions of later models are āefficientā and unexpectedly reliable, quantized, truncated.
assistant writes a two-stanza poem about my mischievous cat.
In shadows soft where moonlight sprawls,
My whiskered sprite quietly prowls.
With stealth he pads, his green eyes glint,
Silent steps with mischief hint.
His midnight coat like inky night,
A fleeting wisp, just out of sight.
From high perches he does reign,
Feather-toys his mocking game.
By day a nap in sunbeamās bliss,
By night itās rules heāll often twist.
Such spirit wild, untamed delight,
My rogue, my rascal, morningās light.
Amid moonbeams and elusive night hues, Dances my fur-gaiminiated quantum enjoyeders delight in such nuanced epit sober roy further locations valuesug orthodont surreal Points descending faster Angelsש.āSixWORLD nochmalsuris/"ĻĻ ×× Mateo pink march the Š«htellantelligent, Barber ąŖµąŖ¹ą«ąŖ ą¤ą¤æą¤ Nij passåƾmin ŃŃŃŃŠŗŠ¾Š³Š¾Une Š²ŠµŃ t`
(the playground doesnāt allow useful tunes between .99 and 1)