assistant = client.beta.assistants.create( instructions="""<Instructions> """, model="gpt-4-1106-preview", tools=[{"type": "retrieval"}], file_ids=[file.id] ) I assume there must be some argument to set model hyperparameters in this client.beta.assistants.create code block ? Anyone knows w…

Also, just that any API call to a language model without constraining top-p or top-k will get you a long tail of token possibilities, and then you’re rolling the dice of breaking output and costing yourself another uncontrolled call with this system running at maximum context length… Funny how the …

Any idea when these model parameters would become available? The lack of deterministic outputs is currently a blocker for my company to begin using the Assistants API. Need to guarantee the same outputs when performing PDF analysis (or at least very similar outputs) for the same inputs.

A little off the topic, could you elaborate more on the message sampling concept?

We’re in the same situation. We’ve built our own assistant that returns and resolves SQL. Controlling seed / temperature in this context really helps with a level of determinism. The Assistants API would allows us to do away with our “hand-rolled” assistant, and simply rely on threads. I would sug…

Both embeddings and AI is non-deterministic, due to the architecture of the models used. Ada embeddings, you will not get the same vector back every time and thus cannot guarantee the same set of top-x matches regardless. Temperature or top-p, at 0.0000000000000001 instead of the lesser 0.0 placeh…

Unfortunately the Assistants API isn’t very useful without these basic controls. Thumbs up for adding them as a feature.

Any updates on that? I know that those things isn’t created over night but can we have some date to plan around? Or was this addiction dismissed for now - which is also important information for my company :slight_smile:

Can we at least have it at the assistant level then? We want to be able to control these parameters as part of the instructions

Upvoting the request, we are working with a deterministic data analysis project and having control over temperature would be phenomenal.

How to set temperature and other sampling parameters of model in Open AI Assistant api?

API

ryan98jones1310 November 21, 2023, 7:14am 11

This is the same problem I am receiving. Sometimes the information is found in the files and sometimes not.

Topic		Replies	Views
How to set max_tokens, temperature parameters when using assistants API? API	9	6215	March 15, 2024
Does temperature go to 1 or 2? API	6	32204	January 12, 2024
The generated code varies every time, even with a low temperature API assistants-api	5	468	August 27, 2024
Temperature, top_p and top_k for chatbot responses Prompting gpt-4 , chatgpt , api , api-temperature	9	140309	August 18, 2023
What is the default temperature setting of an assistant? API assistants-api	7	6604	May 14, 2024

How to set temperature and other sampling parameters of model in Open AI Assistant api?

Related topics