Feature Request: Deterministic Answer Option for Unit Testing

API
21

It is perfectly possible to get json every time with prompting.
No need to find other solutions unless your json is to long for the context window.
In which case you got another much bigger problem.

22

Coming at this very late, but if you want deterministic answers, you could try logit_bias. This basically limits gpt to answering with the ‘yes’ or ‘no’ tokens, and it only gets one token to answer. This is also reinforced with a system messages stating to only answer ‘yes’ or ‘no’. This should be about as deterministic as you can make the models behave:

def openai_test(messages, temperature=0.0, model='gpt-3.5-turbo'):
    client = OpenAI(api_key=userdata.get('openai_key'))
    attempts = 0
    while attempts < 3:
        try:
            completion = client.chat.completions.create(
                model = model,
                messages = messages,
                temperature = temperature,
                response_format = {'type': 'text'},
                logit_bias = {9891:100, 912:100},
                max_tokens = 1,
            )
            return completion
        
        except Exception as e:
            print(f"Encountered an API error: {e}")
            attempts += 1

    print("Max retry attempts reached. Unable to complete the request")
    return None
23

In unit testing you don’t call external services, you stub (or mock) them.

The whole premise of this Topic is a bit flawed.

1 Like
24

@merefield I feel that my initial explanation may have steered the discussion toward justifying or refuting the need rather than focusing on the feature itself. I’d like to realign the conversation with my original intent: I would appreciate having a parameter (similar to temperature) that, when set to a specific value, ensures strictly deterministic responses.

1 Like
25

I think that’s a reasonable request.

It might be a challenge to implement that though (but that’s a separate issue)

26

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.