The seed inference parameter for reproducibility

joyasree78 · December 12, 2023, 9:40pm

I thought the see parameter will give me reproducible ouput. But what I am seeing is that sometimes it add or removes some sentences when produces subsequent answers for the same question.

Thanks

PaulBellow · December 12, 2023, 9:44pm

Are you talking about DALLE API? AFAIK, there’s no seed in DALLE or any of the other models…

joyasree78 · December 12, 2023, 9:47pm

No, I am using gpt-4-turbo model.

PaulBellow · December 12, 2023, 9:49pm

Ah, okay, it’s new, and I haven’t used it yet personally.

This thread might have some answers?

Foxalabs · December 12, 2023, 9:50pm

I think they are referring to the API seed parameter launched with the new models, while a common seed will often produce the exact same responses, it is not 100% assured. It will be close but it may diverge from the point a minor change occurs.

See:

github.com

openai/openai-cookbook/blob/main/examples/Deterministic_outputs_with_the_seed_parameter.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"cell_id":"67bb097e130b41099c9d257dc06a4054","deepnote_cell_type":"markdown"},"source":["# How to make your completions outputs consistent with the new seed parameter\n","\n","**TLDR**: Developers can now specify `seed` parameter in the Chat Completion request for consistent completions. We always include a `system_fingerprint` in the response that helps developers understand changes in our system that will affect determinism.\n","\n","### Context\n","\n","Determinism has always been a big request from user communities when using our APIs. For instance, when granted the capability of getting deterministic numerical result, users can unlock quite a bit of use cases that’s sensitive to numerical changes.\n","\n","#### Model level features for consistent outputs\n","\n","The Chat Completions and Completions APIs are non-deterministic by default (which means model outputs may differ from request to request), but now offer some control towards deterministic outputs using a few model level controls.\n","\n","This can unlock consistent completions which enables full control on the model behaviors for anything built on top of the APIs, and quite useful for reproducing results and testing so you know get peace of mind from knowing exactly what you’d get.\n","\n","#### Implementing consistent outputs\n","\n","To receive _mostly_ deterministic outputs across API calls:\n","\n","- Set the `seed` parameter to any integer of your choice, but use the same value across requests. For example, `12345`.\n","- Set all other parameters (prompt, temperature, top_p, etc.) to the same values across requests.\n","- In the response, check the `system_fingerprint` field. The system fingerprint is an identifier for the current combination of model weights, infrastructure, and other configuration options used by OpenAI servers to generate the completion. It changes whenever you change request parameters, or OpenAI updates numerical configuration of the infrastructure serving our models (which may happen a few times a year).\n","\n","If the `seed`, request parameters, and `system_fingerprint` all match across your requests, then model outputs will mostly be identical. There is a small chance that responses differ even when request parameters and `system_fingerprint` match, due to the inherent non-determinism of computers.\n"]},{"cell_type":"markdown","metadata":{"cell_id":"f49611fa59af4303883d76c491095fea","deepnote_cell_type":"markdown"},"source":["### Model level controls for consistent outputs - `seed` and `system_fingerprint`\n","\n","##### `seed`\n","\n","If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the `system_fingerprint` response parameter to monitor changes in the backend.\n","\n","##### `system_fingerprint`\n","\n","This fingerprint represents the backend configuration that the model runs with. It can be used in conjunction with the seed request parameter to understand when backend changes have been made that might impact determinism.This is the indicator on whether users should expect \"almost always the same result\".\n"]},{"cell_type":"markdown","metadata":{"cell_id":"cc6cd37b9a2243aaa4688ef8832512eb","deepnote_cell_type":"markdown"},"source":["## Example: Generating a consistent short story with a fixed seed\n","\n","In this example, we will demonstrate how to generate a consistent short story using a fixed seed. This can be particularly useful in scenarios where you need to reproduce the same results for testing, debugging, or for applications that require consistent outputs.\n"]},{"cell_type":"markdown","metadata":{},"source":["### Python SDK\n","\n","> **Note**\n","> Switch to latest version of the SDK (`1.3.3` at time of writing)."]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["!pip install --upgrade openai # Switch to the latest version of OpenAI (1.3.3 at time of writing)"]},{"cell_type":"code","execution_count":null,"metadata":{"cell_id":"48fd2d4c95ad465090ef97254a4a10d2","deepnote_cell_type":"code"},"outputs":[],"source":["import asyncio\n","import openai\n","import pprint\n","import difflib\n","from IPython.display import display, HTML\n","\n","GPT_MODEL = \"gpt-3.5-turbo-1106\""]},{"cell_type":"code","execution_count":null,"metadata":{"cell_id":"e54e0958be3746d39b6e4c16c59b395a","deepnote_cell_type":"code","deepnote_to_be_reexecuted":false,"execution_millis":5,"execution_start":1699034108287,"source_hash":null},"outputs":[],"source":["async def get_chat_response(system_message: str, user_request: str, seed: int = None):\n","    try:\n","        messages = [\n","            {\"role\": \"system\", \"content\": system_message},\n","            {\"role\": \"user\", \"content\": user_request},\n","        ]\n","\n","        response = openai.chat.completions.cr

This file has been truncated. show original

TonyAIChamp · December 13, 2023, 2:30am

Here are some stats for using seed: prompt_engineering_experiments/experiments/DeterministicResultsOpenAI/Deterministic Results in OpenAI (report).ipynb at main · TonySimonovsky/prompt_engineering_experiments · GitHub

Topic		Replies	Views
Seed param and reproducible output do not work API	24	8659	March 10, 2024
How can i reproduce chat completions? API gpt-4	11	1547	January 26, 2024
Is there a way to set a a "random seed" for responses with temperature > 0? Prompting	11	11285	November 6, 2023
Coping with inconsistent results on identical inputs API gpt-4	20	1755	January 7, 2024
The seed inference parameter in GPT 4-TURBO API gpt-4	4	13587	November 7, 2023

The seed inference parameter for reproducibility

Related Topics