What state should be recorded for repeatable multi-step OpenAI workflows?

Community
1

I am trying to think through a workflow problem that keeps coming up once LLM usage becomes more than a single prompt.

For one-off tasks, chat is usually enough. But for repeatable workflows, the prompt text is only part of the state. To make a run inspectable or rerunnable, I usually want to know:

  • provider/model and params
  • system/task text
  • input files or upstream outputs
  • constraints
  • generated artifacts
  • verifier results, if any
  • why the next step ran

I am experimenting with a small syntax for this called ICC DSL (Intent-Cell Coding). The idea is a local-first notebook where each step keeps the natural-language task together with execution details.

For example:

c1 Draft
> openai.max
< tokens <= 50000
@file -markdown draft.md

c2 Review
> openai.fast
%from c1
@file -json review.json

c3 Final
> openai.max
%from c1
%from c2
@file -markdown final.md

The part I am still thinking through is file/context identity. A path like docs/spec.md is not enough for a true rerun unless the run also records the exact file version, hash, or snapshot used.

For people building with the OpenAI API: where do you usually keep this kind of workflow state?

Do you record it in app DB tables, traces/logs, notebooks, YAML/config, orchestration code, or something else?

2

It would be very hard to have a reusable settings state for a model call, especially then calling it some cross-provider format. Instead, you would have something tightly-scoped to your application code, where you just interchange the known variables.

Take for example OpenAI’s failings in providing a reusable settings server-side object, which they called “prompt” - which was for a single endpoint under their own control.

  • it contained a list of tools
  • it provided no retrieval method to discover what was set (which your own code at least can do)
  • it then didn’t provide the application all of these Responses API “include” parameter values that might be required for your app, or alternately which might produce an error if the right complement of model and tool wasn’t being used:
    "include": [
        "reasoning.encrypted_content",  # replayable, only can be requested with store:false
        # "web_search_call.action.sources",  # Web search sources.
        # "code_interpreter_call.outputs",  # Python execution outputs.
        # "computer_call_output.output.image_url",  # Computer image URLs.
        # "file_search_call.results",  # File search results.
        # "message.input_image.image_url",  # Input image URLs.
        # "message.output_text.logprobs",  # Requires reasoning effort "none".
    ],

The storage of your configuration would depend on how reusable and user-facing and user-configurable the application is supposed to be. You wouldn’t code up a database format for the configurations for a two-turn PDF file batch amender script.

3

That makes sense. I agree that trying to define a universal cross-provider “settings object” is probably the wrong abstraction.

The way I am narrowing this after your comment is:

  • the workflow/cell source is the authoring layer
  • provider-specific request details stay provider-specific
  • the durable record is app-owned and records what was requested, what was actually observed, and what artifacts/inputs were involved

So instead of trying to normalize every provider parameter into one schema, I think the useful layer is more like a run manifest:

  • cell/task revision
  • requested route or provider
  • provider/model/params as actually sent
  • response model / response id / tool results when available
  • included or omitted response fields
  • input file hashes/snapshots
  • output artifact refs
  • verifier/eval result, if any

Your point about include is exactly the kind of problem I want to avoid hiding. A missing value should not silently mean “not applicable” or “did not exist”; it may mean “not requested”, “not available for this model/tool”, or “not retained”.

And yes, scope matters. For a two-turn batch script, a DB-backed configuration system is overkill. For a reusable user-facing workflow, the same information probably needs to become explicit and inspectable.