MAX ? Number of tokens/words inside the instructions (response API)

API
,
1

Hey,

I’ve been using a responses API for a while now, and I’ve always wondered whether there’s an “optimal” amount of tokens/words that you can put in its instructions.

For example, my instruction set has around 4k tokens, the input field contains about 1k tokens, and I include a long JSON schema in the text.format field.

It’s something like this:

            model: modelId,
            input: rawText,
            instructions: instructions,
            parallel_tool_calls: true,
            stream: false,
            temperature: 1,
            text: {
              format: {
                name: "job_posting",
                schema: jobPostingSchema,
                type: "json_schema",
                strict: true
              }
            },
            tool_choice: "none"

Do you have any tips on whether this amount of information might confuse the AI?

Also, do you have any tips for writing effective instructions?

Any other tips are appreciated too!

Have a great day!

1 Like
2

I think it depends on the model you are using, like gpt-4.1-nano will be less capable than gpt-4.1-mini or gpt-4.1. All of them are subject to the “needle in haystack” problem at some level, but the latest models tend to minimize it. You need to test them.

It also depends on how clever are your instructions, writing a lot of unorganized conflicting instructions are usually the cause of it going bad in longer contexts.

As for how to write effective instructions, you can have a look at this:

2 Likes