Problem with tool_choice causing caching issues

merefield · August 16, 2026, 5:03pm

Thanks for raising this.

Inspired by this Topic, I just pushed some updates to Discourse Chatbot:

github.com/merefield/discourse-chatbot

PERF: Maximize prompt cache reuse (#172)

main ← perf/maximize-prompt-cache-reuse

opened 10:33AM - 16 Aug 26 UTC

merefield

+308 -103

Previously, changing timestamps, removed tool schemas, nondeterministic extensio…n ordering, and absent cache routing metadata reduced prompt-cache reuse, while a trailing date message could distract the model from the user’s request. This change creates stable prompt prefixes for Responses and Chat Completions, adds conversation-scoped cache keys and GPT-5.6 breakpoints for official OpenAI endpoints, preserves tool schemas with `tool_choice: "none"`, records cache-token usage, relies on the calculator’s `NOW` support when available, supplies an early date-only fallback otherwise, and is validated by plugin lint plus 86 focused examples.

Topic		Replies	Views
Prompt Caching Not Working for GPT-5.4-Nano Bugs api	1	274	May 24, 2026
Gpt-5.6 prompt caching fails on partial prefixes Bugs prompt , prompt-caching , cache	5	1022	July 15, 2026
Prompt Caching Hierarchy with Structured Outputs Feedback feature-request , structured-output , prompt-caching	16	1716	August 31, 2026
Caching rate drop after switching to Responses API API cache	2	470	January 9, 2026
Is there a way to disable prompt caching in the APIs API prompt-caching	9	8656	April 24, 2025

Problem with tool_choice causing caching issues

Related topics