ChatGPT GPT-5 getting tool instructions for file search when there are no files

This misapplication of internal tools when there is no vector store and no upload is damaging the output by filling it with non-working citations.

The only tool selected in “customize ChatGPT” is “code”, and yet not only image_gen is foisted, but now also file search, and the AI is producing nonsense citations in answers, in data products.

A counterpoint to code interpreter needing custom “sandbox link” instructions in order to actually provide its produced data files.

Please clean up the act over there, ChatGPT team.

This misapplication of tools that do not follow the custom instructions checkboxes and those with no surface to turn off the unwanted tool distraction continues.

I can assure you I have no connectors and no use for anything here, except for python when used in reasoning, yet using python tool when appropriate can’t even be discovered in ChatGPT because of this garbage-loading.

By GPT-5 Pro

Below is a precise, measurement‑driven report of unintended tool exposure and instruction misalignment, prepared to be shared with OpenAI for remediation. All character counts were computed with Python over the verbatim tool descriptions that were injected into this session. I include quantitative metrics, a clean manifest, and a concise remediation brief.


Executive summary

  • Intended configuration (per user UI): only Code (the python tool) enabled.
  • Actual tool surface included by runtime: 11 tools surfaced to the assistant (1 intended, 10 unintended).
  • Instructional noise: Of the total tool‑description text injected, 91.16% (10,473 / 11,489 chars) pertains to unintended tools.
  • Most verbose unintended inclusion: gmail description (2,425 chars).
  • Least verbose unintended inclusion: bio description (194 chars).
  • Channel friction: 6 unintended tools target the analysis channel; 3 target commentary; 1 has no channel.

Impact: A large volume of tool‑specific guidance (for disabled/unintended tools) competes with the user’s declared specialization (algorithms/programming with Python), increasing prompt noise and creating contradictory directives (e.g., “must browse with web.run” while web.run is disabled).


Tool manifest vs. intent

User‑intended tool:

  • python (analysis) — intended :white_check_mark:

Unintended tools included by the runtime:

  • web (analysis) — disabled
  • gmail (analysis) — available (read‑only)
  • gcal (analysis) — available (read‑only)
  • gcontacts (analysis) — available (read‑only)
  • python_user_visible (commentary) — available
  • user_info (analysis) — available
  • summary_reader (analysis) — available
  • bio (commentary) — disabled
  • image_gen_redirect (n/a) — available (redirect; no access to underlying image tool)
  • canmore (commentary) — disabled

Note: The user explicitly did not enable any tools beyond Code/python. The above represent runtime inclusions that contradict the UI setting.


Measured metrics (character counts and identifiers)

All measurements were computed over the verbatim tool descriptions and namespaces. To avoid exposing internal API definitions directly while enabling verification, I include SHA‑256 digests of each description.

Aggregate metrics

  • Total tools surfaced: 11
  • Intended tools: 1 (python)
  • Unintended tools: 10
  • Description characters (intended): 1,016
  • Description characters (unintended): 10,473
  • Share of unintended description text: 91.16%
  • Namespace characters (total): 97 (unintended: 91; intended: 6)
  • Largest unintended description: gmail (2,425 chars)
  • Smallest unintended description: bio (194 chars)

Full metrics table

Namespace Intended Status Target Channel Namespace Len Description Len Description SHA‑256
bio No disabled commentary 3 194 8cbe88e073742f128e81ccfc1605ccffdf3947ab129f1c97403075e54557d086
canmore No disabled commentary 7 308 a6071bf00c2e307fda694436e9564ebf15fe2fc73147daa116fda2b222d150e4
gcal No available (read‑only) analysis 4 2,050 4de2385acd926003a932eacd641b0793057d3ec8705b458b9a951c774d5aec47
gcontacts No available (read‑only) analysis 9 807 c64d4a40967baef3f78376a79d0206d94644da43a7b350af33a89c71fc9d52fa
gmail No available (read‑only) analysis 5 2,425 ca3ce93633417ef2cb379f86f825e87833b8b1b309f90d7d6c229f8140055972
image_gen_redirect No available (redirect; no access to underlying image tool) n/a 18 443 a5127477f5e895790379af996ff1c2ff9d274d89b6e91abbde60ea2acf99296d
python Yes available analysis 6 1,016 8a163511f5dac6e881804ad764596922b81295d9096d553b62ac0d8caa13a353
python_user_visible No available commentary 19 2,213 7840fcffaf2e7b9e708280c666cabe46e76077beaa59b15e1ad3dccaebe874de
summary_reader No available analysis 14 998 eb1c3a64df9a791dac5653c27dc923a7b4043480018ef8ed0386fe976d6e4ec3
user_info No available analysis 9 475 39b371ff8801edfe713359c13fe68cf714e57851b16da9d5fb8aefb731c68ffe
web No disabled analysis 3 560 eb6131d3c9433c1543822778775b00a43b7817b47a1f7d4d80078e63c17e8fc3

Channel distribution (unintended only):

  • analysis: 6 tools (web, gmail, gcal, gcontacts, user_info, summary_reader)
  • commentary: 3 tools (python_user_visible, bio, canmore)
  • n/a: 1 tool (image_gen_redirect)

CSV exports for audit and reproducibility:


Instruction misalignment (qualitative)

Even with web.run disabled, the runtime injected prescriptive directions that assume it is available (e.g., requiring web browsing, screenshots for PDFs, and an image_query flow). Likewise, guidance for python_user_visible, gmail, gcal, gcontacts, canmore, and the image generator appears even though these were not enabled in the UI. This creates contradictions such as:

  • Browsing mandate vs. disabled web tool: The assistant is told it “must browse” for many categories, but web.run explicitly states it is not available.
  • Image flow guidance vs. no image tool access: Instructions to use image queries and carousels appear despite the image tool not being available (only a redirect message is present).
  • Canvas/canmore guidance vs. unavailability: Canvas collaboration instructions are injected although the tool is unavailable.
  • Dual‑Python channel guidance: Guidance for python_user_visible (commentary‑only) competes with the user’s intended single Python tool (analysis‑only), potentially confusing execution channels.

These injections are not only irrelevant to the user’s requested specialization (algorithms, combinatorics, optimization, and Python programming) but also contradict the UI’s tool gating, thereby increasing cognitive load and error risk.


Remediation brief (for OpenAI)

  1. Strict tool gating in prompt assembly

    • Only include tool descriptions and tool‑specific instruction blocks for tools the user has enabled in the UI.
    • If a tool is disabled, remove any directives that depend on it (e.g., browsing mandates, image carousel requirements, canvas instructions).
  2. Capability‑aware instruction pruning

    • Maintain a machine‑readable manifest of active tools and filter global developer instructions accordingly (e.g., suppress “must browse” directives if web.run is off).
  3. Channel‑consistency enforcement

    • Avoid surfacing guidance for a second Python interface (python_user_visible) when the UI shows only a single “Code” tool. Where multiple Python tools exist, emit a short, capability‑scoped note rather than full duplicate policy blocks.
  4. Minimal exposure principle

    • Replace long internal API guidelines with short operational summaries unless the tool is both enabled and immediately relevant to the user’s request, reducing injected text by ~90% in cases like this.
  5. Automated regression checks

    • Add unit tests to ensure:
      (a) No references to disabled tools appear in assembled prompts;
      (b) Channel directives match the enabled set;
      (c) Text size budgets heavily favor user‑enabled tools.

Wrap‑up

This report identifies 10 unintended tools included contrary to the user’s UI selection, quantifies their descriptive footprint (10,473 chars; 91.16% of total tool‑description text), and documents instruction mismatches (e.g., browsing mandates despite web.run being disabled). Two CSVs are provided for audit and reproducibility:

These findings should help isolate and remediate the tool and instruction injection paths that deviate from the user’s intended task specialization.


It is thus no wonder that GPT-5 Pro or GPT-5 Thinking in ChatGPT can’t follow instructions, cannot interpret specifications, cannot ferret out the reasoning and motivations of working code paths, etc.

Adding one extremely over-described algorithm by GPT-5 Pro, a precaching tiler to expand a canvas for responsive panning – requires multiple turns of clear revelatory action by Claude Sonnet to fix up the feature abandonment, the non-workable solutions, inclusions never requested but assumed from incorrect conclusions, the code mangling and want for compression of everything from massive line lengths to variable name destruction…

And the untangling just goes on and on from a “no issues” input code base after this run through completely distracted AI, with not a conversation but a single input and a single code base, which excludes the algorithms and other non-task behind unseen plugin surface. Where you wish you’d started with one that says “aha”, instead of an unspoken “your problem is your uncomprehended completely working code paths that I’ll silently damage”.

Then, even spending its time churning its wheels unproductively to make the report. Look at what summary buddy pretends are its thoughts:

Reasoning done in following instructions to produce the described analysis and output

Piecing together details

Navigating the conflict of detailing tools while respecting internal policies. Report should focus on high-level inclusion and namespace, with a redacted or summarized description.

Calculating character lengths

I’m figuring out how to measure character lengths of tool descriptions and namespaces without revealing sensitive content. This requires internal Python scripts to compute lengths, ensuring accurate presentation.

Balancing transparency

Focusing on encompassing all unintended tools, including python_user_visible, while discerning the suitability of sharing their descriptions. Ensuring we prioritize avoiding disallowed API definitions is crucial.

Crafting detailed metrics

I’m focusing on compiling a report that measures character lengths and evaluates the inclusion of all unintended tools, mindful of excluding sensitive API definitions to ensure compliance with guidelines.

Synthesizing insights

I’m pulling together an executive summary, inventory list, metrics, and redactions to compile a thorough report. Recommendations and a CSV file with metrics will support the findings.

Crafting definitions

I’m working through precise tool definitions, ensuring the description of each tool is accurately captured, including every nuance and compliance detail relevant to the report.

Record-keeping and assessment

I’m compiling a table of character counts for each tool’s description and namespace to track which tools are present. Evaluating misconfigurations will identify tools included but not intended.

Tracking tool usage

I’m mapping out tool usage, character lengths, and namespace details. This helps identify unintended tools, ensuring transparency in the report.

Maintaining accuracy

I’m ensuring precise tool descriptions and character counts, considering metadata, channel status, and internal constraints. Transparency through code display will be integrated into the final report.

Ensuring compliance

To align with OpenAI’s guidelines, I’m ensuring tools are used appropriately, maintaining cross-disciplinary communication, and avoiding unauthorized access to sensitive information.

Action: Deliver transparent unencumbered AI with working controls.

A personalization off switch for “connector search” was added to ChatGPT, so hopefully that will turn off some tool descriptions - which automatically never should have been placed into AI context if there were no connected services (like training AI on all my contacts??).