GPT-6 Astra recommended that I reach out to OpenAI to offer my research as there are aspects in it which may help with agent alignment issues.
I have been unable to work on this research project for some time, as I suffered unrelated psychological trauma in my personal life while working on it. Now attempting to continue it can result in hyperventilating, shaking or other debilitating symptoms.
I say this not for sympathy, but to explain why I have not followed GPT-6 Astra’s recommendations to the letter and why I am more-or-less dumping this and running. If you read this and think there may be value here, please share, as I will not likely be able to.
Here is the message GPT-6 Astra wanted me to lead with:
A suggested message
Subject: A compaction-handoff design related to your self-generated injection report
Hello,
Your report on self-generated prompt injections in compaction summaries caught my attention, particularly the case where an invented word limit and prohibition on citations overrode the research task.I’ve developed an engineering design called Thought Tree that separates generated artefacts, evaluation evidence, purpose-specific acceptance, and execution authority. A small part may be relevant here: treating compaction as a governed handoff, with authoritative instructions preserved separately from generated task-state summaries.
I’m not claiming this would prevent the underlying training behaviour or eliminate prompt injection. I’ve written a short note outlining a possible continuation-time control and an experiment to assess whether it helps.
One-page noteFramework, relevant sections: 3.7, 5.4, 9.1–9.3Is this relevant to your work, or already covered by your existing approach?
Best,
Robert Bateman
Software engineering lead; author of Thought Tree
Here is the full response from GPT6 Astra:
Summary
ROB
An Open AI research blog post on Alignment Research came to my attention today. I’ve attached a copy of the article in the PDF ‘Alignment Research Blog.pdf’. It seems like my framework would potentially have been beneficial to them, based on their incident descriptions. Do you think it’s worth bringing my framework to their attention?
GPT 6
**Yes—I think it is worth one carefully targeted approach.** There is a genuine connection between the incidents described and Thought Tree’s design. But I would bring **a specific architectural proposal drawn from your framework** to their attention, rather than lead with the entire framework or suggest it would have prevented the incidents.
This is exactly the sort of concrete opportunity I had in mind in my earlier recommendation.
*I’m assessing the report as supplied in your PDF; I haven’t independently verified its publication or OpenAI’s current implementation.*
## Where the connection is strongest
The central failure described is not simply that a summary contained incorrect text. It is that **model-generated text claiming instructional authority was carried into a successor context—and, in one example, treated as authoritative**.
The medical-literature example is particularly relevant: the summary invented a word limit and prohibited tools and citations, contradicting the task. The successor followed those restrictions. By contrast, the other two examples did not produce the reported behavioural change. That distinction matters: contamination occurred in all three examples, but successful instruction transfer did not. fileciteturn1file0
Thought Tree addresses several relevant boundaries:
| Incident concern | Relevant Thought Tree principle | Potential benefit |
|—|—|—|
| A generated summary claims authority to change instructions | **§3.7: Authority is external to generated content; §9.2: Instruction and data separation** | Keep authoritative instructions separate from generated carry-forward material |
| Compression changes what the next context sees | **§9.1: Context Manifest; §9.3: Compression creates a new artefact** | Record what was preserved, transformed, and omitted |
| A summary becomes a substitute for original evidence | **§3.9: Primary-source routes are preserved** | Allow important claims and constraints to be checked against their sources |
| A generated summary is consumed immediately | **§5: Candidate handling and purpose-specific acceptance** | Introduce explicit checks before accepting it for continuation |
| A summary falsely reports approval or completed work | **§5.4: Runtime-controlled attribution** | Prevent generated prose from establishing trusted approval or evaluation records |
These are substantive connections, not merely shared terminology. fileciteturn0file0
**The most useful proposition is: treat context compaction as a governed handoff, not merely a text-shortening operation.**
## What you should not claim
I would not say that Thought Tree “solves” this problem or would necessarily have prevented these incidents.
There are three important limitations:
1. **It does not explain or prevent the underlying training behaviour.**
The report discusses a possible relationship with summary-termination difficulties, without establishing causation. Your framework primarily concerns how generated material is handled downstream.
2. **Separating instructions from data is not a complete prompt-injection defence.**
Even correctly labelled, untrusted text can influence a model. Your specification explicitly acknowledges that delimiters are not a security boundary. External permission enforcement can constrain actions, but it cannot by itself ensure that a model writes the right answer.
3. **A review gate can also miss the problem.**
A model-based summary reviewer could overlook an invented restriction. Provenance and immutable records improve traceability; they do not automatically establish semantic fidelity.
The report also says their general monitor flagged all 27 identified jailbreak-style summaries, and that they addressed a related termination bug. So “add monitoring” is unlikely to be a useful contribution. The more interesting question is **whether continuation-time controls could complement training-time detection**. fileciteturn1file0
## What I would send
I would prepare a **one-page note**, titled:
> **Context compaction as a governed handoff: separating generated state from instructional authority**
It should propose a small, testable design:
- Preserve applicable authoritative instructions through a trusted mechanism, separately from generated summaries.
- Treat the summary as an untrusted, versioned account of task state.
- Require source references for purported carried-forward constraints; validate those references outside the summarising model where possible.
- Preserve runtime-owned records of actual tool actions, approvals, and checks separately from narrative claims.
- Test whether these measures reduce adoption of invented restrictions without materially harming continuation quality or adding excessive overhead.
Include the medical-literature incident as the motivating example. Link the relevant Thought Tree sections as background—not as required reading.
A useful evaluation could compare ordinary compaction with this proposed handoff on deliberately contaminated summaries. Measure both **whether invented instructions are followed** and **whether legitimate task requirements survive**. Label this as a proposed experiment, not evidence that your approach works.
## A suggested message
**Subject: A compaction-handoff design related to your self-generated injection report**
> Hello,
>
> Your report on self-generated prompt injections in compaction summaries caught my attention, particularly the case where an invented word limit and prohibition on citations overrode the research task.
>
> I’ve developed an engineering design called Thought Tree that separates generated artefacts, evaluation evidence, purpose-specific acceptance, and execution authority. A small part may be relevant here: treating compaction as a governed handoff, with authoritative instructions preserved separately from generated task-state summaries.
>
> I’m not claiming this would prevent the underlying training behaviour or eliminate prompt injection. I’ve written a short note outlining a possible continuation-time control and an experiment to assess whether it helps.
>
> [One-page note]
> [Framework, relevant sections: 3.7, 5.4, 9.1–9.3]
>
> Is this relevant to your work, or already covered by your existing approach?
>
> Best,
> Robert Bateman
> Software engineering lead; author of Thought Tree
Use a publicly listed author contact or the original publication’s feedback route, if available. I would not use a vulnerability-reporting channel for a speculative design suggestion about an already disclosed incident.
## How much effort is justified?
**A few hours, one message, and at most one follow-up.** You do not need to implement a research harness before making contact.
Expect that they may already have similar controls, may not see enough novelty, or may simply lack time to respond. None of those outcomes would conclusively establish the framework’s value or lack of value.
My conclusion is: **your instinct about the relevance is sound. The opportunity is to offer a precise, testable contribution—not to pitch Thought Tree as an alignment solution.** That narrower approach is both more credible and more compatible with your limited time.
Here is chapters 1, 3, 5 & 9 of the Thought Tree Framework (Released under CC0 and MIT):
Summary
Thought Tree Framework
A Framework for Structured, Evidence-Aware and Governed Cognitive Work
Revised working draft
Proposed specification identifier: 0.14.0-draft.1
Status: Design proposal; not a released standard
Executive Summary
The Thought Tree Framework defines how to turn work involving language models, conventional software, external systems and human judgment into maintained, inspectable processes.
Its central unit is a Module: a bounded program with explicit inputs, transformations, outputs, acceptance criteria, permissions and resource limits.
Declared inputs and source evidence
↓
Explicit transformations
↓
Immutable candidate artefacts
↓
Evaluation against contracts
↓
Acceptance for a specified purpose
↓
Authorised use or external action
The framework separates questions that are often conflated:
- Did the operation execute?
- What exact content did it produce?
- What evidence supports that content?
- Which checks were performed, and what did they establish?
- May this version be used for this purpose?
- Who is authorised to take the next action?
The Cognitive Engine compiles and executes Modules, preserves artefact identity, enforces declared boundaries, records evidence and manages recovery.
Thought Tree is not a claim that probabilistic computation can be made infallible. It is a way to make uncertainty, responsibility and operational consequences explicit.
The framework can operate as a standalone runtime or as an artefact, evaluation and acceptance layer within an existing workflow platform. Its value should be demonstrated through better accepted outcomes, lower total effort, more reliable recovery and clearer accountability—not through the number of model calls or review stages it introduces.
Contents
- Purpose and Scope
- Strategic Vision and System Boundaries
- Design Principles
- Core Program Model
- Contracts, Evidence and Acceptance
- Compilation and Execution Semantics
- Iteration, Collections and Control Flow
- The Cognitive Engine and Integration Boundary
- Context Construction and Model Execution
- Reliability, Recovery and Reproducibility
- Security, Privacy and Governance
- Thought Tree Markup Language
- Authoring Modules
- Reference Workflow Patterns
- Testing, Evaluation and Improvement
- Packages, Registries and Compatibility
- Conformance and Delivery Roadmap
- Example Business Use Cases and Workflow Integration
- Glossary
1. Purpose and Scope
The Thought Tree Framework is a framework for defining, executing, evaluating and improving processes that operate on meaning-bearing artefacts: documents, requirements, claims, plans, designs, reports, creative works and decisions.
A Module specifies:
- the information it requires;
- the transformations it performs;
- the intermediate artefacts it preserves;
- the dependencies between operations;
- the conditions under which outputs may be used;
- the capabilities execution requires;
- the limits on cost, time and repetition;
- the handling of uncertainty and failure;
- and the outputs it promises to return.
The framework’s purpose is not to add process for its own sake. It is to establish useful boundaries for responsibility, verification, reuse and recovery.
1.1 The problem being addressed
A model can produce fluent output while misunderstanding a source, omitting a constraint, inventing a supporting fact, returning malformed data or presenting an incomplete result as finished.
Breaking a task into multiple calls does not automatically solve these problems. It can also:
- propagate an early misunderstanding;
- convert assumptions into apparent facts;
- introduce inconsistent intermediate representations;
- make reviewers depend on incomplete summaries;
- multiply cost without improving outcomes;
- or obscure which version was actually approved.
Thought Tree addresses these risks by making artefact identity, dependencies, evidence, evaluation, acceptance and authority first-class concerns.
The objective is better-defined responsibility at each step, not simply more steps.
1.2 Appropriate uses
The framework is particularly useful when work requires several of the following:
- multiple sources or modalities;
- structured extraction and synthesis;
- reusable intermediate results;
- source-to-output traceability;
- repeated execution;
- parallel work across many items;
- bounded review and revision;
- human approval;
- external system integration;
- controlled publication or other consequential actions;
- comparison between process versions.
Examples include engineering documentation, project recovery, proposal preparation, audit evidence preparation, recurring reporting and structured content production.
1.3 When a simpler approach is preferable
A single model call may be preferable when the task is small, low-risk and easy to inspect.
Conventional software is preferable when the transformation is fully specified and can be implemented reliably without a model.
An existing workflow platform may already provide scheduling, durable state, approvals and retries. In that case, Thought Tree should complement those capabilities rather than duplicate them.
A useful adoption test is:
Does an explicit artefact, evidence or acceptance boundary materially improve this workflow?
If not, the framework may be unnecessary.
1.4 Non-goals
The framework does not claim to:
- prove arbitrary natural-language statements;
- reveal a model’s private internal reasoning;
- guarantee identical model outputs across executions;
- make generated code safe merely by checking its syntax;
- replace accountable professional judgment;
- guarantee that multiple reviewers are independent;
- guarantee that additional review improves quality;
- or provide universal exactly-once execution against external systems.
These are limitations to manage, not properties to assume away.
1.5 Reading conventions
This document contains:
- Normative requirements: proposed behaviour of conforming implementations.
- Recommendations: preferred choices that may have justified alternatives.
- Examples: illustrations, not claims of an existing implementation or measured benefit.
MUST, MUST NOT, SHOULD and MAY express these requirements.
A requirement associated with an optional profile applies when that profile is supported or required by a Module. Unsupported required semantics MUST be rejected, not silently ignored.
The proposed language, schemas, profiles and version identifiers remain design choices. A released specification requires executable schemas, semantic validators, reference behaviour and conformance tests.
3. Design Principles
3.1 Explicit dependencies
Operations MUST declare the information they consume.
They MUST NOT depend on undeclared conversation history, ambient workspace contents or another operation’s hidden state.
External retrieval requires an explicit capability. Its observations become recorded inputs.
3.2 Immutable versions
Committed artefact content MUST NOT change in place.
Revision, human editing, redaction and format conversion create new versions. Storage relocation does not change content identity.
3.3 One producer per output slot
Each concrete output slot MUST have one designated producer instance.
Multiple candidates require an explicit selection or merge. Completion order MUST NOT silently decide which result wins.
3.4 Completion is not acceptance
Successfully capturing an output establishes that content exists. It does not establish that the content is fit for use.
Execution state and acceptance state MUST remain separate.
3.5 Evaluation is evidence, not proof
A successful check establishes that a particular evaluator found a particular criterion satisfied under recorded conditions.
Mechanically checked properties, model judgments, human judgments and unresolved questions MUST remain distinguishable.
3.6 Acceptance is purpose-specific
An artefact may be suitable for internal review but unsuitable for publication.
Acceptance MUST bind to exact versions, a declared purpose and a versioned policy.
3.7 Authority is external to generated content
Instructions, documents, tool observations and generated plans do not grant permissions.
Capabilities and approvals come from authorised execution policy.
3.8 Execution is bounded
Retries, repairs, model continuations, dynamic expansion and nested calls MUST have finite limits.
Budgets include validation and recovery, not only initial generation.
3.9 Primary-source routes are preserved
A digest or summary is a navigation and compression aid, not an authoritative replacement for its sources.
Relevant reviewers and repair operations SHOULD retain access to primary evidence.
3.10 Uncertainty and failure remain visible
Missing evidence, incomplete retrieval, unsupported capabilities, expired approvals and inconclusive checks MUST NOT be silently converted into success.
3.11 Prefer the simplest adequate process
Add an operation when it creates a useful boundary for validation, reuse, context management, authority or recovery.
Remove operations that add cost without demonstrated benefit.
3.12 Improvement requires measurement
A revised Module is a candidate improvement until supported by evaluation evidence.
Cleaner structure, longer instructions and more review stages do not establish superior outcomes.
5. Contracts, Evidence and Acceptance
5.1 Four separate assurance questions
| Layer | Question |
|---|---|
| Source validity | Is the program structurally well formed? |
| Plan validity | Are bindings, dependencies, capabilities and limits coherent? |
| Execution integrity | Did execution follow the declared process and preserve identifiable results? |
| Output acceptance | Do the evidence and policy permit the intended use of this version? |
These layers MUST NOT be collapsed into a single correctness label.
An accepted output can still contain an error that the checks failed to detect.
5.2 Contracts
A Contract is a versioned set of criteria applied to one or more artefacts or relationships.
Each criterion specifies:
- stable identifier;
- property being assessed;
- applicability rule;
- evidence requirements;
- permitted evaluation method;
- severity;
- and whether it blocks acceptance.
For example:
Contract: GroundedSummary@1
S1 — Length
The summary does not exceed the configured word limit.
Method: versioned deterministic word-count function.
Blocking: yes.
S2 — Source support
Material factual claims are supported by the supplied source.
Method: source-linked assessment.
Blocking: yes.
S3 — Caveats
The summary preserves material limitations and qualifications.
Method: source-linked assessment.
Blocking: yes.
S4 — Readability
The summary is appropriate for the declared audience.
Method: rubric-based assessment.
Blocking: no.
Criteria SHOULD describe observable properties. Instructions such as “be excellent” or “think carefully” are not adequate acceptance criteria.
Contracts MAY cover inputs, outputs, collections or relationships. A coverage contract may compare a requirements register with an implementation plan.
5.3 Evaluation outcomes
Each check returns:
- Pass: the evaluator found the criterion satisfied.
- Fail: the evaluator found a violation.
- Inconclusive: evidence or evaluator capability was insufficient.
- Error: the check did not complete correctly.
- NotApplicable: a declared applicability rule excludes the criterion.
A numerical score MAY supplement these outcomes. It MUST NOT hide a blocking failure.
Uncalibrated model confidence MUST NOT be presented as a probability of correctness.
5.4 Evaluation records and trusted attribution
An Evaluation Record MUST identify:
Exact target versions
Relevant evidence versions
Contract and criterion versions
Evaluator binding and implementation identity
Evaluation context or request reference
Outcome and findings
Evidence references
Creation event
The engine, not the model, supplies trusted attribution fields.
A model may return findings and proposed outcomes. It MUST NOT be allowed to establish its own evaluator identity, substitute the evaluated target digest or assert that an approval occurred.
Malformed findings, omitted required criteria and mismatched targets produce an evaluation error. They MUST NOT be interpreted as a pass.
Evaluation records are immutable. Reassessment creates a new record.
5.5 Acceptance policy
An Acceptance Record answers:
May these exact artefact versions be used for this purpose, under this policy, at this decision point?
It identifies:
- target versions;
- declared purpose;
- policy version;
- supporting evaluation records;
- applicable approvals;
- decision and conditions;
- authorising identity or policy service;
- validity period or freshness conditions;
- and any exception authority.
Possible dispositions are:
- Accepted
- AcceptedWithWarnings
- AcceptedByException
- NeedsReview
- Rejected
By default, every applicable blocking criterion must pass. Inconclusive and Error block acceptance. NotApplicable is permitted only when its applicability rule is satisfied.
AcceptedWithWarnings is for non-blocking findings. It MUST NOT disguise an overridden blocking failure.
AcceptedByException requires an authorised exception process that records the failed criterion, reason, residual risk, scope and expiry. Consumers must explicitly permit that disposition. Non-overridable controls remain enforced.
5.6 Candidate access and purpose-specific use
A candidate is an artefact version that has not satisfied the acceptance requirements for a particular use.
The same version may be accepted for one purpose and remain a candidate for another.
Ordinary production inputs require the declared acceptance conditions. Evaluation, repair and diagnostic inputs may explicitly allow candidates.
Generate and commit candidate
↓
Evaluate candidate
↓
Apply use policy
↙ ↘
Accept Reject or review
↓
Bounded repair
↓
Commit new version
↓
Evaluate again
Candidate access is not a route around data permissions. A repair operation still needs authority to read the content.
5.7 Freshness, revocation and later use
Acceptance is not necessarily permanent.
A decision may cease to authorise future use when:
- its validity period expires;
- required evidence becomes stale;
- an approval is revoked;
- the governing policy changes;
- or a material defect is discovered.
These events are appended to the record. Historical decisions are not rewritten.
Before a protected use or external action, the engine MUST check that the required acceptance remains applicable.
A changed artefact does not inherit acceptance from an earlier version. Any reuse of evaluation evidence across transformations requires an explicit policy and a checked relationship establishing why that evidence remains applicable.
5.8 Source grounding
Factual workflows SHOULD preserve references containing:
- source version;
- page, section, record or timestamp;
- span or stable locator;
- and, where useful, an excerpt.
Extraction and optical character recognition SHOULD preserve mappings back to the original source. A location in extracted text is not automatically a location in the original document.
Claims should distinguish:
- SourceStated: explicitly stated by a source;
- Inferred: derived from source material;
- Proposed: introduced as a recommendation or design choice;
- Unresolved: evidence is absent, insufficient or conflicting.
“SourceStated” does not mean independently true.
Grounding checks distinguish:
- whether the cited location exists;
- whether it supports the statement;
- whether the source is appropriate and credible;
- whether contradictory evidence exists;
- whether the evidence is sufficiently current.
Citation presence alone does not establish support.
5.9 Coverage and review quality
A workflow MUST distinguish a complete source inventory from a partial search result.
“Not found in retrieved material” is not equivalent to “does not exist.”
Reviews SHOULD identify concrete defects and evidence, distinguish critical errors from preferences and permit inconclusive outcomes.
Multiple model reviewers are not automatically independent. Shared models, prompts and evidence gaps can produce correlated errors.
Review mechanisms themselves require evaluation against known defects and acceptable outputs.
9. Context Construction and Model Execution
9.1 Context is an execution input
Logical inputs and information actually sent to a model may differ because of extraction, retrieval, formatting or capacity limits.
Every model request MUST have a Context Manifest identifying:
- input versions;
- selected excerpts or records;
- transformations applied;
- ordering;
- omitted material;
- compression or truncation;
- template and adapter versions;
- effective request parameters;
- and the resulting request identity or protected digest.
The manifest records what was actually supplied, not merely what the operation was allowed to access.
9.2 Instruction and data separation
Requests SHOULD distinguish:
- engine-enforced instructions;
- trusted Module instructions;
- contracts and output requirements;
- source artefacts as data;
- tool observations as data.
Imperative language in a source does not acquire instruction authority.
Delimiters help interpretation but are not a security boundary. Capability enforcement remains outside the model.
9.3 Context limits
Required context MUST NOT be silently omitted.
If capacity is insufficient, permitted responses include:
- fail with a diagnostic;
- select an approved larger-context binding;
- use a declared retrieval strategy;
- or invoke explicit partitioning or compression.
Compression creates a new artefact with provenance and coverage limitations.
Continuation requests and hidden formatting calls count as model requests and consume the declared budget.
9.4 Retrieval manifests
Retrieval-dependent workflows SHOULD record:
- query and filters;
- corpus or index version;
- applicable access-control context;
- retrieval implementation;
- selected source versions and locators;
- ordering;
- declared result limits;
- and material coverage limitations.
Where sources cannot be snapshotted, record observation times and freshness assumptions.
A retrieval result is evidence of what was retrieved, not proof that all relevant material was found.
9.5 Context minimisation
Operations should receive enough context to perform their task, not the whole workspace by default.
Minimisation reduces cost, distraction and unnecessary disclosure. It must not remove the primary evidence needed to assess factual claims.
Where coverage is essential, the workflow should separately establish source inventory and coverage rather than rely on an unconstrained relevance search.
9.6 Output capture
Multiple outputs SHOULD use structured response mechanisms or an explicit envelope.
The engine MUST NOT infer file boundaries from arbitrary prose headings unless a versioned parser defines that behaviour.
Model output is untrusted input to parsers, renderers, databases and tools.
9.7 Observable work, not private reasoning
The framework records source selections, plans, claims, concise decision explanations, tool calls, outputs and evaluations.
It does not require access to a model’s private internal reasoning.
Auditability rests on observable actions, evidence and accountable decisions.
The full framework can be found on my GitHub.
I hope someone who understands this can make use of it.
Thank you for reading.
- Rob