Enhancement Request: Improved Support for Capturing and Submitting Evidence of Bugs
Summary:
When ChatGPT behavior deviates (e.g., reverses previous refinements, loses state, reintroduces resolved issues), users have no structured way to capture and submit contextual evidence. Manual copy/paste is error-prone, omits internal state, and places a heavy burden on both user and support staff.
Problem:
- Users must manually describe issues without access to model state, session ID, or reproduction context.
- Bugs that involve state drift, forgotten context, or logic regressions are nearly impossible to isolate reliably.
- This limits OpenAI’s ability to identify patterns or regressions from real-world usage.
- The current process is akin to copying details from faxes sent from an air-gapped machine — disconnected, tedious, and error-prone.
Suggested Enhancement:
Introduce a built-in “Submit Feedback with Diagnostic Context” feature that:
- Captures selected or recent message range (with option to redact user content)
- Includes internal metadata:
- Model ID/version/hash
- Timestamp range and session ID
- User settings (custom instructions, memory state toggle)
- Provides optional JSON download for offline bug submission (e.g., via forum or email)
Security and Integrity:
- Attach a cryptographic signature or HMAC to verify report authenticity and detect tampering.
- Allows OpenAI engineers to safely analyze external submissions (e.g., shared via forum) with confidence.
User Privacy and Control:
- All diagnostic data is locally viewable and editable before upload.
- Only internal metadata (e.g., model hashes) may be opaque — no encrypted or hidden content.
- Nothing is auto-submitted — users explicitly choose when to download and share.
- This protects privacy while enabling reproducibility for those who opt in.
Why It Matters (to Users):
- Enables frictionless reporting of real bugs, not just vague summaries.
- Improves reproducibility of state-sensitive issues (e.g., memory misfires, reintroduced bugs).
Why It Matters (to OpenAI):
- Reduces support load by allowing early triage of high-confidence reports.
- Helps isolate hard-to-debug model behaviors with user-assisted telemetry.
- Ensures submissions are trustworthy and verifiable without backend guesswork.
Consistency with Industry Best Practices:
This approach mirrors best-in-class enterprise diagnostic frameworks like Oracle’s Automatic Diagnostic Repository (ADR) and Windows Error Reporting (formerly Dr. Watson):
- ADR collects detailed crash diagnostics (logs, dumps, config, history) into a portable, structured repository.
- Windows Error Reporting automates the capture and upload of crash telemetry with user consent.
In both cases, vendors gain the context they need to reproduce and triage problems — without compromising privacy or requiring complex manual reports. ChatGPT should offer a modern, LLM-aligned equivalent.
Strategic Benefit:
The more technical users can help you debug with real telemetry, the less guesswork is needed — meaning lower cost per issue, faster turnarounds, and better feedback loops overall.
Sign-off:
– MDD