Urgent Codex CLI maintainer triage requested: two reproducible workflow-blocking reliability failures in 0.153.4

fengyeelx · September 6, 2026, 7:35am

I am urgently asking for help getting two serious Codex CLI reliability issues in front of the appropriate Codex engineering owners.

I have already reported both issues in the official openai/codex repository, collected diagnostics, inspected the relevant client source, and uploaded additional diagnostics through Codex /feedback. I am posting here because both reports are currently still unassigned and have received no maintainer response, while the failures are severely disrupting real development work.

I am a ChatGPT Pro user and I am not reporting a hypothetical edge case or a generic “my network disconnected” problem. One of the failures now has a deterministic localhost-only reproduction against the actual installed Codex CLI 0.153.4 Windows binary.

1. WebSocket false-dead-stream / idle watchdog failure

GitHub:

github.com/openai/codex

Severe WS stalls in 0.153.4: live Ping/Pong still trips the idle watchdog (Windows loopback-only reproduction)

opened 02:22PM - 05 Sep 26 UTC

FengYeeLx

bug windows-os CLI connectivity

## Urgent reliability report Please urgently investigate the distinction betwee…n WebSocket transport liveness and application-event silence in the Codex CLI idle watchdog. I have now reproduced the relevant behavior using the **actual installed Windows `codex-cli 0.153.4` executable**, an isolated `CODEX_HOME`, and a synthetic server listening only on localhost. **There are no images, no real model inference, no account credentials supplied, and no production proxy in the model-request path.** The peer sends protocol Ping frames, and Codex demonstrably replies with Pong. Nevertheless, Codex reports `idle timeout waiting for websocket` and abandons the WebSocket. Sending application-level JSON heartbeats instead, or increasing only the configured idle threshold, allows the same delayed response to complete over WebSocket. **I am seriously dissatisfied with the repeated multi-minute stalls in real work. Codex CLI becomes practically unusable for the affected task when a short local command completes and the next model step repeatedly disappears into timeout/retry cycles. A paid coding workflow should not require users to reverse-engineer transport internals or build a mock server to understand why “working” produces no useful progress. Please provide urgent triage, an actionable supported mitigation, and a prioritized correction to the liveness/progress handling.** This is a **separate failure mode from my image-history report [#43015](https://github.com/openai/codex/issues/43015)**. It is closely related to [#39771](https://github.com/openai/codex/issues/39771), with additional current-stable Windows evidence and a small local reproduction. Please consolidate into the existing tracker if appropriate; the purpose is to provide reproducible evidence, not duplicate noise. ## Environment and production symptom | Field | Value | |---|---| | Installed CLI | `codex-cli 0.153.4` | | Platform | Windows x64, OS build `10.0.26100` | | Shell | PowerShell 7.6 | | Production provider/auth | Built-in OpenAI provider; ChatGPT Pro authentication | | Production model | `gpt-6-astra`, `ultra` selection | | Production network | Local v2rayN proxy; disclosed separately from the loopback reproduction | | Audited source | `rust-v0.153.4`, commit `3d2ee51ca2d5db578f328aa75e20aa22c0197c9a` | | Lab provider | Custom, unauthenticated localhost Responses provider, solely to run the installed binary against a synthetic peer | The production session was doing primarily text/code maintenance. During the inspected resumed period, **no new image input or image-viewing output was recorded**. The reconstructed post-compaction history contained only one old image of **133,762 data-URL characters** and approximately **1.48 MB of serialized history items overall**. This is a structural history measurement, not a capture of every actual WS request. The most recent completed input was about **278,055 tokens**, with **275,840 cached**, against a reported context window of **828,400**. This is not evidence of a full context window or a tens-of-megabytes image request. Recent local command executions completed successfully in approximately **0.36–0.41 seconds**. The process remained alive; the inspected system snapshot showed about 10% CPU load and ample free memory. These snapshots do not rule out every environmental problem, but they do not support blaming a locally hung command or resource exhaustion for the recurring stream waits. ### Observed production errors On September 5, 2026 (UTC), the same turn recorded: | Time | Error | |---|---| | 13:18:13 | `websocket closed by server before response.completed` | | 13:23:55 | `idle timeout waiting for websocket` | | 13:34:19 | `idle timeout waiting for websocket` | | 13:45:04 | `idle timeout waiting for websocket` | | 13:58:09 | `idle timeout waiting for websocket` | Reconnect handshakes typically succeeded in approximately 1–2 seconds. For example, the 13:58:09 timeout was followed by a successful handshake at 13:58:11. After a tool result at 13:15:09, the next recorded model tool call was not until 13:39:30. That interval includes retries and intervening input/agent events; it is not a pure upload-time measurement. I distinguish the first server-close error from the later client idle errors. I have **not** captured enough production wire evidence to prove that every one of those online failures occurred while the upstream model was still healthy. The controlled experiment below establishes the client's behavior without relying on that assumption. ## Controlled local reproduction: three cases The test runs the unmodified installed CLI against a loopback-only synthetic peer. The peer acknowledges the request with `response.created`, delays the final answer for **3.2 seconds**, then sends valid Responses events containing `OK`. For a short, bounded test, the idle threshold is **1,500 ms** instead of the production default 300,000 ms. The comparison uses the same binary and response sequence. `stream_max_retries=0` avoids wasting time on repeated attempts; the CLI may immediately use HTTP fallback. This is a deliberately scaled reproduction of the same wait path, not a claim that production used a 1.5-second timeout. | Case | Heartbeat during the 3.2-second delay | Idle threshold | Peer observed | Result | |---|---|---:|---|---| | A | WebSocket protocol Ping | 1,500 ms | **7 Ping / 7 valid Pong** | **Idle error and HTTP fallback** | | B | JSON text event `{"type":"keepalive"}` | 1,500 ms | 14 application heartbeats | **Normal WS completion**, no idle error, no HTTP fallback | | C | WebSocket protocol Ping | 6,000 ms | 14 Ping / 14 valid Pong | **Normal WS completion**, no idle error, no HTTP fallback | Measured from the synthetic `response.created` event: - Case A: the HTTP fallback POST arrived after **1.5125 seconds**. - Case A: the last valid Pong was received after **1.4655 seconds**, approximately **47 ms before fallback**. - Case B: WebSocket completion was sent after **3.2036 seconds**. - Case C: WebSocket completion was sent after **3.2052 seconds**. The largest `response.create` frame payload, including warmup, was **42,471 bytes** in every case. This reproduction does not involve a large request or image payload. All three CLI processes exited with code 0 because the fixture permits a successful HTTP fallback in case A. **The demonstrated problem is the idle classification and abandonment of a live delayed WebSocket—not a CLI crash or failed synthetic final answer.** The exact CLI warning in case A was: ```text Falling back from WebSockets to HTTPS transport. stream disconnected before completion: idle timeout waiting for websocket ``` The fixture deliberately uses plain HTTP/WS on loopback. “HTTPS transport” above is the CLI's own generic warning text; the lab does not perform TLS interception or weaken certificate validation. ## What the experiment proves—and what it does not **Proven by the actual binary:** protocol-level liveness can continue while the upper response waiter expires. A valid Pong was observed immediately before the CLI abandoned the connection. An application-level heartbeat resets the effective wait, whereas protocol Ping/Pong does not. Changing only the idle threshold also changes the outcome. **Proven by the fixture controls:** the synthetic application has a viable delayed completion; cases B and C deliver it normally. The failure in A does not require an OpenAI outage, proxy misconfiguration, image history, or a dead TCP path. **Not proven:** that every production idle timeout is a false positive; that Ping/Pong alone proves a real model is making progress; or that simply raising every timeout is a complete fix. A transport can remain alive while an application genuinely stalls. That is exactly why these conditions need separate accounting and diagnostics. The current waiter is effectively measuring **forwarded response-message inactivity**, not transport liveness and not necessarily meaningful model progress. If the intended contract requires application heartbeats during long reasoning, please state and enforce that contract end-to-end rather than leaving users with opaque repeated retries. ## Source-level explanation in 0.153.4 1. [`model-provider-info/src/lib.rs`][provider] defines `DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000`. The production config inspected did not contain a timeout override. 2. In [`responses_websocket.rs`][websocket], the WebSocket pump answers `Ping` with `Pong` and consumes `Pong` frames. These control frames are not forwarded through the message channel used by the response reader. 3. The same file's `run_websocket_response_stream` awaits `timeout(idle_timeout, ws_stream.next())`. Expiry generates the exact `idle timeout waiting for websocket` error seen in the production logs and case A. 4. [`responses_retry.rs`][retry] handles retry/fallback after such stream errors. A new sampling step can have a fresh retry budget; repeated `1/5` notices in a long task are not a total per-task failure counter. 5. In the inspected provider merge function, an existing built-in provider entry wins through `entry(key).or_insert(provider)`. Consequently, the ordinary `[model_providers.openai]` approach is not a reliable way to override this built-in provider's timeout. The custom-provider override used by the lab should not be advertised as an already-validated production ChatGPT workaround. The code explains why an unknown application-level heartbeat can preserve the wait while a recent protocol Pong does not. The local three-case run verifies that this distinction is present in the installed release, rather than only inferred from source or copied from another user's report. ## Expected behavior and requested urgent action Please address **both** false-dead-stream detection and genuinely dead streams: - Track **transport liveness** separately, with explicit Ping/Pong and a bounded failure deadline when transport health checks fail. - Track **application silence/progress** separately for an acknowledged in-flight response. Do not label the transport dead solely because no forwarded application event arrived during a legitimate long reasoning interval. - Ensure a documented application heartbeat contract during long inference, or implement an equivalent client/server mechanism that the correct clock observes. - Retain a bounded way to detect a truly stuck application; merely accepting Ping forever is not a complete solution. - Expose supported timeout/retry/transport controls for the built-in ChatGPT provider, with effective values visible to users. Do not suggest a configuration key that the merge path ignores. - Avoid repeatedly restarting a still-viable sampling operation without explaining what timed out. Show whether the request is acknowledged, when transport activity last occurred, and when application progress last occurred. - Provide a supported mitigation for existing affected sessions and identify the release that will contain the fix. Simply shortening the current timer risks interrupting more healthy long responses; simply lengthening it delays detection of genuine failures. The three-case result demonstrates why one undifferentiated inactivity clock is insufficient for these different states. **Please treat this as a severe, workflow-disrupting reliability issue and respond as soon as possible with an actionable mitigation and fix plan.** I am not asserting data destruction or a universal failure of all prompts; I am reporting repeated real disruption plus a reproducible client behavior that can explain it. ## Related issues and scope - [#39771](https://github.com/openai/codex/issues/39771): directly related high-reasoning false-dead-stream report. This adds an isolated local three-case reproduction on Windows stable 0.153.4. - [#38638](https://github.com/openai/codex/issues/38638): discusses slow detection of dead streams. A fix must also avoid terminating live, application-silent responses prematurely. - [#27625](https://github.com/openai/codex/issues/27625): repeated idle timeouts during one task. - [#23807](https://github.com/openai/codex/issues/23807) and [#33051](https://github.com/openai/codex/issues/33051): related five-minute waits; not all such reports establish the same cause. - [#43015](https://github.com/openai/codex/issues/43015): my separate large-image-history report. The present local reproduction is image-free, and the inspected production session has only a small old image. ## Diagnostic and privacy boundaries No new full `codex doctor --json` report was collected. The investigation used targeted read-only production inspection, pinned source review, and the isolated actual-CLI test below. No production session was modified or interrupted by the test. Its temporary homes were separately verified and removed afterward; the reproduction source and results were preserved. Raw production prompts, source code, images, session IDs, credentials, private request IDs and proxy account details are omitted. The code below uses only synthetic data and a local endpoint. ## Reproduction instructions Requires Python's standard library and an installed Codex CLI 0.153.4 executable. No additional Python packages or real API credentials are needed. Save the code below as `codex_ws_idle_repro.py`, then run: ```powershell python -B .\codex_ws_idle_repro.py --codex 'C:\path\to\codex.exe' --output .\ws-idle-results.json ``` It runs three bounded cases, writes JSON results, and prints its uniquely created temporary directory for inspection and separate cleanup. It does not change the normal Codex home. It uses an isolated unauthenticated custom provider to redirect model requests to localhost; this differs intentionally from production authentication and timing. The source below is the exact tested script, SHA-256 `DB9ADD011C78D06624F1D404F0CB012969CDD78A6A52E8C02689999F723FD8C6`. [provider]: https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/model-provider-info/src/lib.rs [websocket]: https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/codex-api/src/endpoint/responses_websocket.rs [retry]: https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/core/src/responses_retry.rs <details> <summary>Complete tested reproducer (Python standard library)</summary> ```python """Exercise an installed Codex CLI against a loopback-only synthetic peer. No OpenAI credentials, real model requests, project tools, or image data are used. The caller retains the scratch directory for separate, verified cleanup. """ import argparse import base64 import hashlib import http.server import json import os from pathlib import Path import select import socket import struct import subprocess import tempfile import threading import time class Case: def __init__(self, name, heartbeat, idle_ms, delay_s=3.2): self.name = name self.heartbeat = heartbeat self.idle_ms = idle_ms self.delay_s = delay_s self.events = [] self.lock = threading.Lock() self.stop = threading.Event() self.start = time.monotonic() def log(self, event, **fields): with self.lock: self.events.append({"t_s": round(time.monotonic() - self.start, 4), "event": event, **fields}) def recv_exact(sock, count): chunks = bytearray() while len(chunks) < count: part = sock.recv(count - len(chunks)) if not part: raise EOFError("peer closed") chunks.extend(part) return bytes(chunks) def recv_frame(sock): first, second = recv_exact(sock, 2) opcode = first & 15 size = second & 127 if size == 126: size = struct.unpack("!H", recv_exact(sock, 2))[0] elif size == 127: size = struct.unpack("!Q", recv_exact(sock, 8))[0] if size > 2_000_000: raise ValueError("synthetic-test frame limit exceeded") mask = recv_exact(sock, 4) if second & 128 else None payload = recv_exact(sock, size) if mask: payload = bytes(value ^ mask[i % 4] for i, value in enumerate(payload)) return opcode, payload def send_frame(sock, opcode, payload): size = len(payload) if size < 126: header = bytes([128 | opcode, size]) elif size < 65536: header = bytes([128 | opcode, 126]) + struct.pack("!H", size) else: header = bytes([128 | opcode, 127]) + struct.pack("!Q", size) sock.sendall(header + payload) def response_events(response_id, warmup=False): message = {"id": "msg_synthetic", "type": "message", "role": "assistant", "status": "completed", "content": [{"type": "output_text", "text": "OK", "annotations": []}]} response = {"id": response_id, "object": "response", "status": "completed", "model": "gpt-6-astra", "output": [] if warmup else [message], "usage": {"input_tokens": 1, "output_tokens": 0 if warmup else 1, "total_tokens": 1 if warmup else 2, "input_tokens_details": {"cached_tokens": 0}, "output_tokens_details": {"reasoning_tokens": 0}}} if not warmup: yield {"type": "response.output_item.added", "output_index": 0, "item": {**message, "status": "in_progress", "content": []}} yield {"type": "response.content_part.added", "item_id": "msg_synthetic", "output_index": 0, "content_index": 0, "part": {"type": "output_text", "text": "", "annotations": []}} yield {"type": "response.output_text.delta", "item_id": "msg_synthetic", "output_index": 0, "content_index": 0, "delta": "OK"} yield {"type": "response.output_text.done", "item_id": "msg_synthetic", "output_index": 0, "content_index": 0, "text": "OK"} yield {"type": "response.content_part.done", "item_id": "msg_synthetic", "output_index": 0, "content_index": 0, "part": message["content"][0]} yield {"type": "response.output_item.done", "output_index": 0, "item": message} yield {"type": "response.completed", "response": response} class Peer(http.server.BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" def log_message(self, *args): pass def do_GET(self): case = self.server.case if self.headers.get("Upgrade", "").lower() != "websocket": case.log("non_websocket_get", path=self.path) body = b'{"models":[]}' self.send_response(200) self.send_header("Content-Type", "application/json") self.send_header("Content-Length", str(len(body))) self.end_headers() self.wfile.write(body) return key = self.headers.get("Sec-WebSocket-Key", "") accept = base64.b64encode(hashlib.sha1((key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11").encode()).digest()).decode() self.send_response(101) self.send_header("Upgrade", "websocket") self.send_header("Connection", "Upgrade") self.send_header("Sec-WebSocket-Accept", accept) self.end_headers() self.wfile.flush() self.close_connection = True self.connection.settimeout(2) case.log("websocket_upgraded") try: while not case.stop.is_set(): opcode, payload = recv_frame(self.connection) if opcode == 8: case.log("client_close") return if opcode == 10: case.log("pong_received") continue if opcode != 1: case.log("unexpected_client_opcode", opcode=opcode) continue request = json.loads(payload) if request.get("type") != "response.create": case.log("other_client_event", kind=request.get("type")) continue warmup = request.get("generate") is False case.log("response_create_received", warmup=warmup, request_bytes=len(payload)) response_id = "resp_warmup" if warmup else "resp_synthetic" created = {"type": "response.created", "response": {"id": response_id, "status": "in_progress", "model": "gpt-6-astra", "output": []}} self.send_json(created) case.log("response_created_sent", warmup=warmup) if not warmup: start = time.monotonic() next_beat = start + 0.15 while time.monotonic() - start < case.delay_s and not case.stop.is_set(): now = time.monotonic() if now >= next_beat: if case.heartbeat == "protocol_ping": send_frame(self.connection, 9, b"synthetic-liveness") case.log("ping_sent") else: self.send_json({"type": "keepalive"}) case.log("application_keepalive_sent") next_beat = now + 0.2 ready, _, _ = select.select([self.connection], [], [], 0.03) if ready: op, data = recv_frame(self.connection) if op == 10: case.log("pong_received") elif op == 8: case.log("client_close_during_wait") return elif op == 9: send_frame(self.connection, 10, data) case.log("client_ping_received") else: case.log("unexpected_during_wait", opcode=op) if case.stop.is_set(): return for event in response_events(response_id, warmup): self.send_json(event) case.log("completion_sent", transport="websocket", warmup=warmup) except (EOFError, OSError, ValueError) as error: case.log("peer_connection_end", error_type=type(error).__name__) def send_json(self, event): send_frame(self.connection, 1, json.dumps(event, separators=(",", ":")).encode()) def do_POST(self): case = self.server.case size = int(self.headers.get("Content-Length", "0")) if size > 2_000_000: self.send_error(413) return self.rfile.read(size) case.log("http_fallback_post", request_bytes=size) events = [{"type": "response.created", "response": {"id": "resp_http", "status": "in_progress", "output": []}}] events.extend(response_events("resp_http")) body = b"".join(b"data: " + json.dumps(event, separators=(",", ":")).encode() + b"\n\n" for event in events) self.send_response(200) self.send_header("Content-Type", "text/event-stream") self.send_header("Content-Length", str(len(body))) self.end_headers() self.wfile.write(body) self.wfile.flush() case.log("completion_sent", transport="http", warmup=False) def run_case(executable, scratch, name, heartbeat, idle_ms): case = Case(name, heartbeat, idle_ms) home = scratch / name / "home" work = scratch / name / "work" home.mkdir(parents=True) work.mkdir() server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Peer) server.daemon_threads = True server.case = case threading.Thread(target=server.serve_forever, daemon=True).start() port = server.server_address[1] config = f'''model = "gpt-6-astra" model_provider = "local_idle_probe" approval_policy = "never" web_search = "disabled" [features] memories = false plugins = false shell_snapshot = false [memories] use_memories = false generate_memories = false [model_providers.local_idle_probe] name = "Loopback synthetic idle probe" base_url = "http://127.0.0.1:{port}/v1" wire_api = "responses" requires_openai_auth = false supports_websockets = true stream_idle_timeout_ms = {idle_ms} stream_max_retries = 0 request_max_retries = 0 ''' (home / "config.toml").write_text(config, encoding="utf-8") allowed = {"SYSTEMROOT", "WINDIR", "COMSPEC", "PATHEXT", "PATH", "TEMP", "TMP", "USERPROFILE", "APPDATA", "LOCALAPPDATA", "HOMEDRIVE", "HOMEPATH", "NUMBER_OF_PROCESSORS", "PROCESSOR_ARCHITECTURE"} env = {k: v for k, v in os.environ.items() if k.upper() in allowed} env.update(CODEX_HOME=str(home), NO_PROXY="127.0.0.1,localhost,::1") command = [str(executable), "exec", "--json", "--ephemeral", "--skip-git-repo-check", "--ignore-rules", "--color", "never", "--sandbox", "read-only", "--cd", str(work), "Reply with OK. Do not call any tools."] start = time.monotonic() timed_out = False try: result = subprocess.run(command, env=env, capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=25, creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0)) stdout, stderr, code = result.stdout, result.stderr, result.returncode except subprocess.TimeoutExpired as error: timed_out = True stdout = error.stdout.decode("utf-8", "replace") if isinstance(error.stdout, bytes) else error.stdout or "" stderr = error.stderr.decode("utf-8", "replace") if isinstance(error.stderr, bytes) else error.stderr or "" code = None finally: case.stop.set() server.shutdown() server.server_close() combined = stdout + "\n" + stderr counters = {} for event in case.events: counters[event["event"]] = counters.get(event["event"], 0) + 1 errors = [line for line in combined.splitlines() if "idle timeout" in line or "falling back" in line.lower() or '"type":"error"' in line] errors = [line.replace(str(home), "<isolated-home>").replace(str(work), "<isolated-work>") for line in errors] return {"name": name, "heartbeat": heartbeat, "configured_idle_ms": idle_ms, "synthetic_completion_delay_s": case.delay_s, "process_timeout": timed_out, "exit_code": code, "elapsed_s": round(time.monotonic() - start, 3), "idle_timeout_reported": "idle timeout waiting for websocket" in combined, "http_fallback_observed": counters.get("http_fallback_post", 0) > 0, "event_counts": counters, "selected_client_messages": errors, "events": case.events, "client_stdout": stdout.replace(str(home), "<isolated-home>").replace(str(work), "<isolated-work>"), "client_stderr": stderr.replace(str(home), "<isolated-home>").replace(str(work), "<isolated-work>")} def main(): parser = argparse.ArgumentParser() parser.add_argument("--codex", type=Path, required=True) parser.add_argument("--output", type=Path, required=True) args = parser.parse_args() scratch = Path(tempfile.mkdtemp(prefix="codex-ws-idle-probe-")) print(json.dumps({"scratch_directory": str(scratch)}), flush=True) report = {"purpose": "Installed CLI versus loopback-only synthetic peer", "real_model_requests": 0, "production_configuration_modified": False, "codex_binary_sha256": hashlib.sha256(args.codex.read_bytes()).hexdigest(), "cases": []} for spec in [("ping_short_idle", "protocol_ping", 1500), ("application_heartbeat_short_idle", "application_keepalive", 1500), ("ping_long_idle", "protocol_ping", 6000)]: result = run_case(args.codex, scratch, *spec) report["cases"].append(result) args.output.write_text(json.dumps(report, indent=2, ensure_ascii=False), encoding="utf-8") print(json.dumps({k: result[k] for k in ["name", "exit_code", "elapsed_s", "idle_timeout_reported", "http_fallback_observed", "event_counts"]}), flush=True) print(json.dumps({"output": str(args.output), "scratch_directory": str(scratch)}), flush=True) if __name__ == "__main__": main() ``` </details>

This issue has been reduced to a controlled three-case experiment using the unmodified installed Codex CLI executable and a synthetic server listening only on localhost.

There is:

no OpenAI inference request;
no account credential involved;
no image payload;
no production proxy in the model-request path.

In the failing case, the peer continuously sends WebSocket protocol Ping frames and Codex continuously returns valid Pong frames.

Nevertheless, Codex still reports:

idle timeout waiting for websocket

and abandons the WebSocket / falls back.

The last valid Pong was observed only about 47 ms before the fallback occurred.

Two controls are particularly important:

replacing protocol Ping with an application-level JSON heartbeat allows the identical delayed response to complete normally over WebSocket;
changing only the configured idle threshold also allows the identical response to complete normally.

This appears to demonstrate a real distinction between transport liveness and forwarded Responses application-event silence in the current idle-watchdog path.

I am not claiming that Ping/Pong proves that a model is making useful inference progress. The claim is narrower: a live transport and application silence are currently being collapsed into a failure reported as idle timeout waiting for websocket, which can cause a viable long-reasoning request to be abandoned and retried.

In production this has manifested as repeated multi-minute stalls and retry/fallback cycles during long Codex reasoning.

Related earlier report:

github.com/openai/codex

High-reasoning Responses WebSocket idle timeout is a false dead-stream (model resumed after 8.5 min of no text frames)

opened 06:18PM - 20 Aug 26 UTC

aygerix

bug app connectivity

### What version of the Codex App are you using (From “About Codex” dialog)? Ch…atGPT desktop **26.814.41407** (bundled `codex-cli` **0.148.0-alpha.15**) ### What subscription do you have? ChatGPT Pro (`plan_type=pro` in local rate-limit snapshots). Auth mode: ChatGPT (not API key). ### What platform is your computer? ```text Darwin 24.6.0 arm64 arm macOS 15.7.7 (24G720) ``` ### What issue are you seeing? On `gpt-5.6-sol` with `reasoning_effort=high`, a turn that is **still generating** can go several minutes with **no Responses text frames**. The client treats that silence as a dead websocket: ```text WARN codex_core::responses_retry stream disconnected - retrying sampling request (n/5) ... sampling_error=stream disconnected before completion: idle timeout waiting for websocket ``` The UI stays **active** with frozen token counts. There is no user-visible disconnect. After 5 silent retries the turn is wasted. Interrupt shows `turn_aborted`. This is **not** (only) a dropped TCP connection. Protocol ping/pong do not reset the idle timer (they are swallowed in the websocket pump and never reach `ws_stream.next()`). The watchdog is `tokio::time::timeout(stream_idle_timeout, ws_stream.next())` with default `stream_idle_timeout_ms = 300_000`. Only a **text** frame resets it. `stream_idle_timeout_ms` / `supports_websockets` cannot be changed for the built-in `openai` provider (`model_providers.openai` is reserved / `or_insert` ignores overrides). ### What steps can reproduce the bug? 1. Codex Desktop, ChatGPT login, model `gpt-5.6-sol`, reasoning **high**. 2. Run a long turn: many `exec` calls, then a long think/write (large patch / gate file). Transport is `wss://chatgpt.com/backend-api/codex/responses`. 3. After the last commentary + tool result, the model goes quiet. No `agent_reasoning` / tool / token snapshot advances for >5 minutes. 4. Client logs `idle timeout waiting for websocket` and retries the **entire** sampling request (`1/5` … `5/5`). Chat still looks “working”. **Controlled experiment (same machine, same thread, same model):** a localhost reverse proxy in front of `https://chatgpt.com/backend-api/codex` injected a text frame every 45s while upstream was silent: ```json {"type":"codex.client.keepalive"} ``` Codex accepted the frames and ignored them as unknown events: ```text DEBUG codex_api::sse::responses unhandled responses event: "codex.client.keepalive" ``` No idle timeout fired. After **~8.5 minutes** of no real model output (`18:06:20Z` → `18:14:56Z` on 2026-08-20), the **same in-flight sampling request** emitted a real `custom_tool_call` / `patch_apply_end` and continued. Frozen token counts then moved. That is the opposite of a dead stream: the websocket and the server were still in the turn; the 300s text-frame watchdog was a **false positive**. **Earlier incident on the same thread without the keepalive (same day):** | UTC | What happened | |---|---| | 17:00:02 | Last real commentary (“now freezing the … gate”) | | 17:07:34 / 17:13:11 / 18:18:50 | `idle timeout waiting for websocket` retries 1/5, 2/5, 3/5 | | 17:21:49 | User interrupt; `turn_aborted`; promised file never written | Thread id: `01a02010-e5b0-75f1-b3b7-3e0c54879f59` Turn (keepalive run): `01a02059-3cbe-70d2-9c1b-881ae7f1cfa4` Context at last good request: ~99k / 258400 input tokens (~38%). `rate_limit_reached_type` was `null` (weekly usage was high, ~95%, but not marked reached). Same `idle timeout waiting for websocket` pattern on other `gpt-5.6-sol` high/medium Desktop turns the same day (`websocket closed by server before response.completed` and `Connection reset without closing handshake` also occur, but those are distinct). ### What is the expected behavior? - Do not treat **in-progress high-reasoning silence** (no Responses text frames, websocket still open) as a dead stream. - Server or client should emit a liveness text/event that the idle waiter counts, **or** the idle timer should not apply until the socket is actually closed / ping fails. - If you keep a 300s watchdog: surface it in the UI (“waiting for model, last frame Ns ago”) instead of silent `retrying sampling request (n/5)` with frozen tokens. - Allow `stream_idle_timeout_ms` (and/or websocket opt-out) on the built-in ChatGPT `openai` provider. Today the only documented knobs are on custom providers. A faster dead-stream detector (#38638, #17003) would have **killed this turn** at 5 minutes. The model resumed at 8.5 minutes. Faster detection of true RSTs is still good; collapsing “no text frames” into “dead” is not. ### Additional information Related: #38638, #27625, #28579, #24260, #23807, #26696, #33051. #23807 is a 300s stall **between** completed responses (turn state machine). This report is idle timeout **during** an open sampling stream while `gpt-5.6-sol` high is thinking and sending no text. #38638 matches the log line and the “cannot override built-in provider” constraint; the keepalive experiment is extra evidence that the 300s timeout is often a **false dead-stream**, not only slow detection of a real drop. Code references on `main` (behavior matches 0.148.0-alpha.15): - `DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000` in `codex-rs/model-provider-info/src/lib.rs` - `timeout(idle_timeout, ws_stream.next())` → `"idle timeout waiting for websocket"` in `codex-rs/codex-api/src/endpoint/responses_websocket.rs` - Ping/Pong handled inside the pump and **not** forwarded to `next()` - Unknown JSON events: parse failure or `unhandled responses event` → `continue` (timer already reset)

2. Aggregate image-history serialized-request amplification

GitHub:

github.com/openai/codex

Severe CLI reliability failure: 63.8 MB image-history requests before any compaction, WebSocket fallback, and prolonged stalls on Windows

opened 01:54PM - 05 Sep 26 UTC

FengYeeLx

bug windows-os CLI context connectivity performance

## Summary and urgent request Please urgently investigate aggregate image-histo…ry byte growth and the repeated full-request cost after WebSocket fallback, and provide a supported recovery path for affected sessions. In a real image-assisted coding task, Codex accumulated **63,759,837 bytes per request**, still **45,853,691 bytes after zstd compression**. At the corresponding history snapshot, **38 typed image occurrences accounted for 61,279,872 ASCII data-URL characters**. This happened **before any compaction**, not only through images retained in an old compacted replacement history. The failure had a serious practical impact: repeated stream errors, five-attempt retry batches, HTTPS fallback, and prolonged periods in which the primary task did not advance. One failed sequence on 0.153.2 lasted approximately **88 minutes without new effective model output**. Updating to **0.153.4**, the latest stable release when checked on September 5, did not remove the accumulated payload or prevent the observed WebSocket fallback. **I am extremely dissatisfied with this experience. These failure periods made Codex CLI effectively unusable for my paid development workflow. Waiting through prolonged retries without an actionable explanation or reliable recovery procedure is unacceptable. Users should not need to inspect SQLite databases, count Base64 payloads, or study client source code merely to keep an image-assisted coding task usable. I am asking for urgent maintainer triage, a supported mitigation, and a prioritized fix—not merely another generic reconnect warning.** The report also includes a later partial recovery: a successful compaction removed the old tool-image payloads from active replacement history and work resumed. I am not claiming permanent data loss or that every use of `view_image` fails. ## Related reports and the additional evidence here This is closely related to [#41338](https://github.com/openai/codex/issues/41338) and [#24550](https://github.com/openai/codex/issues/24550). Please consolidate this evidence into an existing tracker if that is the preferred triage path. The distinguishing measurements are: - Windows, the built-in OpenAI provider, ChatGPT authentication, and current stable 0.153.4—not a Responses-to-Chat bridge. - **Zero compaction records** at the failure snapshot: ordinary current-window image outputs can already create the large-request condition. - **37 distinct data URLs across 38 occurrences**. Exact deduplication would remove only **1,119,846 bytes, or 1.83% of the image data**. - The largest individual data URL was **5,183,170 bytes**. **None exceeded 10 MiB**. A per-image 10 MiB ceiling alone would not address this workload. - Actual pre/post-compression request measurements, separately from rollout file size and token accounting. - A subsequent compaction that can be inspected structurally: tool-output images were removed, while the original user-message images remained. ## Environment | Field | Observed value | |---|---| | Codex CLI | 0.153.2 during the original long failed sequence; upgraded to 0.153.4 before the later resumed failure | | Current stable release checked | `rust-v0.153.4`, published September 4, 2026 | | Source version audited | `3d2ee51ca2d5db578f328aa75e20aa22c0197c9a` / `rust-v0.153.4` | | Platform | Windows x64, OS build `10.0.26100` | | Shell | PowerShell 7.6; terminal-emulator version not collected | | Subscription / authentication | ChatGPT Pro; ChatGPT authentication | | Provider | Built-in OpenAI provider, Codex Responses endpoint | | Model | `gpt-6-astra` | | Reasoning selection | Initially `xhigh`; the resumed pre-compaction and post-compaction turns both used `ultra` | | Service tier | `default` | | Network | Explicit HTTP/HTTPS proxy variables pointing to a local v2rayN proxy; TUN/routing changes are disclosed below | Relevant **non-default** context settings were: ```toml model_context_window = 1000000 model_auto_compact_token_limit = 900000 ``` The session reported a `model_context_window` of **828,400**. I am disclosing the configured values without asserting that they were the effective runtime limits. This is not a claim of reproduction with an untouched default configuration. It is a supported long-context workflow that lacked an effective safeguard against its much larger serialized image payload. ### Codex doctor report A new full `codex doctor --json` report was not collected for this investigation. The evidence below comes from targeted read-only inspection of the affected rollout, retained client logs, process/network state, and the pinned client source. I am not substituting an older doctor report or claiming that doctor passed. ## What happened: measured request and history sizes The HTTP client recorded the following at **2026-09-05 11:12:14 UTC**: ```text Compressed request body with zstd pre_compression_bytes=63759837 post_compression_bytes=45853691 compression_duration_ms=78 ``` Several earlier requests in the same resumed task were almost as large: | UTC | Before compression | After compression | Compression time | |---|---:|---:|---:| | 10:55:20 | 63,721,050 B | 45,827,415 B | 70 ms | | 10:57:42 | 63,740,637 B | 45,841,192 B | 76 ms | | 10:58:22 | 63,742,371 B | 45,842,100 B | 70 ms | | 10:59:43 | 63,742,371 B | 45,842,094 B | 98 ms | | 11:12:14 | 63,759,837 B | 45,853,691 B | 78 ms | These are **application-recorded request-body sizes**, not inferred wire traffic, billing totals, or the size of the JSONL file. Compression itself was fast; the measurements do not justify blaming local zstd work for multi-minute delays. The corresponding history was inspected through **11:12:15 UTC**, counting only `response_item` records: | Typed image location | Occurrences | Data-URL characters / ASCII bytes | |---|---:|---:| | User-message `content[].image_url` | 7 | 4,307,974 | | Function/custom-tool `output[].image_url` | 31 | 56,971,898 | | Total | **38** | **61,279,872** | All counted image values belonged to typed `input_image` objects. UI/event duplicates were not included in these totals. The main sources were generated images and image-viewing tool outputs; this is not a claim that the whole total came from `view_image` alone. The last completed request recorded at **10:58:22 UTC** reported **482,479 input tokens**, of which **480,000 were cached**. That is the last completed-request usage record, **not** a new usage measurement for the 11:12 submission. It demonstrates that high token-cache reuse coexisted with repeated large requests; unchanged usage counters do not prove the backend performed no work or incurred no cost. ## Failure chronology and severity All times below are September 5, 2026, UTC. | Time / interval | Observation | |---|---| | 08:53:15–10:21:18, CLI 0.153.2 | After the last productive tool result, repeated response-stream failures produced no new effective model output for approximately 88 minutes; the turn eventually failed. | | 09:08:19, 09:23:24, 09:38:29, 09:51:07, 10:06:11 | Five recorded retries in that failed sequence. The roughly 15-minute spacing is observed behavior, not a claim about a proven timeout setting. | | 10:51–10:52 | CLI upgraded to 0.153.4 and restarted; the original session was resumed. | | 10:53 | Multiple WebSocket handshakes succeeded; actual request sends then failed with Windows error 10054 and exhausted five retries. | | 10:54:01 | HTTP fallback was activated. | | 10:55–10:58 | Some productive tool activity resumed, including starting two subagents. | | 10:59:43 | A separate manual route change restarted the proxy and caused a known disconnect. This event is treated separately, as explained below. | | 11:00:19 and 11:12:44 | HTTPS requests received HTTP 200; SSE keepalives were also observed during long waits. | | 11:07:35 and 11:08:44 | The two subagents completed; their progress must not be confused with the primary thread's progress. | | 11:31:23–11:36:38 | The running turn was interrupted, then a compaction ran and completed. | The errors included: ```text stream disconnected before completion: failed to send websocket request: IO error: ... (os error 10054) Falling back from WebSockets to HTTPS transport stream disconnected before completion: Transport error: network error: error decoding response body ``` The original 88-minute terminal failure occurred on **0.153.2**. The later handshake/send failures, fallback and large requests were observed on **0.153.4**. I am not attributing the earlier 88-minute sequence to the newer binary. ## Later recovery: important evidence, not omitted At **11:36:38**, the completed compaction installed a replacement history containing: - 36 items: 35 messages and one compaction item; - **7 embedded images totaling 4,307,974 characters**; - approximately **4,378,186 bytes** when serialized as compact UTF-8 JSON by the inspection code. The retained image multiset exactly matched the earlier user-message image multiset. The old raw tool outputs were no longer present in the replacement history. Relative to the earlier snapshot, embedded image bytes decreased by approximately **92.97%**. Subsequent retained HTTP logs showed requests around **9.8–10.9 MB before compression**, substantially below the earlier 63.8 MB, and the primary thread again performed commands and file edits. Those later requests already included additional post-compaction work, so they are not the size of the first untouched compacted request. This is a **natural before/after observation**, not a controlled benchmark. The model selection remained Astra/Ultra and the same CLI process continued, but the workload changed and upstream conditions were not controlled. Long waits were not eliminated: one later parent-thread tool-result-to-next-tool-call interval was still about **11 minutes**. The observed recovery supports treating active-history size as an important contributor. It does **not** prove that compaction alone fixes every affected thread, or that every minute of delay was upload time. It also matches an important implementation distinction: current compaction can discard original tool outputs while retaining user images. ## Reproduction conditions The observed workload was: 1. Run a long image-assisted coding session using the built-in OpenAI provider and the environment described above. 2. Generate and inspect multiple images through normal tools. Retain several dozen prepared images in active history; individual images need not exceed 10 MiB. 3. Continue model/tool iterations without a compaction having reduced that history. 4. Resume the same history after a CLI restart. 5. Compare request-body byte measurements with token accounting, and observe whether the actual resumed WebSocket request succeeds after the handshake. This describes the real failure conditions. **A standalone minimal deterministic network reproducer and a sanitized-copy A/B test have not been run.** The private workload cannot be pasted into a public issue. The report is not evidence that every image, network, model, or default configuration reproduces the transport failure. ## Technical assessment: what is proven, and what is most likely **Confirmed:** this workload retained tens of megabytes of typed image data and repeatedly constructed very large requests. The current source explains how that payload survives truncation and is resent when the full-history path is used. **Most strongly supported explanation of the amplification:** accumulated inline images become expensive repeatedly when incremental WebSocket continuation is unavailable and the client submits the full active prompt over HTTP. Additional images can enlarge subsequent submissions again. The natural compaction observation and related community controls support this explanation. **Not established:** the exact origin of the first TCP reset, a particular server/proxy message-size ceiling, or a universal causal relationship between image size and every observed stall. Payload size or multimodal replay shape may trigger a transport/endpoint limitation, but local/upstream proxy behavior and backend processing are still possible contributors. High reasoning effort also contributes to ordinary model latency. <details> <summary>Pinned source evidence and the conditional byte-amplification argument</summary> The following code was checked at release commit `3d2ee51ca2d5db578f328aa75e20aa22c0197c9a`: | Component | Relevant behavior | |---|---| | [Image-viewing handler][src-view] | Produces a data URL and a typed image output. Omitting the image from the log preview does not remove it from model input. | | [Tool-output truncation][src-trunc] | Text is budgeted/truncated while `InputImage` URLs are copied in the inspected paths. | | [History/token estimation][src-history] | Replaces the actual Base64 byte contribution with an image-token estimate. Correct token estimation still requires a separate serialized-byte safeguard. | | [Request construction and transports][src-client] | HTTP constructs a request from the formatted active prompt. WS may use incremental input when prior-response state and prefix matching permit it; otherwise it uses full input. | | [Compacted-history retention][src-compact] | Filters ordinary raw tool outputs; retained user images can remain. Thus compaction outcomes depend on the history's shape. | | [WebSocket response stream][src-ws] | The ordinary `Message::Close(_)` branch discards close details and returns a generic stream error. This does not imply that our 10054 included a close frame. | The deterministic part can be stated narrowly. Let `b_i` be the byte length of each prepared inline image URL retained in a history that is serialized in full. These strings must contribute at least `sum(b_i)` bytes to that uncompressed request, before surrounding JSON and other inputs. If the same image strings are retained across `k` such requests, they contribute at least `k * sum(b_i)` bytes across the constructed request bodies. That is a serialization consequence under stated conditions. It is **not** proof that all requests reach the network in full, that those bytes are billable tokens, or that a connection must fail at a particular size. Compression and WS incremental continuation change transfer costs; the recorded zstd sizes establish the substantial cost in this case. For this workload, a single-image ceiling and exact deduplication are insufficient on their own: no image exceeded 10 MiB, and 37 of 38 encoded images were distinct. The safeguard needs an aggregate/lifecycle dimension. </details> ## Confounders and limits explicitly checked - **Proxy route change:** at 10:59:43, changing routing restarted Xray/sing-box and disrupted both the affected session and another active session. That disconnect is not used as proof of image-induced failure. Earlier failures predated it. - **Proxy configuration was present:** explicit HTTP/HTTPS proxy settings were active and actual WebSocket handshakes succeeded. This differs from an older missing-proxy/handshake-timeout problem. - **Connection versus progress:** HTTP 200 and periodic SSE keepalives demonstrate some established stream activity, not completion of model work. I am not calling every long wait a deadlock. - **Local resource evidence:** the inspected snapshots did not show CPU/memory exhaustion; local request compression was measured in milliseconds. These snapshots do not exclude every possible local bottleneck. - **Billing:** no monetary loss, duplicate model billing, or precise quota waste is claimed from the byte counts or unchanged usage counters. - **Recovery:** a compaction later succeeded and work resumed. No permanent data loss or permanently unrecoverable session is claimed. - **Log retention:** older retained-client-log rows rotated during the investigation. Their measurements were captured while present; the persistent rollout and saved diagnostic observations were cross-checked. No unrestricted packet capture or server-side trace was obtained. ## Existing fixes and reports have been considered | Reference | Relevance and boundary | |---|---| | [#41338](https://github.com/openai/codex/issues/41338) | Directly reports the byte/token mismatch from local image viewing; open when checked. | | [#24550](https://github.com/openai/codex/issues/24550) | Reports WS retries/fallback with retained images. Its original compaction-specific reproduction differs from this pre-compaction case. | | [#30441](https://github.com/openai/codex/issues/30441) | Related image-history size failures; this local report does not claim a 413 was observed. | | [#24388 and closure response](https://github.com/openai/codex/issues/24388#issuecomment-4835137176) | Acknowledges an earlier compaction issue and explains a change in compaction implementation. This should not be read as confirmation that every image-byte retention problem is fixed. | | [Merged PR #38681](https://github.com/openai/codex/pull/38681) | Preserves parent HTTP fallback for delegated sessions. Its commit is an ancestor of 0.153.4 and the relevant code is present. I am not claiming that this specific fix is absent. | In particular, I am not requesting that maintainers redo #38681. Please assess the remaining current-window byte growth and repeated full-history submission separately from already-addressed delegated-session fallback behavior. ## Expected behavior and urgent requested action Image viewing is a core, legitimate workflow. The expected solution should preserve useful visual capability rather than requiring users to avoid images altogether. Please: 1. **Urgently triage and assign ownership** for the aggregate image-history/request-byte problem, clarify which existing tracker owns it, and provide a current supported mitigation and expected fix/release path. 2. Add an **aggregate serialized-byte safeguard**, independent of token budgeting, covering ordinary tool outputs before compaction as well as continuation, resume/fork and compacted history. Many individually small images must be covered. 3. Provide a bounded lifecycle or out-of-band representation for older image artifacts, with explicit references and on-demand retrieval. Do not silently discard task-critical current images or the user's original files. 4. Avoid repeatedly resubmitting a deterministically rejected large request unchanged. Preserve actionable close/status details when available; do not infer an oversize error solely from a generic TCP reset. 5. Provide a **supported recovery procedure for existing affected sessions**, with preservation of project files and useful task context, instead of requiring manual JSONL/SQLite surgery or abandoning a long task without a reliable handoff. 6. Make the UI distinguish connection attempts, accepted requests with heartbeats, actual model progress, and repeated request-size failure. Surface the relevant byte measurements and a useful recovery action. Suggested regression coverage should include multiple sub-10-MiB images, current-window tool images before any compaction, high token-cache reuse with large serialized bodies, both user and tool images after compaction, and repeated continuation after fallback. Please verify both the **request bytes** and successful follow-up behavior; a low token count or a successful compaction notification alone is insufficient. **This is a serious interruption to paid, real development work, not a cosmetic warning. Please provide an actionable official response as soon as possible.** ## Privacy and supporting material Only sanitized measurements and code references are included. Raw rollouts, images, prompts, project paths, credentials, cookies, account identifiers and private request IDs are intentionally omitted. Please specify an appropriate secure channel if additional private diagnostic material is necessary for triage. [src-view]: https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/core/src/tools/handlers/view_image.rs [src-trunc]: https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/utils/output-truncation/src/lib.rs [src-history]: https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/core/src/context_manager/history.rs [src-client]: https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/core/src/client.rs [src-compact]: https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/core/src/compact_remote_v2.rs [src-ws]: https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/codex-api/src/endpoint/responses_websocket.rs

A separate affected image-assisted session produced a measured request of:

63,759,837 bytes before compression

and

45,853,691 bytes after zstd compression

The active history contained 38 typed image occurrences accounting for about 61.28 million data-URL characters.

Important details:

zero compaction records existed at the measured failure snapshot;
37 of the 38 image data URLs were distinct;
exact deduplication would remove only about 1.83% of the image data;
no individual image exceeded 10 MiB;
the largest individual data URL was about 5.18 MB.

So this is not adequately addressed by simply deduplicating identical images or imposing only a 10 MiB per-image ceiling.

The concerning architectural boundary appears to be that model-context/token budgeting and actual serialized request byte growth can diverge substantially during an ordinary multimodal working window.

A later successful compaction reduced the retained embedded image bytes by about 93%, after which productive work resumed. I am not claiming this proves that every observed stall was caused solely by payload size, but it is strong evidence that active-history serialized size is an important contributor.

Related existing reports:

github.com/openai/codex

Inline image output costs ~230 tokens but 4.2 MB on the wire, so token-based context management never sees the payload that wedges the thread

opened 01:49PM - 28 Aug 26 UTC

ramuroy

bug context tool-calls app

### Related issues I searched first. This overlaps #18629 (inline base64 tool i…mages poisoning threads) and #31579 (thread stuck after a failed auto-compact), and I have 👍'd both. I am filing separately because I have a measurement that contradicts a stated hypothesis in #18629 and that neither issue reports: **the images are nearly free in tokens and enormous on the wire.** #18629 says the payload is "likely inflating token usage." Measured here, it does not — and that asymmetry looks like the actual reason nothing in the client notices before the thread dies. Happy for this to be folded into #18629 if you'd rather keep it in one place. New here: the per-image token cost, a 34.78 MB payload (32× #31579's 1.08 MB), and `view_image` on a local file as the producer rather than a browser/screenshot plugin. --- ### What version of the Codex App are you using? Codex App `26.810.52044`; `codex-cli 0.150.1` ### What subscription do you have? Not specified here. Observable from the session's own telemetry: `limit_id: "codex"`, a 10,080-minute (weekly) primary window at 81% used, `credits.balance: "0"`. ### What platform is your computer? `Linux 7.0.0-30-generic x86_64 x86_64` ### What issue are you seeing? A thread became permanently unable to continue. Three consecutive user turns each returned: ```json {"detail":"Bad Request"} ``` Immediately before that, three pre-turn auto-compactions failed: ``` 16:14:50 WARN remote compaction v2 stream failed; retrying request after delay retries=1 max_retries=2 compact_error=stream disconnected before completion: failed to send websocket request: Connection closed normally 16:14:52 WARN remote compaction v2 stream failed; retrying request after delay retries=2 max_retries=2 compact_error=stream disconnected before completion: websocket closed by server before response.completed 16:14:54 WARN falling back to HTTP 16:14:54 DEBUG Compressed request body with zstd pre_compression_bytes=34777589 post_compression_bytes=492064 16:14:55 ERROR session_task.run:run_turn: Failed to run pre-sampling compact ``` After the compaction failed, the turn proceeded **on the uncompacted history**: five websocket retries, all `closed by server before response.completed`, HTTP fallback, then `Turn error: {"detail":"Bad Request"}`. Every later message repeated it, with the body growing slightly each time as the failed turns appended (34,777,589 → 34,779,905). ### The measurement The request body was **34.78 MB**. Of that, eight `view_image` outputs were **4,200,348 bytes each**: | `response_item` type | Count | Bytes | Share | | --- | ---: | ---: | ---: | | `custom_tool_call_output` | 226 | 35,243,778 | **96.3%** | | `reasoning` | 281 | 771,319 | 2.1% | | `custom_tool_call` | 226 | 375,302 | 1.0% | | `message` | 56 | 150,834 | 0.4% | | everything else | 4 | 48,958 | 0.1% | The eight images alone are **33.6 MB, 92% of the entire request**. Now the part I think matters. Taking `last_token_usage.input_tokens` on the turn immediately after each image entered history: | Image | Bytes on the wire | Prompt-token delta | | ---: | ---: | ---: | | 1 | 4,200,348 | +239 | | 2 | 4,200,348 | +238 | | 3 | 4,200,348 | +230 | | 4 | 4,200,348 | +232 | | 5 | 4,200,348 | +227 | | 6 | 4,200,348 | +216 | | 7 | 4,200,347 | +235 | | 8 | 4,200,346 | +226 | | **total** | **33.6 MB** | **1,843** | **1,843 tokens — 0.71% of the 258,400 context window — for 92% of the request body.** For comparison, ordinary `exec` outputs in the same session cost +18,516, +11,000 and +7,240 tokens each while contributing almost nothing to the byte count. ### Why this looks like the root cause Context management is **token-denominated**; the transport constraint is **byte-denominated**. Only one of those is tracked. 1. **The client cannot see the problem.** Its own accounting said 233,226 / 258,400 — a normal 90%, the routine trigger for compaction — while the actual body was 34.78 MB. `pre_compression_bytes` is computed and logged immediately before the POST and feeds no decision. 2. **Compaction cannot fix it, even when it works.** A token-budget compactor has no incentive to evict images: they are the *cheapest* items per token and the *most expensive* per byte. It will evict the 18,516-token text output and keep the 4.2 MB picture. A textbook-perfect compaction could leave the body essentially unchanged. 3. **Compaction cannot run at all.** It is itself a model request carrying the same payload, so when the payload is what fails, the only shrink mechanism fails with it — then the turn proceeds uncompacted anyway. That converts one bad turn into a permanently dead thread. 4. **The error says nothing.** `{"detail":"Bad Request"}` gives no size, no token count, no offending item, no remedy. ### What steps can reproduce the bug? - **Session id:** `01a04764-a97d-71c2-ace0-6fcf5816cc42` - **Model:** `gpt-5.6-sol`, `reasoning_effort=max` - **Context window:** 258,400. Last successful turn: 233,226 input tokens (90.3%). - Two earlier auto-compactions in the same session **succeeded** — both before any image entered history. 1. Run a long session with ordinary tool use until it sits near the auto-compact threshold. 2. Have a tool write a large PNG to disk. Ours was 1280×820 at 3,149,928 bytes. 3. Inline it: `const r = await tools.view_image({path, detail:"original"}); image(r.image_url);` — repeat ~8 times. 4. Send another message. Pre-turn auto-compact fires, fails, and every turn from then on returns `{"detail":"Bad Request"}`. The image producer does not matter — #18629 collects browser-use, Playwright, `image_gen` and plugin cases. What matters is an unbounded artifact reaching history via base64. ### What is the expected behavior? 1. **Bound the inlined artifact.** A 1280×820 frame tiles to the same image tokens whether the file is 13 KB or 3.1 MB — the measurement above shows the extra bytes buy the model nothing. Downscale or re-encode to a byte ceiling before the item enters history. This is lossless in every sense that matters and removes ~99% of the payload on its own. 2. **Track bytes as a budget, not only tokens.** The number is already computed at `pre_compression_bytes`. Evicting by bytes-per-token would have dropped eight items worth 1,843 tokens and reclaimed 33.6 MB. 3. **Give compaction a local fallback, and never proceed uncompacted.** If the compaction request fails, elide the heaviest items client-side and retry without the network. Running the turn anyway on the uncompacted history is what makes the state permanent rather than transient. 4. **Report the real reason.** A 400 on a 34.78 MB body should say so. ### Additional information - `detail: "original"` was requested; the stored item records `detail: "high"`. Nothing downscaled or capped it either way. - **On the producer side:** the oversized PNG came from a local MCP server we maintain, which was writing PNGs as stored deflate blocks — 3,149,928 bytes for a line drawing that compresses to 16,687. We have fixed that (299× smaller, byte-identical pixel data), so this particular trigger is gone for us. It only removes one producer, though: nothing client-side stops the next large image from any source. - **We depend on these images and are not going to stop sending them.** The text alternatives our tooling exposes — a 400×200 character grid, a 32-entry colour ranking, single-pixel reads — answer "what colour is at this point", not "what does this drawing show". The right fix is bounding the payload, not rationing the capability. - **On whether this is a size limit:** I want to be careful. #31579 hit the same 400 at 1,084,457 bytes, ~32× smaller than ours, alongside a dangling `No tool output found for tool search call`, so the rejection may be structural rather than a byte cap. What I can state from this session is the payload composition, the token/byte asymmetry, and that the client had no recovery path. I could not observe the server's reason. - **Possible recovery, from #31579:** archiving and immediately unarchiving the thread moved it from `systemError` to `notLoaded`, unloading the stale in-memory history, after which the same prompt succeeded. Worth documenting if confirmed — right now a user's only visible option is to abandon the thread.

github.com/openai/codex

Responses WebSocket falls back when compacted replacement_history contains large inline images

opened 07:56AM - 26 May 26 UTC

KwakYounHo

bug CLI context connectivity

### What version of Codex CLI is running? `codex-cli 0.133.0`. The original af…fected session metadata includes `codex-cli 0.130.0`, but the WebSocket fallback was reproduced from a copied session using the current Codex CLI/App around `0.133.0`. ### What subscription do you have? ChatGPT Pro 20x. ### Which model were you using? `gpt-5.5` with `reasoning_effort=xhigh`. ### What platform is your computer? macOS arm64. ### What terminal emulator and version are you using (if applicable)? iTerm2 3.6.10. The affected thread was resumed through Codex CLI/App flows. The reproduction used copied `CODEX_HOME` directories and `codex resume` against the same session JSONL payload. No terminal multiplexer was intentionally used for the reproduction session; separate monitoring/caffeinate processes may have been run in other terminals. ### Codex doctor report Redacted summary from `codex doctor --json`: - overallStatus: `ok` - codexVersion: `0.133.0` - install method: `brew` - runtime platform: `macos-aarch64` - installation: `installation looks consistent` - config: `config loaded` - model: `gpt-5.5` - model provider: `openai` - auth: `auth is configured` - auth mode: `chatgpt` - provider reachability: `active provider endpoints are reachable over HTTP` - WebSocket reachability: `Responses WebSocket handshake succeeded` - WebSocket handshake result: `HTTP 101 Switching Protocols` - WebSocket wire API: `responses` - terminal: `iTerm2` - terminal version: `3.6.10` - terminal size: `211x98` - state paths: `state paths and databases are inspectable` - active rollout files: `110 files, 1371653678 total bytes, 12469578 average bytes` I am not pasting the raw doctor JSON because it contains local usernames, paths, project names, and state locations. ### What issue are you seeing? A long-running Codex thread consistently falls back from Responses WebSocket to HTTPS when resumed. This may share a root cause with #24388, because both involve large `input_image` payloads retained inside compacted `replacement_history`. The observed failure mode here is different: instead of remote compaction failing with `context_length_exceeded`, the Responses WebSocket stream closes before `response.completed`, retries several times, and then falls back to HTTP. I was able to reproduce the fallback using a copied `CODEX_HOME`, and then eliminate the fallback by removing only inline image payloads from compacted `replacement_history`. Observed WebSocket failure pattern: ```text stream disconnected - retrying sampling request (1/5 ...) stream disconnected - retrying sampling request (2/5 ...) stream disconnected - retrying sampling request (3/5 ...) stream disconnected - retrying sampling request (4/5 ...) stream disconnected - retrying sampling request (5/5 ...) falling back to HTTP ``` The HTTP fallback then proceeds, but the affected thread becomes prone to long idle periods. ### What steps can reproduce the bug? I cannot attach the raw session JSONL because it contains local paths, prompts, tool output, and image payloads. The structural reproduction I used was: 1. Start from a long-running Codex thread with multiple compactions and historical image inputs. 2. Confirm that compacted `replacement_history` contains retained `input_image` payloads. 3. Copy the affected session JSONL into a clean temporary `CODEX_HOME`. 4. Resume the same thread from the copied session. 5. Observe that WebSocket warmup succeeds, but the actual resumed turn payload fails over WebSocket, retries 5 times, and falls back to HTTP. 6. Create another copy of the same session JSONL. 7. Replace only `input_image` items inside `type: "compacted"` records with small text placeholders. 8. Resume the same thread from that sanitized copy. 9. Observe that `responses_websocket` completes normally, with no HTTP fallback and no `stream disconnected` retries. Control reproduction with copied session JSONL: - WebSocket warmup succeeded. - The actual resumed turn payload failed over WebSocket. - `stream disconnected` occurred 5 times. - Codex fell back to HTTP. This suggests the issue is reproducible from the session/history payload itself, not from the original live `~/.codex` state. Compaction-only sanitized reproduction: - I replaced only `input_image` items inside `type: "compacted"` records with text placeholders. - Non-compacted history was left mostly unchanged. Before sanitization: - Session JSONL size: ~703 MB - Largest compacted JSONL line: ~17.4 MB - Total `input_image` entries: 1789 - `input_image` entries inside compacted records: 1277 - `input_image` entries outside compacted records: 512 - Latest compacted `replacement_history`: ~17.39 MB - Inline image data in that replacement history: ~17.17 MB - Image data share of replacement history: ~98.75% After compaction-only sanitization: - Session JSONL size: ~195 MB - Largest compacted JSONL line: ~241 KB - `input_image` entries inside compacted records: 0 - `input_image` entries outside compacted records: 509 Result: - `responses_websocket` completed normally. - `responses_http`: 0 - `stream disconnected`: 0 - fallback: 0 - turn completed successfully. This suggests the failure is specifically tied to inline images retained in compacted `replacement_history`, not simply to any historical `input_image` item anywhere in the session JSONL. I also tested a full-image-sanitized copy, where all `input_image` items were replaced with text placeholders. That also completed normally over WebSocket. ### What is the expected behavior? Either: - compacted `replacement_history` should not retain large raw inline image payloads unnecessarily, - the Responses WebSocket transport should handle this payload shape reliably, - Codex should surface a more specific close reason, such as payload size / unsupported multimodal replay / request shape issue, or - Codex should automatically recover from this history shape without requiring manual session JSONL surgery. A session should not repeatedly fail Responses WebSocket and silently fall back to HTTP because compacted history retained large inline image payloads. ### Additional information The thread did not always fail WebSocket. It worked over WebSocket for several days before the first hard fallback. The first hard fallback appeared after a compaction where `replacement_history` had grown to ~17.38 MB and contained 42 inline images. In the latest compaction: - Messages in replacement history: 382 - Messages containing images: 18 - `input_image` count in replacement history: 42 - Image bytes inside image-containing messages: ~99.88% of those messages The images appear to be retained as historical user message content inside compacted replacement history. Privacy note: I am not attaching raw rollout/session files because they contain private prompts, tool output, local paths, and inline image data. I can provide further redacted structural data if useful.

Diagnostics already provided

I also uploaded additional diagnostics through Codex /feedback.

Uploaded thread:

01a0755c-377e-75a1-9085-6f6a34bda5cf

Diagnostic bridge:

github.com/openai/codex

Uploaded Codex diagnostic thread for existing CLI issues #43022 and #43015

opened 06:26AM - 06 Sep 26 UTC

closed 07:24AM - 06 Sep 26 UTC

FengYeeLx

bug windows-os CLI context connectivity

### What version of Codex CLI is running? codex-cli 0.153.4 ### What subscript…ion do you have? ChatGPT Pro 20x ### Which model were you using? gpt-6-astra, reasoning selection: ultra ### What platform is your computer? Microsoft Windows NT 10.0.26100.0 x64 ### What terminal emulator and version are you using (if applicable)? Windows Terminal Microsoft.WindowsTerminal 1.24.11911.0 winget ### Codex doctor report ```json > codex doctor --json { "schemaVersion": 1, "generatedAt": "1788675783s since unix epoch", "overallStatus": "warning", "codexVersion": "0.153.4", "checks": { "app_server.status": { "id": "app_server.status", "category": "app-server", "status": "ok", "summary": "background server is not running", "details": { "control socket": "C:\\Users\\FengYeeLx\\.codex\\app-server-control\\app-server-control.sock", "daemon state dir": "C:\\Users\\FengYeeLx\\.codex\\app-server-daemon", "mode": "ephemeral", "pid file": "C:\\Users\\FengYeeLx\\.codex\\app-server-daemon\\app-server.pid (missing)", "settings": "C:\\Users\\FengYeeLx\\.codex\\app-server-daemon\\settings.json (missing)", "status": "not running", "update-loop pid file": "C:\\Users\\FengYeeLx\\.codex\\app-server-daemon\\app-server-updater.pid (missing)" }, "remediation": null, "durationMs": 0 }, "auth.credentials": { "id": "auth.credentials", "category": "auth", "status": "ok", "summary": "auth is configured", "details": { "auth file": "C:\\Users\\FengYeeLx\\.codex\\auth.json", "auth storage mode": "File", "stored API key": "false", "stored ChatGPT tokens": "true", "stored agent identity": "false", "stored auth mode": "chatgpt" }, "remediation": null, "durationMs": 0 }, "config.load": { "id": "config.load", "category": "config", "status": "ok", "summary": "config loaded", "details": { "CODEX_HOME": "C:\\Users\\FengYeeLx\\.codex", "config.toml": "C:\\Users\\FengYeeLx\\.codex\\config.toml", "config.toml parse": "ok", "cwd": "C:\\Users\\FengYeeLx\\Desktop", "enabled feature flags": "<redacted>", "feature flag overrides": "apps=false, external_migration=true, prevent_idle_sleep=true", "feature flags enabled": "49", "log dir": "C:\\Users\\FengYeeLx\\.codex\\log", "mcp servers": "3", "model": "gpt-6-astra", "model provider": "openai", "sqlite home": "C:\\Users\\FengYeeLx\\.codex" }, "remediation": null, "durationMs": 0 }, "desktop.app.version": { "id": "desktop.app.version", "category": "desktop", "status": "ok", "summary": "the desktop application is installed", "details": { "log directory": "$HOME\\AppData\\Local\\Codex/Logs", "running": "false", "version": "26.901.5280.0" }, "remediation": null, "durationMs": 0 }, "desktop.app_server.handshake": { "id": "desktop.app_server.handshake", "category": "desktop", "status": "ok", "summary": "the desktop application is not running", "details": {}, "remediation": null, "durationMs": 0 }, "desktop.security.enforcement": { "id": "desktop.security.enforcement", "category": "desktop", "status": "ok", "summary": "no locally visible recent Codex security enforcement was found", "details": {}, "remediation": null, "durationMs": 0 }, "git.environment": { "id": "git.environment", "category": "git", "status": "ok", "summary": "git version 2.55.0.windows.5", "details": { "PATH git #1": "C:\\Git\\cmd\\git.exe", "PATH git entries": "1", "git build options": "git version 2.55.0.windows.5; cpu: x86_64; built from commit: 32c4f7689275d233577576630e1ac5b7eb354eb0; sizeof-long: 4; sizeof-size_t: 8; shell-path: D:/git-sdk-64-build-installers/usr/bin/sh; rust: disabled; feature: fsmonitor--daemon; gettext: enabled; libcurl: 8.21.0; OpenSSL: OpenSSL 3.5.7 9 Jun 2026; zlib: 1.3.2; SHA-1: SHA1_DC; SHA-256: SHA256_BLK; default-ref-format: files; default-hash: sha1", "git exec path": "C:/Git/mingw64/libexec/git-core", "git version": "git version 2.55.0.windows.5", "repo detected": "false", "selected git": "C:\\Git\\cmd\\git.exe" }, "remediation": null, "durationMs": 35 }, "git.worktree.dev_drive": { "id": "git.worktree.dev_drive", "category": "git", "status": "ok", "summary": "no Git worktree is active", "details": {}, "remediation": null, "durationMs": 0 }, "installation": { "id": "installation", "category": "install", "status": "ok", "summary": "installation looks consistent", "details": { "PATH codex #1": "C:\\Users\\FengYeeLx\\AppData\\Local\\fnm_multishells\\6380_1788675365695\\codex", "PATH codex #2": "C:\\Users\\FengYeeLx\\AppData\\Local\\fnm_multishells\\6380_1788675365695\\codex.cmd", "PATH codex entries": "2", "current executable": "C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\bin\\codex.exe", "install context": "npm (package C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc, bin C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\bin, resources C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\codex-resources, path C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\codex-path)", "managed by Vite+": "false", "managed by bun": "false", "managed by npm": "true", "managed by pnpm": "false", "managed package root": "C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex", "npm update target": "C:\\Users\\FengYeeLx\\AppData\\Local\\fnm_multishells\\6380_1788675365695\\node_modules\\@openai\\codex" }, "remediation": null, "durationMs": 213 }, "mcp.config": { "id": "mcp.config", "category": "mcp", "status": "ok", "summary": "MCP configuration is locally consistent", "details": { "configured servers": "3", "disabled servers": "0", "stdio servers": "3" }, "remediation": null, "durationMs": 0 }, "network.env": { "id": "network.env", "category": "network", "status": "ok", "summary": "network-related environment looks readable", "details": { "managed proxy": "not configured", "proxy env vars present": "HTTP_PROXY, HTTPS_PROXY, NO_PROXY, http_proxy, https_proxy, no_proxy", "respect system proxy": "disabled" }, "remediation": null, "durationMs": 0 }, "network.provider_reachability": { "id": "network.provider_reachability", "category": "reachability", "status": "ok", "summary": "active provider endpoints are reachable over HTTP", "details": { "ChatGPT inference URL": "https://chatgpt.com/backend-api/<redacted> reachable (HTTP 405)", "desktop assets CDN": "https://persistent.oaistatic.com/codex-app-prod/<redacted> reachable (HTTP 200)", "reachability mode": "ChatGPT auth" }, "remediation": null, "durationMs": 1791 }, "network.websocket_reachability": { "id": "network.websocket_reachability", "category": "websocket", "status": "ok", "summary": "Responses WebSocket handshake succeeded", "details": { "DNS": "2 IPv4, 0 IPv6, first IPv4", "auth mode": "chatgpt", "connect timeout": "15000 ms", "endpoint": "wss://chatgpt.com/backend-api/<redacted>", "handshake result": "HTTP 101 Switching Protocols", "model provider": "openai", "provider name": "OpenAI", "proxy env vars present": "HTTP_PROXY, HTTPS_PROXY, NO_PROXY, http_proxy, https_proxy, no_proxy", "reasoning header": "false", "server model present": "false", "supports websockets": "true", "wire API": "responses" }, "remediation": null, "durationMs": 3661 }, "runtime.provenance": { "id": "runtime.provenance", "category": "runtime", "status": "ok", "summary": "running npm on windows-x86_64", "details": { "commit": "unknown", "current executable": "C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\bin\\codex.exe", "install method": "npm (package C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc, bin C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\bin, resources C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\codex-resources, path C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\codex-path)", "platform": "windows-x86_64", "version": "0.153.4" }, "remediation": null, "durationMs": 0 }, "runtime.search": { "id": "runtime.search", "category": "search", "status": "ok", "summary": "search is OK (bundled)", "details": { "search command": "C:\\Users\\FengYeeLx\\AppData\\Roaming\\fnm\\node-versions\\v24.14.0\\installation\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\codex-path\\rg.exe", "search command readiness": "file exists", "search provider": "bundled" }, "remediation": null, "durationMs": 0 }, "sandbox.helpers": { "id": "sandbox.helpers", "category": "sandbox", "status": "ok", "summary": "sandbox configuration is readable", "details": { "approval policy": "Never", "codex-linux-sandbox helper": "none", "denied-read restrictions": "false", "execve wrapper helper": "none", "filesystem sandbox": "unrestricted", "network sandbox": "enabled", "sandbox backend": "elevated", "sandbox provisioning": "complete" }, "remediation": null, "durationMs": 1 }, "security.endpoint": { "id": "security.endpoint", "category": "security", "status": "ok", "summary": "no supported endpoint protection detected", "details": { "endpoint products": "none detected" }, "remediation": null, "durationMs": 10 }, "state.paths": { "id": "state.paths", "category": "state", "status": "ok", "summary": "state paths and databases are inspectable", "details": { "CODEX_HOME": "C:\\Users\\FengYeeLx\\.codex (dir)", "active rollout files": "622 files, 10652642126 total bytes, 17126434 average bytes", "archived rollout files": "19 files, 81508500 total bytes, 4289921 average bytes", "goals DB": "C:\\Users\\FengYeeLx\\.codex\\goals_1.sqlite (file)", "goals DB integrity": "ok", "log DB": "C:\\Users\\FengYeeLx\\.codex\\logs_2.sqlite (file)", "log DB integrity": "ok", "log dir": "C:\\Users\\FengYeeLx\\.codex\\log (dir)", "memories DB": "C:\\Users\\FengYeeLx\\.codex\\memories_1.sqlite (file)", "memories DB integrity": "ok", "queue DB": "C:\\Users\\FengYeeLx\\.codex\\queue_1.sqlite (file)", "queue DB integrity": "ok", "sqlite home": "C:\\Users\\FengYeeLx\\.codex (dir)", "state DB": "C:\\Users\\FengYeeLx\\.codex\\state_5.sqlite (file)", "state DB integrity": "ok", "thread history DB": "C:\\Users\\FengYeeLx\\.codex\\thread_history_1.sqlite (file)", "thread history DB integrity": "ok" }, "remediation": null, "durationMs": 6355 }, "state.rollout_db_parity": { "id": "state.rollout_db_parity", "category": "threads", "status": "warning", "summary": "rollout files and state DB thread inventory differ", "details": { "default model provider": "openai", "rollout DB active files": "622", "rollout DB active rows": "1541", "rollout DB archive mismatches": "0", "rollout DB archived files": "19", "rollout DB archived rows": "19", "rollout DB duplicate DB paths": "0", "rollout DB duplicate rollout thread ids": "0", "rollout DB malformed file names": "0", "rollout DB missing active rows": "0", "rollout DB missing archived rows": "0", "rollout DB model providers": "<redacted>", "rollout DB rows": "1560", "rollout DB scan cap reached": "false", "rollout DB scan errors": "0", "rollout DB sources": "subagent:thread_spawn=1420, cli=91, exec=21, vscode=21, subagent:other=7", "rollout DB stale row sample": [ "C:\\Users\\FengYeeLx\\.codex\\sessions\\2026\\04\\02\\rollout-2026-04-02T04-44-54-019d4c82-31af-73d3-8cec-4db55ef6ad4c.jsonl", "C:\\Users\\FengYeeLx\\.codex\\sessions\\2026\\04\\03\\rollout-2026-04-03T00-39-44-019d4f10-a78d-74e2-892f-5f1b7d004e54.jsonl", "C:\\Users\\FengYeeLx\\.codex\\sessions\\2026\\05\\19\\rollout-2026-05-19T20-42-49-019e4042-a50f-7d53-80f5-663b2424dff2.jsonl", "C:\\Users\\FengYeeLx\\.codex\\sessions\\2026\\05\\19\\rollout-2026-05-19T20-48-49-019e4048-21d2-76e0-ac4b-d088c784bb6a.jsonl", "C:\\Users\\FengYeeLx\\.codex\\sessions\\2026\\05\\19\\rollout-2026-05-19T16-34-54-019e3f5f-a8a3-7110-b319-1c7d416df691.jsonl" ], "rollout DB stale rows": "919" }, "issues": [ { "severity": "warning", "cause": "state DB rows point at missing or unusable rollout files", "measured": "919 stale rows", "expected": "every state DB rollout path is a file on disk", "remedy": null, "fields": [] } ], "remediation": null, "durationMs": 304 }, "system.disk": { "id": "system.disk", "category": "disk", "status": "ok", "summary": "sufficient free disk space (410.6 GiB)", "details": { "CODEX_HOME available": "410.6 GiB", "failure threshold": "1.0 GiB", "warning threshold": "5.0 GiB", "worktree available": "410.6 GiB" }, "remediation": null, "durationMs": 0 }, "system.environment": { "id": "system.environment", "category": "system", "status": "ok", "summary": "OS language zh-CN", "details": { "EDITOR": "not set", "VISUAL": "not set", "os": "Windows 10.0.26100 (Windows 11 EnterpriseS) [64-bit]", "os language": "zh-CN", "os type": "Windows", "os version": "10.0.26100" }, "remediation": null, "durationMs": 0 }, "terminal.env": { "id": "terminal.env", "category": "terminal", "status": "ok", "summary": "terminal metadata was detected", "details": { "WT_SESSION": "present", "color output": "enabled", "console input code page": "936", "console output code page": "936", "stderr console mode": "0x00000007 (VT processing: true)", "stderr is terminal": "true", "stdin is terminal": "true", "stdout console mode": "0x00000007 (VT processing: true)", "stdout is terminal": "true", "terminal": "Windows Terminal", "terminal size": "120x30" }, "remediation": null, "durationMs": 0 }, "terminal.title": { "id": "terminal.title", "category": "title", "status": "ok", "summary": "terminal title default", "details": { "terminal title activity": "true", "terminal title items": "activity, project-name", "terminal title project source": "cwd", "terminal title project value": "Desktop", "terminal title source": "default" }, "remediation": null, "durationMs": 0 }, "updates.status": { "id": "updates.status", "category": "updates", "status": "ok", "summary": "update configuration is locally consistent", "details": { "cached latest version": "0.153.4", "check for update on startup": "true", "desktop application": "OpenAI.Codex", "desktop latest build": "26.901.6511.0", "desktop update status": "available", "last checked at": "2026-09-05T10:43:22.463336400Z", "latest version": "0.153.4", "latest version status": "current version is not older", "npm update target": "C:\\Users\\FengYeeLx\\AppData\\Local\\fnm_multishells\\6380_1788675365695\\node_modules\\@openai\\codex", "update action": "npm install -g @openai/codex", "version cache": "C:\\Users\\FengYeeLx\\.codex\\version.json" }, "remediation": null, "durationMs": 3234 } } } ``` ### What issue are you seeing? This issue entry exists only to associate an internally uploaded Codex diagnostic thread with two already-filed public reliability reports. It is not intended to introduce a third independent bug report. Uploaded thread: `01a0755c-377e-75a1-9085-6f6a34bda5cf` Existing public reports: * #43022 — Severe WS stalls in 0.153.4: live Ping/Pong still trips the idle watchdog (Windows loopback-only reproduction) * #43015 — Severe CLI reliability failure: 63.8 MB image-history requests before any compaction, WebSocket fallback, and prolonged stalls on Windows #43022 documents a deterministic loopback-only reproduction using the actual installed Windows Codex CLI binary. WebSocket protocol Ping frames receive valid Pong responses, but the application-event idle watchdog still expires and abandons the WebSocket. An application-level JSON heartbeat or a longer idle threshold allows the same delayed response to complete. #43015 separately documents an image-history/request-size failure in which active history produced an approximately 63.8 MB request before compression and approximately 45.85 MB after zstd compression, followed by repeated stream failures, retries, fallback, and severe workflow stalls. The two reports describe distinct mechanisms that can produce overlapping user-visible reliability symptoms. Please associate the uploaded internal diagnostic thread and its available logs/diagnostics with #43022 and #43015 and route the relevant evidence to Codex CLI engineering. ### What steps can reproduce the bug? Uploaded thread: 01a0755c-377e-75a1-9085-6f6a34bda5cf ### What is the expected behavior? The complete sanitized reproduction is already documented in #43022. For the WebSocket failure: 1. Run the actual Codex CLI 0.153.4 binary against the localhost-only synthetic peer provided in #43022. 2. Use a 1,500 ms stream idle threshold and delay the synthetic completion for 3.2 seconds. 3. Send WebSocket protocol Ping frames during the delay. 4. Observe that Codex replies with valid Pong frames but nevertheless reports: `idle timeout waiting for websocket` 5. Observe HTTP fallback. 6. Repeat with an application-level JSON heartbeat instead of protocol Ping. 7. Observe normal WebSocket completion without the idle error. 8. Repeat with protocol Ping but a 6,000 ms idle threshold. 9. Observe normal WebSocket completion. For the separate image-history failure, see the measured production reproduction conditions in #43015. Uploaded diagnostic thread: `01a0755c-377e-75a1-9085-6f6a34bda5cf` --- For #43022, transport liveness and application-level response silence should not be represented by one undifferentiated inactivity condition. A live WebSocket with valid protocol-level Ping/Pong activity should not be reported as a dead WebSocket merely because no forwarded Responses application event arrived during a legitimate long reasoning interval. Genuine transport failures must still be detected within a bounded time. For #43015, Codex should prevent unbounded aggregate serialized image-history growth, account for serialized request bytes in addition to model-token estimates, and provide a supported recovery path before an affected session becomes severely degraded. The uploaded internal diagnostics should be associated with the existing public reports rather than treated as evidence for an unrelated new issue. ### Additional information Uploaded through Codex /feedback: Thread: 01a0755c-377e-75a1-9085-6f6a34bda5cf Existing public issues: #43022 #43015 Closely related existing trackers: #39771 #41338 #24550 The public reports intentionally omit credentials, private repository contents, raw private prompts, private images, and other sensitive session material. The uploaded feedback bundle is intended to provide OpenAI with additional internal diagnostic context that cannot appropriately be placed in the public GitHub issue.

A current codex doctor --json run reports, among other things:

Codex CLI 0.153.4;
Windows x64;
ChatGPT authentication;
active provider HTTP reachability succeeding;
Responses WebSocket handshake succeeding with HTTP 101.

That does not prove every production connection is healthy, but it makes a simple “endpoint is unreachable” explanation insufficient for the controlled failure above.

What I am asking for

I would sincerely appreciate help from anyone who can route these reports to the relevant Codex engineering owners.

For #43022, I am asking for triage by the owner of the Responses WebSocket transport / retry / idle-watchdog path.

For #43015, I am asking for triage by the owner of image preparation, conversation history, context management, and request serialization.

I am especially looking for:

confirmation that the issues have reached the appropriate engineering team;
a supported short-term mitigation for affected users;
clarification on whether fixes are already being developed;
identification of the release that will contain a correction, if known.

These issues are currently making some long-running Codex development workflows extremely difficult to use reliably.

I understand that the repository has a very large issue backlog and that today may not be a normal engineering workday. I am not asking for special treatment without evidence. I am asking for urgent triage because substantial reproduction work has already been done, the user impact is severe, and the reports now contain enough technical detail for an engineer to begin investigating without first having to reconstruct the problem from scratch.

If anyone from OpenAI or the Codex team sees this, I would be very grateful if you could help route #43022 and #43015 to the appropriate maintainers.

Thank you.

fengyeelx · September 7, 2026, 2:02pm

Please, please help me. I really don’t know what to do now, and I’ve already tried everything I could, but I still can’t solve this problem on my own. Your help is extremely important to me. Even if you can only offer some advice, information, or whatever support you’re able to provide, I would be immensely grateful. If you need to know more about the situation, I’m willing to explain everything honestly. I sincerely ask you not to ignore my request. Please.

fengyeelx · September 8, 2026, 3:48am

Follow-up — September 8, 2026

Hello Codex team — apologies for one brief follow-up, and thank you to anyone who can help with routing.

I have not yet seen a public maintainer response on the two GitHub reports. I appreciate that they were filed recently, and I am not asking anyone to commit to an immediate fix or release date. The documented failures have substantially disrupted my development work, so even a brief indication of the appropriate team or a supported mitigation would be very helpful.

The two canonical reports are:

#43022 — a controlled localhost reproduction using the installed Windows 0.153.4 binary, showing the distinction between ongoing protocol Ping/Pong activity and the application-message idle timeout.
#43015— measured aggregate image-history growth producing a 63.8 MB request before compaction, with the environment, recovery observations, and causality limits documented.

Additional diagnostics were submitted through Codex /feedback, with the reference recorded in #43168:
01a0755c-377e-75a1-9085-6f6a34bda5cf

Could someone from the relevant team please help route these reports, or point me to an existing tracker or supported recovery procedure?

I am keeping the technical evidence in the two GitHub issues so that this community thread does not become a separate investigation.

Thank you very much for your time and assistance.

Topic		Replies	Views
OpenAI network infrastructure and or bad network connection? Bugs	1	175	September 10, 2026
Custom MCP connector: web client drops the stream mid-turn ("Connection interrupted"), works fine on mobile — same server, same connector ChatGPT Apps SDK apps-sdk	8	1359	August 18, 2026
Long-running unit tests make ChatGPT unrecoverable Bugs	29	614	September 25, 2026
Codex can't use with problem "Selected model is at capacity" Codex codex	33	3709	October 12, 2026
Infinite wait for OpenAI server response with GPT5 on /completions under specific conditions API gpt5	13	1651	January 12, 2026

Urgent Codex CLI maintainer triage requested: two reproducible workflow-blocking reliability failures in 0.153.4

1. WebSocket false-dead-stream / idle watchdog failure

2. Aggregate image-history serialized-request amplification

Diagnostics already provided

What I am asking for

Related topics