Introducing Codexometer: keep track of remaining quota, session telemetry & model benchmarks!

Codexometer is a small, retro terminal dashboard for monitoring Codex quota, local session activity, and benchmark runs. Leave it open in a second terminal window or pane to see every active usage window, its remaining capacity, and its reset time without repeatedly opening /status in the Codex session where you are working.

Why use it?

/status is useful, but it lives inside the session where you are working. Codexometer turns the same quota information into an always-visible companion display, then adds local session monitoring and a benchmark runner around it.

It is useful when you want to:

  • keep quota and reset timing visible without interrupting your current task;
  • see whether local Codex sessions are active, waiting for input, or awaiting approval;
  • understand locally observed token activity and API-equivalent cost;
  • compare available models and reasoning levels on the same checked tasks; or
  • inspect how a benchmark was solved rather than seeing only a final score.

Everything runs locally in a terminal, with mouse controls, keyboard shortcuts, responsive layouts, five colour themes, and support for additional rate-limit windows when Codex returns them.

What does it show?

The interface has three main tabs:

  • Quota — switch between Bars, Consumption Pace, Pie, and Fuel Tank presentations. A pace-aware health signal distinguishes quota that is safely tracking the reset cycle from quota burning too quickly. Codexometer can also learn an observed standard API-equivalent estimate for primary Codex windows, with conservative confidence and visible pricing provenance.
  • Monitor — record locally observed token activity by root Codex session, with scrolling graphs and compact call, activity, output, and time-to-first-token telemetry. It estimates each session’s local share of observed account activity, highlights sessions needing input or approval when Codex exposes that state, and folds explicitly linked spawned agents into their parent session.
  • Benchmark — run programmatically checked challenges across selected model and reasoning-level combinations, then compare outcomes, wall time, tokens, estimated API-equivalent cost, and rankings.

Benchmarking

The Benchmark tab is a scoped runner with live, inspectable run details.

The always-available benchmark catalogue is divided into:

  • Codexometer Core — the original easy and moderate deterministic coding tasks; and
  • Codexometer Extended — the later, harder deterministic tasks.

Open Scope to select individual tasks, models, and reasoning levels. Compatibility cues show which efforts each model supports, while bulk controls make it easy to check or clear a complete group. Run Scope executes the selected intersections; Run All runs the active suite’s complete catalogue across every compatible model and reasoning level.

Each trial appears in the result matrix as soon as it starts and remains clickable while in progress. The detail view includes:

  • the benchmark prompt and safe context supplied to the model;
  • model responses and verifier results;
  • live progress, elapsed time, tokens, and API-equivalent cost;
  • tool requests and responses where the benchmark uses them;
  • move and state transitions for interactive games; and
  • a complete copy-to-clipboard action for sharing or further analysis.

You can enter and leave a live detail view without interrupting the run, navigate long transcripts with the keyboard, and stop an active suite while retaining completed work and the captured incomplete result.

Deterministic suites can be ranked with cost-, speed-, or balanced weighting. External randomized suites remain deliberately unranked.

Optional DigBench integration

Codexometer also includes an experimental integration with DigBench, or “Discovery in Games”. DigBench is a scientific-discovery benchmark containing 70 interactive games with undisclosed rules. Humans and AI agents receive the same states, available actions, and step budgets, and must discover each game’s mechanics through experimentation.

Supplying a DIGBENCH_API_TOKEN adds DigBench to the suite selector. Codexometer fetches the current game catalogue at launch rather than compiling a fixed list, so Scope can expose every game returned by DigBench alongside the model and reasoning controls.

Before a run, Codexometer shows the planned number of persisted remote sessions and asks for confirmation. During each game, the result table reports live game and level progress. Its detail view documents the solving workflow as safe prompts and tool definitions followed by each tool request, tool response, move, authoritative state, and final response. Win detection comes from the remote DigBench state rather than from interpreting the model’s prose.

Without a DigBench token, the integration remains hidden and the normal Codexometer experience is unchanged. Tokens can be created from the DigBench account page.

Privacy, authentication, and cost

Quota monitoring uses the prevailing Codex login and does not inspect ordinary conversation content. Detailed interaction capture is restricted to isolated benchmark turns created by Codexometer; ordinary Codex session messages remain excluded.

Benchmark transcripts are bounded and sanitized. Credentials, request headers, known runtime identifiers, temporary paths, terminal controls, Codex reasoning, and unrelated local session content are not retained in the detail view.

By default, benchmark model calls use the prevailing Codex login and quota. If you choose to benchmark with Sign in with ChatGPT subscription authentication, you remain responsible for the applicable terms and policies and do so at your own risk. Codexometer is a local client for user-triggered trials; it is not intended to re-serve or share one person’s subscription access.

For a clearer usage-based billing boundary, set CODEXOMETER_BENCHMARK_API_KEY to your own OpenAI API key. OPENAI_API_KEY is accepted as a fallback. The selected key is isolated in a benchmark-only Codex app-server and takes precedence for all benchmark model discovery and runs, while quota monitoring continues to use the prevailing login.

The DigBench token authorizes only the external game service. Codexometer removes it from the environment before spawning Codex and gives the model only session-scoped game tools. DigBench sessions are persisted remotely, and benchmark model calls consume either Codex subscription quota or usage-billed API tokens according to the authentication choice above.

Where does it fit?

Codexometer works particularly well in:

  • another Windows Terminal tab or split pane;
  • a second Terminal or iTerm window on macOS;
  • a tmux, Zellij, or other terminal-multiplexer pane; or
  • an Ubuntu terminal beside the Codex CLI.

It is written in Go and builds as a standalone binary for macOS, Windows, and Linux.

Install and upgrade

The easiest installation is also the easiest way to upgrade an existing Go-installed copy:

go install github.com/merefield/codexometer@latest
codexometer --version

Go installs the executable into GOBIN, or normally $HOME/go/bin. Ensure that directory is on PATH. If the version command still finds an older installation, command -v codexometer on macOS/Linux or Get-Command codexometer in PowerShell will show which executable is being run.

If you installed a manually built binary, pull the latest source and rebuild it, or replace the existing executable with a newly compiled copy.

Full installation, authentication, privacy, monitoring, and benchmarking guidance is available in the README.

GitHub: GitHub - merefield/codexometer: A terminal widget that allows you to keep track of Codex usage against your current quota · GitHub

If you enjoy it, please give the project a :star: on GitHub!

I need to token max!! :sweat_smile:

:smiling_face_with_sunglasses:

I solved the same problem directly in my Linux Codex client.

Codex Tray now shows the current weekly usage and the exact reset time directly in the UI, so there is no need to run /status or use a separate monitoring tool.

The client is written in PyQt6 and communicates directly with codex app-server over JSON/stdio — no Electron and no terminal output parsing.

Screenshot below shows the current implementation: Weekly limit: 7% (resets 08:56 Aug 18).

GitHub: https://github.com/bezoar74-hash/codex-tray-linux

Feedback is welcome, especially if anyone wants to test it on other Linux distributions.

I think “need” is only part of the point here :wink:

Codexometer is deliberately a practical toy :locomotive:

On top of that a lot of serious devs use Codex CLI, and with that there is significantly more need …

Thanks for sharing this!

Idea: Can we create a few standard tasks and compare token usage across different runs?

I’m asking because we regularly see user reports stating that token consumption has suddenly changed and rate limits are reached faster.

If we can establish a community baseline for different use cases, this should help bring some objectivity to these reports.

What do you think? Is that a good use case for your tool?

That’s a really interesting feature request. Like a usage “stop watch”? I love it!

I stand ready to help with the concept if needed. :saluting_face:

OK so task benchmarking is more challenging as we’d have to determine what constitutes as done and signal that in some way.

Nevertheless I’ve made a start on the stopwatch it self:

this is now merged as another style you can cycle to :chart_increasing:

Oh and now all the buttons are both hotkey and mouse clickable :computer_mouse:

And I’ve cleaned up the interface and added tabs (use tab to switch or click them with mouse):

which I think are a lot more logical.

That releases the S hotkey for our stopwatch.

Regarding the stopwatch feature, let’s take another look at what we want to achieve.

A) Whether the same task uses the same amount of tokens at two different points in time, ceteris paribus.

B) How much usage is deducted from the rate limits when consuming the same amount of tokens at different points in time.

Looking at it this way, I believe B is an objective measure.
A can potentially cause more confusion because of the randomness introduced by the models.

But what if Codex is having a high demand day and it gets through less tokens as service is slower?

I think it would be great if we could measure against actual task completion (which is surely what matters ultimately?)

That is quite a fun project.

Ah, you mean a stopwatch for latency, time to first token, and time to completion?

Here’s an older implementation as a reference for what to expect: https://openai-status.llm-utils.org/

Or do you mean token usage per task?

My initial concern is that there will be some randomness in the results because of the models.

But, thinking about it, many reports claim a massive difference in rate limit consumption compared to another day. If we can define a standard task with a standard harness and instructions, then we can run it often enough to determine an interval where token consumption is likely to land.

We can then answer the question of whether usage is being consumed a lot faster with some degree of confidence.

Ideally we re-use the codex harness, we just change models and reasoning levels and send the same task independently, without memory, to each combination. then we can summarise the amount of time it took, the number of tokens and the api equivalent cost. the hard part is working out a sufficiently complex task which has a very simple answer that can ideally be judged by deterministic code

Yes, this could include a standard set of config files like config.toml, but also the model catalog and maybe others.

Regarding the task’s result, couldn’t that simply be a Boolean produced by the model’s reasoning? Something along the lines of: “Create solution.txt with only 1 or 0 as the answer.”

I was also thinking about whether users could use their own reference task, which would require a project and Codex snapshot. But that’s maybe something for later.

Work in progress:

ok @VeitB that’s merged and deployed - consider it Beta for time being.

details of the benchmark tasks here:

Given there are at least three dimensions of interesting results, this is kind of begging for a 3D output! :smiley:

A fresh, ephemeral, read-only app-server thread is used for each trial, so benchmark history does not clutter normal Codex sessions.

Clever! Looking forward to try it and even more importantly looking forward to use it as an actual tool in our daily conversations here.

Found some time to dig into the benchmarks, and this looks really interesting! Hat tip for finding the test cases.

I am currently running the benchmarks, and it appears Terra sometimes struggles with writing Starlark.

Otherwise, I think this is a functional basis for confirming token usage and limit consumption.

Thanks a lot!

np, I’ve literally just merged another three, supposedly harder, benchmarks - but not run them much to examine impact.

yeah, focussing on starlark might bias some models but it has a few properties which make it sensible for this kind of thing …