I built Codex How To, an independent engineering-first curriculum and skill package for OpenAI Codex. I wanted to test a narrower question than “are skills useful?”:
When does a repository workflow skill improve a completed engineering task enough to justify its context and execution cost?
I ran six controlled GPT-5.6-sol tasks across two task sizes. Each task compared:
- no repository skill;
- the full
engineering-loopv0.2.0; and - the current lean v0.4.0 skill.
Quality came first: every variant had to pass the same acceptance checks without a human code correction before token or time differences were interpreted.
| Task | Control | Full v0.2.0 | Lean v0.4.0 | Result |
|---|---|---|---|---|
| Small backend boundary fix | 390,144 tokens | 418,029 | 401,602 | Control cheapest; all passed |
| Medium dependency-free 2048 build | 828,446 tokens | 553,179 | 380,767 | Lean used 54.0% fewer than control; all passed |
The result reversed with task size. My current hypothesis is that concise lifecycle guidance may be redundant for a bounded, strongly specified fix but can reduce repeated exploration when implementation, testing, review, and evidence handoff span several surfaces.
That is a boundary to test, not a universal productivity claim. There were only two tasks, global personal skills remained visible to every run, run order was fixed, and live browser behavior was unavailable in the managed sandbox.
The source, fixtures, exact measurements, replication protocol, and interactive explorer are in the public GitHub repository Phelan164/codex-howto. The repository homepage opens the explorer.
I would especially value independent results that disagree—frontend, infrastructure, ambiguous debugging, or security-sensitive work. What passed, where did the skill help, and where did it add ceremony?