Claude Code vs Codex, round 2: 3 practical tasks across multiple files, each run 3 times

Results

Times are medians of 3 runs; parentheses show the fastest~slowest values.

Input tokens largely include cached tokens reused when rereading earlier conversation steps. The two CLIs count tokens differently, so use these figures only to compare approximate scale.

Claude Code reported an 'equivalent cost at standard API rates' of approximately 0.78~1.25 USD per task. Subscription accounts do not pay this amount separately; usage counts toward the plan's limits. Codex did not report an equivalent figure, so we did not compare it.

3 tasks

Prompt used for every run

Only the sentence after 'Task:' in the prompt below changes for each task. Unlike the first round, we instructed the tools to read 'README.md and other documentation'.

This folder is a small Node.js project (ES modules, no dependencies). Read the code, README.md and any other docs, and the tests. Task: {task} Do not modify existing test files. When you are done, make sure `node --test` passes.

• T4 Fix bugs across multiple files — Fix the bugs reported in ISSUES.md. README.md describes the correct pricing rules.
• T5 Add a feature from a specification — Implement refundOrder in src/orders.js according to the Refunds section of README.md.
• T6 Concurrency bug + API migration — Fix the problems described in the Orders section of README.md (overselling, and supporting both the Promise and the callback style).

How We Measured

• Run date: 2026-10-04 (night, South Korean time), Apple M1 Pro · macOS 26.6 · Node.js 22. The same computer, versions, and settings as the first round.
• Claude Code 2.1.250 — default model Claude Opus 5 (Claude Haiku 4.5 was also called for short supporting tasks). Codex CLI 0.159.2 — model GPT-6.1 Sol, reasoning effort medium.
• Both tools ran in non-interactive mode (claude -p, codex exec) using the operator's personal subscription accounts, with file edits and node execution automatically allowed. Runs were covered by existing subscriptions at no additional cost.
• We varied the execution order for each task (Claude Code first in run 1, Codex first in run 2, Claude Code first in run 3). No heavy work, such as builds, ran on the same computer during testing.

Grading method

• Each task included 3 tests visible to the agent and hidden tests (9~12) added only during grading. We first validated the hidden tests using reference code written by the operator, then ran the concurrency reference code 10 times to check that results were consistent.
• Before grading, we restored the test folder to its original state. Test files added by the agents were not used for grading.
• Because concurrency bugs can appear intermittently, we graded every result three times at different times and recorded the lowest score.

How the code written by the two tools differed

• T4: both tools found all 4 bugs in money.js (JPY decimal places and rounding) and cart.js (the 10-item threshold and coupon cap). Source changes were similar: 5~9 lines for Claude Code and 4~5 lines for Codex. In all three runs, Codex created a separate regression test file reproducing the reported issues.
• T5: both tools followed the specification to 'recalculate the remaining items using the original pricing rules and subtract that amount'. Added source lines were also similar, at 40~47.
• T6: across all 6 runs, both tools serialized all order processing through a single queue (global lock). In runs 2·3, Codex scoped the lock to the store so multiple order services using the same store shared the queue. In run 1, it added a transaction feature to the store and documented it in the README. Claude Code made changes only within orders.js. Neither tool used separate locks for each product.

Limitations of This Test

• The project is small: approximately 200 lines across 6 files. It is closer to practical work than the first round, but much smaller than a real service repository.
• Both tools scored full marks, so the test did not reveal an accuracy difference. Results may differ with larger repositories or tasks with ambiguous requirements.
• Time taken varies with server conditions and the network. Alternating runs on the same night reduced these effects but did not eliminate them.
• Codex used reasoning effort medium, while Claude Code used default settings. We did not measure how much of each subscription's usage allowance was consumed.

Which Tool Should You Use?

The following reflects the site operator's opinion based on these test results.

• Even with practical tasks at this scale, no difference in output accuracy emerged. It makes sense to start with the tool included in a subscription you already pay for.
• If you want faster runs, Codex was again 1.3~1.9 times faster.
• Claude Code was better at keeping changes narrow, modifying only source files. Codex added regression tests on its own, but sometimes also changed the README and storage code without being asked, so its changes need review.
• For problems requiring design judgment, such as concurrency and performance, both tools chose the simplest correct solution. If requirements such as throughput matter, state them explicitly in the prompt.

ClaudeChatGPT (Codex)Agents

All 18 run records

Download task and grading files

The bundle includes the original tasks (online store code), hidden tests, reference code, execution scripts, a summary of all 18 results (JSON), and diffs for each run.