Ora, a 16-person startup measuring web agent-readiness for enterprise customers, published benchmark results comparing six agent harnesses—Claude Code, ChatGPT, Gemini, Hermes, OpenClaw, and Vercel's eve—against live production sites. The metrics are operational: cost per journey, latency, step count, and native task-completion rate, captured across hundreds of real customer workflows.
The methodology enforces a critical constraint. No two harnesses expose steps identically, so Ora runs a separate runtime for each and traces every step. Ido Finder, engineering lead, explains: without per-step traces, a benchmark score is hollow. When an agent fails in a signup flow, customers see exactly which step broke and what the agent attempted. That trace is the product.
Eve vs. Claude Code produced concrete differentials. Running both harnesses on Claude Fable 5 and Haiku 4.5 against identical integration tasks on customer sites, Ora measured: eve completed tasks in 7% fewer steps, achieved 2x native success (finishing on the customer site vs. web search fallback), and found 9% more valid endpoints. One run caught a prompt-caching bug in eve. Vercel shipped a fix within the testing window. The next round showed roughly 15% lower total cost.
| Metric | Eve | Claude Code | Delta |
|---|---|---|---|
| Step count per journey | Fewer | Baseline | −7% |
| Native task-completion rate | Higher | Baseline | ×2 (2× more completions on live site vs. web-search fallback) |
| Valid endpoints found | More | Baseline | +9% |
| Total run cost (post-bug-fix) | Lower | Baseline | ~−15% |
This feedback loop matters to architects. Ora partners with Vercel, giving the eve team direct access to Ora's trace data. The bug surfaced not from unit tests but from cross-harness comparison runs on production-representative journeys. The cost reduction followed directly.
After publishing results, Ora decided to build its own agent layer on eve—a significant signal from a team whose business is running every other harness side by side. Technical reason: eve's sandbox override. Eve ships with its own isolated execution environment, which normally runs outside Ora's instrumented tracing. The override lets Ora swap in its own environment, so eve agents get the same per-step recording as other harnesses without new infrastructure. journey.ora.ai now straddles both: eve is a harness under test and the framework the platform runs on.
The deployment pattern shows scale pressure. Sixteen engineers ship hundreds of commits daily. Coding agents handle routine infrastructure tasks—reading logs, deploying, updating environment variables. Finder estimates the time saved at a few hours a week. Consolidating front-end, back-end, and agent runtime into a single Vercel deployment lets coding agents reason about a unified infrastructure surface instead of stitching separate systems.
By Ora's measure, 99% of the web can't handle an agent that signs up, integrates, and pays. That figure drives Ora's pitch—the platform shows companies where agents fail and what to fix. Ora is now splitting into microservices on Vercel, with internal eve agents running as one more service in the graph.
Lesson for architects: harness choice is not model choice. Claude Fable 5 ran inside both eve and Claude Code in this benchmark. The harness produced measurable differences in step count, endpoint validity, and cost. Without isolating harness from model in your evals, you don't know which lever you're pulling.