Harness published a two-part engineering guide on practices it adopted after six months redesigning its SDLC around AI agents. The goal is operational control. Harness sells CI/CD infrastructure; its customers measure it on reliability, not iteration pace.
The four pillars map to where agentic systems fail in practice. Pillar one: spec-based development. All product, tech, UI, and test specs live in version-controlled repositories (Harness uses its own Code Repository). AI agents need structured, co-located context. Confluence pages don't work. Result: teams prototype UI mockups in 30 minutes instead of weeks because specs are machine-readable and always current.
Pillar two quantifies where agents belong. Harness measured 15–20% of engineering work as fully autonomous—Jira ticket to pull request—while 80% stays AI-assisted, with developers using agents as tools rather than delegating entire workflows. Four in five tasks still require human-in-the-loop oversight. The architecture combines traditional microservices with agent endpoints exposed via Model Context Protocol. Each agent has bounded permissions and a single responsibility: a code review agent cannot implement changes, a testing agent cannot touch production, a specification agent has no authority over architecture. Specialized agents outperform omniscient overloaded ones on cost and reliability.
Testing reveals the largest gap. The guide specifies six layers: front-end verification (Vitest, Playwright), product and API testing (unit, integration, functional), agent efficacy testing against baselines, data verification to catch hallucination sources, continuous efficacy monitoring in production, and agentic feedback loops. Layer five—continuous efficacy monitoring—is most skipped and most painful. Change a system prompt, update a knowledge base, or swap a model version: agent output quality degrades silently without a monitoring harness watching it.
| Layer | Focus Area | Tools / Method |
|---|---|---|
| 1 | Front-end verification | Vitest, Playwright |
| 2 | Product & API testing | Unit, integration, functional tests |
| 3 | Agent efficacy testing | Baseline comparison evaluations |
| 4 | Data verification | Hallucination source detection |
| 5 | Continuous efficacy monitoring (most skipped) | Production monitoring harness |
| 6 | Agentic feedback loops | Automated agent-output feedback |
Harness adopted Operational Readiness Reviews from the hyperscaler playbook. ORRs run immediately after design review, not development, and produce a checklist with documented evidence. The guide frames this evidence—load test results, game day results, security audit sign-offs—as a template teams should follow, not Harness figures. Severity tiers are enforced: high items block launch, medium items require resolution within 90 days, low items go to backlog. Post-launch, service teams meet weekly to review data-plane and control-plane health, customer metrics, and RCAs.
| Severity | Enforcement Action |
|---|---|
| High | Blocks launch — must be resolved before ship |
| Medium | Must be resolved within 90 days post-launch |
| Low | Logged to backlog, no hard deadline |
Forrester found that coding-only AI improvements of 30–40% often translate to less than 10% whole-team productivity gain when planning, testing, and release pipelines stay manual. Bottlenecks shift rather than disappear.
Architects expanding agent autonomy should instrument efficacy monitoring on day one, enforce agent scope at the permission layer not the prompt layer, and treat ORRs as a forcing function to surface operational concerns before code ships.