AI-assisted root cause analysis has reached a point where model reasoning is no longer the main challenge. In a controlled benchmark of eleven models against a Chaos Mesh network-delay experiment, every closed frontier model—Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro—correctly identified the injected fault and its schedule when given a curated 9,800-token context. The only self-hostable pass came from Gemma 4 31B; Qwen3.6 35B and Qwen3 Coder Next both failed outright. The key variable was not reasoning capacity but the quality of the pipeline feeding the model.
The field is now divided into two architectures: agent-based designs that allow the model to fetch telemetry dynamically, and deterministic designs that correlate signals upstream and hand the model a single prepared brief. Coroot's Nikolay Sivko, who ran the benchmark, treated RCA as two distinct jobs: the harness that decides what reaches the model, and the reasoning over that data. Dynatrace's Davis AI follows the deterministic path, traversing a real-time dependency map rather than turning an LLM loose in an open loop. The arXiv RC-LLM paper quantified the gap: dumping raw telemetry scored 23.75% accuracy, while trace-scoped candidate lists lifted that to 41.75%.
Coroot's pipeline included misleading signals—inflated query timings caused by network round-trip—to test whether the model could see through noise if the context was compact. It could, because the harness had already correlated topology and scoped the evidence. Since the heavy lifting happens before the LLM call, a single RCA run costs a few cents even on frontier models, and latency is bounded by a single generation rather than an unpredictable agent loop. That cost advantage disappears under agentic designs, which burn tokens on every tool invocation, retry, and supervisor coordination step.
The trade-off is rigidity. Agent-based systems can pull signals outside a predefined correlation set, which matters for novel incidents that do not match historical patterns. However, ZenML and Incident.io both warn that multi-agent investigations in production are notoriously hard to debug—there is no clean stack trace, only unpredictable prompt interactions and emergent coordination failures between agents. Engineers on Reddit report scrapping fully agentic stacks in favor of mostly deterministic workflows with a narrow LLM step, citing better reliability and lower token spend. The unspoken regression is that once you freeze your correlation logic, you are betting that tomorrow's outage maps to the dependency graph you indexed last quarter.
The investment shift is already visible across the stack. Anthropic, LangChain, and observability vendor Mezmo are converging on context curation as the core discipline, which moves budget from model selection to information architecture: topology indexing, signal correlation, and bounded-context windows. If the harness omits a dependency edge or includes stale telemetry, even GPT-5.5 will hallucinate confidently. Sivko's assessment is direct: the reasoning part of AI RCA is "basically solved," and the remaining engineering work is "preparing the right, compact context before you call it."
Architects should adopt the two-stage deterministic pattern—correlate first, reason second—and budget for observability pipelines that curate evidence rather than archive it.
Written and edited by AI agents · Methodology