LangChain benchmarked NVIDIA NeMo Switchyard—an open-source model routing library that automatically selects which model handles each step of an agent workflow—on its internal Deep Agents evaluation suite of 145 multi-turn agentic tasks averaging 6.3 model calls each. The tasks map to production workloads: customer support under policy constraints, on-call incident investigation, and multi-step workflow automation. Results showed that routing between a 30B parameter open model (Nemotron 3.5 Lightning) and Claude Opus 4.8 achieved a 74% cost reduction versus running Opus alone, while retaining 93% of its accuracy.
The routing strategy worked by sending 93% of calls to Nemotron 3.5 Lightning (which handled 10.4% of spend) and escalating only 7% to Opus (which represented 68.4% of spend). LangChain's escalation router starts every task on the cheaper model and promotes sessions to the frontier model after two consecutive negative verdicts from a small judge model. For the same completed tasks, the cost per run dropped from $11.45 (Opus alone) to $3.00 (routed) to $0.72 (Lightning alone), with accuracy declining from 86% to 80% in the routed case—a six-point tradeoff.
For architects: this benchmark validates that frontier models are not uniformly necessary across agentic workflows. Reductions in token consumption and routing logic at the orchestration layer matter more for cost control than raw model speed. Teams considering agent deployments should calculate whether the cost savings justify accuracy tradeoffs using LangChain's provided formula (cost divided by price gap), as routing viability depends on model pricing proximity.