Databricks shipped Smart Routing in Beta inside Unity AI Gateway, a task-aware model router that matched Opus 5 performance on public coding benchmarks at 56% lower cost and cut per-task spend to 65% of Opus 5 on Databricks' internal codebase. The feature routes coding agent sessions to the cheapest model capable of completing the work without changing developer workflow, plugging directly into Claude Code and Codex.
The routing decision locks in at task start for the full session. Per-request routing collapses cache hit rates because consecutive turns must land on the same model and effort level to reuse cache. Session-level routing preserves cache efficiency while capturing most cost savings from cheaper models on simpler work.
| Routing Granularity | Cache Hit Rate | Cost Savings Captured | Mechanism |
|---|---|---|---|
| Per-request routing | Collapses — consecutive turns must land on the same model and effort level | High potential, unrealized | Route independently each API call |
| Session-level routing | Preserved — all turns share one model selection | Most savings captured | Lock routing decision at task start for full session |
The classifier making the routing call is intentionally cheap: a small, low-latency model that reads the task description and annotates it with semantic fields—which system part is changing, what code evidence the prompt carries (a snippet, traceback, or nothing), how localized the fix is, what failure mode appears, and what project type. From those labels it derives a task-type family and language family. No repository content, no tests, no answers—just the description and metadata. The router defaults to a medium-sized model and escalates to frontier or cheaper models based on those labels.
Smart Routing delivered 35% cost savings against Databricks' own engineering benchmark versus routing everything to Opus 5. On public benchmarks (SWE-bench style) savings reached 56% while matching Opus 5 on quality. The gap reflects real-world diversity: internal tasks skew toward complex multi-file work; public benchmarks have more single-file edits that cheaper models handle cleanly. 2026 brought 33 new coding models to market, widening the routing surface.
| Benchmark | Cost Saving vs. Opus 5 | Quality vs. Opus 5 | Characteristic Workload |
|---|---|---|---|
| Databricks internal engineering | 35% | Matched | Complex, multi-file work |
| Public (SWE-bench style) | 56% | Matched | More single-file edits |
Teams optimizing across harnesses can use Omnigent, a meta-harness pairing Smart Routing with harness selection. Omnigent runs atop the routing decision and picks the agent harness best suited to the model and task. It sits in the same Unity AI Gateway control plane that handles spend management and access policy.
Databricks acknowledges the current router leaves significant headroom. A perfect-foresight router would beat every model at a fraction of cost; the current system closes only part of that gap. Real sessions surface complexity mid-task—a scoped bug fix becoming a multi-file refactor. The router has no reassessment mechanism once underway, though the team targets mid-session re-evaluation when cache goes stale as a near-term goal. Improving the complexity-sizing model requires production feedback loops, which is why the Beta is shipping now.
If your coding agent infrastructure defaults every task to a frontier model, you are overpaying by 35–56%. The fix is a classifier that never sees your codebase—just the task description.