NVIDIA Releases Nemotron Lightning and Switchyard for Agent Routing

NVIDIA is releasing Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model for high-volume, specialized-task execution in multi-agent systems, and NeMo Switchyard, an open-source routing library that dispatches requests across model ensembles. Both launch immediately: Lightning is available on Hugging Face, ModelScope, OpenRouter, and build.nvidia.com as an NVIDIA NIM microservice. Switchyard is open source.

Lightning sits below frontier planners like Nemotron Ultra or GPT-5.6 — handling code review, tool invocation, security-alert triage, and billing queries rather than high-level orchestration. NVIDIA claims 4x faster output speed and 30% faster agentic task completion versus peer models, per PinchBench benchmarks. The MoE architecture keeps active parameters low while maintaining 30B total capacity. Post-training integrates with NVIDIA NeMo for domain customization. NVIDIA is also releasing Nemotron-RL-Agentic-Terminal-Pivot, the reinforcement-learning dataset used to tune Lightning for coding-agent workloads, published alongside the weights for traceability.

NeMo Switchyard solves the routing tax. In single-model deployments, every call hits the same endpoint regardless of task complexity. In manually-routed stacks, engineers maintain per-pipeline dispatch logic that breaks when models change. Switchyard plugs into existing agent frameworks and routes prompts automatically based on configurable priorities: quality, latency, or cost. NVIDIA's internal benchmark puts system-level task-completion cost at roughly one-third of Opus 4.8 run alone at frontier quality, though the comparison mixes routing-eligible tasks with simpler calls, so savings depend on traffic distribution.

Early partner results show production impact. Boomi reported 100% domain-routing accuracy across five capabilities, redirecting 59% of traffic to a fine-tuned model that runs 5x faster than baseline, cutting later-turn latency by 21%. Ramp ran Switchyard on its internal SWE-Bench variant and matched frontier-model performance while reducing costs by 58% and runtime by 33%. Cadence saw 9.9% efficiency gains on formal-verification workloads using the ChipStack AI Super Agent. These runs span integration tooling, software engineering, and EDA — limiting typical single-benchmark skepticism.

Lightning's deployment story is broad by design. Local targets include NVIDIA RTX PCs, DGX Spark, DGX Station, and Jetson. Enterprise deployments cover RTX PRO workstations, on-premises data centers, and cloud. CrowdStrike fine-tuned it for cybersecurity, Harvey with Trajectory for legal services, CodeRabbit with Baseten for code review, Lila Sciences for life-sciences reasoning, and Fastino Labs for software development, finance, and healthcare — all reporting domain-accuracy gains after fine-tuning on proprietary datasets via NeMo.

Switchyard's operational cost is real. It requires a model catalog and routing rules. Plugging it into LangGraph or CrewAI without rewriting application logic is the goal, but routing quality degrades if the catalog is stale or domain boundaries are poorly specified. NVIDIA's 100% routing accuracy from Boomi applies to a five-capability taxonomy; production agent graphs typically have messier task decompositions. Cadence's 9.9% efficiency gain on formal verification is modest versus Ramp's 58% cost cut — the spread suggests savings are workload-dependent.

For architects: Lightning is a credible self-hosted sub-agent for high-volume, latency-sensitive tasks where per-token API costs are the constraint. Switchyard's value grows with catalog depth — more models and fine-tuned variants enable better optimization. Open weights and published training data make both defensible for regulated-industry deployments. Start narrow: one Lightning NIM behind one Switchyard route for a single high-volume task, measure latency and cost delta, then expand.