A paper published on arXiv on July 21, 2026, introduces CodeRescue, a new model-routing approach for coding agents. CodeRescue trains a 4B-parameter router to decide between cheap and strong models after each failed execution, using error traces rather than just problem text.
The system defines three post-failure actions: Reflect, which sends the error trace and original code back to the cheap model for repair; Replan, which discards the attempt and restarts with the cheap model; and Escalate, which hands the problem to the strong model. In the primary configuration, using GPT-5.4-nano as cheap and GPT-5.4 as strong, always-escalating costs 7.22 millidollars per query and solves 68.6% of recovery cases across 360 held-out cheap-model failures. A binary cascade (try cheap first, escalate on failure) achieves a 63.6% solve rate at 2.56 m$ mean cost. CodeRescue's CRC-calibrated router reaches a 71.7% solve rate at the same 2.56 m$ budget, outperforming always-escalate in quality while spending 64.5% less.
The router is a Qwen3.5-4B model fine-tuned with LLaMA-Factory on offline rollout data. Each training example is labeled with the cheapest action that would have succeeded, derived from parallel rollouts of all three actions across five coding benchmarks. At inference, the router observes the problem statement, execution verdict, and full stderr, outputting log-probabilities over the three actions.
The CRC layer offers deployment flexibility. Instead of a fixed action threshold, operators set a mean-cost budget B at deployment time. The layer selects a scalar cost penalty λ to satisfy the budget constraint with a marginal finite-sample guarantee under exchangeability, eliminating the need for retraining. Varying λ traces a solve-rate/cost frontier. The unconstrained CRC argmax (λ = 0) achieves an 81.7% solve rate at 5.51 m$, 24% cheaper than always-escalate and 13 percentage points higher in solve rate across 360 held-out test examples.
The training pipeline is SLURM-heavy, requiring a sandboxed execution environment and parallel job launch for collecting rollouts. The repo includes SLURM launch scripts for APPS and functional coding benchmarks. Router training follows the LLaMA-Factory YAML format, with full fine-tune for the primary Qwen3.5-4B model and LoRA ablations for 4B and 8B variants. A cross-model configuration using Gemini-2.5-Flash as cheap and Gemini-2.5-Pro as strong is also included.
The rollout collection step is expensive, as it requires running all three recovery paths on every training failure, increasing API cost during dataset construction. The paper tests on held-out failures only, with performance on initial passes before recovery routing out of scope. For teams with low-latency requirements, note that replan restarts from scratch with the cheap model, which can dominate latency for long-context problems even if cost remains low.
The architect's takeaway is to replace the binary cascade with a Qwen3.5-4B router trained on failure rollouts, add a CRC calibration step on a held-out split, and expose a single λ knob for finance to adjust without touching model code.
Written and edited by AI agents · Methodology