Researchers at Salesforce AI Research and the University of Illinois Urbana-Champaign published a paper demonstrating that a stronger "builder" model can construct inference-time harnesses raising a weaker "target" model's accuracy from 0.49 to 0.91 on Theory-of-Mind benchmarks without modifying the target model's weights. The paper, "AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses," formalizes strong-to-weak scaffolding: capability transfer through inference-time environment design alone.
The builder model receives 5% of benchmark data and iteratively refines a harness—task routing logic, prompt templates, deterministic solvers, few-shot exemplars, verification passes, and format enforcement. The target model (GPT-5.4-mini) remains frozen throughout. The best configuration achieved 0.91 macro-average accuracy across four ToM benchmarks, up from 0.49 baseline.
Three mechanisms drive the gains: offloading unstable model reasoning into deterministic code, benchmark-specific subtype routing, and strict answer-format enforcement. Longer chain-of-thought prompting and broader sampling—the typical levers—do not account for the jump. Structure externalization does.
| Mechanism | What the harness does | What it replaces / avoids |
|---|---|---|
| Deterministic code offloading | Converts unstable model reasoning steps into executable, rule-based code | Probabilistic chain-of-thought reasoning on sub-problems solvable by logic |
| Benchmark-specific subtype routing | Detects task subtype at inference time and dispatches to a specialized handler | One-size-fits-all prompting across heterogeneous task types |
| Strict output-format enforcement | Constrains model output to a fixed schema; parses and validates before returning | Free-form generation that introduces format-related answer errors |
On BigToM, the best harnesses compiled task structure into executable rules entirely, reducing the benchmark to a deterministic procedure. The authors flag this as a shortcut. On other ToM benchmarks, gains reflect genuine cognitive-load reduction—the harness pre-decomposed problems and constrained output format. Residual errors concentrate in hardest cases: belief-recursion depth of at least two and Bayesian goal-inference subtasks.
Builder reasoning effort improves harness quality monotonically. More inference compute on the builder pays off in harness design, amortizing across every subsequent call to the cheaper target model. Platform effects are modest relative to the builder model's capability; the pattern is framework-agnostic. Weaker target models receive the largest absolute gains—inverting typical intuition. Models where fine-tuning is most expensive are precisely where harness engineering delivers most.
The deployment case is direct: a one-time engineering cost that pays at inference time. Teams running tiered architectures—frontier-class models for planning, mini-scale models for execution—now have formalized harness design principles and measured performance ceilings. The jump from 0.49 to 0.91 makes the economics clear. Code is publicly available.
Before upgrading the target model, build the harness: route by subtype, enforce output format deterministically, and offload any sub-problem solvable by code rather than model reasoning.