EEVEE Surpasses Self-Improving Agents with 48% Margin on Multi-Domain Inference

EEVEE, a test-time prompt-learning framework developed by Princeton and Shanghai Jiao Tong University, has demonstrated a +41.53 cumulative retention gain on sequential multi-benchmark streams, outperforming previous self-improving methods GEPA and ACE, which saw declines of -15.36 and -18.58 respectively due to cross-dataset interference. The framework addresses the production scenario where an inference endpoint receives heterogeneous queries from multiple domains and task formats in a single traffic stream, a condition often overlooked by existing prompt optimizers.

The framework's architecture is based on a router that partitions incoming inputs into task clusters and assigns each cluster to a specific prompt configuration. Updates to both the router and prompts are interleaved across three stages: initializing useful prompt slots, efficiently exploring coupled router-prompt changes, and converging under a stable routing policy. Both the router and prompts continue to refine as new task distributions emerge.

The authors evaluated EEVEE on frozen-weight Qwen3-4B-Instruct and DeepSeek-V3.2, with no fine-tuning, GRPO, or RL involved. The benchmark suite included GPQA Diamond, Formula, TheoremQA, and HumanEval—tasks covering reasoning, mathematics, and code to represent heterogeneous production demand. The paper does not disclose the serving stack, inference framework, hardware topology, or orchestration layer, implying that architects may need to budget for an additional forward pass or lightweight classifier in front of the main model call.

EEVEE improved average multi-benchmark scores by 10.38 points over Qwen3-4B-Instruct and 24.32 points over DeepSeek-V3.2, surpassing GEPA by up to 37.2% and ACE by up to 48.2% when benchmarks are evaluated together. The router's ability to isolate prompt mutations to specific task clusters also avoids the prompt expansion overhead that affects ACE, which accumulates incremental delta updates into ever-longer contexts, increasing token cost and latency.

However, the paper does not provide any measurement of the routing step's serving tax, such as p50 or p99 latency, token throughput, GPU-hour budget, or per-call economics. All performance data comes from static benchmark splits, not live systems under organic distribution shift, meaning the retention curves are laboratory proofs rather than deployed-system guarantees. The three-stage training pipeline also imposes a non-trivial integration cost, as onboarding EEVEE to a new task stream requires running the exploration phase before the router stabilizes. The authors do not quantify degradation when the router misclassifies a query into the wrong prompt slot, a critical failure mode for agents exposed to adversarial or out-of-distribution user prompts. Additionally, the router is learned, not engineered, introducing a meta-training loop whose convergence behavior on real-world query distributions remains unvalidated.

The transferable pattern from EEVEE is to co-evolve a lightweight router alongside domain-specific prompt slots, partitioning heterogeneous traffic before adaptation rather than forcing it through a single monolithic prompt.