A new paper from Princeton and partner institutions argues that the right way to stress-test a reasoning model is not harder math problems—it is forcing the model to switch skills mid-chain. The work, published 5 August 2026 on arXiv, introduces Skill Entropy, a formal metric for benchmarking long-horizon reasoning, and Skill²-Bench, a benchmark covering 558 skills across 9 domains. Tests of 8 frontier and 4 open-source models exposed the skill-switching gap: accuracy drops as task-level entropy rises.

Skill²-Bench builds on cross-skill long-horizon tasks—multi-step problems where each step requires different reasoning and depends on prior outputs. An example: derive a mathematical result, then use it in a scheduling problem. Existing benchmarks evaluate math, code, and planning separately. Skill²-Bench forces sequential dependency. Each task receives a scalar skill-entropy score from reference-model calibration: entropy is high when a reference model handles two skills well in isolation but fails when they are combined. The benchmark groups tasks into three tiers by entropy score.

Skill²-Bench task structure: cross-skill tasks are calibrated via a reference model and grouped into three entropy tiers.
FIG. 02 Skill²-Bench task structure: cross-skill tasks are calibrated via a reference model and grouped into three entropy tiers. — arxiv.org/abs/2608.05139v1

The 9 domains span math, science, coding, logic, information extraction, planning, creative writing, context retrieval, and instruction following. This breadth is deliberate. The authors want entropy to surface wherever agents chain heterogeneous steps, not just in math-to-code transitions. The benchmark is available on Hugging Face at Gen-Verse/Skill2-Bench with evaluation scripts for vLLM-served open-weight checkpoints and API models.

Benchmark FeatureValue
Total skills covered558
Domains9
Entropy score tiers3
Frontier models evaluated8
Open-source models evaluated4
Benchmark locationHugging Face — Gen-Verse/Skill2-Bench
FIG. 03 Skill²-Bench at a glance: key benchmark statistics — Princeton et al., arXiv 2608.05139v1

On the training side, the paper converts skill entropy from a measurement into a reward signal. Skill-Entropy RL uses GRPO with skill-annotated rollouts: at each step, the model predicts both the answer and the skill used to produce it. Reward combines step-level correctness with a skill-entropy signal measuring alignment between predicted and gold skill sequences. The model receives a training signal for skill transitions, not only final correctness.

Skill-Entropy RL training loop: GRPO receives a combined reward from step-level correctness and skill-transition alignment.
FIG. 04 Skill-Entropy RL training loop: GRPO receives a combined reward from step-level correctness and skill-transition alignment. — arxiv.org/abs/2608.05139v1
Skill-Entropy RL training pipeline: four sequential stages from calibration to a trained model
FIG. 05 Skill-Entropy RL training pipeline: four sequential stages from calibration to a trained model — Princeton et al., arXiv 2608.05139v1; github.com/Gen-Verse/Skill-Entropy-RL

The gains are substantial. On Qwen3-4B-Instruct, Skill-Entropy RL lifts the Skill²-Bench score from 34.4% to 68.4%—a 34 point gain. On Qwen3-1.7B, improvement runs from 14.6% to 40.1%, nearly tripling baseline. Both outperform reported baselines. The training signal transfers: applying it to OpenR1-Math data produces comparable improvements, showing skill entropy is a reusable objective, not benchmark-specific scaffolding.

Skill²-Bench accuracy before and after Skill-Entropy RL training: Qwen3-4B-Instruct improves from 34.4% to 68.4%; Qwen3-1.7B from 14.6% to 40.1%.
FIG. 06 Skill²-Bench accuracy before and after Skill-Entropy RL training: Qwen3-4B-Instruct improves from 34.4% to 68.4%; Qwen3-1.7B from 14.6% to 40.1%. — arxiv.org/abs/2608.05139v1
ModelBaseline ScoreSkill-Entropy RL ScoreAbsolute Gain
Qwen3-4B-Instruct34.4%68.4%+34.0 pp
Qwen3-1.7B14.6%40.1%+25.5 pp
FIG. 07 Model-by-model Skill²-Bench results: baseline, Skill-Entropy RL, and absolute gain — Princeton et al., arXiv 2608.05139v1
Skill²-Bench scores: baseline vs Skill-Entropy RL for Qwen3-4B-Instruct and Qwen3-1.7B
FIG. 08 Skill²-Bench scores: baseline vs Skill-Entropy RL for Qwen3-4B-Instruct and Qwen3-1.7B — Princeton et al., arXiv 2608.05139v1

The engineering cost is significant. The full pipeline requires Python 3.10+, CUDA 12.x, and PyTorch 2.8. The sequence—entropy calibration via reference-model API gateway, task generation, SFT warm-up, GRPO training—spans multiple days on a GPU cluster. Entropy calibration alone requires API calls for labeling and deduplication before training begins. Teams without access to GPU clusters or Slurm will not reproduce this from the README in an afternoon.

The paper does not yet address skill entropy on tool-use and multi-agent chains, where skill boundaries are often function calls rather than reasoning steps. The benchmark domains fit within a single model's context window. Production pipelines that route subproblems to specialized agents or external APIs introduce system-level entropy, not token-level entropy. Whether the metric generalizes is open.

The practitioner takeaway: if you are selecting a reasoning model for an agentic pipeline chaining math, retrieval, and planning, MATH or HumanEval scores tell you nothing about cross-skill degradation. Skill²-Bench provides measurement; Skill-Entropy RL provides a training recipe to close the gap on small models for constrained hardware.