Reasoning Core Beats Rival Procedural Toolkits on Three Reasoning Benchmarks

Inria researchers Damien Sileo, Valentin Lacombe, and Dimitri Kachler have released Reasoning Core, a library of 50 procedural generators for verifiable reasoning problems designed for completion-supervised fine-tuning. The paper, published August 5, benchmarks the collection against Procedural Warmup, Reasoning Gym, and SynLogic across four base-model settings. In the 3B primary comparison, Reasoning Core achieves the highest mean scores on DROP, LogiQA, and ARC-Challenge.

The 50 generators span nine domains: mathematics, formal logic, planning, state tracking, graph reasoning, formal mathematics, structured data, games, causality, and code. Each generator exposes a unified Python API: `get_task("task_name").generate_example()` returns a prompt, canonical answer, and optional reasoning trace. The `score_answer` function provides a deterministic scorer, eliminating the need for an LLM judge.

The design prioritizes distributional breadth over task count. Training on a single PDDL domain like BlocksWorld does not generalize to minor variations. Reasoning Core randomizes domain parameters within each generator — PDDL planning runs over randomized object sets and operator definitions rather than fixed instances. Difficulty controls enable curriculum schedules tuned to each base model's solvability curve. The library's task authoring guide exposes a public training-and-influence API for reproducible paired baseline/treatment experiments.

The paper's key finding: semantic validity alone does not determine training utility. Problems that parse correctly can fail to improve performance if their targets are verbose or difficulty is miscalibrated. The audit procedure applied to Reasoning Core and rival collections surfaced subtle mismatches in generation, rendering, target format, and scoring.

For teams preferring pre-generated data, more than 10B tokens are available in the HuggingFace `reasoning-core/procedural-pile` dataset with train and test splits. The library integrates with Prime Intellect's Environments Hub, OpenReward, OpenEnv, reasoning-gym, and SynLogic. Installation is `uv pip install reasoning-core`. The library, datasets, and audit materials are publicly available under the MIT license.

Procedural pipelines that look correct at the task-definition level can silently introduce mismatches — a generator producing one token format while the scorer expects another, or difficulty controls clustering samples in a region with no gradient signal. Teams building custom eval harnesses should run the regression suite on each task before adding it to training, not just spot-check examples. The public influence API makes this tractable.

Reasoning Core is the current best-documented completion-supervised procedural toolkit at 3B scale, but correctness requires running the audit suite — the generators are tools, not a turnkey solution.