A PricewaterhouseCoopers research paper published August 6 on arXiv compares two ways to make language models call tools: native JSON versus programmatic tool calling (PTC), where models write executable Python. The benefit runs along model generation lines: all five Anthropic models and the three newest GPT generations gain from PTC; three older GPT models do not.

The study tested 14 language models on BFCL v4, Berkeley's Function Calling Leaderboard, which weights multi-step agentic tasks at 40% and multi-turn performance at 30%. In JSON mode, models emit a structured JSON object per function call. In PTC, tools are exposed as typed Python stubs; the model writes a script executed in a sandbox subprocess, returning all results in one inference turn rather than one per call.

JSON tool calling requires one inference turn per function call; PTC batches all calls into a single sandbox execution and returns results in one turn.
FIG. 02 JSON tool calling requires one inference turn per function call; PTC batches all calls into a single sandbox execution and returns results in one turn. — PwC / arXiv:2608.06370

PTC matched or exceeded JSON on 11 of 14 models. The GPT-5.6 family gained 10.6%. Under parallel fan-out, PTC matched or outperformed JSON on 13 of 14 models. The key finding: performance split runs along model generation lines, not family lines.

Model GroupModelsPTC vs JSON (Standard)PTC vs JSON (Parallel Fan-out)Notable Gain
AnthropicAll 5 modelsPTC betterPTC better—
GPT — newest 3 generations (incl. GPT-5.6)3 modelsPTC betterPTC better+10.6% (GPT-5.6 family)
GPT — older 3 generations (pre-late 2024)3 modelsPTC underperformsPTC underperforms—
All models — standard eval11 of 14PTC matched/exceeded——
All models — parallel fan-out13 of 14—PTC matched/outperformed—
FIG. 03 PTC vs JSON performance across model families tested on BFCL v4 (14 models total) — PwC / arXiv:2608.06370

Chain length matters most. At 12 or more sequential tool calls, PTC opens an 18.8% accuracy gap over JSON—driven by the extra inference turn JSON requires per link. Pipelines chaining more than a dozen calls make scaffolding choice a primary reliability variable.

JSON hits a structural wall on parallel calls. At model-specific thresholds—N=70–72 for Claude Sonnet 5—JSON silently drops entire calls. PTC maintained 100% accuracy at N=100. Agents crawling API catalogs or running simultaneous database queries hit this limit without warning.

Under adversarial context load, JSON degrades 2.3% on average; a filesystem-discovery approach drops 32%. PTC stays stable. The subprocess isolation prevents growing conversation context from corrupting function call arguments.

ScenarioJSONPTCDelta
Short chains (< 12 sequential tool calls)BaselineSimilar to JSON~0%
Long chains (≥ 12 sequential tool calls)BaselineHigher accuracy+18.8%
Parallel calls at N ≤ 70–72 (Claude Sonnet 5 threshold)100%100%0%
Parallel calls at N = 100Silent call drops100% accuracySignificant
Adversarial context load (average across models)−2.3% degradationStable+2.3%
Adversarial context — filesystem-discovery approach−32% degradationStable+32%
FIG. 04 JSON vs PTC accuracy across key agentic scenarios (BFCL v4 benchmark) — PwC / arXiv:2608.06370

PTC has one genuine cost: it requires code-capable models and a sandboxed runtime, demanding dependency management and permission scoping. JSON tool calling needs neither. For models released before late 2024, those infrastructure costs buy no performance gain; PTC underperforms on older GPT generations, suggesting a capability threshold.

If your agent stack runs a current-generation model and chains 12 or more steps, or approaches model-specific parallel-call limits, switch to programmatic tool calling—given a secure subprocess sandbox.