Together AI ran 904 rollouts—452 per model, four trials on each of 113 DeepSWE tasks—comparing GLM-5.3 and GPT-5.6 Sol on a coding benchmark designed to stress-test real software engineering across languages and domains. A two-stage cascade running GLM-5.3 first and escalating to Sol on test failure solves 85.9% of tasks at $6.61 per rollout. Sol alone solves 72.7% at $8.37. The cascade beats single-model Sol by 13.2 percentage points and costs 21% less.
On pass@1, Sol leads 72.7% to GLM-5.3's 69.0%—a 3.7-point gap within statistical noise. Pass@2 reverses the order: GLM-5.3 reaches 81.1%, Sol 81.0%. At pass@4, GLM-5.3 pulls ahead to 87.6% versus 85.8%. The cost gap is sharper. GLM-5.3 runs $3.99 per rollout, Sol $8.37—a 2.1x difference. Per $100 of spend, GLM-5.3 solves 17 tasks; Sol solves 9.
| Metric | GLM-5.3 | Sol |
|---|---|---|
| pass@1 | 69.0% | 72.7% |
| pass@2 | 81.1% | 81.0% |
| pass@4 | 87.6% | 85.8% |
| Cost per rollout | $3.99 | $8.37 |
| Tasks solved per $100 | 17 | 9 |
Latency cuts the other way. Sol completes a rollout in 19 minutes across 61 steps with 60k output tokens. GLM-5.3 takes 35 minutes, 124 steps, and 80k tokens. For synchronous, human-in-the-loop workflows the latency gap justifies Sol's cost. For batch pipelines and offline agents, the extra time is irrelevant and cost dominates.
| Metric | GLM-5.3 | Sol |
|---|---|---|
| Rollout time | 35 min | 19 min |
| Steps per rollout | 124 | 61 |
| Output tokens | ~80 k | ~60 k |
| Best fit | Batch / offline agents | Synchronous / human-in-the-loop |
Failure modes show where the routing decision becomes operational. Sol breaks the repository's existing test suite in 20% of failures—the regression signature across the GPT family. GLM-5.3 breaks it in 11%. When GLM-5.3 fails, it misses forward: a 61% near-miss rate versus Sol's 54%. Any pipeline accepting Sol's diffs needs a full regression gate. GLM-5.3 needs that gate less.
The models split cleanly by domain and language. Sol wins data modeling and serialization (92%), build and ops tooling (73%), concurrency (72%), and protocol conformance (59%). GLM-5.3 wins query and config languages (88%), language internals (83%), stateful reactivity (73%), and JavaScript (90% versus 75%) and Rust (70% versus 60%). Sol takes Python (74% to 66%), Go (79% to 76%), and TypeScript (66% to 61%). GLM-5.3's clear hole is protocol conformance at 44%, 15 points below Sol.
| Domain / Language | GLM-5.3 | Sol | Winner |
|---|---|---|---|
| JavaScript | 90% | 75% | GLM-5.3 |
| Rust | 70% | 60% | GLM-5.3 |
| Query & config languages | 88% | — | GLM-5.3 |
| Language internals | 83% | — | GLM-5.3 |
| Stateful reactivity | 73% | — | GLM-5.3 |
| Python | 66% | 74% | Sol |
| Go | 76% | 79% | Sol |
| TypeScript | 61% | 66% | Sol |
| Data modeling & serialization | — | 92% | Sol |
| Build & ops tooling | — | 73% | Sol |
| Concurrency | — | 72% | Sol |
| Protocol conformance | 44% | 59% | Sol |
The cascade works because the models disagree substantially. Per-task correlation is 0.43. Both solve 90 tasks. GLM-5.3 exclusively solves 9 that Sol never lands; Sol exclusively solves 7 that GLM-5.3 misses; 7 defeat both. Their union covers 106 of 113 tasks (93.8%). Low correlation means escalation routes to Sol only where GLM-5.3 genuinely fails, not duplicating work.
The architect's decision tree is straightforward. Default to GLM-5.3 for batch workloads and JavaScript/Rust-heavy repositories. Route to Sol when latency is user-visible, when the task is protocol conformance or systems infrastructure, or when working in Python, Go, or TypeScript at scale. Run the cascade if your workload spans domains with no strong prior on hard tasks. Put a regression suite around Sol's diffs regardless; skip it for GLM-5.3 and you will be mostly right.
The open-weight tier has closed to within 3.7 pass@1 points of the frontier leader on a hard coding benchmark, at 2.1x lower cost, with better accuracy-per-dollar in multi-attempt settings. The single-model question is less interesting than six months ago. The routing question is where efficiency gains live now.