Together AI ran DeepSeek-V4 Flash 0731 and GPT-5.6 Luna head-to-head on DeepSWE — 113 real, long-horizon coding tasks from live open-source repos, four trials per task, graded pass/fail by hidden test suite. Luna is the better engineer. DeepSeek is cheap enough that routing some work to DeepSeek first and escalating to Luna on failure beats Luna alone on both accuracy and cost.
Luna's lead on single-shot quality is clear. Pass@1: 67.2% vs 53.3%. The gap holds at every equal-attempt count: 81.6% vs 70.1% at k=2, 90.3% vs 80.5% at k=4. Luna also runs faster — 16 minutes median vs 23, 92 steps vs 148, and produces less: 70k tokens vs 104k. For raw capability, this is not a close race.
| Metric | DeepSeek-V4 Flash 0731 | GPT-5.6 Luna |
|---|---|---|
| Pass@1 | 53.3% | 67.2% |
| Pass@2 | 70.1% | 81.6% |
| Pass@4 | 80.5% | 90.3% |
| Median runtime | 23 min | 16 min |
| Median steps | 148 | 92 |
| Median output tokens | 104k | 70k |
Cost inverts the story. DeepSeek runs $0.10 per attempt; Luna $0.61 — a 6x spread. That yields 532 DeepSeek solves per $100 versus 110 for Luna. Concretely: DeepSeek's pass@2 (70.1%) already matches Luna's pass@1 (67.2%), and two DeepSeek attempts cost $0.20 — a third of a single Luna run. With a verifier to pick the winning attempt, DeepSeek matches Luna's first-try quality at one-third the cost.
| Metric | DeepSeek-V4 Flash 0731 | GPT-5.6 Luna |
|---|---|---|
| Cost per attempt | $0.10 | $0.61 |
| Solves per $100 | 532 | 110 |
| Pass@2 accuracy | 70.1% | — |
| Cost for 2 attempts | $0.20 | — |
| Pass@1 accuracy | — | 67.2% |
| Cost for 1 attempt | — | $0.61 |
The cascade is the practical answer. Run DeepSeek first; escalate to Luna on failure. Together AI measured this at 78.9% task accuracy and $0.385 per task. That beats Luna alone (67.2%) and cuts Luna-for-everything by 37%. For teams running code-agent workloads at scale, this is the arbitrage the bench reveals.
Failure modes also favor DeepSeek. When DeepSeek fails, it breaks existing tests in 9% of cases. Luna does so 15% of the time — consistent with the 15–20% regression signature Together AI sees across GPT-lineage models including Sol. Both fail mostly by near-miss (DeepSeek 69%, Luna 66%), but the pricier model is more likely to corrupt working code. The practical implication: Luna deployments need full regression gates. DeepSeek needs them less.
Domain and language breakdown reveal where each model belongs. Luna wins 7 of 8 task domains. Its edges are sharpest in program analysis (69% vs 33%), concurrency and durability (70% vs 38%), and language runtime internals (86% vs 59%) — roughly 30-point gaps in reasoning work. DeepSeek's single domain win is query and config languages (78% vs 70%): SQL builders, window functions, keyset pagination, config parsers. Structured, schema-shaped, convention-following code is where DeepSeek holds its ground. Luna wins all five programming languages. DeepSeek is respectable on Rust (55% vs 60%) and Go (62% vs 79%). JavaScript is a cliff: 35% for DeepSeek against 60% for Luna. If your agent stack touches JavaScript, DeepSeek is a false economy. If it lives in config, query, or Rust, DeepSeek closes most of the gap.
At $0.10 per task, DeepSeek-V4 Flash earns its place as the first stage of any cost-aware code-agent pipeline. Route query and config work to it by default. Use it as a cheap fan-out layer for pass@k. Escalate to Luna for reasoning-heavy or JavaScript tasks where DeepSeek's capability gap is real and measurable.