GravityOCR, a parameter-shared AR-block-diffusion model from Trillion Labs and Korea University, achieves 3.94× decode-only speedup on document region crops and 1.32× end-to-end page-processing speedup by separating parallel token drafting from causal verification, according to an arXiv preprint. The model commits an average of 9.7 output tokens per forward pass in SGLang serving while retaining 99.7% of the original GLM-OCR model's accuracy on OmniDocBench v1.6.
The core problem GravityOCR solves is a failure mode in direct parallel diffusion decoding: when multiple tokens are predicted simultaneously from a partially masked block, each token is predicted before the others are known, and committing them directly can introduce structural errors—omissions, repetitions, or malformed markup. The paper illustrates this with a confidence-based parallel decoder that commits "commercial" and "enter" while the intervening position remains unresolved, ultimately omitting "vehicles" from the output.
GravityOCR adapts a pretrained causal AR OCR model (GLM-OCR) into a parameter-shared architecture supporting both block-diffusion drafting and causal AR verification. The model adds only a learned embedding for a mask token; no separate drafting network or auxiliary head is required. During joint training, a single forward pass processes three response streams conditioned on the same image and prompt: a clean stream for causal next-token supervision, and two complementary corrupted streams with masking ratios t and 1−t for denoising supervision. This ensures every response token receives denoising supervision in exactly one corrupted stream while image and prompt tokens are never masked.
At inference, self-speculative decoding works in rounds. Each round starts from the last committed token, followed by 32 mask tokens. The shared decoder produces the next causal AR token and 32 parallel draft tokens in a single forward pass. A verification pass then feeds the causal token plus the draft tokens through the model under token-level causal attention, producing AR predictions. The decoder accepts the longest draft prefix matching the AR predictions, committing those tokens plus the next AR token. This design preserves exact causal AR greedy output while allowing successful drafts to advance generation by multiple tokens.
The paper reports results on OmniDocBench v1.6, a 1,651-page benchmark. GravityOCR achieves an Overall score of 95.16, compared with 95.48 for the original GLM-OCR, retaining quality within 0.32 points. In SGLang serving on a single H100, self-speculative decoding reaches 1,047 tok/s decode-only throughput and 844 tok/s end-to-end throughput including vision encoding and prompt prefill, compared with 794 tok/s and 486 tok/s for causal AR decoding—a 1.32× end-to-end page-processing speedup. The model achieves 0.730 pages/s, exceeding MinerU2.5-Pro at 0.399 pages/s and HunyuanOCR-1.5 with DFlash at 0.579 pages/s.
The paper further optimizes the jointly trained model using GRPO applied only through the causal AR path, with task-aware OCR rewards: normalized edit similarity for plain text, structure-aware TEDS combined with cell-content similarity for tables, and canonicalized LaTeX edit similarity for formulas. GRPO improves the OmniDocBench Overall score from 94.92 to 95.16 while tokens per forward remain essentially unchanged at 9.61 and 9.68, indicating that drafter-verifier agreement is preserved. The AR loss itself is critical: removing it lowers the Overall score from 95.02 to 93.64 while TPF remains similar, showing the AR objective preserves verifier accuracy with little change in drafting efficiency.
Speedup varies by content type. Table regions accept 25.0 of the 32 drafted tokens per round and achieve 3.36× speedup, compared with text regions accepting 15.0 tokens and achieving 1.54× speedup. This reflects stronger syntactic constraints in structured outputs. At higher batch sizes, the advantage narrows: self-speculative decoding leads by 1.85× at batch size one but only 1.13× at batch size 64, as batching exposes more parallel work from the AR decoder and improves GPU utilization.
The key architectural insight—parameter-shared AR-block-diffusion with causal verification—sidesteps the accuracy loss of direct parallel commitment without requiring a separate drafting network. For teams deploying vision-language models on structured output tasks, this pattern offers a deployable way to extract parallelism from grounded generation without sacrificing quality.