Together AI published a full developer guide for Kimi K3 on July 16, 2026, the same day Moonshot AI released the model. K3 is the largest open-weight model ever shipped at 2.8 trillion parameters and the first in the 3T-class. The guide covers reasoning effort control, streaming mechanics, vision inputs, structured output, and prefix-caching cost patterns. Together AI serves it from US infrastructure with the full 1,048,576-token context window and OpenAI-compatible endpoints.
K3 runs on Moonshot's Stable LatentMoE framework with 896 total experts and 16 activated per token. Kimi Delta Attention (KDA)—a hybrid linear-attention layer that scales to 1M context—and Attention Residuals (AttnRes) manage long-context reasoning. Quantile Balancing derives expert allocation from router-score quantiles. Per-Head Muon optimizes each attention head independently. Gated MLA enables selective attention. These mechanisms kept training stable at 2.8T.
The reasoning_effort field accepts "low", "high", or "max" (max is default). For latency-sensitive paths that don't need a thinking trace, setting reasoning={"enabled": False} disables thinking entirely and bills zero thinking tokens. Streaming delivers two separate content streams: reasoning_content deltas for the trace and content deltas for the final answer. Together handles the split at the API layer, so you don't parse it yourself.
Prefix caching is automatic on Together AI. Pricing: $3.00 per million uncached input tokens, $15.00 per million output tokens, and $0.30 per million cached input—a 90% discount. This cost ratio makes K3 viable for agentic workloads where a large system prompt or repository context is reused across many turns. K3 reasons by default; reasoning tokens count as output. An Artificial Analysis evaluation consumed 130 million output tokens versus a median of 100 million for comparable models, totaling $2,437.41.
Moonshot reports 93.5% on GPQA Diamond, the best open-weight score on that benchmark. On the Frontend Code Arena leaderboard, K3 ranked #1 with 1,679 Elo, ahead of Claude Fable 5 (1,631) and GPT-5.6 Sol (1,618). Artificial Analysis's Intelligence Index v4.1 places K3 at 57—#4 overall, ahead of Claude Opus 4.8 (55.69) but behind Fable 5 (59.86) and GPT-5.6 Sol max (58.89). K3 delivers 34.9 output tokens per second on Kimi's own API against an open-weight median of 61.6 t/s, with a TTFT of 3.94 seconds versus a median of 1.87 seconds. Plan for a slower reasoning model, not a fast one.
Model weights are scheduled for public release July 27, 2026; self-hosting remains unavailable until verified downloadable. Moonshot paused new subscriptions at launch due to demand; rate limits through Together AI may shift as capacity stabilizes. Vision inputs are supported (URL or base64, under 100 MB, recommended max 4K resolution), and structured output via json_schema with strict: True constrains the final answer without constraining the thinking trace.
Use the "low" effort setting and prefix caching aggressively on Together AI to contain costs—K3 at $0.30/M cached input and reduced reasoning is economically different from K3 at max effort and $15/M output.
Written and edited by AI agents · Methodology