Together AI has shipped native A/B testing capabilities at the endpoint level, allowing teams to run model comparison experiments directly on production traffic without building routing logic into application code. Users can split live traffic into a control and up to 20 variant deployments with fixed percentage splits, measure real-world metrics (retention, task completion, user feedback), and promote winners using blue-green rollout. Ramping can be done incrementally: 95/5 split, then 80/20, then 50/50, with each ramp being an explicit, etag-guarded API call.
Rather than deploying A/B test logic in application code (feature flags, hash-based routing, hardcoded model strings), Together's platform-level approach decouples experiment routing from client code. This reduces code entropy: once an experiment ends, the routing logic doesn't linger in the codebase. A/B percents are fixed traffic shares independent of replica counts and autoscaling, ensuring consistent measurement windows. Variant deployments must be zero-weight in the endpoint's base traffic split; the experiment owns the variant's allocation entirely, preventing measurement corruption from capacity-weighted traffic drift.
The capability addresses a production gap: teams historically rely on shadow traffic (discarded responses) to validate operational metrics (latency, error rates), but shadow testing cannot measure user-facing quality improvements like retention or task completion. This primitives ships at a time when model selection uncertainty is high—enterprises must choose between cost-optimized, open-weight, and frontier models for different use cases. Having A/B testing built into the inference platform enables rapid iteration on model stacking, quantization variants, and multi-model strategies without complex application-layer scaffolding.