Together AI launches Dedicated Model Inference with canary deploy, A/B testing, and auto-rollback

Together AI released a major update to its inference platform on July 23, adding production-grade deployment controls to its open-weight model hosting service. The Dedicated Model Inference platform ships with canary rollout, blue-green and rolling updates, automatic rollback on metric thresholds, A/B testing against live traffic, shadow testing, and multi-region autoscaling. Teams can now deploy new model versions, fine-tuned weights, or LoRA adapters behind a single stable endpoint without rebuilding or switching platforms.

The platform abstracts away quantization, tensor parallelism, speculative decoding, and serving engine configuration through pre-tested deployment profiles, while still exposing expert-level control for teams that need to optimize latency, throughput, or cost balance per workload. Together is also launching a closed beta for custom training, including full-weight and LoRA reinforcement learning and supervised fine-tuning with checkpoints deployable straight to production. The company reports serving over 400 trillion tokens per month across 40+ models.

For teams running production agentic AI, code agents, or voice systems on open-weight models, this removes the operational burden of building and maintaining a homegrown model serving stack. Decagon, a Together AI customer, noted they previously relied on manual versioning; the new platform lets them canary fine-tuned models on live traffic weekly, automatically rolling back if key metrics regress. Architects evaluating open-source inference should assess whether platform features like auto-rollback and safe iteration velocity outweigh the operational cost of self-hosting or the lock-in trade-off of closed-model APIs.