Modal rebuilt its sandbox infrastructure to eliminate Kubernetes' centralized coordination bottleneck, achieving 1 million concurrent sandboxes with median startup time under 0.5 seconds and a peak creation rate of 50,000 sandboxes per second, according to staff engineers Colin Weld and Connor Adams writing on the platform's engineering blog as reported by InfoQ.
The core problem Weld and Adams identified is that Kubernetes and similar container orchestrators rely on centralized scheduling and a strongly consistent state store. At scale, operations that are O(containers) or O(nodes) become bottlenecks: the scheduling algorithm and etcd, Kubernetes' central durable store, both grow in load with cluster size, and etcd is not natively shardable within a keyspace. Running 1 million sandboxes across tens of thousands of compute nodes pushes these limits past viability. Overcoming the limitation is possible but requires "serious work", including rewriting or replacing etcd and parallelizing the scheduling algorithm.
Modal's solution inverts the coordination model. Instead of a global source of truth, each worker became its own source of truth. Rather than a single serialized scheduler, Modal deployed a fleet of scheduling servers operating in parallel, allowing the scheduling layer to scale horizontally. Once a scheduling server decides which worker should host a sandbox, it contacts that worker directly via RPC. Workers accept the request if they have free resources or reject it otherwise. The only remaining centralized component is a Redis stream where all workers publish their state, but load testing showed this remains viable until well over 100,000 workers.
In their benchmark, Modal created 1 million sandboxes in under a minute. The median time from sandbox creation to code execution was under 0.5 seconds, with the platform sustaining 50,000 sandbox creations per second. These numbers represent the operational floor for multi-tenant agent platforms: teams building agentic systems that spawn isolated execution contexts need sub-second provisioning and the ability to handle tens of thousands of concurrent requests.
The architectural shift reflects a deeper insight about GenAI infrastructure. AWS principal AI engineer Alex Jones, commenting on the announcement, noted that Modal's key achievement was not extending Kubernetes but "walking around the whole thing" after understanding its limitations. Jones argues that "Kubernetes isn't adapting fast enough to what GenAI infrastructure actually needs" and predicts a decoupling: the execution plane needs what Modal built—isolation boundaries appearing in milliseconds—while the coordination plane, where multi-agent workflows need shared memory and overlapping security boundaries, still benefits from Kubernetes-shaped systems.
Modal is not alone in this direction. Other projects pursuing similar goals include Unikraft, Google Substrate, and Overdrive, all seeking to rebuild cloud infrastructure around highly scalable execution with sub-10-millisecond cold starts. The pattern is clear: teams building serverless inference stacks or multi-tenant agent platforms should expect that general-purpose orchestrators will become a constraint before a solution, and plan for infrastructure that decouples scheduling from execution.