Microsoft AKS on bare metal reaches public preview; adds Ray and Fleet Manager for edge-to-cloud AI
Microsoft announced at Build 2026 that Azure Kubernetes Service (AKS) now runs directly on bare-metal hardware in public preview, eliminating the hypervisor layer and giving workloads direct access to NVLink, RDMA, and high-performance networking. The feature targets large language model training and latency-sensitive inference where virtualization overhead measurably impacts performance and cost. AKS on bare metal integrates with Azure Arc for cloud-native provisioning and management, so the experience remains consistent whether running on edge devices, data centers, or cloud regions.
Simultaneously, Microsoft announced Anyscale on Azure (managed Ray in public preview) for orchestrating distributed AI workloads across CPUs and GPUs on dynamically scaling AKS clusters. Azure Kubernetes Fleet Manager is now generally available, extending centralized policy, workload placement, and RBAC governance across hybrid and multi-cloud cluster estates. These updates aim to make Kubernetes—rather than bespoke AI infrastructure stacks—the operational backbone for training and inference at scale.
Managed System Node Pools are now GA, auto-managing system component capacity and patching to prevent resource contention with GPU workloads. AI Runway, a Kubernetes-native model-serving framework, enables users to select models, validate GPU requirements, estimate costs, and launch production endpoints via KAITO provisioning and vLLM runtimes. The bare-metal support is initially available on validated Dell and HPE servers; broader certification follows by year-end.
For infrastructure architects: The bare-metal option signals enterprise acceptance of Kubernetes for AI production when the virtualization tax becomes unacceptable. Combined with Fleet Manager (now GA), this positions AKS as a unified control plane from edge to cloud, reducing operational fragmentation for organizations managing diverse hardware topologies and sensitive workloads.
Sources
- Primary source
- infoq.com
“AKS on Bare Metal, currently in public preview. By removing the virtualization layer, AKS can now provide direct access to technologies such as NVLink, RDMA, and high-performance networking”
- blog.aks.azure.com
“AKS now runs directly on bare-metal, small-form-factor devices at the edge, available today in public preview”
- techcommunity.microsoft.com
“Anyscale on Azure, now in public preview, brings managed Ray to AKS”
- techcommunity.microsoft.com
“Azure Kubernetes Service (AKS), the fully-managed enterprise-grade Kubernetes service, now runs directly on bare metal”