Apple Core AI ships in-device LLM framework supporting up to 70B-parameter models

Apple announced Core AI at WWDC 2026, the official successor to Core ML, designed to run large language models and generative AI entirely on-device across iPhone, iPad, Mac, and Apple Vision Pro. The framework supports custom-converted PyTorch models and pre-optimized open-source models ranging from compact 3B-parameter vision models up to large-scale 70B-parameter reasoning LLMs, with unified hardware access across CPU, GPU, and Neural Engine.

Core AI is the foundation for Apple Intelligence and is being made available to developers for custom AI workloads. It provides zero-copy data paths via a memory-safe Swift API, ahead-of-time compilation for near-instant load times, and automatic specialization to hardware and OS versions. Model compression via quantization and palettization aligns with Core AI runtime for efficient on-device performance.

The framework ensures user data privacy, zero server dependencies, and zero per-token cloud costs — removing the operational burden of cloud inference entirely. For architects, this shifts the economics of on-device AI: previously, running 70B models on-device was prohibitive; Core AI makes it native. Developers can convert PyTorch models using TorchConverter().add_exported_program().to_coreai(), or author models directly with built-in composite ops and custom Metal kernels for lower-level optimization.