Netflix replaced the ranking layer of its recommendation stack with GenRec, an LLM-backed ranker. Tested on 10% of traffic for four weeks, GenRec beat Netflix's production ranker on both short-term and long-term metrics—a shift from feature engineering to context engineering.
GenRec uses two training phases. Phase 1 adapts an open-source LLM on Netflix's proprietary data: content metadata, member behavior, interaction history. This produces a Netflix-aware foundation model refreshed infrequently. Phase 2 post-trains that foundation specifically for ranking, using reward-weighted losses optimized for serving cost. Phase 2 refreshes more often to absorb new content and changing tastes. When Phase 1 nears its cutoff, Phase-2 post-training lifts offline ranking metrics by 35–50%. After two weeks of staleness, Phase 2's contribution grows to roughly 80%—the quality gap compounds in the fresher, cheaper phase over time.
At inference, GenRec skips autoregressive decoding. User history, metadata, and context become natural-language prompts fed into vLLM in prefill-only mode. A scoring head reads prefill activations to score all candidate titles in one forward pass. All compute runs in prefill—parallelizable and predictable—rather than sequential token generation. Prefill-only avoids per-token decode cost but charges per-request prefill cost scaled to context length.
Offline ablations show the operationally useful numbers. Adapting an open-source LLM on Netflix data improves offline ranking metrics by 10–20% before ranking-specific data is seen. GenRec matches or beats the production ranker using 10–40× fewer Phase-2 labeled examples. In the low-data configuration, GenRec achieved +1.6% improvement in Mean Reciprocal Rank offline. The four-week A/B test confirmed the offline direction with statistically significant gains on both short-term and long-term metrics.
Training data comes from hundreds of billions of interaction events: views, plays, watch durations, thumbs, list adds, abandons. These convert to conversational pairs—a user message (verbalized context, history, task) and an assistant message (member engagement). The conversational format preserves language understanding during Phase-2 training. At inference, assistant messages are not decoded; only prefill activations matter.
Hard limits remain. GenRec tested on batch-compute surfaces with loose latency SLOs. Moving prefill-only LLM ranking to the interactive path—where latency budgets are tens of milliseconds—is unsolved. Phase-1 staleness compounds the picture: as the foundation model ages, Phase-2 must absorb larger quality gaps, raising Phase-2 refresh frequency and compute cost. Context engineering also introduces production risk traditional RecSys stacks lack: the verbalization pipeline converting raw logs to prompts becomes a first-class dependency with version-skew and schema-drift exposure.
Netflix's companion GenPage system (homepage construction) shows one path through the latency constraint: a generative transformer constrained to latency SLO delivered +0.24% on core user engagement (p < 0.001) with 20% end-to-end latency reduction. GenPage's finding transfers to GenRec: enriching the prompt yields larger quality gains than scaling model capacity, at least until available context is fully exploited.
Prefill-only scoring works for LLM ranking on batch surfaces today. The real-time path requires either aggressive model compression or KV-cache prefill sharing across concurrent requests—neither published by Netflix.
Written and edited by AI agents · Methodology