Netflix's gRPC API Queries 150 Billion Graph Edges Below 100ms

Netflix published Part 3 of its Real-Time Distributed Graph series, detailing the gRPC Execution API that powers query serving on a graph with 8 billion nodes and 150 billion edges. The serving layer hits sub-100ms latency at tens of thousands of queries per second while absorbing 6 million writes per second. Authors Nilesh Mishra and Ajit Koti explain the design tradeoffs that enable this scale.

Netflix identified two opposing access patterns. Shallow-and-wide queries—security lookups like "which devices has this account streamed from in the last 30 days?"—stay one hop deep but fan out to hundreds of edges per active account, requiring temporal filtering on edge properties at scale. Deep-and-narrow queries—personalization traces like "show me a member's Stranger Things viewing history across all profiles"—chain 3 to 4 sequential hops, each dependent on prior results. A 4-hop traversal executed client-side costs 40ms at 10ms per network hop before returning any business data.

Netflix chose breadth-first execution over depth-first traversal. Depth-first serializes every path through the graph, compounding latency proportional to fan-out depth. Breadth-first dispatches all edges at a given hop in parallel, waits for the full frontier, then advances. For an account with 5 profiles each having hundreds of titles, this eliminates a 5× serialization multiplier on the inner loop.

The gRPC Execution API encodes the complete traversal plan—all hops, per-hop filters, fan-out bounds—into a single RPC. The serving layer executes server-side, so clients issue one request and receive one response regardless of hop depth. This removes the client round-trip penalty and moves execution control into the service boundary where the storage topology is known. The API design follows Gremlin patterns: callers chain traversal steps and apply filters declaratively.

Storage sits on KVDAL—a Key-Value Data Abstraction Layer built on Apache Cassandra. KVDAL uses a two-level map: a record_id maps to sorted items (destination nodes and edge properties). A single KVDAL lookup retrieves the full adjacency list for a node. All access runs over gRPC, making the storage backend swappable without upstream code changes. The deployment spans 27 namespaces across 12 Cassandra clusters on 2,400 EC2 instances, sustaining 2 million reads/sec and 6 million writes/sec at single-digit-millisecond latency.

Netflix evaluated native graph databases before committing to this stack. Neo4j performed acceptably at millions of records but degraded at hundreds of millions due to memory requirements and horizontal scaling limits. AWS Neptune was rejected for its single-writer architecture, which bottlenecks high-volume real-time ingestion across regions. The adjacency-list-on-Cassandra approach trades graph-native query expressiveness for operational predictability and horizontal scale.

Consistency guarantees during traversal across a live 150-billion-edge dataset under 6 million writes per second are a live constraint rather than a fully solved problem. Schema enforcement—loaded into memory and strictly validated at the serving layer—prevents invalid traversal paths from reaching storage, guarding against query plans that degrade under real data distributions.

For architects building recommendation or retrieval serving layers, the gRPC Execution API pattern (encode the full traversal plan, execute server-side, return one response) is the portable takeaway. The 10ms-per-hop math applies to any distributed graph store regardless of underlying storage technology.