KOEN
Read Original
#Backend

Stop Answering the Same Question Twice: Interval-Aware Caching for Druid at Netflix Scale

Stop Answering the Same Question Twice: Interval-Aware Caching for Druid at Netflix Scale
01

Summary

How Netflix Reduced DB Load by 33% Amidst 15 Million Events Per Second

A technical deep dive into interval-aware caching and exponential TTLs for rolling-window dashboards.

This article explores how Netflix addressed scalability challenges in its real-time monitoring systems by building an experimental caching layer for Apache Druid. It introduces an innovative approach of breaking queries into time-aligned buckets to maximize partial result reuse and adjusting cache TTLs based on data confidence.

  • 01Time-based bucketing to prevent redundant processing for rolling-window queries
  • 02Exponential TTL strategy that doubles cache duration as data ages and stabilizes
  • 03Transparent proxy architecture intercepting requests at the Druid Router level
  • 0466% improvement in P90 response times and up to 14x reduction in result bytes
  • 05Negative caching implementation to handle sparse metrics and avoid unnecessary re-queries

RECOMMENDATION

Highly recommended for engineers facing database scaling issues due to real-time dashboards, particularly those utilizing rolling time windows in time-series environments.

The Problem

Netflix's large-scale real-time dashboards generated hundreds of queries every 10 seconds, causing standard Apache Druid caches to miss due to shifting time windows and creating redundant database load.

The Solution

An interval-aware caching layer was developed to bucket query results by time, combined with an 'Exponential TTL' strategy that increases cache duration based on the age of the data.

The Result

Achieved an 82% partial cache hit rate for user queries, a 33% reduction in queries reaching Druid, and a 66% improvement in P90 query latency.

Trade-off

The system accepts up to 5 seconds of data staleness and introduces additional architectural complexity and network hops through an external proxy service.

03

Key Concepts

Concept · 01

Interval-Aware Caching

An intelligent caching method that slices query intervals into granular time buckets, allowing previously computed results to be retrieved from cache while fetching only the new data from the DB.

  • Decouples time intervals from query hashes to maximize hit rates for overlapping windows
  • Rebuilds queries with narrowed intervals to fetch missing 'tails' from Druid
Concept · 02

Exponential TTL

A caching strategy where fresh data gets a short TTL for accuracy, while older, settled data receives exponentially longer TTLs to increase cache persistence.

  • Addresses the late-arriving data problem in real-time analytics
  • Ranges from a 5-second minimum TTL to a 1-hour maximum based on data age
Concept · 03

Negative Caching

The practice of caching empty results for sparse metrics to prevent repetitive queries to the database for time periods where no data exists.

  • Uses sentinel values to distinguish between valid 'no data' and missing cache entries
  • Carefully avoids caching trailing empty buckets to prevent artificial delays for arriving data