KOEN
Read Original
#DevOps

Single-tenant Titus: VM-level isolation on a multi-tenant container platform

Single-tenant Titus: VM-level isolation on a multi-tenant container platform
01

Summary

Why Netflix Returned to Dedicated VMs for Critical Container Workloads

Navigating hardware-level noisy neighbor problems on Titus and introducing VM-level isolation without losing container platform agility

Netflix's Titus platform powers millions of containers, but hardware-level resource sharing introduced unpredictable performance degradation for high-throughput streaming services. After evaluating alternatives like microVMs and gVisor, Netflix chose Single-tenant Titus—a strategy that provisions dedicated EC2 instances via their federation layer. This article explores how Netflix achieved full performance isolation and superior fleet efficiency while keeping operational burdens off application developers.

  • 01Analysis of non-cgroup-enforceable hardware bottlenecks including LLC cache eviction, memory bandwidth, and softirq contention
  • 02Comparative evaluation of isolation techniques: shared kernel, microVMs (Firecracker), gVisor, and dedicated EC2 instances
  • 03Automated just-in-time provisioning of right-sized EC2 instances using Titus federation and customized autoscaling
  • 04Strict single-tenant guarantees enforced via pod anti-affinity, node taints, and disabled defragmentation
  • 05Fleet efficiency optimization by isolating disruptive workloads, resulting in higher bin-packing density on remaining shared hosts

RECOMMENDATION

Essential reading for infrastructure engineers and platform architects dealing with high-throughput data processing or strict tail-latency SLAs. It provides a realistic architectural playbook on balancing container density against hardware-level isolation.

The Problem

In multi-tenant container environments, Linux cgroups and namespaces fail to prevent hardware-level resource interference, such as Last-Level Cache (LLC) eviction, memory bandwidth saturation, and softirq processing overhead. High-throughput stateful streaming jobs experienced tail-latency spikes and costly stop-the-world recoveries due to these intermittent noisy neighbor dynamics.

The Solution

Instead of adopting microVMs or gVisor, Netflix built 'Single-tenant Titus' by integrating their existing federation layer and custom autoscaling to provision dedicated, right-sized EC2 instances per task. Enforcing pod anti-affinity and node taints provides full performance and resource isolation while preserving the managed platform experience for developers.

The Result

Isolating high-interference and latency-sensitive workloads onto single-tenant nodes completely eliminated hardware contention. Furthermore, removing noisy neighbors relaxed anti-affinity constraints across shared hosts, enabling denser container bin-packing and elevating overall fleet compute efficiency.

Trade-off

Replacing large shared hosts with numerous smaller dedicated instances increases control plane tracking overhead and EC2 launch API call volumes during regional failovers. Additionally, imperfect alignment between task requests and standardized EC2 instance shapes can leave minor unused resource headroom stranded.

03

Key Concepts

Concept · 01

Noisy Neighbor Contention

A phenomenon where co-located containers degrade neighbor performance by competing for underlying shared physical hardware resources that kernel quotas cannot partition.

  • In Titus, HTB lock contention during small packet floods and L3 cache eviction caused tail-latency spikes even when containers remained within cgroup limits.
Concept · 02

Single-tenant Titus

An execution mode in Netflix's Titus platform that assigns a dedicated EC2 virtual machine to a single container task while preserving standard platform APIs and managed host lifecycle.

  • It uses the Titus federation layer to dynamically provision the smallest matching EC2 instance family for a given task.
Concept · 03

Pod Anti-Affinity and Taints

Kubernetes scheduling features used to prohibit co-locating specific pods on the same host and restrict un-tolerated workloads from being scheduled onto designated nodes.

  • Single-tenant nodes are tainted and configured with anti-affinity to ensure no other containers can land on the dedicated instance.