KOEN
Read Original
#AI

From Activity to Intent: Generating User Journeys with LLMs

From Activity to Intent: Generating User Journeys with LLMs
01

Summary

How Pinterest Consolidated Its Recommendation Pipeline into a Single 4B LLM

Shifting from reactive item filtering to long-horizon user intent generation at global scale.

Pinterest modernized its user journey inference architecture by replacing a complex multi-stage clustering pipeline with a unified 4-billion parameter generative LLM. By distilling frontier model reasoning into a Qwen3 student model and leveraging vLLM with PagedAttention, Pinterest delivers highly personalized and multilingual intent summaries to hundreds of millions of active users daily.

  • 01Collapsed extraction, clustering, and ranking into a single activity-to-intent LLM prompt.
  • 02Distilled reasoning from frontier models into a cost-effective 4B Qwen3 model using LoRA SFT.
  • 03Built a scalable serving stack achieving 775 RPS over ~100 GPUs using vLLM, PagedAttention, and KV prefix caching.
  • 04Implemented defensive JSON output handling and safety policies directly inside prompt instructions.
  • 05Pioneered RQ-VAE Semantic IDs (SIDs) to compress Pin representations and drastically lower token overhead.

RECOMMENDATION

ML architects and tech leads working on recommendation systems should study this framework to replace complex legacy ML pipelines with distilled generative models and high-throughput vLLM serving stacks.

The Problem

Pinterest's legacy user journey system relied on a brittle multi-stage pipeline of keyword extraction, embeddings, and clustering, which caused error compounding, fragmentation, generic bucket naming, and high vulnerability to noise.

The Solution

The architecture was consolidated into a single generative task powered by a fine-tuned 4B Qwen3 student distilled from frontier LLMs, served over a high-throughput vLLM and NVIDIA Dynamo infrastructure.

The Result

The system achieved a 1.1% lift in email CTR, a 1.3% increase in push opens, and statistically significant platform-wide engagement and retention gains across hundreds of millions of users globally.

Trade-off

Operating a 4B parameter generative LLM entails higher GPU infrastructure costs compared to classical clustering, and emerging enhancements like Semantic IDs reduce human readability during prompt debugging.

03

Key Concepts

Concept · 01

User Journey

A high-level intent or project a user aims to accomplish over weeks or months across multiple sessions.

  • Refers to multi-month projects on Pinterest like kitchen remodeling or planning a summer wedding.
  • Serves as the primary personalization unit instead of single item clicks or broad taxonomy labels.
Concept · 02

Knowledge Distillation

A technique that transfers knowledge and reasoning capabilities from a large teacher model to a smaller student model.

  • Used frontier models to generate high-quality synthetic training examples for student models.
  • Enabled a 4B Qwen3 student model to achieve near-teacher quality while maintaining manageable inference costs.
Concept · 03

vLLM & PagedAttention

A high-performance LLM serving framework that optimizes GPU memory management through paged KV cache allocation.

  • Deployed alongside NVIDIA Dynamo to achieve 775 RPS throughput across roughly 100 GPUs.
  • Leveraged continuous batching and prefix caching to minimize decode latency for daily batch scoring.