TikTok Recommendation and Delivery System
Walk through a TikTok-style short-video platform from creator upload through encoder output, Object Storage arrival, item embeddings, Two-Tower ranking, CDN pre-fetching, viewer playback, and feedback recovery.
Continue with the guided workspace
Use the paired workspace to connect this guide's mental model with live component state, visible signals, and the outcome each step or scenario is meant to prove.
A short-video platform like TikTok combines a high-throughput ingestion pipeline, a recommendation engine, and a global CDN. Creators upload raw video, a transcoding fanout cuts it into streamable segments, a Two-Tower recommendation model scores candidate videos against the viewer’s engagement profile, and CDN edges cache the hottest segments so playback starts in under a second.
For parameter experimentation, the TikTok System Simulator is also available.
Mental Model
Think of TikTok’s architecture like a high-speed library combined with a personal book matcher. Creators upload raw drafts (uploads) that are automatically formatted, cut into short chapters, and placed in a central warehouse (transcoding & object storage). The head librarian (CDN edge) keeps copies of popular chapters nearby for immediate reading. To recommend books, the librarian uses two matching boards (Two-Tower model): one representing book subjects (Item Tower) and one representing the reader’s mood (User Tower). As the reader flips pages (user engagement), the librarian updates their mood profile in real-time to hand them better matched chapters next.
Components and Why They Exist
Uploader Client
Provides an interface for creators to publish raw video content to the platform. Records and uploads video files, transmitting them to the ingestion backend. In the demo, observe the creator client uploading raw video assets into the platform.
Video Upload API
The gateway handling incoming video uploads, metadata creation, and worker triggering. Performs auth and format validation, writes metadata, and registers transcoding tasks. In the demo, observe the upload API receiving the raw video and spawning a background transcoding task.
Video Transcoding Service
Prepares video streams for varied network bitrates and smooth playback. Slices uploaded videos into uniform 3-second segments and encodes them at multiple resolutions. In the demo, encoder-local rendition output appears when queued work arrives; the durable manifest remains unchanged until a separate particle reaches Object Storage.
Object Storage (Origin)
Serves as the central, highly durable persistent store for all static chunk assets. Holds all segmented video media files and serves as the origin backend for CDN edges. In the demo, observe the manifest and segment count change only when the transcoder-to-storage particle reaches the origin node, before the media is embedded or cached at the edge.
Metadata DB
The visualization models a persisted publish-metadata record and an analytics counter in this card. Its metadata becomes video_482 persisted on the upload route, and the counter increments when the watch event arrives; it does not store or update the User Tower vector.
Two-Tower Recommendation Engine
Matches the modeled Item Tower candidate set with the separate User Tower vector to produce an ordered feed. In the demo, the Event Stream feedback route changes the User Tower vector from baseline to fresh: dance + diy; a later User Tower → Ranker route updates the ranking, followed by the Ranker → CDN cache plan.
CDN Edge Node
Caches transcoded video segments near users to reduce latency and origin read congestion. Caches chunk files in memory and serves them locally, only requesting from origin on cache misses. In the demo, observe the CDN caching segments to deliver fast, low-latency playback to nearby viewers.
Viewer Client
The client-side player application that renders video streams and captures user telemetry. Buffers and decodes video segments, pre-fetches future recommendations, and reports user interactions. In the demo, watch the viewer player buffer segments in advance to offer a zero-latency swipe experience.
User Engagement Queue
Captures the modeled viewer watch event. In the demo, the Viewer Delivery Client appends event 01; that arrival increments Metadata + Analytics’ analytics counter, and the separate Event Stream → User Tower route later refreshes the user vector.
Edge Cases and Misconceptions
- Two-Tower Model Latency: A common misconception is that the recommendation system runs heavy deep neural network inference for every viewer swipe in real-time. In reality, the Item Tower embeddings are pre-computed offline. The User Tower computes a user vector in real-time, and candidate selection is performed using fast Approximate Nearest Neighbors (ANN) search (e.g. Hierarchical Navigable Small World graphs) on the pre-computed item vectors.
- Pre-fetching Trade-offs: Pre-fetching the first few chunks of the next recommended videos creates a seamless, zero-latency swiping experience. However, if a user swipes rapidly (low watch time), pre-fetched chunks are discarded, wasting massive network egress bandwidth. Recommenders must dynamically scale the pre-fetch window based on the viewer’s swipe speed and network conditions.
- Thundering Herd on Cache Eviction: If a highly viral video’s chunks expire or are manually flushed from the CDN simultaneously, thousands of viewer clients will query the Object Storage origin at once. This thundering herd can overwhelm database metadata services and origin storage throughput.
Common Questions
How does the recommendation feedback loop update the user vector in real-time?
In this visualization, viewer playback appends one watch event to the Event Stream. When its feedback particle reaches User Tower, the bounded user-vector state changes from `baseline` to `fresh: dance + diy`; the visualization does not model an online-learning service or learning-rate calculation.
Why are videos sliced into short 3-second chunks rather than streamed as whole files?
Slicing videos into small chunks allows the player to start playback instantly with minimal buffering. It also enables dynamic adaptive streaming (bitrate switching), where the player can download a lower-resolution chunk if network bandwidth drops, preventing playback stalls.
What is the difference between candidate generation and ranking in recommendation systems?
Candidate generation (retrieval) uses lightweight models (like the Two-Tower model) to narrow down a catalog of millions of videos to a few hundred candidates. The ranking phase then runs a more complex, feature-rich neural network (incorporating user demographics, historical interactions, and freshness) on those few hundred candidates to generate the final sorted feed.
How does the CDN edge cache mitigate origin load?
A larger CDN cache capacity stores more transcoded segments, increasing cache hit rates (~15ms response) and protecting the Object Storage origin from direct read spikes (~220ms response).
What happens if the recommendation feedback updates are disabled or slow?
If user vector updates are delayed or disabled, the recommendations become stagnant ('recommendation drift'). The system continues serving videos matching outdated interests, leading to lower engagement, rapid swiping, and higher session abandonment rates.
How does the system handle video transcoding failures?
Transcoding is treated as an asynchronous, retryable background job. If transcoding fails, the upload remains hidden from the public catalog. Legitimate clients only see videos once their metadata status is updated to 'ready' after successful transcoding and chunk upload.
How does CDN cache capacity affect origin scalability?
Low cache capacity forces the CDN edge to evict active segments frequently (cache thrashing). When a viewer requests evicted segments, the CDN must pull them from the Object Storage origin, increasing request latency and compounding the load on origin servers.