URL Shortener System Design

A URL shortener turns a long link into a short code, stores the mapping, and redirects anyone who hits the short code back to the original URL. The design tension is between the write path (generating unique codes at high throughput) and the read path (resolving redirects in milliseconds under massive traffic), solved with Base62 encoding, sharding, and cache-aside Redis.

For parameter experimentation, the URL Shortener Simulator is also available.

Mental Model

Think of a URL shortener like a coat check at a theater. The write path is the drop-off: you hand over a large coat (long URL), the clerk prints a tiny ticket index (Base62 code), and hangs the coat on a specific numbered rack (database shard) after verifying the ticket number isn’t already taken (collision check). The read path is the pick-up: you show your ticket (short URL) and get your coat immediately. To save steps, popular coats are kept on a hanger right next to the counter (Redis cache), or you might even keep a permanent duplicate at home (browser cache) so you don’t have to visit the theater at all.

Components and Why They Exist

Client

The user browser interface that submits long links to be shortened and requests short URLs. Sends POST payloads for URL shortening and receives HTTP 301/302 headers for redirection. In the demo, observe the client submitting a long URL or initiating a redirect request by clicking a shortened link.

API Gateway

The ingress gateway regulating incoming traffic and providing routing rules. Validates input parameters, checks rate limit policies, and forwards traffic to write or redirect microservices. In the demo, observe the API gateway routing writes to the Write Service or resolving cache routes.

Write Service

The microservice managing the creation of short URLs, code validation, and database insertion. Invokes the hash engine, executes collision checks, and coordinates sharded database writes. In the demo, watch the write service orchestrate the base62 encoding and collision verification checks.

ID Generator / Hash Engine

Generates short alphanumeric keys to represent long target URLs uniquely. Converts sequential auto-incrementing IDs to Base62 or generates cropped MD5/MurmurHash values. In the demo, observe the conversion of an internal numeric ID into a Base62 alphanumeric string like gXyZ7.

Collision Checker

Ensures no two different long URLs are assigned the same short code. Queries the database or cache to verify if a generated code is already in use, adding a salt if a duplicate is found. In the demo, watch the checker flag a hash collision and trigger a salted regenerator routine.

Shard Router

Determines the correct database shard for each short URL to spread data load. Hashes the short code to calculate the index of the database node responsible for storing the record. In the demo, observe the router hashing the short code to select a target database shard.

URL DB Shards

Provides horizontal scalability and partitioning for high-volume mapping records. Stores mapping pairs (short_code to long_url) in split tables across multiple database nodes. In the demo, observe data being saved to a specific shard node based on the router’s assignment.

Redirect Service

Resolves short keys back to their original long URLs to perform HTTP redirection. Checks the caching layers or database shards for the short code, returning an HTTP redirect header. In the demo, watch the service return an HTTP 301 Moved Permanently header containing the target long URL.

Redis Cache

Speeds up lookups of popular short URLs to prevent database bottleneck congestion. Caches the mapping of frequently accessed short links in-memory to provide sub-millisecond responses. In the demo, watch read lookups hit the Redis cache and bypass the database shards entirely.

Browser Cache

Reduces subsequent network traffic by caching HTTP 301 redirects directly at the client. Locally stores the destination URL corresponding to a 301 short link to resolve future clicks locally. In the demo, notice subsequent clicks on a 301 URL bypassing the server network completely.

Edge Cases and Misconceptions

• Collisions in Hashing: If using MD5/MurmurHash and taking the first 6 characters, collisions will eventually occur. The system must query the database to verify if the code exists. If it does and maps to a different long URL, the system appends a salt (random suffix) to the long URL and hashes again.
• 301 vs. 302 Analytics Impact: While a cacheable HTTP 301 can reduce server traffic, it can hide later user behavior from the shortener. Advertisers or analytics-heavy services prefer HTTP 302 (Temporary Redirect) because it forces the browser to request the server every time, enabling full logging of click analytics (IPs, referrers, and user agents).
• ID Generation Coordination: Using a central database for auto-incrementing IDs creates a single point of failure. Autocomplete and URL systems use distributed ID generators (like Twitter Snowflake) to generate non-overlapping unique IDs lock-free.