Load Balancer Algorithms

Load balancing is a core design pattern for scaling modern applications, distributing incoming network traffic across multiple backend servers to prevent overload and ensure high availability.

Mental Model

By separating the client-facing IP from individual backend nodes, load balancers allow servers to be added, removed, or restarted without interrupting user traffic. The load balancer intercepts client requests and applies a routing algorithm: Round Robin distributes queries sequentially; Least Connections routes to the node with the fewest active sessions; IP Hash hashes the client IP to ensure session affinity to a single node; and Weighted routing scales request distribution to match node capacities. A background health checker probes backend status and removes failing servers from the routing pool; health checks can lag or produce false positives, so draining and retry policy still matter.

Components and Why They Exist

Clients

The source of user request traffic consuming the application. Dispatches request payloads to the load balancer’s virtual IP. In the demo, watch request dots originate from clients and flow to the load balancer.

Load Balancer

The central router that intercepts client requests and applies the selected routing algorithm. Evaluates algorithm logic (like Round Robin or Least Connections) and forwards requests to the chosen backend server. In the demo, watch it route incoming client request traffic based on the selected algorithm.

Backend Servers / Nodes (Server 1 / Server 2 / Server 3)

The backend application pool processing the requests. Accepts redirected connections, tracks active connections, records response times, and returns response payloads. In the demo, watch connection gauges and response times update per request.

Health Checker / Sentinel

Automatically detects and isolates broken backend instances to preserve system availability. Periodically checks the health of each server, marking them offline if they fail to respond. In the demo, watch server nodes fade out or show an offline status badge, and the load balancer reroute traffic.

Edge Cases and Misconceptions

• Misconception: Round Robin is always the simplest and best algorithm: Round Robin distributes traffic evenly by count, which is only effective if all requests consume similar resources and all backends have identical CPU/memory capacities. If server capacities differ, Round Robin will quickly overload weaker nodes.
• Misconception: IP Hash guarantees perfect load distribution: IP Hash maps client IP addresses to servers, ensuring session affinity. However, if many clients are behind a single corporate proxy or NAT gateway, they share the same public IP and will all route to the same backend server, causing an imbalance.
• Edge Case: Least Connections herd effect during cold start: When a new or recovered server joins the cluster with 0 connections, Least Connections will route all incoming requests to it until its connection count matches other nodes. This sudden storm can immediately overload the server before it warms up.
• Recovery is a measured ramp, not a periodic exception: In the slow-start lesson, Server 2 begins at 0% of its configured traffic weight after its health probe succeeds. Its modeled admission rises continuously to 100% over a 30-second recovery window, then its health changes from warming to healthy. Its actual share also reflects the other backends’ configured weights.