Auto-Scaling (HPA) Simulator

Mental Model

Auto-scaling is the process of adjusting computing resources dynamically based on actual demand. In Kubernetes, the Horizontal Pod Autoscaler (HPA) monitors resource metrics (typically CPU utilization) and adjusts the replica count of a deployment to keep average resource utilization near a target threshold.

The core mental model is that auto-scaling is a feedback loop: external traffic increases CPU usage, the metrics server collects CPU samples, the HPA controller evaluates those samples against the target threshold, and the deployment controller adjusts the replica count. A startup delay means new replicas are not ready immediately, and cooldowns prevent the system from overreacting to noisy metric spikes.

Components and Why They Exist

Traffic Source

To generate realistic, fluctuating workloads that simulate real-world usage patterns, such as gradual morning ramps or sudden Black Friday spikes. Emits request messages to the ingress path, representing user or client requests hitting the application. In the demo, watch the incoming traffic rate slider and the resulting visual flow of request dots entering the application.

Application Service Parent

It acts as the load balancer and routing layer, representing the Kubernetes Service that distributes traffic to running pods. Receives incoming requests and distributes them using a round-robin routing strategy across all active, healthy pod children. In the demo, watch how request dots are distributed to active pods, and see if any are dropped when the pods are overloaded.

Metrics Server Parent

HPA cannot make scaling decisions without visibility. The metrics server aggregates utilization stats across the cluster. Samples CPU and memory metrics from running pods and calculates the average utilization across the active replica set. In the demo, observe the average CPU utilization percentage and how it changes in response to traffic fluctuations.

HPA Controller Parent

It implements the control logic that evaluates metrics and decides whether to scale up, scale down, or stay stable. Computes the desired replica count using the ratio formula, checks cooldown constraints, and issues scaling commands to the deployment controller. In the demo, watch the desired replica count update, and observe the scale-up and scale-down cooldown timers.

Deployment Controller Parent

It executes the scaling decisions by managing the lifecycle of individual pod replicas. Creates new pods (which start in a pending state) or terminates existing pods to match the desired replica count. In the demo, watch pods change color and state (e.g., Pending, Ready, Running, or Terminating) on the canvas.

Edge Cases and Misconceptions

• Scaling is not instantaneous: New pods take time to initialize, pull container images, and pass readiness probes. During this startup delay, the existing pods must absorb the load, which can lead to temporary performance degradation or dropped requests.
• CPU is not the only metric: Scaling purely on CPU means the autoscaler will miss memory exhaustion issues (like memory leaks), queue backups, or downstream database bottlenecks.
• Aggressive scale-down is dangerous: Restoring capacity is expensive. Scaling down too quickly during a temporary dip in traffic can leave the system vulnerable to the next traffic wave, causing thrashing.
• Autoscaling has limits: Min and max replica boundaries protect against unbounded cost or resource exhaustion, but a max limit that is set too low will cause request drops under extreme traffic.
• Failed pods consume capacity: Pods that are crashing or restarting still consume cluster management resources and count toward the deployment\x27s replicas, even though they cannot serve traffic.