Auto-Scaling Component Runtime Simulator

Mental Model

Modern cloud platforms represent services not as monolithic blocks but as systems of interacting components connected by communication pathways. The Horizontal Pod Autoscaler (HPA) works in tandem with a Service proxy, a Metrics Server, a Deployment Controller, and a pool of Pods to dynamically adjust replica counts.

In this component-runtime model, resource scaling is represented through explicit message propagation. Request events flow through input and output ports, are queued, and get dispatched to individual pod instances. This models realistic operational overheads: network delay, worker queues, and the fact that scaling decisions depend on scraped metrics that reflect past rather than instantaneous system states.

Components and Why They Exist

Traffic Source

To generate realistic workloads (e.g. spikes, ramps, or normal patterns) to test how the autoscaler responds under pressure. Periodically emits requests through its output port, which travel along connector pathways to the ingress service. In the demo, observe the volume and rate of request dots being dispatched from the traffic source to the Service component.

Application Service Parent

Acts as the ingress gateway and load balancer for the application pool. Receives external requests on its requestIn port, queues them internally, and distributes them to ready pod children using round-robin routing. If the queue overflows, it performs load shedding. In the demo, watch incoming requests enter the parent queue, and note if queue depth builds up or requests are dropped (load shed) when pods are slow.

Pod Children

To run the actual application workload and process requests. Accept requests on their input ports, consume CPU/memory resources during processing, and report metrics. They transition through lifecycle states (Pending, Ready, Running, Crashed). In the demo, observe individual pod CPU utilization, readiness timers when starting up, and state changes (such as crashing under heavy queue pressure).

Metrics Server Parent

Aggregates resource usage across the system so the autoscaler has a data source for decisions. Scrapes and pools CPU and memory metrics from all active, ready pods, calculates the average utilization, and exposes it on its query port. In the demo, watch the metrics query logs and the resulting computed average CPU utilization update at regular intervals.

HPA Controller Parent

Implements the core auto-scaling logic and feedback loop. Periodically queries the Metrics Server, evaluates average CPU utilization against the target CPU using the ratio formula, applies tolerance thresholds, and handles stabilization cooldowns before deciding to scale. In the demo, watch the HPA controller evaluate recommendations, check the scale-up and scale-down stabilization gates, and output scaling commands.

Deployment Controller Parent

Reconciles the desired replica state requested by HPA with the actual state of running pods. Spins up new pod children (which start as Pending and must run readiness checks) or terminates existing pods. In the demo, watch the deployment controller trigger pod creation or deletion animations on the canvas when it receives commands.

Edge Cases and Misconceptions

• Scrape and Startup Delays: A common misconception is that scaling is instant. In reality, the scrape interval of the metrics server combined with the startup delay of new containers means there is a significant lag between a traffic spike and new ready capacity.
• Memory Blindness: HPA configured for CPU utilization is blind to memory leaks. A memory-leaking application may crash due to Out-Of-Memory (OOM) errors, but if the CPU usage remains low, HPA will not scale up to replace the crashing replicas.
• Scaling Thrashing: If the scale-up cooldown or scale-down stabilization windows are too short, noisy metrics will cause the system to oscillate rapidly between scaling up and scaling down, leading to resource churn and instability.
• Load Shedding vs. Service Collapse: Releasing pressure via load shedding (dropping requests) is necessary when the system is overloaded. Without it, worker queues grow boundlessly, memory utilization spikes, and all pods may crash simultaneously, causing a complete outage.