Scaling a microservice under high load means increasing its capacity to handle traffic, typically by running more instances rather than making a single instance bigger.
Key Points: • Horizontal scaling, adding more instances behind a load balancer, is usually preferred over vertical scaling because it improves both throughput and fault tolerance. • First diagnose whether the bottleneck is CPU-bound, memory-bound, I/O-bound, or caused by a slow downstream dependency, since each needs a different fix. • Autoscaling policies driven by metrics like CPU utilization, request latency, or queue depth let the platform react without manual intervention. • Scaling one service can just push the bottleneck downstream, so dependent services and shared resources like the database need capacity planning too. • Cost and infrastructure limits, such as database connection pool size, matter as much as raw compute when deciding how far to scale.
Example: If a payment service is CPU-bound during checkout spikes, adding two more pods in Kubernetes with a horizontal pod autoscaler tied to CPU usage lets the system absorb the spike, while the database connection pool is checked to make sure it can support the extra instances.
Interview Tip: A concise interview answer is:
"I'd start by figuring out whether the service is CPU, memory, or dependency bound, then scale horizontally with autoscaling driven by the right metric. I'd also check that downstream dependencies like the database or a rate-limited external API can absorb the extra load, since scaling one service can just move the bottleneck."