An API Gateway handles load balancing by spreading incoming requests evenly across the available instances of a service, so no single instance is overwhelmed while others sit idle.
Key Points: • The gateway maintains, or queries, the list of healthy instances for each service, typically through integration with a service registry. • It applies a load-balancing algorithm, such as round-robin, least-connections, or weighted distribution, to decide which instance handles each request. • Health checks let the gateway skip instances that are failing or overloaded, routing traffic only to instances known to be healthy. • Some gateways also factor in response times or geographic proximity to route requests to the fastest or nearest available instance. • This offloads load-balancing logic from individual services and clients, centralizing it in one place that's easier to tune and monitor.
Example: If the Product service has four running instances, the API Gateway distributes incoming requests across all four using round-robin, and automatically stops sending traffic to any instance that starts failing its health check until it recovers.
Interview Tip: A concise interview answer is:
"An API Gateway load balances by tracking the healthy instances of each service, usually via a service registry, and distributing requests across them with an algorithm like round-robin or least-connections. Health checks keep it from routing to unhealthy instances, which keeps the system responsive and prevents any single instance from being overloaded."