What Is Load Balancing
The mechanism is straightforward. Clients connect to the load balancer rather than to any individual server, and it selects a destination for each request according to a configured algorithm. It also monitors the health of each instance and stops sending traffic to any that stop responding correctly. That health checking is what turns a group of servers into a resilient pool.
The mechanism is straightforward. Clients connect to the load balancer rather than to any individual server, and it selects a destination for each request according to a configured algorithm. It also monitors the health of each instance and stops sending traffic to any that stop responding correctly. That health checking is what turns a group of servers into a resilient pool.
Clients address the load balancer, unaware of how many instances exist behind it. Instances can be added or removed without any client-side change.
Each request is assigned to an instance based on configured rules, which may consider connection counts, response times, or simple rotation.
The balancer tests each instance periodically and removes unhealthy ones from rotation. This is the feature that converts redundancy into actual availability.
Recovered instances rejoin the pool once health checks pass again, so recovery needs no manual intervention. Our DevOps services work configures these thresholds carefully.
The algorithm determines how requests get assigned, and the right choice depends on whether your requests are uniform in cost. Where every request takes similar effort, simple rotation works well. Where request cost varies widely, connection-aware or latency-aware methods prevent one instance accumulating all the expensive work.
Requests distributed in sequence across instances. Simple, predictable, and appropriate when instances are identical and request cost is broadly uniform.
Sends each request to the instance handling fewest active connections. Better where request duration varies substantially, since it accounts for actual current load.
Selects the instance responding fastest. Adapts to instances degrading gradually rather than failing outright, which health checks alone may not catch.
Routes consistently based on a request attribute such as client address. Useful for cache locality, though it can distribute load unevenly.
The main complication in load balancing is state. If an application holds session data in the memory of the instance that created it, subsequent requests must reach the same instance, which undermines even distribution and breaks when that instance fails. Solving this properly is usually a prerequisite for effective load balancing rather than an optimisation afterwards.
State held on one instance means requests must return there. That constrains distribution and loses the session entirely when the instance restarts.
The balancer routes a client consistently to one instance. This works but reduces distribution quality and still loses sessions on instance failure.
Storing sessions in a shared cache or database makes every instance equivalent. This is the durable solution and enables genuinely even distribution.
Tokens carrying their own validity remove server-side session storage entirely. Our API development work favours this approach for new services.
Building with What Is Load Balancing? Let's talk.
Load balancing means spreading incoming requests across several servers so none becomes overwhelmed. A load balancer receives every request, sends it to an available server, and stops sending traffic to any server that fails its health checks.
A load balancer distributes traffic across identical instances of one service, concerned with capacity and availability. An API gateway inspects request content, applies policy such as authentication and rate limiting, and routes to different services. They are commonly used together.
Round robin suits identical instances handling requests of broadly similar cost. Least connections is better where request duration varies substantially. Least response time helps detect gradually degrading instances. Hash-based routing suits cache locality at the cost of even distribution.
Periodic tests the balancer performs against each instance to confirm it is responding correctly. Failing instances are removed from rotation and reinstated automatically once they recover, which is what turns redundant servers into genuine availability.
Sticky sessions route a client consistently to one instance, which works around session data held in server memory. They reduce distribution quality and still lose sessions when an instance fails, so externalising session state is the better long-term solution.
Only if you run more than one instance, which you should if availability matters, since a single instance means any restart is downtime. Managed cloud load balancers are inexpensive and remove a meaningful category of failure.