Load balancer

An L4 balancer works at the TCP/UDP level: it sees IPs and ports, but not HTTP. It is very fast and cheap and fits any protocol (databases, gRPC, WebSocket). Examples: AWS NLB, HAProxy in TCP mode, Linux IPVS.

An L7 balancer understands HTTP: it routes by path and headers (/api to one pool, /static to another), terminates TLS, adds headers and retries. Examples: Nginx, Caddy, Traefik, Envoy, AWS ALB.

Round robin hands requests to each copy in turn, simple and good for similar short requests. Least connections sends a request to the copy with the fewest active connections and works better when requests vary in duration (say, generating a PDF certificate next to a regular API call).

Consistent hashing binds a key (user id, URL) to a specific backend, and adding a node remaps only a small share of keys. This is useful for caches and stateful services.

The balancer calls /healthz on each copy every few seconds. After 2 or 3 failed checks the copy leaves rotation, and users never see it.

Rolling deploys build on this: the new version starts alongside, passes the health check and joins the pool, while the old one first stops receiving new requests (draining), waits for current ones to finish and only then stops.

One key condition: copies must be stateless. If the session lives in process memory, the user gets «logged out» when routed to another copy. Keep state in the database, Redis or a token. Sticky sessions help for a while but break even distribution.