Load Balancing Algorithms for Proxy Pools: Round-Robin, P2C, and the Cases Where Each Wins

A proxy pool is a collection of exit IPs that a request router has to pick from on each request. The picking algorithm determines how evenly load is spread, how well the pool handles heterogeneous exit performance, and how quickly the system can route around a slow or failing exit. The defaults in most off-the-shelf proxy software are round-robin or random, both of which work poorly once exits have meaningfully different latencies or success rates. This post walks through the five algorithms in practical use in proxy infrastructure, explains the math of why power-of-two-choices outperforms pure random, and covers when consistent hashing is the only correct answer.

Round-Robin

Round-robin maintains a counter. Each incoming request increments the counter and picks exit counter mod N. It is trivial to implement, stateless between requests if you are happy with global counter contention, and produces perfectly even distribution when requests are homogeneous and exits are identical.

The failure mode is heterogeneity. If exit 3 is twice as slow as the others, round-robin still sends it one out of every N requests, and requests queued behind exit 3 pile up while other exits sit idle. A pool of 1,000 exits with one slow exit sees about 0.1 percent of requests stuck in the slow queue at any moment. A pool with 10 exits sees 10 percent. Round-robin is a reasonable default only when exits are genuinely interchangeable.

Weighted Round-Robin

Weighted round-robin assigns each exit an integer weight and visits it proportionally more often. If exit A has weight 3 and exit B has weight 1, the sequence is A, A, A, B, A, A, A, B. Weights let you account for known capacity differences (one exit is on a faster uplink) but you have to set and maintain the weights manually, and they do not react to real-time conditions.

Random

Random selection picks a uniformly random exit on each request. It has the same expected distribution as round-robin but without the sequential counter, which eliminates contention in a multi-threaded router. The failure mode is the same: heterogeneous exits are treated identically, and the worst exit receives the same share as the best.

Random has one advantage over round-robin in adversarial settings: it is harder to predict which exit will handle the next request, which matters for certain anti-bot and anti-scraping defenses that look for sequential patterns.

Least-Connections

Least-connections tracks the number of in-flight requests per exit and routes new requests to the exit with the fewest. This adapts to heterogeneity automatically: a slow exit accumulates a backlog, its in-flight count climbs, and the algorithm sends new requests elsewhere until the backlog clears.

The implementation cost is that you need to track per-exit state with atomic updates, and selecting the minimum across N exits is O(N) per request unless you maintain a heap or priority queue. For small pools (< 100 exits) a linear scan is fine. For pools of 10,000+ exits common in residential proxy infrastructure, the selection cost becomes nontrivial.

Least-connections has a subtle pathology: when multiple exits are tied for minimum, the tiebreaker (often the first one scanned) always gets the new request. If exits are ordered consistently, one exit receives all traffic during idle periods. A small random shuffle in the tiebreaker fixes this.

Power-of-Two-Choices (P2C)

Power-of-two-choices is the algorithm that most production-grade load balancers have converged on. The idea is simple: pick two exits uniformly at random, then send the request to whichever one has fewer in-flight connections. You only compare two, so selection is O(1). You get most of the benefit of least-connections without the scan cost.

The mathematical result that makes this work is due to Mitzenmacher, Richa, and Sitaraman (2001). For N bins and M balls thrown uniformly, the maximum load is O(log M / log log M). For N bins and M balls thrown using P2C, the maximum load is log log N + O(1). The improvement is exponential, not incremental. A pool of 10,000 exits receiving uniform traffic has a theoretical max-load of around 4 balls per bin under P2C versus around 20 under pure random.

Python pseudocode:

def choose_exit(pool, inflight):
    a, b = random.sample(pool, 2)
    return a if inflight[a] <= inflight[b] else b

P2C is the right default for proxy pools where exits are interchangeable and you want automatic load-aware routing. It is what Envoy, HAProxy, and Nginx Plus use internally for their "least request" load balancers. Hex Proxies uses P2C by default for the ISP proxy rotation router.

Consistent Hashing

The previous four algorithms assume any exit can serve any request. Consistent hashing exists for the case where requests must be pinned to specific exits based on some identifier, and you need the mapping to be stable even when the pool size changes.

The canonical use case is session stickiness. A client establishes a session cookie with a target website, and subsequent requests in that session must come from the same exit IP or the target will invalidate the session. You hash the session ID modulo the pool size and route to pool[hash mod N]. Works fine until you add or remove an exit: now every session's hash is different, and every session breaks.

Consistent hashing (Karger et al., 1997) solves this by mapping both exits and session IDs onto a ring of 2^32 or 2^64 positions. Each session is routed to the next exit clockwise on the ring. When an exit is added, only sessions between the new exit and its counter-clockwise neighbor are rerouted. When an exit is removed, only sessions on that exit are rerouted. On average, a pool change of 1/N fraction of exits affects 1/N fraction of sessions, not all of them.

The classic implementation places each exit at multiple ring positions (virtual nodes) to smooth out the distribution. 100-200 virtual nodes per exit is typical.

import hashlib
class Ring:
    def __init__(self, exits, vnodes=150):
        self.ring = []
        for exit in exits:
            for i in range(vnodes):
                h = int(hashlib.md5(f'{exit}:{i}'.encode()).hexdigest(), 16)
                self.ring.append((h, exit))
        self.ring.sort()

    def get(self, key):
        h = int(hashlib.md5(key.encode()).hexdigest(), 16)
        for pos, exit in self.ring:
            if pos >= h:
                return exit
        return self.ring[0][1]

For residential proxy providers offering sticky sessions, consistent hashing is the correct underlying data structure. A naive modulo-based router breaks every session on every pool change, which happens continuously in a residential pool as peers connect and disconnect.

Rendezvous Hashing

Rendezvous hashing (Thaler and Ravishankar, 1998) is a simpler alternative with the same rebalancing property. For each request, hash the session ID with each exit name and pick the exit with the highest resulting hash. Adding or removing an exit only affects sessions whose best choice changes, which is O(1/N) on average. It is O(N) per lookup instead of O(log N), which matters for large pools, but the implementation is much simpler and has no tuning knobs.

Health-Aware Variants

Every algorithm above assumes exits are either in the pool or out. Real pools need health awareness: an exit that is slow or returning errors should be deweighted, then removed, then eventually retried.

The standard pattern is an outlier detection layer in front of the load balancer. For each exit, track rolling success rate and latency over the last N requests. If an exit drops below a success threshold or exceeds a latency threshold, temporarily remove it from the pool. Retry after a cooldown. Envoy calls this "outlier detection" and the parameters are configurable per upstream cluster.

For proxy pools specifically, health signals are noisier than for traditional backends because target behavior varies. An exit that returns 403 on target A might return 200 on target B a moment later. Health must be tracked per (exit, target) pair, not just per exit. Hex Proxies tracks success rates per-exit-per-target-class and uses the intersection to drive P2C selection.

Which Algorithm for Which Workload

• High-throughput homogeneous scraping: P2C with per-exit health tracking. Handles variance automatically, O(1) selection cost.
• Account management with sticky sessions: consistent hashing or rendezvous hashing keyed on account ID.
• Mixed batch and interactive workloads: weighted least-connections, with weights tuned to separate latency-sensitive and throughput-sensitive traffic.
• Research workloads that need reproducibility: seeded random. Deterministic given the same seed, useful for debugging.
• Tiny pools under 10 exits: full least-connections. The O(N) cost is trivial at that scale.

Conclusion

Load balancing algorithms are not interchangeable. Round-robin is the wrong default for any pool with heterogeneous exits, which is every real-world proxy pool. Power-of-two-choices is the right default for stateless routing. Consistent hashing is the right default when sessions must stay pinned across pool changes. The difference between a pool that delivers advertised performance and one that collapses under load often comes down to this single design decision, and it is worth spending a day getting it right rather than accepting the default in whatever reverse proxy you happened to deploy first.