Proxy Rotation for API Calls

API rate limits are per-IP by default. A single IP hitting a public API at 100 requests per minute gets throttled quickly. Distributing those requests across multiple proxy IPs multiplies your effective rate limit proportionally. This guide covers rotation strategies from simple to sophisticated.

Understanding API Rate Limits

Most APIs enforce rate limits at multiple levels: - **Per-IP**: 60-1000 requests per minute per source IP - **Per-API-Key**: Cumulative across all IPs using the same key - **Per-Endpoint**: Different limits for different endpoints - **Burst vs Sustained**: Short burst allowance with lower sustained rate

Proxy rotation addresses per-IP limits. Combined with key rotation (when allowed by ToS), it can multiply your effective throughput dramatically.

Basic Round-Robin Rotation

The simplest approach rotates through proxy configurations sequentially:

from dataclasses import dataclass

@dataclass(frozen=True) class RotatingProxy: username: str password: str gateway: str = "gate.hexproxies.com" port: int = 8080

@property def url(self) -> str: return f"http://{self.username}:{self.password}@{self.gateway}:{self.port}"

class RoundRobinRotator: def __init__(self, proxy: RotatingProxy): """With Hex Proxies residential, each request auto-rotates to a new IP.""" self._proxy = proxy

def next(self) -> str: return self._proxy.url

# With Hex Proxies rotating residential, every request gets a new IP automatically rotator = RoundRobinRotator(RotatingProxy(username="YOUR_USER", password="YOUR_PASS")) ```

Session-Aware Rotation

Some APIs require consistent IP per session (e.g., OAuth flows). Use sticky sessions for these:

class SessionAwareRotator: def __init__(self, username: str, password: str): self._username = username self._password = password

def get_proxy(self, session_key: str = "") -> str: if session_key: session_id = hashlib.md5(session_key.encode()).hexdigest()[:8] return f"http://{self._username}-session-{session_id}:{self._password}@gate.hexproxies.com:8080" return f"http://{self._username}:{self._password}@gate.hexproxies.com:8080" ```

Adaptive Rotation with Backpressure

Monitor response codes and adjust rotation behavior based on API feedback:

import time
from collections import defaultdict

@dataclass(frozen=True) class RequestResult: status: int timestamp: float proxy_session: str

class AdaptiveRotator: def __init__(self, username: str, password: str): self._username = username self._password = password self._rate_limited_until: dict[str, float] = {} self._request_log: list[RequestResult] = [] self._session_counter = 0

def get_proxy(self, domain: str) -> str: """Get a proxy, respecting backpressure from rate-limited domains.""" cooldown_until = self._rate_limited_until.get(domain, 0) if time.monotonic() < cooldown_until: # Domain is rate-limited — force a new session self._session_counter += 1 session = f"adaptive-{self._session_counter}" return f"http://{self._username}-session-{session}:{self._password}@gate.hexproxies.com:8080"

def record_response(self, domain: str, status: int) -> None: result = RequestResult( status=status, timestamp=time.monotonic(), proxy_session=str(self._session_counter), ) self._request_log = [*self._request_log, result] if status == 429: self._rate_limited_until = { **self._rate_limited_until, domain: time.monotonic() + 60, } self._session_counter += 1 ```

Node.js Implementation

class ProxyRotator { constructor(username, password) { this.username = username; this.password = password; this.sessionCounter = 0; }

getAgent(sessionKey = '') { const user = sessionKey ? `${this.username}-session-${sessionKey}` : this.username; const url = `http://${user}:${this.password}@gate.hexproxies.com:8080`; return new HttpsProxyAgent(url); }

getRotatingAgent() { this.sessionCounter++; return this.getAgent(`rotate-${this.sessionCounter}`); } }

// Usage const rotator = new ProxyRotator('YOUR_USER', 'YOUR_PASS'); const agent = rotator.getRotatingAgent();

const response = await fetch('https://api.example.com/data', { agent }); ```

Rate Limit Multiplier Calculation

If an API allows 100 requests/minute per IP and you rotate across fresh IPs, your effective rate is limited only by Hex Proxies' throughput capacity. With our 50B requests/week infrastructure and residential pool offering thousands of IPs, the proxy layer is never your bottleneck.

Best Practices for API Proxy Rotation

1. **Respect ToS**: Always check the API's terms of service. Proxy rotation is a tool — use it for legitimate load distribution.
2. **Monitor 429 Responses**: Track rate limit responses and implement exponential backoff per session.
3. **Use Sticky Sessions for Auth**: OAuth and session-based APIs need consistent IPs — use Hex Proxies sticky sessions.
4. **Log Everything**: Record which proxy session handled each request for debugging.
5. **Set Conservative Limits**: Stay at 80% of the rate limit to avoid triggering burst protections.