Proxy Session Management: Sticky vs Rotating

Session management determines whether each request uses the same IP or a different one. The right choice depends on your use case — and getting it wrong means unnecessary blocks or broken workflows.

Rotating Sessions (New IP Per Request)

Each request goes through a different IP address. Best for broad data collection where no request depends on a previous one.

# Rotating: no session parameter = new IP per request proxy = "http://YOUR_USER:YOUR_PASS@gate.hexproxies.com:8080"

with httpx.Client(proxy=proxy) as client: # Each request uses a different IP r1 = client.get("https://httpbin.org/ip") r2 = client.get("https://httpbin.org/ip") # r1 and r2 will show different IPs ```

Sticky Sessions (Same IP Across Requests)

All requests with the same session ID use the same IP. Essential for multi-step workflows.

# Sticky: session parameter keeps same IP

with httpx.Client(proxy=session_proxy) as client: # All requests use the same IP client.get("https://shop.example.com/cart") client.get("https://shop.example.com/checkout") client.get("https://shop.example.com/confirm") ```

When to Use Each

| Scenario | Session Type | Why | |----------|-------------|-----| | Price scraping across sites | Rotating | Maximum IP diversity | | Login + authenticated scraping | Sticky | Session cookies tied to IP | | Social media account management | Sticky | Platforms track IP consistency | | Search engine scraping | Sticky (per batch) | Natural search behavior | | Product catalog scraping | Rotating | Speed over consistency | | E-commerce checkout | Sticky | Checkout flow needs same IP | | API data collection | Rotating | Distribute rate limits |

Session Pool Pattern

Maintain a pool of sticky sessions, each assigned to a specific task:

from dataclasses import dataclass

@dataclass(frozen=True) class SessionProxy: session_id: str proxy_url: str

class SessionPool: def __init__(self, username: str, password: str): self._username = username self._password = password self._sessions: dict[str, SessionProxy] = {}

def get_session(self, task_key: str) -> SessionProxy: """Get or create a sticky session for a task.""" if task_key in self._sessions: return self._sessions[task_key]

session_id = hashlib.md5(task_key.encode()).hexdigest()[:12] proxy = SessionProxy( session_id=session_id, proxy_url=f"http://{self._username}-session-{session_id}:{self._password}@gate.hexproxies.com:8080", ) self._sessions = {**self._sessions, task_key: proxy} return proxy

def rotate_session(self, task_key: str) -> SessionProxy: """Force a new session for a task (e.g., after being blocked).""" new_key = f"{task_key}-{len(self._sessions)}" session_id = hashlib.md5(new_key.encode()).hexdigest()[:12] proxy = SessionProxy( session_id=session_id, proxy_url=f"http://{self._username}-session-{session_id}:{self._password}@gate.hexproxies.com:8080", ) self._sessions = {**self._sessions, task_key: proxy} return proxy

# Usage pool = SessionPool("YOUR_USER", "YOUR_PASS") amazon_session = pool.get_session("amazon-account-1") ebay_session = pool.get_session("ebay-account-1") ```

Hybrid Strategy: Sticky for Sites, Rotating for Discovery

class HybridSessionManager:
    def __init__(self, username: str, password: str):
        self._pool = SessionPool(username, password)
        self._username = username
        self._password = password

def get_proxy(self, url: str, needs_sticky: bool = False) -> str: if needs_sticky: from urllib.parse import urlparse domain = urlparse(url).netloc return self._pool.get_session(domain).proxy_url return self._rotating_proxy ```

Session Lifetime and Cleanup

Hex Proxies sticky sessions persist for the duration of your plan. Periodically rotate sessions for long-running operations to prevent stale IP assignments:

• **Short tasks** (minutes): Single sticky session is fine
• **Medium tasks** (hours): Rotate session every 2-4 hours
• **Long tasks** (days): Rotate session every 12-24 hours

Monitoring Session Health

Track per-session success rates. If a session starts getting blocked, rotate it immediately rather than continuing to send requests through a degraded IP.