How to Set Up Rotating Proxies: Step-by-Step Guide

Rotating proxies automatically cycle through a pool of IP addresses, assigning a different IP for each request or at defined time intervals. This is one of the most effective ways to distribute traffic, avoid IP blocks, and collect data at scale. In this tutorial, we'll walk through everything from initial configuration to advanced rotation strategies, with working code examples in Python, JavaScript, and cURL.

What You'll Need

Before starting, make sure you have:

• A proxy plan with rotating proxy access (gateway-based rotation is easiest to configure)
• Your proxy credentials (host, port, username, password)
• Python 3.10+ or Node.js 20+ installed (for code examples)
• Basic familiarity with HTTP requests

If you don't have a proxy plan yet, Hex Proxies offers rotating residential and ISP proxy plans with flexible rotation options.

Understanding Rotation Methods

There are two primary ways rotating proxies work, and the method affects how you configure them.

Gateway-Based Rotation

With gateway rotation, you connect to a single proxy endpoint (the "gateway"), and the proxy provider handles IP assignment behind the scenes. Each request through the gateway gets routed through a different IP from the pool.

This is the simplest approach because you only need to configure one proxy address. The provider's infrastructure manages the pool, rotation logic, and IP health monitoring.

How it works:

Your Request → Provider Gateway (gate.hexproxies.com:8080) → Random IP from Pool → Target Website

Proxy List Rotation

With list rotation, the provider gives you a list of individual proxy IP:port combinations, and you rotate through them in your own code. This gives you more control but requires more implementation work.

How it works:

Your Request → Your Rotation Logic → proxy-1.hexproxies.com:8080 → Target Website
Your Request → Your Rotation Logic → proxy-2.hexproxies.com:8081 → Target Website

For most users, gateway-based rotation is the recommended approach. It's simpler to configure and the provider handles IP health monitoring automatically.

Step 1: Get Your Proxy Credentials

After purchasing a rotating proxy plan, you'll receive credentials in your dashboard. They typically look like this:

Gateway Host: gate.hexproxies.com
Gateway Port: 8080
Username: YOUR_USERNAME
Password: YOUR_PASSWORD

Note: The gateway address, port, and credentials shown in this guide are examples. Your actual proxy credentials are available in the Hex Proxies dashboard after purchasing a plan.

Most providers allow you to append targeting and session parameters to the username string. For example:

YOUR_USERNAME-country-us             → US IPs only
YOUR_USERNAME-country-us-state-ny    → New York IPs only
YOUR_USERNAME-session-abc123         → Sticky session (same IP for session duration)

Step 2: Test Your Connection

Before writing any code, verify your proxy works with a simple cURL command:

# Test with basic authentication
curl -x http://YOUR_USERNAME:YOUR_PASSWORD@gate.hexproxies.com:8080 https://httpbin.org/ip

# Test with country targeting
curl -x http://YOUR_USERNAME-country-us:YOUR_PASSWORD@gate.hexproxies.com:8080 https://httpbin.org/ip

# Run multiple times to verify rotation
for i in {1..5}; do
  echo "Request $i:"
  curl -s -x http://YOUR_USERNAME:YOUR_PASSWORD@gate.hexproxies.com:8080 https://httpbin.org/ip
  echo ""
done

Each request should return a different IP address, confirming that rotation is working.

Step 3: Configure in Python

Basic Rotating Proxy with Requests

import requests

PROXY_HOST = "gate.hexproxies.com"
PROXY_PORT = "7777"
PROXY_USER = "YOUR_USERNAME"
PROXY_PASS = "YOUR_PASSWORD"

def get_proxy_config(country=None, session_id=None):
    """Build proxy configuration with optional targeting."""
    username = PROXY_USER

    if country:
        username += f"-country-{country}"
    if session_id:
        username += f"-session-{session_id}"

    proxy_url = f"http://{username}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
    return {"http": proxy_url, "https": proxy_url}


# Per-request rotation (new IP each time)
def fetch_with_rotation(url):
    proxy = get_proxy_config(country="us")
    response = requests.get(url, proxies=proxy, timeout=30)
    return response


# Sticky session (same IP for related requests)
def fetch_with_session(urls, session_id="my-session-001"):
    proxy = get_proxy_config(country="us", session_id=session_id)
    results = []

    for url in urls:
        response = requests.get(url, proxies=proxy, timeout=30)
        results.append(response)

    return results

Concurrent Scraping with Rotation

For higher throughput, use concurrent requests with concurrent.futures:

import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import random
import time

PROXY_HOST = "gate.hexproxies.com"
PROXY_PORT = "7777"
PROXY_USER = "YOUR_USERNAME"
PROXY_PASS = "YOUR_PASSWORD"

def make_proxy_url(country="us"):
    return f"http://{PROXY_USER}-country-{country}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"

def scrape_url(url):
    """Scrape a single URL with rotating proxy."""
    proxy_url = make_proxy_url()
    proxies = {"http": proxy_url, "https": proxy_url}

    headers = {
        "User-Agent": random.choice([
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/133.0.0.0",
            "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) Chrome/133.0.0.0",
            "Mozilla/5.0 (X11; Linux x86_64) Firefox/134.0",
        ])
    }

    try:
        response = requests.get(url, proxies=proxies, headers=headers, timeout=30)
        return {"url": url, "status": response.status_code, "data": response.text}
    except Exception as e:
        return {"url": url, "status": "error", "data": str(e)}

def scrape_batch(urls, max_workers=10):
    """Scrape multiple URLs concurrently with rotating proxies."""
    results = []

    with ThreadPoolExecutor(max_workers=max_workers) as executor:
        future_to_url = {executor.submit(scrape_url, url): url for url in urls}

        for future in as_completed(future_to_url):
            result = future.result()
            results.append(result)

            # Brief delay to be polite
            time.sleep(random.uniform(0.1, 0.5))

    return results

# Usage
urls = [f"https://example.com/products/page/{i}" for i in range(1, 101)]
results = scrape_batch(urls, max_workers=5)

successful = [r for r in results if r["status"] == 200]
print(f"Success rate: {len(successful)}/{len(results)}")

Async Scraping with aiohttp

For even higher concurrency, use async I/O:

import aiohttp
import asyncio
import random

PROXY_URL = "http://YOUR_USERNAME-country-us:YOUR_PASSWORD@gate.hexproxies.com:8080"

async def fetch(session, url):
    try:
        async with session.get(url, proxy=PROXY_URL, timeout=aiohttp.ClientTimeout(total=30)) as response:
            data = await response.text()
            return {"url": url, "status": response.status, "data": data}
    except Exception as e:
        return {"url": url, "status": "error", "data": str(e)}

async def scrape_all(urls, concurrency=20):
    connector = aiohttp.TCPConnector(limit=concurrency)
    async with aiohttp.ClientSession(connector=connector) as session:
        tasks = [fetch(session, url) for url in urls]
        results = await asyncio.gather(*tasks)
    return results

# Usage
urls = [f"https://example.com/item/{i}" for i in range(1, 501)]
results = asyncio.run(scrape_all(urls, concurrency=15))

Step 4: Configure in JavaScript / Node.js

Basic Setup with Axios

const axios = require('axios');
const { HttpsProxyAgent } = require('https-proxy-agent');

const PROXY_HOST = 'gate.hexproxies.com';
const PROXY_PORT = 7777;
const PROXY_USER = 'YOUR_USERNAME';
const PROXY_PASS = 'YOUR_PASSWORD';

function getProxyAgent(country = 'us', sessionId = null) {
  let username = `${PROXY_USER}-country-${country}`;
  if (sessionId) {
    username += `-session-${sessionId}`;
  }

  const proxyUrl = `http://${username}:${PROXY_PASS}@${PROXY_HOST}:${PROXY_PORT}`;
  return new HttpsProxyAgent(proxyUrl);
}

async function fetchWithRotation(url, country = 'us') {
  const agent = getProxyAgent(country);

  const response = await axios.get(url, {
    httpsAgent: agent,
    timeout: 30000,
    headers: {
      'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/133.0.0.0'
    }
  });

  return response.data;
}

// Scrape multiple URLs with concurrency control
async function scrapeBatch(urls, concurrency = 5) {
  const results = [];

  for (let i = 0; i < urls.length; i += concurrency) {
    const batch = urls.slice(i, i + concurrency);
    const promises = batch.map(url => fetchWithRotation(url).catch(err => ({
      error: err.message,
      url
    })));

    const batchResults = await Promise.allSettled(promises);
    results.push(...batchResults);

    // Delay between batches
    await new Promise(resolve => setTimeout(resolve, 1000));
  }

  return results;
}

Using node-fetch with Proxy Support

const fetch = require('node-fetch');
const { HttpsProxyAgent } = require('https-proxy-agent');

const proxyAgent = new HttpsProxyAgent(
  'http://YOUR_USERNAME-country-us:YOUR_PASSWORD@gate.hexproxies.com:8080'
);

async function scrape(url) {
  const response = await fetch(url, {
    agent: proxyAgent,
    headers: {
      'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/133.0.0.0',
      'Accept': 'text/html,application/xhtml+xml',
      'Accept-Language': 'en-US,en;q=0.9',
    }
  });

  if (!response.ok) {
    throw new Error(`HTTP ${response.status}: ${response.statusText}`);
  }

  return response.text();
}

Step 5: Configure Rotation Intervals

Different use cases require different rotation strategies. Here's how to implement each:

Per-Request Rotation (Default)

With gateway-based proxies, per-request rotation is usually the default. Every time you make a new connection through the gateway, you get a new IP. No special configuration needed.

Timed Rotation with Sticky Sessions

To maintain the same IP for a set duration, use sticky sessions. Most providers implement this through a session ID in the username:

import time
import uuid

def create_timed_session(duration_minutes=10):
    """Create a proxy session that rotates after the specified duration."""
    session_id = str(uuid.uuid4())[:8]
    start_time = time.time()
    duration_seconds = duration_minutes * 60

    def get_proxy():
        nonlocal session_id, start_time
        elapsed = time.time() - start_time

        if elapsed > duration_seconds:
            session_id = str(uuid.uuid4())[:8]
            start_time = time.time()

        username = f"YOUR_USERNAME-session-{session_id}"
        return f"http://{username}:YOUR_PASSWORD@gate.hexproxies.com:8080"

    return get_proxy

N-Requests Rotation

Rotate after a fixed number of requests:

class RequestCountRotator:
    def __init__(self, rotate_every=50):
        self.rotate_every = rotate_every
        self.request_count = 0
        self.session_id = self._new_session()

    def _new_session(self):
        import uuid
        return str(uuid.uuid4())[:8]

    def get_proxy(self):
        self.request_count += 1
        if self.request_count >= self.rotate_every:
            self.session_id = self._new_session()
            self.request_count = 0

        username = f"YOUR_USERNAME-session-{self.session_id}"
        return f"http://{username}:YOUR_PASSWORD@gate.hexproxies.com:8080"

Step 6: Troubleshooting Common Issues

Connection Timeouts

If requests frequently time out, try:

1. Increase your timeout value to 60 seconds
2. Switch to a geographically closer proxy location
3. Verify the gateway host and port are correct

407 Proxy Authentication Required

Double-check your username and password. Common issues:

• Special characters in password not URL-encoded
• Username format doesn't match provider's specification
• Credentials expired or plan inactive

High Block Rate Despite Rotation

If you're getting blocked even with rotating proxies:

1. Reduce your request rate
2. Rotate User-Agent headers alongside IPs
3. Add realistic headers (Accept, Accept-Language, Referer)
4. Consider upgrading from datacenter to residential proxies
5. Check our guide on avoiding IP bans

Duplicate IPs

If you notice the same IP appearing frequently, your proxy pool may be too small for your request volume. Contact your provider about pool size, or add a session ID with a random value to force new IP assignment per request.

Best Practices

Start slow and scale up. Begin with low concurrency (3-5 concurrent requests) and increase gradually while monitoring success rates.

Log everything. Track which IPs you receive, response codes, and response times. This data helps you optimize your rotation strategy and identify problems early.

Use appropriate proxy types. Rotating residential proxies are best for large-scale scraping of protected sites. ISP proxies are better for session-based tasks. Datacenter proxies work for low-protection targets. See our web scraping proxy guide for detailed recommendations.

Handle errors gracefully. Never crash on a failed request. Implement retry logic, proxy switching on failure, and exponential backoff.

Respect robots.txt. While proxies give you the ability to access any site, always check and respect the site's robots.txt file and terms of service.

Conclusion

Setting up rotating proxies is straightforward once you understand the mechanics. Gateway-based rotation handles the complexity for you — configure your proxy credentials, add targeting parameters as needed, and let the provider's infrastructure manage IP assignment and health monitoring.

For most use cases, start with gateway-based rotating residential proxies, test your success rates, and adjust your rotation strategy (per-request vs. sticky sessions) based on your specific requirements.

Ready to get started? Check out Hex Proxies rotating residential plans for flexible rotation options, or explore our ISP proxy plans if you need static IPs with rotation managed in your own code.