How to Manage Proxy IP Reputation
IP reputation is the invisible currency of proxy operations. A clean IP with good reputation passes anti-bot checks effortlessly. A burned IP gets blocked on sight. Managing reputation is the difference between 95% and 50% success rates.
How IP Reputation Works
Anti-bot systems maintain databases that track every IP's behavior:
- **Request Volume**: How many requests per minute from this IP?
- **Request Patterns**: Are requests machine-like (regular intervals) or human-like (variable)?
- **Target Diversity**: Is this IP hitting one site or many?
- **Historical Behavior**: Has this IP been associated with bot activity before?
- **ASN Classification**: Is this IP from a hosting provider, ISP, or mobile carrier?
Rule 1: Pace Your Requests
The single most impactful reputation practice — never exceed natural request rates:
import asyncioasync def human_paced_request(url: str, proxy: str) -> str: """Add human-like delays between requests.""" delay = random.uniform(2.0, 5.0) # 2-5 seconds random delay await asyncio.sleep(delay)
async with httpx.AsyncClient(proxy=proxy, timeout=30) as client: resp = await client.get(url) return resp.text ```
Rule 2: Rotate Intelligently
Do not rotate IPs on every request — this looks unnatural. Match rotation to your use case:
| Use Case | Rotation Strategy | Why | |----------|------------------|-----| | Account management | No rotation (sticky) | Accounts need consistent IPs | | Price monitoring | Rotate per domain | Different site = different IP | | Broad scraping | Rotate per request | Maximum diversity | | Search queries | Rotate per batch (10-20) | Mimics search session |
Rule 3: Maintain Human Behavioral Patterns
Anti-bot systems analyze request patterns beyond just rate:
def build_realistic_headers(referer: str = "") -> dict: """Build headers that mimic real browser behavior.""" headers = { "User-Agent": random.choice([ "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36", ]), "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Language": "en-US,en;q=0.9", "Accept-Encoding": "gzip, deflate, br", "Connection": "keep-alive", "Sec-Fetch-Dest": "document", "Sec-Fetch-Mode": "navigate", "Sec-Fetch-Site": "none" if not referer else "same-origin", } if referer: headers["Referer"] = referer return headers ```
Rule 4: Monitor Success Rates
Track success rates per domain to detect reputation degradation:
from collections import defaultdict@dataclass(frozen=True) class DomainStats: domain: str total: int successful: int blocked: int rate_limited: int
class ReputationMonitor: def __init__(self): self._stats: dict[str, dict[str, int]] = defaultdict( lambda: {"total": 0, "success": 0, "blocked": 0, "rate_limited": 0} )
def record(self, domain: str, status: int) -> None: s = self._stats[domain] new_stats = {**s, "total": s["total"] + 1} if 200 <= status < 400: new_stats["success"] = s["success"] + 1 elif status == 403: new_stats["blocked"] = s["blocked"] + 1 elif status == 429: new_stats["rate_limited"] = s["rate_limited"] + 1 self._stats[domain] = new_stats
def get_stats(self, domain: str) -> DomainStats: s = self._stats[domain] return DomainStats( domain=domain, total=s["total"], successful=s["success"], blocked=s["blocked"], rate_limited=s["rate_limited"], )
def success_rate(self, domain: str) -> float: s = self._stats[domain] return s["success"] / max(s["total"], 1) ```
Rule 5: Use Dedicated IPs for High-Value Targets
ISP proxies provide dedicated IP addresses that no one else uses. This prevents other users' behavior from contaminating your IP reputation. For your most important targets, dedicated ISP proxies from Hex Proxies — backed by Comcast, Windstream, RCN, and Frontier IPs — provide the cleanest reputation possible.
Recovery from Burned IPs
If an IP's reputation drops: 1. Stop using it immediately for the affected domain 2. Rotate to a new session (residential) or request a new IP (ISP) 3. Reduce request rate for the domain 4. Wait 24-48 hours before retrying with the original IP
Hex Proxies Reputation Advantage
Our ISP proxies are dedicated — each IP is assigned to one user. No shared usage means no reputation contamination. Our residential pool of thousands of IPs provides fresh addresses on every rotation. With 50 billion requests per week across our infrastructure, we actively monitor and maintain IP health across our entire network.