IP Reputation and ASN Diversity: Why Your Proxy Subnet Matters
IP reputation is the single largest factor determining whether a proxy request succeeds or fails against protected targets. A proxy with a perfect TLS fingerprint and realistic browser behavior will still be blocked if the IP has a poor reputation score. Yet most proxy buyers evaluate providers on pool size alone -- "10 million IPs" -- without understanding that 10 million IPs concentrated in 50 subnets are functionally equivalent to 50 IPs for detection purposes.
This post explains how IP reputation systems work, why ASN and subnet diversity matter more than raw pool size, and how to evaluate these metrics when choosing a proxy provider.
How IP Reputation Works
The Reputation Pipeline
Anti-bot systems and fraud detection platforms maintain IP reputation databases that score every IP address on the internet. The scoring pipeline operates continuously:
Data Sources Scoring Engine Output
───────────── ────────────── ──────
┌─────────────┐
Honeypot networks ────────────>│ │
│ │
Abuse complaint feeds ────────>│ Reputation │──> Score: 0-100
│ Scoring │
Cross-site behavioral data ──>│ Model │──> Classification:
│ │ residential/dc/
Blacklist databases ──────────>│ │ hosting/vpn/proxy
│ │
Registration databases ───────>│ │──> Risk level:
(WHOIS, RIR data) │ │ low/medium/high
└─────────────┘What Determines an IP's Score
Registration data (static). Every IP block is registered in a Regional Internet Registry (RIR) database. The registration record includes the ASN, organization name, and registration date. IPs registered to hosting companies (AWS, DigitalOcean, Hetzner) start with a "datacenter" classification regardless of how they are used. IPs registered to ISPs (Comcast, BT, Deutsche Telekom) start with a "residential" classification.
ASN classification (static). The Autonomous System Number identifies the network operator. Anti-bot vendors maintain ASN classification databases:
| ASN Type | Example Organizations | Base Trust Score |
|---|---|---|
| Consumer ISP | Comcast, AT&T, Vodafone, BT | 80-95 |
| Business ISP | Cogent, Zayo, Lumen | 60-80 |
| Cloud/Hosting | AWS, GCP, Azure, DigitalOcean | 10-30 |
| VPN/Proxy provider | Known VPN ASNs | 5-20 |
| CDN | Cloudflare, Akamai, Fastly | 50-70 |
Behavioral history (dynamic). This is where individual IP reputation diverges from ASN-level defaults. When a specific IP:
- Sends high volumes of requests to protected sites
- Gets caught by honeypot traps
- Triggers CAPTCHA challenges repeatedly
- Appears on abuse complaint lists
...its individual score degrades, independent of its ASN classification. A residential IP that scrapes aggressively can drop from 90 to 20 within hours.
Subnet correlation (dynamic). When multiple IPs in the same /24 subnet (256 addresses) exhibit suspicious behavior simultaneously, the entire subnet's score degrades. This is the primary mechanism that makes subnet diversity critical -- one burned IP in a /24 can taint the other 255 addresses.
Reputation Databases Used by Anti-Bot Vendors
The major reputation data sources as of 2026:
| Database | Operator | Used By |
|---|---|---|
| MaxMind GeoIP2 / minFraud | MaxMind | Nearly universal |
| IPQualityScore | IPQualityScore | Cloudflare, enterprise sites |
| IP2Location | IP2Location | E-commerce, ad platforms |
| Spur.us | Spur | HUMAN (PerimeterX), DataDome |
| Shodan InternetDB | Shodan | Security-focused sites |
| Project Honey Pot | Unspam | Open-source integration |
| Spamhaus XBL/CSS | Spamhaus | Email platforms, some WAFs |
Why ASN Diversity Matters
The /24 Subnet Problem
Consider two proxy providers:
Provider A: 100,000 IPs across 50 /24 subnets (2,000 IPs per subnet average)
Provider B: 100,000 IPs across 5,000 /24 subnets (20 IPs per subnet average)
When Provider A's customers scrape aggressively, a /24 subnet gets flagged. All 2,000 IPs in that subnet see degraded success rates. The provider's effective pool shrinks from 100,000 to 98,000, then 96,000, accelerating as remaining subnets carry more load and burn faster.
Provider B's customers burn individual /24s at the same rate, but each burned subnet contains only 20 IPs. The effective pool shrinks from 100,000 to 99,980. The diversity acts as a buffer against reputation decay.
The math:
Effective pool life (simplified model):
Provider A:
Subnets: 50
IPs per subnet: 2,000
If 10 subnets burn per month: 20,000 IPs lost/month
Pool half-life: 2.5 months
Provider B:
Subnets: 5,000
IPs per subnet: 20
If 200 subnets burn per month (4x the absolute rate): 4,000 IPs lost/month
Pool half-life: 12.5 monthsEven if Provider B burns subnets at 4 times the absolute rate, their pool lasts 5 times longer because the damage per burned subnet is 100 times smaller.
ASN Diversity Beyond Subnets
Subnet diversity within a single ASN has diminishing returns. If all 5,000 subnets belong to one ASN (one ISP), detection systems can flag the ASN itself when enough subnets show suspicious patterns.
True diversity requires distributing IPs across multiple ASNs from different ISPs, different regions, and different network types:
| Diversity Metric | Why It Matters |
|---|---|
| Number of unique ASNs | Prevents ASN-level flagging |
| Geographic distribution of ASNs | Enables geo-targeted scraping |
| ISP diversity (different parent companies) | Prevents upstream provider-level blocking |
| /24 subnet count | Limits blast radius of individual IP burns |
| IPs per /24 ratio | Lower is better (less correlated damage) |
How to Evaluate Provider Network Diversity
Metrics to Request
When evaluating a proxy provider, ask for these specific metrics:
- Total unique ASNs in the residential pool
- Total unique /24 subnets in the residential pool
- IP-to-subnet ratio (total IPs / total subnets)
- Gini coefficient of ASN distribution (measures how evenly IPs are distributed across ASNs; lower is more diverse)
- Percentage of IPs in the top 10 ASNs (high concentration = low effective diversity)
Testing Diversity Yourself
You can empirically test a provider's diversity by sampling IPs and analyzing the ASN distribution:
import requests
import json
from collections import Counter
from dataclasses import dataclass
@dataclass(frozen=True)
class IPInfo:
"""Immutable IP information record."""
ip: str
asn: int
asn_org: str
subnet_24: str
country: str
def sample_proxy_ips(proxy_url, sample_size=500):
"""Sample IPs from a proxy provider and collect ASN info.
Args:
proxy_url: Proxy URL with rotating credentials.
sample_size: Number of IPs to sample.
Returns:
Tuple of IPInfo records.
"""
samples = []
seen_ips = set()
for i in range(sample_size):
try:
resp = requests.get(
"https://ipinfo.io/json",
proxies={"https": proxy_url},
timeout=15,
)
data = resp.json()
ip = data.get("ip", "")
if ip in seen_ips:
continue # Skip duplicates
seen_ips.add(ip)
# Extract /24 subnet
octets = ip.split(".")
subnet_24 = f"{octets[0]}.{octets[1]}.{octets[2]}.0/24"
# Parse ASN
org_str = data.get("org", "")
asn_parts = org_str.split(" ", 1)
asn_num = int(asn_parts[0].replace("AS", "")) if asn_parts[0].startswith("AS") else 0
asn_org = asn_parts[1] if len(asn_parts) > 1 else org_str
info = IPInfo(
ip=ip,
asn=asn_num,
asn_org=asn_org,
subnet_24=subnet_24,
country=data.get("country", ""),
)
samples.append(info)
except Exception:
continue # Skip failed requests
return tuple(samples)
def analyze_diversity(samples):
"""Analyze the network diversity of sampled IPs."""
if not samples:
return {"error": "No samples collected"}
asn_counts = Counter(s.asn for s in samples)
subnet_counts = Counter(s.subnet_24 for s in samples)
country_counts = Counter(s.country for s in samples)
total = len(samples)
unique_asns = len(asn_counts)
unique_subnets = len(subnet_counts)
# Top 10 ASN concentration
top_10_asn_ips = sum(count for _, count in asn_counts.most_common(10))
top_10_concentration = top_10_asn_ips / total * 100
# IP-to-subnet ratio
ip_subnet_ratio = total / unique_subnets if unique_subnets > 0 else float('inf')
return {
"total_samples": total,
"unique_ips": total,
"unique_asns": unique_asns,
"unique_subnets": unique_subnets,
"unique_countries": len(country_counts),
"ip_to_subnet_ratio": round(ip_subnet_ratio, 2),
"top_10_asn_concentration": round(top_10_concentration, 1),
"top_5_asns": [
{"asn": asn, "count": count, "pct": round(count / total * 100, 1)}
for asn, count in asn_counts.most_common(5)
],
}
# Example usage
samples = sample_proxy_ips(
"http://USER:PASS@gate.hexproxies.com:8080",
sample_size=500,
)
report = analyze_diversity(samples)
print(json.dumps(report, indent=2))What Good Diversity Looks Like
Based on our analysis of 12 proxy providers (Hex Proxies internal testing, February 2026):
| Metric | Poor | Adequate | Excellent |
|---|---|---|---|
| Unique ASNs per 1,000 sampled IPs | < 50 | 50-200 | > 200 |
| IP-to-subnet ratio | > 20 | 5-20 | < 5 |
| Top 10 ASN concentration | > 60% | 30-60% | < 30% |
| Countries represented | < 20 | 20-80 | > 80 |
Impact on Success Rates
We tested the same scraping workload against Cloudflare-protected targets using proxy pools with different diversity levels (source: Hex Proxies internal testing, March 2026, 50,000 requests per configuration):
| Pool Configuration | Unique ASNs | IP/Subnet Ratio | Success Rate | Success Rate After 24h |
|---|---|---|---|---|
| Low diversity | 15 ASNs | 35:1 | 91.2% | 72.4% |
| Medium diversity | 120 ASNs | 8:1 | 94.7% | 89.3% |
| High diversity | 500+ ASNs | 2.3:1 | 96.8% | 95.1% |
Practical Recommendations
For Residential Proxy Users
- Ask your provider about subnet diversity, not just pool size. "10 million IPs" means nothing if they are concentrated in 100 subnets.
- Monitor per-session success rates. Sudden drops indicate your assigned IP or its subnet has been flagged. Rotate to a new session immediately.
- Use geo-targeted rotation strategically. Requesting IPs from a specific country narrows the available pool and may reduce diversity. Only geo-target when the use case requires it.
- Distribute scraping load across time. Concentrated bursts from the same subnet are more likely to trigger subnet-level flagging than the same volume spread over hours.
For ISP Proxy Users
- Request IPs across multiple subnets. When purchasing ISP proxies, request allocation across diverse /24 blocks rather than a contiguous range.
- Monitor individual IP health. ISP proxies are static -- if an IP's reputation degrades, it stays degraded until the behavioral signals age out (typically 7-30 days).
- Rotate unused IPs into service gradually. Adding 100 new IPs to active scraping simultaneously looks abnormal. Phase them in over days to build organic behavioral history.
Frequently Asked Questions
Can I check my proxy IP's reputation before using it?
Several free tools provide basic IP reputation scores: IPQualityScore (ipqualityscore.com), Spur.us, and Shodan InternetDB. These give you the same data that anti-bot systems use. Check a sample of your provider's IPs periodically to ensure quality.
Does IP age matter?
Yes. Newly allocated IP addresses have thin reputation profiles -- detection systems are uncertain about them, which can trigger cautious challenges. IPs with 6+ months of clean behavioral history have established trust. This is one advantage of ISP proxies: you hold them long enough to build history.
How quickly does a burned IP recover?
It depends on the reputation database. Behavioral signals typically decay over 7-30 days. If an IP was only flagged in one system (e.g., only Cloudflare's internal database), it may recover in a week. If it was listed on multiple blacklists, recovery can take months.
Is IPv6 proxy support relevant for diversity?
IPv6 provides astronomically more address space, but adoption for web scraping is limited because most target websites still serve IPv4 traffic. IPv6 proxies are useful for specific targets (Google, Facebook) that fully support IPv6, but they do not replace IPv4 diversity for general scraping.
Does Hex Proxies manage IP reputation proactively?
Yes. We monitor the reputation of our IP pool continuously using multiple reputation databases. IPs that show degraded scores are removed from the active rotation pool and either retired or placed in a recovery queue. This is a key differentiator from providers that simply add IPs and let customers discover burned IPs through failed requests. See our IP freshness page for details.
IP reputation and ASN diversity are the invisible infrastructure behind proxy success rates. A provider's network composition matters more than headline pool size. Hex Proxies maintains 1,400+ unique ASNs across 195+ countries with aggressive IP health management. Residential proxies start at $4.25/GB; ISP proxies at $2.08/IP. See our network stats or explore pricing.