Proxy Routing for Web-Capable Coding Agents

How Coding Agents Route Web Traffic Through Proxies

Web-capable coding agents — Claude Code running a research loop, Cursor agents fetching documentation, or a custom tool-use harness that calls fetch — generate two completely different kinds of network traffic. Treating them as one stream is the most common proxy misconfiguration in agent deployments, and it causes failures in both directions.

The Two-Channel Problem

Channel A is model-API traffic: authenticated HTTPS requests to api.anthropic.com or api.openai.com. These endpoints expect your traffic, authenticate it with an API key, and never block you for volume within your rate limits. Routing this channel through a scraping proxy adds a network hop, burns metered bandwidth on multi-megabyte streaming responses, and can trip the provider's anomaly detection when your account suddenly originates from consumer IP space.

Channel B is agent-generated web traffic: the pages your agent fetches while working — package registry lookups, documentation pages, GitHub raw files, the staging site it is testing against. This traffic leaves your cloud instance with a datacenter source IP, and that is exactly the signature that anti-bot layers score against. This is the channel that needs a proxy.

The design rule: proxy Channel B, never Channel A.

Channel Separation with Environment Variables

Most HTTP stacks (curl, httpx, requests, Go's net/http) honor the standard proxy environment variables. The NO_PROXY list is what implements channel separation at the process level:

# Route agent web traffic through Hex Proxies
export HTTP_PROXY="http://user:pass@gate.hexproxies.com:8080"
export HTTPS_PROXY="http://user:pass@gate.hexproxies.com:8080"

# Keep model-API and internal traffic direct
export NO_PROXY="api.anthropic.com,api.openai.com,localhost,127.0.0.1,.internal"

This works for development-time agents like Claude Code, which inherit the shell environment: every WebFetch-style tool call and subprocess the agent spawns picks up the proxy, while its own model-API loop stays direct. One caveat: NO_PROXY handling varies across runtimes (some match suffixes, some require exact hosts), so verify with a test request before trusting it.

Wiring the Web Channel Explicitly in Python

Environment variables are process-global. In a custom harness you usually want explicit clients instead, so the two channels can never bleed into each other:

import os
import httpx
from anthropic import Anthropic

# Channel A: model API — direct connection, no proxy.
llm = Anthropic()  # reads ANTHROPIC_API_KEY from the environment

# Channel B: agent web traffic — proxied per agent context.
def make_web_client(context_id: str) -> httpx.Client:
    """One proxied client per agent context, with a sticky session."""
    proxy_user = os.environ["HEX_PROXY_USER"]
    proxy_pass = os.environ["HEX_PROXY_PASS"]
    proxy_url = (
        f"http://{proxy_user}-session-{context_id}:{proxy_pass}"
        "@gate.hexproxies.com:8080"
    )
    return httpx.Client(proxy=proxy_url, timeout=30.0, follow_redirects=True)

def fetch_page_tool(web: httpx.Client, url: str) -> str:
    """Tool implementation the agent loop calls."""
    response = web.get(url)
    response.raise_for_status()
    return response.text[:50_000]

web = make_web_client("ctx-refactor-auth")
result = llm.messages.create(
    model="claude-opus-4-8",
    max_tokens=2048,
    messages=[{"role": "user", "content": "Summarize the httpx timeout docs."}],
)

The model call never touches the proxy; the tool fetch never leaves without it.

Wiring the Web Channel in TypeScript

In Node 18+, scope an undici ProxyAgent to individual fetch calls rather than installing a global dispatcher — a global dispatcher would drag your SDK's model-API calls through the proxy too:

import { ProxyAgent } from 'undici';

const HEX_GATEWAY = 'http://gate.hexproxies.com:8080';

function makeContextDispatcher(contextId: string): ProxyAgent {
  const user = process.env.HEX_PROXY_USER;
  const pass = process.env.HEX_PROXY_PASS;
  if (!user || !pass) throw new Error('HEX_PROXY_USER / HEX_PROXY_PASS not set');
  return new ProxyAgent({
    uri: HEX_GATEWAY,
    token: 'Basic ' + Buffer.from(user + '-session-' + contextId + ':' + pass).toString('base64'),
  });
}

// Tool implementation: only this fetch is proxied.
async function fetchPageTool(contextId: string, url: string): Promise<string> {
  const response = await fetch(url, { dispatcher: makeContextDispatcher(contextId) } as RequestInit);
  if (!response.ok) throw new Error('Fetch failed: ' + response.status);
  return (await response.text()).slice(0, 50_000);
}

Per-Context Identity

A coding agent is not one browsing user — it is many short-lived working contexts. A Claude Code session refactoring one repo, a second session running in a git worktree, and a CI-triggered agent run are three contexts. Give each its own -session- suffix so each holds one IP for its lifetime:

• Requests within a context stay consistent: a docs site that set cookies on request 1 sees the same IP on request 40.
• Contexts do not contaminate each other: if one context trips a rate limit on a registry, the other contexts' IPs are unaffected.
• Long-running tool loops survive: multi-step fetch sequences (login, paginate, download) do not break mid-session. This session-stability property is the core requirement for agent workloads — rotating IPs mid-loop is the single most common cause of tool-chain failures.

Development-Time vs Runtime Routing

These two deployment modes want different IP strategies:

Development-time agents (Claude Code on your laptop, Cursor agents in your IDE) generate low, bursty volume against a small set of mostly developer-facing sites. A single dedicated ISP proxy is the right tool: one static IP registered to a consumer carrier (Comcast, Frontier, Windstream, or RCN), unlimited bandwidth at $2.08–$2.47/IP/month, so an agent that fetches 2 GB of documentation costs the same as one that fetches 20 MB. Hex ISP IPs are US-only (Ashburn VA, New York, San Francisco) — for development workflows that is rarely a constraint.

Runtime agents (production tool loops fetching arbitrary URLs on behalf of users) hit diverse, sometimes heavily protected sites and may need to appear from specific countries. Rotating residential proxies ($4.25–$4.75/GB across 195+ countries) cover that breadth; switch to a sticky session whenever the agent enters a multi-step sequence. The honest trade-off: residential bandwidth is metered, so a runaway agent loop has a direct dollar cost — set per-context byte budgets.

If your agent runs against Claude specifically, the best proxy for Claude page covers the access-side considerations, and the AI web agents guide covers full browser-automation deployments.

Verifying the Split Before You Trust It

Channel separation is easy to get wrong silently — a library that ignores NO_PROXY, a Docker layer that drops the env vars, a global dispatcher someone added later. Verify both directions explicitly:

# 1. Web channel egresses through the proxy (should print a Hex IP)
curl -s https://api.ipify.org

# 2. Model channel stays direct (should print your real machine IP)
curl -s --noproxy '*' https://api.ipify.org

# 3. Confirm the SDK path: a model call should succeed even if the
#    proxy credentials are deliberately broken — if it fails, Channel A
#    is leaking through the proxy.
HEX_PROXY_PASS=wrong python -c "from anthropic import Anthropic; \
print(Anthropic().messages.create(model='claude-opus-4-8', max_tokens=16, \
messages=[{'role':'user','content':'ping'}]).content[0].text)"

Run the third check in CI whenever the harness's HTTP plumbing changes. It is the only test that catches the failure mode where everything works but your metered bandwidth bill quietly includes every streamed model token.

Two environment details bite teams repeatedly. In Docker, ENV lines bake proxy settings into the image — prefer passing them at docker run time so the same image works inside and outside the proxied environment. In CI, runners often inject their own HTTP_PROXY for an internal egress gateway; your agent job must override it or the two proxy layers will chain, which breaks CONNECT tunneling on most gateways.

Credential Hygiene

Agent harnesses are an easy place to leak proxy credentials, because tool code gets logged, traced, and sometimes echoed back into model context. Keep credentials in environment variables (never in tool source or tool descriptions), redact the userinfo portion of proxy URLs in logs and error messages, and never include the proxy URL in any string the model can see — models repeat what they read, and a traced tool error containing user:pass@gate... can surface in a transcript reviewed by people who should not have it.

Compliance Notes

Routing through a proxy does not change your obligations. Respect target sites' robots.txt and terms of service, keep fetch rates polite, and stay within your model provider's usage policies (Anthropic's and OpenAI's policies both apply to what your agent does on the web, not just what it generates). Log which context fetched which URL — provenance makes audits tractable.