Proxy Patterns for MCP Tools and Function Calling

Proxy Patterns for MCP Tools and Function-Calling Stacks

The MCP server setup guide covers wiring a proxy into a single web-fetching tool. Production tool-use stacks hit problems that setup guides don't: different tools need different network identities, a model re-fetches the same URL five times in one conversation, and one flaky fetch in the middle of a six-tool chain wastes the whole chain's token spend. This guide is a catalog of four patterns that address those, in the order you will need them.

Where Proxies Sit in a Tool-Use Stack

In both MCP servers and direct function-calling integrations, the tool implementation is the only place network traffic originates — the model never fetches anything itself. That makes the tool layer the single enforcement point for proxy policy: which pool a request uses, what gets cached, and what error shape the model sees. Everything below lives in that layer.

Pattern 1: Per-Tool Proxy Pools

Different tools have different network requirements, and a shared global proxy config forces them all to the lowest common denominator. Map each tool to the pool that matches its workload:

• fetch_docs — hits developer documentation repeatedly; a dedicated ISP IP with unlimited bandwidth is ideal (large pages, high repeat rate, US egress fine).
• check_price — needs geo-accurate retail pages; rotating residential with a country port.
• fetch_account_page — authenticated; must hold one sticky identity (see Pattern 2).

import { ProxyAgent, type Dispatcher } from 'undici';

type PoolName = 'isp-static' | 'resi-rotating' | 'resi-sticky';

function buildPools(): Record<PoolName, Dispatcher> {
  const user = process.env.HEX_PROXY_USER!;
  const pass = process.env.HEX_PROXY_PASS!;
  const gateway = 'gate.hexproxies.com:8080';
  return {
    'isp-static': new ProxyAgent('http://' + user + ':' + pass + '@isp.hexproxies.com:8080'),
    'resi-rotating': new ProxyAgent('http://' + user + ':' + pass + '@' + gateway),
    'resi-sticky': new ProxyAgent(
      'http://' + user + '-session-conv-default:' + pass + '@' + gateway,
    ),
  };
}

const pools = buildPools();

const TOOL_POOL: Record<string, PoolName> = {
  fetch_docs: 'isp-static',
  check_price: 'resi-rotating',
  fetch_account_page: 'resi-sticky',
};

async function toolFetch(toolName: string, url: string): Promise<string> {
  const dispatcher = pools[TOOL_POOL[toolName] ?? 'resi-rotating'];
  const response = await fetch(url, { dispatcher } as RequestInit);
  if (!response.ok) {
    throw new Error('HTTP ' + response.status + ' fetching ' + url);
  }
  return await response.text();
}

The mapping table is policy-as-data: adding a tool means adding one line, and auditing which tools share network identity takes one glance.

Pattern 2: Conversation-Scoped Sticky Sessions

Tools are stateless between calls, but conversations are not. When a model calls search_site then open_result then download_report, those three tool calls form one logical browsing session — if each gets a different IP, sites that bind state to IP will break the chain in the middle. Scope a sticky session to the conversation and pass it through your tool context:

import hashlib
import os

import httpx

def conversation_client(conversation_id: str) -> httpx.AsyncClient:
    """All tools in one conversation share one egress identity."""
    sessid = hashlib.sha256(f"conv:{conversation_id}".encode()).hexdigest()[:12]
    user = os.environ["HEX_PROXY_USER"]
    password = os.environ["HEX_PROXY_PASS"]
    proxy = f"http://{user}-session-{sessid}:{password}@gate.hexproxies.com:8080"
    return httpx.AsyncClient(proxy=proxy, timeout=30.0, follow_redirects=True)

This is the session-stability principle scaled down to tool-use: the long-running loop is the conversation, and identity must survive it. Stateless lookup tools (Pattern 1's check_price) can stay on rotation; anything that continues what a previous tool call started belongs on the conversation session.

Pattern 3: Tool-Result Caching in Front of the Proxy

Models re-request. In long conversations the model routinely re-fetches a URL it saw twenty turns ago — context got compacted, or it simply wants to re-check a quote. Without a cache, every re-request is metered bandwidth and added latency. Put a TTL cache in the tool implementation, keyed on the normalized request, and make the TTL content-aware:

interface CacheEntry { body: string; expiresAt: number }

const cache = new Map<string, CacheEntry>();

function ttlFor(url: string): number {
  if (url.includes('/docs/') || url.endsWith('.md')) return 60 * 60 * 1000; // docs: 1h
  if (url.includes('price') || url.includes('search')) return 60 * 1000;    // volatile: 1m
  return 10 * 60 * 1000;                                                    // default: 10m
}

async function cachedToolFetch(toolName: string, url: string): Promise<string> {
  const key = toolName + ':' + new URL(url).toString();
  const hit = cache.get(key);
  if (hit && hit.expiresAt > Date.now()) return hit.body;

  const body = await toolFetch(toolName, url); // Pattern 1's pooled fetch
  cache.set(key, { body, expiresAt: Date.now() + ttlFor(url) });
  return body;
}

Include the tool name in the cache key: two tools fetching the same URL through different pools may legitimately see different content (geo-variant pages), and a shared key would leak one pool's view into the other.

Pattern 4: Failure Recovery in Multi-Tool Chains

A six-step tool chain with a 95%-reliable fetch step completes only ~74% of the time if failures abort the chain. The recovery rules that matter:

1. Retry inside the tool, not via the model. A model-visible failure costs a full reasoning round-trip; an in-tool retry with a fresh IP costs milliseconds.
2. Return structured, actionable errors when retries are exhausted. "HTTP 403" teaches the model nothing; "this site is blocking automated access — try the search tool for an alternative source" redirects the chain productively.
3. Never silently substitute. Returning stale cache or a different source without labeling it corrupts downstream tool calls that trust the result.

import httpx

async def resilient_tool_fetch(client_factory, url: str, attempts: int = 3) -> dict:
    """Returns a structured result the model can reason about."""
    last_error = "unknown"
    for attempt in range(attempts):
        # fresh client per attempt -> fresh egress IP on rotation
        async with client_factory() as client:
            try:
                response = await client.get(url)
                if response.status_code == 200:
                    return {"ok": True, "content": response.text[:50_000]}
                if response.status_code in (403, 429):
                    last_error = f"blocked ({response.status_code})"
                    continue
                return {"ok": False, "error": f"HTTP {response.status_code}", "retryable": False}
            except httpx.TimeoutException:
                last_error = "timeout"
                continue
    return {
        "ok": False,
        "error": f"unreachable after {attempts} attempts ({last_error})",
        "suggestion": "The site may block automated access. Consider an alternative source.",
        "retryable": False,
    }

Choosing Pool Types

Observability Per Tool, Not Per Server

Aggregate proxy metrics hide the problems that matter in tool-use stacks. Export four series with a tool label: fetch latency (p50/p95 — slow tools stall whole conversations because the model waits synchronously), proxy error rate (a single tool spiking usually means one target started blocking, not a network issue), cache hit rate, and bytes through the proxy (your cost attribution per tool). When something regresses, the per-tool breakdown turns "the agent feels slow" into "check_price's target added a challenge page on Tuesday" in one query.

Deployment Checklist

Before shipping: credentials come from environment variables, never tool source; every tool has a per-domain rate limit (the model will happily loop a tool); cache hit rate and proxy error rate are exported as metrics per tool; and tool descriptions tell the model what the tool can and cannot reach, which reduces doomed calls. If you orchestrate through LangChain or LlamaIndex rather than raw MCP, the same four patterns apply at the loader level — see proxies for LangChain and LlamaIndex, and for MCP data-server architecture itself, the MCP data servers post.