Why SuperAgent for Proxy Workflows

SuperAgent's chainable API makes it one of the most readable HTTP clients in the Node.js ecosystem. For proxy-based workflows, readability matters because proxy configuration, header manipulation, retry logic, and response handling often stack together in complex request chains. SuperAgent's fluent interface keeps these configurations scannable and maintainable.

The library integrates with proxies through the superagent-proxy plugin, which extends every request with a .proxy() method. This per-request proxy assignment is more flexible than agent-level configuration because you can route different requests through different proxy endpoints in the same chain without managing multiple agent instances.

Complete Configuration Example

import request from 'superagent';

proxy(request);

const proxyUrl = \`http://\${process.env.PROXY_USER}:\${process.env.PROXY_PASS}@gate.hexproxies.com:8080\`;

async function fetchWithProxy(url) { try { const response = await request .get(url) .proxy(proxyUrl) .set('User-Agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)') .set('Accept', 'text/html,application/xhtml+xml') .set('Accept-Language', 'en-US,en;q=0.9') .timeout({ response: 20000, deadline: 30000 }) .retry(3, (err, res) => { if (err) return true; return res && [403, 429, 503].includes(res.status); }) .buffer(true) .parse(request.parse['text/html']);

console.log(\`[\${response.status}] \${url} - \${response.text.length} bytes\`); return response.text; } catch (err) { console.error(\`Failed: \${url} - \${err.message}\`); return null; } }

const urls = ['https://example.com', 'https://example.org']; const results = await Promise.all(urls.map(fetchWithProxy)); ```

SuperAgent-Specific Proxy Features

SuperAgent's .retry() method accepts a custom retry function that receives both the error and the response object. This is uniquely useful for proxy workflows because you can retry on specific HTTP status codes (like 403 from a block) without retrying on legitimate 404s. Most other HTTP clients only retry on network errors by default and require additional configuration for status-code-based retries.

The .proxy() method from superagent-proxy handles both HTTP and HTTPS targets automatically, selecting the correct tunneling strategy based on the target URL scheme. This eliminates the need to manage separate HTTP and HTTPS proxy agents that other Node.js clients require.

Common Pitfalls with SuperAgent

The most frequent mistake is forgetting to call proxy(request) to extend SuperAgent before using .proxy() on requests. Without this initialization step, the .proxy() method does not exist and your requests go direct with no error. This silent failure makes it difficult to diagnose during initial setup.

Another pitfall is SuperAgent's response parsing. By default, SuperAgent tries to parse responses based on content type. When scraping HTML through a proxy, the default parser may attempt to parse HTML as JSON if the content type header is missing or incorrect. Always set .buffer(true) and provide an explicit parser via .parse() to avoid this.

Timeout Configuration for Proxy Connections

SuperAgent supports two timeout values: response (time until first byte) and deadline (total request duration). For proxy connections, set the response timeout to at least 20 seconds because the proxy CONNECT handshake adds latency before the first byte arrives. The deadline should be response timeout plus your expected page load time.

Debugging Proxy Connections

SuperAgent does not log proxy connection details by default. Enable the debug module with DEBUG=superagent* node app.js to see the full request lifecycle including proxy tunneling. This is invaluable when diagnosing connection failures that occur during the proxy handshake before the upstream request even starts.