Puppeteer Proxy Setup
Puppeteer has a significant advantage over Selenium for proxy authentication: the page.authenticate() method sends Proxy-Authorization headers natively through the Chrome DevTools Protocol, eliminating the need for third-party interception tools. This means lower memory overhead and simpler setup compared to Selenium Wire.
Proxy configuration in Puppeteer happens at two levels. The --proxy-server launch argument routes all browser traffic through the proxy gateway. The page.authenticate() call supplies credentials per page context. This separation means you can change credentials without relaunching the browser, enabling per-tab proxy rotation.
Prerequisites
Before you begin, make sure you have: - An active Hex Proxies account with proxy credentials - Node.js 18+ - npm or yarn - Puppeteer (auto-downloads Chromium)
Installation
npm install puppeteerBasic Proxy Configuration
Pass the proxy server as a Chrome launch argument and authenticate per page using page.authenticate(). This approach requires no additional packages beyond Puppeteer itself.
const browser = await puppeteer.launch({ args: ['--proxy-server=http://gate.hexproxies.com:8080'], headless: 'new', });
const page = await browser.newPage();
// Authenticate with the proxy await page.authenticate({ username: 'user', password: 'pass', });
// Set a realistic viewport and user agent await page.setViewport({ width: 1920, height: 1080 }); await page.setUserAgent( 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) ' + 'AppleWebKit/537.36 (KHTML, like Gecko) ' + 'Chrome/120.0.0.0 Safari/537.36' );
try { await page.goto('https://httpbin.org/ip', { waitUntil: 'networkidle0', timeout: 30000, }); const body = await page.evaluate( () => document.body.innerText ); console.log(body); } finally { await browser.close(); } ```
Request Interception for Advanced Control
Puppeteer's page.setRequestInterception(true) lets you modify, block, or redirect requests before they reach the proxy. This is useful for blocking ads, images, or tracking scripts that waste proxy bandwidth. Combined with Hex Proxies, you can reduce data consumption by 40-60% on media-heavy pages.
Per-Tab Proxy Rotation
Open multiple pages with browser.newPage() and call page.authenticate() with different credentials on each tab. This creates separate proxy sessions per tab without restarting the browser, enabling parallel scraping with different IPs from a single Puppeteer instance.
Configuration Options
- **Proxy Server** -- gate.hexproxies.com:8080 via --proxy-server launch argument.
- **Authentication** -- page.authenticate({ username, password }) per page context.
- **Navigation Timeout** -- page.setDefaultNavigationTimeout(60000) for proxied pages.
- **Viewport** -- page.setViewport({ width: 1920, height: 1080 }) for realistic rendering.
- **User Agent** -- page.setUserAgent() with a current Chrome user agent string.
Error Handling
Puppeteer errors during proxied navigation throw specific error types that indicate the failure point.
- net::ERR_PROXY_CONNECTION_FAILED
- - Chromium cannot establish a connection to the proxy server
- - Verify gate.hexproxies.com:8080 is reachable; test with curl
- - Check that no firewall blocks outbound connections on port 8080
2. Navigation Timeout Exceeded (30000ms) - The page did not finish loading within the timeout - Increase timeout in page.goto() options to 60000 - Use waitUntil: 'domcontentloaded' instead of 'networkidle0' for faster resolution
3. net::ERR_TUNNEL_CONNECTION_FAILED - The proxy rejected the CONNECT tunnel for HTTPS - Verify proxy credentials are correct in page.authenticate() - Check that the proxy supports HTTPS CONNECT on port 8081
4. Page Crash (Page.crash event) - Chromium ran out of memory rendering a heavy page through the proxy - Block images and CSS with request interception to reduce memory usage - Use --disable-dev-shm-usage flag in Docker environments
5. Protocol Error: Target closed - The browser was closed while a navigation was in progress - Ensure browser.close() is called only after all page operations complete - Use try/finally to guarantee cleanup ordering ```
Listen for page.on('pageerror') and page.on('error') events to catch runtime errors during proxied navigation.