v1.10.82-f67ee7d
Skip to main content
← Back to Hex Proxies

Python Proxy Setup

Learn how to configure HTTP and SOCKS5 proxies in Python using requests, urllib, and httpx. Step-by-step setup with Hex Proxies residential IPs.

Get Started

Requirements

  • Python 3.8+
  • pip package manager
  • requests or httpx library

Installation

pip install requests

Code Example

import requests

proxies = {
    "http": "http://user:pass@gate.hexproxies.com:8080",
    "https": "http://user:pass@gate.hexproxies.com:8080"
}

# Basic GET request through proxy
response = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=30)
print(response.json())

# Session-based proxy (reuses connection)
session = requests.Session()
session.proxies.update(proxies)
session.headers.update({"User-Agent": "Mozilla/5.0"})

for url in ["https://example.com", "https://httpbin.org/headers"]:
    resp = session.get(url, timeout=30)
    print(resp.status_code)

Setup Steps

1

Install requests library

Run pip install requests to add the HTTP library. For async support, also install httpx.

2

Build the proxies dictionary

Create a dict mapping "http" and "https" to your Hex Proxies gateway URL with embedded credentials.

3

Create a Session object

Use requests.Session() and call session.proxies.update(proxies) to reuse connections across requests.

4

Verify the connection

Send a GET to httpbin.org/ip and confirm the response shows a Hex Proxies IP, not your origin IP.

5

Add error handling

Catch ProxyError, ConnectTimeout, and SSLError separately. Log each with context for debugging.

6

Move credentials to env vars

Read proxy URL from os.environ["PROXY_URL"] instead of hardcoding in source files.

Configuration Options

OptionDescription
Proxy URLhttp://user:pass@gate.hexproxies.com:8080 -- full endpoint with credentials embedded.
TimeoutPass timeout=30 to every request call. Proxied traffic has additional latency.
SSL VerificationKeep verify=True. Update certifi if you get SSLError on valid certificates.
Connection PoolingUse Session objects. Mount HTTPAdapter(pool_maxsize=20) for concurrent workloads.
Retry LogicAttach urllib3 Retry adapter with backoff_factor=0.5 and status_forcelist=[429, 500, 502, 503].

Best Practices

  • Use requests.Session() to reuse TCP connections and reduce handshake overhead by 30-50ms per request.
  • Store credentials in environment variables using os.environ, never hardcode in source.
  • Mount a urllib3 Retry adapter with exponential backoff for automatic retry on transient failures.
  • Set explicit timeouts on every request call; the default is no timeout, which can hang indefinitely.
  • Use per-request rotation for scraping and sticky sessions for login flows or multi-step form submissions.
  • Log response status codes and latency per request to monitor proxy health over time.
  • Pin your requests and urllib3 versions to avoid breaking changes in proxy handling.
  • For high-concurrency scraping, switch to httpx.AsyncClient or aiohttp for event-loop-based parallelism.

Python Proxy Setup

Python is the most popular language for web scraping and data collection, with three major HTTP libraries that each handle proxies differently. The requests library uses a dictionary-based proxy configuration, urllib relies on environment variables or ProxyHandler objects, and httpx supports both sync and async proxying through a unified client interface. Understanding these differences lets you pick the right tool for your workload.

This guide walks you through configuring Python to route HTTP and HTTPS requests through Hex Proxies, including installation, authentication, error handling, and production best practices.

Prerequisites

Before you begin, make sure you have: - An active Hex Proxies account with proxy credentials - Python 3.8+ - pip package manager - requests or httpx library

Installation

pip install requests

Basic Proxy Configuration

The requests library accepts a proxies dictionary mapping protocol schemes to proxy URLs. Both HTTP and HTTPS traffic should point to gate.hexproxies.com:8080. The library handles CONNECT tunneling for HTTPS automatically.

proxies = { "http": "http://user:pass@gate.hexproxies.com:8080", "https": "http://user:pass@gate.hexproxies.com:8080" }

# Basic GET request through proxy response = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=30) print(response.json())

# Session-based proxy (reuses connection) session = requests.Session() session.proxies.update(proxies) session.headers.update({"User-Agent": "Mozilla/5.0"})

for url in ["https://example.com", "https://httpbin.org/headers"]: resp = session.get(url, timeout=30) print(resp.status_code) ```

Session Objects vs Per-Request Proxies

Python's requests.Session() reuses TCP connections across multiple requests, reducing handshake overhead by 30-50ms per request. For scraping workflows that make hundreds of sequential requests, sessions cut total execution time significantly. Per-request proxy dictionaries create a new connection each time, which is useful when you need a fresh IP on every call but adds latency.

urllib and httpx Alternatives

For stdlib-only deployments, urllib.request.ProxyHandler works without any pip dependencies. For async workloads, httpx.AsyncClient supports the same proxy URL format and integrates with asyncio event loops natively.

Configuration Options

  • **Proxy URL** -- http://user:pass@gate.hexproxies.com:8080 with credentials embedded in the URL string.
  • **Timeout** -- Always pass timeout=30 to requests.get() to prevent indefinite hangs on unresponsive destinations.
  • **SSL Verification** -- Keep verify=True (the default) in production. Disabling it masks certificate errors that could indicate MITM attacks.
  • **Connection Pooling** -- Use requests.Session() to reuse connections. The default pool size is 10; increase via HTTPAdapter for high-concurrency workloads.
  • **Retry Logic** -- Mount a urllib3.util.retry.Retry adapter on your session for automatic retries with backoff.

Error Handling

Python's requests library raises specific exceptions for proxy-related failures that you should catch individually.

  1. requests.exceptions.ProxyError
  2. - Raised when the proxy server is unreachable or rejects the connection
  3. - Verify gate.hexproxies.com:8080 is reachable from your network
  4. - Check that your credentials are URL-encoded if they contain special characters like @ or #

2. requests.exceptions.ConnectTimeout - The proxy accepted the connection but the destination did not respond - Increase timeout from 30 to 60 seconds for slow destinations - Check if the destination is blocking the proxy IP and try a different session

3. requests.exceptions.SSLError - Certificate verification failed during the HTTPS CONNECT tunnel - Update your certifi package: pip install --upgrade certifi - Never set verify=False in production; it silently bypasses security

4. HTTP 407 Proxy Authentication Required - Your credentials were rejected by the proxy gateway - Double-check username and password on the Hex Proxies dashboard - Ensure special characters in credentials are percent-encoded

5. HTTP 429 Too Many Requests - The destination is rate-limiting your proxy IP - Add time.sleep() delays between requests (1-3 seconds) - Switch to per-request rotation to distribute load across IPs ```

Always wrap proxy requests in try/except blocks and log the exception type for debugging.

Error Handling

## Error Handling

Python's requests library raises specific exceptions for proxy-related failures that you should catch individually.

1. requests.exceptions.ProxyError - Raised when the proxy server is unreachable or rejects the connection - Verify gate.hexproxies.com:8080 is reachable from your network - Check that your credentials are URL-encoded if they contain special characters like @ or #

2. requests.exceptions.ConnectTimeout - The proxy accepted the connection but the destination did not respond - Increase timeout from 30 to 60 seconds for slow destinations - Check if the destination is blocking the proxy IP and try a different session

3. requests.exceptions.SSLError - Certificate verification failed during the HTTPS CONNECT tunnel - Update your certifi package: pip install --upgrade certifi - Never set verify=False in production; it silently bypasses security

4. HTTP 407 Proxy Authentication Required - Your credentials were rejected by the proxy gateway - Double-check username and password on the Hex Proxies dashboard - Ensure special characters in credentials are percent-encoded

5. HTTP 429 Too Many Requests - The destination is rate-limiting your proxy IP - Add time.sleep() delays between requests (1-3 seconds) - Switch to per-request rotation to distribute load across IPs ```

Always wrap proxy requests in try/except blocks and log the exception type for debugging.

Frequently Asked Questions

How do I use proxies with Python requests?▾

Pass a proxies dictionary to requests.get() with keys "http" and "https" mapped to your proxy URL. For repeated use, set it on a Session object.

Can I use async proxies in Python?▾

Yes. Use httpx.AsyncClient(proxy="http://user:pass@gate.hexproxies.com:8080") for native async/await proxy support.

How do I rotate IPs in Python?▾

Create a new requests.Session() for each request, or close and reopen the session. Each new connection to the gateway gets a fresh IP from the rotation pool.

What timeout should I use for proxied requests?▾

Start with 30 seconds. Proxied requests add 50-150ms overhead, so timeout values should be higher than direct connections. Increase to 60 seconds for large pages.

Ready to Get Started?

Set up Python with Hex Proxies in minutes.

Create AccountView Pricing

Related Resources

Python Requests Proxy Generator

Create a ready-to-run Python Requests proxy configuration with Hex Proxies.

Residential Proxies

High-quality residential proxies with rotating IPs from 100+ countries. Perfect for web scraping, data collection, and market research.

ISP Proxies

Ultra-fast ISP proxies with static IPs and unlimited bandwidth. Optimized for sneaker sites, social media, and high-speed tasks.

Rotating Proxies

Automatic IP rotation with every request or on a timed interval. Built for large-scale scraping and data collection.