Go net/http Proxy Integration

Go net/http Proxy Setup

Go's standard library `net/http` package provides a production-grade HTTP client with built-in proxy support through the Transport layer. Go is popular for building high-performance scrapers and data pipelines because of its efficient goroutine concurrency model and low memory footprint. Integrating Hex Proxies with Go requires configuring a custom Transport with a proxy URL.

Basic Proxy Configuration

package main

import (
    "fmt"
    "io"
    "net/http"
    "net/url"
    "time"
)

func main() {
    proxyURL, err := url.Parse("http://user:pass@gate.hexproxies.com:8080")
    if err != nil {
        panic(err)
    }

    transport := &http.Transport{
        Proxy: http.ProxyURL(proxyURL),
    }

    client := &http.Client{
        Transport: transport,
        Timeout:   30 * time.Second,
    }

    resp, err := client.Get("https://httpbin.org/ip")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    body, _ := io.ReadAll(resp.Body)
    fmt.Println(string(body))
}

IP Whitelist Authentication

Whitelist your server IP in the Hex Proxies dashboard and use the proxy URL without credentials:

proxyURL, _ := url.Parse("http://gate.hexproxies.com:8080")

SOCKS5 Proxy with Go

Use the `golang.org/x/net/proxy` package for SOCKS5 support:

import "golang.org/x/net/proxy"

auth := proxy.Auth{User: "user", Password: "pass"}
dialer, err := proxy.SOCKS5("tcp", "gate.hexproxies.com:1080", &auth, proxy.Direct)

transport := &http.Transport{
    DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
        return dialer.Dial(network, addr)
    },
}
client := &http.Client{Transport: transport}

Geo-Targeting

Append country codes to your username in the proxy URL:

proxyURL, _ := url.Parse("http://user-country-us:pass@gate.hexproxies.com:8080")

Concurrent Requests with Goroutines

var wg sync.WaitGroup
sem := make(chan struct{}, 20) // Limit to 20 concurrent requests

for _, target := range urls {
    wg.Add(1)
    sem <- struct{}{}
    go func(u string) {
        defer wg.Done()
        defer func() { <-sem }()
        resp, err := client.Get(u)
        if err == nil {
            defer resp.Body.Close()
            // Process response
        }
    }(target)
}
wg.Wait()

Best Practices

• Reuse the http.Client — creating a new client per request wastes TCP connections. Go's client handles connection pooling automatically.
• Set timeouts on both the client (`Timeout`) and transport (`ResponseHeaderTimeout`, `TLSHandshakeTimeout`) to prevent goroutine leaks.
• Rotate IPs by creating new transport instances with different proxy credentials for each request, or use Hex Proxies' automatic rotation.
• Use context.Context for cancellation and timeout propagation across goroutines.

Troubleshooting

• Proxy authentication errors: Ensure credentials are correctly URL-encoded in `url.Parse`. Special characters in passwords need percent-encoding.
• TLS handshake timeout: Increase `TLSHandshakeTimeout` in the Transport. Residential proxies add latency to the TLS negotiation.
• Too many open files: Increase ulimit when running high-concurrency scrapers. Set `Transport.MaxIdleConns` and `Transport.MaxIdleConnsPerHost`.
• Connection reset: The proxy or target closed the connection. Implement retry logic with exponential backoff.