Proxies for CI/CD Pipelines

Proxies for CI/CD Pipelines

CI/CD runners execute from cloud provider IP ranges — AWS, GCP, Azure. These IPs are frequently rate-limited or blocked by external services, and they always originate from the same region. Integrating proxy infrastructure into your pipeline enables reliable external API testing, geo-verification, and distributed E2E test execution.

GitHub Actions Configuration

name: Geo and Integration Tests
on: [push, pull_request]

jobs:
  geo-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install dependencies
        run: pip install httpx pytest

      - name: Run geo-targeted tests
        env:
          PROXY_USER: ${{ secrets.HEX_PROXY_USER }}
          PROXY_PASS: ${{ secrets.HEX_PROXY_PASS }}
        run: pytest tests/geo/ -v

      - name: Run external API integration tests
        env:
          HTTP_PROXY: http://${{ secrets.HEX_PROXY_USER }}:${{ secrets.HEX_PROXY_PASS }}@gate.hexproxies.com:8080
          HTTPS_PROXY: http://${{ secrets.HEX_PROXY_USER }}:${{ secrets.HEX_PROXY_PASS }}@gate.hexproxies.com:8080
        run: pytest tests/integration/ -v

GitLab CI Configuration

geo_tests:
  image: python:3.12
  stage: test
  variables:
    HTTP_PROXY: "http://$HEX_PROXY_USER:$HEX_PROXY_PASS@gate.hexproxies.com:8080"
    HTTPS_PROXY: "http://$HEX_PROXY_USER:$HEX_PROXY_PASS@gate.hexproxies.com:8080"
  script:
    - pip install httpx pytest
    - pytest tests/geo/ -v
  only:
    - merge_requests
    - main

Jenkins Pipeline

pipeline {
    agent any
    environment {
        HEX_PROXY_USER = credentials('hex-proxy-user')
        HEX_PROXY_PASS = credentials('hex-proxy-pass')
        HTTP_PROXY = "http://${HEX_PROXY_USER}:${HEX_PROXY_PASS}@gate.hexproxies.com:8080"
        HTTPS_PROXY = "http://${HEX_PROXY_USER}:${HEX_PROXY_PASS}@gate.hexproxies.com:8080"
    }
    stages {
        stage('Geo Tests') {
            steps {
                sh 'pytest tests/geo/ -v'
            }
        }
    }
}

Writing Proxy-Aware Tests

import os
import httpx
import pytest

PROXY_USER = os.environ.get("PROXY_USER", "")
PROXY_PASS = os.environ.get("PROXY_PASS", "")

def get_proxy_url(country: str = "") -> str:
    user = f"{PROXY_USER}-country-{country.lower()}" if country else PROXY_USER
    return f"http://{user}:{PROXY_PASS}@gate.hexproxies.com:8080"

@pytest.mark.parametrize("country,expected_currency", [
    ("US", "USD"),
    ("GB", "GBP"),
    ("DE", "EUR"),
    ("JP", "JPY"),
])
def test_pricing_page_currency(country: str, expected_currency: str):
    proxy = get_proxy_url(country)
    with httpx.Client(proxy=proxy, timeout=30) as client:
        resp = client.get("https://yoursite.com/pricing")
        assert resp.status_code == 200
        assert expected_currency in resp.text

def test_api_rate_limit_not_hit():
    """Verify external API calls from CI succeed through proxy."""
    proxy = get_proxy_url()
    with httpx.Client(proxy=proxy, timeout=30) as client:
        resp = client.get("https://api.external-service.com/v1/data")
        assert resp.status_code != 429, "Rate limited — proxy rotation needed"
        assert resp.status_code == 200

Security: Secrets Management

Never hardcode proxy credentials in pipeline files. Use your CI platform's secrets manager:

Performance Considerations

CI pipelines are time-sensitive. Use ISP proxies for the lowest latency — sub-50ms overhead keeps your pipeline fast. For geo-specific tests that require country targeting, residential proxies add slightly more latency but provide accurate geographic placement.

Hex Proxies' multi-Gbps capacity comfortably absorbs CI pipeline test traffic.