Proxy API Service

🔗 View Source

Overview

Proxy API Service is a FastAPI-based application that collects, filters, and tests public HTTP/SOCKS proxies from GitHub repositories, delivering raw and verified proxy lists through simple REST endpoints.

With automated scraping and health-checking, it empowers developers needing reliable proxy infrastructure for scraping, automation, or anonymity, providing both comprehensive and quality-verified proxy data.

Key Features

  • Scrapes proxy files from recently updated GitHub repositories
  • Extracts public proxy IP:PORT using smart filtering and regex
  • Validates proxies concurrently across HTTP and SOCKS protocols
  • Provides secured download endpoints for raw and working lists
  • Automates background tasks for scraping and verification

Purpose & Vision

The purpose of Proxy API Service is to simplify proxy list gathering and validation, saving developers time and effort. It envisions a world where reliable proxies are accessible via a secure, programmatic API.

The vision is to evolve into a resilient, customizable platform supporting advanced filtering, usage analytics, and extensible proxy sources for a wider range of automation and privacy tools.

Technologies Used

  • FastAPI — Web API framework
  • httpx — For async HTTP client operations
  • PySocks — To test SOCKS proxies (SOCKS4/SOCKS5)
  • dotenv — For managing secrets and environment variables
  • ThreadPoolExecutor — Enables concurrent proxy validation

Workflow

  1. Scrape GitHub repos for proxy list files using custom keywords and filters
  2. Extract proxy IP:PORT strings using regex and valid file extensions
  3. Validate proxies concurrently via HTTP/SOCKS tests using DNS ping and httpbin methodologies
  4. Save all gathered proxies to output/raw.txt and working ones to output/proxies.txt
  5. Serve raw and verified lists via password-protected REST endpoints

Results & Impact

  • Delivers reliable, real-time proxy data via a programmable interface
  • Eliminates manual scraping and verification overhead
  • Makes secure proxy access effortlessly automatable for workflows

Future Enhancements

  • Support advanced filtering (location, anonymity level, latency)
  • Add API analytics to track usage, success rates, and response times
  • Enable pluggable proxy sources, beyond GitHub (e.g., RSS feeds, public APIs)

Conclusion

Proxy API Service offers a robust, automated solution for collecting and validating proxies via a secure REST API. Its modular design and clear architecture make it ideal for developers seeking reliable proxy infrastructure with minimal setup.