### Install FreeProxy from GitHub (Method 2) Source: https://freeproxy.readthedocs.io/en/latest/Install.html Clone the repository and install using the setup script. This method allows for local modifications before installation. ```bash git clone https://github.com/CharlesPikachu/freeproxy.git cd freeproxy python setup.py install ``` -------------------------------- ### Minimal ProxiedSessionClient Example Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt Demonstrates the basic setup and usage of ProxiedSessionClient to fetch content from a URL using a specified proxy source and filter rule. ```python from freeproxy.freeproxy import ProxiedSessionClient proxy_sources = ["KuaidailiProxiedSession"] init_proxied_session_cfg = {"filter_rule": {"country_code": ["CN", "US"]}} client = ProxiedSessionClient( proxy_sources=proxy_sources, init_proxied_session_cfg=init_proxied_session_cfg, ) headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36" } resp = client.get("https://space.bilibili.com/406756145", headers=headers) print(resp.text) ``` -------------------------------- ### Install FreeProxy from GitHub (Method 1) Source: https://freeproxy.readthedocs.io/en/latest/Install.html Install the library directly from the master branch of the GitHub repository using pip. ```bash pip install git+https://github.com/CharlesPikachu/freeproxy.git@master ``` -------------------------------- ### Install FreeProxy using pip Source: https://freeproxy.readthedocs.io/en/latest/Install.html Use this command to install the library directly from the Python Package Index. ```bash pip install pyfreeproxy ``` -------------------------------- ### Example Terminal Output Source: https://freeproxy.readthedocs.io/en/latest/Quickstart.html This shows the expected output in the terminal after running the proxy scraping script. It displays a table summarizing the proxies retrieved from each source. ```text C:\Users\Charles\Desktop>python test.py KuaidailiProxiedSession >>> adding country_code: 37it [00:05, 6.57it/s] | 1/4 [00:18<00:56, 18.95s/it] 100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:28<00:00, 7.17s/it] The proxy distribution for each source you specified is as follows: +-----------+-------------------------------+------+-------+--------+--------+------------+-------+-------+ | Source | Retrieved Example | HTTP | HTTPS | SOCKS4 | SOCKS5 | Chinese IP | Elite | Total | +-----------+-------------------------------+------+-------+--------+--------+------------+-------+-------+ | Proxifly | http://195.231.69.203:443 | 5112 | 0 | 1043 | 477 | 48 | 2157 | 6632 | | Kuaidaili | http://113.45.158.25:3128 | 20 | 13 | 0 | 0 | 19 | 33 | 33 | | Qiyunip | https://114.103.88.18:8089 | 6 | 9 | 0 | 0 | 15 | 14 | 15 | | Proxylist | socks4://184.181.217.206:4145 | 420 | 59 | 182 | 156 | 54 | 699 | 817 | +-----------+-------------------------------+------+-------+--------+--------+------------+-------+-------+ ``` -------------------------------- ### Example Terminal Output for Proxy Scraping Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt This is an example of the terminal output generated by the proxy scraping script. It shows the progress of scraping from different sources and a formatted table summarizing the retrieved proxies by type and other statistics. ```bash C:\Users\Charles\Desktop>python test.py KuaidailiProxiedSession >>> adding country_code: 37it [00:05, 6.57it/s] | 1/4 [00:18<00:56, 18.95s/it] 100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:28<00:00, 7.17s/it] The proxy distribution for each source you specified is as follows: +-----------+-------------------------------+------+-------+--------+--------+------------+-------+-------+ | Source | Retrieved Example | HTTP | HTTPS | SOCKS4 | SOCKS5 | Chinese IP | Elite | Total | +-----------+-------------------------------+------+-------+--------+--------+------------+-------+-------+ | Proxifly | http://195.231.69.203:443 | 5112 | 0 | 1043 | 477 | 48 | 2157 | 6632 | | Kuaidaili | http://113.45.158.25:3128 | 20 | 13 | 0 | 0 | 19 | 33 | 33 | | Qiyunip | https://114.103.88.18:8089 | 6 | 9 | 0 | 0 | 15 | 14 | 15 | | Proxylist | socks4://184.181.217.206:4145 | 420 | 59 | 182 | 156 | 54 | 699 | 817 | +-----------+-------------------------------+------+-------+--------+--------+------------+-------+-------+ ``` -------------------------------- ### Apply Stricter Filtering: US Elite Proxies Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt This example demonstrates filtering for proxies that are both from the US and have an 'elite' anonymity level. It initializes a session with these specific filter rules. ```python from freeproxy.modules.proxies import SpysoneProxiedSession sess = SpysoneProxiedSession(filter_rule={"anonymity": ["elite"], "country_code": ["US"]}) sess.refreshproxies() print(sess.getrandomproxy(proxy_format="freeproxy")) ``` -------------------------------- ### Filter Scraped Proxies with ProxiedSessionClient Source: https://freeproxy.readthedocs.io/en/latest/Quickstart.html An example demonstrating advanced filtering of scraped proxies using multiple sources and specific criteria like country code, anonymity, and protocol. It also shows setting max_tries and making multiple requests. ```python from freeproxy.freeproxy import ProxiedSessionClient client = ProxiedSessionClient( proxy_sources=["ProxyScrapeProxiedSession", "ProxylistProxiedSession"], init_proxied_session_cfg={ "max_pages": 2, "filter_rule": { "country_code": ["CN"], "anonymity": ["elite"], "protocol": ["http", "https"], }, }, disable_print=False, max_tries=20, ) headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36', } resp = client.get("https://www.baidu.com/", timeout=10, headers=headers) print(resp.text) resp = client.get("https://httpbin.org/ip", timeout=5) print(resp.json()) resp = client.get("https://httpbin.org/anything", timeout=15) print(resp.json()) print("origin:", resp.json().get("origin")) print("X-Forwarded-For:", resp.json()["headers"].get("X-Forwarded-For")) print("Forwarded:", resp.json()["headers"].get("Forwarded")) ``` -------------------------------- ### Apply Stricter Filtering: HTTP/S Proxies with Speed Constraints Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt This example filters for HTTP or HTTPS proxies with maximum TCP and HTTP request latencies set to 10000 milliseconds. Note that applying speed constraints can slow down the crawling process. ```python from freeproxy.modules.proxies import FreeproxylistProxiedSession sess = FreeproxylistProxiedSession( filter_rule={ "protocol": ["http", "https"], "max_tcp_ms": 10000, "max_http_ms": 10000, } ) sess.refreshproxies() print(sess.getrandomproxy(proxy_format="freeproxy")) ``` -------------------------------- ### Apply Stricter Filtering: China Proxies Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt This example shows how to filter proxies to include only those from mainland China. It initializes a session with a specific country code filter and refreshes the proxies. ```python from freeproxy.modules.proxies import IP3366ProxiedSession sess = IP3366ProxiedSession(filter_rule={"country_code": ["CN"]}) sess.refreshproxies() print(sess.getrandomproxy(proxy_format="freeproxy")) ``` -------------------------------- ### Scrape, Summarize, and Save Proxies Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt This Python script scrapes proxies from predefined sources, calculates statistics for each source, prints a formatted table of the results to the terminal, and saves all retrieved proxies to a JSON file named 'free_proxies.json'. Ensure you have the 'freeproxy' library installed. ```python import json, random from tqdm import tqdm from freeproxy.modules import BaseProxiedSession, ProxyInfo, BuildProxiedSession, printtable, colorize '''settings''' SOURCES = ["ProxiflyProxiedSession", "KuaidailiProxiedSession", "QiyunipProxiedSession", "ProxylistProxiedSession"] TITLES = ["Source", "Retrieved Example", "HTTP", "HTTPS", "SOCKS4", "SOCKS5", "Chinese IP", "Elite", "Total"] '''scrape''' def scrape(src: str) -> list[ProxyInfo]: try: sess: BaseProxiedSession = BuildProxiedSession({"max_pages": 1, "type": src, "disable_print": False}) return sess.refreshproxies() except Exception: return [] '''stats''' def stats(proxies: list[ProxyInfo]) -> dict: return { "http": sum(p.protocol.lower() == "http" for p in proxies), "https": sum(p.protocol.lower() == "https" for p in proxies), "socks4": sum(p.protocol.lower() == "socks4" for p in proxies), "socks5": sum(p.protocol.lower() == "socks5" for p in proxies), "cn": sum(bool(p.in_chinese_mainland) for p in proxies), "elite": sum(p.anonymity.lower() == "elite" for p in proxies), "total": len(proxies), "ex": (random.choice(proxies).proxy if proxies else "NULL"), } '''row''' def row(src: str, s: dict) -> list: ex = colorize(s["ex"], "green") if s["total"] else "NULL" return [ src.removesuffix("ProxiedSession"), ex, colorize(s["http"], "number"), colorize(s["https"], "number"), colorize(s["socks4"], "number"), colorize(s["socks5"], "number"), colorize(s["cn"], "number"), colorize(s["elite"], "number"), colorize(s["total"], "number"), ] '''main''' def main(): free_proxies, items = {}, [] for src in tqdm(SOURCES): proxies = scrape(src) items.append(row(src, stats(proxies))) free_proxies[src] = [p.todict() for p in proxies] print("The proxy distribution for each source you specified is as follows:") printtable(titles=TITLES, items=items, terminal_right_space_len=1) json.dump(free_proxies, open("free_proxies.json", "w"), indent=2) '''tests''' if __name__ == "__main__": main() ``` -------------------------------- ### List Supported Proxy Sources Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt Use this command to display all proxy sources that your current version of freeproxy supports. This helps in understanding the available options for proxy retrieval. ```bash python -c "from freeproxy.modules import ProxiedSessionBuilder; print(ProxiedSessionBuilder.REGISTERED_MODULES.keys())" ``` -------------------------------- ### Advanced ProxiedSessionClient Configuration and Filtering Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt Illustrates advanced configuration of ProxiedSessionClient, including multiple proxy sources, detailed filtering rules (country, anonymity, protocol), and setting max retries. It then makes multiple requests to different endpoints to demonstrate proxy usage and response inspection. ```python from freeproxy.freeproxy import ProxiedSessionClient client = ProxiedSessionClient( proxy_sources=["ProxyScrapeProxiedSession", "ProxylistProxiedSession"], init_proxied_session_cfg={ "max_pages": 2, "filter_rule": { "country_code": ["CN"], "anonymity": ["elite"], "protocol": ["http", "https"], }, }, disable_print=False, max_tries=20, ) headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36', } resp = client.get("https://www.baidu.com/", timeout=10, headers=headers) print(resp.text) resp = client.get("https://httpbin.org/ip", timeout=5) print(resp.json()) resp = client.get("https://httpbin.org/anything", timeout=15) print(resp.json()) print("origin:", resp.json().get("origin")) print("X-Forwarded-For:", resp.json()["headers"].get("X-Forwarded-For")) print("Forwarded:", resp.json()["headers"].get("Forwarded")) ``` -------------------------------- ### ProxiedSessionClient with Quiet Mode Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt Shows how to initialize ProxiedSessionClient in quiet mode to suppress proxy usage logs, using a different proxy source. ```python from freeproxy import freeproxy client = freeproxy.ProxiedSessionClient( proxy_sources=["ProxydbProxiedSession"], disable_print=True, ) ``` -------------------------------- ### Saved Proxies JSON Structure Source: https://freeproxy.readthedocs.io/en/latest/_sources/Quickstart.md.txt This JSON structure represents the output file 'free_proxies.json' where all scraped proxies are saved. Each key corresponds to a source, and its value is a list of proxy objects, each containing detailed information about the proxy. ```json { "KuaidailiProxiedSession": [ { "source": "KuaidailiProxiedSession", "protocol": "http", "ip": "58.216.109.17", "port": "800", "country_code": "CN", "in_chinese_mainland": true, "anonymity": "elite", "delay": 124, "test_timeout": 5, "test_url": "http://www.baidu.com", "test_headers": { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36" }, "failed_connection_default_timeout": 3600000, "created_at": "2025-12-03T12:43:25.018208", "extra": {} } ], "ProxiflyProxiedSession": [], "QiyunipProxiedSession": [], "ProxylistProxiedSession": [] } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.