### Install Dependencies with Poetry Source: https://github.com/an0nx/ads-trackers-list/blob/master/README.md Command to install project dependencies using Poetry, including development tools. This is a prerequisite for building the blocklist locally. ```bash poetry install --with dev ``` -------------------------------- ### Compile Blocklists with CLI Tool (Bash) Source: https://context7.com/an0nx/ads-trackers-list/llms.txt Command-line instructions to install dependencies, compile protobuf schemas, and run the blocklist compiler. It shows how to use default settings, specify custom input/output files, and generate build statistics. ```bash # Install dependencies pip install aiohttp protobuf # Compile protobuf schema (required before first run) python -m grpc_tools.protoc -I./proto --python_out=. ./proto/router_common.proto # Run with default settings (reads blocklists.txt, outputs dlc.dat) python main.py # Run with custom input file and output location python main.py --input my-blocklists.txt --output ./output --name custom.dat # Generate build statistics JSON file python main.py --input blocklists.txt --output . --name dlc.dat --stats-file stats.json # Example output: # 2024-01-15 10:30:00 [INFO] Found 28 blocklist(s) to process # 2024-01-15 10:30:00 [INFO] Downloading: adguard-dns-filter (https://...) # 2024-01-15 10:30:02 [INFO] Parsed 50000 rules from adguard-dns-filter # 2024-01-15 10:30:15 [INFO] Category 'ADGUARD-DNS-FILTER': 50000 rules # 2024-01-15 10:30:15 [INFO] Compiled 28 categories with 350000 total rules to ./dlc.dat ``` -------------------------------- ### Example V2Ray/Xray Configuration with Blocklist Source: https://github.com/an0nx/ads-trackers-list/blob/master/README.md Demonstrates how to integrate the `dlc.dat` file into a V2Ray/Xray routing configuration. It shows how to use the `geosite` tag to block all aggregated domains or specific lists. ```json { "routing": { "domainStrategy": "AsIs", "rules": [ { "type": "field", "outboundTag": "block", "domain": [ // Block all domains from all lists "geosite:blocklists-all", // Or block domains from a specific list "geosite:adguard-dns" ] } // ... other rules ] }, "outbounds": [ { "protocol": "blackhole", "tag": "block" } // ... other outbounds ] } ``` -------------------------------- ### GitHub Actions Workflow for Automated Blocklist Builds Source: https://context7.com/an0nx/ads-trackers-list/llms.txt A GitHub Actions workflow that automates the hourly rebuilding of the blocklist and publishing to a 'latest' release. It includes steps for checking out code, setting up Python, installing dependencies, compiling Protobuf, building the blocklist, and uploading artifacts. ```yaml # .github/workflows/build.yml (example usage) name: Build Blocklist on: schedule: - cron: '0 * * * *' # Run every hour push: branches: [main] workflow_dispatch: jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: '3.13' - name: Install dependencies run: pip install aiohttp protobuf grpcio-tools - name: Compile protobuf run: python -m grpc_tools.protoc -I./proto --python_out=. ./proto/router_common.proto - name: Build blocklist run: python main.py --output . --name dlc.dat --stats-file stats.json - name: Upload to release uses: softprops/action-gh-release@v1 with: tag_name: latest files: | dlc.dat stats.json # Download URL for users: # https://github.com/An0nX/ads-trackers-list/releases/latest/download/dlc.dat ``` -------------------------------- ### Run Local Blocklist Build Script Source: https://github.com/an0nx/ads-trackers-list/blob/master/README.md Command to execute the main Python script for building the `dlc.dat` file locally. It takes the input list of sources and specifies the output file name. ```bash poetry run python main.py --input blocklists.txt --output-name dlc.dat ``` -------------------------------- ### Compile Protobuf Schema Source: https://github.com/an0nx/ads-trackers-list/blob/master/README.md Command to compile the Protobuf schema using `grpc_tools.protoc`. This step is necessary for generating Python code from the `.proto` file, which is used in the build process. ```bash poetry run python -m grpc_tools.protoc -I./proto --python_out=. ./proto/router_common.proto ``` -------------------------------- ### Protobuf Schema for GeoSiteList and Domain Rules Source: https://context7.com/an0nx/ads-trackers-list/llms.txt Defines the Protobuf schema for GeoSiteList, used in V2Ray/Xray routing. It includes message definitions for Domain (with types like Plain, Regex, RootDomain, Full), GeoSite (categorized domains), and the root GeoSiteList. ```protobuf // Domain rule types used in V2Ray/Xray routing message Domain { enum Type { Plain = 0; // Keyword matching (e.g., "ads" matches "ads.example.com") Regex = 1; // Regular expression pattern RootDomain = 2; // Matches domain and all subdomains (e.g., "example.com" matches "*.example.com") Full = 3; // Exact domain match only } Type type = 1; string value = 2; repeated Attribute attribute = 3; } // GeoSite entry containing domains for a category message GeoSite { string country_code = 1; // Category name (e.g., "ADGUARD-DNS-FILTER") repeated Domain domain = 2; } // Root container for all geosite entries message GeoSiteList { repeated GeoSite entry = 1; } ``` -------------------------------- ### Build GeoSiteList Protobuf Message from Categorized Rules Source: https://context7.com/an0nx/ads-trackers-list/llms.txt Converts categorized domain rules into a GeoSiteList protobuf message. This message can then be serialized into a binary .dat format compatible with V2Ray/Xray. It requires the `router_common_pb2` module and a dictionary of categorized rules. ```python from main import build_geosite_list, parse_blocklist import router_common_pb2 # Prepare categorized rules from multiple sources categorized_rules = { 'ads': {('domain', 'ads.example.com'), ('domain', 'banner.example.net')}, 'trackers': {('domain', 'analytics.example.com'), ('full', 'tracker.specific.org')}, 'malware': {('domain', 'malicious.example.com'), ('regexp', '.*\.badsite\.net$')} } # Build the protobuf message geosite_list = build_geosite_list(categorized_rules) # Serialize to binary .dat file with open('custom.dat', 'wb') as f: f.write(geosite_list.SerializeToString()) # Inspect the structure for entry in geosite_list.entry: print(f"Category: {entry.country_code}, Domains: {len(entry.domain)}") # Output: # Category: ADS, Domains: 2 # Category: MALWARE, Domains: 2 # Category: TRACKERS, Domains: 2 ``` -------------------------------- ### Asynchronously Fetch Blocklist Content Source: https://context7.com/an0nx/ads-trackers-list/llms.txt Asynchronously downloads blocklist content from a given URL using the aiohttp library. It includes configuration for connection limits, user agents, and request timeouts, along with basic error handling for network issues. ```python import asyncio import aiohttp from main import fetch_blocklist, REQUEST_TIMEOUT, USER_AGENT async def download_example(): connector = aiohttp.TCPConnector(limit=10) headers = {"User-Agent": USER_AGENT} async with aiohttp.ClientSession( connector=connector, headers=headers, timeout=REQUEST_TIMEOUT ) as session: try: content = await fetch_blocklist( session, "https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts" ) print(f"Downloaded {len(content)} bytes") print(f"First 200 chars: {content[:200]}") except aiohttp.ClientError as e: print(f"Download failed: {e}") asyncio.run(download_example()) # Output: # Downloaded 125000 bytes # First 200 chars: # Peter Lowe's Ad and tracking server list... ``` -------------------------------- ### Parse Raw Domain List to Blocklist Rules Source: https://context7.com/an0nx/ads-trackers-list/llms.txt Parses a raw string of domain names into a list of blocklist rules. Each rule is a tuple containing the rule type (e.g., 'domain') and the domain value. This is a foundational step for processing domain lists. ```python raw_content = """ ads.example.com tracker.example.net malware.example.org """ rules = parse_blocklist(raw_content) # Result: [('domain', 'ads.example.com'), ('domain', 'tracker.example.net'), ('domain', 'malware.example.org')] ``` -------------------------------- ### Download Pre-compiled Blocklist File Source: https://github.com/an0nx/ads-trackers-list/blob/master/README.md Provides a direct URL to download the latest pre-compiled `dlc.dat` file. This file is automatically updated by GitHub Actions and can be used directly in V2Ray/Xray configurations. ```plaintext https://github.com/An0nX/ads-trackers-list/releases/latest/download/dlc.dat ``` -------------------------------- ### Parse Blocklist Content (Python) Source: https://context7.com/an0nx/ads-trackers-list/llms.txt Demonstrates the `parse_blocklist` function's ability to process various blocklist formats including AdGuard, hosts file, V2Fly domain-list-community, and raw domain lists. The function automatically detects the format and returns a list of domain rules. ```python from main import parse_blocklist # AdGuard format adguard_content = """ ||ads.example.com^ ||tracker.example.net^ ! This is a comment ||analytics.example.org^ """ rules = parse_blocklist(adguard_content) # Result: [('domain', 'ads.example.com'), ('domain', 'tracker.example.net'), ('domain', 'analytics.example.org')] # Hosts file format hosts_content = """ # Blocklist 0.0.0.0 ads.example.com 127.0.0.1 tracker.example.net 0.0.0.0 localhost """ rules = parse_blocklist(hosts_content) # Result: [('domain', 'ads.example.com'), ('domain', 'tracker.example.net')] # V2Fly domain-list-community format v2fly_content = """ domain:example.com full:ads.specific.com regexp:.*\.tracker\.net$ keyword:advertising """ rules = parse_blocklist(v2fly_content) # Result: [('domain', 'example.com'), ('full', 'ads.specific.com'), ('regexp', '.*\.tracker\.net$'), ('keyword', 'advertising')] ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.