### Development Installation Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/10-package-structure.md Command to install pybgpkit with development dependencies, including testing tools like pytest. This setup is recommended for contributors to the project. ```bash pip install pybgpkit[dev] # Includes pytest ``` -------------------------------- ### Installation from Source Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/10-package-structure.md Steps to clone the pybgpkit repository and install it locally in editable mode. This is useful for development or when using the latest unreleased code. ```bash git clone https://github.com/bgpkit/pybgpkit.git cd pybgpkit pip install -e . ``` -------------------------------- ### Environment Variable Configuration Example Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Demonstrates how to load API URLs and page sizes from environment variables for flexible configuration. ```python import os import bgpkit # Example: Load API URL from environment broker_api = os.getenv( 'BGPKIT_BROKER_API', 'https://api.bgpkit.com/v3/broker' ) broker = bgpkit.Broker(api_url=broker_api) # Example: Load page size from environment page_size = int(os.getenv('BGPKIT_PAGE_SIZE', '100')) broker = bgpkit.Broker(page_size=page_size) ``` -------------------------------- ### PyPI Installation Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/10-package-structure.md Command to install the pybgpkit package using pip from the Python Package Index. This is the standard method for installing Python libraries. ```bash pip install pybgpkit ``` -------------------------------- ### Initialize RouteParser with Filters Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Configure the RouteParser with filters to refine the route data. This example shows filtering by peer IP addresses. ```python import bgpkit # With filters route_parser = bgpkit.RouteParser( url="https://spaces.bgpkit.org/parser/update-example", filters={"peer_ips": "192.0.2.1"} ) ``` -------------------------------- ### Test/Demo Endpoints Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/00-index.md This section provides access to test and demo endpoints for the pybgpkit library, useful for quick testing and examples. ```APIDOC ## Test/Demo Endpoints This section provides access to test and demo endpoints for the pybgpkit library, useful for quick testing and examples. ### Parser - **Endpoint**: `https://spaces.bgpkit.org/parser/update-example` ``` -------------------------------- ### List Result Response Structure Example Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/00-index.md Demonstrates the JSON structure for responses that return a list of items, common for broker queries or community lookups. Each item contains relevant details like timestamps. ```json # Broker.query() → List[BrokerItem] # CommunityLookup.query() → List[CommunityEntry] [ { "ts_start": "...", "ts_end": "...", ... }, ... ] ``` -------------------------------- ### Manual Python Package Release Source: https://github.com/bgpkit/pybgpkit/blob/main/README.md Manually builds and uploads a Python package to PyPI using the `build` and `twine` tools. Ensure these are installed before running. ```bash python -m pip install --upgrade build twine python -m build twine upload --skip-existing dist/* ``` -------------------------------- ### Get RIPE Collectors and Print Details Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Retrieves all collectors from the RIPE project and prints their ID, name, and location if available. ```python ripe_collectors = broker.collectors(project="ripe") for collector in ripe_collectors: print(f"{collector.id}: {collector.name}") if collector.latitude: print(f" Location: ({collector.latitude}, {collector.longitude})") ``` -------------------------------- ### Retry with Exponential Backoff Example Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/MANIFEST.txt Demonstrates a comprehensive error handling pattern with retry logic using exponential backoff. This is useful for handling transient network failures or API rate limiting. ```python import time def retry_with_exponential_backoff(func, max_retries=5, initial_delay=1): """Retry a function with exponential backoff.""" delay = initial_delay for attempt in range(max_retries): try: return func() except Exception as e: print(f"Attempt {attempt + 1} failed: {e}") time.sleep(delay) delay *= 2 raise Exception("Max retries exceeded") # Example usage: def my_api_call(): # Replace with your actual API call raise ConnectionError("Simulated network error") try: result = retry_with_exponential_backoff(my_api_call) print("API call successful:", result) except Exception as e: print("API call failed after multiple retries:", e) ``` -------------------------------- ### BGP Community Analysis Example Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/00-index.md This snippet shows how to find BGP communities managed by a specific organization, such as Google, using the CommunityLookup class. ```python import bgpkit lookup = bgpkit.CommunityLookup() # Find all communities managed by Google google_communities = lookup.query(as_name="Google", page_size=100) print(f"Google manages {len(google_communities)} communities:") for entry in google_communities: print(f" {entry.value}: {entry.description}") ``` -------------------------------- ### Single Result Response Structure Example Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/00-index.md Illustrates the expected JSON structure for a single result, such as from an IP lookup. Includes fields for IP address, country, and ASN information. ```json # IpLookup.query() → IpInfo { "ip": "1.1.1.1", "country": "AU", "asn": {"asn": 13335, "name": "Cloudflare"} } ``` -------------------------------- ### Handle IP Lookup Errors Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/03-ip-lookup.md Provides an example of how to handle potential network and API errors during an IP lookup operation using try-except blocks. ```python import bgpkit import requests lookup = bgpkit.IpLookup() try: info = lookup.query(ip="192.0.2.1") print(f"Country: {info.country}, AS: {info.as_number}") except requests.exceptions.Timeout: print("API request timed out") except requests.exceptions.ConnectionError: print("Unable to reach API endpoint") except Exception as e: print(f"Unexpected error: {e}") ``` -------------------------------- ### Query Broker Peers with Full Feed Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Retrieve full BGP feed information for peers from the Broker service. Set `full_feed` to True to get comprehensive data. ```python broker.peers( full_feed=True, asn=64512 ) ``` -------------------------------- ### Paginated Result Response Structure Example Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/00-index.md Shows the typical JSON format for paginated API responses. Includes data, counts, current page, page size, and pagination links for navigation. ```json # AsnLookup.query() → AsnLookupResult { "data": [...], # List of items "count": 1000, # Total across all pages "page": 1, # Current page "page_size": 100, # Items per page "pagination": { "next": "...", # Next page URL "previous": "..." # Previous page URL } } ``` -------------------------------- ### BGP Route Security Validation Workflow Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/00-index.md This example demonstrates how to validate BGP route security using pybgpkit. It checks if the origin AS for a given prefix is authorized according to RPKI data. ```python import bgpkit roas = bgpkit.Roas() parser = bgpkit.Parser(url="https://spaces.bgpkit.org/parser/update-example") for elem in parser: # Check if this origin AS is authorized for this prefix roa_entries = roas.query(prefix=elem.prefix, current=True) authorized_asns = {roa.asn for roa in roa_entries} origin_asn = elem.as_path[-1] if elem.as_path else None if origin_asn in authorized_asns: print(f"✓ {elem.prefix} correctly originated by AS{origin_asn}") else: print(f"✗ {elem.prefix} originated by unauthorized AS{origin_asn}") ``` -------------------------------- ### Configure Broker with Default Settings Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the Broker class with default API URL, page size, and SSL verification settings. ```python import bgpkit # Default configuration (standard use) broker = bgpkit.Broker() ``` -------------------------------- ### Get RouteViews Collectors Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Retrieves all active collectors from the RouteViews project. ```python rv_collectors = broker.collectors( project="routeviews", active=True ) ``` -------------------------------- ### Configuration Options Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/MANIFEST.txt Details on how to configure pybgpkit, including constructor parameters, query method parameters, environment variables, and API endpoint URLs. ```APIDOC ## Configuration Options Details on how to configure pybgpkit, including constructor parameters, query method parameters, environment variables, and API endpoint URLs. ### Constructor Parameters - Documented for 7 classes. ### Query Method Parameters - Documented for 8 methods. ### Parameter Validation - Ranges and default values are documented. ### Environment Variables - Guidance on using environment variables for configuration. ### API Endpoint URLs - Documentation for public service endpoint URLs. ``` -------------------------------- ### Initialize and Parse All Elements with Parser Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/01-parser.md Instantiate the Parser with a URL and optional filters, then parse all route elements. Use this when detailed attribute access is required for all elements. ```python import bgpkit parser = bgpkit.Parser( url="https://spaces.bgpkit.org/parser/update-example", filters={"peer_ips": "185.1.8.65, 2001:7f8:73:0:3:fa4:0:1"} ) elements = parser.parse_all() for elem in elements: print(elem.prefix, elem.as_path) ``` -------------------------------- ### Initialize Roas Client Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/06-roas-lookup.md Instantiate the Roas client. Use the default alpha API endpoint or specify a custom URL. ```python import bgpkit # Standard initialization roas = bgpkit.Roas() # Custom API endpoint (alpha API) roas = bgpkit.Roas(api_url="https://alpha.api.bgpkit.com") ``` -------------------------------- ### Initialize Broker Client Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Instantiate the Broker client for interacting with the BGPKIT Broker API. Supports standard and custom configurations. ```python import bgpkit # Standard initialization broker = bgpkit.Broker() # Custom configuration broker = bgpkit.Broker( api_url="https://custom-api.example.com/v3/broker", page_size=500, verify=False # For self-signed certificates ) ``` -------------------------------- ### Initialize Parser with Filters Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Configure the Parser with a dictionary of filters to pre-process the data. This allows for targeted data retrieval based on peer IPs and origin ASNs. ```python import bgpkit # With dictionary-style filters parser = bgpkit.Parser( url="https://spaces.bgpkit.org/parser/update-example", filters={ "peer_ips": "185.1.8.65, 2001:7f8:73:0:3:fa4:0:1", "origin_asns": "13335" } ) ``` -------------------------------- ### Configure AsnLookup with Default Settings Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the AsnLookup class using the default API URL for utilities. ```python import bgpkit # Default configuration asn_lookup = bgpkit.AsnLookup() ``` -------------------------------- ### Initialize and Count Routes with RouteParser Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/01-parser.md Instantiate the RouteParser with a URL and count the total number of routes without loading them into memory. This is useful for quick statistics. ```python route_parser = bgpkit.RouteParser( url="https://spaces.bgpkit.org/parser/update-example" ) total_routes = route_parser.count() print(f"Total routes: {total_routes}") ``` -------------------------------- ### Retrieve Latest MRT Data Files Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Get the most recent MRT data files available across all collectors and projects. Requires the Broker client to be initialized. ```python broker = bgpkit.Broker() latest_files = broker.latest() for item in latest_files: print(f"{item.collector_id}: {item.url}") print(f" Time range: {item.ts_start} to {item.ts_end}") print(f" Size: {item.exact_size} bytes") ``` -------------------------------- ### Initialize AsnLookup Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/04-asn-lookup.md Instantiate the AsnLookup class for querying ASN data. Use the default API endpoint or provide a custom one. ```python import bgpkit # Standard initialization asn_lookup = bgpkit.AsnLookup() # Custom API endpoint asn_lookup = bgpkit.AsnLookup(api_url="https://custom-api.example.com/v3/utils") ``` -------------------------------- ### BrokerItem Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/07-types-reference.md Represents metadata for a single MRT data file, including start and end timestamps, collector ID, data type, URL, and size information. ```APIDOC ## BrokerItem ### Description Represents metadata for a single MRT data file, including start and end timestamps, collector ID, data type, URL, and size information. ### Source `bgpkit/bgpkit_broker.py:17-24` ### Type Dataclass ```python @dataclass class BrokerItem: ts_start: str ts_end: str collector_id: str data_type: str url: str rough_size: int exact_size: int ``` ### Fields - **ts_start** (str) - Start timestamp (ISO 8601) - **ts_end** (str) - End timestamp (ISO 8601) - **collector_id** (str) - Collector identifier - **data_type** (str) - "update" or "rib" - **url** (str) - Download URL - **rough_size** (int) - Approximate size in bytes - **exact_size** (int) - Exact size in bytes ### Returned by - `Broker.query()` → `List[BrokerItem]` - `Broker.latest()` → `List[BrokerItem]` ``` -------------------------------- ### Configure Broker with Combined Custom Settings Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the Broker class with a custom API URL, page size, and SSL verification disabled. ```python import bgpkit # Combined configuration broker = bgpkit.Broker( api_url="https://internal-api.example.com/v3/broker", page_size=500, verify=False ) ``` -------------------------------- ### Get Latest BGP Data Files with Broker Source: https://github.com/bgpkit/pybgpkit/blob/main/README.md Retrieves the latest available MRT data files using the BGPKIT Broker. This is useful for accessing the most recent BGP information. ```python # Get latest files latest = broker.latest() ``` -------------------------------- ### Standard Import Pattern Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/10-package-structure.md Demonstrates how to import the entire bgpkit package and use its classes with the package namespace prefix. This is a common way to access library functionalities. ```python import bgpkit # Use with namespace prefix parser = bgpkit.Parser(url="...") broker = bgpkit.Broker() ``` -------------------------------- ### Access AS Name from IpInfo Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/03-ip-lookup.md Demonstrates how to access the AS organization name from the IpInfo object. Handles cases where AS information might not be available. ```python ip_info = lookup.query(ip="8.8.8.8") if ip_info.as_name: print(f"Organization: {ip_info.as_name}") ``` -------------------------------- ### AsnLookup Constructor Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/04-asn-lookup.md Initializes the AsnLookup client. You can specify a custom API endpoint URL if needed. ```APIDOC ## AsnLookup Constructor Initializes the AsnLookup client. ### Parameters - **api_url** (str) - Optional - Default: "https://api.bgpkit.com/v3/utils" - Base URL for the utilities API ### Example ```python import bgpkit # Standard initialization asn_lookup = bgpkit.AsnLookup() # Custom API endpoint asn_lookup = bgpkit.AsnLookup(api_url="https://custom-api.example.com/v3/utils") ``` ``` -------------------------------- ### Roas Constructor Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/06-roas-lookup.md Initializes the ROAS client. You can specify a custom API URL for the alpha ROAS API. ```APIDOC ## Roas Constructor Initializes the ROAS client. You can specify a custom API URL for the alpha ROAS API. ### Parameters #### Query Parameters - **api_url** (str) - Optional - Default: "https://alpha.api.bgpkit.com" - Base URL for the alpha ROAS API. ``` -------------------------------- ### Access AS Number from IpInfo Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/03-ip-lookup.md Demonstrates how to access the AS number from the IpInfo object. Handles cases where AS information might not be available. ```python ip_info = lookup.query(ip="1.1.1.1") if ip_info.as_number: print(f"AS{ip_info.as_number}") else: print("AS information not found") ``` -------------------------------- ### Configure IpLookup with Default Settings Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the IpLookup class using the default API URL for utilities. ```python import bgpkit # Default configuration lookup = bgpkit.IpLookup() ``` -------------------------------- ### Import Roas and RoasItem Classes Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/06-roas-lookup.md Import the necessary Roas and RoasItem classes from the bgpkit library for use in your application. ```python from bgpkit import Roas, RoasItem ``` -------------------------------- ### Initialize Parser with Filters Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/01-parser.md Use this method to create a Parser instance with a list of pre-constructed Filter objects. Specify the URL of the MRT data file and the filters to apply. ```python from bgpkit import Filter, Parser f1 = Filter.peer_ip("185.1.8.65") f2 = Filter.origin_asn("13335") parser = Parser.from_filters( url="https://spaces.bgpkit.org/parser/update-example", filters=[f1, f2] ) ``` -------------------------------- ### Initialize Parser with URL Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Basic initialization of the Parser class requires a URL to the MRT data. This is used for direct parsing of data files. ```python import bgpkit # Basic initialization parser = bgpkit.Parser( url="https://spaces.bgpkit.org/parser/update-example" ) ``` -------------------------------- ### Initialize RouteParser with URL Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Basic initialization of the RouteParser class, similar to Parser, requires a URL for data processing. This is suitable for scenarios where only route data is needed. ```python import bgpkit # Basic initialization route_parser = bgpkit.RouteParser( url="https://spaces.bgpkit.org/parser/update-example" ) ``` -------------------------------- ### Configure Broker with Custom Page Size Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the Broker class with a custom page size for handling large result sets. ```python import bgpkit # Custom page size for large result sets broker = bgpkit.Broker(page_size=1000) ``` -------------------------------- ### IpLookup Constructor Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/03-ip-lookup.md Initializes the IpLookup client. You can specify a custom API URL if needed. ```APIDOC ## IpLookup Constructor Initializes the IpLookup client. You can specify a custom API URL if needed. ### Parameters #### Query Parameters - **api_url** (str) - Optional - Default: "https://api.bgpkit.com/v3/utils" - Base URL for the utilities API ``` -------------------------------- ### Access AsnInfo Fields Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/04-asn-lookup.md Demonstrates how to access individual fields from an AsnInfo object after performing a query. This shows the structure of the returned data. ```python import bgpkit asn_lookup = bgpkit.AsnLookup() result = asn_lookup.query(asn="15169") asn_info = result.data[0] print(f"ASN: {asn_info.asn}") print(f"Name: {asn_info.name}") print(f"Organization: {asn_info.org_name}") print(f"Country: {asn_info.country}") print(f"Website: {asn_info.website}") print(f"Description: {asn_info.description}") ``` -------------------------------- ### Query All RIB Snapshots from a Project Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Retrieve all RIB snapshots for a specific project, such as RouteViews. Requires the Broker client to be initialized. ```python ribs = broker.query( project="routeviews", data_type="rib" ) ``` -------------------------------- ### Build and Push Git Tag for Release Source: https://github.com/bgpkit/pybgpkit/blob/main/README.md Tags a new version and pushes it to the origin repository to trigger automated builds and releases via GitHub Actions. ```bash git tag v0.7.0 git push origin v0.7.0 ``` -------------------------------- ### Configure AsnLookup with Custom API URL Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the AsnLookup class with a custom API endpoint URL for utilities. ```python import bgpkit # Custom API endpoint asn_lookup = bgpkit.AsnLookup(api_url="https://internal-api.example.com/v3/utils") ``` -------------------------------- ### Initialize IpLookup with Default API Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/03-ip-lookup.md Standard initialization of the IpLookup class using the default API endpoint. ```python import bgpkit # Standard initialization with default API lookup = bgpkit.IpLookup() ``` -------------------------------- ### Import Public Classes Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/04-asn-lookup.md Imports all public classes from the bgpkit library into the current namespace. ```python from bgpkit import AsnLookup, AsnInfo, AsnLookupResult ``` -------------------------------- ### Standard Import Pattern Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/00-index.md Import the entire bgpkit library and access its classes through the main namespace. This is a straightforward way to begin using the library's functionalities. ```python # Standard pattern import bgpkit # Access all classes via namespace parser = bgpkit.Parser(url="...") broker = bgpkit.Broker() lookup = bgpkit.IpLookup() ``` -------------------------------- ### Configure Broker with Custom API URL Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the Broker class with an alternative API endpoint URL. ```python import bgpkit # Alternative API endpoint broker = bgpkit.Broker( api_url="https://custom-broker-api.example.com/v3/broker" ) ``` -------------------------------- ### Broker Constructor Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Initializes the Broker API client. Allows customization of the API endpoint, page size for paginated results, and SSL certificate verification. ```APIDOC ## Broker Constructor ### Description Initializes the Broker API client. Allows customization of the API endpoint, page size for paginated results, and SSL certificate verification. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import bgpkit # Standard initialization broker = bgpkit.Broker() # Custom configuration broker = bgpkit.Broker( api_url="https://custom-api.example.com/v3/broker", page_size=500, verify=False # For self-signed certificates ) ``` ### Response #### Success Response (200) Returns an instance of the Broker client. #### Response Example ```json { "message": "Broker client initialized" } ``` ``` -------------------------------- ### Configure Broker with SSL Verification Disabled Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the Broker class with SSL verification disabled, suitable for self-signed certificates. ```python import bgpkit # Disable SSL verification (for self-signed certificates) broker = bgpkit.Broker(verify=False) ``` -------------------------------- ### Initialize CommunityLookup Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/05-community-lookup.md Instantiate the CommunityLookup class for querying BGP communities. You can use the default API endpoint or specify a custom one. ```python import bgpkit # Standard initialization community_lookup = bgpkit.CommunityLookup() # Custom API endpoint community_lookup = bgpkit.CommunityLookup( api_url="https://custom-api.example.com/v3/communities" ) ``` -------------------------------- ### Configure CommunityLookup with Default Settings Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the CommunityLookup class using the default API URL for communities. ```python import bgpkit # Default configuration community_lookup = bgpkit.CommunityLookup() ``` -------------------------------- ### Configure Roas with Default Alpha API Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the Roas class using the default alpha API endpoint. ```python import bgpkit # Default alpha API roas = bgpkit.Roas() ``` -------------------------------- ### BGP Parsing with Reusable Filters Source: https://github.com/bgpkit/pybgpkit/blob/main/README.md Demonstrates creating and using reusable filter objects, such as peer IP filters, with the BGPKIT Parser. This allows for more modular filter management. ```python # Reusable filter objects from bgpkit import Filter f = Filter.peer_ip("185.1.8.65") parser = bgpkit.Parser.from_filters(url, [f]) ``` -------------------------------- ### Import Broker Classes Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Imports the necessary classes from the bgpkit library for use. ```python from bgpkit import Broker, BrokerItem, CollectorItem, PeerItem ``` -------------------------------- ### Initialize IpLookup with Custom API URL Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/03-ip-lookup.md Initialize the IpLookup class with a custom API endpoint URL. ```python import bgpkit # Custom API endpoint lookup = bgpkit.IpLookup(api_url="https://custom-api.example.com/v3/utils") ``` -------------------------------- ### Module Hierarchy Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/10-package-structure.md Visual representation of the pybgpkit package's directory and file structure. ```tree bgpkit/ ├── __init__.py # Top-level exports ├── bgpkit_parser.py # Parser and RouteParser (re-exports from pybgpkit-parser) ├── bgpkit_broker.py # Broker, BrokerItem, PeerItem, CollectorItem ├── bgpkit_ip.py # IpLookup, IpInfo, AsnDetail ├── bgpkit_asn.py # AsnLookup, AsnInfo, AsnLookupResult, Pagination ├── bgpkit_community.py # CommunityLookup, CommunityEntry, CommunitySource └── bgpkit_roas.py # Roas, RoasItem ``` -------------------------------- ### List Community Sources Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/05-community-lookup.md Retrieves and prints a list of available community data sources, including their IDs, names, and optional URLs. ```python lookup = bgpkit.CommunityLookup() sources = lookup.sources() for source in sources: status = f" ({source.url})" if source.url else "" print(f"{source.id}: {source.name}{status}") ``` -------------------------------- ### Selective Import Pattern Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/10-package-structure.md Shows how to import specific classes directly from the top-level bgpkit package. This allows for using class names without the package prefix. ```python from bgpkit import Parser, RouteParser, Broker # Use classes directly parser = Parser(url="...") broker = Broker() ``` -------------------------------- ### Import Community Classes Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/05-community-lookup.md Imports necessary classes for working with BGP communities from the bgpkit library. ```python from bgpkit import CommunityLookup, CommunityEntry, CommunitySource ``` -------------------------------- ### Iterate and Display ROAS Item Details Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/06-roas-lookup.md Fetch current ROAs for a given AS and iterate through the results to print detailed information for each ROA, including prefix, origin AS, max length, TAL, and validity date ranges. ```python roas = bgpkit.Roas() items = roas.query(asn=15169, current=True, page_size=50) for item in items: print(f"Prefix: {item.prefix}") print(f"Originating AS: AS{item.asn}") if item.max_len: print(f"Max allowed length: /{item.max_len}") else: print("Exact length only") if item.tal: print(f"Trust Anchor: {item.tal}") if item.date_ranges: print(f"Valid during:") for start, end in item.date_ranges: print(f" {start} to {end}") print() ``` -------------------------------- ### Retrieve BGP Community Data Sources Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/05-community-lookup.md Fetch a list of all available data sources that contribute information to the BGP community database. Displays the ID, name, and URL of each source. ```python import bgpkit lookup = bgpkit.CommunityLookup() sources = lookup.sources() for source in sources: print(f"{source.id}: {source.name}") if source.url: print(f" Source: {source.url}") ``` -------------------------------- ### Query BGP Peers by IP Address Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Find BGP peer information using a specific IP address. Requires the Broker client to be initialized. ```python peer_info = broker.peers(ip="192.0.2.1") ``` -------------------------------- ### Query ROAs for a Prefix Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/06-roas-lookup.md Identifies all ASNs that are authorized to originate routes for a given IP prefix. ```python # Check which ASNs are authorized for a specific prefix prefix_authorizations = roas.query(prefix="203.0.113.0/24") print(f"{len(prefix_authorizations)} ASNs authorized for this prefix:") for item in prefix_authorizations: print(f" AS{item.asn}") ``` -------------------------------- ### Query Current ROAs for an AS Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/06-roas-lookup.md Retrieve current Route Origination Authorizations for a given Autonomous System Number. Displays the authorized prefix, maximum prefix length, and Trust Anchor Location. ```python import bgpkit roas = bgpkit.Roas() # Get current ROAs for a specific AS current_roas = roas.query(asn=15169, current=True) for item in current_roas: print(f"Prefix: {item.prefix}") print(f"Max Length: {item.max_len}") print(f"TAL: {item.tal}\n") ``` -------------------------------- ### List Active Collectors with Broker Source: https://github.com/bgpkit/pybgpkit/blob/main/README.md Lists active collectors for a given project using the BGPKIT Broker. This helps in identifying available data sources. ```python # List collectors collectors = broker.collectors(project="routeviews", active=True) ``` -------------------------------- ### Query Collectors by Project Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md List collectors associated with a specific project, like RouteViews or RIPE. Requires the Broker client to be initialized. ```python broker = bgpkit.Broker() # Example: Get collectors for RouteViews project routeviews_collectors = broker.collectors(project="routeviews") ``` -------------------------------- ### AsnLookup.query() Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/04-asn-lookup.md Searches the ASN database with optional filters and pagination. Returns a list of ASN information objects. ```APIDOC ## AsnLookup.query() Search the ASN database with optional filters and pagination. ### Parameters #### Query Parameters - **asn** (str) - Optional - None - Exact AS number to look up (e.g., "13335") - **country** (str) - Optional - None - ISO 3166-1 alpha-2 country code filter - **search** (str) - Optional - None - Free-text search across AS names and descriptions - **page** (int) - Optional - 1 - Page number for pagination (1-indexed) - **page_size** (int) - Optional - 100 - Results per page (capped at 10000) ### Returns `AsnLookupResult` — Paginated results with metadata. ### Raises - `requests.exceptions.RequestException` — On network or API errors - `requests.exceptions.JSONDecodeError` — On invalid API response ### Example ```python import bgpkit asn_lookup = bgpkit.AsnLookup() # Exact AS number lookup result = asn_lookup.query(asn="13335") for asn_info in result.data: print(f"AS{asn_info.asn}: {asn_info.name}") print(f" Country: {asn_info.country}") print(f" Description: {asn_info.description}") print(f" Website: {asn_info.website}") # Search by country cloudflare_asns = asn_lookup.query(country="US", search="cloudflare") print(f"Found {result.count} Cloudflare ASNs in the US") # Paginated search across all US-based ASNs all_us_asns = [] page = 1 while True: result = asn_lookup.query(country="US", page=page, page_size=1000) all_us_asns.extend(result.data) if len(result.data) < 1000: break page += 1 # Free-text search google_results = asn_lookup.query(search="google") for asn_info in google_results.data: print(f"AS{asn_info.asn}: {asn_info.name}") # Search with pagination results = asn_lookup.query(search="cloudflare", page=2, page_size=50) for asn in results.data: print(f"AS{asn.asn}") ``` ``` -------------------------------- ### High-Volume Queries with Increased Page Size Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Configure the Broker with a larger page size to efficiently paginate through extensive result sets. ```python import bgpkit # Increase page size for faster pagination through large result sets broker = bgpkit.Broker(page_size=1000) items = broker.query(project="routeviews") # Collect all results all_items = [] # Pagination is handled automatically by _paginate() ``` -------------------------------- ### Import Core pybgpkit Classes Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/01-parser.md All essential classes for pybgpkit, including Parser, RouteParser, RouteElem, and Filter, are exported at the top-level `bgpkit` namespace for easy access. ```python from bgpkit import Parser, RouteParser, RouteElem, Filter ``` -------------------------------- ### Typical BGP Operations Workflow with IP Lookup Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/03-ip-lookup.md Integrates IP lookup into a BGP parsing workflow to identify the country of origin for update peers. ```python from bgpkit import Parser, IpLookup parser = bgpkit.Parser(url="https://spaces.bgpkit.org/parser/update-example") lookup = bgpkit.IpLookup() for elem in parser: peer_info = lookup.query(ip=elem.peer_ip) print(f"Update from AS{elem.peer_asn} in {peer_info.country}") ``` -------------------------------- ### Perform Simple IP Lookup Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/03-ip-lookup.md Retrieve minimal geolocation information for an IP address for faster queries. ```python import bgpkit lookup = bgpkit.IpLookup() # Fast/simple lookup simple_info = lookup.query(ip="8.8.8.8", simple=True) print(f"Country: {simple_info.country}") ``` -------------------------------- ### Query ROAs for a Specific Prefix Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/06-roas-lookup.md Search for ROA entries that authorize a particular IP prefix. Lists the AS number and maximum allowed prefix length for each authorization. ```python import bgpkit roas = bgpkit.Roas() # Find ROAs for a specific prefix prefix_roas = roas.query(prefix="1.1.1.0/24") for item in prefix_roas: print(f"AS{item.asn} authorizes {item.prefix}") if item.max_len: print(f" Max length: {item.max_len}") ``` -------------------------------- ### Basic Parser Exception Handling Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/08-errors-and-exceptions.md Demonstrates how to catch exceptions when parsing MRT files with the bgpkit.Parser. This is useful for handling invalid URLs, corrupted files, or filter mismatches. ```python import bgpkit parser = bgpkit.Parser(url="https://example.com/nonexistent.mrt") try: # May raise an exception if file is invalid or unreachable elems = parser.parse_all() except Exception as e: print(f"Parser error: {type(e).__name__}: {e}") ``` -------------------------------- ### Perform Full IP Lookup Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/03-ip-lookup.md Retrieve comprehensive geolocation and AS information for a given IP address. ```python import bgpkit lookup = bgpkit.IpLookup() # Full lookup ip_info = lookup.query(ip="1.1.1.1") print(f"Country: {ip_info.country}") print(f"AS Number: {ip_info.as_number}") print(f"AS Name: {ip_info.as_name}") ``` -------------------------------- ### Top-Level Public API Exports Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/10-package-structure.md Imports all public symbols directly from the 'bgpkit' namespace for easy access. ```python from bgpkit import ( # Parser module Parser, RouteParser, RouteElem, Filter, # Broker module Broker, BrokerItem, CollectorItem, PeerItem, # ROAS module Roas, RoasItem, # IP Lookup module IpLookup, IpInfo, # ASN Lookup module AsnLookup, AsnInfo, AsnLookupResult, # Community Lookup module CommunityLookup, CommunityEntry, CommunitySource, ) ``` -------------------------------- ### RouteParser Class Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/01-parser.md Offers fast, route-level BGP parsing with fewer fields than the full `Parser`. It's suitable when only prefix and AS path information is needed, providing lower memory consumption. ```APIDOC ## RouteParser Class Fast, route-level BGP parsing with fewer fields than full element parsing. ### Constructor ```python RouteParser( url: str, filters: Optional[dict] = None ) -> RouteParser ``` | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | url | str | Yes | — | URL to MRT data file or remote endpoint | | filters | dict | No | None | Filter criteria to apply during parsing | ### Methods #### count() Count the total number of routes in the data without loading all elements into memory. ```python def count() -> int ``` **Returns:** `int` — Total number of routes matching filters. **Example:** ```python route_parser = bgpkit.RouteParser( url="https://spaces.bgpkit.org/parser/update-example" ) total_routes = route_parser.count() print(f"Total routes: {total_routes}") ``` #### Iterator Protocol Iterate over routes one at a time. ```python for route in route_parser: # route is a RouteElem with fewer attributes print(route.prefix, route.as_path) ``` ### Performance Notes - RouteParser is faster than Parser due to reduced field parsing - Use RouteParser when only prefix and AS path information is needed - Memory consumption is lower with iterator-based access vs. `parse_all()` ``` -------------------------------- ### BGP Data Processing Pipeline Workflow Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/00-index.md This snippet demonstrates a typical workflow for processing BGP data using pybgpkit. It involves finding MRT files with the Broker, parsing BGP updates with the Parser, and enriching the data with IP and ASN information. ```python import bgpkit from datetime import datetime, timedelta # Step 1: Find data files broker = bgpkit.Broker() end_time = datetime.utcnow() start_time = end_time - timedelta(hours=1) files = broker.query( ts_start=start_time.isoformat() + "Z", ts_end=end_time.isoformat() + "Z", collector_id="rrc00" ) # Step 2: Parse BGP updates for file in files[:1]: # Process first file parser = bgpkit.Parser(url=file.url) for elem in parser: # Step 3: Enrich with IP/ASN info ip_info = bgpkit.IpLookup().query(ip=elem.peer_ip) asn_info = bgpkit.AsnLookup().query(asn=str(elem.peer_asn)) print(f"Update from {ip_info.country} AS{elem.peer_asn}") print(f" Prefix: {elem.prefix}") print(f" AS Path: {elem.as_path}") ``` -------------------------------- ### bgpkit_asn.py Module Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/10-package-structure.md Facilitates lookup and search for Autonomous System (AS) number information. ```APIDOC ## Module: bgpkit_asn.py **Purpose:** AS number information lookup and search. **Exported Symbols:** | Symbol | Type | |-----------------|----------| | AsnLookup | class | | AsnInfo | dataclass| | AsnLookupResult | dataclass| **Key Methods:** | Method | Returns | Purpose | |--------------------|-----------------|----------------------| | AsnLookup.query() | AsnInfo | Look up AS number | | AsnLookup.search() | List[AsnInfo] | Search AS numbers | | AsnLookup.list() | Pagination | List all AS numbers | ``` -------------------------------- ### Query MRT Data Files by Time Range Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Search for MRT data files within a specified time range. Requires the Broker client to be initialized. ```python broker = bgpkit.Broker() # Search by time range items = broker.query( ts_start="2024-01-01T00:00:00Z", ts_end="2024-01-01T01:00:00Z" ) print(f"Found {len(items)} files") ``` -------------------------------- ### Configure Roas with Custom API Endpoint Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the Roas class with a custom API endpoint URL. Note that the ROAS API is in alpha status. ```python import bgpkit # Custom API endpoint roas = bgpkit.Roas(api_url="https://internal-roas-api.example.com") ``` -------------------------------- ### Query RIPE Collectors and Print Details Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Retrieves active RIPE collectors and prints their ID, name, country, location, and status. ```python collectors = broker.collectors(project="ripe", active=True) for collector in collectors: status = "active" if collector.active else "inactive" location = f"({collector.latitude}, {collector.longitude})" if collector.latitude else "unknown" print(f"{collector.id}: {collector.name}") print(f" Country: {collector.country} {location} [{status}]") ``` -------------------------------- ### Query Broker Collectors by Project and Country Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md List active collectors from the Broker service, filtered by project and country. This helps in identifying available data sources in specific regions. ```python broker.collectors( project="ripe", country="DE", active=True ) ``` -------------------------------- ### Configure CommunityLookup with Custom API URL Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Instantiate the CommunityLookup class with a custom API endpoint URL for communities. ```python import bgpkit # Custom API endpoint community_lookup = bgpkit.CommunityLookup( api_url="https://internal-api.example.com/v3/communities" ) ``` -------------------------------- ### Query Historical ROAs by Date Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/06-roas-lookup.md Retrieve ROA data for a specific AS on a particular historical date. Prints the authorized prefix and its maximum allowed length. ```python import bgpkit roas = bgpkit.Roas() # Historical data for a date date_roas = roas.query(asn=3333, date="2020-06-15") for item in date_roas: print(f"{item.prefix} - Max Length: {item.max_len}") ``` -------------------------------- ### Query BGP Peers by ASN and Feed Type Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Search for BGP peers based on ASN and whether they receive the full BGP feed. Requires the Broker client to be initialized. ```python broker = bgpkit.Broker() # Get all full-feed peers for a specific ASN peers = broker.peers(asn=13335, full_feed=True) print(f"Found {len(peers)} full-feed peers for AS13335") ``` -------------------------------- ### Paginate Through All ASNs in a Country Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/04-asn-lookup.md Retrieve all ASNs within a specific country using pagination. This loop continues until all results are fetched. ```python import bgpkit asn_lookup = bgpkit.AsnLookup() # Paginated search across all US-based ASNs all_us_asns = [] page = 1 while True: result = asn_lookup.query(country="US", page=page, page_size=1000) all_us_asns.extend(result.data) if len(result.data) < 1000: break page += 1 ``` -------------------------------- ### Search Communities by Description Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/05-community-lookup.md Searches for communities whose descriptions contain specific keywords, such as 'backup' or 'redundancy'. ```python # Find backup/redundancy-related communities backup_communities = lookup.query(description="backup", page_size=50) for entry in backup_communities: print(f"{entry.value}: {entry.description} (AS{entry.asn})") ``` -------------------------------- ### Display Detailed Community Entry Information Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/05-community-lookup.md Query for BGP communities and then iterate through the results to print detailed information for each community entry, including its owner, purpose, country, and data source. ```python import bgpkit lookup = bgpkit.CommunityLookup() entries = lookup.query(asn="15169", page_size=50) for entry in entries: print(f"Community {entry.value}") print(f" Owner: AS{entry.asn} ({entry.as_name})") print(f" Purpose: {entry.description}") print(f" Country: {entry.country}") print(f" Source: {entry.source}\n") ``` -------------------------------- ### Query BGP Peers by Collector Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/02-broker.md Retrieve BGP peer information for a specific collector. Requires the Broker client to be initialized. ```python rrc_peers = broker.peers(collector="rrc00") ``` -------------------------------- ### Query AsnLookup with Pagination Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/09-configuration.md Retrieve results from AsnLookup with custom pagination. Specify the page number and the number of results per page to manage large result sets. ```python asn_lookup.query( page=2, page_size=50 ) ``` -------------------------------- ### Query BGP Communities by AS Number Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/05-community-lookup.md Search the BGP community database for communities managed by a specific AS number. The results display the community value and its description. ```python import bgpkit lookup = bgpkit.CommunityLookup() # Search by AS number result = lookup.query(asn="13335") for entry in result: print(f"{entry.value}: {entry.description}") ``` -------------------------------- ### Handling Timeout for API Requests Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/08-errors-and-exceptions.md Illustrates catching a Timeout exception when an API request to pybgpkit services takes too long to complete. Suggests retrying or reducing request parameters like page_size. ```python import bgpkit from requests.exceptions import Timeout lookup = bgpkit.AsnLookup() try: result = lookup.query(search="google", page_size=10000) except Timeout: print("API request timed out; try again or reduce page_size") ``` -------------------------------- ### bgpkit_broker.py Module Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/10-package-structure.md Client for the BGPKIT Broker service, enabling search for MRT files and metadata. ```APIDOC ## Module: bgpkit_broker.py **Purpose:** REST API client for BGPKIT Broker service; metadata search for MRT files. **Exported Symbols:** | Symbol | Type | |---------------|----------| | Broker | class | | BrokerItem | dataclass| | PeerItem | dataclass| | CollectorItem | dataclass| **Key Methods:** | Method | Returns | Purpose | |-----------------|------------------|-----------------| | Broker.query() | List[BrokerItem] | Search MRT files| | Broker.latest() | List[BrokerItem] | Get latest files| | Broker.peers() | List[PeerItem] | Query peer info | | Broker.collectors()| List[CollectorItem]| Query collectors| ``` -------------------------------- ### Iterate Through All Pages Source: https://github.com/bgpkit/pybgpkit/blob/main/_autodocs/04-asn-lookup.md Iterates through all available pages of results using the pagination 'next' URL. This is useful for fetching large datasets. ```python result = asn_lookup.query(search="amazon", page=1, page_size=100) while result.pagination and result.pagination.next: print(f"Page {result.page}: {len(result.data)} results") # Fetch next page by incrementing page number result = asn_lookup.query(search="amazon", page=result.page + 1, page_size=100) print(f"Final page {result.page}: {len(result.data)} results") ```