### Install XMLRiver Pro from GitHub Source: https://github.com/eapwrk/xmlriver-pro/blob/main/docs/llms.txt Install the latest version of the XMLRiver Pro library directly from its GitHub repository. This is the recommended installation method. ```bash pip install git+https://github.com/Eapwrk/xmlriver-pro.git ``` -------------------------------- ### Complete Configuration Example for Google and Yandex Clients Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/configuration.md This example shows how to load credentials from environment variables and configure both GoogleClient and YandexClient with advanced options like timeout, retries, and retry delay. It also demonstrates checking API limits. ```python import os from dotenv import load_dotenv from xmlriver_pro import GoogleClient, YandexClient # Load environment load_dotenv() # Get credentials user_id = int(os.getenv("XMLRIVER_USER_ID", "0")) api_key = os.getenv("XMLRIVER_API_KEY", "") # Configure Google client google = GoogleClient( user_id=user_id, api_key=api_key, timeout=60, max_retries=3, retry_delay=1.0, enable_retry=True ) # Configure Yandex client yandex = YandexClient( user_id=user_id, api_key=api_key, timeout=60, max_retries=3, retry_delay=1.0, enable_retry=True ) # Check configuration print("=== Configuration Check ===") print(f"Balance: {google.get_balance()} rubles") print(f"Google cost: {google.get_cost('google')} rubles per 1000") print(f"Yandex cost: yandex.get_cost('yandex')}} rubles per 1000") limits = google.get_api_limits() print(f"Max concurrent: {limits['max_concurrent_streams']}") print(f"Daily Google limit: {limits['daily_limits']['google']:,}") print(f"Daily Yandex limit: {limits['daily_limits']['yandex']:,}") ``` -------------------------------- ### Example Usage of SearchParams Type Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Shows an example of creating a dictionary that conforms to the SearchParams type alias for search queries. ```python from xmlriver_pro.core.types import SearchParams params: SearchParams = { "query": "python", "groupby": 10, "page": 1 } ``` -------------------------------- ### OneBoxDocument Example Usage Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Illustrates how to access and print basic information from a OneBoxDocument. Assumes 'onebox' is a OneBoxDocument instance. ```python onebox: OneBoxDocument print(f"Type: {onebox.content_type}") print(f"Title: {onebox.title}") print(f"Content: {onebox.snippet}") ``` -------------------------------- ### Batch Processing Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Shows how to process a list of search queries in batches using AsyncGoogleClient and asyncio.gather. Exceptions during processing are caught and reported. ```python import asyncio from xmlriver_pro import AsyncGoogleClient async def batch_search(queries: list): async with AsyncGoogleClient(user_id=123, api_key="key") as client: # Process multiple queries concurrently tasks = [] for query in queries: task = client.search(query) tasks.append(task) # Wait for all to complete results = await asyncio.gather(*tasks, return_exceptions=True) # Process results for query, result in zip(queries, results): if isinstance(result, Exception): print(f"{query}: ERROR - {result}") else: print(f"{query}: {result.total_results} results") # Usage queries = ["python", "javascript", "rust", "go", "cpp"] asyncio.run(batch_search(queries)) ``` -------------------------------- ### AdResult Example Usage Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Shows how to print key information from an AdResult object. The 'ad' variable must be an instance of AdResult. ```python ad: AdResult print(f"Headline: {ad.title}") print(f"Description: {ad.snippet}") print(f"URL: {ad.url}") ``` -------------------------------- ### Example Usage of SearchResult Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Demonstrates how to access and print information from a SearchResult object. Ensure a SearchResult object is instantiated before use. ```python result: SearchResult print(f"#{result.rank}: {result.title}") print(f"URL: {result.url}") print(f"Rating: {result.stars}★") print(f"Snippet: {result.snippet}") ``` -------------------------------- ### Combined Yandex Search Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md A comprehensive example demonstrating a complex search query with multiple parameters. ```APIDOC ## Combined Yandex Search Example ### Description This example showcases a complex search query using the `YandexClient`, combining various parameters such as search terms, grouping, pagination, region, domain, device type, and language. ### Method `YandexClient(user_id: int, api_key: str)` `search(query: str, groupby: int, page: int, lr: int, domain: str, device: DeviceType, lang: str, **kwargs)` ### Parameters #### `YandexClient` Initialization - **user_id** (int) - Your Yandex user ID. - **api_key** (str) - Your Yandex API key. #### `search` Method Parameters - **query** (str) - The search query string (e.g., "python programming"). - **groupby** (int) - Parameter for grouping search results (e.g., 10). - **page** (int) - The page number for pagination (e.g., 0). - **lr** (int) - The region code (e.g., 213 for Moscow). - **domain** (str) - The Yandex domain to search on (e.g., "ru"). - **device** (DeviceType) - The type of device for the search (e.g., `DeviceType.DESKTOP`). - **lang** (str) - The language for the search results (e.g., "ru"). ### Request Example ```python from xmlriver_pro import YandexClient, DeviceType client = YandexClient(user_id=123, api_key="your_key") # Complex search results = client.search( "python programming", groupby=10, page=0, lr=213, # Moscow domain="ru", # Russian domain device=DeviceType.DESKTOP, lang="ru" ) print(f"Query: {results.query}") print(f"Total results: {results.total_results}") for result in results.results: print(f"{result.rank}. {result.title}") ``` ### Response - The `search` method returns a results object containing query details (`query`, `total_results`) and a list of individual search `results`. ``` -------------------------------- ### YandexClient Basic Search Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Demonstrates how to initialize the YandexClient and perform a basic search query, then print the total number of results found. ```python from xmlriver_pro import YandexClient, DeviceType client = YandexClient(user_id=123, api_key="your_key") # Basic search results = client.search("программирование python") print(f"Found {results.total_results} results") ``` -------------------------------- ### KnowledgeGraph Example Usage Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Demonstrates printing the entity name and description from a KnowledgeGraph object. The 'kg' variable should be a KnowledgeGraph instance. ```python kg: KnowledgeGraph print(f"Entity: {kg.entity_name}") print(f"Description: {kg.description}") ``` -------------------------------- ### Pagination Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-news.md Demonstrates how to retrieve news articles across multiple pages using the `page` parameter and `groupby` for potentially better results. ```APIDOC ## Pagination Example ### Example Usage ```python from xmlriver_pro import YandexNews news = YandexNews(user_id=123, api_key="key") # Yandex pages start at 0 all_results = [] for page in range(0, 3): # Fetching 3 pages results = news.search_news("python", page=page, groupby=10) all_results.extend(results.results) print(f"Page {page}: {len(results.results)} articles") print(f"Total: {len(all_results)} articles") ``` ``` -------------------------------- ### YandexClient Search Results Processing Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Provides an example of iterating through the search results and printing the rank, title, and snippet for each result. ```python # Process results for result in results.results: print(f"{result.rank}. {result.title}") print(f" {result.snippet}") ``` -------------------------------- ### Combined Search and Formatting Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/utils.md Demonstrates a complete workflow: validating a search query, executing the search, and then formatting the response and statistics. Use this to see how different utility functions integrate. ```python from xmlriver_pro import GoogleClient from xmlriver_pro.utils import ( validate_query, validate_coords, format_search_response, format_search_stats ) client = GoogleClient(user_id=123, api_key="key") # Validate input query = "python programming" if not validate_query(query): print("Invalid query") else: # Execute search response = client.search(query) # Format output formatted = format_search_response(response) stats = format_search_stats(response) print(f"Query: {formatted['query']}") print(f"Total: {stats['total_results']:,}") print(f"Returned: {stats['returned_results']}") for result in formatted['results'][:5]: print(f"\n{result['rank']}. {result['title']}") print(f" {result['snippet']}") ``` -------------------------------- ### YandexClient Mobile Search Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Demonstrates how to perform a search query specifically for mobile devices using the DeviceType enum. ```python # Mobile search results = client.search("python", device=DeviceType.MOBILE) ``` -------------------------------- ### Compare Synchronous and Asynchronous Google Client Performance Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Compares the performance of synchronous and asynchronous Google Client operations. The synchronous example performs queries sequentially, while the asynchronous example uses `asyncio.gather` for concurrent execution, typically resulting in significant speed improvements. ```python import asyncio import time from xmlriver_pro import GoogleClient, AsyncGoogleClient # Synchronous: Sequential def sync_example(): client = GoogleClient(user_id=123, api_key="key") queries = ["python", "java", "cpp", "rust"] start = time.time() for query in queries: results = client.search(query) print(f"{query}: {results.total_results}") print(f"Total time: {time.time() - start:.1f}s") # Asynchronous: Concurrent async def async_example(): async with AsyncGoogleClient(user_id=123, api_key="key") as client: queries = ["python", "java", "cpp", "rust"] start = time.time() tasks = [client.search(q) for q in queries] results = await asyncio.gather(*tasks) for query, result in zip(queries, results): print(f"{query}: {result.total_results}") print(f"Total time: {time.time() - start:.1f}s") # Async is typically 3-4x faster for multiple queries ``` -------------------------------- ### Example Usage of SearchResponse Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Demonstrates how to access and display information from a SearchResponse object. Useful for iterating through results and checking for query suggestions. ```python response: SearchResponse print(f"Query: {response.query}") print(f"Total: {response.total_results}") print(f"Returned: {len(response.results)}") for result in response.results: print(f" - {result.title}") if response.correct: print(f"Did you mean: {response.correct}?") ``` -------------------------------- ### Install Specific Version of XMLRiver Pro Source: https://github.com/eapwrk/xmlriver-pro/blob/main/docs/llms.txt Install a specific version of the XMLRiver Pro library from GitHub using a version tag. This is useful for maintaining compatibility with older projects. ```bash pip install git+https://github.com/Eapwrk/xmlriver-pro.git@v1.0.2 ``` -------------------------------- ### AsyncGoogleClient Context Manager Usage Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Demonstrates the correct way to instantiate and use AsyncGoogleClient with an `async with` statement. This ensures that connections are properly managed and released. The example also shows an incorrect usage pattern that should be avoided. ```python import asyncio from xmlriver_pro import AsyncGoogleClient async def main(): # Correct: Using context manager async with AsyncGoogleClient(user_id=123, api_key="key") as client: results = await client.search("python") # Incorrect: Missing context manager # client = AsyncGoogleClient(user_id=123, api_key="key") # results = await client.search("python") # Will fail asyncio.run(main()) ``` -------------------------------- ### Retrieve and Print Ad Details Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-ads.md Example of fetching ads for a given query and printing the title, snippet, and URLs of the top ad. ```python response = ads.get_ads("python tutorial") ad = response.top_ads[0] print(f"Title: {ad.title}") print(f"Snippet: {ad.snippet}") print(f"URL: {ad.url}") print(f"Ads URL: {ad.ads_url}") ``` -------------------------------- ### YandexClient Search by Region Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Shows how to perform a search query filtered by a specific Yandex region ID, using Moscow (213) as an example. ```python # Search by region (Moscow = 213) results = client.search("python", lr=213) ``` -------------------------------- ### Example Usage of ImageResult Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Demonstrates how to display information from an ImageResult object. Ensure an ImageResult object is defined prior to execution. ```python img: ImageResult print(f"Title: {img.title}") print(f"Image: {img.img_url}") print(f"Size: {img.original_width}x{img.original_height}") ``` -------------------------------- ### Concurrent Request Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Demonstrates how to make multiple concurrent search requests using AsyncGoogleClient and asyncio.gather. Configure the maximum number of concurrent requests using the max_concurrent parameter. ```python import asyncio from xmlriver_pro import AsyncGoogleClient async def main(): async with AsyncGoogleClient( user_id=123, api_key="key", max_concurrent=5 ) as client: # Multiple concurrent requests queries = ["python", "java", "c++", "rust", "go"] tasks = [client.search(q) for q in queries] results = await asyncio.gather(*tasks) for query, result in zip(queries, results): print(f"{query}: {result.total_results} results") ``` -------------------------------- ### Combined Usage: Search and Format News Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-news.md An example demonstrating how to search for news from the last week and then format the results for display. ```python from xmlriver_pro import GoogleNews, TimeFilter from xmlriver_pro.utils import format_news_result news = GoogleNews(user_id=123, api_key="your_key") # Get recent news results = news.search_news_last_week("machine learning") # Format and display print(f"=== Machine Learning News (Last Week) ===") print(f"Total found: {results.total_results}") print(f"Returned: {len(results.results)}") print() for result in results.results: formatted = format_news_result(result) print(f"#{formatted['rank']} {formatted['title']}") print(f" Source: {formatted['media']}") print(f" Date: {formatted['pub_date']}") print(f" URL: {formatted['url']}") print() ``` -------------------------------- ### Example Usage of NewsResult Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Shows how to print key details from a NewsResult object. A NewsResult object must be available to run this code. ```python news_result: NewsResult print(f"Title: {news_result.title}") print(f"Source: {news_result.media}") print(f"Date: {news_result.pub_date}") ``` -------------------------------- ### Update to Latest Version from GitHub Source: https://github.com/eapwrk/xmlriver-pro/blob/main/docs/versioning.md Upgrade the installed XMLRiver Pro package to the latest version available on GitHub. ```bash pip install --upgrade git+https://github.com/Eapwrk/xmlriver-pro.git ``` -------------------------------- ### Retrieve API Limits Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/configuration.md Shows how to use the `get_api_limits()` method to fetch current system limits and provides an example of the returned data structure. ```APIDOC ## Get API Limits ### Description Retrieves the current API usage limits and system configurations. This method helps in understanding the operational constraints and best practices for using the API. ### Method ```python client.get_api_limits() ``` ### Usage Example ```python from xmlriver_pro import GoogleClient client = GoogleClient(user_id=123, api_key="key") limits = client.get_api_limits() print(f"Max concurrent streams: {limits['max_concurrent_streams']}") print(f"Default timeout: {limits['default_timeout']}") print(f"Max timeout: {limits['max_timeout']}") print(f"Typical response time: {limits['typical_response_time']}") print(f"Daily limits: {limits['daily_limits']}") print(f"Recommendations: {limits['recommendations']}") ``` ### Response #### Success Response (200) Returns a dictionary containing API limits and recommendations: - **max_concurrent_streams** (int): The maximum number of concurrent streams allowed. - **default_timeout** (int): The default timeout setting in seconds. - **max_timeout** (int): The maximum allowed timeout setting in seconds. - **typical_response_time** (tuple): A tuple representing the typical response time in seconds. - **daily_limits** (dict): A dictionary detailing daily usage limits for different services (e.g., google, yandex, wordstat). - **recommendations** (dict): A dictionary providing recommendations for timeout, concurrent requests, daily volume, and error handling. #### Response Example ```json { "max_concurrent_streams": 10, "default_timeout": 60, "max_timeout": 60, "typical_response_time": [3, 6], "daily_limits": { "google": 200000, "yandex": 150000, "wordstat": 150000 }, "recommendations": { "timeout": "Use 60 seconds for reliability", "concurrent_requests": "Max 10 concurrent per system", "daily_volume": "Respect daily limits to avoid blocking", "error_handling": "Handle codes 110, 111, 115 as temporary" } } ``` ``` -------------------------------- ### Check XMLRiver Pro Version Source: https://github.com/eapwrk/xmlriver-pro/blob/main/docs/llms.txt Verify the currently installed version of the XMLRiver Pro library. This command can be run from the terminal. ```python python -c "import xmlriver_pro; print(xmlriver_pro.__version__)" ``` -------------------------------- ### Async Client Usage with aiohttp Session Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/configuration.md Example of using AsyncGoogleClient with a default session and reusing an existing aiohttp ClientSession. Demonstrates setting max_concurrent requests. ```python from xmlriver_pro import AsyncGoogleClient import aiohttp async def main(): # Create with default session async with AsyncGoogleClient(user_id=123, api_key="key") as client: results = await client.search("python") # Reuse session async with aiohttp.ClientSession() as session: async with AsyncGoogleClient( user_id=123, api_key="key", session=session, max_concurrent=5 ) as client: results = await client.search("python") ``` -------------------------------- ### Paginate Through Search Results Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-news.md Provides an example of how to paginate through search results, as Yandex pages are zero-indexed. The `groupby=10` parameter is used here for demonstration. ```python from xmlriver_pro import YandexNews news = YandexNews(user_id=123, api_key="key") # Yandex pages start at 0 all_results = [] for page in range(0, 3): # 3 pages results = news.search_news("python", page=page, groupby=10) all_results.extend(results.results) print(f"Page {page}: {len(results.results)} articles") print(f"Total: {len(all_results)} articles") ``` -------------------------------- ### MapResult Example Usage Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Demonstrates how to access fields from a MapResult object. Ensure the 'location' variable is populated with a MapResult instance before use. ```python location: MapResult print(f"Name: {location.title}") print(f"Rating: {location.stars}★") print(f"Address: {location.address}") print(f"Phone: {location.phone}") print(f"Coords: {location.latitude}, {location.longitude}") ``` -------------------------------- ### YandexClient Search with Time Filter (Last 2 Weeks) Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Demonstrates searching for results from the last two weeks using 'within=1'. ```python # Last 2 weeks results = client.search_with_time_filter("python news", within=1) ``` -------------------------------- ### Search News with Regional and Time Filters Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-news.md Shows how to combine regional filtering (using `lr` for location ID) with time filtering (`within`) to get specific news results. For example, finding recent news in Moscow. ```python from xmlriver_pro import YandexNews news = YandexNews(user_id=123, api_key="key") # Regional + time filter moscow_recent = news.search_news( "python", lr=213, # Moscow within=77 # Last day ) print(f"Moscow news (last 24h): {moscow_recent.total_results}") ``` -------------------------------- ### GoogleClient Initialization with Environment Variables Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/configuration.md Demonstrates how to load user ID and API key from environment variables using python-dotenv to initialize the GoogleClient. ```APIDOC ## Initialize GoogleClient with Environment Variables ### Description This snippet shows how to configure the `GoogleClient` by loading credentials from a `.env` file using the `python-dotenv` library. It reads `XMLRIVER_USER_ID` and `XMLRIVER_API_KEY` from the environment. ### Usage ```python import os from dotenv import load_dotenv from xmlriver_pro import GoogleClient # Load environment variables from .env file load_dotenv() # Initialize the client using environment variables client = GoogleClient( user_id=int(os.getenv("XMLRIVER_USER_ID")), api_key=os.getenv("XMLRIVER_API_KEY") ) ``` ### Environment Variables - **XMLRIVER_USER_ID** (str): The user identifier. This value will be converted to an integer. - **XMLRIVER_API_KEY** (str): The API key for authentication. ``` -------------------------------- ### Competitive Analysis with Google Ads API Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-ads.md Example of using the GoogleAds client to track competitors' ad presence for a list of search queries. It retrieves and prints the number of top and bottom ads, and lists the titles of the top ads. ```python from xmlriver_pro import GoogleAds ads = GoogleAds(user_id=123, api_key="your_key") # Track competitors' ad presence competitors = [ "best python online course", "learn python programming", "python tutorial free", ] for query in competitors: response = ads.get_ads(query) print(f"\n'{query}'") print(f" Top ads: {len(response.top_ads)}") print(f" Bottom ads: {len(response.bottom_ads)}") # Show top ads for ad in response.top_ads[:2]: print(f" - {ad.title}") ``` -------------------------------- ### Get API limits and recommendations Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-client.md Fetch the current API limits, including maximum concurrent streams, default and maximum timeouts, and typical response times. It also provides daily limits per system and configuration recommendations. ```python def get_api_limits() -> Dict[str, Any]: limits = client.get_api_limits() print(f"Max concurrent: {limits['max_concurrent_streams']}") print(f"Daily limit (Google): {limits['daily_limits']['google']:,}") ``` -------------------------------- ### Configure GoogleClient with Environment Variables Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-client.md Shows how to initialize the GoogleClient using environment variables for sensitive credentials like user ID and API key. It also configures network parameters such as timeout, maximum retries, retry delay, and enables retry logic. This is recommended for secure and flexible API integration. ```python from xmlriver_pro import GoogleClient import os from dotenv import load_dotenv load_dotenv() client = GoogleClient( user_id=int(os.getenv("XMLRIVER_USER_ID")), api_key=os.getenv("XMLRIVER_API_KEY"), timeout=60, # 60 seconds (recommended) max_retries=3, # 3 retry attempts retry_delay=1.0, # 1 second base delay enable_retry=True # Enable exponential backoff ) # Check configuration limits = client.get_api_limits() balance = client.get_balance() cost = client.get_cost("google") ``` -------------------------------- ### Configure Async Google Client Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/configuration.md Instantiate and use the asynchronous Google client with custom configuration. Ensure proper context management with 'async with'. ```python import asyncio from xmlriver_pro import AsyncGoogleClient async def main(): # Configure async client async with AsyncGoogleClient( user_id=123, api_key="key", timeout=60, max_retries=3, retry_delay=1.0, enable_retry=True, max_concurrent=10 ) as client: # Perform searches results = await client.search("python") print(f"Found {results.total_results} results") asyncio.run(main()) ``` -------------------------------- ### get_concurrent_status Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Get the current concurrent request status synchronously. ```APIDOC ## get_concurrent_status ### Description Get current concurrent request status (synchronous). ### Method `def get_concurrent_status() -> Dict[str, int]` ### Parameters #### Path Parameters None #### Query Parameters None ### Request Example ```python async with AsyncGoogleClient(user_id=123, api_key="key") as client: status = client.get_concurrent_status() print(f"Active: {status['active_requests']}") print(f"Available: {status['available_slots']}") ``` ### Response #### Success Response (200) - **Dict[str, int]** - Dictionary containing: - `active_requests` - Currently running requests - `available_slots` - Available concurrent slots #### Response Example None provided in source. ``` -------------------------------- ### Basic Google Search with Python Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/INDEX.md Demonstrates how to perform a basic search using the GoogleClient and iterate through the results. ```python from xmlriver_pro import GoogleClient client = GoogleClient(user_id=123, api_key="your_key") results = client.search("python programming") for result in results.results: print(f"{result.rank}. {result.title}") print(f" {result.url}") ``` -------------------------------- ### Get Yandex API Limits Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Retrieves the current API limits and any associated recommendations for usage. ```python def get_api_limits() -> Dict[str, Any] ``` -------------------------------- ### Example Usage of Coords Type Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Demonstrates how to use the Coords type alias to define and unpack geographic coordinates. ```python from xmlriver_pro.core.types import Coords coords: Coords = (37.7749, -122.4194) # San Francisco latitude, longitude = coords ``` -------------------------------- ### AsyncGoogleClient Constructor Parameters Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Details the parameters for initializing the AsyncGoogleClient, including user ID, API key, timeout, retry settings, concurrency limit, and an optional aiohttp session. ```python AsyncGoogleClient( user_id: int, api_key: str, timeout: int = 60, max_retries: int = 3, retry_delay: float = 1.0, enable_retry: bool = True, max_concurrent: int = 10, session: Optional[aiohttp.ClientSession] = None ) ``` -------------------------------- ### Yandex Client Initialization and Search Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Demonstrates how to initialize the YandexClient and perform searches with region-specific parameters. ```APIDOC ## Yandex Client Initialization and Search ### Description Initialize the YandexClient with your user ID and API key. You can then perform searches, specifying parameters like `lr` (region code) to narrow down results to a particular geographic area. ### Method `YandexClient(user_id: int, api_key: str)` `search(query: str, **kwargs)` ### Parameters #### `YandexClient` Initialization - **user_id** (int) - Your Yandex user ID. - **api_key** (str) - Your Yandex API key. #### `search` Method Parameters - **query** (str) - The search query string. - **lr** (int) - Optional. The region code for the search. See 'Region Codes (lr)' for a list of common IDs. ### Request Example ```python from xmlriver_pro import YandexClient client = YandexClient(user_id=123, api_key="your_key") # Search in Moscow region moscow_results = client.search("python", lr=213) # Search in St. Petersburg region spb_results = client.search("python", lr=2) ``` ### Response - The `search` method returns a results object containing query details and a list of search results. ``` -------------------------------- ### Using AsyncGoogleClient as an Async Context Manager Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Demonstrates how to instantiate and use AsyncGoogleClient within an async context manager to perform searches. Ensure you have the necessary imports and an asyncio event loop. ```python import asyncio from xmlriver_pro import AsyncGoogleClient async def main(): async with AsyncGoogleClient(user_id=123, api_key="key") as client: # Use client results = await client.search("python") print(results) asyncio.run(main()) ``` -------------------------------- ### Configure Google Client with Timeout Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/configuration.md Instantiate a Google client and set a 60-second timeout. This is a recommended setting for production environments to prevent indefinite waits. ```python client = GoogleClient( user_id=user_id, api_key=api_key, timeout=60 # Recommended ) ``` -------------------------------- ### Initialize GoogleClient Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-client.md Instantiate the GoogleClient with your user ID and API key. Optional parameters control retry behavior and request timeouts. ```python from xmlriver_pro import GoogleClient, DeviceType client = GoogleClient(user_id=123, api_key="your_key") ``` -------------------------------- ### YandexClient Search with Time Filter (Last Month) Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Illustrates how to fetch results from the last month by setting 'within=2'. ```python # Last month results = client.search_with_time_filter("programming trends", within=2) ``` -------------------------------- ### Initialize GoogleImages Client Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-images.md Instantiate the GoogleImages client with your user ID and API key. This client is used for all subsequent image search operations. ```python from xmlriver_pro import GoogleImages images = GoogleImages(user_id=123, api_key="your_key") ``` -------------------------------- ### Get Yandex API Cost Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Fetches the cost per 1000 requests for the Yandex API. The default system is 'yandex'. ```python def get_cost(system: str = "yandex") -> float ``` -------------------------------- ### Example Usage of AdsResponse Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Shows how to access and print the counts and titles of ads from an AdsResponse object. Useful for displaying ad information. ```python ads_response: AdsResponse print(f"Top ads: {len(ads_response.top_ads)}") print(f"Bottom ads: {len(ads_response.bottom_ads)}") for ad in ads_response.top_ads: print(f" - {ad.title}") ``` -------------------------------- ### Tracking Search Progress Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Illustrates how to implement progress tracking for a list of search queries using AsyncGoogleClient and asyncio.gather. ```python import asyncio from xmlriver_pro import AsyncGoogleClient async def search_with_progress(queries: list): async with AsyncGoogleClient(user_id=123, api_key="key") as client: completed = 0 total = len(queries) async def search_and_track(query): nonlocal completed result = await client.search(query) completed += 1 print(f"Progress: {completed}/{total}") return result tasks = [search_and_track(q) for q in queries] results = await asyncio.gather(*tasks) return results queries = ["python", "java", "javascript", "cpp", "rust"] asyncio.run(search_with_progress(queries)) ``` -------------------------------- ### Initialize GoogleNews Client Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-news.md Instantiate the GoogleNews client with your user ID and API key. Configure optional parameters like timeout and retry settings. ```python from xmlriver_pro import GoogleNews news = GoogleNews(user_id=123, api_key="your_key") ``` -------------------------------- ### AsyncGoogleClient Constructor Parameters Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/configuration.md Parameters for AsyncGoogleClient, including common client options and async-specific settings like max_concurrent requests and session reuse. ```python from xmlriver_pro import AsyncGoogleClient async_client = AsyncGoogleClient( user_id: int, api_key: str, timeout: int = 60, max_retries: int = 3, retry_delay: float = 1.0, enable_retry: bool = True, max_concurrent: int = 10, # Max concurrent requests session: Optional[aiohttp.ClientSession] = None # Existing session ) ``` -------------------------------- ### Search Yandex News (Specific Region) Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-news.md Filters search results by a specific Yandex region ID. For example, '213' for Moscow. ```python # Search in specific region (Moscow = 213) results = news.search_news("python", lr=213) ``` -------------------------------- ### Development vs Production Google Client Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/configuration.md Set up distinct Google client configurations for development and production environments. Development clients are configured to fail fast, while production clients are set up for resilience. ```python import os from xmlriver_pro import GoogleClient # Development configuration dev_client = GoogleClient( user_id=int(os.getenv("DEV_USER_ID")), api_key=os.getenv("DEV_API_KEY") , enable_retry=False, # Fail fast for debugging max_retries=1 ) # Production configuration prod_client = GoogleClient( user_id=int(os.getenv("PROD_USER_ID")), api_key=os.getenv("PROD_API_KEY") , enable_retry=True, # Resilient max_retries=5, retry_delay=2.0 # Longer delays ) ``` -------------------------------- ### YandexClient Search on Specific Domain Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Illustrates how to conduct a search targeting a particular Yandex domain, such as the Russian domain ('ru'). ```python # Search on specific domain results = client.search("python", domain="ru") ``` -------------------------------- ### Manually Create Search Result and Response Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Illustrates how to manually instantiate a SearchResult object with specific attributes and then create a SearchResponse containing that result. ```python from xmlriver_pro.core.types import SearchResult, SearchResponse, Coords # Create single result result = SearchResult( rank=1, url="https://example.com", title="Example", snippet="Example snippet", stars=4.5 ) # Create response response = SearchResponse( query="example", total_results=1000, results=[result] ) ``` -------------------------------- ### Get Yandex Account Balance Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Retrieves the current account balance in rubles. This information is shared across all systems associated with the account. ```python def get_balance() -> float ``` -------------------------------- ### Runtime Dependencies Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/module-structure.md Lists the essential libraries required for the project to run, including version specifiers. ```text requests>=2.31.0 # HTTP client (sync) xmltodict>=0.13.0 # XML parsing aiohttp>=3.8.0 # HTTP client (async) ``` -------------------------------- ### Get account balance Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-client.md Retrieve the current account balance, which is shared across all search engine services. The balance is returned in rubles. ```python def get_balance() -> float: balance = client.get_balance() print(f"Account balance: {balance} rubles") ``` -------------------------------- ### Error Codes Quick Reference Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/INDEX.md A quick reference guide to common XMLRiver API error codes, their categories, and recommended actions. ```APIDOC ## Error Codes Quick Reference | Code | Category | Action | |------|----------|--------| | 2 | Validation | Check query | | 15 | NoResults | Try different query | | 31, 42, 45 | Auth | Check credentials | | 101, 201 | Service | Retry later | | 102-108, 120, 121 | Validation | Fix parameters | | 110, 111, 115 | RateLimit | Wait and retry | | 200 | Funds | Top up account | | 202, 500 | Network | Retry (auto-handled) | See [complete error codes](errors.md#error-code-mapping) for all codes. ``` -------------------------------- ### GoogleClient Constructor Parameters Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/configuration.md Common parameters for initializing XMLRiver Pro client classes like GoogleClient. Ensure user_id and api_key are provided. Timeout is capped at 60 seconds. ```python client = GoogleClient( user_id: int, api_key: str, timeout: int = 60, max_retries: int = 3, retry_delay: float = 1.0, enable_retry: bool = True, **kwargs: Any ) ``` -------------------------------- ### Usage of SearchType Enumeration Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/types.md Demonstrates how to import and use the SearchType enumeration to set search parameters. ```python from xmlriver_pro.core.types import SearchType search_type = SearchType.ORGANIC search_type = SearchType.NEWS search_type = SearchType.IMAGES ``` -------------------------------- ### Development Dependencies Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/module-structure.md Lists the libraries needed for development, testing, and code quality checks. ```text pytest>=7.0.0 # Testing pytest-cov>=4.0.0 # Coverage black>=23.0.0 # Formatting pylint>=2.17.0 # Linting mypy>=1.0.0 # Type checking pre-commit>=3.0.0 # Git hooks ``` -------------------------------- ### YandexClient Search with Time Filter (Last Day) Example Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Shows how to use search_with_time_filter to retrieve results from the last day, using 'within=77'. ```python # Last day results = client.search_with_time_filter("python", within=77) ``` -------------------------------- ### Initialize GoogleAds Client Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-ads.md Instantiate the GoogleAds client with your user ID and API key. This client is used to interact with the Google Ads API. ```python from xmlriver_pro import GoogleAds ads = GoogleAds(user_id=123, api_key="your_key") ``` -------------------------------- ### Handle RateLimitError with Retry Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/errors.md Example of catching RateLimitError and implementing a manual retry mechanism with a delay. This is useful for temporary rate limiting issues. ```python from xmlriver_pro.core import RateLimitError import time try: results = client.search("python") except RateLimitError as e: print(f"Rate limited (code {e.code})") time.sleep(5) results = client.search("python") # Retry after delay ``` -------------------------------- ### AsyncGoogleClient Constructor Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Initializes the AsyncGoogleClient with user credentials and optional configuration for retries, timeouts, and concurrency. ```APIDOC ## Constructor ```python AsyncGoogleClient( user_id: int, api_key: str, timeout: int = 60, max_retries: int = 3, retry_delay: float = 1.0, enable_retry: bool = True, max_concurrent: int = 10, session: Optional[aiohttp.ClientSession] = None ) ``` ### Parameters | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | user_id | int | Yes | — | XMLRiver user ID | | api_key | str | Yes | — | XMLRiver API key | | timeout | int | No | 60 | Request timeout in seconds | | max_retries | int | No | 3 | Maximum retry attempts | | retry_delay | float | No | 1.0 | Base delay between retries | | enable_retry | bool | No | True | Enable automatic retries | | max_concurrent | int | No | 10 | Max concurrent requests | | session | Optional[aiohttp.ClientSession] | No | None | Existing aiohttp session | ``` -------------------------------- ### Handle Generic XMLRiverError Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/errors.md Example of catching the base XMLRiverError to handle any API-related exceptions. Useful for general error logging or fallback mechanisms. ```python try: results = client.search("query") except XMLRiverError as e: print(f"Error [{e.code}]: {e.message}") ``` -------------------------------- ### Reusing aiohttp.ClientSession Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Demonstrates how to create and reuse an aiohttp.ClientSession for multiple AsyncGoogleClient instances to improve performance. ```python import asyncio import aiohttp from xmlriver_pro import AsyncGoogleClient async def main(): # Create session once async with aiohttp.ClientSession() as session: # Use same session for multiple clients async with AsyncGoogleClient( user_id=123, api_key="key", session=session ) as google: results = await google.search("python") print(f"Found {results.total_results} results") asyncio.run(main()) ``` -------------------------------- ### Get Account Balance Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Asynchronously retrieve the current account balance. The balance is returned as a float value, typically representing currency units. ```python async with AsyncGoogleClient(user_id=123, api_key="key") as client: balance = await client.get_balance() print(f"Balance: {balance} rubles") ``` -------------------------------- ### Get Top Ads for a Query Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-ads.md Fetch only the advertisements that appear above the organic search results for a specific query. Useful for identifying premium ad placements. ```python top_ads = ads.get_top_ads("best laptops 2024") print(f"Found {len(top_ads)} top ads") for ad in top_ads: print(f"- {ad.title}") print(f" {ad.snippet}") ``` -------------------------------- ### Fix Authentication Errors with Environment Variables Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/errors.md Demonstrates how to fix authentication errors by loading credentials from environment variables using dotenv and providing user guidance. ```python import os from dotenv import load_dotenv from xmlriver_pro import GoogleClient from xmlriver_pro.core import AuthenticationError load_dotenv() try: client = GoogleClient( user_id=int(os.getenv("XMLRIVER_USER_ID")), api_key=os.getenv("XMLRIVER_API_KEY") ) results = client.search("python") except AuthenticationError as e: print(f"Authentication failed: {e.message}") print("Please verify XMLRIVER_USER_ID and XMLRIVER_API_KEY") ``` -------------------------------- ### Get All Ads for a Query Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-ads.md Retrieve all paid advertisements (top and bottom) for a given search query. The response object contains lists for top_ads and bottom_ads. ```python response = ads.get_ads("python programming course") print(f"Top ads: {len(response.top_ads)}") print(f"Bottom ads: {len(response.bottom_ads)}") print(f"Total ads: {len(response.top_ads) + len(response.bottom_ads)}") # Process top ads for ad in response.top_ads: print(f"Title: {ad.title}") print(f"URL: {ad.url}") print(f"Snippet: {ad.snippet}") ``` -------------------------------- ### GoogleAds Client Initialization Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-ads.md Initialize the GoogleAds client with your user ID and API key. The client inherits from BaseClient, so refer to its documentation for additional parameters. ```APIDOC ## Initialize GoogleAds Client ### Description Instantiate the `GoogleAds` client by providing your `user_id` and `api_key`. ### Code Example ```python from xmlriver_pro import GoogleAds # Replace with your actual user ID and API key ads_client = GoogleAds(user_id=123, api_key="your_key") ``` ``` -------------------------------- ### Handle NoResultsError Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/errors.md Example of catching NoResultsError when a search query returns no matching results. This allows for graceful handling, such as informing the user or attempting a different query. ```python from xmlriver_pro.core import NoResultsError try: results = client.search("xyzabc123notaword") except NoResultsError: print("No results found") # Handle gracefully, use different query, etc. ``` -------------------------------- ### Implement Rate Limiting with AsyncGoogleClient Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Demonstrates how to implement rate limiting between requests when using the AsyncGoogleClient. A delay is introduced after each search operation, except for the last one, to manage request frequency. ```python import asyncio from xmlriver_pro import AsyncGoogleClient async def rate_limited_search(queries: list, delay: float = 1.0): async with AsyncGoogleClient(user_id=123, api_key="key") as client: results = [] for query in queries: result = await client.search(query) results.append(result) # Rate limit: delay between requests if query != queries[-1]: # Not last query await asyncio.sleep(delay) return results queries = ["python", "java", "cpp"] results = asyncio.run(rate_limited_search(queries, delay=2.0)) ``` -------------------------------- ### Get Ads by Specific Domain Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-ads.md Filter advertisements to retrieve only those from a specified domain for a given search query. This is useful for targeting ads from particular advertisers or platforms. ```python # Get ads only from Udemy udemy_ads = ads.get_ads_by_domain("python course", "udemy.com") print(f"Udemy ads: {len(udemy_ads)}") for ad in udemy_ads: print(f"- {ad.title}") ``` -------------------------------- ### Create Project Release Source: https://github.com/eapwrk/xmlriver-pro/blob/main/docs/versioning.md This script assists in creating a new release by performing necessary checks. Specify the version to be released. ```bash python create_release.py 1.1.0 ``` -------------------------------- ### Get cost per 1000 requests for a search system Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/google-client.md Obtain the cost for 1000 requests for a specified search system, such as Google or Yandex. The cost is provided in rubles. ```python def get_cost(system: str = "google") -> float: # Get Google cost google_cost = client.get_cost("google") print(f"Google cost: {google_cost} rubles per 1000 requests") ``` -------------------------------- ### AsyncGoogleClient Usage as Context Manager Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/async-google-client.md Demonstrates how to use AsyncGoogleClient as an asynchronous context manager to perform searches. ```APIDOC ## Context Manager AsyncGoogleClient must be used as async context manager: ```python import asyncio from xmlriver_pro import AsyncGoogleClient async def main(): async with AsyncGoogleClient(user_id=123, api_key="key") as client: # Use client results = await client.search("python") print(results) asyncio.run(main()) ``` ``` -------------------------------- ### Paginate Search Results Source: https://github.com/eapwrk/xmlriver-pro/blob/main/_autodocs/api-reference/yandex-client.md Iterate through multiple pages of search results. Yandex pages start at index 0. Ensure to handle the number of pages you need to fetch. ```python from xmlriver_pro import YandexClient client = YandexClient(user_id=123, api_key="your_key") # Yandex pages start at 0 all_results = [] for page in range(0, 3): # 3 pages results = client.search("python", page=page, groupby=10) all_results.extend(results.results) print(f"Page {page}: {len(results.results)} results") print(f"Total: {len(all_results)} results") ```