### Full Setup with Environment Variables Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md Example demonstrating how to set up the SDK using environment variables for API key and URL. Ensure the .env file is correctly configured. ```bash # .env file OXYLABS_AI_STUDIO_API_KEY=your-api-key OXYLABS_AI_STUDIO_API_URL=https://api-aistudio.oxylabs.io ``` -------------------------------- ### Install and Setup Oxylabs AI Studio Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/examples.md Install the library using pip and load your API key from a .env file for authentication. ```python # Install # pip install oxylabs-ai-studio # Create .env file # OXYLABS_AI_STUDIO_API_KEY=your-api-key # In code import os from dotenv import load_dotenv load_dotenv() # Load from .env file ``` -------------------------------- ### Full Setup with Python Code Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md Complete Python application setup including logging configuration and client initialization using environment variables. Ensure logging level is set appropriately. ```python # app.py import os from oxylabs_ai_studio.apps.ai_scraper import AiScraper from oxylabs_ai_studio.logger import configure_logging import logging # Configure logging configure_logging(level=logging.INFO) ``` -------------------------------- ### Configuration: Logging Setup Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/examples.md Configure logging for the Oxylabs AI Studio SDK. Examples show enabling debug logging, file logging, and custom log message formats. ```python from oxylabs_ai_studio.logger import configure_logging import logging # Debug logging configure_logging(level=logging.DEBUG) # File logging import logging.handlers handler = logging.FileHandler("oxylabs.log") configure_logging(handler=handler) # Custom format configure_logging( level=logging.INFO, format_string="%(asctime)s | %(name)s | %(message)s" ) ``` -------------------------------- ### AiSearch Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Demonstrates how to use the AiSearch client to perform an instant search query. ```python from oxylabs_ai_studio.apps.ai_search import AiSearch search = AiSearch(api_key="key") results = search.instant_search(query="python") ``` -------------------------------- ### AiMap Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Shows how to initialize and use the AiMap client to map a URL with specific search keywords. ```python from oxylabs_ai_studio.apps.ai_map import AiMap mapper = AiMap(api_key="key") result = mapper.map( url="https://example.com", search_keywords=["api", "docs"] ) ``` -------------------------------- ### Install Oxylabs AI Studio Python SDK Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/overview.md Install the SDK using pip. Ensure you have Python 3.10 or higher. ```bash pip install oxylabs-ai-studio ``` -------------------------------- ### Configure Logging Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Demonstrates how to configure logging for the SDK, including enabling debug logging and setting a custom format. ```python from oxylabs_ai_studio.logger import configure_logging import logging # Enable debug logging configure_logging(level=logging.DEBUG) # Custom format configure_logging( level=logging.INFO, format_string="%(asctime)s | %(name)s | %(levelname)s | %(message)s" ) ``` -------------------------------- ### Full Scrape and Schema Usage Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aiscraper.md A complete example showing how to generate a schema, then use it to scrape a product page and process the results. It includes error handling for failed scrapes. ```python scraper = AiScraper(api_key="key") schema = scraper.generate_schema(prompt="name, price, rating") result = scraper.scrape( url="https://example.com/product", output_format="json", schema=schema ) if result.data is None: print(f"Scrape failed: {result.message}") else: print(f"Job {result.run_id} succeeded") print(f"Data: {result.data}") ``` -------------------------------- ### Quick Start Usage of AiScraper Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Basic example of using AiScraper after setting the API key in the environment. Reads API key automatically. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper scraper = AiScraper() # Reads from environment result = scraper.scrape( url="https://example.com", output_format="markdown" ) if result.data: print(result.data) ``` -------------------------------- ### BrowserAgent Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Illustrates how to initialize and use the BrowserAgent client to interact with a webpage and execute a user-defined prompt. ```python from oxylabs_ai_studio.apps.browser_agent import BrowserAgent agent = BrowserAgent(api_key="key") result = agent.run( url="https://example.com", user_prompt="Search for 'widget'" ) ``` -------------------------------- ### Example Usage of AiMap Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/types.md Illustrates how to initialize the AiMap client, perform a mapping operation, and then process the discovered URLs or handle potential errors. ```python from oxylabs_ai_studio.apps.ai_map import AiMap mapper = AiMap(api_key="key") job = mapper.map(url="...", search_keywords=["api"]) if job.data is None: print(f"Failed: {job.message}") else: print(f"Discovered {len(job.data)} URLs:") for url in job.data: print(f" {url}") ``` -------------------------------- ### Start a Crawl Job Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Initiates a crawl job by providing the starting URL and a natural language prompt. Specify output format, JavaScript rendering, and location for tailored crawling. ```json { "url": "string", "user_prompt": "string", "output_format": "json|markdown|csv|toon", "openapi_schema": null|{...}, "render_javascript": boolean, "return_sources_limit": integer, "geo_location": null|"string", "max_credits": null|integer } ``` -------------------------------- ### Basic Crawling with AiCrawler (Markdown) Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/examples.md Initiate a basic crawl from a starting URL and provide a user prompt to guide the crawler. Results are returned in Markdown format. ```python from oxylabs_ai_studio.apps.ai_crawler import AiCrawler crawler = AiCrawler(api_key="your-api-key") result = crawler.crawl( url="https://example.com", user_prompt="Find all product pages" ) if result.data: for page in result.data: print(page) else: print(f"Error: {result.message}") ``` -------------------------------- ### AiScraper Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Shows how to use the AiScraper client to generate a schema from a prompt and then scrape a URL using that schema. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper scraper = AiScraper(api_key="key") schema = scraper.generate_schema(prompt="title, price") result = scraper.scrape( url="https://example.com", output_format="json", schema=schema ) ``` -------------------------------- ### Async Scraper Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Provides an example of using the asynchronous version of the AiScraper client to scrape a URL. ```python import asyncio from oxylabs_ai_studio.apps.ai_scraper import AiScraper async def main(): scraper = AiScraper(api_key="key") # Use async version result = await scraper.scrape_async( url="https://example.com" ) return result result = asyncio.run(main()) ``` -------------------------------- ### Start Browser Agent Session Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Initiates a browser agent session to perform tasks like navigating URLs and extracting information. Specify the starting URL, user prompt, and desired output format. ```json { "url": "string", "user_prompt": "string", "output_format": "json|markdown|html|screenshot|csv|toon", "openapi_schema": null|{...}, "geo_location": null|"string" } ``` -------------------------------- ### Example Usage of Browser Agent DataModel Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/types.md Demonstrates how to instantiate the BrowserAgent, run a job, and process the returned DataModel, checking the type and content. ```python from oxylabs_ai_studio.apps.browser_agent import BrowserAgent, DataModel agent = BrowserAgent(api_key="key") job = agent.run(url="...", output_format="json", schema=schema) if job.data: data: DataModel = job.data if data.type == "json" and isinstance(data.content, dict): print(f"JSON data: {data.content}") ``` -------------------------------- ### Example Usage of AiSearch Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/types.md Shows how to initialize the AiSearch client, perform a search, and then iterate through the results or print an error message if the search failed. ```python from oxylabs_ai_studio.apps.ai_search import AiSearch search = AiSearch(api_key="key") job = search.search(query="...", limit=10) if job.data is None: print(f"Failed: {job.message}") else: for result in job.data: print(f"{result.title} - {result.url}") ``` -------------------------------- ### Configure Logging with Basic Settings Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md Use the `configure_logging` function to set up basic logging for the SDK. This example sets the logging level to `DEBUG` to capture detailed information. ```python from oxylabs_ai_studio.logger import configure_logging import logging # Set to DEBUG level configure_logging(level=logging.DEBUG) ``` -------------------------------- ### API Key Validation Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Shows how to validate an API key using the `is_api_key_valid` utility function. ```python from oxylabs_ai_studio.utils import is_api_key_valid if not is_api_key_valid("your-key"): print("Invalid or unreachable API key") ``` -------------------------------- ### Example Usage of SearchResult with Content Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/types.md Demonstrates performing a search with `return_content=True` and then printing details of each SearchResult, including a snippet of the content if available. ```python from oxylabs_ai_studio.apps.ai_search import SearchResult results = search.search(query="python", limit=5, return_content=True) for result in results.data: print(f"Title: {result.title}") print(f"URL: {result.url}") print(f"Description: {result.description}") if result.content: print(f"Content: {result.content[:200]}...") ``` -------------------------------- ### AiCrawler Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Demonstrates how to initialize and use the AiCrawler client to crawl a URL and extract specific information based on a user prompt. ```python from oxylabs_ai_studio.apps.ai_crawler import AiCrawler crawler = AiCrawler(api_key="key") result = crawler.crawl( url="https://example.com", user_prompt="Find all pricing pages" ) ``` -------------------------------- ### Start Website Mapping Job Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Initiates a website mapping job to discover URLs starting from a given URL. Configure search keywords, crawl depth, JavaScript rendering, and domain inclusion rules. ```json { "url": "string", "search_keywords": ["string"], "user_prompt": null|"string", "max_crawl_depth": integer, "limit": integer, "geo_location": null|"string", "render_javascript": boolean, "include_sitemap": boolean, "max_credits": null|integer, "allow_subdomains": boolean, "allow_external_domains": boolean } ``` ```json { "run_id": "string" } ``` -------------------------------- ### Use AiScraper after setup Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md After setting up your environment and API key, you can instantiate and use the `AiScraper` client for your AI Studio tasks. The client will automatically pick up the configuration. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper # No api_key parameter needed scraper = AiScraper() result = scraper.scrape(url="https://example.com") ``` -------------------------------- ### Initialize AiScraper with API Key from Runtime Environment Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md This example demonstrates how to fetch the API key from the environment at runtime using `os.environ.get` and raise an error if it's not found. This provides a robust way to handle API key configuration. ```python import os from oxylabs_ai_studio.apps.ai_scraper import AiScraper api_key = os.environ.get('OXYLABS_AI_STUDIO_API_KEY') if not api_key: raise ValueError("API key not set") scraper = AiScraper(api_key=api_key) ``` -------------------------------- ### Quick Start: Initialize and Use AiScraper Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/overview.md Demonstrates the basic pattern for initializing an AiScraper client and performing a scrape operation. This includes generating a schema, calling the scrape method, and accessing the results. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper # 1. Initialize client with API key scraper = AiScraper(api_key="your-api-key") # 2. Generate schema if using structured extraction (optional) schema = scraper.generate_schema(prompt="extract: title, price, rating") # 3. Call service method with parameters result = scraper.scrape( url="https://example.com", output_format="json", schema=schema ) # 4. Access results from job object for item in result.data: print(item) ``` -------------------------------- ### Browser-Agent Sync Interface Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/agentic_code_guide.md Use the synchronous interface of the BrowserAgent to automate browser actions based on a user prompt. Ensure you have your API key configured. ```python from oxylabs_ai_studio.apps.browser_agent import BrowserAgent browser_agent = BrowserAgent(api_key="") prompt = "Find if there is game 'super mario odyssey' in the store." url = "https://sandbox.oxylabs.io/" result = browser_agent.run( url=url, user_prompt=prompt, output_format="json", schema={"type": "object", "properties": {"page_url": {"type": "string"}}, "required": []}, ) print(result.data) ``` -------------------------------- ### Complete Application: E-commerce Product Scraping Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/examples.md A full-stack example demonstrating how to scrape product details (name, price, rating, stock, description) from multiple e-commerce URLs in parallel. Includes schema generation, JavaScript rendering, and error handling. ```python """ Complete example: Scrape product data from an e-commerce site """ import asyncio from oxylabs_ai_studio.apps.ai_scraper import AiScraper from oxylabs_ai_studio.logger import configure_logging import logging # Configure logging configure_logging(level=logging.INFO) async def scrape_products(product_urls: list[str]) -> list[dict]: """Scrape product data from multiple URLs""" scraper = AiScraper(api_key="your-api-key") # Generate schema once schema = await scraper.generate_schema_async( prompt="name, price, rating, in_stock, description" ) if not schema: raise ValueError("Failed to generate schema") # Scrape all products in parallel tasks = [ scraper.scrape_async( url=url, output_format="json", schema=schema, render_javascript=True ) for url in product_urls ] results = await asyncio.gather(*tasks) # Extract data products = [] for result in results: if result.data and isinstance(result.data, dict): products.append(result.data) else: print(f"Failed to scrape: {result.message}") return products async def main(): # Product URLs to scrape urls = [ "https://example.com/product/1", "https://example.com/product/2", "https://example.com/product/3", ] try: products = await scrape_products(urls) print(f"\nScraped {len(products)} products:\n") for p in products: print(f"Name: {p.get('name')}") print(f"Price: ${p.get('price')}") print(f"Rating: {p.get('rating')}/5") print(f"In Stock: {p.get('in_stock')}") print(f"Description: {p.get('description', 'N/A')[:100]}...") print() except Exception as e: print(f"Error: {e}") if __name__ == "__main__": asyncio.run(main()) ``` -------------------------------- ### Example Usage of SchemaResponse Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/types.md Demonstrates how to create and access data from a SchemaResponse object. Useful for validating structured data extraction. ```python from oxylabs_ai_studio.models import SchemaResponse schema_response: SchemaResponse = { "openapi_schema": { "type": "object", "properties": { "title": {"type": "string"}, "price": {"type": "number"} } } } schema = schema_response["openapi_schema"] ``` -------------------------------- ### Job Status Lifecycle Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Illustrates the typical sequence of requests for a job that requires polling for status updates. ```http POST /xxx/run → 200 OK ↓ GET /xxx/run/data (status: "processing") ↓ GET /xxx/run/data (status: "processing") ↓ GET /xxx/run/data (status: "completed" | "failed") ``` -------------------------------- ### Quick Instant Search Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aisearch.md Performs a fast, instant search without content extraction. Ideal for quickly retrieving a list of relevant URLs and titles. ```python search = AiSearch(api_key="key") # Fast, no content, no polling results = search.instant_search( query="python libraries", limit=10 ) for result in results.data: print(f"{result.title} - {result.url}") ``` -------------------------------- ### Example Usage of BrowserAgentJob Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/types.md Illustrates how to run a browser agent task and process the BrowserAgentJob result, accessing the type and content of the returned data. ```python from oxylabs_ai_studio.apps.browser_agent import BrowserAgent agent = BrowserAgent(api_key="key") job = agent.run(url="...", user_prompt="...") if job.data is None: print(f"Failed: {job.message}") else: print(f"Type: {job.data.type}") print(f"Content: {job.data.content}") ``` -------------------------------- ### Find Documentation URLs with AiMap Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aimap.md Maps a documentation site to find relevant pages using keywords like 'api' and 'guide'. Limits the results to 50 URLs. ```python result = mapper.map( url="https://docs.example.com", search_keywords=["api", "reference", "guide", "tutorial"], max_crawl_depth=2, limit=50 ) print("Documentation URLs found:") for url in result.data: print(f" {url}") ``` -------------------------------- ### Map Website Using Natural Language Prompt Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aimap.md Discover URLs by providing a natural language prompt describing the content you are looking for. This example also enables JavaScript rendering, which is useful for dynamic websites. ```python # Map using natural language prompt result = mapper.map( url="https://career.example.com", user_prompt="job listings and career pages", max_crawl_depth=2, limit=10, render_javascript=True ) ``` -------------------------------- ### Example Usage of AiScraperJob Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/types.md Demonstrates initiating a scrape and handling the AiScraperJob result, differentiating between dictionary and string data formats. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper scraper = AiScraper(api_key="key") job = scraper.scrape(url="...", output_format="json", schema=schema) if isinstance(job.data, dict): print(f"Product: {job.data}") elif isinstance(job.data, str): print(f"Content: {job.data}") ``` -------------------------------- ### Example Usage of AiCrawlerJob Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/types.md Shows how to initiate a crawl and process the AiCrawlerJob result. Handles potential job failures and iterates through crawled data. ```python from oxylabs_ai_studio.apps.ai_crawler import AiCrawler crawler = AiCrawler(api_key="key") job = crawler.crawl(url="...", user_prompt="...") if job.data is None: print(f"Failed: {job.message}") else: for item in job.data: print(item) ``` -------------------------------- ### POST /browser-agent/run Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Starts a browser agent session to perform tasks based on a URL and user prompt. Supports various output formats and optional schema for structured extraction. ```APIDOC ## POST /browser-agent/run ### Description Start a browser agent session. ### Method POST ### Endpoint /browser-agent/run ### Parameters #### Request Body - **url** (string) - Yes - Starting URL - **user_prompt** (string) - No - Instructions for agent (default: "") - **output_format** (enum) - No - Output format (default: "markdown") - **openapi_schema** (object) - No - Schema for structured extraction - **geo_location** (string) - No - Proxy location ### Response #### Success Response (200 OK) - **run_id** (string) - Description ### Request Example { "url": "string", "user_prompt": "string", "output_format": "json|markdown|html|screenshot|csv|toon", "openapi_schema": null|{...}, "geo_location": null|"string" } ### Response Example { "run_id": "string" } ``` -------------------------------- ### POST /crawl/run Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Starts a crawl job by providing a URL and a natural language prompt. It supports various output formats and JavaScript rendering. ```APIDOC ## POST /crawl/run ### Description Start a crawl job. ### Method POST ### Endpoint /crawl/run ### Parameters #### Request Body - **url** (string) - Yes - Starting URL to crawl - **user_prompt** (string) - Yes - Natural language instruction for what to crawl - **output_format** (enum) - No - "json", "markdown", "csv", "toon" (default: "markdown") - **openapi_schema** (object) - No - JSON schema for structured extraction. Required if output_format is json/csv/toon - **render_javascript** (boolean) - No - Render JavaScript (default: false) - **return_sources_limit** (integer) - No - Max number of sources (default: 25, range: 1-50) - **geo_location** (string) - No - Proxy location (e.g., "US", "United States") - **max_credits** (integer) - No - Max credits to consume ### Request Example ```json { "url": "string", "user_prompt": "string", "output_format": "json|markdown|csv|toon", "openapi_schema": null|{...}, "render_javascript": boolean, "return_sources_limit": integer, "geo_location": null|string", "max_credits": null|integer } ``` ### Response #### Success Response (200 OK) - **run_id** (string) - Unique job identifier for polling #### Response Example ```json { "run_id": "string" } ``` #### Status Codes - 200: Job created successfully - 4xx: Bad request (invalid parameters) - 5xx: Server error ``` -------------------------------- ### Asynchronous Crawl Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aicrawler.md Demonstrates how to perform an asynchronous crawl using `crawl_async` within an async function. This method is non-blocking and suitable for integration into event-driven applications. ```python import asyncio from oxylabs_ai_studio.apps.ai_crawler import AiCrawler async def crawl_site(): crawler = AiCrawler(api_key="key") result = await crawler.crawl_async( url="https://example.com", user_prompt="Find all product pages", output_format="markdown" ) return result results = asyncio.run(crawl_site()) ``` -------------------------------- ### Start a Scrape Job Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Initiates a scrape job for a single page, specifying the URL and desired output format. Options include JavaScript rendering, geolocation, and user agent. ```json { "url": "string", "output_format": "json|markdown|csv|screenshot|toon", "openapi_schema": null|{...}, "render_javascript": boolean|"auto", "geo_location": null|"string", "user_agent": null|"string" } ``` -------------------------------- ### Browser-Agent Async Interface Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/agentic_code_guide.md Utilize the asynchronous interface of the BrowserAgent for non-blocking browser automation. This is suitable for applications requiring concurrent operations. ```python import asyncio from oxylabs_ai_studio.apps.browser_agent import BrowserAgent browser_agent = BrowserAgent(api_key="") async def main(): prompt = "Find if there is game 'super mario odyssey' in the store." url = "https://sandbox.oxylabs.io/" result = await browser_agent.run_async( url=url, user_prompt=prompt, output_format="json", schema={"type": "object", "properties": {"page_url": {"type": "string"}}, "required": []}, ) print(result.data) if __name__ == "__main__": asyncio.run(main()) ``` -------------------------------- ### Use Asynchronous HTTP Client Context Manager Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/oxystudioaiclient.md Utilize the async_client() method as an asynchronous context manager to get an httpx.AsyncClient. This ensures proper cleanup of the client after use. ```python client = OxyStudioAIClient(api_key="key") async with client.async_client() as http_client: response = await http_client.post("/endpoint", json=body) ``` -------------------------------- ### Map Website Using Keywords Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aimap.md Crawl a website to find URLs matching specific keywords. This example sets a maximum crawl depth and a limit for the number of results, and specifies a geographic location for the crawl. ```python mapper = AiMap(api_key="key") # Map using keywords result = mapper.map( url="https://oxylabs.io", search_keywords=["proxy", "pricing", "product"], max_crawl_depth=2, limit=50, geo_location="United States" ) if result.data: print(f"Found {len(result.data)} matching URLs") for url_item in result.data: print(f" {url_item}") ``` -------------------------------- ### Natural Language Site Mapping with AiMap Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/examples.md Map a website using a natural language prompt to guide the discovery of specific types of pages. This is powerful for finding content based on user intent. ```python result = mapper.map( url="https://career.example.com", user_prompt="job listing pages and career opportunities", max_crawl_depth=2 ) if result.data: for url in result.data: print(url) ``` -------------------------------- ### Check API Health and Validate Key Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Use the GET /status endpoint to check if the API is healthy and if your provided API key is valid. This is useful for initial setup and ongoing validation. ```http GET /status ``` -------------------------------- ### Initialize AiMap Client Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aimap.md Instantiate the AiMap client. You can provide your API key directly or rely on the `OXYLABS_AI_STUDIO_API_KEY` environment variable. ```python from oxylabs_ai_studio.apps.ai_map import AiMap mapper = AiMap(api_key="your-api-key") # or use OXYLABS_AI_STUDIO_API_KEY environment variable mapper = AiMap() ``` -------------------------------- ### Example Schema Output Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aiscraper.md This is an example of a JSON schema that can be used with the AI Scraper to define the structure of the data to be extracted. ```python { "type": "object", "properties": { "title": {"type": "string"}, "price": {"type": "number"}, "genres": {"type": "array", "items": {"type": "string"}}, "release_date": {"type": "string"} }, "required": ["title"] } ``` -------------------------------- ### Start Search Job with Content Extraction Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Initiates a search job to find information on the web and extract content from the results. Configure the search query, result limit, JavaScript rendering, and content extraction preferences. ```json { "query": "string", "limit": integer, "render_javascript": boolean, "return_content": boolean, "geo_location": null|"string" } ``` ```json { "run_id": "string" } ``` -------------------------------- ### Accessing Settings Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/utilities.md Demonstrates how to import and access global settings like API key and URL. ```python from oxylabs_ai_studio.settings import settings # Access configuration api_key = settings.OXYLABS_AI_STUDIO_API_KEY api_url = settings.OXYLABS_AI_STUDIO_API_URL print(f"Using API: {api_url}") ``` -------------------------------- ### AiCrawlerJob Usage Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aicrawler.md Provides a practical example of using the `AiCrawlerJob` object after a crawl operation. It demonstrates checking for crawl success, accessing the run ID, and iterating through the extracted data. ```python crawler = AiCrawler(api_key="key") result = crawler.crawl( url="https://example.com", user_prompt="Find all product listings" ) if result.data is None: print(f"Crawl failed: {result.message}") else: print(f"Job {result.run_id} completed with {len(result.data)} items") for item in result.data: print(item) ``` -------------------------------- ### Initialize AI Studio Client Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Instantiate the client by providing your API key. This client object will be used to make authenticated requests to the API. ```python from oxylabs_ai_studio.client import OxyStudioAIClient client = OxyStudioAIClient(api_key="your-key") ``` -------------------------------- ### GET /scrape/run/data Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Polls the status and retrieves the results of a scrape job using its run ID. ```APIDOC ## GET /scrape/run/data ### Description Poll scrape job status and results. ### Method GET ### Endpoint /scrape/run/data ### Parameters #### Query Parameters - **run_id** (string) - Yes - Job identifier ### Response #### Success Response (200 OK) - **status** (enum) - "completed" or "failed" - **data** (string|object) - Scraped data, null if failed - **error_code** (string) - Error code if status is "failed", otherwise null #### Response Example ```json { "status": "completed|failed", "data": null|string|{...}, "error_code": null|string" } ``` ``` -------------------------------- ### GET /crawl/run/data Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Polls the status and retrieves results of a crawl job using its unique run ID. ```APIDOC ## GET /crawl/run/data ### Description Poll crawl job status and results. ### Method GET ### Endpoint /crawl/run/data ### Parameters #### Query Parameters - **run_id** (string) - Yes - Job identifier from POST /crawl/run ### Response #### Success Response (200 OK) - **status** (enum) - "processing" = still running, "completed" = done, "failed" = error - **data** (array) - Crawled data. List of strings (markdown) or objects (json/csv) - **error_code** (string) - Error code if status is "failed", otherwise null #### Response Example ```json { "status": "processing|completed|failed", "data": [...], "error_code": null|string" } ``` #### Response (202 Accepted) Job still processing, poll again later. #### Status Codes - 200: Status available - 202: Still processing - 4xx: Invalid job ID - 5xx: Server error ``` -------------------------------- ### Initialize and Scrape with AI Studio SDK Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md This snippet shows how to retrieve your API key from environment variables, initialize the AiScraper client, generate a scraping schema from a prompt, and perform a scrape operation. It includes basic error handling for missing API keys and checks the scrape result. ```python import os from oxylabs.ai_studio import AiScraper # Get API key from environment api_key = os.environ.get('OXYLABS_AI_STUDIO_API_KEY') if not api_key: raise ValueError("OXYLABS_AI_STUDIO_API_KEY not set") # Initialize client (could also be AiScraper() without api_key param) scraper = AiScraper(api_key=api_key) # Generate schema schema = scraper.generate_schema(prompt="name, price, rating") # Scrape result = scraper.scrape( url="https://example.com/product", output_format="json", schema=schema ) if result.data: print(result.data) else: print(f"Failed: {result.message}") ``` -------------------------------- ### Initialize OxyStudioAIClient Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/oxystudioaiclient.md Initialize the client with your API key and an optional timeout. If the API key is not provided, it will attempt to read it from the OXYLABS_AI_STUDIO_API_KEY environment variable. ```python from oxylabs_ai_studio.client import OxyStudioAIClient client = OxyStudioAIClient( api_key="your-api-key", timeout=30.0 ) ``` -------------------------------- ### Localized Search Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aisearch.md Performs a search targeted to a specific geographic location. This is useful for finding location-specific information. ```python # Search results in specific location results = search.search( query="restaurants near me", limit=20, geo_location="France", return_content=False # Optional, speeds up ) for result in results.data: print(f"{result.title} - {result.url}") ``` -------------------------------- ### Initialize BrowserAgent Client Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/browseragent.md Instantiate the BrowserAgent client. You can provide an API key directly or rely on the OXYLABS_AI_STUDIO_API_KEY environment variable. ```python from oxylabs_ai_studio.apps.browser_agent import BrowserAgent agent = BrowserAgent(api_key="your-api-key") # or use OXYLABS_AI_STUDIO_API_KEY environment variable agent = BrowserAgent() ``` -------------------------------- ### Configuration: Custom Timeout Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/examples.md Set a custom timeout for API requests when initializing OxyStudioAIClient. This example sets a 60-second timeout. ```python from oxylabs_ai_studio.client import OxyStudioAIClient client = OxyStudioAIClient( api_key="your-api-key", timeout=60.0 # 60 second timeout ) ``` -------------------------------- ### Client Initialization with Environment Variable Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md This is the recommended pattern for initializing the `AiScraper` client. It automatically reads the API key from the `OXYLABS_AI_STUDIO_API_KEY` environment variable. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper # Reads from OXYLABS_AI_STUDIO_API_KEY environment variable scraper = AiScraper() ``` -------------------------------- ### Comprehensive Logging Pattern Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/utilities.md Demonstrates setting up SDK logging and obtaining a logger for your application code to log custom messages and exceptions. ```python from oxylabs_ai_studio.logger import configure_logging, get_logger import logging # Configure SDK logging configure_logging(level=logging.DEBUG) # Get logger for your code logger = get_logger("my_scraping_app") logger.info("Starting scraping operation") try: result = scraper.scrape(url="...") except Exception as e: logger.exception(f"Scraping failed: {e}") ``` -------------------------------- ### Get Specific Loggers Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md Retrieve specific loggers for different modules within the SDK. Useful for targeted logging configuration. ```python import logging # Get specific loggers client_logger = logging.getLogger("oxylabs_ai_studio.client") crawler_logger = logging.getLogger("oxylabs_ai_studio.apps.ai_crawler") scraper_logger = logging.getLogger("oxylabs_ai_studio.apps.ai_scraper") agent_logger = logging.getLogger("oxylabs_ai_studio.apps.browser_agent") search_logger = logging.getLogger("oxylabs_ai_studio.apps.ai_search") map_logger = logging.getLogger("oxylabs_ai_studio.apps.ai_map") ``` -------------------------------- ### Initialize AiSearch Client Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aisearch.md Instantiate the AiSearch client with your API key. Alternatively, the client can read the API key from the OXYLABS_AI_STUDIO_API_KEY environment variable. ```python from oxylabs_ai_studio.apps.ai_search import AiSearch search = AiSearch(api_key="your-api-key") # or use OXYLABS_AI_STUDIO_API_KEY environment variable search = AiSearch() ``` -------------------------------- ### Get Max Crawl Time Constant Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md Retrieve the maximum duration a job, such as a crawl, can run before timing out. This value is predefined in the SDK. ```python from oxylabs_ai_studio.apps.ai_crawler import CRAWLER_TIMEOUT_SECONDS print(f"Max crawl time: {CRAWLER_TIMEOUT_SECONDS}s") # 600 seconds (10 minutes) ``` -------------------------------- ### AiSearch Constructor Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aisearch.md Initializes the AiSearch client. You can provide an API key directly or rely on the environment variable `OXYLABS_AI_STUDIO_API_KEY`. ```APIDOC ## AiSearch Constructor ### `AiSearch(api_key: str | None = None)` Initialize the search client. ```python from oxylabs_ai_studio.apps.ai_search import AiSearch search = AiSearch(api_key="your-api-key") # or use OXYLABS_AI_STUDIO_API_KEY environment variable search = AiSearch() ``` | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `api_key` | `str \| None` | No | None | API key. If None, reads from `OXYLABS_AI_STUDIO_API_KEY` environment variable. | **Inherits**: `api_key`, `base_url`, `timeout` from [`OxyStudioAIClient`](./oxystudioaiclient.md). ``` -------------------------------- ### Initialize AiScraper Client Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aiscraper.md Instantiate the AiScraper client. You can provide an API key directly or rely on the OXYLABS_AI_STUDIO_API_KEY environment variable. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper scraper = AiScraper(api_key="your-api-key") # or use OXYLABS_AI_STUDIO_API_KEY environment variable scraper = AiScraper() ``` -------------------------------- ### BrowserAgent Constructor Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/browseragent.md Initializes the BrowserAgent client. It can be configured with an API key or by using the OXYLABS_AI_STUDIO_API_KEY environment variable. ```APIDOC ## BrowserAgent Constructor ### `BrowserAgent(api_key: str | None = None)` Initialize the browser agent client. ```python from oxylabs_ai_studio.apps.browser_agent import BrowserAgent agent = BrowserAgent(api_key="your-api-key") # or use OXYLABS_AI_STUDIO_API_KEY environment variable agent = BrowserAgent() ``` #### Parameters - **api_key** (`str | None`) - Optional - API key. If None, reads from `OXYLABS_AI_STUDIO_API_KEY` environment variable. ``` -------------------------------- ### Initialize and Use AiScraper Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Demonstrates the common pattern for initializing and using the AiScraper service, including schema generation and result checking. Reads API key from environment. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper # 1. Initialize client scraper = AiScraper(api_key="your-key") # 2. Generate schema (optional, for structured extraction) schema = scraper.generate_schema(prompt="title, price, rating") # 3. Call service method result = scraper.scrape( url="https://example.com", output_format="json", schema=schema ) # 4. Check result if result.data: print(result.data) # Extracted data else: print(f"Failed: {result.message}") # Error code ``` -------------------------------- ### Basic Page Scraping with AiScraper (Markdown) Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/examples.md Scrape a web page and get the content in Markdown format. Ensure you have your API key configured. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper scraper = AiScraper(api_key="your-api-key") result = scraper.scrape( url="https://example.com/product", output_format="markdown" ) if result.data: print(result.data) else: print(f"Error: {result.message}") ``` -------------------------------- ### Get Synchronous HTTP Client Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/oxystudioaiclient.md Obtain a configured synchronous httpx.Client instance. The caller is responsible for closing this client when it's no longer needed. ```python client = OxyStudioAIClient(api_key="key") http_client = client.get_client() # Use the client for requests response = http_client.get("/some/endpoint") ``` -------------------------------- ### Rate Limiting Retry Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/README.md Illustrates a basic retry mechanism for handling rate limiting errors (HTTP 429) by sleeping and retrying the request. ```python import time try: result = scraper.scrape(url="...") except Exception as e: if "429" in str(e): time.sleep(60) result = scraper.scrape(url="...") ``` -------------------------------- ### Configure Logging Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/oxystudioaiclient.md Set up logging for the oxylabs_ai_studio.client module to monitor API call events like rate limits and errors. ```python from oxylabs_ai_studio.logger import configure_logging import logging configure_logging(level=logging.DEBUG) ``` -------------------------------- ### AiCrawlerJob Attributes Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aicrawler.md Illustrates how to access attributes of the `AiCrawlerJob` object returned by crawl operations. This includes retrieving the run ID and the extracted data. ```python from oxylabs_ai_studio.apps.ai_crawler import AiCrawlerJob job: AiCrawlerJob = crawler.crawl(url="...", user_prompt="...") print(job.run_id) print(job.data) ``` -------------------------------- ### get_client() Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/oxystudioaiclient.md Creates and returns a synchronous HTTP client configured with API credentials and necessary headers. ```APIDOC ## get_client() ### Description Create and return a synchronous HTTP client configured with API credentials. ### Signature `get_client() -> httpx.Client` ### Returns `httpx.Client` with headers: - `x-api-key`: The API key - `Content-Type`: `application/json` - `User-Agent`: `python-sdk` (or custom value if set) ### Note Caller is responsible for closing the client when done. ``` -------------------------------- ### AI-Scraper Sync Interface Example Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/agentic_code_guide.md Employ the synchronous interface of the AiScraper to extract website content into JSON format. A JSON schema is required for structured output. ```python from oxylabs_ai_studio.apps.ai_scraper import AiScraper scraper = AiScraper(api_key="") url = "https://sandbox.oxylabs.io/products/3" result = scraper.scrape( url=url, output_format="json", schema={"type": "object", "properties": {"price": {"type": "string"}}, "required": []}, render_javascript=False, ) print(result) ``` -------------------------------- ### Run BrowserAgent for Product Search Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/browseragent.md Use the run method to search for a product on a website and extract specific details like name, platform, and price into a JSON format. A schema is generated first to define the extraction structure. ```python agent = BrowserAgent(api_key="key") # Search for a product schema = agent.generate_schema( prompt="game name, platform, review stars, price" ) result = agent.run( url="https://store.example.com", user_prompt="Find if 'Super Mario Odyssey' is available. If yes, get the price.", output_format="json", schema=schema, ) if result.data and result.data.content: print("Found:", result.data.content) ``` -------------------------------- ### Initialize AiCrawler Client Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/aicrawler.md Instantiate the AiCrawler client. You can provide an API key directly or rely on the OXYLABS_AI_STUDIO_API_KEY environment variable. ```python from oxylabs_ai_studio.apps.ai_crawler import AiCrawler crawler = AiCrawler(api_key="your-api-key") # or use OXYLABS_AI_STUDIO_API_KEY environment variable crawler = AiCrawler() ``` -------------------------------- ### Configure Custom File Handler for Logging Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/configuration.md Set up a custom file handler to direct SDK logs to a specified file. Ensure the 'configure_logging' function is imported. ```python from oxylabs_ai_studio.logger import configure_logging import logging # Custom file handler file_handler = logging.FileHandler("oxylabs_sdk.log") configure_logging(handler=file_handler) ``` -------------------------------- ### GET /browser-agent/run/data Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/endpoints.md Polls the status and retrieves results of a browser agent session using its run ID. Provides status, data content, and error information. ```APIDOC ## GET /browser-agent/run/data ### Description Poll browser agent status and results. ### Method GET ### Endpoint /browser-agent/run/data ### Parameters #### Query Parameters - **run_id** (string) - Yes - ### Response #### Success Response (200 OK) - **status** (enum) - processing|completed|failed - **data** (object) - - **error_code** (string) - null|"string" ### Response Example { "status": "processing|completed|failed", "data": { "type": "json|markdown|html|screenshot|csv|toon", "content": null|"string"|{...} }, "error_code": null|"string" } ``` -------------------------------- ### Log Level Constants Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/utilities.md Demonstrates the usage of predefined constants for logger name and default log level. ```python from oxylabs_ai_studio.logger import LOGGER_NAME, DEFAULT_LOG_LEVEL LOGGER_NAME = "oxylabs_ai_studio" DEFAULT_LOG_LEVEL = logging.INFO ``` -------------------------------- ### Extract Data with JSON Schema Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/browseragent.md Run the agent with a specified `output_format` of 'json' and a provided `schema`. This example shows how to access the structured JSON content from the `data.content` attribute. ```python result = agent.run( url="https://store.example.com", user_prompt="Find product 'Widget'", output_format="json", schema=schema ) if result.data: print(f"Type: {result.data.type}") # "json" if isinstance(result.data.content, dict): product = result.data.content print(f"Product: {product}") ``` -------------------------------- ### OxyStudioAIClient Constructor Source: https://github.com/oxylabs/oxylabs-ai-studio-py/blob/main/_autodocs/api-reference/oxystudioaiclient.md Initializes the OxyStudioAIClient with an API key and an optional timeout for HTTP requests. ```APIDOC ## OxyStudioAIClient Constructor ### Description Initializes the client with API credentials. ### Signature `OxyStudioAIClient(api_key: str | None = None, timeout: float = 30.0)` ### Parameters #### Parameters - **api_key** (`str | None`) - Optional - Default: `None` - API key for Oxylabs AI Studio API. If None, reads from `OXYLABS_AI_STUDIO_API_KEY` environment variable. Raises `ValueError` if not provided either way. - **timeout** (`float`) - Optional - Default: `30.0` - HTTP request timeout in seconds for all requests made by this client. ### Raises - `ValueError` if `api_key` is not provided and `OXYLABS_AI_STUDIO_API_KEY` environment variable is not set. ### Attributes - `api_key: str` — The resolved API key - `base_url: str` — API base URL (from settings, default: `https://api-aistudio.oxylabs.io`) - `timeout: float` — Request timeout in seconds ```