### Install Evomi Go Client Source: https://docs.evomi.com/integrations/go-client Use 'go get' to install the Evomi Go Client library. ```bash go get github.com/evomi/evomi-go-client ``` -------------------------------- ### Evomi Rust Client - Quick Start Source: https://docs.evomi.com/integrations/rust-client A basic example demonstrating how to initialize the client and perform a scrape operation. ```APIDOC ## Quick Start ```rust use evomi_client::Client; #[tokio::main] async fn main() -> Result<(), Box> { // Initialize with API key (or set EVOMI_API_KEY env var) let client = Client::new("your-api-key", None, None)?; // Scrape a URL let result = client.scrape("https://example.com", None).await?; println!("{:?}", result); Ok(()) } ``` ``` -------------------------------- ### JS Client Quick Start Source: https://docs.evomi.com/integrations/js-client A quick start guide to initialize the Evomi JavaScript Client and perform a basic scrape operation. ```APIDOC ## Quick Start ```javascript import { EvomiClient } from 'evomi-client'; const client = new EvomiClient({ apiKey: 'your-api-key' }); // Scrape a webpage const result = await client.scrape('https://example.com'); console.log(result.content); ``` ``` -------------------------------- ### Quick Start: Scrape a URL with Evomi Go Client Source: https://docs.evomi.com/integrations/go-client A basic example demonstrating how to initialize the client and scrape a single URL. Ensure you have your API key configured. ```go package main import ( "context" "fmt" "log" "github.com/evomi/evomi-go-client/evomi" ) func main() { client := evomi.NewClient("your-api-key", "", "") result, err := client.Scrape(context.Background(), "https://example.com", evomi.ScrapeOptions{}) if err != nil { log.Fatal(err) } fmt.Println(result["content"]) } ``` -------------------------------- ### Start Small Request Example Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/tools/url-discovery/usage-examples Test API usage with a small `max_urls` value to understand potential costs before performing larger jobs. ```json { "domain": "example.com", "sources": ["sitemap"], "max_urls": 10, "check_if_live": false } ``` -------------------------------- ### Start Small Crawl Configuration Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/tools/domain-crawling/usage-examples Example configuration for testing with a small number of URLs to manage costs and verify functionality. ```json { "domain": "example.com", "max_urls": 10 } ``` -------------------------------- ### Install Evomi Client Source: https://docs.evomi.com/integrations/python-client Install the required package via pip. ```bash pip install evomi-client ``` -------------------------------- ### Install Evomi Rust Client Source: https://docs.evomi.com/integrations/rust-client Add the evomi-client and tokio dependencies to your Cargo.toml file to get started. ```toml [dependencies] evomi-client = "0.1.2" tokio = { version = "1", features = ["full"] } ``` -------------------------------- ### Quick Start: Crawl Domain and Scrape All URLs Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/tools/domain-crawling This is a basic example to crawl a domain and scrape all discovered URLs. It uses default scraper settings. Replace YOUR_API_KEY with your actual API key. ```bash curl -X POST "https://scrape.evomi.com/api/v1/scraper/crawl" \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "domain": "example.com", "max_urls": 10 }' ``` -------------------------------- ### JavaScript Example with Local DNS Resolution Source: https://docs.evomi.com/proxy-instructions/residential-proxies/expert-settings/localdns This Node.js example demonstrates how to use `node-fetch` to make a request through a proxy with local DNS resolution enabled. You need to install the `node-fetch` package. ```javascript const fetch = require('node-fetch'); const proxyUrl = 'http://testuser:testpassword_localdns-1@rp.evomi.com:1000'; const url = 'https://ip.evomi.com/s'; fetch(url, { proxy: proxyUrl }) .then(res => res.text()) .then(body => console.log(body)); ``` -------------------------------- ### Install Scrapy Source: https://docs.evomi.com/proxy-instructions/integration-guides/scrapy Install the Scrapy framework via pip. ```bash pip install scrapy ``` -------------------------------- ### Install Evomi JS Client Source: https://docs.evomi.com/integrations/js-client Install the Evomi JavaScript Client package using npm. ```bash npm install evomi-client ``` -------------------------------- ### Wait For Action Example Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/parameters/javascript-automation Example of waiting for search results to appear after initiating a search. ```json { "js_instructions": [ {"click": "#search-button"}, {"wait_for": ".search-results"} // Wait until results appear ] } ``` -------------------------------- ### JS Client Installation Source: https://docs.evomi.com/integrations/js-client Install the Evomi JavaScript Client package from npm. ```APIDOC ## Installation Install the package from npm: ``` npm install evomi-client ``` ``` -------------------------------- ### Install Node.js Dependencies Source: https://docs.evomi.com/proxy-instructions/mobile-proxies/how-to-connect Install node-fetch and https-proxy-agent to enable proxy support in Node.js. ```bash npm install node-fetch http-proxy-agent ``` -------------------------------- ### Install Selenium Source: https://docs.evomi.com/proxy-instructions/integration-guides/selenium Command to install the Selenium package via pip. ```bash pip install selenium ``` -------------------------------- ### Install Beautiful Soup and Requests Source: https://docs.evomi.com/proxy-instructions/integration-guides/beautiful-soup Install the necessary Python libraries for web scraping and making HTTP requests. For SOCKS5 proxy support, install the `requests` library with the `socks` extra. ```bash pip install beautifulsoup4 requests ``` ```bash pip install "requests[socks]" ``` -------------------------------- ### Install Python Requests Source: https://docs.evomi.com/proxy-instructions/mobile-proxies/how-to-connect Install the required library for handling HTTP requests in Python. ```bash pip install requests ``` -------------------------------- ### Install Python Requests Source: https://docs.evomi.com/proxy-instructions/integration-guides/python-requests Commands to install the requests library and optional SOCKS5 support. ```bash pip install requests ``` ```bash pip install "requests[socks]" ``` -------------------------------- ### Install Evomi MCP Server Source: https://docs.evomi.com/integrations/mcp Install the Evomi MCP Server package from PyPI using pip. ```bash pip install evomi-mcp ``` -------------------------------- ### Custom Scraper Settings Examples Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/tools/domain-crawling/usage-examples Examples for advanced configurations including JS rendering, Markdown output, and AI enhancement. ```json { "domain": "example.com", "max_urls": 10, "scraper_config": { "mode": "request", "js_rendering": true, "wait_for": 2000 } } ``` ```json { "domain": "blog.example.com", "max_urls": 20, "scraper_config": { "content": "markdown" } } ``` ```json { "success": true, "domain": "blog.example.com", "discovered_count": 20, "scraper_tasks_submitted": 20, "results": [ { "url": "https://blog.example.com/post-1", "source": "crawl", "scrape_task_id": "md-001", "scrape_check_url": "/api/v1/scraper/tasks/md-001" } ], "credits_used": 60.0, "credits_remaining": 940.0 } ``` ```json { "domain": "news.example.com", "max_urls": 10, "scraper_config": { "ai_enhance": true, "ai_source": "markdown", "ai_prompt": "Extract: headline, author, publish_date, summary. Return as JSON." } } ``` -------------------------------- ### Install Axios for Node.js Source: https://docs.evomi.com/proxy-instructions/integration-guides/nodejs Install the Axios library using npm. This is a prerequisite for using Axios in your Node.js project. ```bash npm install axios ``` -------------------------------- ### Capture Product API Data Example Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/parameters/network-capture Example configuration to capture product data from an e-commerce site. ```APIDOC ## Capture Product API Data Extract product data from an e-commerce site that loads products via API: ```json { "url": "https://shop.example.com/category/electronics", "mode": "browser", "wait_until": "networkidle", "networkCapture": [ { "filterType": "composite", "conditions": [ { "filterType": "url", "value": "/products", "matchType": "contains" }, { "filterType": "resourceType", "resourceType": "fetch" } ], "httpResponseBody": true } ] } ``` ``` -------------------------------- ### Basic Crawling Examples Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/tools/domain-crawling/usage-examples Examples for discovering and scraping pages from a domain, including depth control. ```json { "domain": "example.com", "max_urls": 10 } ``` ```json { "success": true, "domain": "example.com", "discovered_count": 10, "scraper_tasks_submitted": 10, "results": [ { "url": "https://example.com/", "source": "crawl", "scrape_task_id": "abc-123", "scrape_check_url": "/api/v1/scraper/tasks/abc-123" }, { "url": "https://example.com/about", "source": "crawl", "scrape_task_id": "def-456", "scrape_check_url": "/api/v1/scraper/tasks/def-456" } ], "credits_used": 20.0, "credits_remaining": 980.0 } ``` ```json { "domain": "example.com", "depth": 2, "max_urls": 50 } ``` ```json { "success": true, "domain": "example.com", "discovered_count": 50, "scraper_tasks_submitted": 50, "results": [ { "url": "https://example.com/", "source": "crawl" }, { "url": "https://example.com/page1", "source": "crawl" } ], "credits_used": 100.0, "credits_remaining": 900.0 } ``` -------------------------------- ### Install Playwright Dependencies Source: https://docs.evomi.com/proxy-instructions/integration-guides/playwright Commands to install the Playwright library and browser binaries. ```bash npm install playwright npx playwright install ``` ```bash pip install playwright playwright install ``` -------------------------------- ### Wait Action Example Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/parameters/javascript-automation Example of using wait after clicking a 'load more' button to allow new content to load before further interaction. ```json { "js_instructions": [ {"click": "#load-more"}, {"wait": 3000}, // Wait for content to load {"click": "#load-more"} ] } ``` -------------------------------- ### Sticky Session Example Source: https://docs.evomi.com/frequently-asked-questions Use the `_session` parameter to create a sticky session that maintains the same IP for a specified duration. This example keeps the same IP for 20 minutes. ```HTTP http://rp.evomi.com:1000:username:password_session-sgn34f3e ``` -------------------------------- ### Delivery Mode Examples Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/cloud-storage Query parameters demonstrating different content delivery formats. ```text ?url=https://example.com&delivery=raw&use_default_storage=true&content=screenshot ``` ```text ?url=https://example.com&delivery=raw&use_default_storage=true&content=markdown ``` ```text ?url=https://example.com&delivery=raw&use_default_storage=true ``` -------------------------------- ### Capture GraphQL Responses Example Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/parameters/network-capture Example configuration to capture GraphQL API responses. ```APIDOC ## Capture GraphQL Responses Capture GraphQL API responses: ```json { "url": "https://app.example.com/dashboard", "mode": "browser", "networkCapture": [ { "filterType": "composite", "conditions": [ { "filterType": "url", "value": "/graphql", "matchType": "endsWith" }, { "filterType": "method", "methods": ["POST"] } ], "httpResponseBody": true } ] } ``` ``` -------------------------------- ### Install Puppeteer Source: https://docs.evomi.com/proxy-instructions/integration-guides/puppeteer Install the Puppeteer library using npm. This command also automatically downloads a compatible Chromium binary. ```bash npm install puppeteer ``` -------------------------------- ### Configuration Creation Response Source: https://docs.evomi.com/integrations/agent-requests Example response confirming the creation of a new scraper configuration. ```json { "success": true, "response": "I've created a config named 'Amazon Products' with ID cfg_xyz789. It extracts product title, price, rating, and review count from Amazon product pages.", "actions_taken": [ { "tool": "generate_config", "params": { "name": "Amazon Products", "prompt": "Scrape Amazon product pages - extract title, price, rating, and review count" }, "result": { "success": true, "config_id": "cfg_xyz789" } } ], "credits_used": 60, "credits_remaining": 940 } ``` -------------------------------- ### Install node-fetch and socks-proxy-agent for SOCKS5 Source: https://docs.evomi.com/proxy-instructions/integration-guides/nodejs Install `node-fetch` and `socks-proxy-agent` to enable SOCKS5 proxy support when using `node-fetch`. This combination allows for SOCKS5 proxy connections. ```bash npm install node-fetch socks-proxy-agent ``` -------------------------------- ### Evomi API Specific Request Example Source: https://docs.evomi.com/integrations/agent-requests This example demonstrates a specific and clear request for scraping product information, including desired fields. ```text # Good — clear intent and target "Scrape https://example.com/products and extract product name, price, and availability" # Less clear "Get products from example.com" ``` -------------------------------- ### Example Password with Quality Mode Source: https://docs.evomi.com/proxy-instructions/integration-guides/windows-proxifier This example demonstrates the password format when using the 'quality' mode for Evomi residential proxies. Replace 'mysecretpassword' and 'abc123' with your actual credentials and session identifier. ```plaintext mysecretpassword_session-abc123_mode-quality ``` -------------------------------- ### Claude Desktop Configuration (Installed Package) Source: https://docs.evomi.com/integrations/mcp Add this configuration to your Claude Desktop settings to enable the Evomi MCP server when installed from PyPI. Ensure your API key is correctly set. ```json { "mcpServers": { "evomi": { "command": "evomi-mcp", "env": { "EVOMI_API_KEY": "your-api-key-here" } } } } ``` -------------------------------- ### Install node-fetch and https-proxy-agent Source: https://docs.evomi.com/proxy-instructions/integration-guides/nodejs Install `node-fetch` for making HTTP requests and `https-proxy-agent` for HTTP proxy support in Node.js. This is needed for `node-fetch` HTTP proxy configurations. ```bash npm install node-fetch https-proxy-agent ``` -------------------------------- ### Click Action Example Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/parameters/javascript-automation Example of using the click action within a scraper request to interact with a 'show more' button. ```bash curl -X POST "https://scrape.evomi.com/api/v1/scraper/realtime" \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com", "mode": "browser", "js_instructions": [ {"click": ".show-more-button"} ] }' ``` -------------------------------- ### Error Response Example (Invalid Product) Source: https://docs.evomi.com/public-api/endpoints/rotate_session Example error response for an invalid product type specified during session rotation. ```json { "error": "Invalid product", "success": false } ``` -------------------------------- ### Claude Desktop Configuration (Installed from Source) Source: https://docs.evomi.com/integrations/mcp Add this configuration to your Claude Desktop settings if you have installed the Evomi MCP server from source. This specifies how to run the server script. ```json { "mcpServers": { "evomi": { "command": "python", "args": ["-m", "evomi_mcp.server"], "env": { "EVOMI_API_KEY": "your-api-key-here" } } } } ``` -------------------------------- ### Use JavaScript for Mobile Proxy Sessions Source: https://docs.evomi.com/proxy-instructions/mobile-proxies/proxy-sessions Provides examples of using `node-fetch` in JavaScript to connect to mobile proxies with `session` and `hardsession` parameters. The `lifetime` parameter is demonstrated with the `session` example. ```javascript const fetch = require('node-fetch'); const proxyUrl = 'http://testuser:testpassword_country-US_session-sgn34f3e_lifetime-10@mp.evomi.com:3000'; const url = 'https://ip.evomi.com/s'; fetch(url, { proxy: proxyUrl }) .then(res => res.text()) .then(body => console.log(body)); const proxyUrl2 = 'http://testuser:testpassword_country-US_hardsession-JS9nsq2n@mp.evomi.com:3000'; fetch(url, { proxy: proxyUrl2 }) .then(res => res.text()) .then(body => console.log(body)); ``` -------------------------------- ### CURL Example with Local DNS Resolution Source: https://docs.evomi.com/proxy-instructions/residential-proxies/expert-settings/localdns Use the `curl` command with the `_localdns-1` parameter appended to the proxy password to test local DNS resolution. This example fetches information from `https://ip.evomi.com/s`. ```bash curl -x http://testuser:testpassword_localdns-1@rp.evomi.com:1000 https://ip.evomi.com/s ``` ```bash curl -x http://testuser:testpassword_country-DE_localdns-1@rp.evomi.com:1000 https://ip.evomi.com/s ``` -------------------------------- ### Session Management Examples Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/proxy-types Examples demonstrating how to use `proxy_session_id` to maintain the same IP address across multiple requests for various use cases like login flows, shopping carts, or paginated scraping. ```APIDOC ## Session Management Examples ### Description Session IDs allow you to maintain the same IP address across multiple requests. This is crucial for workflows involving user logins, multi-step processes like shopping carts, or scraping paginated results, helping to avoid detection and ensure consistency. ### Use Cases #### Login and Browse To maintain the same IP address when navigating from a login page to a subsequent page: **Step 1: Login Page** ```curl curl -X POST "https://scrape.evomi.com/api/v1/scraper/realtime" \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com/login", "proxy_session_id": "user001", "proxy_country": "US" }' ``` **Step 2: Dashboard (same IP)** ```curl curl -X POST "https://scrape.evomi.com/api/v1/scraper/realtime" \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com/dashboard", "proxy_session_id": "user001", "proxy_country": "US" }' ``` #### Shopping Cart Flow Use the same session ID for all steps in a shopping cart process: **Request Body Example:** ```json { "proxy_session_id": "cart789", "proxy_country": "US" } ``` This applies to actions such as: 1. Browsing products 2. Adding items to the cart 3. Viewing the cart 4. Proceeding to checkout #### Paginated Scraping Maintain the same IP when scraping multiple pages of results to avoid detection: **Request Body Example:** ```json { "proxy_session_id": "scrape1", "proxy_country": "GB" } ``` ### Session ID Requirements - **Length:** 6-8 characters - **Format:** Alphanumeric (a-z, A-Z, 0-9) - **Case-sensitive:** `session1` is different from `SESSION1` - **Persistence:** Using the same ID ensures the same IP address across requests within the session. ``` -------------------------------- ### Initialize Evomi Clients Source: https://docs.evomi.com/integrations/python-client Demonstrates how to initialize both asynchronous and synchronous clients. ```python import asyncio from evomi_client import EvomiClient async def main(): # Initialize with API key (or set EVOMI_API_KEY env var) client = EvomiClient(api_key="your-api-key") # Scrape a URL result = await client.scrape("https://example.com") print(result["content"]) asyncio.run(main()) ``` ```python from evomi_client import EvomiClientSync client = EvomiClientSync(api_key="your-api-key") result = client.scrape("https://example.com") print(result["content"]) ``` -------------------------------- ### Submit Async Request Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/result-polling Example of initiating an asynchronous scraping job. ```bash curl -X POST "https://scrape.evomi.com/api/v1/scraper/realtime" \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com", "async": true }' ``` -------------------------------- ### Quick Start: Initialize and Scrape Source: https://docs.evomi.com/integrations/js-client Initialize the EvomiClient with an API key and perform a basic web scrape. ```javascript import { EvomiClient } from 'evomi-client'; const client = new EvomiClient({ apiKey: 'your-api-key' }); // Scrape a webpage const result = await client.scrape('https://example.com'); console.log(result.content); ``` -------------------------------- ### Fill and Click Login Example Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/parameters/javascript-automation Demonstrates filling username and password fields followed by a click on the login button. ```json { "js_instructions": [ {"fill": ["#username", "john@example.com"]}, {"fill": ["#password", "secret123"]}, {"click": "#login-button"} ] } ``` -------------------------------- ### Quick Start with Evomi Rust Client Source: https://docs.evomi.com/integrations/rust-client Initialize the client with an API key and scrape a URL. Ensure the EVOMI_API_KEY environment variable is set or pass the key directly. ```rust use evomi_client::Client; #[tokio::main] async fn main() -> Result<(), Box> { // Initialize with API key (or set EVOMI_API_KEY env var) let client = Client::new("your-api-key", None, None)?; // Scrape a URL let result = client.scrape("https://example.com", None).await?; println!("{:?}", result); Ok(()) } ``` -------------------------------- ### Set Appropriate Crawl Depth Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/tools/domain-crawling/usage-examples Example configuration to set the crawl depth, starting with depth 1 and increasing as needed to discover more pages. ```json { "domain": "example.com", "depth": 2, "max_urls": 100 } ``` -------------------------------- ### Quick Start Proxy Connections Source: https://docs.evomi.com/proxy-instructions/integration-guides/curl Basic commands to test HTTP, HTTPS, and SOCKS5 proxy connections using Evomi credentials. ```bash curl -x http://your_username:your_password_session-anychars_mode-speed@rp.evomi.com:1000 https://ip.evomi.com/s ``` ```bash curl -x https://your_username:your_password_session-anychars_mode-speed@rp.evomi.com:1001 https://ip.evomi.com/s ``` ```bash curl -x socks5h://your_username:your_password_session-anychars_mode-speed@rp.evomi.com:1002 https://ip.evomi.com/s ``` -------------------------------- ### Use Latency Filter with Node.js Fetch Source: https://docs.evomi.com/proxy-instructions/residential-proxies/expert-settings/latency Example of implementing the latency filter in a Node.js application using the `node-fetch` library. Make sure `node-fetch` is installed. ```javascript const fetch = require('node-fetch'); const proxyUrl = 'http://testuser:testpassword_latency-550_session-mysession@rp.evomi.com:1000'; const url = 'https://ip.evomi.com/s'; fetch(url, { proxy: proxyUrl }) .then(res => res.text()) .then(body => console.log(body)); ``` -------------------------------- ### Initialize Evomi Client with API Key Source: https://docs.evomi.com/integrations/go-client Instantiate the Evomi client by passing the API key directly during initialization. ```go client := evomi.NewClient("your-api-key", "", "") ``` -------------------------------- ### Python Example with Local DNS Resolution Source: https://docs.evomi.com/proxy-instructions/residential-proxies/expert-settings/localdns This Python script uses the `requests` library to make a request through a proxy configured with local DNS resolution enabled. Ensure you have the `requests` library installed. ```python import requests proxies = { "http": "http://testuser:testpassword_localdns-1@rp.evomi.com:1000", } response = requests.get("https://ip.evomi.com/s", proxies=proxies) print(response.text) ``` -------------------------------- ### Good Prompt Example for Structured Output Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/parameters/ai-enhancement Provide specific instructions for data extraction, including field names and desired format (e.g., JSON with keys: name, price, available). ```json {"ai_prompt": "Extract product name, price in USD, and availability status as JSON with keys: name, price, available"} ``` -------------------------------- ### Evomi Rust Client - Installation Source: https://docs.evomi.com/integrations/rust-client Instructions on how to add the Evomi Rust Client and Tokio to your project's Cargo.toml file. ```APIDOC ## Installation Add to your `Cargo.toml`: ```toml [dependencies] evomi-client = "0.1.2" tokio = { version = "1", features = ["full"] } ``` ``` -------------------------------- ### Success Response Example Source: https://docs.evomi.com/public-api/endpoints/rotate_session Example of a successful response when a proxy session is rotated. ```json { "success": true, "message": "Session reset successfully" } ``` -------------------------------- ### Storage Path Template Examples Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/cloud-storage JSON configurations for defining dynamic file paths using template variables. ```json { "path_prefix": "scrapes/{{domain}}/{{date}}/{{task_id}}.{{extension}}" } ``` ```json { "path_prefix": "data/{{year}}/{{month}}/{{day}}/{{domain}}_{{task_id}}.{{extension}}" } ``` ```json { "path_prefix": "results/{{status}}/{{domain}}/{{timestamp}}.{{extension}}" } ``` ```json { "path_prefix": "{{domain}}/{{task_id}}.{{extension}}" } ``` -------------------------------- ### Initialize and Scrape with Python Client Source: https://docs.evomi.com/integrations Requires the evomi-client package. Supports asynchronous scraping operations. ```python from evomi_client import EvomiClient client = EvomiClient(api_key="your-api-key") result = await client.scrape("https://example.com") ``` -------------------------------- ### WebSocket Configuration Examples Source: https://docs.evomi.com/scraping-products-instructions/scraping-browser/websocket-parameters Common configurations for browser sessions, including OS targeting, proxy routing, and feature toggles. ```text # Windows fingerprint, US proxy routing, adblock on wss://browser.evomi.com?key=YOUR_API_KEY&os=windows&proxy_country=US&adblock=true # Disable Turnstile solving for debugging wss://browser.evomi.com?key=YOUR_API_KEY&solve_cloudflare=false # Minimal: just your key wss://browser.evomi.com?key=YOUR_API_KEY ``` -------------------------------- ### Configure and Test Evomi Proxy Source: https://docs.evomi.com/proxy-instructions/integration-guides/beautiful-soup Set up proxy credentials and URL, then use `requests` to fetch your public IP through the proxy. This verifies the proxy configuration and connection. Ensure to replace placeholders with your actual Evomi credentials. ```python import requests from bs4 import BeautifulSoup proxy_host = "rp.evomi.com" proxy_port = "1000" proxy_user = "your_username" proxy_pass = "your_password_session-anychars_mode-speed" proxy_url = f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}" proxies = { "http": proxy_url, "https": proxy_url, } response = requests.get("https://ip.evomi.com/s", proxies=proxies, timeout=15) response.raise_for_status() soup = BeautifulSoup(response.content, "html.parser") print("Proxy IP:", soup.get_text().strip()) ``` -------------------------------- ### URL Pattern Filtering Examples Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/tools/domain-crawling/usage-examples Examples for restricting crawling to specific URL patterns. ```json { "domain": "example.com", "url_pattern": "/blog/.*", "max_urls": 20 } ``` ```json { "success": true, "domain": "example.com", "discovered_count": 18, "scraper_tasks_submitted": 18, "results": [ { "url": "https://example.com/blog/post-1", "source": "crawl", "scrape_task_id": "blog-001", "scrape_check_url": "/api/v1/scraper/tasks/blog-001" } ], "credits_used": 36.0, "credits_remaining": 964.0 } ``` ```json { "domain": "shop.example.com", "url_pattern": "/products?/[a-z0-9-]+", "max_urls": 100 } ``` ```json { "success": true, "domain": "shop.example.com", "discovered_count": 87, "scraper_tasks_submitted": 87, "results": [ { "url": "https://shop.example.com/products/laptop", "source": "crawl", "scrape_task_id": "prod-001", "scrape_check_url": "/api/v1/scraper/tasks/prod-001" } ], "credits_used": 174.0, "credits_remaining": 826.0 } ``` -------------------------------- ### Python Best Practices Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/usage-examples Recommended patterns for environment configuration, error handling, and monitoring. ```python import os API_KEY = os.environ.get('EVOMI_API_KEY') if not API_KEY: raise ValueError("EVOMI_API_KEY environment variable not set") ``` ```python try: response = scrape(url) response.raise_for_status() except requests.exceptions.HTTPError as e: if e.response.status_code == 402: alert("Insufficient credits!") elif e.response.status_code == 429: time.sleep(60) # Rate limited except requests.exceptions.Timeout: log("Request timed out") ``` ```python credits_remaining = float(response.headers.get('X-Credits-Remaining', 0)) if credits_remaining < 100: send_alert(f"Low credits: {credits_remaining}") ``` ```python # For static sites with datacenter proxy scrape(url, mode='request', proxy_type='datacenter') # 1 credit # For static sites with residential proxy (better success rate) scrape(url, mode='request', proxy_type='residential') # 2 credits # For JS-heavy sites (requires residential proxy) scrape(url, mode='browser', proxy_type='residential') # 5 credits # Let API decide (requires residential proxy) scrape(url, mode='auto', proxy_type='residential') # 2-6 credits depending on needs ``` -------------------------------- ### Granular Targeting Request Example Source: https://docs.evomi.com/public-api/endpoints/settings Example of combining multiple parameters for specific proxy targeting. ```json { "country": "DE", "city": "berlin", } ``` -------------------------------- ### Debug Failed Requests Example Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/parameters/network-capture Example configuration to capture failed API calls for debugging purposes. ```APIDOC ## Debug Failed Requests Capture failed API calls for debugging: ```json { "url": "https://example.com/app", "mode": "browser", "networkCapture": [ { "filterType": "statusCode", "statusCodes": [400, 401, 403, 404, 500, 502, 503], "httpResponseBody": true } ] } ``` ``` -------------------------------- ### Error Response Example (Missing Session ID) Source: https://docs.evomi.com/public-api/endpoints/rotate_session Example error response when the session ID is not provided. ```json { "error": "Session ID not provided", "success": false } ``` -------------------------------- ### Initialize Evomi Client with API Key Source: https://docs.evomi.com/integrations/rust-client Initialize the Evomi client by passing the API key directly during instantiation. ```rust let client = Client::new("your-api-key", None, None)?; ``` -------------------------------- ### Error Response Example (Rotation Failure) Source: https://docs.evomi.com/public-api/endpoints/rotate_session Example error response indicating a failure during the session rotation process. ```json { "error": "Failed to reset session", "success": false } ``` -------------------------------- ### Install proxy-chain for SOCKS5 Source: https://docs.evomi.com/proxy-instructions/integration-guides/puppeteer Install the proxy-chain package, which is required for using SOCKS5 proxies with Puppeteer by creating a local forwarder. ```bash npm install proxy-chain ``` -------------------------------- ### Enterprise Configuration Strings Source: https://docs.evomi.com/proxy-instructions/residential-proxies/expert-settings/udp Examples of combining the UDP parameter with other enterprise-grade proxy configuration flags. ```text _udp-1_asn-AS15169_fraudscore-0 _country-DE_udp-1_isp-telekom_latency-50 _zip-10115_udp-1_localdns-1_http3-1 ``` -------------------------------- ### Python Example for Monitoring Success Rates Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/proxy-types This Python snippet demonstrates how to track the success rates of datacenter and residential proxies. It includes a fallback mechanism to try residential proxies if datacenter proxies fail. ```python # Python example results = { 'datacenter_success': 0, 'residential_success': 0 } # Try datacenter first response = scrape(url, proxy_type='datacenter') if response.status_code == 200: results['datacenter_success'] += 1 else: # Fall back to residential response = scrape(url, proxy_type='residential') results['residential_success'] += 1 ``` -------------------------------- ### CURL Zipcode Targeting Examples Source: https://docs.evomi.com/proxy-instructions/residential-proxies/expert-settings/zipcode Examples showing how to use zipcode targeting with CURL, including combining it with country parameters. ```bash curl -x http://testuser:testpassword_zip-10001@rp.evomi.com:1000 https://ip.evomi.com/s curl -x http://testuser:testpassword_country-US_zip-90210@rp.evomi.com:1000 https://ip.evomi.com/s ``` -------------------------------- ### Install socks-proxy-agent for Axios SOCKS5 Source: https://docs.evomi.com/proxy-instructions/integration-guides/nodejs Install the `socks-proxy-agent` package to enable SOCKS5 proxy support with Axios. This is required as Axios does not natively support SOCKS5. ```bash npm install socks-proxy-agent ``` -------------------------------- ### Build Proxy String with Manual Configuration Source: https://docs.evomi.com/integrations/go-client Manually configure proxy settings using `ProxyConfig` and build the proxy string. This method allows for detailed control over proxy parameters. ```go config := &evomi.ProxyConfig{ ProxyType: evomi.ProxyTypeResidential, Protocol: evomi.ProxyProtocolHTTP, Country: "US", City: "New York", Username: "your-username", Password: "your-password", } proxyString := config.BuildProxyString() ``` -------------------------------- ### Initialize EvomiClient with API Key Source: https://docs.evomi.com/integrations/js-client Initialize the EvomiClient by passing the API key directly during instantiation. ```javascript const client = new EvomiClient({ apiKey: 'your-api-key' }); ``` -------------------------------- ### Example Proxy Connection with Combined Filters Source: https://docs.evomi.com/proxy-instructions/residential-proxies/expert-settings Demonstrates how to combine multiple expert filters (fraudscore, device, latency) by appending them to the proxy password in a cURL command. ```bash # Combined filters: fraudscore + device + latency curl -x http://testuser:testpassword_fraudscore-5_device-windows_latency-100@rp.evomi.com:1000 https://ip.evomi.com/s ``` -------------------------------- ### Configure Manual Proxy Settings Source: https://docs.evomi.com/integrations/js-client Use this to set up proxy configurations with specific types, protocols, and geographical targeting. Ensure you have the 'evomi-client' package installed. ```typescript import { ProxyConfig, ProxyType, ProxyProtocol } from 'evomi-client'; const config = new ProxyConfig({ proxyType: ProxyType.RESIDENTIAL, protocol: ProxyProtocol.HTTP, country: 'US', city: 'New York', username: 'your-username', password: 'your-password' }); const proxyString = config.buildProxyString(); ``` -------------------------------- ### GET /account/schedule Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/schedules Retrieve a list of all configured schedules. ```APIDOC ## GET /account/schedule ### Description Returns a paginated list of all schedules associated with the account. ### Method GET ### Endpoint https://scrape.evomi.com/api/v1/account/schedule ### Parameters #### Query Parameters - **page** (integer) - Optional - Page number for pagination - **per_page** (integer) - Optional - Results per page (max 100) - **active_only** (boolean) - Optional - Filter to show only active schedules ### Response #### Success Response (200) - **schedules** (array) - List of schedule objects - **pagination** (object) - Pagination metadata #### Response Example { "success": true, "schedules": [ { "id": "job_xyz789", "name": "Hourly Price Monitor", "config_id": "cfg_abc123", "interval_minutes": 60, "is_active": true, "last_run": "2026-02-20T13:00:00", "last_status": "success", "next_run": "2026-02-20T14:00:00" } ], "pagination": { "page": 1, "per_page": 20, "total": 1 } } ``` -------------------------------- ### GET /schedule-runs Source: https://docs.evomi.com/integrations/go-client Retrieves a list of runs for a specific schedule. ```APIDOC ## GET /schedule-runs ### Description Retrieves a list of runs for a specific schedule. ### Parameters #### Path Parameters - **scheduleID** (string) - Required - The ID of the schedule to list runs for. #### Request Body - **Page** (int) - Optional - Page number. - **PerPage** (int) - Optional - Number of items per page. ``` -------------------------------- ### Initialize Evomi Client with Custom Base URL Source: https://docs.evomi.com/integrations/go-client Configure the client to use a custom base URL, typically for testing purposes. ```go client := evomi.NewClient("your-api-key", "https://custom.evomi.com", "") ``` -------------------------------- ### JSON Response Example Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/quickstart The structure of the JSON response when delivery=json is specified. ```json { "success": true, "url": "https://example.com", "domain": "example.com", "title": "Example Domain", "status_code": 200, "credits_used": 1.0, "credits_remaining": 99.0, "mode_used": "auto (request)" } ``` -------------------------------- ### Manage Storage Configurations Source: https://docs.evomi.com/integrations/go-client Methods for listing, creating, updating, and deleting cloud storage configurations for scrape results. ```go configs, err := client.ListStorageConfigs(ctx) ``` ```go // S3-compatible storage storage, err := client.CreateStorageConfig(ctx, "My S3", "s3_compatible", map[string]any{ "bucket": "my-bucket", "region": "us-east-1", "access_key": "...", "secret_key": "...", }, true) // Google Cloud Storage storage, err := client.CreateStorageConfig(ctx, "My GCS", "gcs", map[string]any{ "bucket": "my-bucket", "credentials_json": "...", }, false) // Azure Blob Storage storage, err := client.CreateStorageConfig(ctx, "My Azure", "azure_blob", map[string]any{ "container": "my-container", "connection_string": "...", }, false) ``` ```go setAsDefault := true storage, err := client.UpdateStorageConfig(ctx, "stor_abc123", evomi.UpdateStorageOptions{ Name: "Renamed Storage", SetAsDefault: &setAsDefault, }) ``` ```go result, err := client.DeleteStorageConfig(ctx, "stor_abc123") ``` -------------------------------- ### Continent Targeting Structure Source: https://docs.evomi.com/public-api/endpoints/settings Example of the continent-based country grouping structure. ```json "europe": { "name": "Europe", "countries": { "Germany": "DE", "France": "FR" } } ``` -------------------------------- ### Enable Ad Blocking with Node.js Source: https://docs.evomi.com/proxy-instructions/residential-proxies/expert-settings/adblocking Set up Node.js `node-fetch` to utilize proxies with ad blocking. Append `_adblock-1` to the proxy password for activation. ```javascript const fetch = require('node-fetch'); const proxyUrl = 'http://testuser:testpassword_country-US_session-sgn34f3e_lifetime-10@rp.evomi.com:1000'; const url = 'https://ip.evomi.com/s'; fetch(url, { proxy: proxyUrl }) .then(res => res.text()) .then(body => console.log(body)); const proxyUrl2 = 'http://testuser:testpassword_country-US_hardsession-JS9nsq2n@rp.evomi.com:1000'; fetch(url, { proxy: proxyUrl2 }) .then(res => res.text()) .then(body => console.log(body)); ``` -------------------------------- ### GET /api/v1/scraper/search/tasks/{task_id} Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/tools/search Check the status of an asynchronous search task. ```APIDOC ## GET /api/v1/scraper/search/tasks/{task_id} ### Description Retrieves the status and results of a previously submitted asynchronous search task. ### Method GET ### Endpoint https://scrape.evomi.com/api/v1/scraper/search/tasks/{task_id} ### Parameters #### Path Parameters - **task_id** (string) - Required - The unique identifier for the search task ### Response #### Success Response (200) - **status** (string) - Current status of the task - **results** (array) - Search results if the task is completed ``` -------------------------------- ### Run an Existing Config using Evomi API Source: https://docs.evomi.com/integrations/agent-requests Execute a pre-saved scraping configuration by its name. Replace 'YOUR_API_KEY' with your actual API key. ```bash curl -X POST "https://scrape.evomi.com/api/v1/agent/request" \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "message": "Run my config named \"Product Scraper\" and show me the results" }' ``` -------------------------------- ### GET /account/configs Source: https://docs.evomi.com/scraping-products-instructions/scraper-api/scraper-config List all saved scraper configurations with pagination support. ```APIDOC ## GET /account/configs ### Description Retrieves a list of all saved scraper configurations. ### Method GET ### Endpoint https://scrape.evomi.com/api/v1/account/configs ### Parameters #### Query Parameters - **page** (integer) - Optional - Page number for pagination (default: 1) - **per_page** (integer) - Optional - Results per page (default: 20, max: 100) - **sort_by** (string) - Optional - Sort field: created_at, updated_at, name (default: created_at) - **sort_order** (string) - Optional - Sort direction: asc or desc (default: desc) ### Response #### Success Response (200) - **configs** (array) - List of configuration objects - **total** (integer) - Total number of configs #### Response Example { "success": true, "configs": [ { "id": "cfg_abc123", "name": "Product Scraper", "created_at": "2026-02-19T12:00:00" } ], "total": 1 } ```