### Start rayobrowse Docker Container Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md This snippet shows how to start the rayobrowse Docker container locally. It involves copying an example environment file, setting a required environment variable, and then running the Docker Compose command. Ensure Docker and Docker Compose are installed. ```bash cp .env.example .env # Edit .env → set STEALTH_BROWSER_ACCEPT_TERMS=true docker compose up -d ``` -------------------------------- ### Start rayobrowse Docker Container Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/scrapy/README.md This command sequence clones the rayobrowse repository, navigates into the directory, copies the example environment file, and starts the Docker Compose services in detached mode. This is the initial setup step to run the stealth browser daemon. ```bash git clone https://github.com/rayobyte-data/rayobrowse.git cd rayobrowse cp .env.example .env # Edit .env and set STEALTH_BROWSER_ACCEPT_TERMS=true docker compose up -d ``` -------------------------------- ### Docker Setup and Health Check Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt This snippet demonstrates how to start the Rayobrowse Docker container, configure environment variables, and verify the daemon's health status using a curl command. ```bash # Copy environment file and accept terms cp .env.example .env # Edit .env and set: STEALTH_BROWSER_ACCEPT_TERMS=true # Start the container docker compose up -d # Verify the daemon is healthy curl -s http://localhost:9222/health # Response: {"success": true, "data": {"status": "healthy", ...}} ``` -------------------------------- ### Start Rayobrowse with Docker Compose Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Command to start the Rayobrowse service in detached mode using Docker Compose. ```bash docker compose up -d ``` -------------------------------- ### Rayobrowse Connect Endpoint URL Parameters Examples Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt These examples illustrate how to configure Rayobrowse browser instances by modifying the query parameters in the `/connect` WebSocket URL. Options include setting the OS, proxy, specific Chrome versions, and enabling headful mode. ```bash # Windows fingerprint with proxy ws://localhost:9222/connect?headless=true&os=windows&proxy=http://user:pass@proxy.example.com:8080 # Specific Chrome version ws://localhost:9222/connect?headless=true&os=windows&browser_version_min=144&browser_version_max=144 # Android fingerprint for mobile sites ws://localhost:9222/connect?headless=true&os=android # Headful mode (visible in noVNC) ws://localhost:9222/connect?headless=false&os=windows ``` -------------------------------- ### Initialize and Start Rayobrowse Environment Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Commands to configure the environment variables and launch the Rayobrowse daemon using Docker Compose. ```bash cp .env.example .env docker compose up -d ``` -------------------------------- ### Install Scrapy and Scrapy-Playwright Dependencies Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/scrapy/README.md This command installs the necessary Python packages for Scrapy web scraping and the `scrapy-playwright` integration. It includes Scrapy itself, the Playwright Python library, and the `scrapy-playwright` plugin. Browser binaries are managed by the rayobrowse container, so `playwright install` is not needed. ```bash pip install scrapy scrapy-playwright ``` -------------------------------- ### GET /connect Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md Establishes a WebSocket connection to the browser automation service to initialize a new browser instance with specific fingerprinting parameters. ```APIDOC ## GET /connect ### Description Establishes a WebSocket connection to initialize a browser instance. The browser lifecycle is tied to the WebSocket connection; it is created upon connection and terminated shortly after disconnection. ### Method GET (WebSocket upgrade) ### Endpoint ws://localhost:9222/connect ### Parameters #### Query Parameters - **headless** (boolean) - Required - Set to 'true' for background execution or 'false' for visible mode (viewable via noVNC). - **os** (string) - Required - The target OS fingerprint: 'windows', 'linux', 'android', or 'macos'. - **proxy** (string) - Optional - Proxy URL in format http://user:pass@host:port. - **browser_version_min** (integer) - Optional - Minimum Chrome version to pin. - **browser_version_max** (integer) - Optional - Maximum Chrome version to pin. - **api_key** (string) - Optional - Authentication key if remote mode is enabled. ### Request Example ws://localhost:9222/connect?headless=true&os=windows&proxy=http://user:pass@proxy.example.com:8080 ### Response #### Success Response (101 Switching Protocols) - **Connection** (WebSocket) - Successful upgrade to WebSocket connection for CDP (Chrome DevTools Protocol) communication. #### Response Example { "status": "connected", "message": "Browser instance initialized with windows fingerprint" } ``` -------------------------------- ### Configure Rayobrowse Browser Instances Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Examples of configuring browser instances using the SDK, including proxy integration, specific fingerprint versions, multiple instances, and loading static fingerprint files from disk. ```python # With Proxy ws_url = create_browser( headless=False, target_os="windows", proxy="http://user:pass@proxy.example.com:8000", ) # Specific Fingerprint Version ws_url = create_browser( headless=False, target_os="windows", browser_name="chrome", browser_version_min=144, browser_version_max=144, ) # Multiple Browsers urls = [create_browser(headless=False, target_os="windows") for _ in range(3)] # Static Fingerprint Files ws_url = create_browser( fingerprint_file="fingerprints/windows_chrome.json" ) ``` -------------------------------- ### GET /browsers Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Retrieves a list of all currently active browser instances. ```APIDOC ## GET /browsers ### Description Returns a list of all active browser sessions managed by the daemon. ### Method GET ### Endpoint /browsers ### Response #### Success Response (200) - **browsers** (array) - List of active browser objects. ``` -------------------------------- ### Configure Rayobrowse Production Environment Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Example configuration for a production environment using environment variables. Defines daemon mode, API authentication, and network port settings. ```bash STEALTH_BROWSER_ACCEPT_TERMS=true STEALTH_BROWSER_DAEMON_MODE=remote STEALTH_BROWSER_API_KEY=your-production-api-key RAYOBROWSE_PORT=80 ``` -------------------------------- ### GET /connect Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md Establishes a WebSocket connection to a stealth-fingerprinted browser instance. This endpoint handles the automatic creation and cleanup of browser sessions based on provided query parameters. ```APIDOC ## GET /connect ### Description Establishes a CDP-compatible WebSocket connection to a new stealth browser instance. The browser is automatically created upon connection and destroyed when the connection closes. ### Method GET ### Endpoint /connect ### Parameters #### Query Parameters - **headless** (boolean) - Optional - Whether to run the browser in headless mode (default: true). - **os** (string) - Optional - Target OS fingerprint (windows, linux, android, macos). - **browser_name** (string) - Optional - Browser fingerprint type (default: chrome). - **browser_version_min** (string) - Optional - Minimum Chrome version. - **browser_version_max** (string) - Optional - Maximum Chrome version. - **proxy** (string) - Optional - Proxy URL (e.g., http://user:pass@host:port). - **browser_language** (string) - Optional - Accept-Language header. - **ui_language** (string) - Optional - Browser UI locale. - **api_key** (string) - Required in remote mode - Authentication key for remote access. ### Request Example ws://localhost:9222/connect?headless=true&os=windows&browser_name=chrome ### Response #### Success Response (101 Switching Protocols) - **WebSocket** - Connection established for CDP communication. #### Response Example { "status": "connected", "browser_id": "uuid-12345" } ``` -------------------------------- ### Example WebSocket Connection URLs for RayoBrowse Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md These URLs demonstrate various ways to connect to the RayoBrowse WebSocket endpoint, allowing customization of browser fingerprinting, operating system emulation, and proxy settings. They are essential for setting up different browser environments for web scraping or testing. ```url ws://localhost:9222/connect?headless=true&os=windows ``` ```url ws://localhost:9222/connect?headless=true&os=android&proxy=http://user:pass@proxy.example.com:8080 ``` ```url ws://localhost:9222/connect?headless=true&os=windows&browser_version_min=130&browser_version_max=132 ``` ```url ws://localhost:9222/connect?headless=false&os=windows ``` -------------------------------- ### GET /connect Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/scrapy/README.md Establishes a WebSocket connection to the Rayobrowse browser instance with specified configuration parameters. ```APIDOC ## GET /connect ### Description Connects to the Rayobrowse CDP service to initiate a browser session. This endpoint supports configuring the browser environment, proxy settings, and authentication for remote instances. ### Method GET ### Endpoint ws://[host]:[port]/connect ### Parameters #### Query Parameters - **headless** (boolean) - Optional - Whether to run the browser in headless mode. Default: true. - **os** (string) - Optional - Fingerprint OS: windows, linux, android, macos. Default: linux. - **browser_name** (string) - Optional - Browser fingerprint type. Default: chrome. - **proxy** (string) - Optional - Proxy URL in format http://user:pass@host:port. - **api_key** (string) - Optional - Required for remote mode authentication. - **browser_language** (string) - Optional - Accept-Language header value. - **ui_language** (string) - Optional - Browser UI locale. ### Request Example ws://localhost:9222/connect?headless=true&os=windows&proxy=http://user:pass@proxy.example.com:8080 ### Response #### Success Response (101 Switching Protocols) - **Connection** (header) - Upgraded to WebSocket connection for CDP communication. #### Response Example { "status": "connected", "message": "CDP session established" } ``` -------------------------------- ### GET /browsers Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Retrieves a list of all currently active browser sessions managed by the daemon. ```APIDOC ## GET /browsers ### Description List all active browser sessions. ### Method GET ### Endpoint /browsers ### Response #### Success Response (200) - **success** (bool) - Status of the request - **data** (object) - List of active browsers #### Response Example { "success": true, "data": { "browsers": [ { "browser_id": "br_59245e8658532863", "ws_endpoint": "ws://localhost:9222/cdp/br_59245e8658532863", "created_at": "2024-01-15T10:30:00Z" } ] } } ``` -------------------------------- ### Connect to Rayobrowse via CDP WebSocket (Playwright) Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Connects to a Rayobrowse instance using Playwright's `connect_over_cdp` method. This example uses the `/connect` endpoint with an API key for authentication. ```python from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.connect_over_cdp( "ws://your-server/connect?headless=true&os=windows&api_key=your_api_key_here" ) page = browser.new_context().new_page() page.goto("https://example.com") ``` -------------------------------- ### Rayobrowse REST API Response for Browser Creation Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Example JSON response from the `POST /browser` endpoint, containing the browser ID and the WebSocket endpoint for connecting via CDP. ```json { "success": true, "data": { "browser_id": "br_59245e8658532863", "ws_endpoint": "ws://your-server/cdp/br_59245e8658532863" } } ``` -------------------------------- ### Checking Docker Compose Logs Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md This command retrieves the last 50 lines of logs from the Docker Compose services. It is useful for diagnosing issues with the RayoBrowse setup or other related containers. ```bash docker compose logs --tail=50 ``` -------------------------------- ### GET /connect Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Establishes a direct WebSocket connection to a new stealth browser instance. The browser is automatically created upon connection and cleaned up upon disconnection. ```APIDOC ## GET /connect ### Description Connects to a stealth browser instance via WebSocket. This endpoint supports various query parameters to configure the browser's fingerprint and behavior. ### Method GET (WebSocket upgrade) ### Endpoint ws://localhost:9222/connect ### Parameters #### Query Parameters - **headless** (boolean) - Optional - Run without GUI (default: true) - **os** (string) - Optional - Fingerprint OS: windows, linux, android, macos (default: linux) - **browser_name** (string) - Optional - Browser fingerprint type (default: chrome) - **browser_version_min** (integer) - Optional - Minimum Chrome version - **browser_version_max** (integer) - Optional - Maximum Chrome version - **proxy** (string) - Optional - Proxy URL (e.g., http://user:pass@host:port) - **api_key** (string) - Optional - Required in remote mode ### Request Example ws://localhost:9222/connect?headless=true&os=windows&proxy=http://user:pass@proxy.example.com:8080 ### Response #### Success Response (101 Switching Protocols) - **WebSocket** - Connection established to the Chrome DevTools Protocol (CDP) session. ``` -------------------------------- ### Verify Rayobrowse Setup with Python Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt A script to verify the health of the Rayobrowse daemon and perform an end-to-end test using Playwright. It checks the health endpoint, initializes a browser instance, and navigates to a test URL. ```python import json import urllib.request from rayobrowse import create_browser from playwright.sync_api import sync_playwright # Step 1: Check daemon health with urllib.request.urlopen("http://localhost:9222/health", timeout=10) as r: data = json.loads(r.read()) assert data.get("success"), "Daemon health check failed" print("Daemon healthy.") # Step 2: Create a browser ws_url = create_browser(headless=True, target_os="windows") print(f"Browser ready: {ws_url}") # Step 3: Load a page with Playwright with sync_playwright() as p: browser = p.chromium.connect_over_cdp(ws_url) context = browser.contexts[0] if browser.contexts else browser.new_context() page = context.pages[0] if context.pages else context.new_page() page.goto("https://example.com", wait_until="domcontentloaded", timeout=30000) title = page.title() browser.close() print(f'Page title: "{title}"') print("Setup verified successfully.") ``` -------------------------------- ### GET /health Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Checks the health status of the Rayobrowse daemon. ```APIDOC ## GET /health ### Description Returns the current health status of the Rayobrowse daemon to ensure it is ready to accept connections. ### Method GET ### Endpoint http://localhost:9222/health ### Response #### Success Response (200) - **success** (boolean) - Indicates if the request was successful - **data** (object) - Contains status information including 'healthy' #### Response Example { "success": true, "data": { "status": "healthy" } } ``` -------------------------------- ### GET /health Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Checks the operational status and version of the Rayobrowse daemon. ```APIDOC ## GET /health ### Description Check daemon health status. ### Method GET ### Endpoint /health ### Response #### Success Response (200) - **status** (string) - Health status - **version** (string) - Daemon version ``` -------------------------------- ### POST /browser Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Provisions a new remote browser instance and returns a WebSocket endpoint for CDP connection. ```APIDOC ## POST /browser ### Description Creates a new stealth browser instance. Returns a unique browser ID and a WebSocket endpoint for direct CDP connection. ### Method POST ### Endpoint /browser ### Parameters #### Request Body - **headless** (boolean) - Optional - Whether to run the browser in headless mode. - **os** (string) - Optional - The operating system fingerprint to emulate (e.g., "windows"). ### Request Example { "headless": true, "os": "windows" } ### Response #### Success Response (200) - **success** (boolean) - Indicates if the request was successful. - **data** (object) - Contains browser_id and ws_endpoint. #### Response Example { "success": true, "data": { "browser_id": "br_59245e8658532863", "ws_endpoint": "ws://your-server/cdp/br_59245e8658532863" } } ``` -------------------------------- ### Connect via WebSocket (/connect) Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Connect any CDP client directly to the /connect endpoint without needing an SDK. Customize the browser via query parameters. ```APIDOC ## WebSocket Connection (/connect) ### Description Connect any CDP client directly to the `/connect` endpoint. No SDK needed. ### Method GET ### Endpoint `ws://localhost:9222/connect` ### Query Parameters - **headless** (boolean) - Optional - `true` or `false`. Defaults to `true`. - **os** (string) - Optional - Fingerprint OS: `windows`, `linux`, `android`, `macos`. Defaults to `linux`. - **browser_name** (string) - Optional - Browser fingerprint type. Defaults to `chrome`. - **browser_version_min** (integer) - Optional - Minimum Chrome version. Defaults to latest. - **browser_version_max** (integer) - Optional - Maximum Chrome version. Defaults to latest. - **proxy** (string) - Optional - Proxy URL, e.g. `http://user:pass@host:port`. - **browser_language** (string) - Optional - Accept-Language value. Defaults to auto. - **ui_language** (string) - Optional - Browser UI locale. Defaults to auto. - **screen_width_min** (integer) - Optional - Minimum screen width. Defaults to auto. - **screen_height_min** (integer) - Optional - Minimum screen height. Defaults to auto. - **api_key** (string) - Optional - Required in remote mode. ### Request Example ``` ws://localhost:9222/connect?headless=true&os=windows&proxy=http://user:pass@host:port ``` ### Response #### Success Response (101 Switching Protocols) - A WebSocket connection is established. #### Response Example (No JSON response, a WebSocket connection is established) ``` -------------------------------- ### Python SDK - create_browser Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Use the Python SDK for more control over the browser lifecycle. This function returns a WebSocket URL for connecting. ```APIDOC ## Python SDK - create_browser ### Description Use the Python SDK for more control over the browser lifecycle. This function returns a WebSocket URL for connecting. ### Method N/A (Python function) ### Endpoint N/A (Python function) ### Parameters - **headless** (boolean) - Optional - `true` or `false`. - **target_os** (string) - Optional - Fingerprint OS: `windows`, `linux`, `android`, `macos`. - **browser_name** (string) - Optional - Browser fingerprint type. - **browser_version_min** (integer) - Optional - Minimum Chrome version. - **browser_version_max** (integer) - Optional - Maximum Chrome version. - **proxy** (string) - Optional - Proxy URL, e.g. `http://user:pass@proxy.example.com:8000`. - **fingerprint_file** (string) - Optional - Path to a JSON file containing fingerprint data. ### Request Example ```python from rayobrowse import create_browser ws_url = create_browser( headless=False, target_os="windows", browser_name="chrome", browser_version_min=144, browser_version_max=144, proxy="http://user:pass@proxy.example.com:8000" ) ``` ### Response #### Success Response - **ws_url** (string) - The WebSocket URL to connect to the browser. #### Response Example ```json { "ws_url": "ws://localhost:9222/connect?headless=false&os=windows&browser_name=chrome&browser_version_min=144&browser_version_max=144&proxy=http://user:pass@proxy.example.com:8000" } ``` ``` -------------------------------- ### Create Browser Sessions Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Demonstrates how to initialize browser instances with specific configurations such as headless mode, proxy settings, and version pinning. Supports both the Python SDK and REST API interfaces. ```python from rayobrowse import create_browser # Multiple browsers concurrently urls = [create_browser(headless=False, target_os="windows") for _ in range(3)] # With proxy for geo-targeted scraping ws_url = create_browser( headless=True, target_os="windows", proxy="http://user:pass@proxy.example.com:8080", ) # Specific browser version ws_url = create_browser( target_os="windows", browser_name="chrome", browser_version_min=144, browser_version_max=144 ) ``` ```bash curl -X POST http://localhost:9222/browser \ -H "Content-Type: application/json" \ -d '{ "headless": true, "os": "windows", "browser_name": "chrome", "browser_version_min": 144, "browser_version_max": 144 }' ``` -------------------------------- ### GET /health Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Checks the health status of the Rayobrowse daemon to ensure the service is running and ready to accept requests. ```APIDOC ## GET /health ### Description Verifies that the Rayobrowse daemon is operational and ready to process browser requests. ### Method GET ### Endpoint http://localhost:9222/health ### Response #### Success Response (200) - **success** (boolean) - Indicates if the daemon is healthy. #### Response Example { "success": true } ``` -------------------------------- ### Environment Variables Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Configure Rayobrowse daemon behavior using environment variables in a `.env` file. ```APIDOC ## Environment Variables Configure the daemon behavior through environment variables in `.env`. | Variable | Default | Description | |----------|---------|-------------| | `STEALTH_BROWSER_ACCEPT_TERMS` | `false` | **Required.** Set to `true` to accept license and enable daemon | | `STEALTH_BROWSER_API_KEY` | *(empty)* | API key for paid plans and remote mode authentication | | `STEALTH_BROWSER_NOVNC` | `true` | Enable browser viewer at `http://localhost:6080` | | `STEALTH_BROWSER_DAEMON_MODE` | `local` | `local` or `remote`. Remote enables API key auth | | `STEALTH_BROWSER_PUBLIC_URL` | *(empty)* | Base URL for CDP endpoints in remote mode | | `RAYOBROWSE_PORT` | `9222` | Host port for daemon | ```bash # Example .env for local development STEALTH_BROWSER_ACCEPT_TERMS=true STEALTH_BROWSER_NOVNC=true ``` ``` -------------------------------- ### GET /health Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/scrapy/README.md Checks the health status of the Rayobrowse daemon to ensure the service is running and ready to accept connections. ```APIDOC ## GET /health ### Description Returns the current health status of the Rayobrowse container. ### Method GET ### Endpoint /health ### Response #### Success Response (200) - **success** (boolean) - Indicates if the request was successful. - **data** (object) - Contains status details. #### Response Example { "success": true, "data": { "status": "healthy" } } ``` -------------------------------- ### Configure Environment Variables for Rayobrowse Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Sets up the necessary environment variables for Rayobrowse to run in remote mode. This includes API keys, port configurations, and terms acceptance. ```bash STEALTH_BROWSER_ACCEPT_TERMS=true STEALTH_BROWSER_API_KEY=your_api_key_here STEALTH_BROWSER_DAEMON_MODE=remote RAYOBROWSE_PORT=80 # Optional: set if you have a domain, otherwise public IP is auto-detected # STEALTH_BROWSER_PUBLIC_URL=http://browser.example.com ``` -------------------------------- ### POST /browser Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Creates a new stealth browser session with specific configuration parameters like OS, proxy, and browser version. ```APIDOC ## POST /browser ### Description Creates a new stealth browser session programmatically. ### Method POST ### Endpoint /browser ### Request Body - **headless** (bool) - Optional - Run without GUI - **os** (str) - Required - Target OS: windows, android, linux, macos - **browser_name** (str) - Optional - Browser fingerprint type - **browser_version_min** (int) - Optional - Min Chrome version - **browser_version_max** (int) - Optional - Max Chrome version - **proxy** (str) - Optional - Proxy URL ### Request Example { "headless": true, "os": "windows", "browser_name": "chrome" } ### Response #### Success Response (200) - **success** (bool) - Status of the request - **data** (object) - Contains browser_id and ws_endpoint #### Response Example { "success": true, "data": { "browser_id": "br_59245e8658532863", "ws_endpoint": "ws://localhost:9222/cdp/br_59245e8658532863" } } ``` -------------------------------- ### Manage Browser Lifecycle with Python SDK Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Uses the Rayobrowse Python SDK to programmatically create browser instances with specific configurations. This approach provides more control over the browser lifecycle compared to direct CDP connection. ```python from rayobrowse import create_browser from playwright.sync_api import sync_playwright ws_url = create_browser(headless=False, target_os="windows") with sync_playwright() as p: browser = p.chromium.connect_over_cdp(ws_url) page = browser.contexts[0].pages[0] page.goto("https://example.com") browser.close() ``` -------------------------------- ### Upgrade Rayobrowse Components Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Commands to update the Docker container image and the Python SDK to the latest versions. ```bash docker compose pull && docker compose up -d pip install --upgrade -r requirements.txt ``` -------------------------------- ### Enable and Access noVNC Browser Viewer Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Configuration to enable the noVNC interface for real-time browser monitoring. Requires setting the environment variable and navigating to the VNC web interface. ```bash # Enable noVNC (enabled by default) STEALTH_BROWSER_NOVNC=true # Connect with headless=false to see the browser # ws://localhost:9222/connect?headless=false&os=windows # Open browser viewer # Navigate to: http://localhost:6080/vnc.html ``` -------------------------------- ### Enable Debug Logging Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Python configuration to enable verbose debug logging for troubleshooting anti-scraping issues. ```python import logging logging.basicConfig(level=logging.DEBUG) ``` -------------------------------- ### Request a New Browser Instance via REST API Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Makes a POST request to the Rayobrowse `/browser` endpoint to programmatically request a new browser instance. Requires authentication via the `X-API-Key` header. ```bash curl -X POST http://your-server/browser \ -H "Content-Type: application/json" \ -H "X-API-Key: your_api_key_here" \ -d '{"headless": true, "os": "windows"}' ``` -------------------------------- ### Connect to Rayobrowse via Playwright Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Demonstrates how to connect a Playwright script to the Rayobrowse CDP endpoint to perform web automation with stealth features enabled. ```python from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.connect_over_cdp( "ws://localhost:9222/connect?headless=false&os=windows" ) page = browser.new_context().new_page() page.goto("https://example.com") print(page.title()) input("Browser open — view at http://localhost:6080/vnc.html. Press Enter to close...") browser.close() ``` -------------------------------- ### Verify OpenClaw Browser Profile Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md This command verifies that the OpenClaw browser profile is correctly set up and connected to rayobrowse. It checks the status of the specified browser profile, confirming that OpenClaw can utilize rayobrowse for automation. ```bash openclaw browser --browser-profile rayobrowse status ``` -------------------------------- ### Connect to Rayobrowse via CDP Endpoint Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Demonstrates how to connect a Playwright client directly to the Rayobrowse /connect endpoint using WebSocket URLs. This method allows for browser configuration via query parameters like OS and headless mode. ```python from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.connect_over_cdp( "ws://localhost:9222/connect?headless=true&os=windows" ) page = browser.new_context().new_page() page.goto("https://example.com") print(page.title()) browser.close() ``` -------------------------------- ### Remote Mode Configuration and Connection Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Details how to deploy Rayobrowse in remote mode with API key authentication. Includes server environment variables, client connection code, and REST API usage. ```bash STEALTH_BROWSER_ACCEPT_TERMS=true STEALTH_BROWSER_DAEMON_MODE=remote STEALTH_BROWSER_API_KEY=your-secret-key RAYOBROWSE_PORT=80 ``` ```python from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.connect_over_cdp( "ws://your-server.example.com/connect?headless=true&os=windows&api_key=your-secret-key" ) page = browser.new_context().new_page() page.goto("https://example.com") print(page.title()) browser.close() ``` ```bash curl -X POST http://your-server.example.com/browser \ -H "Content-Type: application/json" \ -H "Authorization: Bearer your-secret-key" \ -d '{"headless": true, "os": "windows"}' ``` -------------------------------- ### Configure rayobrowse for Remote Mode Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md This snippet shows the environment variable settings for running rayobrowse in remote mode on a server. Key variables include enabling terms acceptance, setting the daemon mode to 'remote', providing an API key for authentication, and specifying the port. ```bash STEALTH_BROWSER_ACCEPT_TERMS=true STEALTH_BROWSER_DAEMON_MODE=remote STEALTH_BROWSER_API_KEY=your-secret-key #contact support@rayobyte.com for a key RAYOBROWSE_PORT=80 ``` -------------------------------- ### Python SDK: Create Browser Instance (Playwright) Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt This Python code utilizes the Rayobrowse SDK's `create_browser` function to programmatically launch a stealth browser with specific configurations. It then connects to the returned WebSocket URL using Playwright to interact with the browser. ```python from rayobrowse import create_browser from playwright.sync_api import sync_playwright # Create a browser with specific fingerprint configuration ws_url = create_browser( headless=False, # Run in headful mode target_os="windows", # Windows fingerprint (windows, android tested; macos, linux experimental) browser_name="chrome", # Browser type browser_version_min=144, # Minimum Chrome version browser_version_max=144, # Maximum Chrome version # proxy="http://user:pass@host:port", # Optional proxy # browser_language="en-US,en;q=0.9", # Optional language # fingerprint_file="fingerprints/custom.json", # Optional static fingerprint # launch_args=["--disable-extensions"], # Optional Chromium flags ) print(f"Browser ready: {ws_url}") # Output: Browser ready: ws://localhost:9222/cdp/br_59245e8658532863 # Connect with Playwright with sync_playwright() as p: browser = p.chromium.connect_over_cdp(ws_url) context = browser.contexts[0] if browser.contexts else browser.new_context() page = context.pages[0] if context.pages else context.new_page() page.goto("https://example.com", wait_until="domcontentloaded", timeout=30000) print(f"Page title: {page.title()}") # Output: Page title: Example Domain # Browser auto-cleans up 2 seconds after connection closes browser.close() ``` -------------------------------- ### Configure OpenClaw for Remote rayobrowse Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md This JSON configuration shows how to connect a local OpenClaw instance to a remotely deployed rayobrowse server. The `cdpUrl` includes the server's address, the API key for authentication, and standard connection parameters like `headless` and `os`. ```json // ~/.openclaw/openclaw.json { browser: { enabled: true, defaultProfile: "rayobrowse", profiles: { rayobrowse: { cdpUrl: "ws://your-server.example.com/connect?headless=true&os=windows&api_key=your-secret-key", }, }, }, } ``` -------------------------------- ### Testing RayoBrowse Endpoint with Playwright (Python) Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md This Python script uses the Playwright library to connect to the RayoBrowse WebSocket endpoint and perform a basic web navigation task. It demonstrates how to establish a connection, open a page, retrieve the page title, and close the browser instance. ```python from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.connect_over_cdp( "ws://localhost:9222/connect?headless=true&os=windows" ) page = browser.contexts[0].pages[0] page.goto("https://example.com") print(page.title()) browser.close() ``` -------------------------------- ### Configure OpenClaw with rayobrowse (JSON) Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md This JSON configuration snippet defines a 'rayobrowse' browser profile for OpenClaw. It specifies the `cdpUrl` to connect to the rayobrowse daemon, enabling stealthy browser automation. This configuration can be added to `~/.openclaw/openclaw.json`. ```json { browser: { enabled: true, defaultProfile: "rayobrowse", profiles: { rayobrowse: { cdpUrl: "ws://localhost:9222/connect?headless=true&os=windows", }, }, }, } ``` -------------------------------- ### Verify Daemon Health and Logs Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Commands to check the status of the Rayobrowse daemon via HTTP health checks and to inspect container logs for troubleshooting. ```bash curl http://localhost:9222/health ``` ```bash docker compose logs -f ``` -------------------------------- ### OpenClaw AI Agent Integration Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Integrate Rayobrowse with OpenClaw for browser-based AI agent tasks by configuring OpenClaw's settings. ```APIDOC ## OpenClaw AI Agent Integration Configure OpenClaw to use rayobrowse for browser-based AI agent tasks. ### JSON Configuration ```json5 // ~/.openclaw/openclaw.json { browser: { enabled: true, defaultProfile: "rayobrowse", profiles: { rayobrowse: { cdpUrl: "ws://localhost:9222/connect?headless=true&os=windows", }, }, }, } ``` ### CLI Configuration ```bash # Or configure via CLI openclaw config set browser.enabled true openclaw config set browser.defaultProfile rayobrowse openclaw config set 'browser.profiles.rayobrowse.cdpUrl' 'ws://localhost:9222/connect?headless=true&os=windows' # Verify configuration openclaw browser --browser-profile rayobrowse status ``` ``` -------------------------------- ### Configure OpenClaw with rayobrowse (CLI) Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/openclaw/README.md These bash commands configure OpenClaw to use rayobrowse via its command-line interface. They enable the browser feature, set 'rayobrowse' as the default profile, and specify the `cdpUrl` for the connection. OpenClaw automatically reloads these settings. ```bash openclaw config set browser.enabled true openclaw config set browser.defaultProfile rayobrowse openclaw config set 'browser.profiles.rayobrowse.cdpUrl' 'ws://localhost:9222/connect?headless=true&os=windows' ``` -------------------------------- ### Remote Mode Configuration Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Deploy Rayobrowse on a server for remote access, enabling API key authentication for secure connections. ```APIDOC ## Remote Mode Configuration Deploy rayobrowse on a server for remote access with API key authentication. ### Server .env Configuration ```bash # Server .env configuration STEALTH_BROWSER_ACCEPT_TERMS=true STEALTH_BROWSER_DAEMON_MODE=remote STEALTH_BROWSER_API_KEY=your-secret-key RAYOBROWSE_PORT=80 # Optional: STEALTH_BROWSER_PUBLIC_URL=http://browser.example.com ``` ### Client Connection (Python) ```python # Client connection with API key from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.connect_over_cdp( "ws://your-server.example.com/connect?headless=true&os=windows&api_key=your-secret-key" ) page = browser.new_context().new_page() page.goto("https://example.com") print(page.title()) browser.close() ``` ### REST API Request ```bash # REST API with authentication headers curl -X POST http://your-server.example.com/browser \ -H "Content-Type: application/json" \ -H "X-API-Key: your-secret-key" \ -d '{"headless": true, "os": "windows"}' # Or using Authorization header curl -X POST http://your-server.example.com/browser \ -H "Content-Type: application/json" \ -H "Authorization: Bearer your-secret-key" \ -d '{"headless": true, "os": "windows"}' ``` ``` -------------------------------- ### Integrate with Playwright Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Connects Playwright to a Rayobrowse instance using either the direct /connect endpoint or the Python SDK for advanced session control. ```python from playwright.sync_api import sync_playwright from rayobrowse import create_browser # Method 1: Direct /connect endpoint with sync_playwright() as p: browser = p.chromium.connect_over_cdp( "ws://localhost:9222/connect?headless=true&os=windows" ) page = browser.new_context().new_page() page.goto("https://example.com") browser.close() # Method 2: Using Python SDK ws_url = create_browser(headless=False, target_os="windows") with sync_playwright() as p: browser = p.chromium.connect_over_cdp(ws_url) page = browser.contexts[0].pages[0] page.goto("https://example.com", wait_until="commit") browser.close() ``` -------------------------------- ### DELETE /browser/{id} Source: https://github.com/rayobyte-data/rayobrowse/blob/main/README.md Terminates a specific browser instance by its ID. ```APIDOC ## DELETE /browser/{id} ### Description Shuts down and removes the specified browser instance. ### Method DELETE ### Endpoint /browser/{id} ### Parameters #### Path Parameters - **id** (string) - Required - The unique identifier of the browser to terminate. ``` -------------------------------- ### Run Scrapy Spider with Rayobrowse Source: https://github.com/rayobyte-data/rayobrowse/blob/main/integrations/scrapy/README.md Executes a Scrapy spider named 'quotes' and saves the scraped data to a JSON file. This command initiates the crawling process, and the output will show connection details and scraping statistics. ```bash scrapy crawl quotes -o quotes.json ``` -------------------------------- ### Manage Browser Sessions via REST API Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Provides endpoints for listing active browser sessions and terminating specific sessions by their ID. ```bash # List all active browsers curl http://localhost:9222/browsers # Delete a specific browser session curl -X DELETE http://localhost:9222/browser/br_59245e8658532863 ``` -------------------------------- ### Check Daemon Health Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Verifies the status and version of the running Rayobrowse daemon. ```bash curl http://localhost:9222/health ``` -------------------------------- ### Configure OpenClaw for Rayobrowse Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt Configures the OpenClaw AI agent to utilize Rayobrowse as its default browser profile. Supports both JSON configuration files and CLI commands. ```json5 { browser: { enabled: true, defaultProfile: "rayobrowse", profiles: { rayobrowse: { cdpUrl: "ws://localhost:9222/connect?headless=true&os=windows", }, }, }, } ``` ```bash openclaw config set browser.enabled true openclaw config set browser.defaultProfile rayobrowse openclaw config set 'browser.profiles.rayobrowse.cdpUrl' 'ws://localhost:9222/connect?headless=true&os=windows' ``` -------------------------------- ### Connect Selenium to Rayobrowse via CDP Shim (Python) Source: https://context7.com/rayobyte-data/rayobrowse/llms.txt This snippet shows how to integrate Selenium with Rayobrowse using a custom CDP shim. The shim acts as a bridge between ChromeDriver and the Rayobrowse daemon's WebSocket endpoint. It requires the 'rayobrowse' and 'selenium' Python packages, along with 'webdriver-manager'. The code dynamically determines the ChromeDriver version needed based on the browser version provided by the daemon. ```python import http.server import json import socket import threading import urllib.request from contextlib import contextmanager from urllib.parse import urlparse from rayobrowse import create_browser from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.chrome.service import Service from webdriver_manager.chrome import ChromeDriverManager def _free_port(): """Find an available TCP port on localhost.""" with socket.socket() as s: s.bind(("127.0.0.1", 0)) return s.getsockname()[1] @contextmanager def cdp_shim(ws_url): """Bridge ChromeDriver to the daemon's CDP proxy.""" parsed = urlparse(ws_url) daemon_endpoint = f"http://{parsed.hostname}:{parsed.port}" browser_id = parsed.path.strip("/").split("/")[-1] base_url = f"{daemon_endpoint}/cdp/{browser_id}" class _ShimHandler(http.server.BaseHTTPRequestHandler): def do_GET(self): path = self.path.lstrip("/") proxy_url = f"{base_url}/{path}" if path else f"{base_url}/json/version" try: with urllib.request.urlopen(proxy_url, timeout=5) as resp: body = resp.read() self.send_response(200) self.send_header("Content-Type", "application/json") self.end_headers() self.wfile.write(body) except Exception as exc: self.send_response(500) self.end_headers() self.wfile.write(json.dumps({"error": str(exc)}).encode()) def log_message(self, *args): pass port = _free_port() server = http.server.HTTPServer(("127.0.0.1", port), _ShimHandler) thread = threading.Thread(target=server.serve_forever, daemon=True) thread.start() try: yield f"127.0.0.1:{port}" finally: server.shutdown() # Create stealth browser ws_url = create_browser( headless=False, target_os="windows", browser_version_min=144, browser_version_max=144, ) # Detect Chrome version for matching ChromeDriver parsed = urlparse(ws_url) daemon_endpoint = f"http://{parsed.hostname}:{parsed.port}" browser_id = parsed.path.strip("/").split("/")[-1] with urllib.request.urlopen(f"{daemon_endpoint}/cdp/{browser_id}/json/version") as resp: version_info = json.loads(resp.read()) chrome_version = version_info["Browser"].split("/")[1].split(".")[0] # Connect Selenium via CDP shim with cdp_shim(ws_url) as shim_addr: options = Options() options.debugger_address = shim_addr service = Service(ChromeDriverManager(driver_version=chrome_version).install()) driver = webdriver.Chrome(service=service, options=options) driver.get("https://example.com") print(f"Page title: {driver.title}") # Output: Page title: Example Domain driver.quit() ```