### Install and use the example provider plugin Source: https://github.com/yail259/frankensurf/blob/main/examples/frankensurf-plugin-example/README.md Install the plugin into Frankensurf's environment, list plugins, trust the provider, set the EXAMPLE_API_URL and EXAMPLE_API_KEY environment variables, and read a page using the example_fetch provider with the --paid flag. ```bash pip install -e examples/frankensurf-plugin-example # into Frankensurf's environment frankensurf plugins # listed as PLUGIN_NOT_TRUSTED frankensurf plugins trust provider example_fetch frankensurf-plugin-example export EXAMPLE_API_URL=https://your-service.example/read EXAMPLE_API_KEY=... frankensurf read https://example.com --provider example_fetch --paid ``` -------------------------------- ### Install Frankensurf with uv or pipx and run initial commands Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/install.mdx Installs the frankensurf command and frankensurf-mcp for agents using uv tool install or pipx install. Then runs frankensurf install and an example read command. ```bash uv tool install "frankensurf[mcp]" # or: pipx install "frankensurf[mcp]" frankensurf install frankensurf read "https://news.ycombinator.com" --explain ``` -------------------------------- ### Install Frankensurf and read a page Source: https://github.com/yail259/frankensurf/blob/main/README.md Installs Frankensurf with MCP support, installs all free tools, and reads a Walmart search page with explanation. The read command climbs through tools until it gets the real page, as shown in the output. ```bash pip install "frankensurf[mcp]" frankensurf install # every free tool: browsers, stealth browsers, local search, Crawl4AI frankensurf read "https://www.walmart.com/search?q=air+fryer" --explain ``` ```text https://www.walmart.com/search?q=air+fryer ✗ http CAPTCHA 0.4s ✗ local CAPTCHA 1.5s ✓ camoufox got the page 15.6s air fryer - Walmart.com 24,192 characters via camoufox, complete (107 results), free ``` -------------------------------- ### Install, stop, and uninstall SearXNG with frankensurf CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/searxng.mdx Commands to install and start SearXNG (about 140 MB, a few seconds), stop or check status, and remove it. The install downloads a pinned commit, verifies SHA-256, installs Python packages in its own environment, and runs it in the background on 127.0.0.1:8088 with JSON output. ```bash frankensurf install search # install and start it (about 140 MB, a few seconds) frankensurf search-engine stop # any time; also: start, status frankensurf uninstall search # remove it ``` -------------------------------- ### Install and run the Astro site Source: https://github.com/yail259/frankensurf/blob/main/site/README.md Install dependencies, run the dev server on http://localhost:4322, or build the static output into dist/. ```bash npm install npm run dev # http://localhost:4322 npm run build # static output in dist/ ``` -------------------------------- ### Install Frankensurf from a clone for development Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/install.mdx Clones the repository and installs with pip in editable mode including test and mcp extras, then runs frankensurf install. ```bash git clone https://github.com/yail259/frankensurf.git pip install -e '.[test,mcp]' frankensurf install ``` -------------------------------- ### Set up development environment and run tests Source: https://github.com/yail259/frankensurf/blob/main/README.md Creates a Python virtual environment, installs the package with test and MCP extras, installs Frankensurf, and runs the test suite with PYTHONPATH set to include the source directory. ```bash python3 -m venv .venv .venv/bin/pip install -e '.[test,mcp]' .venv/bin/frankensurf install PYTHONPATH=.:src .venv/bin/pytest -q ``` -------------------------------- ### Install optional extras: nodriver, Steel, SOCKS Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/install.mdx Commands to install optional extras: nodriver via frankensurf install, Steel via a script, and SOCKS support via pip or uv tool install. ```bash frankensurf install nodriver ``` ```bash bash scripts/start-steel.sh ``` ```bash pip install "frankensurf[socks]" ``` ```bash uv tool install "frankensurf[mcp,socks]" ``` -------------------------------- ### Install Frankensurf on Windows via WSL Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/install.mdx Installs Python venv support, creates a virtual environment, installs the package with the MCP extra, installs tools with system dependencies, and reads a page. This is the second step for Windows users, run inside Ubuntu. ```bash sudo apt install -y python3-venv python3 -m venv ~/.venvs/frankensurf && source ~/.venvs/frankensurf/bin/activate pip install "frankensurf[mcp]" frankensurf install --with-deps frankensurf read "https://www.walmart.com/search?q=air+fryer" --explain ``` -------------------------------- ### Installation commands for the plugin Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/more/plugins.mdx Shows how to install the plugin into Frankensurf's environment using pip, uv tool, or pipx inject. ```bash pip install -e ./frankensurf-plugin-example # pip or a venv uv tool install frankensurf --with-editable ./frankensurf-plugin-example # uv tool pipx inject frankensurf -e ./frankensurf-plugin-example # pipx ``` -------------------------------- ### Install Browser Use SDK in a virtual environment Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/browser-use.mdx Creates a Python virtual environment and installs the pinned SDK from requirements-browser-use.txt. ```bash python3 -m venv ~/.local/share/frankensurf/agent-venv ~/.local/share/frankensurf/agent-venv/bin/python -m pip install -r requirements-browser-use.txt ``` -------------------------------- ### Install Frankensurf with MCP extra and tools on macOS Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/install.mdx Creates a virtual environment, installs the package with the MCP extra, and then installs every free tool. This is the second step in the macOS installation workflow. ```bash python3 -m venv ~/.venvs/frankensurf && source ~/.venvs/frankensurf/bin/activate pip install "frankensurf[mcp]" frankensurf install ``` -------------------------------- ### Install WSL with Ubuntu on Windows Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/install.mdx Installs WSL with Ubuntu from an administrator PowerShell. This is the first step for Windows users. ```powershell wsl --install -d Ubuntu ``` -------------------------------- ### Discover a site module (Python, CLI, MCP) Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/site-modules.mdx Use `module discover` to draft a module from the best list of items found on a page. The Python example shows getting drafts and saving with `save=True`; the CLI and MCP equivalents are shown. Check the sample before saving, as drafts can pick the wrong list. ```python found = await web.discover_module("https://www.example.com/search?q=lamp") best = found["drafts"][0] # module, count, fields, sample await web.discover_module("https://www.example.com/search?q=lamp", save=True) ``` ```bash frankensurf module discover "https://www.example.com/search?q=lamp" --save ``` ```json { "tool": "discover_site_module", "arguments": { "url": "https://www.example.com/search?q=lamp", "save": true } } ``` -------------------------------- ### Example .env file with API keys Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/blocked/paid-tools.mdx Example of a .env file at ~/.config/frankensurf/.env containing API keys for ZenRows, Firecrawl, and Exa. The file should be readable only by you (chmod 600). Keys never appear in results, receipts, or errors. ```bash FRANKENSURF_ZENROWS_API_KEY=... FRANKENSURF_FIRECRAWL_API_KEY=... FRANKENSURF_EXA_API_KEY=... ``` -------------------------------- ### Install Frankensurf with MCP extra on Linux Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/install.mdx Creates a virtual environment and installs the frankensurf package with the MCP extra. This is the first step in the Linux installation workflow. ```bash python3 -m venv ~/.venvs/frankensurf && source ~/.venvs/frankensurf/bin/activate pip install "frankensurf[mcp]" ``` -------------------------------- ### Install frankensurf package and browsers Source: https://github.com/yail259/frankensurf/blob/main/plugin/README.md Install the frankensurf package with MCP extras and set up free browsers and the local search engine. ```bash pip install "frankensurf[mcp]" # or: uv tool install "frankensurf[mcp]" frankensurf install # the free browsers and the local search engine ``` -------------------------------- ### Install Frankensurf tools with system dependencies on Linux Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/install.mdx Installs every free tool and the system libraries the browsers need. This command asks for sudo and takes a few minutes. ```bash frankensurf install --with-deps ``` -------------------------------- ### Set up self-hosted Steel Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/steel.mdx Starts the self-hosted Steel container and sets the environment variable to point Frankensurf at it. ```bash bash scripts/start-steel.sh export FRANKENSURF_STEEL_URL=http://127.0.0.1:3100 ``` -------------------------------- ### Install Chromium for Playwright Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/local-chromium.mdx Installs the Chromium browser binary required by the `local` provider. Run this in your virtual environment before using the provider. ```bash .venv/bin/playwright install chromium ``` -------------------------------- ### Model factory example for Browser Use Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/browser-use.mdx Example model factory file that creates a ChatOpenAI model wrapped in ModelBinding with billing set to 'paid'. ```python from browser_use import ChatOpenAI from frankensurf.browser_use_binding import ModelBinding def create_model(): return ModelBinding(ChatOpenAI(model="gpt-4.1-mini", max_retries=0), billing="paid") ``` -------------------------------- ### Install Patchright into stealth environment Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/patchright.mdx Install Patchright into the provider virtual environment using pip. This is required before using the provider. ```bash ~/.local/share/frankensurf/provider-venv/bin/pip install patchright ``` -------------------------------- ### Install Frankensurf for LangChain Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/frameworks.mdx Install the package with the LangChain extra using pip. ```bash pip install "frankensurf[langchain]" ``` -------------------------------- ### Example receipt JSON Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/reference/receipts.mdx A trimmed example of a receipt from a real read. It shows the structure and typical fields like trace_id, status, operation, method, observed_at, freshness_seconds, cache_hit, requested_url, final_url, http_status, latency_ms, cost_usd, evidence, attempts, and routing. The routing.provider_plan.ordered lists the planned route after skipping unavailable providers. ```json { "trace_id": "6f1c…", "status": "observed", "operation": "read", "method": "local", "observed_at": "2026-10-06T03:14:07Z", "freshness_seconds": 0, "cache_hit": false, "requested_url": "https://example.com", "final_url": "https://example.com/", "http_status": 200, "latency_ms": 2267, "cost_usd": 0, "evidence": [{"path": "state/evidence/2a8c….html", "sha256": "2a8c…", "bytes": 23}], "attempts": [ {"provider": "http", "status": "failed", "failure": "VISUAL_REQUIRED", "latency_ms": 90}, {"provider": "local", "status": "observed", "latency_ms": 2176} ], "routing": {"provider_plan": { "ordered": ["http", "local", "camoufox", "scrapling", "scrapling_http", "crawl4ai", "jina_reader"], "basis": "registration order; insufficient exact-scope evidence" }} } ``` -------------------------------- ### Install Frankensurf for OpenAI Agents SDK Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/frameworks.mdx Install the package with the openai-agents extra using pip. ```bash pip install "frankensurf[openai-agents]" ``` -------------------------------- ### Install Crawl4AI provider Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/crawl4ai.mdx Installs the Crawl4AI provider. On Linux (or WSL on Windows), not on macOS yet. Creates `~/.local/share/frankensurf/crawl4ai-venv` and ends with a live render check. Can be moved with `FRANKENSURF_CRAWL4AI_DIR`, `FRANKENSURF_CRAWL4AI_PYTHON`, and `FRANKENSURF_CRAWL4AI_BROWSERS_PATH`. ```bash frankensurf install crawl4ai # part of plain `frankensurf install` too ``` -------------------------------- ### Install Camoufox with frankensurf install stealth Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/camoufox.mdx Installs Camoufox and Scrapling in a separate environment (FRANKENSURF_PROVIDER_PYTHON). ```bash frankensurf install stealth ``` -------------------------------- ### Python: Run do with WebPolicy Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/act.mdx Python example that loads the intent JSON and runs it with a WebPolicy configured for the identity, action classes, allowed contracts, origins, and tools. Uses the provider 'browser_use_local_cdp_do'. ```python import json from frankensurf import WebPolicy policy = WebPolicy( identity="shop-me", action_classes=("WRITE_REVERSIBLE", "WRITE_EXTERNAL", "PURCHASE/FINANCIAL", "ACCOUNT_SECURITY"), browser_do_allowed_contracts=("browser.raw_control.v1",), browser_do_allowed_origins=("https://www.example-shop.com",), browser_do_allowed_tools=("fill", "click", "assert_text"), ) result = await web.do(json.load(open("intent.json")), policy, provider="browser_use_local_cdp_do") ``` -------------------------------- ### Install, show, and uninstall the FrankenSurf skill Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/agents.mdx Commands to install the skill to the default location or a custom directory, show its contents, and uninstall it. The skill is automatically sent to MCP clients, but agents that load skills from a folder (like Claude Code) need this installation. ```bash frankensurf skill install # ~/.claude/skills/frankensurf/SKILL.md frankensurf skill install --skill-dir ./my-agent/skills frankensurf skill show frankensurf skill uninstall # remove it again (--skill-dir for another folder) ``` -------------------------------- ### Add and install Frankensurf plugin in Claude Code Source: https://github.com/yail259/frankensurf/blob/main/plugin/README.md Commands to add the plugin marketplace and install the Frankensurf plugin. ```bash /plugin marketplace add yail259/frankensurf /plugin install frankensurf@frankensurf ``` -------------------------------- ### Read a page with Frankensurf on Linux Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/install.mdx Reads a Walmart search page and shows each tool it tried. This is the final step in the Linux installation workflow. ```bash frankensurf read "https://www.walmart.com/search?q=air+fryer" --explain ``` -------------------------------- ### Response from read Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/index.mdx Example JSON response from the read tool, showing the page data and receipt with status, method, latency, and cost. ```json { "url": "https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/", "title": "Markdown for Agents", "content_type": "text/markdown; charset=utf-8", "text": "---\ndescription: Cloudflare's Markdown for Agents converts HTML to Markdown at the edge…", "structured": { "format": "markdown" }, "receipt": { "status": "observed", "method": "http", "latency_ms": 261, "cost_usd": 0.0, "attempts": [{ "provider": "http", "status": "observed", "latency_ms": 261 }] } } ``` -------------------------------- ### Start Chrome with remote debugging and set CDP environment variable Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/blocked/real-browser.mdx Use this to connect FrankenSurf to an already running Chrome browser. Start Chrome with remote debugging on port 9222 and a dedicated profile, then set FRANKENSURF_REAL_BROWSER_CDP to the debugging endpoint. Each read opens a fresh private window and closes it afterwards, so no cookies persist. ```bash google-chrome --remote-debugging-port=9222 --user-data-dir="$HOME/.frankensurf-chrome" export FRANKENSURF_REAL_BROWSER_CDP=http://127.0.0.1:9222 ``` -------------------------------- ### Use Browserless provider in Python read Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/browserless.mdx Set the FRANKENSURF_BROWSERLESS_API_KEY and optionally FRANKENSURF_BROWSERLESS_USD_PER_HOUR. This example shows a read call with provider='browserless' and policy_overrides allowing paid fallbacks. ```python page = await web.read( "https://example.com/products/2231", provider="browserless", policy_overrides={"allow_paid_fallbacks": True}, ) ``` -------------------------------- ### Response with prefer_markdown enabled Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/markdown.mdx Example response from the read tool when prefer_markdown is enabled. Shows the content_type as text/markdown and a structured format of markdown, with a receipt indicating the HTTP method and latency. ```json { "title": "Markdown for Agents", "content_type": "text/markdown; charset=utf-8", "text": "---\ndescription: Cloudflare's Markdown for Agents converts HTML to Markdown at the edge…", "structured": { "format": "markdown" }, "receipt": { "method": "http", "latency_ms": 261 } } ``` -------------------------------- ### Batch read usage in Python, CLI, and MCP Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/batch.mdx Shows how to call the batch tool in Python, CLI, and MCP. The Python example uses the web.batch method with a list of URLs. The CLI command uses the frankensurf batch subcommand. The MCP example uses the batch tool with a urls argument. ```python pages = await web.batch([ "https://example.com/a", "https://example.com/b", "https://another.example/c", ]) ``` ```bash frankensurf batch https://example.com/a https://example.com/b https://another.example/c ``` ```json { "tool": "batch", "arguments": { "urls": ["https://example.com/a", "https://example.com/b"] } } ``` -------------------------------- ### Response (trimmed) Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/search.mdx Example response from a search, showing the query, source, coverage, results with URL, title, snippet, engines, and verification, plus upstream_failures. ```json { "query": "playwright python tutorial", "source": "searxng", "coverage": "partial", "results": [ { "url": "https://www.browserstack.com/guide/playwright-python-tutorial", "title": "The ultimate Playwright Python tutorial | BrowserStack", "snippet": "Learn end-to-end web testing with Playwright Python…", "engines": ["duckduckgo"], "verification": "indexed_discovery" }, { "url": "https://playwright.dev/python/docs/library", "title": "Getting started - Library | Playwright Python", "snippet": "These commands download the Playwright package and install browser binaries…", "engines": ["duckduckgo"], "verification": "indexed_discovery" } ], "upstream_failures": [["brave", "too many requests"], ["google cse", "too many requests"]] } ``` -------------------------------- ### Discover a module from a home page with a query (CLI) Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/site-modules.mdx Start from a home page with a query when you don't know the site's search URL. Frankensurf finds the search itself via schema.org `SearchAction`, the search form, or by typing the query into a visible box. Saves the module with `--save`. ```bash frankensurf module discover https://www.example.com/ --query "desk lamp" --save ``` -------------------------------- ### Deploy to Cloudflare Pages Source: https://github.com/yail259/frankensurf/blob/main/site/README.md Install dependencies with npm ci and deploy the static site to the frankensurf Cloudflare Pages project on its production branch (main). The deploy command builds the site and uploads only dist/. ```bash npm ci npm run deploy ``` -------------------------------- ### Response example for read Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/index.mdx Shows the structure of the response object returned by `read`, including the receipt with trace ID, status, method, HTTP status, latency, cost, and attempts. ```json { "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP", "title": "HTTP: Hypertext Transfer Protocol | MDN", "text": "HTTP: Hypertext Transfer Protocol | MDN Skip to main content …", "content_type": "text/html", "structured": { "jsonld": [], "embedded_json": [] }, "image_urls": [], "receipt": { "trace_id": "7700792749c345eeb2060f3a4edfb6e2", "status": "observed", "method": "http", "http_status": 200, "latency_ms": 250, "cost_usd": 0.0, "attempts": [{ "provider": "http", "status": "observed", "latency_ms": 249 }] } } ``` -------------------------------- ### Use Browserless provider in MCP read Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/browserless.mdx Set the FRANKENSURF_BROWSERLESS_API_KEY and optionally FRANKENSURF_BROWSERLESS_USD_PER_HOUR. This example shows an MCP tool call with provider='browserless' and acquisition_policy allowing paid fallbacks. ```json { "tool": "read", "arguments": { "url": "https://example.com/products/2231", "provider": "browserless", "acquisition_policy": { "allow_paid_fallbacks": true } } } ``` -------------------------------- ### Example site module JSON structure Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/site-modules.mdx Shows the complete JSON schema for a site module, including match, templates, sources, items, assertions, and notes. The module matches a search page, extracts embedded JSON hits, and asserts minimum item counts. ```json { "id": "example.search", "version": "1", "match": { "origin": "https://www.example.com", "path_pattern": "/search" }, "templates": { "search": { "url": "https://www.example.com/search?q={query}", "params": { "query": "required" } } }, "sources": { "hits": { "kind": "embedded_json", "decode": "next_flight", "marker": "\"hits\":" } }, "items": { "from": "hits", "fields": { "title": "ObjectTitle", "url": { "path": "ITEM_URL", "type": "url" }, "price": { "path": "ObjectPrice", "type": "number" } } }, "assertions": { "schema": "frankensurf.workload-assertions/v1", "checks": [ { "path": "items", "operator": "minimum_count", "value": 5 }, { "path": "matching_query", "operator": "minimum_count", "value": 3 } ] }, "notes": "Search takes q; searchTerm is silently ignored." } ``` -------------------------------- ### Initialize key and print public directory Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/blocked/signed-requests.mdx Run these commands to create a key and output its public directory to a file named directory.json. The private key never leaves your machine. ```bash frankensurf bot-auth-init frankensurf bot-auth-directory > directory.json ``` -------------------------------- ### Search receipt JSON example Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/search.mdx Shows the JSON structure of the search receipt, including search_ladder, search_checks, and search_attempts fields. It illustrates the ladder configuration, check verdict, and per-source attempt details. ```json { "search_ladder": {"enabled": true, "query_type": "news|recent|open", "order": ["bing_news_rss", "searxng", "duckduckgo_html"], "first_rung": ["bing_news_rss", "searxng"], "winner": "merged", "stopped": "good", "hedged": ["bing_news_rss", "searxng"]}, "search_checks": {"verdict": "good", "need": 3, "passed": 3, "reasons": {"off_query": 2, "out_of_window": 1}, "unknown_dates": 1, "results": [{"url": "https://example.com/2026/10/11/story", "source": "bing_news_rss", "in_window": true, "on_query": true, "published": "2026-10-11T23:59:59Z", "date_source": "url", "kind": "item"}]}, "search_attempts": [{"source": "bing_news_rss", "status": "observed", "count": 10, "passed": 2, "check": "weak", "cost_usd": 0, "latency_ms": 412}] } ``` -------------------------------- ### Trust and use the plugin Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/more/plugins.mdx Commands to trust the plugin, list plugins, set environment variables, and read a URL with the provider. ```bash frankensurf plugins trust provider example_fetch frankensurf-plugin-example frankensurf plugins # now under "loaded" export EXAMPLE_API_URL=https://your-service.example/read EXAMPLE_API_KEY=... frankensurf read https://example.com --provider example_fetch --paid ``` -------------------------------- ### Install Frankensurf for CrewAI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/frameworks.mdx Install the package with the crewai extra using pip. ```bash pip install "frankensurf[crewai]" ``` -------------------------------- ### Entry point declaration in pyproject.toml Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/more/plugins.mdx Declares the plugin's entry point under the provider group, mapping the plugin ID to the ExampleFetch class. Includes project metadata and dependencies. ```toml [project] name = "frankensurf-plugin-example" version = "0.1.0" dependencies = ["frankensurf>=0.34", "httpx>=0.28,<1"] [project.entry-points."frankensurf.plugins.provider"] example_fetch = "frankensurf_plugin_example:ExampleFetch" ``` -------------------------------- ### Runtime constructor Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/reference/python.mdx Creates a Runtime instance that owns the state directory and configuration for cache, evidence, traces, route memory, and pacing. It is opened with `async with`. ```APIDOC ## Runtime(state_dir, steel_api_url, local_cdp_url, concurrency, per_domain, domain_delay, identity_registry, plugin_config_path) ### Description Creates a Runtime instance that owns the state directory and configuration for cache, evidence, traces, route memory, and pacing. It is opened with `async with`. ### Parameters - **state_dir** (string) - Optional - Path to the state directory for cache, evidence, traces, route memory and pacing. - **steel_api_url** (string) - Optional - URL for a self-hosted Steel, or use FRANKENSURF_STEEL_URL environment variable. - **local_cdp_url** (string) - Optional - Used only through a registered identity. - **concurrency** (integer) - Optional - Number of pages at once in a batch. Default: 4. - **per_domain** (integer) - Optional - Number of pages at once per site in a batch. Default: 1. - **domain_delay** (number) - Optional - Seconds between batch requests to one site. Default: 0.25. - **identity_registry** (object) - Optional - Identity registry, or use FRANKENSURF_IDENTITIES environment variable. - **plugin_config_path** (string) - Optional - Path to plugin configuration, or use ~/.config/frankensurf/plugins.json. ### Example ```python Runtime( state_dir="state", steel_api_url=None, local_cdp_url=None, concurrency=4, per_domain=1, domain_delay=0.25, identity_registry=None, plugin_config_path=None, ) ``` ``` -------------------------------- ### Use Frankensurf from Python Source: https://github.com/yail259/frankensurf/blob/main/README.md Shows how to use the Frankensurf Runtime in Python to read a page and perform a search. ```python from frankensurf import Runtime async with Runtime("state") as web: page = await web.read("https://example.com", policy_overrides={"prefer_markdown": True}) hits = await web.search("playwright python tutorial") ``` -------------------------------- ### receipt.attempts: automatic escalation example Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/javascript.mdx Example of the receipt.attempts output showing automatic escalation from HTTP to a local browser when plain HTTP fails with VISUAL_REQUIRED. ```json [ { "provider": "http", "status": "failed", "failure": "VISUAL_REQUIRED", "latency_ms": 88 }, { "provider": "local", "status": "observed", "latency_ms": 1310 } ] ``` -------------------------------- ### Provider implementation with ExampleFetch class Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/more/plugins.mdx Defines a provider plugin with a manifest and an acquire method that calls an HTTP API. It checks for an environment variable, handles a 403 response, and returns page data. The manifest declares rendering, paid, and remote capabilities. ```python import os import httpx from frankensurf.providers import ProviderManifest from frankensurf.runtime import WebFailure class ExampleFetch: manifest = ProviderManifest( "example_fetch", "0.1.0", # must match the entry point name rendering=True, # the service runs a browser paid=True, # only with allow_paid_fallbacks / --paid remote=True, # fetches from its own network ) def available(self, configured): # Unavailable means skipped, never failed. return bool(os.environ.get("EXAMPLE_API_KEY")) async def acquire(self, request, services): if not self.available(None): # named explicitly, it runs anyway raise WebFailure("PROVIDER_UNAVAILABLE", "set EXAMPLE_API_KEY first") async with httpx.AsyncClient(timeout=request.policy.unblocker_timeout_seconds) as client: response = await client.post(os.environ["EXAMPLE_API_URL"], json={"url": request.url}, headers={"Authorization": "Bearer " + os.environ["EXAMPLE_API_KEY"]}) body = response.json() if body["status"] == 403: raise WebFailure("BLOCKED", "page answered 403", 403) return {"url": body["url"], "content": body["html"], "content_type": "text/html", "http_status": body["status"], "cost_usd": 0.001} ``` -------------------------------- ### Use Frankensurf from the command line Source: https://github.com/yail259/frankensurf/blob/main/README.md Shows command-line usage for reading, searching, and watching pages. ```bash frankensurf read https://example.com --markdown frankensurf search playwright python tutorial --recency week frankensurf watch https://news.ycombinator.com/ --link-pattern 'item\?id=\d+' ``` -------------------------------- ### Install nodriver 0.47.0 with pip Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/nodriver.mdx Installs the nodriver package version 0.47.0 into the Frankensurf provider virtual environment. Later versions are not recommended because they ship a file Python 3.14 refuses to import. ```bash ~/.local/share/frankensurf/provider-venv/bin/pip install "nodriver==0.47.0" ``` -------------------------------- ### Read with auto_items to get items and auto_module Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/site-modules.mdx Pass `auto_items` (MCP: `items=True`) on any read to get `items` from the page's own data, plus `auto_module` in the receipt. Save it with `site_modules` `put` to shape future reads. ```python page = await web.read(url, policy_overrides={"auto_items": True}) page["items"][:3]; page["receipt"]["auto_items"] # {"found", "count", "source"} ``` -------------------------------- ### Use HTTP provider with CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/http.mdx Use the HTTP provider from the command line with the --provider http flag. ```bash frankensurf read https://example.com/products/2231 --provider http ``` -------------------------------- ### Sign in to a profile with frankensurf profile-login Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/profiles.mdx Run this command to open a Frankensurf browser window for signing in to a site. Sign in normally (including 2FA), then close the window; Frankensurf never types or clicks. ```bash frankensurf profile-login shopping --site example-shop.com ``` -------------------------------- ### Use Scrapling provider with CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/scrapling.mdx Use the 'scrapling' provider from the command line. ```bash frankensurf read https://example.com/products/2231 --provider scrapling ``` -------------------------------- ### Use Patchright provider with CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/patchright.mdx Use the Patchright provider from the command line with the --provider patchright flag. ```bash frankensurf read https://example.com/products/2231 --provider patchright ``` -------------------------------- ### web.search with merge mode Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/search.mdx Example of using the web.search method with mode='merge' to query multiple sources at once. ```APIDOC ## web.search ### Description Performs a web search with configurable options. When `mode="merge"`, it queries every eligible source in the vertical simultaneously (up to `search_merge_sources`, default 4) and fuses their results by reciprocal rank. Duplicate pages are merged into a single result listing all engines that found it, with `fusion.sources` providing each engine's rank. ### Method `search` ### Parameters - `query` (string) - Required - The search query. - `mode` (string) - Optional - Set to `"merge"` to query all eligible sources at once. Default is `"fallback"`. - `source` (string) - Optional - Restrict to a single source by name. - `limit` (integer) - Optional - Number of results. Default is `10`. - `site` (string) - Optional - Restrict results to a specific site. - `exclude_domains` (string) - Optional - Exclude results from specified domains. - `recency` (string) - Optional - Filter results by recency. - `region` (string) - Optional - Filter results by region. - `vertical` (string) - Optional - The kind of source. Inferred from the query if not provided; explicit value always wins. - `search_source_prefer` (list) - Optional - Try these sources first. - `search_source_allow` (list) - Optional - Only these sources may run. Default is all. - `search_source_timeout_seconds` (integer) - Optional - Time per source in seconds. Default is `8`. - `allow_paid_fallbacks` (boolean) - Optional - Allow paid sources to join. Default is `false`. ### Request Example ```python results = await web.search("film camera repair sydney", mode="merge") ``` ### Response Returns a list of search results. Each result may include `fusion.sources` indicating the engines that found it and their ranks. The `receipt.search_merge` field reports each source used. ``` -------------------------------- ### Set environment variables for Browser Use Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/browser-use.mdx Exports the model factory path and billing mode for Frankensurf to use the Browser Use provider. ```bash export FRANKENSURF_BROWSER_USE_MODEL_FACTORY=/home/you/models/agent.py:create_model export FRANKENSURF_BROWSER_USE_BILLING=paid ``` -------------------------------- ### Runtime constructor with parameters Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/reference/python.mdx Instantiate Runtime with optional parameters for state directory, API URLs, concurrency, delays, identity registry, and plugin config. Environment variables can be used as alternatives. ```python Runtime( state_dir="state", # cache, evidence, traces, route memory steel_api_url=None, # a self-hosted Steel, or FRANKENSURF_STEEL_URL local_cdp_url=None, # used only through a registered identity concurrency=4, # pages at once in a batch per_domain=1, # pages at once per site in a batch domain_delay=0.25, # seconds between batch requests to one site identity_registry=None, # or FRANKENSURF_IDENTITIES plugin_config_path=None, # or ~/.config/frankensurf/plugins.json ) ``` -------------------------------- ### Read main content with CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/main-content.mdx Use the frankensurf read command with the --main flag to get the main content of a page. ```bash frankensurf read https://news.example.com/story --main ``` -------------------------------- ### CLI: Perform a search with limit Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/search.mdx Use the CLI to search for 'playwright python tutorial' with a limit of 10 results. ```bash frankensurf search playwright python tutorial --limit 10 ``` -------------------------------- ### Rejected plugin output from frankensurf plugins Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/more/plugins.mdx Shows the JSON output when the plugin is installed but not yet trusted, including the hint command to trust it. ```json "rejected": [{"kind": "provider", "id": "example_fetch", "code": "PLUGIN_NOT_TRUSTED", "distribution": "frankensurf-plugin-example", "hint": "installed but not trusted yet: frankensurf plugins trust provider example_fetch frankensurf-plugin-example"}] ``` -------------------------------- ### Search with --source searxng in CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/searxng.mdx Use the CLI to search with the SearXNG source. Named sources never fall back to another. ```bash frankensurf search playwright python tutorial --source searxng ``` -------------------------------- ### Use local provider with CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/local-chromium.mdx Reads a URL using the `local` provider from the command line. The `--provider local` flag selects the headless Chromium browser. ```bash frankensurf read https://example.com/products/2231 --provider local ``` -------------------------------- ### batch_template(module, operation, params) Source: https://github.com/yail259/frankensurf/blob/main/plugin/skills/frankensurf/SKILL.md Runs a batch of operations using a saved site module. For example, many searches on one site. ```APIDOC ## batch_template(module, operation, params) ### Description Runs a batch of operations using a saved site module. For example, many searches on one site. ### Parameters - **module** (object) - Required - The module object obtained from `discover_site_module`. - **operation** (string) - Required - The operation to perform, e.g., `"search"`. - **params** (array) - Required - List of parameter objects, each containing the operation's parameters. ### Response Returns a list of results for each operation. ### Example ``` batch_template(module, "search", [{"query": "term1"}, {"query": "term2"}]) ``` ``` -------------------------------- ### Use Steel provider with CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/steel.mdx Uses the Steel provider with the Frankensurf CLI to read a URL. ```bash frankensurf read https://example.com/products/2231 --provider steel ``` -------------------------------- ### CLI: module add, list, read-template, read, disable Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/site-modules.mdx Shows CLI commands for managing modules: adding a module file, listing modules, reading a template with parameters, reading a URL with modules turned off (`--module none`), and disabling a module. ```bash frankensurf module add example-search.json frankensurf module list frankensurf read-template example.search search --param query="desk lamp" frankensurf read https://www.example.com/search?q=lamp --module none frankensurf module disable example.search ``` -------------------------------- ### Response (trimmed) Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/page-data.mdx Example response showing the `structured` and `captured_json` fields. The `structured.jsonld` contains JSON-LD blocks, and `captured_json.items` contains the captured JSON responses. ```json { "structured": { "jsonld": [{ "@type": "Product", "name": "…", "offers": { "price": "…" } }], "embedded_json": [] }, "captured_json": { "items": [{ "url": "https://shop.example.com/api/product/2231", "http_status": 200, "content_type": "application/json", "format": "json", "data": { "title": "Widget", "price": 42 } }], "skipped": 0 } } ``` -------------------------------- ### Search with Serper using Python Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/serper.mdx Use the WebPolicy with allow_paid_fallbacks=True to enable paid sources like Serper. The source is specified by name, and named sources never fall back to another. ```python from frankensurf import WebPolicy results = await web.search("rio tinto acquisition", source="serper", policy=WebPolicy(allow_paid_fallbacks=True)) ``` -------------------------------- ### Response for a PDF read Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/pdfs.mdx Example response from the read tool for a PDF. It includes the URL, extracted text, content type, structured metadata, and receipt with latency. ```json { "url": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf", "text": "Dummy PDF file", "content_type": "application/pdf; qs=0.001", "structured": { "format": "pdf", "pages": 1 }, "receipt": { "method": "http", "latency_ms": 414 } } ``` -------------------------------- ### Use Browser Use provider with CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/browser-use.mdx Reads a URL using the browser_use provider via the command line. ```bash frankensurf read https://example.com/products/2231 --provider browser_use ``` -------------------------------- ### Use Scrapfly provider in MCP Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/scrapfly.mdx MCP JSON request to the read tool with provider='scrapfly' and acquisition_policy allowing paid fallbacks. Requires the same environment variables as the Python example. ```json { "tool": "read", "arguments": { "url": "https://example.com/products/2231", "provider": "scrapfly", "acquisition_policy": { "allow_paid_fallbacks": true } } } ``` -------------------------------- ### Python: Use real browser with policy_overrides and provider Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/blocked/real-browser.mdx Shows two ways to use the real browser in Python: with policy_overrides to allow it when walls block, and directly with provider='real_browser'. ```python # Try your real browser when walls keep blocking page = await web.read(url, policy_overrides={"allow_real_browser": True}) # Straight to your real browser page = await web.read(url, provider="real_browser") ``` -------------------------------- ### Read page data with Python Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/page-data.mdx Use the `web.read` method with `render` and `capture_json_responses` policy overrides to capture JSON responses. The example then extracts the `data` field from each captured item. ```python page = await web.read("https://shop.example.com/p/2231", policy_overrides={ "render": True, "capture_json_responses": True, }) offers = [item["data"] for item in page.get("captured_json", {}).get("items", [])] ``` -------------------------------- ### Enable Lightning mode via CLI Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/lightning.mdx Use the --lightning flag with the frankensurf read command to enable Lightning mode for a single read. ```bash frankensurf read "https://www.bandcamp.com/search?q=jazz" --lightning ``` -------------------------------- ### Read main content with Python Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/read/main-content.mdx Use the read method with the main_content policy override to get the article text. The result includes main_text with the article and a receipt with main_content details. ```python page = await web.read(url, policy_overrides={"main_content": True}) page["main_text"] # the article page["receipt"]["main_content"] # {"method", "chars", "of_chars"} ``` -------------------------------- ### Example result object Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/reference/python.mdx Shows the structure of a result returned by read-like methods, including URL, title, text, content, headers, structured data, images, field status, and receipt. ```python { "url": "https://example.com/", "title": "Example Domain", "text": "…", "content": "…", # raw body "content_type": "text/html", "headers": {…}, # never cookies "structured": {"jsonld": [], "embedded_json": []}, "image_urls": […], "images": […], # with include_images=True "field_status": {"availability": "unknown", "transaction_price": "unknown", "content": "observed"}, "receipt": {…}, } ``` -------------------------------- ### Response JSON for watch Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/watch.mdx Example response from a `watch` poll. Shows the `first_poll` flag, the list of new links, and the receipt status. The `seen_count` and `links_on_page` indicate the number of links remembered and present. ```json { "url": "https://news.ycombinator.com/", "status": "observed", "first_poll": true, "new": [ { "url": "https://news.ycombinator.com/item?id=49969183", "text": "4 hours ago" }, { "url": "https://news.ycombinator.com/item?id=49971230", "text": "1 hour ago" } ], "links_on_page": 30, "seen_count": 30, "receipt": { "status": "observed", "method": "http" } } ``` -------------------------------- ### Search Google News RSS with Python Source: https://github.com/yail259/frankensurf/blob/main/site/src/content/docs/docs/integrations/google-news.mdx Use the web.search method with the source parameter set to 'google_news_rss' to query Google News. The example searches for the quoted phrase "Rio Tinto" acquisition. ```python results = await web.search("\"Rio Tinto\" acquisition", source="google_news_rss") ```