### Start Maxun Frontend and Backend Manually Source: https://github.com/getmaxun/maxun/blob/develop/SETUP.md Run this command from the project root to start both the frontend and backend services after manual installation. Ensure all dependencies are installed. ```bash npm run start ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/getmaxun/maxun/blob/develop/SETUP.md Install the necessary Node.js dependencies for the root of the Maxun project. ```bash npm install ``` -------------------------------- ### Install Maxun Core Dependencies Source: https://github.com/getmaxun/maxun/blob/develop/SETUP.md Navigate into the maxun-core directory and install its specific Node.js dependencies. ```bash cd maxun-core npm install ``` -------------------------------- ### Start Maxun with Docker Compose Source: https://github.com/getmaxun/maxun/blob/develop/SETUP.md Use this command to start the Maxun backend and frontend services when using Docker Compose. Ensure your .env file is correctly configured. ```bash docker-compose up -d ``` -------------------------------- ### Start a run for a robot Source: https://context7.com/getmaxun/maxun/llms.txt Starts and queues a robot execution, returning the run ID and browser ID. ```APIDOC ## PUT /storage/runs/:id — Start a run for a robot ### Description Creates and queues a robot execution. Returns `runId` and `browserId`. ### Method PUT ### Endpoint /storage/runs/:id ### Parameters #### Path Parameters - **id** (string) - Required - The ID of the robot to run. ### Request Body - `{}` - An empty JSON object is expected. ### Request Example ```bash curl -s -b cookies.txt -X PUT "http://localhost:8080/storage/runs/${ROBOT_ID}" \ -H "Content-Type: application/json" \ -d '{}' | jq . ``` ### Response #### Success Response (200) - **browserId** (string) - The ID of the browser instance used for the run. - **runId** (string) - The unique ID for this run. - **robotMetaId** (string) - The ID of the robot being run. - **queued** (boolean) - Indicates if the run has been queued. ### Response Example ```json { "browserId": "uuid", "runId": "uuid", "robotMetaId": "uuid", "queued": false } ``` ``` -------------------------------- ### Clone Maxun Repository Source: https://github.com/getmaxun/maxun/blob/develop/SETUP.md Clone the Maxun project from GitHub. This is the first step for manual installation. ```bash git clone https://github.com/getmaxun/maxun ``` -------------------------------- ### Install Playwright with Dependencies Source: https://github.com/getmaxun/maxun/blob/develop/SETUP.md Install the Playwright browser automation tool and its system dependencies. This is required for running tests or specific features. ```bash npx playwright install --with-deps chromium ``` -------------------------------- ### Start Recording Session Source: https://context7.com/getmaxun/maxun/llms.txt Initiates a new browser recording session. Requires cookies to be managed. The response is a browser UUID string. ```bash curl -s -b cookies.txt http://localhost:8080/record/start # Response: "browser-uuid-string" ``` -------------------------------- ### Start a recording session Source: https://context7.com/getmaxun/maxun/llms.txt Initiates a new browser recording session. The response is a unique browser ID. ```APIDOC ## GET /record/start — Start a recording session ### Description Starts a new browser recording session. ### Method GET ### Endpoint /record/start ### Response #### Success Response (200) - **browser-uuid-string** (string) - The unique identifier for the browser session. ### Response Example ``` "browser-uuid-string" ``` ``` -------------------------------- ### Initiate Airtable OAuth (PKCE flow) Source: https://context7.com/getmaxun/maxun/llms.txt Starts the OAuth 2.0 PKCE flow for authenticating a robot with Airtable. ```APIDOC ## GET /auth/airtable?robotId=:id — Initiate Airtable OAuth (PKCE flow) ### Description Initiates the Airtable OAuth 2.0 authentication process using the PKCE flow for a specified robot. ### Method GET ### Endpoint /auth/airtable ### Parameters #### Query Parameters - **robotId** (string) - Required - The ID of the robot to authenticate. ### Response #### Success Response (200) Redirects the user to the Airtable authorization page. The PKCE verifier is stored in the session. ### Request Example ```bash open "http://localhost:8080/auth/airtable?robotId=${ROBOT_ID}" ``` ``` -------------------------------- ### Initiate Airtable OAuth (PKCE Flow) Source: https://context7.com/getmaxun/maxun/llms.txt Starts the Airtable OAuth process using the PKCE flow for a robot. This opens a browser to the Airtable authorization page. ```bash open "http://localhost:8080/auth/airtable?robotId=${ROBOT_ID}" # Redirects to Airtable authorization page; PKCE verifier stored in session. ``` -------------------------------- ### Initiate Google OAuth for Robot Source: https://context7.com/getmaxun/maxun/llms.txt Starts the Google OAuth flow for a robot. This opens a browser to the Google consent screen; the server handles the redirect. ```bash # Opens browser to Google consent screen; redirect handled server-side. open "http://localhost:8080/auth/google?robotId=${ROBOT_ID}" ``` -------------------------------- ### Execute a Web Scraping Workflow with maxun-core Interpreter Source: https://context7.com/getmaxun/maxun/llms.txt This TypeScript example demonstrates initializing and running the maxun-core Interpreter. It requires Playwright and defines a workflow to scrape data from Hacker News. ```typescript import Interpreter from 'maxun-core'; import { chromium } from 'playwright-core'; const browser = await chromium.launch({ headless: true }); const page = await browser.newPage(); const workflowFile = { workflow: [ { where: { url: 'about:blank' }, what: [ { action: 'goto', args: ['https://news.ycombinator.com'] }, { action: 'waitForLoadState', args: ['networkidle'] } ] }, { where: { url: 'https://news.ycombinator.com' }, what: [ { action: 'scrapeList', args: [{ listSelector: '.athing', fields: { title: { selector: '.titleline > a', tag: 'a', attribute: 'innerText' }, link: { selector: '.titleline > a', tag: 'a', attribute: 'href' } }, limit: 10, pagination: null }] } ] } ] }; const interpreter = new Interpreter(workflowFile, { maxRepeats: 1, maxConcurrency: 1, debug: false, serializableCallback: (data) => { console.log('Scraped data:', JSON.stringify(data, null, 2)); // Output: [{ title: "Show HN: ...", link: "https://..." }, ...] }, binaryCallback: async (data, mimeType) => { console.log(`Binary output (${mimeType}), size: ${data.length}`); }, debugChannel: { progressUpdate: (current, total, pct) => process.stdout.write(`\r${pct}%`), } }); await interpreter.run(page, null); await browser.close(); ``` -------------------------------- ### Initiate Google OAuth for a robot Source: https://context7.com/getmaxun/maxun/llms.txt Starts the OAuth 2.0 flow for authenticating a robot with Google services. ```APIDOC ## GET /auth/google?robotId=:id — Initiate Google OAuth for a robot ### Description Initiates the Google OAuth 2.0 authentication process for a specified robot. ### Method GET ### Endpoint /auth/google ### Parameters #### Query Parameters - **robotId** (string) - Required - The ID of the robot to authenticate. ### Response #### Success Response (200) Opens a browser to the Google consent screen. The server handles the redirect. ### Request Example ```bash open "http://localhost:8080/auth/google?robotId=${ROBOT_ID}" ``` ``` -------------------------------- ### Start Robot Run API Source: https://context7.com/getmaxun/maxun/llms.txt Initiate a robot execution by providing the robot ID. This endpoint creates and queues the job, returning a `runId` and `browserId`. The request body can be empty. ```bash curl -s -b cookies.txt -X PUT "http://localhost:8080/storage/runs/${ROBOT_ID}" \ -H "Content-Type: application/json" \ -d '{}' | jq . ``` -------------------------------- ### Docker Compose Configuration for Maxun Source: https://github.com/getmaxun/maxun/blob/develop/docs/self-hosting-docker.md Defines the services required for Maxun, including databases, storage, backend, and frontend. This setup is production-ready for local access and requires a reverse proxy for external access. ```yaml services: postgres: image: postgres:17 container_name: maxun-postgres mem_limit: 512M environment: POSTGRES_USER: ${DB_USER} POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_DB: ${DB_NAME} volumes: - /home/$USER/Docker/maxun/db:/var/lib/postgresql/data healthcheck: test: ["CMD-SHELL", "pg_isready -U postgres"] interval: 10s timeout: 5s retries: 5 redis: image: docker.io/library/redis:7 container_name: maxun-redis restart: always mem_limit: 128M volumes: - /home/$USER/Docker/maxun/redis:/data minio: image: minio/minio container_name: maxun-minio mem_limit: 512M environment: MINIO_ROOT_USER: ${MINIO_ACCESS_KEY} MINIO_ROOT_PASSWORD: ${MINIO_SECRET_KEY} command: server /data --console-address :${MINIO_CONSOLE_PORT:-9001} volumes: - /home/$USER/Docker/maxun/minio:/data backend: image: getmaxun/maxun-backend:latest container_name: maxun-backend ports: - "127.0.0.1:${BACKEND_PORT:-8080}:${BACKEND_PORT:-8080}" env_file: .env environment: BACKEND_URL: ${BACKEND_URL} PLAYWRIGHT_BROWSERS_PATH: /ms-playwright PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 0 # DEBUG: pw:api # PWDEBUG: 1 # Enables debugging CHROMIUM_FLAGS: '--disable-gpu --no-sandbox --headless=new' security_opt: - seccomp=unconfined # This might help with browser sandbox issues shm_size: '2gb' mem_limit: 4g depends_on: - postgres - minio volumes: - /var/run/dbus:/var/run/dbus frontend: image: getmaxun/maxun-frontend:latest container_name: maxun-frontend mem_limit: 512M ports: - "127.0.0.1:${FRONTEND_PORT:-5173}:5173" env_file: .env environment: PUBLIC_URL: ${PUBLIC_URL} BACKEND_URL: ${BACKEND_URL} depends_on: - backend ``` -------------------------------- ### Get schedule details Source: https://context7.com/getmaxun/maxun/llms.txt Retrieves the schedule details for a specific robot. ```APIDOC ## GET /storage/schedule/:id — Get schedule details ### Description Retrieves the schedule details for a specific robot. ### Method GET ### Endpoint /storage/schedule/:id ### Parameters #### Path Parameters - **id** (string) - Required - The ID of the robot whose schedule details are to be retrieved. ### Response #### Success Response (200) - **schedule** (object) - An object containing the schedule details. - **runEvery** (integer) - The frequency interval. - **runEveryUnit** (string) - The unit for the frequency interval. - **cronExpression** (string) - The generated cron expression for the schedule. - **nextRunAt** (string) - The timestamp for the next scheduled run. ### Response Example ```json { "runEvery": 6, "runEveryUnit": "HOURS", "cronExpression": "0 */6 * * *", "nextRunAt": "2025-01-01T14:00:00.000Z", ... } ``` ``` -------------------------------- ### List All Runs API Source: https://context7.com/getmaxun/maxun/llms.txt Retrieve a list of all robot runs. The example shows how to extract the `runId`, `status`, `name`, and `startedAt` from the first run in the list. ```bash curl -s -b cookies.txt http://localhost:8080/storage/runs | jq '.[0] | {runId, status, name, startedAt}' ``` -------------------------------- ### Create Crawl Robot API Source: https://context7.com/getmaxun/maxun/llms.txt Create a crawl robot to traverse a website and follow internal links. Configure the starting URL, robot name, crawl settings (selector, depth, limit, URL filter), and output formats. ```bash curl -s -b cookies.txt -X POST http://localhost:8080/storage/recordings/crawl \ -H "Content-Type: application/json" \ -d '{ "url": "https://blog.example.com", "name": "Blog Crawler", "crawlConfig": { "selector": "a[href]", "maxDepth": 3, "limit": 50, "urlFilter": "blog.example.com" }, "formats": ["markdown", "links"] }' | jq '.robot.recording_meta.type' ``` -------------------------------- ### Get a robot by ID Source: https://context7.com/getmaxun/maxun/llms.txt Fetches the details of a specific robot using its unique ID. The `ROBOT_META_ID` variable should be set to the robot's ID. ```bash curl -s -H "x-api-key: ${API_KEY}" "http://localhost:8080/api/sdk/robots/${ROBOT_META_ID}" | jq '.data.recording_meta' ``` -------------------------------- ### Define a Maxun WorkflowFile Structure Source: https://context7.com/getmaxun/maxun/llms.txt This TypeScript code defines the structure for a `WorkflowFile` used by maxun-core. It includes type imports and an example demonstrating workflow steps, actions like 'goto' and 'scrapeList', and pagination configuration. ```typescript import type { WorkflowFile, WhereWhatPair, Where, What } from 'maxun-core'; // Full type reference: const exampleWorkflow: WorkflowFile = { meta: { name: "Example Robot", desc: "Extracts items from a list page" }, workflow: [ // Step 1: Entry point — always starts from about:blank { where: { url: 'about:blank' }, what: [ { action: 'goto', args: ['https://shop.example.com/products'] }, { action: 'waitForLoadState', args: ['networkidle'] } ] }, // Step 2: Execute scraping on the target page { where: { url: 'https://shop.example.com/products' }, what: [ { action: 'scrapeList', args: [{ listSelector: '.product-card', fields: { name: { selector: 'h2.name', tag: 'h2', attribute: 'innerText' }, price: { selector: 'span.price', tag: 'span', attribute: 'innerText' }, image: { selector: 'img', tag: 'img', attribute: 'src' } }, limit: 50, pagination: { type: 'button', selector: 'button.next-page' } }] }, { // Capture a screenshot after scraping action: 'screenshot', args: [{ fullPage: false }] } ] } ] }; // Available 'action' values in What.action: // Playwright Page methods: goto, click, fill, type, press, hover, waitForSelector, waitForLoadState, evaluate, ... // Custom Maxun functions: // scrape - scrape text/attributes via a CSS selector // scrapeSchema - scrape a structured schema of key/selector mappings // scrapeList - scrape a list of items with field definitions and optional pagination // scrapeListAuto - auto-detect list items from a selector // screenshot - capture a screenshot (visible or fullpage) // scroll - scroll the page (up/down) // crawl - crawl pages following links from a start URL // search - run a web search query // enqueueLinks - add discovered links to the crawl queue // flag - internal metadata flag (e.g. 'generated') // script - execute an arbitrary JavaScript snippet in the page context ``` -------------------------------- ### Get Schedule Details API Source: https://context7.com/getmaxun/maxun/llms.txt Retrieve the current schedule details for a robot using its `ROBOT_ID`. The response includes scheduling parameters and the calculated `cronExpression` and `nextRunAt` time. ```bash curl -s -b cookies.txt "http://localhost:8080/storage/schedule/${ROBOT_ID}" | jq .schedule ``` -------------------------------- ### Set Recurring Schedule API Source: https://context7.com/getmaxun/maxun/llms.txt Configure a recurring schedule for a robot using its `ROBOT_ID`. Specify the frequency (`runEvery`, `runEveryUnit`), start day (`startFrom`), time window (`atTimeStart`, `atTimeEnd`), and timezone. ```bash curl -s -b cookies.txt -X PUT "http://localhost:8080/storage/schedule/${ROBOT_ID}" \ -H "Content-Type: application/json" \ -d '{ "runEvery": 6, "runEveryUnit": "HOURS", "startFrom": "MONDAY", "atTimeStart": "08:00", "atTimeEnd": "20:00", "timezone": "America/New_York" }' | jq '.message' ``` -------------------------------- ### Navigate to Project Directory Source: https://github.com/getmaxun/maxun/blob/develop/SETUP.md Change the current directory to the root of the cloned Maxun project. ```bash cd maxun ``` -------------------------------- ### Create Search Robot via SDK Source: https://context7.com/getmaxun/maxun/llms.txt Use this endpoint to create a new search robot. Specify the robot's name, search configuration (query, mode, limit), and desired output formats. ```bash curl -s -X POST http://localhost:8080/api/sdk/search \ -H "x-api-key: ${API_KEY}" \ -H "Content-Type: application/json" \ -d '{ "name": "Competitor Research", "searchConfig": { "query": "best project management tools 2025", "mode": "scrape", "limit": 5 }, "formats": ["markdown", "text"] }' | jq '.data.recording_meta.id' ``` -------------------------------- ### Get runs for a specific robot Source: https://context7.com/getmaxun/maxun/llms.txt Retrieves the runs associated with a particular robot. ```APIDOC ## GET /storage/recordings/:id/runs — Get runs for a specific robot ### Description Gets all runs associated with a specific robot. ### Method GET ### Endpoint /storage/recordings/:id/runs ### Parameters #### Path Parameters - **id** (string) - Required - The ID of the robot. ### Response #### Success Response (200) - **runs.totalCount** (integer) - The total number of runs for the specified robot. ### Response Example ```bash curl -s -b cookies.txt "http://localhost:8080/storage/recordings/${ROBOT_ID}/runs" | jq '.runs.totalCount' ``` ``` -------------------------------- ### Get current workflow Source: https://context7.com/getmaxun/maxun/llms.txt Retrieves the current workflow associated with a specific browser session. ```APIDOC ## GET /workflow/:browserId — Get current workflow ### Description Retrieves the current workflow for a given browser session. ### Method GET ### Endpoint /workflow/:browserId ### Parameters #### Path Parameters - **browserId** (string) - Required - The ID of the browser session. ### Response #### Success Response (200) - **workflow** (array) - An array representing the workflow steps. ### Response Example ```json { "workflow": [ { "where": { "url": "https://example.com/page" }, "what": [{ "action": "screenshot", "args": [] }] } ] } ``` ``` -------------------------------- ### Get a robot by ID Source: https://context7.com/getmaxun/maxun/llms.txt Retrieves details of a specific robot using its unique ID. ```APIDOC ## GET /api/sdk/robots/:id — Get a robot by ID ### Description Retrieves details of a specific robot using its unique ID. ### Method GET ### Endpoint /api/sdk/robots/:id ### Parameters #### Path Parameters - **id** (string) - Required - The unique identifier of the robot. ### Headers - **x-api-key** (string) - Required - Your generated API key. ### Response #### Success Response (200) - **data.recording_meta** (object) - The metadata of the specified robot. ``` -------------------------------- ### Get a specific robot - Bash Source: https://context7.com/getmaxun/maxun/llms.txt Retrieves details for a specific robot using its ID. Requires authentication. ```bash ROBOT_ID="your-robot-meta-id" curl -s -b cookies.txt "http://localhost:8080/storage/recordings/${ROBOT_ID}" | jq '.recording_meta' ``` -------------------------------- ### Create a crawl robot via SDK Source: https://context7.com/getmaxun/maxun/llms.txt Defines and creates a robot specifically for crawling websites. Configurable with URL, name, crawl settings (selector, depth, limit), and desired output formats. ```bash curl -s -X POST http://localhost:8080/api/sdk/crawl \ -H "x-api-key: ${API_KEY}" \ -H "Content-Type: application/json" \ -d '{ "url": "https://docs.example.com", "name": "Docs Crawler SDK", "crawlConfig": { "selector": "a[href]", "maxDepth": 2, "limit": 100, "urlFilter": "docs.example.com" }, "formats": ["markdown", "links"] }' | jq '.data.recording_meta.id' ``` -------------------------------- ### Get Current Workflow Source: https://context7.com/getmaxun/maxun/llms.txt Retrieves the current workflow for a given browser ID. Requires cookies to be managed. The output is the length of the workflow. ```bash curl -s -b cookies.txt "http://localhost:8080/workflow/${BROWSER_ID}" | jq '.workflow | length' ``` -------------------------------- ### Get current authenticated user - Bash Source: https://context7.com/getmaxun/maxun/llms.txt Retrieves information about the currently logged-in user, excluding sensitive details like the password. ```bash curl -s -b cookies.txt http://localhost:8080/auth/current-user | jq .# Response: # { "ok": true, "user": { "id": 1, "email": "user@example.com", "api_key": null } } ``` -------------------------------- ### Register a new user - Bash Source: https://context7.com/getmaxun/maxun/llms.txt Registers a new user account and returns a JWT token in an httpOnly cookie. Requires email and password. ```bash curl -s -c cookies.txt -X POST http://localhost:8080/auth/register \ -H "Content-Type: application/json" \ -d '{"email": "user@example.com", "password": "secret123"}' | jq . # Response: # { "id": 1, "email": "user@example.com" } ``` -------------------------------- ### Create a search robot via SDK Source: https://context7.com/getmaxun/maxun/llms.txt This endpoint allows you to create a new search robot by providing its name, search configuration (query, mode, limit), and desired output formats. ```APIDOC ## POST /api/sdk/search — Create a search robot via SDK ### Description Creates a new search robot with specified configurations. ### Method POST ### Endpoint /api/sdk/search ### Request Body - **name** (string) - Required - The name of the search robot. - **searchConfig** (object) - Required - Configuration for the search. - **query** (string) - Required - The search query. - **mode** (string) - Required - The search mode (e.g., "scrape"). - **limit** (integer) - Required - The number of results to limit. - **formats** (array) - Required - The desired output formats (e.g., ["markdown", "text"]) ### Request Example ```json { "name": "Competitor Research", "searchConfig": { "query": "best project management tools 2025", "mode": "scrape", "limit": 5 }, "formats": ["markdown", "text"] } ``` ### Response #### Success Response (200) Returns metadata including the recording ID. ### Response Example ```json { "data": { "recording_meta": { "id": "recording-id-string" } } } ``` ``` -------------------------------- ### Create a robot from a WorkflowFile Source: https://context7.com/getmaxun/maxun/llms.txt Constructs and registers a new robot based on a provided workflow definition. The workflow specifies scraping or navigation logic. ```bash curl -s -X POST http://localhost:8080/api/sdk/robots \ -H "x-api-key: ${API_KEY}" \ -H "Content-Type: application/json" \ -d '{ "meta": { "name": "Product Extractor", "type": "extract" }, "workflow": [ { "where": { "url": "https://shop.example.com/products" }, "what": [ { "action": "waitForLoadState", "args": ["networkidle"] }, { "action": "scrapeList", "args": [{ "listSelector": ".product-card", "fields": { "title": ".title", "price": ".price" }, "limit": 20, "pagination": null }] } ] }, { "where": { "url": "about:blank" }, "what": [ { "action": "goto", "args": ["https://shop.example.com/products"] }, { "action": "waitForLoadState", "args": ["networkidle"] } ] } ] }' | jq '.data.recording_meta.id' ``` -------------------------------- ### Log in a user - Bash Source: https://context7.com/getmaxun/maxun/llms.txt Authenticates a user and sets a JWT token cookie. Requires email and password. ```bash curl -s -c cookies.txt -X POST http://localhost:8080/auth/login \ -H "Content-Type: application/json" \ -d '{"email": "user@example.com", "password": "secret123"}' | jq . # Response: # { "id": 1, "email": "user@example.com" } ``` -------------------------------- ### Check API key and plan status Source: https://context7.com/getmaxun/maxun/llms.txt Verifies the validity of an API key and retrieves information about the associated plan and credits. Requires an `x-api-key` header. ```bash API_KEY="mxn_abc123..." curl -s -H "x-api-key: ${API_KEY}" http://localhost:8080/api/sdk/status | jq . # Response: { "email": "user@example.com", "plan": "OSS", "credits": 999999 } ``` -------------------------------- ### Get Runs for a Specific Robot API Source: https://context7.com/getmaxun/maxun/llms.txt Fetch the number of runs associated with a particular robot ID. This is useful for tracking the execution history of a specific robot. ```bash curl -s -b cookies.txt "http://localhost:8080/storage/recordings/${ROBOT_ID}/runs" | jq '.runs.totalCount' ``` -------------------------------- ### Run Recorded Workflow Source: https://context7.com/getmaxun/maxun/llms.txt Executes the currently recorded workflow. Requires cookies to be managed. The response indicates completion. ```bash curl -s -b cookies.txt http://localhost:8080/record/interpret # Response: "interpretation done" ``` -------------------------------- ### Create a Crawl robot Source: https://context7.com/getmaxun/maxun/llms.txt Creates a Crawl robot that traverses entire websites by following internal links. ```APIDOC ## POST /storage/recordings/crawl — Create a Crawl robot ### Description Crawl robots traverse entire websites following internal links. ### Method POST ### Endpoint /storage/recordings/crawl ### Request Body - **url** (string) - Required - The starting URL for the crawl. - **name** (string) - Required - The name of the crawl robot. - **crawlConfig** (object) - Configuration for the crawl. - **selector** (string) - CSS selector to identify links to follow. - **maxDepth** (integer) - Maximum depth of the crawl. - **limit** (integer) - Maximum number of pages to crawl. - **urlFilter** (string) - A filter to restrict crawling to specific domains or paths. - **formats** (array of strings) - Required - The desired output formats (e.g., "markdown", "links"). ### Request Example ```json { "url": "https://blog.example.com", "name": "Blog Crawler", "crawlConfig": { "selector": "a[href]", "maxDepth": 3, "limit": 50, "urlFilter": "blog.example.com" }, "formats": ["markdown", "links"] } ``` ### Response #### Success Response (200) - **robot.recording_meta.type** (string) - The type of the created robot, which will be "crawl". ### Response Example ``` "crawl" ``` ``` -------------------------------- ### Create a robot from a WorkflowFile Source: https://context7.com/getmaxun/maxun/llms.txt Creates a new robot by defining its metadata and workflow steps. ```APIDOC ## POST /api/sdk/robots — Create a robot from a WorkflowFile ### Description Creates a new robot by defining its metadata and workflow steps. ### Method POST ### Endpoint /api/sdk/robots ### Headers - **x-api-key** (string) - Required - Your generated API key. - **Content-Type** (string) - Required - application/json ### Request Body - **meta** (object) - Required - Metadata for the robot. - **name** (string) - Required - The name of the robot. - **type** (string) - Required - The type of the robot (e.g., "extract"). - **workflow** (array) - Required - An array of workflow steps defining the robot's actions. ### Request Example ```json { "meta": { "name": "Product Extractor", "type": "extract" }, "workflow": [ { "where": { "url": "https://shop.example.com/products" }, "what": [ { "action": "waitForLoadState", "args": ["networkidle"] }, { "action": "scrapeList", "args": [{ "listSelector": ".product-card", "fields": { "title": ".title", "price": ".price" }, "limit": 20, "pagination": null }] } ] }, { "where": { "url": "about:blank" }, "what": [ { "action": "goto", "args": ["https://shop.example.com/products"] }, { "action": "waitForLoadState", "args": ["networkidle"] } ] } ] } ``` ### Response #### Success Response (200) - **data.recording_meta.id** (string) - The ID of the newly created robot. ``` -------------------------------- ### LLM-based Extraction via SDK Source: https://context7.com/getmaxun/maxun/llms.txt Perform LLM-based data extraction from a given URL. Requires a prompt, URL, LLM provider and model, LLM API key, and a robot name. ```bash curl -s -X POST http://localhost:8080/api/sdk/extract/llm \ -H "x-api-key: ${API_KEY}" \ -H "Content-Type: application/json" \ -d '{ "prompt": "Extract all job titles, company names, and locations from LinkedIn job search results", "url": "https://www.linkedin.com/jobs/search?keywords=software+engineer", "llmProvider": "openai", "llmModel": "gpt-4o", "llmApiKey": "sk-...", "robotName": "LinkedIn Jobs Extractor" }' | jq '.data.robotId' ``` -------------------------------- ### Create a crawl robot via SDK Source: https://context7.com/getmaxun/maxun/llms.txt Creates a new robot specifically designed for crawling web pages, with configurable crawl settings. ```APIDOC ## POST /api/sdk/crawl — Create a crawl robot via SDK ### Description Creates a new robot specifically designed for crawling web pages, with configurable crawl settings. ### Method POST ### Endpoint /api/sdk/crawl ### Headers - **x-api-key** (string) - Required - Your generated API key. - **Content-Type** (string) - Required - application/json ### Request Body - **url** (string) - Required - The starting URL for the crawl. - **name** (string) - Required - The name of the crawl robot. - **crawlConfig** (object) - Optional - Configuration for the crawl process: - **selector** (string) - Optional - CSS selector for links to follow. - **maxDepth** (integer) - Optional - Maximum depth of the crawl. - **limit** (integer) - Optional - Maximum number of pages to crawl. - **urlFilter** (string) - Optional - Regex filter for URLs to include in the crawl. - **formats** (array of strings) - Optional - Desired output formats for the crawl results. ### Request Example ```json { "url": "https://docs.example.com", "name": "Docs Crawler SDK", "crawlConfig": { "selector": "a[href]", "maxDepth": 2, "limit": 100, "urlFilter": "docs.example.com" }, "formats": ["markdown", "links"] } ``` ### Response #### Success Response (200) - **data.recording_meta.id** (string) - The ID of the newly created crawl robot. ``` -------------------------------- ### List all robots - Bash Source: https://context7.com/getmaxun/maxun/llms.txt Retrieves a list of all created robots. Requires authentication. ```bash curl -s -b cookies.txt http://localhost:8080/storage/recordings | jq '.[0].recording_meta' # Response: # { "name": "My Robot", "id": "uuid-here", "type": "extract", "url": "https://example.com", "pairs": 3, "createdAt": "..." } ``` -------------------------------- ### Duplicate a robot for a new URL Source: https://context7.com/getmaxun/maxun/llms.txt Creates a copy of an existing robot, configured to target a different URL. Useful for adapting a robot to a similar but distinct website. ```bash curl -s -X POST "http://localhost:8080/api/sdk/robots/${ROBOT_META_ID}/duplicate" \ -H "x-api-key: ${API_KEY}" \ -H "Content-Type: application/json" \ -d '{"targetUrl": "https://other-shop.com/products"}' | jq '.data.recording_meta.url' ``` -------------------------------- ### List webhooks for a robot Source: https://context7.com/getmaxun/maxun/llms.txt Retrieve a list of all configured webhooks associated with a given robot ID. ```bash curl -s -b cookies.txt "http://localhost:8080/webhook/list/${ROBOT_ID}" | jq '.webhooks' ``` -------------------------------- ### Create LLM-Powered Extraction Robot API Source: https://context7.com/getmaxun/maxun/llms.txt Create an LLM-powered robot for natural-language-driven data extraction. Specify a prompt, optional URL, LLM provider, model, API key, and robot name. The system can auto-detect the target site if no URL is provided. ```bash curl -s -b cookies.txt -X POST http://localhost:8080/storage/recordings/llm \ -H "Content-Type: application/json" \ -d '{ "prompt": "Extract product name, price, and rating from Amazon search results", "url": "https://www.amazon.com/s?k=laptops", "llmProvider": "anthropic", "llmModel": "claude-opus-4-5", "llmApiKey": "sk-ant-...", "robotName": "Amazon Laptop Extractor" }' | jq '.robot.recording_meta.isLLM' ``` -------------------------------- ### Check API key and plan status Source: https://context7.com/getmaxun/maxun/llms.txt Verifies the provided API key and returns information about the associated user account and plan. ```APIDOC ## GET /api/sdk/status — Check API key and plan status ### Description Verifies the provided API key and returns information about the associated user account and plan. ### Method GET ### Endpoint /api/sdk/status ### Headers - **x-api-key** (string) - Required - Your generated API key. ### Response #### Success Response (200) - **email** (string) - The email address associated with the API key. - **plan** (string) - The current subscription plan. - **credits** (integer) - The number of remaining credits. ``` -------------------------------- ### Create a Search robot Source: https://context7.com/getmaxun/maxun/llms.txt Creates a Search robot that runs DuckDuckGo queries and optionally scrapes the result pages. ```APIDOC ## POST /storage/recordings/search — Create a Search robot ### Description Search robots run DuckDuckGo queries and optionally scrape the result pages. ### Method POST ### Endpoint /storage/recordings/search ### Request Body - **name** (string) - Required - The name of the search robot. - **searchConfig** (object) - Configuration for the search. - **query** (string) - Required - The search query. - **mode** (string) - Required - The mode of operation ("scrape" or other). - **limit** (integer) - The maximum number of results to retrieve. - **provider** (string) - The search provider (e.g., "duckduckgo"). - **formats** (array of strings) - Required - The desired output formats (e.g., "markdown"). ### Request Example ```json { "name": "AI News Search", "searchConfig": { "query": "latest AI research papers 2025", "mode": "scrape", "limit": 10, "provider": "duckduckgo" }, "formats": ["markdown"] } ``` ### Response #### Success Response (200) - **robot.recording_meta** (object) - Metadata about the created robot. - **name** (string) - The name of the robot. - **id** (string) - The unique identifier of the robot. - **type** (string) - The type of robot (e.g., "search"). - **searchConfig** (object) - The search configuration used. - **formats** (array of strings) - The output formats configured for the robot. ``` -------------------------------- ### Create a Scrape robot Source: https://context7.com/getmaxun/maxun/llms.txt Creates a Scrape robot that converts a webpage into various formats like Markdown, HTML, text, links, or screenshots. ```APIDOC ## POST /storage/recordings/scrape — Create a Scrape robot ### Description Scrape robots convert a webpage into Markdown, HTML, text, links, or screenshots. ### Method POST ### Endpoint /storage/recordings/scrape ### Request Body - **url** (string) - Required - The URL of the webpage to scrape. - **name** (string) - Required - The name of the scrape robot. - **formats** (array of strings) - Required - The desired output formats (e.g., "markdown", "screenshot-fullpage"). ### Request Example ```json { "url": "https://docs.example.com/guide", "name": "Docs Scraper", "formats": ["markdown", "screenshot-fullpage"] } ``` ### Response #### Success Response (200) - **robot.recording_meta** (object) - Metadata about the created robot. - **name** (string) - The name of the robot. - **id** (string) - The unique identifier of the robot. - **type** (string) - The type of robot (e.g., "scrape"). - **url** (string) - The URL the robot targets. - **formats** (array of strings) - The output formats configured for the robot. ### Response Example ```json { "name": "Docs Scraper", "id": "...", "type": "scrape", "url": "https://docs.example.com/guide", "formats": ["markdown", "screenshot-fullpage"] } ``` ``` -------------------------------- ### Duplicate a robot for a new URL - Bash Source: https://context7.com/getmaxun/maxun/llms.txt Creates a copy of an existing robot, allowing it to be targeted at a different URL. Requires authentication and robot ID. ```bash curl -s -b cookies.txt -X POST "http://localhost:8080/storage/recordings/${ROBOT_ID}/duplicate" \ -H "Content-Type: application/json" \ -d '{"targetUrl": "https://other-site.com/products"}' | jq '.robot.recording_meta.name' # Response: # "My Robot (products)" ``` -------------------------------- ### Create Scrape Robot API Source: https://context7.com/getmaxun/maxun/llms.txt Use this endpoint to create a scrape robot that converts a webpage into Markdown, HTML, text, links, or screenshots. Specify the URL, robot name, and desired output formats. ```bash curl -s -b cookies.txt -X POST http://localhost:8080/storage/recordings/scrape \ -H "Content-Type: application/json" \ -d '{ "url": "https://docs.example.com/guide", "name": "Docs Scraper", "formats": ["markdown", "screenshot-fullpage"] }' | jq '.robot.recording_meta' ``` -------------------------------- ### Link Robot to Google Sheet Source: https://context7.com/getmaxun/maxun/llms.txt Connects a robot to a specific Google Sheet by providing the robot ID, spreadsheet ID, and spreadsheet name. Requires cookies. ```bash curl -s -b cookies.txt -X POST http://localhost:8080/auth/gsheets/update \ -H "Content-Type: application/json" \ -d '{"robotId": "'"${ROBOT_ID}"'", "spreadsheetId": "1BxiM...", "spreadsheetName": "Extracted Data"}' | jq .message # Response: "Robot updated with selected Google Sheet ID" ``` -------------------------------- ### Duplicate a robot for a new URL Source: https://context7.com/getmaxun/maxun/llms.txt Creates a copy of an existing robot, configured to target a new URL. ```APIDOC ## POST /api/sdk/robots/:id/duplicate — Duplicate a robot for a new URL ### Description Creates a copy of an existing robot, configured to target a new URL. ### Method POST ### Endpoint /api/sdk/robots/:id/duplicate ### Parameters #### Path Parameters - **id** (string) - Required - The unique identifier of the robot to duplicate. ### Headers - **x-api-key** (string) - Required - Your generated API key. - **Content-Type** (string) - Required - application/json ### Request Body - **targetUrl** (string) - Required - The new URL for the duplicated robot. ### Request Example ```json {"targetUrl": "https://other-shop.com/products"} ``` ### Response #### Success Response (200) - **data.recording_meta.url** (string) - The URL of the newly duplicated robot. ``` -------------------------------- ### List webhooks for a robot Source: https://context7.com/getmaxun/maxun/llms.txt Retrieve a list of all configured webhooks associated with a specific robot. ```APIDOC ## GET /webhook/list/:robotId — List webhooks for a robot ### Description Retrieve a list of all configured webhooks associated with a specific robot. ### Method GET ### Endpoint /webhook/list/:robotId ### Parameters #### Path Parameters - **robotId** (string) - Required - The ID of the robot whose webhooks are to be listed. ### Response #### Success Response (200) - **webhooks** (array) - A list of webhook objects associated with the robot. ``` -------------------------------- ### Execute a robot and wait for results Source: https://context7.com/getmaxun/maxun/llms.txt Synchronously executes a robot and returns all extracted data. This operation can take up to 3 hours to complete. Specify desired output formats like `markdown`. ```bash curl -s -X POST "http://localhost:8080/api/sdk/robots/${ROBOT_META_ID}/execute" \ -H "x-api-key: ${API_KEY}" \ -H "Content-Type: application/json" \ -d '{"formats": ["markdown"]}' | jq '.data.data' # Response: # { # "textData": {}, # "listData": [{"title": "Widget A", "price": "$9.99"}, ...], # "crawlData": [], # "searchData": {}, # "text": null, # "markdown": "# Product Page\n...", # "html": null, # "promptResult": null # } ``` -------------------------------- ### Generate API key - Bash Source: https://context7.com/getmaxun/maxun/llms.txt Generates a persistent API key for programmatic access. Requires authentication. ```bash curl -s -b cookies.txt -X POST http://localhost:8080/auth/generate-api-key | jq . # Response: # { "message": "API key generated successfully", "api_key": "mxn_abc123...", "api_key_created_at": "2025-01-01T00:00:00.000Z" } ``` -------------------------------- ### Create Search Robot API Source: https://context7.com/getmaxun/maxun/llms.txt Create a search robot to perform DuckDuckGo queries and optionally scrape results. Define the robot name, search configuration (query, mode, limit, provider), and output formats. ```bash curl -s -b cookies.txt -X POST http://localhost:8080/storage/recordings/search \ -H "Content-Type: application/json" \ -d '{ "name": "AI News Search", "searchConfig": { "query": "latest AI research papers 2025", "mode": "scrape", "limit": 10, "provider": "duckduckgo" }, "formats": ["markdown"] }' | jq '.robot.recording_meta' ```