### Quick Start: Creating Your First Plugin Source: https://github.com/steel-dev/steel-browser/blob/main/docs/PLUGIN_DEVELOPMENT.md This example demonstrates how to create a basic 'Hello World' plugin that logs messages on browser launch and page creation events. ```APIDOC ## Quick Start: Creating Your First Plugin ### Description This example demonstrates how to create a basic 'Hello World' plugin that logs messages on browser launch and page creation events. ### Code Example ```typescript import { BasePlugin, PluginOptions } from '@steel-browser/api/cdp-plugin'; import { Browser, Page } from 'puppeteer-core'; export class HelloWorldPlugin extends BasePlugin { constructor(options: PluginOptions) { super({ name: 'hello-world', ...options }); } async onBrowserLaunch(browser: Browser): Promise { console.log('Hello from the browser launch event!'); } async onPageCreated(page: Page): Promise { console.log(`New page created: ${page.url()}`); } } // Usage const plugin = new HelloWorldPlugin({}); cdpService.registerPlugin(plugin); ``` ``` -------------------------------- ### Start Development Server (Bash) Source: https://github.com/steel-dev/steel-browser/blob/main/CONTRIBUTING.md Starts the Steel Browser development environment, including the API and UI servers, or individually. ```bash # Start both API and UI in development mode npm run dev # Or start individually: npm run dev -w api # API server on http://localhost:3000 npm run dev -w ui # UI server on http://localhost:5173 ``` -------------------------------- ### Run Steel Browser with Node.js Source: https://github.com/steel-dev/steel-browser/blob/main/README.md Starts the Steel server and UI directly using Node.js. This method requires Node.js and Chrome to be installed. It installs dependencies and runs the development server, exposing the server on port 3000 and the UI on port 5173. ```bash npm install npm run dev ``` -------------------------------- ### Clone and Install Dependencies (Bash) Source: https://github.com/steel-dev/steel-browser/blob/main/CONTRIBUTING.md Clones the Steel Browser repository and installs project dependencies using npm. ```bash git clone https://github.com/steel-dev/steel-browser.git cd steel-browser npm install ``` -------------------------------- ### Install Node.js Environment Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Commands to install Node.js version 22+ using package managers or version managers across Linux, macOS, and Windows. ```bash curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash source ~/.bashrc nvm install 22 nvm use 22 sudo apt update sudo apt install nodejs npm brew install node@22 ``` ```powershell choco install nodejs --version=22.0.0 ``` -------------------------------- ### Clone and Build Project Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Standard workflow for cloning the repository, installing dependencies, and building the project workspaces. ```bash git clone https://github.com/YOUR_USERNAME/steel-browser.git cd steel-browser git remote add upstream https://github.com/steel-dev/steel-browser.git npm install npm run build ``` -------------------------------- ### Configure Environment Variables Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Example configuration for the .env file to define server ports, Chrome paths, and logging levels. ```bash NODE_ENV=development HOST=0.0.0.0 PORT=3000 CDP_REDIRECT_PORT=9223 CHROME_HEADLESS=false CHROME_EXECUTABLE_PATH=/usr/bin/google-chrome API_URL=http://localhost:3000 ``` -------------------------------- ### Install Testing Dependencies and Run Tests Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Installs necessary testing dependencies like Vitest and jsdom, and provides commands to run unit tests, watch mode, and the Vitest UI. ```bash # Install testing dependencies (if not already installed) npm install -D vitest @vitest/ui jsdom # Run tests npm test # Run tests in watch mode npm test -- --watch # Run tests with UI npm test -- --ui ``` -------------------------------- ### Configure and Run Integration Tests Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Sets up the environment for integration testing by starting the API development server and then executing integration tests using a dedicated npm script. ```bash # Start test environment NODE_ENV=test npm run dev -w api # Run integration tests npm run test:integration ``` -------------------------------- ### Example Plugin Implementation Source: https://github.com/steel-dev/steel-browser/blob/main/docs/ARCHITECTURE.md Demonstrates how to implement a custom plugin for Steel Browser, such as an ad-blocker, using the BasePlugin class and Puppeteer. ```APIDOC ## Example Plugin Implementation ### Description This code snippet shows an example of creating a custom plugin for Steel Browser. The `AdBlockPlugin` extends `BasePlugin` and utilizes Puppeteer's request interception to block specified domains. ### Method N/A (This is a class implementation example) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response N/A ### Code Example ```typescript import { BasePlugin, PluginOptions } from '@steel-browser/api/cdp-plugin'; import { Browser, Page } from 'puppeteer-core'; export class AdBlockPlugin extends BasePlugin { private blockedDomains: Set; constructor(options: PluginOptions & { blockedDomains?: string[] }) { super({ name: 'ad-blocker', ...options }); this.blockedDomains = new Set(options.blockedDomains || []); } async onPageCreated(page: Page): Promise { await page.setRequestInterception(true); page.on('request', (request) => { const url = new URL(request.url()); if (this.blockedDomains.has(url.hostname)) { request.abort(); } else { request.continue(); } }); } } ``` ``` -------------------------------- ### Docker Development Setup (Bash) Source: https://github.com/steel-dev/steel-browser/blob/main/CONTRIBUTING.md Builds and runs the Steel Browser development environment using Docker Compose. ```bash # Build and run with Docker Compose docker-compose -f docker-compose.dev.yml up --build # Or use production images docker-compose up ``` -------------------------------- ### Install and Run End-to-End Tests Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Installs Playwright for end-to-end testing and provides commands to execute tests and launch the Playwright test runner UI. ```bash # Install E2E testing framework npm install -D playwright @playwright/test # Run E2E tests npx playwright test # Run E2E tests with UI npx playwright test --ui ``` -------------------------------- ### Install VS Code Extensions Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Installs recommended Visual Studio Code extensions for enhanced development experience, including support for TypeScript, Prettier, Tailwind CSS, and ESLint. ```bash # Install VS Code extensions code --install-extension ms-typescript.typescript code --install-extension esbenp.prettier-vscode code --install-extension bradlc.vscode-tailwindcss code --install-extension ms-vscode.vscode-typescript-next code --install-extension ms-vscode.vscode-eslint ``` -------------------------------- ### Run Development and Debugging Commands Source: https://github.com/steel-dev/steel-browser/blob/main/docs/ARCHITECTURE.md Provides CLI commands for starting the development server with hot reloading and enabling verbose logging for API debugging. ```bash npm run dev node --inspect ./api/build/index.js ENABLE_VERBOSE_LOGGING=true npm run dev -w api ``` -------------------------------- ### Create a Basic Steel Browser Plugin (TypeScript) Source: https://github.com/steel-dev/steel-browser/blob/main/docs/PLUGIN_DEVELOPMENT.md Demonstrates how to create a simple 'Hello World' plugin by extending the BasePlugin class and implementing lifecycle hooks like onBrowserLaunch and onPageCreated. This serves as a starting point for custom plugin development. ```typescript import { BasePlugin, PluginOptions } from '@steel-browser/api/cdp-plugin'; import { Browser, Page } from 'puppeteer-core'; export class HelloWorldPlugin extends BasePlugin { constructor(options: PluginOptions) { super({ name: 'hello-world', ...options }); } async onBrowserLaunch(browser: Browser): Promise { console.log('Hello from the browser launch event!'); } async onPageCreated(page: Page): Promise { console.log(`New page created: ${page.url()}`); } } // Usage const plugin = new HelloWorldPlugin({}); cdpService.registerPlugin(plugin); ``` -------------------------------- ### Example: Request Logger Plugin Source: https://github.com/steel-dev/steel-browser/blob/main/docs/PLUGIN_DEVELOPMENT.md A practical example of a plugin that logs all outgoing HTTP requests made by a page, including method, URL, and headers. ```APIDOC ## Example: Request Logger Plugin ### Description A practical example of a plugin that logs all outgoing HTTP requests made by a page, including method, URL, and headers. ### Code Example ```typescript export class RequestLoggerPlugin extends BasePlugin { private logFile: string; constructor(options: PluginOptions & { logFile?: string }) { super({ name: 'request-logger', ...options }); this.logFile = options.logFile || 'requests.log'; } async onPageCreated(page: Page): Promise { await page.setRequestInterception(true); page.on('request', (request) => { const logEntry = { timestamp: new Date().toISOString(), method: request.method(), url: request.url(), headers: request.headers() }; // Log to file or console console.log('Request:', logEntry); request.continue(); }); } } ``` ``` -------------------------------- ### Example package.json for Steel Browser Plugin Source: https://github.com/steel-dev/steel-browser/blob/main/docs/PLUGIN_DEVELOPMENT.md Illustrates a typical package.json file for a Steel Browser plugin. It specifies package metadata, entry points, keywords, and peer dependencies like the Steel Browser API and puppeteer-core. ```json { "name": "steel-plugin-example", "version": "1.0.0", "description": "Example plugin for Steel Browser", "main": "dist/index.js", "types": "dist/index.d.ts", "keywords": ["steel-browser", "plugin", "automation"], "peerDependencies": { "@steel-browser/api": "^1.0.0", "puppeteer-core": "^23.0.0" }, "files": ["dist/**/*"] } ``` -------------------------------- ### Development Setup with Docker Compose Source: https://github.com/steel-dev/steel-browser/blob/main/README.md Sets up the Steel browser for local development using a specific development Docker Compose file. This ensures local code changes are reflected in the running application. The `--build` flag is used to re-build Docker images on each run. ```bash docker compose -f docker-compose.dev.yml up ``` ```bash docker compose -f docker-compose.dev.yml up --build ``` -------------------------------- ### Install Dependencies for Browser Launch Failures on Linux Source: https://github.com/steel-dev/steel-browser/blob/main/docs/TROUBLESHOOTING.md Install essential system dependencies required for Steel Browser to function correctly, particularly when encountering browser launch or permission issues on Linux systems. This ensures the browser environment is properly configured. ```bash sudo apt-get update sudo apt-get install -y \ libnss3-dev \ libatk-bridge2.0-dev \ libdrm-dev \ libxcomposite-dev \ libxdamage-dev \ libxrandr-dev \ libgbm-dev \ libxss-dev \ libasound2-dev ``` -------------------------------- ### Daily Development Workflow Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Outlines a typical daily development workflow, including updating the main branch, creating feature branches, starting the development server, making changes, running quality checks, committing, and pushing for a Pull Request. ```bash # 1. Update your fork git fetch upstream git checkout main git merge upstream/main # 2. Create feature branch git checkout -b feature/my-new-feature # 3. Start development environment npm run dev # 4. Make changes and test # ... develop your feature ... # 5. Run quality checks npm run pretty -w api npm run lint -w ui npm run build # 6. Commit changes git add . git commit -m "feat: add new feature" # 7. Push and create PR git push origin feature/my-new-feature ``` -------------------------------- ### Example: Ad Blocker Plugin Source: https://github.com/steel-dev/steel-browser/blob/main/docs/PLUGIN_DEVELOPMENT.md An example of an Ad Blocker plugin that intercepts requests and aborts those originating from a predefined list of advertising domains. ```APIDOC ## Example: Ad Blocker Plugin ### Description An example of an Ad Blocker plugin that intercepts requests and aborts those originating from a predefined list of advertising domains. ### Code Example ```typescript export class AdBlockerPlugin extends BasePlugin { private blockedDomains: Set; private blockedCount: number = 0; constructor(options: PluginOptions & { blockedDomains?: string[] }) { super({ name: 'ad-blocker', ...options }); this.blockedDomains = new Set(options.blockedDomains || [ 'doubleclick.net', 'googleadservices.com', 'googlesyndication.com' ]); } async onPageCreated(page: Page): Promise { await page.setRequestInterception(true); page.on('request', (request) => { const url = new URL(request.url()); if (this.blockedDomains.has(url.hostname)) { this.blockedCount++; console.log(`Blocked ad request: ${url.hostname}`); request.abort(); } else { request.continue(); } }); } async onShutdown(): Promise { console.log(`Ad Blocker: Blocked ${this.blockedCount} requests`); } } ``` ``` -------------------------------- ### Run Steel Browser Docker Image Source: https://github.com/steel-dev/steel-browser/blob/main/README.md Starts the Steel browser server using a Docker image. It exposes ports for the main application (3000) and the console debugger (9223). This allows for immediate use of Steel's session creation, scraping, and screenshot functionalities. ```bash docker run -p 3000:3000 -p 9223:9223 ghcr.io/steel-dev/steel-browser ``` -------------------------------- ### Document Directory Structure Source: https://github.com/steel-dev/steel-browser/blob/main/docs/README.md A visual representation of the project's documentation directory structure, illustrating the organization of markdown files for architecture, setup, and plugin development. ```text docs/ ├── README.md # This file - documentation overview ├── ARCHITECTURE.md # System architecture and design ├── DEVELOPMENT_SETUP.md # Development environment setup ├── PLUGIN_DEVELOPMENT.md # Plugin creation guide └── TROUBLESHOOTING.md # Common issues and solutions ``` -------------------------------- ### Integrating Steel Browser as a Fastify Plugin Source: https://context7.com/steel-dev/steel-browser/llms.txt Example of registering the Steel Browser plugin within a Fastify application. Demonstrates configuring storage, logging, lifecycle hooks, and accessing the CDP service for browser automation. ```typescript import Fastify from 'fastify'; import steelBrowserPlugin from '@steel-browser/api/plugin'; const fastify = Fastify({ logger: true }); await fastify.register(steelBrowserPlugin, { fileStorage: { maxSizePerSession: 100 * 1024 * 1024 }, logging: { enableStorage: true, storagePath: './logs', enableConsoleLogging: true, enableLogsRoutes: true } }); fastify.registerCDPLaunchHook(async (config) => { console.log('Browser launching with config:', config); }); fastify.registerCDPShutdownHook(async (config) => { console.log('Browser shutting down'); }); fastify.get('/custom-endpoint', async (request, reply) => { const page = await fastify.cdpService.getPrimaryPage(); await page.goto('https://example.com'); const title = await page.title(); return { title }; }); await fastify.listen({ port: 3000, host: '0.0.0.0' }); ``` -------------------------------- ### Register Fastify Plugins for Steel Browser Source: https://github.com/steel-dev/steel-browser/blob/main/docs/ARCHITECTURE.md Shows the main plugin registration process for Steel Browser using Fastify, including configuration for file storage. It also lists examples of individual plugins that can be registered. ```typescript // Main plugin registration await fastify.register(steelBrowserPlugin, { fileStorage: { maxSizePerSession: 100 * MB } }); // Individual plugins await fastify.register(browserInstancePlugin); await fastify.register(sessionPlugin); await fastify.register(fileStoragePlugin); ``` -------------------------------- ### Environment Verification Commands for Steel Browser Source: https://github.com/steel-dev/steel-browser/blob/main/docs/TROUBLESHOOTING.md Verify that the necessary software versions and dependencies are installed and accessible for Steel Browser. This includes checking Node.js, npm, Chrome/Chromium, and Docker. ```bash node --version npm --version google-chrome --version chromium --version docker --version docker-compose --version ``` -------------------------------- ### Schema Validation Example Source: https://github.com/steel-dev/steel-browser/blob/main/docs/ARCHITECTURE.md Shows an example of a Zod schema used for validating request payloads, specifically for a 'scrape' action. ```APIDOC ## POST /v1/scrape ### Description Performs a web scraping operation on a given URL, with options for delay, output format, and capturing screenshots or PDFs. ### Method POST ### Endpoint `/v1/scrape` ### Parameters #### Request Body - **url** (string) - Optional - The URL to scrape. - **delay** (number) - Optional - Delay in milliseconds before scraping. - **format** (array of strings) - Optional - Desired output formats (e.g., 'html', 'cleaned_html', 'markdown', 'readability'). - **screenshot** (boolean) - Optional - Whether to capture a screenshot. - **pdf** (boolean) - Optional - Whether to generate a PDF. ### Request Example ```json { "url": "https://example.com", "delay": 1000, "format": ["html", "markdown"], "screenshot": true, "pdf": false } ``` ### Response #### Success Response (200) - **data** (object) - The scraped content and any requested artifacts (e.g., screenshot, PDF). #### Response Example ```json { "data": { "html": "...", "markdown": "# Example Content", "screenshot": "base64_encoded_image_string", "pdf": "base64_encoded_pdf_string" } } ``` ### Code Example ```typescript const ScrapeRequestSchema = z.object({ url: z.string().url().optional(), delay: z.number().optional(), format: z.array(z.enum(['html','cleaned_html','markdown','readability'])).optional(), screenshot: z.boolean().optional(), pdf: z.boolean().optional(), }); ``` ``` -------------------------------- ### Conventional Commit Message Examples Source: https://github.com/steel-dev/steel-browser/blob/main/CONTRIBUTING.md Examples of commit messages following the Conventional Commits specification, categorized by type such as feat, fix, docs, and test. ```text feat(api): add session timeout configuration fix(ui): resolve session list refresh issue docs: update plugin development guide test(api): add CDP service unit tests ``` -------------------------------- ### GET /v1/sessions Source: https://context7.com/steel-dev/steel-browser/llms.txt Retrieves a list of all active browser sessions with their metadata and current status. ```APIDOC ## GET /v1/sessions ### Description Lists all active browser sessions. ### Method GET ### Endpoint /v1/sessions ### Response #### Success Response (200) - **sessions** (array) - List of session objects containing status, duration, and usage metrics. #### Response Example { "sessions": [ { "id": "550e8400-e29b-41d4-a716-446655440000", "status": "live", "websocketUrl": "ws://localhost:3000/ws" } ] } ``` -------------------------------- ### GET /v1/sessions/{id}/context Source: https://context7.com/steel-dev/steel-browser/llms.txt Retrieve the browser context data including cookies, local storage, and session storage for a specific session. ```APIDOC ## GET /v1/sessions/{id}/context ### Description Retrieve the browser context data including cookies, local storage, and session storage for a session. ### Method GET ### Endpoint /v1/sessions/{id}/context ### Parameters #### Path Parameters - **id** (string) - Required - The unique session identifier. ### Request Example curl http://localhost:3000/v1/sessions/550e8400-e29b-41d4-a716-446655440000/context ### Response #### Success Response (200) - **cookies** (array) - List of cookies. - **localStorage** (object) - Local storage key-value pairs. - **sessionStorage** (object) - Session storage key-value pairs. ``` -------------------------------- ### Configure Headless Mode and Virtual Display for Steel Browser Source: https://github.com/steel-dev/steel-browser/blob/main/docs/TROUBLESHOOTING.md Manage headless mode and set up a virtual display (Xvfb) for Steel Browser. This is useful for debugging launch issues or running the browser in environments without a physical display. ```bash # Disable headless mode for debugging export CHROME_HEADLESS=false # Or run with virtual display export DISPLAY=:99 Xvfb :99 -screen 0 1024x768x24 & ``` -------------------------------- ### Configure Network Proxy and SSL Settings for Steel Browser Source: https://github.com/steel-dev/steel-browser/blob/main/docs/TROUBLESHOOTING.md Set up proxy configurations and manage SSL certificate verification for Steel Browser. This is essential for environments with specific network requirements or when dealing with proxy authentication and SSL errors. ```bash # Set proxy configuration export PROXY_URL="http://proxy.company.com:8080" export PROXY_URL="http://username:password@proxy.company.com:8080" # Disable SSL verification (development only) export NODE_TLS_REJECT_UNAUTHORIZED=0 # Check network connectivity curl -I https://google.com ``` -------------------------------- ### Integration Testing Steel Browser Plugins with CDPService Source: https://github.com/steel-dev/steel-browser/blob/main/docs/PLUGIN_DEVELOPMENT.md Demonstrates integration testing for Steel Browser plugins by interacting with the CDPService. It covers setting up the CDPService, registering a plugin, launching the browser, and includes placeholders for testing plugin behavior within the integrated environment. ```typescript // integration.test.ts import { CDPService } from '@steel-browser/api'; import { MyPlugin } from './my-plugin'; describe('Plugin Integration', () => { let cdpService: CDPService; let plugin: MyPlugin; beforeEach(async () => { cdpService = new CDPService({}, console); plugin = new MyPlugin({ name: 'test-plugin' }); cdpService.registerPlugin(plugin); }); afterEach(async () => { await cdpService.shutdown(); }); it('should work with CDP service', async () => { await cdpService.launch(); // Test plugin behavior }); }); ``` -------------------------------- ### Implement Structured Logging Source: https://github.com/steel-dev/steel-browser/blob/main/docs/ARCHITECTURE.md Demonstrates how to log structured events using the Pino logger within a Fastify application context. ```typescript fastify.log.info({ sessionId, action: 'page_created', url: page.url() }, 'New page created'); ``` -------------------------------- ### Create Browser Session Source: https://context7.com/steel-dev/steel-browser/llms.txt Initializes a new browser session with specific configurations such as proxy settings, viewport dimensions, and ad blocking. Returns session metadata including the WebSocket URL required for automation connections. ```bash curl -X POST http://localhost:3000/v1/sessions \ -H "Content-Type: application/json" \ -d '{ "blockAds": true, "dimensions": {"width": 1920, "height": 1080} }' ``` -------------------------------- ### Get Log Statistics Source: https://context7.com/steel-dev/steel-browser/llms.txt Fetches statistical information about the stored browser logs, including the total number of events, the time range of the oldest and newest events, and the total storage size in bytes. ```bash curl http://localhost:3000/v1/logs/stats ``` -------------------------------- ### Configure Node.js Inspector for API Debugging Source: https://github.com/steel-dev/steel-browser/blob/main/docs/TROUBLESHOOTING.md Start the Steel Browser API with the Node.js inspector enabled to allow debugging via Chrome DevTools. This enables remote debugging capabilities for the backend services. ```bash # Start API with inspector node --inspect ./api/build/index.js # Or with specific port node --inspect=0.0.0.0:9229 ./api/build/index.js # Then open Chrome and go to: # chrome://inspect ``` -------------------------------- ### Deploying Steel Browser with Docker Source: https://context7.com/steel-dev/steel-browser/llms.txt Common Docker commands for running Steel Browser. Includes options for standard execution, docker-compose, development builds, and environment variable configuration. ```bash docker run -p 3000:3000 -p 9223:9223 ghcr.io/steel-dev/steel-browser docker-compose up docker-compose -f docker-compose.dev.yml up --build DOCKER_DEFAULT_PLATFORM=linux/arm64 docker-compose up docker run -p 3000:3000 -p 9223:9223 -e CHROME_HEADLESS=true -e DEFAULT_TIMEZONE=America/New_York ghcr.io/steel-dev/steel-browser ``` -------------------------------- ### Get Session Context (Bash) Source: https://context7.com/steel-dev/steel-browser/llms.txt Retrieves the browser context data, including cookies, local storage, and session storage for a specified session. This is useful for inspecting the state of a web page within a session. ```bash # Get session context (cookies, storage) curl http://localhost:3000/v1/sessions/550e8400-e29b-41d4-a716-446655440000/context ``` -------------------------------- ### Execute Development Workflow Commands Source: https://github.com/steel-dev/steel-browser/blob/main/CONTRIBUTING.md Provides standard CLI commands for formatting, linting, and building the project. These commands are essential for maintaining code quality across the API and UI workspaces. ```bash # Format code (API) npm run pretty -w api # Lint code (UI) npm run lint -w ui # Build and type check npm run build ``` -------------------------------- ### Resolve Port Conflicts in Steel Browser Source: https://github.com/steel-dev/steel-browser/blob/main/docs/TROUBLESHOOTING.md Identify and resolve port conflicts that prevent Steel Browser services from starting or connecting. This involves checking which processes are using specific ports and optionally reconfiguring ports. ```bash # Check what's using the ports lsof -i :3000 # API port lsof -i :5173 # UI port lsof -i :9223 # CDP port # Kill processes using the ports kill -9 $(lsof -t -i:3000) # Use different ports export PORT=3001 export CDP_REDIRECT_PORT=9224 ``` -------------------------------- ### Configurable Plugin for Steel Browser Source: https://github.com/steel-dev/steel-browser/blob/main/docs/PLUGIN_DEVELOPMENT.md Illustrates creating a configurable plugin by defining custom options and merging them with base plugin options. It shows how to set default values for configuration properties like API keys, endpoints, retries, and timeouts. ```typescript interface MyPluginOptions extends PluginOptions { apiKey?: string; endpoint?: string; retries?: number; timeout?: number; } export class ConfigurablePlugin extends BasePlugin { private config: MyPluginOptions; constructor(options: MyPluginOptions) { super(options); this.config = { apiKey: options.apiKey || process.env.API_KEY, endpoint: options.endpoint || 'https://api.example.com', retries: options.retries || 3, timeout: options.timeout || 5000, ...options }; } } ``` -------------------------------- ### Real-time Session Updates Source: https://github.com/steel-dev/steel-browser/blob/main/docs/ARCHITECTURE.md Demonstrates how to use React Query to fetch session data with a refetch interval for real-time updates. ```APIDOC ## GET /v1/sessions ### Description Retrieves a list of active sessions. This endpoint is designed to be polled for real-time updates on session status. ### Method GET ### Endpoint `/v1/sessions` ### Parameters N/A ### Request Example N/A ### Response #### Success Response (200) - **sessions** (array) - An array of session objects. - **id** (string) - The unique identifier for the session. - **status** (string) - The current status of the session (e.g., 'running', 'closed'). - **createdAt** (string) - Timestamp when the session was created. #### Response Example ```json { "sessions": [ { "id": "session-123", "status": "running", "createdAt": "2023-10-27T10:00:00Z" } ] } ``` ### Code Example ```typescript import { useQuery } from '@tanstack/react-query'; import { steelClient } from './steelClient'; // Assuming steelClient is your API client instance // Session monitoring const { data: sessions } = useQuery({ queryKey: ['sessions'], queryFn: () => steelClient.sessions.getSessions(), refetchInterval: 1000 // Real-time updates every 1 second }); ``` ``` -------------------------------- ### Configure Custom Chrome Executable for Node.js Source: https://github.com/steel-dev/steel-browser/blob/main/README.md Allows specifying a custom path for the Chrome executable when running the Steel browser with Node.js. This is useful if Chrome is installed in a non-standard location. The `CHROME_EXECUTABLE_PATH` environment variable is used for this configuration. ```bash export CHROME_EXECUTABLE_PATH=/path/to/your/chrome npm run dev ``` -------------------------------- ### Stream Browser Logs in Real-time Source: https://context7.com/steel-dev/steel-browser/llms.txt Establishes a Server-Sent Events (SSE) connection to stream browser logs in real-time. This is useful for live monitoring of browser activities as they occur. ```bash # Stream logs (SSE endpoint) curl -N http://localhost:3000/v1/logs/stream ``` -------------------------------- ### Perform Quick Actions (Scrape, Screenshot, PDF) Source: https://github.com/steel-dev/steel-browser/blob/main/README.md Executes on-demand tasks to extract HTML, capture screenshots, or generate PDFs from a target URL using the Actions API. ```bash # Scrape curl -X POST http://0.0.0.0:3000/v1/scrape -H "Content-Type: application/json" -d '{"url": "https://example.com", "delay": 1000}' # Screenshot curl -X POST http://0.0.0.0:3000/v1/screenshot -H "Content-Type: application/json" -d '{"url": "https://example.com", "fullPage": true}' --output screenshot.png # PDF curl -X POST http://0.0.0.0:3000/v1/pdf -H "Content-Type: application/json" -d '{"url": "https://example.com"}' --output output.pdf ``` -------------------------------- ### Get Specific Steel Session Details (API) Source: https://context7.com/steel-dev/steel-browser/llms.txt Fetches detailed information for a specific Steel browser session identified by its ID. This includes connection URLs (WebSocket, debug) and usage statistics like proxy bandwidth. ```bash # Get session by ID curl http://localhost:3000/v1/sessions/550e8400-e29b-41d4-a716-446655440000 # Response includes full session details with WebSocket URL, debug URL, # proxy bandwidth usage, and current status ``` -------------------------------- ### Use Steel Python SDK with Playwright Source: https://context7.com/steel-dev/steel-browser/llms.txt Shows how to use the Steel Python SDK to create and manage browser sessions, then integrates with Playwright for Python to perform browser automation. Includes session creation, interaction, and release. ```python from steel import Steel client = Steel(base_url="http://localhost:3000") try: # Create session session = client.sessions.create( block_ads=True, proxy_url="http://user:pass@proxy.example.com:8080", dimensions={"width": 1280, "height": 800}, timezone="America/New_York" ) print(f"Session created: {session.id}") print(f"WebSocket URL: {session.websocket_url}") # Use with Playwright Python from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.connect_over_cdp(session.websocket_url) context = browser.contexts[0] page = context.pages[0] if context.pages else context.new_page() page.goto("https://example.com") print(f"Page title: {page.title()}") browser.close() # Release session client.sessions.release(session.id) print("Session released") except Exception as e: print(f"Error: {e}") ``` -------------------------------- ### Get All Active Steel Sessions (API) Source: https://context7.com/steel-dev/steel-browser/llms.txt Retrieves a list of all currently active Steel browser sessions using a simple cURL command. The response includes metadata for each session such as ID, creation time, status, and dimensions. ```bash # List all sessions curl http://localhost:3000/v1/sessions # Response: # { # "sessions": [ # { # "id": "550e8400-e29b-41d4-a716-446655440000", # "createdAt": "2024-01-15T10:30:00.000Z", # "status": "live", # "duration": 120000, # "eventCount": 45, # "dimensions": {"width": 1920, "height": 1080}, # "timeout": 900000, # "creditsUsed": 5, # "websocketUrl": "ws://localhost:3000/ws", # "debugUrl": "http://localhost:3000/v1/sessions/debug", # "proxyTxBytes": 1024000, # "proxyRxBytes": 5120000 # } # ] # } ``` -------------------------------- ### Connect with Puppeteer Source: https://context7.com/steel-dev/steel-browser/llms.txt Demonstrates how to connect to a Steel browser session using Puppeteer and perform basic automation tasks like navigating, taking screenshots, and extracting data. ```typescript import puppeteer from 'puppeteer-core'; async function automateWithPuppeteer() { // First create a session const sessionResponse = await fetch('http://localhost:3000/v1/sessions', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ blockAds: true }) }); const session = await sessionResponse.json(); // Connect Puppeteer to the session const browser = await puppeteer.connect({ browserWSEndpoint: session.websocketUrl }); const page = await browser.newPage(); await page.goto('https://example.com'); // Perform automation const title = await page.title(); console.log('Page title:', title); // Take screenshot await page.screenshot({ path: 'screenshot.png', fullPage: true }); // Extract data const links = await page.evaluate(() => { return Array.from(document.querySelectorAll('a')).map(a => ({ href: a.href, text: a.textContent?.trim() })); }); console.log('Found links:', links); await browser.disconnect(); // Release the session when done await fetch(`http://localhost:3000/v1/sessions/${session.id}/release`, { method: 'POST' }); } automateWithPuppeteer().catch(console.error); ``` -------------------------------- ### Get Steel Session Live Details (API) Source: https://context7.com/steel-dev/steel-browser/llms.txt Retrieves the real-time state of a Steel browser session, including details about open pages, tabs, and the overall browser state. It also provides URLs for live viewing and full-screen access. ```bash # Get live session state curl http://localhost:3000/v1/sessions/550e8400-e29b-41d4-a716-446655440000/live-details # Response: # { # "sessionViewerUrl": "http://localhost:3000/ui/sessions/", # "sessionViewerFullscreenUrl": "http://localhost:3000/ui/sessions/.../fullscreen", # "websocketUrl": "ws://localhost:3000/ws", # "pages": [ # { ``` -------------------------------- ### Service Restart Procedures (Bash) Source: https://github.com/steel-dev/steel-browser/blob/main/docs/TROUBLESHOOTING.md Commands to restart Steel Browser services, including graceful restarts, force restarts using pkill, and Docker-specific restarts. ```bash # Graceful restart npm run dev # Ctrl+C then restart # Force restart pkill -f "node.*steel" npm run dev # Docker restart docker-compose restart ``` -------------------------------- ### Run Steel Browser Locally with Docker Source: https://github.com/steel-dev/steel-browser/blob/main/README.md This command pulls and executes the pre-built Steel browser Docker image locally. It is the recommended method for developers to quickly spin up a browser instance for testing and integration. ```bash docker run -p 3000:3000 ghcr.io/steel-dev/steel-browser:latest ``` -------------------------------- ### Run Steel Browser with Docker Compose Source: https://github.com/steel-dev/steel-browser/blob/main/README.md Launches the Steel browser and its components using Docker Compose. This is a convenient way to manage multiple services. For Mac Silicon users, a specific environment variable is provided to ensure correct platform execution. ```bash docker compose up ``` ```bash DOCKER_DEFAULT_PLATFORM=linux/arm64 docker compose up ``` -------------------------------- ### Error Handling in Steel Browser Plugin Source: https://github.com/steel-dev/steel-browser/blob/main/docs/PLUGIN_DEVELOPMENT.md Provides an example of robust error handling within a Steel Browser plugin. It demonstrates using try-catch blocks to gracefully manage errors during plugin operations, such as setting up page interception, and emphasizes that plugin errors are isolated by the PluginManager. ```typescript export class RobustPlugin extends BasePlugin { async onPageCreated(page: Page): Promise { try { // Plugin logic here await this.setupPageInterception(page); } catch (error) { console.error(`Error in ${this.name} plugin:`, error); // Plugin errors are isolated by the PluginManager // but you should handle them gracefully } } private async setupPageInterception(page: Page): Promise { // Implementation with proper error handling } } ``` -------------------------------- ### POST /v1/sessions Source: https://context7.com/steel-dev/steel-browser/llms.txt Creates a new browser automation session with configurable parameters like proxy settings, viewport dimensions, and timezone. ```APIDOC ## POST /v1/sessions ### Description Initializes a new browser session for automation. ### Method POST ### Endpoint /v1/sessions ### Request Body - **blockAds** (boolean) - Optional - Whether to block ads during the session. - **dimensions** (object) - Optional - Viewport dimensions (width, height). - **proxyUrl** (string) - Optional - Proxy server URL for the session. - **timezone** (string) - Optional - Timezone for the browser instance. ### Request Example { "blockAds": true, "dimensions": {"width": 1920, "height": 1080} } ### Response #### Success Response (200) - **id** (string) - Unique session identifier. - **websocketUrl** (string) - WebSocket URL for browser connection. - **debugUrl** (string) - URL for debugging the session. #### Response Example { "id": "550e8400-e29b-41d4-a716-446655440000", "websocketUrl": "ws://localhost:3000/ws", "debugUrl": "http://localhost:3000/v1/sessions/debug" } ``` -------------------------------- ### Implement Real-time Session Updates with React Query Source: https://github.com/steel-dev/steel-browser/blob/main/docs/ARCHITECTURE.md Demonstrates how to use React Query to fetch session data and enable real-time updates by setting a refetch interval. This is part of the frontend state management strategy. ```typescript // Session monitoring const { data: sessions } = useQuery({ queryKey: ['sessions'], queryFn: () => steelClient.sessions.getSessions(), refetchInterval: 1000 // Real-time updates }); ``` -------------------------------- ### Integrate Steel Browser Plugin (TypeScript) Source: https://github.com/steel-dev/steel-browser/blob/main/CONTRIBUTING.md Demonstrates how to register the Steel Browser plugin within a Fastify application, including configuration options for file storage and custom WebSocket handlers. ```typescript import Fastify from 'fastify'; import steelBrowserPlugin, { SteelBrowserConfig } from './api/src/steel-browser-plugin.js'; const fastify = Fastify({ logger: true }); // Register Steel Browser plugin with configuration const config: SteelBrowserConfig = { fileStorage: { maxSizePerSession: 100 * 1024 * 1024, // 100MB }, customWsHandlers: [ // Your custom WebSocket handlers ], }; await fastify.register(steelBrowserPlugin, config); // Your additional routes and plugins await fastify.register(myCustomPlugin); await fastify.listen({ port: 3000 }); ``` -------------------------------- ### Configure VS Code Settings Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Sets up the `.vscode/settings.json` file to configure editor behavior, including import preferences, format-on-save, default formatter, ESLint integration, and file exclusion patterns for build artifacts and dependencies. ```json { "typescript.preferences.importModuleSpecifier": "relative", "editor.formatOnSave": true, "editor.defaultFormatter": "esbenp.prettier-vscode", "editor.codeActionsOnSave": { "source.fixAll.eslint": true }, "typescript.preferences.includePackageJsonAutoImports": "on", "files.exclude": { "**/node_modules": true, "**/build": true, "**/dist": true, "**/.cache": true } } ``` -------------------------------- ### Manage Docker Development Containers Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Commands to orchestrate the development environment using Docker Compose or individual container builds. ```bash docker-compose -f docker-compose.dev.yml up --build docker build -t steel-browser-api -f ./api/Dockerfile . docker run -p 3000:3000 -p 9223:9223 steel-browser-api ``` -------------------------------- ### System Diagnostic Commands (Bash) Source: https://github.com/steel-dev/steel-browser/blob/main/docs/TROUBLESHOOTING.md Execute these bash commands to gather system information, including system details, memory usage, disk usage, and running Chrome/Node processes. ```bash # Get system info uname -a # System information free -h # Memory usage df -h # Disk usage ps aux | grep chrome # Chrome processes ps aux | grep node # Node processes ``` -------------------------------- ### Create a Selenium Session Source: https://github.com/steel-dev/steel-browser/blob/main/README.md Launches a browser session compatible with the Selenium WebDriver protocol. This allows teams to integrate Steel into existing Selenium-based workflows. ```typescript const session = await client.sessions.create({ isSelenium: true }); ``` ```python session = client.sessions.create(is_selenium=True) ``` ```bash curl -X POST http://localhost:3000/v1/sessions \ -H "Content-Type: application/json" \ -d '{ "isSelenium": true }' ``` -------------------------------- ### Base Plugin Class and Options Source: https://github.com/steel-dev/steel-browser/blob/main/docs/PLUGIN_DEVELOPMENT.md Details the structure of the `BasePlugin` class and the `PluginOptions` interface, which form the foundation for all custom plugins. ```APIDOC ## Base Plugin Class and Options ### Description Details the structure of the `BasePlugin` class and the `PluginOptions` interface, which form the foundation for all custom plugins. ### Base Plugin Class All plugins extend the `BasePlugin` abstract class: ```typescript abstract class BasePlugin { public name: string; protected options: PluginOptions; protected cdpService: CDPService | null; constructor(options: PluginOptions); public setService(service: CDPService): void; // Lifecycle hooks (all optional) public async onBrowserLaunch(browser: Browser): Promise {} public async onPageCreated(page: Page): Promise {} public async onPageNavigate(page: Page): Promise {} public async onPageUnload(page: Page): Promise {} public async onBrowserClose(browser: Browser): Promise {} public async onBeforePageClose(page: Page): Promise {} public async onShutdown(): Promise {} } ``` ### Plugin Options ```typescript interface PluginOptions { name: string; [key: string]: any; // Additional plugin-specific options } ``` ``` -------------------------------- ### Configure VS Code Debugging Source: https://github.com/steel-dev/steel-browser/blob/main/docs/DEVELOPMENT_SETUP.md Defines launch configurations in `.vscode/launch.json` for debugging the Node.js API and Vitest tests. Includes settings for program entry points, output files, environment variables, and runtime arguments. ```json { "version": "0.2.0", "configurations": [ { "name": "Debug API", "type": "node", "request": "launch", "program": "${workspaceFolder}/api/build/index.js", "outFiles": ["${workspaceFolder}/api/build/**/*.js"], "env": { "NODE_ENV": "development", "ENABLE_VERBOSE_LOGGING": "true", "CHROME_HEADLESS": "false" }, "console": "integratedTerminal", "restart": true, "runtimeArgs": ["--inspect"] }, { "name": "Debug Tests", "type": "node", "request": "launch", "program": "${workspaceFolder}/node_modules/.bin/vitest", "args": ["run"], "cwd": "${workspaceFolder}", "console": "integratedTerminal" } ] } ``` -------------------------------- ### POST /v1/sessions Source: https://github.com/steel-dev/steel-browser/blob/main/README.md Creates a new browser session with custom options such as proxy settings, ad blocking, and dimensions, or initializes a Selenium-compatible session. ```APIDOC ## POST /v1/sessions ### Description Creates a new browser session. This endpoint allows for fine-grained control over the browser environment, including proxy configuration and viewport dimensions. ### Method POST ### Endpoint /v1/sessions ### Parameters #### Request Body - **proxyUrl** (string) - Optional - The proxy server URL to use for the session. - **blockAds** (boolean) - Optional - Whether to block ads during the session. - **dimensions** (object) - Optional - Viewport dimensions (width, height). - **isSelenium** (boolean) - Optional - Set to true to initialize a Selenium-compatible session. ### Request Example { "proxyUrl": "user:pass@host:port", "blockAds": true, "dimensions": { "width": 1280, "height": 800 } } ### Response #### Success Response (200) - **id** (string) - The unique identifier for the created session. #### Response Example { "id": "session-12345" } ``` -------------------------------- ### Implement Custom CDP Plugin in TypeScript Source: https://github.com/steel-dev/steel-browser/blob/main/CONTRIBUTING.md Demonstrates how to extend the BasePlugin class to hook into browser lifecycle events such as launch, page creation, navigation, and shutdown. It includes the registration of the plugin within the Fastify application context. ```typescript import { BasePlugin, PluginOptions } from './api/src/services/cdp/plugins/core/base-plugin.js'; import { Browser, Page } from 'puppeteer-core'; export class MyCustomPlugin extends BasePlugin { constructor(options: PluginOptions) { super({ name: 'my-custom-plugin', ...options }); } async onBrowserLaunch(browser: Browser): Promise { this.cdpService?.logger.info('Custom plugin: Browser launched'); } async onPageCreated(page: Page): Promise { this.cdpService?.logger.info('Custom plugin: New page created'); await page.setUserAgent('MyCustomBot/1.0'); } async onPageNavigate(page: Page): Promise { const url = page.url(); this.cdpService?.logger.info(`Custom plugin: Navigated to ${url}`); } async onBrowserClose(browser: Browser): Promise { this.cdpService?.logger.info('Custom plugin: Browser closed'); } async onShutdown(): Promise { this.cdpService?.logger.info('Custom plugin: Service shutting down'); } } fastify.cdpService.registerPlugin(new MyCustomPlugin({})); ``` -------------------------------- ### Network Connectivity Testing (Bash) Source: https://github.com/steel-dev/steel-browser/blob/main/docs/TROUBLESHOOTING.md Perform basic network connectivity tests using ping, nslookup, and traceroute to diagnose network-related issues. ```bash # Test network connectivity ping google.com nslookup google.com traceroute google.com ```