### Quickstart: Launch Puppeteer with REPL plugin Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-repl/readme.md Launch Puppeteer and enable the REPL plugin. This example demonstrates how to use the plugin to start interactive REPL sessions with `page` and `browser` instances. The plugin is required and then used with `puppeteer.use()`. ```javascript const puppeteer = require('puppeteer-extra') puppeteer.use(require('puppeteer-extra-plugin-repl')()) puppeteer.launch({ headless: true }).then(async browser => { const page = await browser.newPage() await page.goto('https://example.com') // Start an interactive REPL here with the `page` instance. await page.repl() // Afterwards start REPL with the `browser` instance. await browser.repl() await browser.close() }) ``` -------------------------------- ### Quickstart: Using puppeteer-extra with Stealth and Adblocker Plugins Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/readme.md Demonstrates how to use puppeteer-extra as a drop-in replacement for Puppeteer, integrating the stealth and adblocker plugins. It launches a browser, navigates to specified URLs, takes screenshots, and logs output. This example requires `puppeteer-extra-plugin-stealth` and `puppeteer-extra-plugin-adblocker` to be installed. ```javascript // puppeteer-extra is a drop-in replacement for puppeteer, // it augments the installed puppeteer with plugin functionality. // Any number of plugins can be added through `puppeteer.use()` const puppeteer = require('puppeteer-extra') // Add stealth plugin and use defaults (all tricks to hide puppeteer usage) const StealthPlugin = require('puppeteer-extra-plugin-stealth') puppeteer.use(StealthPlugin()) // Add adblocker plugin to block all ads and trackers (saves bandwidth) const AdblockerPlugin = require('puppeteer-extra-plugin-adblocker') puppeteer.use(AdblockerPlugin({ blockTrackers: true })) // That's it, the rest is puppeteer usage as normal 😊 puppeteer.launch({ headless: true }).then(async browser => { const page = await browser.newPage() await page.setViewport({ width: 800, height: 600 }) console.log(`Testing adblocker plugin..`) await page.goto('https://www.vanityfair.com') await page.waitForTimeout(1000) await page.screenshot({ path: 'adblocker.png', fullPage: true }) console.log(`Testing the stealth plugin..`) await page.goto('https://bot.sannysoft.com') await page.waitForTimeout(5000) await page.screenshot({ path: 'stealth.png', fullPage: true }) console.log(`All done, check the screenshots. ✨`) await browser.close() }) ``` -------------------------------- ### Quickstart: Launch Playwright with Stealth Plugin (TypeScript/ESM) Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/playwright-extra/readme.md Provides a TypeScript example for using playwright-extra with the stealth plugin, leveraging ES Module syntax. It demonstrates importing modules and using the stealth plugin similarly to the JavaScript example. Requires 'playwright-extra-plugin-stealth' to be installed. ```typescript // playwright-extra is a drop-in replacement for playwright, // it augments the installed playwright with plugin functionality import { chromium } from 'playwright-extra' // Load the stealth plugin and use defaults (all tricks to hide playwright usage) // Note: playwright-extra is compatible with most puppeteer-extra plugins import StealthPlugin from 'puppeteer-extra-plugin-stealth' // Add the plugin to playwright (any number of plugins can be added) chromium.use(StealthPlugin()) // ...(the rest of the quickstart code example is the same) chromium.launch({ headless: true }).then(async browser => { const page = await browser.newPage() console.log('Testing the stealth plugin..') await page.goto('https://bot.sannysoft.com', { waitUntil: 'networkidle' }) await page.screenshot({ path: 'stealth.png', fullPage: true }) console.log('All done, check the screenshot. ✨') await browser.close() }) ``` -------------------------------- ### Setup TypeScript Project for Playwright-Extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/playwright-extra/readme.md Guides through setting up a new TypeScript project for use with Playwright-Extra. It covers installing necessary dependencies like TypeScript, Node types, esbuild, and Playwright related packages, along with configuring tsconfig.json and running TypeScript files directly. ```bash # Optional: If you don't have yarn yet npm i --global yarn # Optional: Create new package.json if it's a new project yarn init -y # Add basic typescript dependencies yarn add --dev typescript @types/node esbuild esbuild-register # Bootstrap a tsconfig.json yarn tsc --init --target ES2020 --lib ES2020 --module commonjs --rootDir src --outDir dist # Add dependencies used in the quick start example yarn add playwright playwright-extra puppeteer-extra-plugin-stealth # Create source folder for the .ts files mkdir src # Now place the example code above in `src/index.ts` # Run the typescript code without the need of compiling it first node -r esbuild-register src/index.ts ``` -------------------------------- ### Install puppeteer-extra-plugin-adblocker Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-adblocker/readme.md Installs the adblocker plugin and its core dependencies using either Yarn or npm. This is the initial setup step for using the plugin. ```bash yarn add puppeteer-extra-plugin-adblocker # - or - npm install puppeteer-extra-plugin-adblocker ``` ```bash yarn add puppeteer puppeteer-extra puppeteer-extra-plugin-adblocker # - or - npm install puppeteer puppeteer-extra puppeteer-extra-plugin-adblocker ``` -------------------------------- ### REPL Usage Examples Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-repl/readme.md Examples of interacting with the REPL. These demonstrate common Puppeteer actions like getting the current URL, clicking elements, navigating, typing into input fields, and evaluating JavaScript within the page context, all from within the interactive REPL. ```javascript > page.url() // => https://example.com > page.click('a') > page.url() // => https://www.iana.org/domains/reserved > page.content() // => ... > page.goto('https://google.com') > page.type('input', 'what is the answer to life the universe and everything') > page.click('input[type=submit]') > page.url() // => https://www.google.com/search?source=hp&ei=u9oXW5HpO8a ... > page.evaluate(() => document.querySelector('h3 a').textContent) // => Question 42 (The Impossible Quiz) - The Impossible Quiz Wiki - Fandom ``` -------------------------------- ### Install puppeteer-extra and stealth plugin Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-stealth/readme.md Installs puppeteer, puppeteer-extra, and the stealth plugin together. This is recommended for first-time users of puppeteer-extra plugins. ```bash yarn add puppeteer puppeteer-extra puppeteer-extra-plugin-stealth # - or - pm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth ``` -------------------------------- ### Install puppeteer-extra-plugin-block-resources Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-block-resources/readme.md Installs the puppeteer-extra-plugin-block-resources package using Yarn. This is the first step to integrate the plugin into your project. ```bash yarn add puppeteer-extra-plugin-block-resources ``` -------------------------------- ### Quickstart: Launch Playwright with Stealth Plugin (JavaScript) Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/playwright-extra/readme.md Demonstrates how to use the playwright-extra library with the stealth plugin to hide Playwright's usage. It launches a browser, navigates to a test page, takes a screenshot, and then closes the browser. Requires 'playwright-extra-plugin-stealth' to be installed. ```javascript // playwright-extra is a drop-in replacement for playwright, // it augments the installed playwright with plugin functionality const { chromium } = require('playwright-extra') // Load the stealth plugin and use defaults (all tricks to hide playwright usage) // Note: playwright-extra is compatible with most puppeteer-extra plugins const stealth = require('puppeteer-extra-plugin-stealth')() // Add the plugin to playwright (any number of plugins can be added) chromium.use(stealth) // That's it, the rest is playwright usage as normal 😊 chromium.launch({ headless: true }).then(async browser => { const page = await browser.newPage() console.log('Testing the stealth plugin..') await page.goto('https://bot.sannysoft.com', { waitUntil: 'networkidle' }) await page.screenshot({ path: 'stealth.png', fullPage: true }) console.log('All done, check the screenshot. ✨') await browser.close() }) ``` -------------------------------- ### Install Playwright-Extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/playwright-extra/readme.md Installs the playwright-extra package and playwright itself using either yarn or npm. This is the initial step to use the plugin framework. ```bash yarn add playwright playwright-extra # - or - npm install playwright playwright-extra ``` -------------------------------- ### Install puppeteer-extra-plugin-devtools Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-devtools/readme.md Installs the puppeteer-extra-plugin-devtools package using Yarn. ```bash yarn add puppeteer-extra-plugin-devtools ``` -------------------------------- ### Install puppeteer-extra-plugin-click-and-wait Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-click-and-wait/readme.md Installs the puppeteer-extra-plugin-click-and-wait package using Yarn. This is a prerequisite for using the plugin in your project. ```bash yarn add puppeteer-extra-plugin-click-and-wait ``` -------------------------------- ### Starting and Managing 3proxy Service Source: https://github.com/berstend/puppeteer-extra/wiki/Using-proxies These bash commands demonstrate how to start the 3proxy service using a specified configuration file, monitor its log output in real-time, and reload its configuration by signaling the process. This allows for dynamic updates to proxy settings, such as changing IP addresses by modifying the authentication credentials. ```bash 3proxy /path/to/config-file.cfg ``` ```bash tail -f /tmp/3proxy.log ``` ```bash kill -SIGUSR1 $(cat /tmp/3proxy.pid) ``` -------------------------------- ### Install puppeteer-extra-plugin-font-size Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-font-size/readme.md Installs the puppeteer-extra-plugin-font-size package using Yarn. This is the first step before using the plugin in your project. ```bash yarn add puppeteer-extra-plugin-font-size ``` -------------------------------- ### Install puppeteer-extra-plugin-repl Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-repl/readme.md Install the puppeteer-extra-plugin-repl package using Yarn. This is the first step to integrating the REPL functionality into your Puppeteer project. ```bash yarn add puppeteer-extra-plugin-repl ``` -------------------------------- ### Install puppeteer-extra-plugin-anonymize-ua Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-anonymize-ua/readme.md Install the puppeteer-extra-plugin-anonymize-ua package using yarn. ```bash yarn add puppeteer-extra-plugin-anonymize-ua ``` -------------------------------- ### Install puppeteer-extra-plugin Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin/readme.md This command installs the puppeteer-extra-plugin package using Yarn. It's a prerequisite for developing custom plugins. ```bash yarn add puppeteer-extra-plugin ``` -------------------------------- ### Quickstart: Launch Puppeteer with DevTools Tunnel Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-devtools/readme.md Demonstrates how to use the puppeteer-extra-plugin-devtools to launch a headless Puppeteer browser and create a public tunnel to its DevTools frontend. ```javascript const puppeteer = require('puppeteer-extra') const devtools = require('puppeteer-extra-plugin-devtools')() puppeteer.use(devtools) puppeteer .launch({ headless: true, defaultViewport: null }) .then(async browser => { console.log('Start') const tunnel = await devtools.createTunnel(browser) console.log(tunnel.url) const page = await browser.newPage() await page.goto('https://example.com') console.log('All setup.') }) ``` -------------------------------- ### Plugin Name Convention Example Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin/readme.md Provides an example of how to define the 'name' property for a custom puppeteer-extra plugin, following the recommended convention. ```javascript get name () { return 'anonymize-ua' } ``` -------------------------------- ### Install puppeteer-extra-plugin-flash with Yarn Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-flash/readme.md This snippet shows how to install the puppeteer-extra-plugin-flash package using the yarn package manager. This is the first step to using the plugin in a project. ```bash yarn add puppeteer-extra-plugin-flash ``` -------------------------------- ### Basic Plugin Example with Puppeteer Extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin/readme.md Illustrates how to create a custom plugin ('hello-world-plugin.js') that extends PuppeteerExtraPlugin and how to use it with puppeteer-extra ('foo.js'). The plugin logs page creation events and user agent information. ```javascript // hello-world-plugin.js const { PuppeteerExtraPlugin } = require('puppeteer-extra-plugin') class Plugin extends PuppeteerExtraPlugin { constructor(opts = {}) { super(opts) } get name() { return 'hello-world' } async onPageCreated(page) { this.debug('page created', page.url()) const ua = await page.browser().userAgent() this.debug('user agent', ua) } } module.exports = function(pluginConfig) { return new Plugin(pluginConfig) } // foo.js const puppeteer = require('puppeteer-extra') puppeteer.use(require('./hello-world-plugin')()) ;(async () => { const browser = await puppeteer.launch({ headless: false }) const page = await browser.newPage() await page.goto('http://example.com', { waitUntil: 'domcontentloaded' }) await browser.close() })() ``` -------------------------------- ### Install puppeteer-extra-plugin-user-preferences with Yarn Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-user-preferences/readme.md This snippet shows how to install the puppeteer-extra-plugin-user-preferences package using the Yarn package manager. This is a prerequisite for using the plugin in your project. ```bash yarn add puppeteer-extra-plugin-user-preferences ``` -------------------------------- ### TypeScript and ESM Usage with Proxy Router (Playwright & Puppeteer) Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/plugin-proxy-router/readme.md Demonstrates how to import and use the ProxyRouter plugin in a TypeScript environment using ECMAScript Modules (ESM). Separate examples are provided for Playwright (using `firefox` as an example) and Puppeteer, showing the import syntax for both the browser automation library and the plugin. ```javascript // You can use any browser: chromium, firefox, webkit import { firefox } from 'playwright-extra' import ProxyRouter from '@extra/proxy-router' // ... firefox.use(proxyRouter) ``` ```javascript import puppeteer from 'puppeteer-extra' import ProxyRouter from '@extra/proxy-router' // ... puppeteer.use(proxyRouter) ``` -------------------------------- ### Install puppeteer-extra-plugin-stealth Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-stealth/readme.md Installs the puppeteer-extra-plugin-stealth package using either yarn or npm. This is the initial step to integrate stealth functionalities. ```bash yarn add puppeteer-extra-plugin-stealth # - or - pm install puppeteer-extra-plugin-stealth ``` -------------------------------- ### Configure Single Proxy for All Connections (Playwright) Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/plugin-proxy-router/readme.md This example demonstrates how to configure the ProxyRouter plugin to use a single default proxy for all browser connections. It shows setting up the proxy with authentication and launching a browser to verify the outbound IP address. This is useful for ensuring all traffic goes through a specific proxy. ```javascript // playwright-extra is a drop-in replacement for playwright, // it augments the installed playwright with plugin functionality // Note: Instead of chromium you can use firefox and webkit as well. const { chromium } = require('playwright-extra') // Configure and add the proxy router plugin with a default proxy const ProxyRouter = require('@extra/proxy-router') chromium.use( ProxyRouter({ proxies: { DEFAULT: 'http://user:pass@proxyhost:port' }, }) ) // That's it, the default proxy will be used and proxy authentication handled automatically chromium.launch({ headless: false }).then(async (browser) => { const page = await browser.newPage() await page.goto('https://canhazip.com', { waitUntil: 'domcontentloaded' }) const ip = await page.evaluate('document.body.innerText') console.log('Outbound IP:', ip) await browser.close() }) ``` -------------------------------- ### Install puppeteer-extra and core Puppeteer Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/readme.md Installs the puppeteer-extra library along with the core Puppeteer package. Supports both yarn and npm package managers. You can also specify a particular version of Puppeteer. ```bash yarn add puppeteer puppeteer-extra # - or - npm install puppeteer puppeteer-extra # puppeteer-extra works with any puppeteer version: yarn add puppeteer@2.0.0 puppeteer-extra ``` -------------------------------- ### Dynamic Proxy Routing by Hostname (Playwright) Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/plugin-proxy-router/readme.md This example illustrates advanced proxy routing using the ProxyRouter plugin. It configures multiple proxies and defines a `routeByHost` function to dynamically select which proxy to use for different hostnames, or to block/use direct connections. It also logs connection statistics, including bytes used per proxy and host. ```javascript // playwright-extra is a drop-in replacement for playwright, // it augments the installed playwright with plugin functionality // Note: Instead of chromium you can use firefox and webkit as well. const { chromium } = require('playwright-extra') // Configure the proxy router plugin const ProxyRouter = require('@extra/proxy-router') const proxyRouter = ProxyRouter({ // define the available proxies (replace this with your proxies) proxies: { // the default browser proxy, can be `null` as well for direct connections DEFAULT: 'http://user:pass@proxyhost:port', // optionally define more proxies you can use in `routeByHost` // you can use whatever names you'd like for them DATACENTER: 'http://user:pass@proxyhost2:port', RESIDENTIAL_US: 'http://user:pass@proxyhost3:port', }, // optional function for flexible proxy routing // if this is not specified the `DEFAULT` proxy will be used for all connections routeByHost: async ({ host }) => { if (['pagead2.googlesyndication.com', 'fonts.gstatic.com'].includes(host)) { return 'ABORT' // block connection to certain hosts } if (host.includes('google')) { return 'DIRECT' // use a direct connection for all google domains } if (host.endsWith('.tile.openstreetmap.org')) { return 'DATACENTER' // route heavy images through datacenter proxy } if (host === 'canhazip.com') { return 'RESIDENTIAL_US' // special proxy for this domain } // everything else will use `DEFAULT` proxy }, }) // Add the plugin chromium.use(proxyRouter) // Launch a browser and run some IP checks chromium.launch({ headless: true }).then(async (browser) => { const page = await browser.newPage() await page.goto('https://showmyip.com/', { waitUntil: 'domcontentloaded' }) const ip1 = await page.evaluate("document.querySelector('#ipv4').innerText") console.log('Outbound IP #1:', ip1) // => 77.191.128.0 (the DEFAULT proxy) await page.goto('https://canhazip.com', { waitUntil: 'domcontentloaded' }) const ip2 = await page.evaluate('document.body.innerText') console.log('Outbound IP #2:', ip2) // => 104.179.129.27 (the RESIDENTIAL_US proxy) console.log(proxyRouter.stats.connectionLog) // list of connections (host => proxy name) // { id: 0, proxy: 'DIRECT', host: 'accounts.google.com' }, // { id: 1, proxy: 'DEFAULT', host: 'www.showmyip.com' }, // { id: 2, proxy: 'ABORT', host: 'pagead2.googlesyndication.com' }, // { id: 3, proxy: 'DEFAULT', host: 'unpkg.com' }, // ... console.log(proxyRouter.stats.byProxy) // bytes used by proxy // { // DATACENTER: 441734, // DEFAULT: 125823, // DIRECT: 100457, // RESIDENTIAL_US: 4764, // ABORT: 0 // } console.log(proxyRouter.stats.byHost) // bytes used by host // { // 'a.tile.openstreetmap.org': 150685, // 'c.tile.openstreetmap.org': 147054, // 'b.tile.openstreetmap.org': 143995, // 'unpkg.com': 57621, // 'www.googletagmanager.com': 49572, // 'www.showmyip.com': 40408, // ... await browser.close() }) ``` -------------------------------- ### API: Using repl.repl() standalone Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-repl/readme.md Example of using the `repl.repl()` method directly to create an interactive REPL for a given object. This is useful when the plugin's methods are not automatically attached to Puppeteer instances. ```javascript const repl = require('puppeteer-extra-plugin-repl')() await repl.repl() ``` -------------------------------- ### Usage Example with puppeteer-extra (TypeScript) Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-stealth/readme.md Shows how to integrate the puppeteer-extra library and the stealth plugin using TypeScript. It includes necessary imports and demonstrates launching the browser, navigating, and capturing a screenshot. ```typescript import puppeteer from 'puppeteer-extra' import StealthPlugin from 'puppeteer-extra-plugin-stealth' puppeteer .use(StealthPlugin()) .launch({ headless: true }) .then(async browser => { const page = await browser.newPage() await page.goto('https://bot.sannysoft.com') await page.waitForTimeout(5000) await page.screenshot({ path: 'stealth.png', fullPage: true }) await browser.close() }) ``` -------------------------------- ### Configure DevTools Tunnel with Authentication Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-devtools/readme.md Shows how to initialize the devtools plugin with custom authentication credentials for the public tunnel. This example uses `puppeteer-extra` and the `puppeteer-extra-plugin-devtools`. ```javascript const puppeteer = require('puppeteer-extra') const devtools = require('puppeteer-extra-plugin-devtools')({ auth: { user: 'francis', pass: 'president' } }) puppeteer.use(devtools) puppeteer.launch().then(async browser => { console.log('tunnel url:', (await devtools.createTunnel(browser)).url) // => tunnel url: https://devtools-tunnel-n9aogqwx3d.localtunnel.me }) ``` -------------------------------- ### Local Execution of Stealth Evasion Extraction Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/extract-stealth-evasions/readme.md This sequence of commands outlines the local installation and execution process for the 'extract-stealth-evasions' script. It involves installing project dependencies using Yarn and then running the main Node.js script ('index.js') to generate the 'stealth.min.js' file. ```bash yarn install node index.js ``` -------------------------------- ### Install puppeteer-extra with Plugin Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-recaptcha/readme.md Installs puppeteer, puppeteer-extra, and the puppeteer-extra-plugin-recaptcha together. This command is useful when setting up a new project that requires both browser automation and CAPTCHA solving. ```bash yarn add puppeteer puppeteer-extra puppeteer-extra-plugin-recaptcha # - or - npm install puppeteer puppeteer-extra puppeteer-extra-plugin-recaptcha ``` -------------------------------- ### Usage Example with puppeteer-extra (JavaScript) Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-stealth/readme.md Demonstrates how to use the puppeteer-extra library with the stealth plugin in JavaScript. It launches a headless browser, navigates to a test page, waits, takes a screenshot, and then closes the browser. ```javascript // puppeteer-extra is a drop-in replacement for puppeteer, // it augments the installed puppeteer with plugin functionality const puppeteer = require('puppeteer-extra') // add stealth plugin and use defaults (all evasion techniques) const StealthPlugin = require('puppeteer-extra-plugin-stealth') puppeteer.use(StealthPlugin()) // puppeteer usage as normal puppeteer.launch({ headless: true }).then(async browser => { console.log('Running tests..') const page = await browser.newPage() await page.goto('https://bot.sannysoft.com') await page.waitForTimeout(5000) await page.screenshot({ path: 'testresult.png', fullPage: true }) await browser.close() console.log('All done, check the screenshot. ✨') }) ``` -------------------------------- ### Install Stealth Plugin for Playwright-Extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/playwright-extra/readme.md Installs the puppeteer-extra-plugin-stealth package, which is required to use the stealth plugin with playwright-extra. This can be done using either yarn or npm. ```bash yarn add puppeteer-extra-plugin-stealth # - or - npm install puppeteer-extra-plugin-stealth ``` -------------------------------- ### Install Adblocker and Stealth Plugins for puppeteer-extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/readme.md Installs the necessary plugins for puppeteer-extra, specifically the stealth plugin for evading detection and the adblocker plugin for blocking ads and trackers. Supports both yarn and npm. ```bash yarn add puppeteer-extra-plugin-stealth puppeteer-extra-plugin-adblocker # - or - npm install puppeteer-extra-plugin-stealth puppeteer-extra-plugin-adblocker ``` -------------------------------- ### Use puppeteer-extra-plugin-flash in a JavaScript Project Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-flash/readme.md This JavaScript example demonstrates how to integrate and use the puppeteer-extra-plugin-flash with puppeteer-extra. It launches a browser in non-headless mode and navigates to a page, ensuring Flash content is allowed. ```javascript const puppeteer = require('puppeteer-extra') puppeteer.use(require('puppeteer-extra-plugin-flash')()) ;(async () => { const browser = await puppeteer.launch({ headless: false }) const page = await browser.newPage() await page.goto('http://ultrasounds.com', { waitUntil: 'domcontentloaded' }) })() ``` -------------------------------- ### Create Multiple DevTools Tunnels for Browser Instances Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-devtools/readme.md Illustrates how to create separate public DevTools tunnels for multiple Puppeteer browser instances. This example utilizes `puppeteer-extra` and its `devtools` plugin. ```javascript const puppeteer = require('puppeteer-extra') const devtools = require('puppeteer-extra-plugin-devtools')() devtools.setAuthCredentials('bob', 'swordfish') puppeteer.use(devtools) ;(async () => { const browserFleet = await Promise.all( [...Array(3)].map(slot => puppeteer.launch()) ) for (const [index, browser] of browserFleet.entries()) { const { url } = await devtools.createTunnel(browser) console.info(`Browser ${index}'s devtools frontend can be found at: ${url}`) } })() // => // Browser 0's devtools frontend can be found at: https://devtools-tunnel-fzenb4zuav.localtunnel.me // Browser 1's devtools frontend can be found at: https://devtools-tunnel-qe2t5rghme.localtunnel.me // Browser 2's devtools frontend can be found at: https://devtools-tunnel-pp83sdi4jo.localtunnel.me ``` -------------------------------- ### Take Screenshot with Browserless and Stealth Plugin Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/readme.md This example shows how to integrate puppeteer-extra with the browserless library to take a screenshot of a specific device. It utilizes the stealth plugin for enhanced evasion. Requires 'puppeteer-extra', 'puppeteer-extra-plugin-stealth', and 'browserless' packages. ```javascript const puppeteer = require('puppeteer-extra') const StealthPlugin = require('puppeteer-extra-plugin-stealth') puppeteer.use(StealthPlugin()) const browserless = require('browserless')({ puppeteer }) const saveBufferToFile = (buffer, fileName) => { const wstream = require('fs').createWriteStream(fileName) wstream.write(buffer) wstream.end() } browserless .screenshot('https://bot.sannysoft.com', { device: 'iPhone 6' }) .then(buffer => { const fileName = 'screenshot.png' saveBufferToFile(buffer, fileName) console.log(`your screenshot is here: `, fileName) }) ``` -------------------------------- ### Puppeteer Integration with Proxy Router Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/plugin-proxy-router/readme.md This section shows how to use the @extra/proxy-router plugin with Puppeteer, highlighting the minimal changes required compared to Playwright. It involves changing the import statement and the method calls for plugin registration and browser launching. ```diff - const { chromium } = require('playwright-extra') + const puppeteer = require('puppeteer-extra') // ... - chromium.use(proxyRouter) + puppeteer.use(proxyRouter) // ... - chromium.launch() + puppeteer.launch() // ... ``` -------------------------------- ### Launch Browser with Puppeteer Extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/readme.md Launches a browser instance using puppeteer-extra, augmenting the standard puppeteer.launch method with plugin lifecycle hooks. All registered plugins with a 'beforeLaunch' method will be called to potentially modify launch options before the browser starts. The browser closes when the Node.js process exits. ```javascript const browser = await puppeteer.launch({ headless: false, defaultViewport: null }) ``` -------------------------------- ### Integrate Plugins with Puppeteer-Extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/readme.md Demonstrates how to use the puppeteer-extra library to enhance Puppeteer with plugins. This example shows how to require puppeteer-extra, apply plugins like anonymize-ua and font-size, and then launch a browser instance with these enhancements. It utilizes a common asynchronous pattern for browser automation. ```javascript const puppeteer = require('puppeteer-extra') puppeteer.use(require('puppeteer-extra-plugin-anonymize-ua')()) puppeteer.use( require('puppeteer-extra-plugin-font-size')({ defaultFontSize: 18 }) ) ;(async () => { const browser = await puppeteer.launch({ headless: false }) const page = await browser.newPage() await page.goto('http://example.com', { waitUntil: 'domcontentloaded' }) await browser.close() })() ``` -------------------------------- ### Bypass Bot Detection with Stealth Plugin Source: https://context7.com/berstend/puppeteer-extra/llms.txt Utilizes the `puppeteer-extra-plugin-stealth` to automatically apply 17 evasion techniques, making headless browsers harder to detect. This example demonstrates basic usage, launching a browser, navigating to detection sites, and taking screenshots. ```javascript const puppeteer = require('puppeteer-extra') const StealthPlugin = require('puppeteer-extra-plugin-stealth') // Use all evasions (default) puppeteer.use(StealthPlugin()) const browser = await puppeteer.launch({ headless: true, args: ['--no-sandbox', '--disable-setuid-sandbox'] }) const page = await browser.newPage() // Navigate to bot detection test site await page.goto('https://bot.sannysoft.com') await page.screenshot({ path: 'stealth.png', fullPage: true }) // Test another detection service await page.goto('https://arh.antoinevastel.com/bots/areyouheadless') const detectionResults = await page.evaluate(() => document.body.innerText) console.log('Detection results:', detectionResults) await browser.close() ``` -------------------------------- ### Debug Puppeteer Code with Node.js Inspector Source: https://github.com/berstend/puppeteer-extra/wiki/How-to-debug-puppeteer-and-headless-browsers Demonstrates how to debug Puppeteer code locally using Node.js's `--inspect-brk` flag. Save the provided JavaScript code to a file (e.g., `demo.js`) and run it with the flag. You can then connect to the debugger via `about:inspect` in Chrome. This method allows pausing execution at the start and interacting with browser instances via the console. ```javascript const puppeteer = require('puppeteer-extra') puppeteer.launch().then(async browser => { const page = await browser.newPage() Object.assign(global, {browser, page}) await page.goto('https://example.com') // await browser.close() }) ``` ```bash $ node --inspect-brk demo.js ``` -------------------------------- ### Plugin Class Documentation Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-stealth/evasions/defaultArgs/readme.md Documentation for the Plugin class, which extends PuppeteerExtraPlugin. It handles browser launch arguments to improve browser mimicry. ```APIDOC ## class: Plugin ### Description A CDP driver like puppeteer can make use of various browser launch arguments that are adversarial to mimicking a regular browser and need to be stripped when launching the browser. ### Extends PuppeteerExtraPlugin ### Parameters #### Path Parameters * None #### Query Parameters * None #### Request Body * **opts** (object) - Optional, default `{}` - Options for the plugin. ### Request Example ```json { "opts": {} } ``` ### Response #### Success Response (200) * This class primarily handles internal browser configurations and does not return a direct response in typical API terms. #### Response Example ```json { "message": "Plugin initialized successfully." } ``` ``` -------------------------------- ### Use puppeteer-extra-plugin-user-preferences to Launch Browser Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-user-preferences/readme.md Demonstrates how to integrate and use the puppeteer-extra-plugin-user-preferences. It shows how to require puppeteer-extra, apply the plugin with custom user preferences (e.g., default font size), and then launch the browser with these settings. ```javascript const puppeteer = require('puppeteer-extra') puppeteer.use(require('puppeteer-extra-plugin-user-preferences')({userPrefs: { webkit: { webprefs: { default_font_size: 22 } } }})) const browser = await puppeteer.launch() ``` -------------------------------- ### Launch Puppeteer with Chrome on Linux/Docker Source: https://github.com/berstend/puppeteer-extra/wiki/Using-Google-Chrome-instead-of-Chromium Launches Puppeteer with the stealth plugin, pointing to a Google Chrome executable on Linux or in a Docker environment. It navigates to a test page, checks media type support, and extracts text content. Requires `puppeteer-extra` and `puppeteer-extra-plugin-stealth`. ```javascript const puppeteer = require('puppeteer-extra') puppeteer.use(require('puppeteer-extra-plugin-stealth')()); puppeteer .launch({ executablePath: '/usr/bin/google-chrome', args: ['--no-sandbox'] }).then(async browser => { const page = await browser.newPage() await page.goto('https://w3c-test.org/media-source/mediasource-is-type-supported.html') console.log(await page.evaluate(() => MediaSource.isTypeSupported('audio/aac'))) console.log(await (await (await page.$('#summary')).getProperty('textContent')).jsonValue()) await browser.close() }) ``` -------------------------------- ### Install puppeteer-extra-plugin-recaptcha Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-recaptcha/readme.md Installs the puppeteer-extra-plugin-recaptcha package using either yarn or npm. This is the primary step to add CAPTCHA solving capabilities to your automation projects. ```bash yarn add puppeteer-extra-plugin-recaptcha # - or - npm install puppeteer-extra-plugin-recaptcha ``` -------------------------------- ### Registering Multiple Plugins in Puppeteer-Extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/readme.md Illustrates the chaining of the `.use()` method in puppeteer-extra to register multiple plugins sequentially. This allows for a concise way to add several functionalities to the puppeteer instance before launching it. This pattern is useful for managing multiple extensions efficiently. ```javascript puppeteer.use(plugin1).use(plugin2) ``` -------------------------------- ### Basic Puppeteer Screenshot in TypeScript Source: https://github.com/berstend/puppeteer-extra/wiki/Newbie-Guide-To-Scraping-With-Puppeteer This snippet demonstrates how to launch Puppeteer, navigate to a URL, take a screenshot, and close the browser. It requires the 'puppeteer' package and is written in TypeScript for type safety. ```typescript // We'll only need to import one package for this, make sure you've installed it with `npm install puppeteer/puppeteer`... import Puppeteer from "puppeteer" // We'll start with a self-executing async function. (async () => { // First let's create a new Browser instance. const browser: Puppeteer.Browser = await Puppeteer.launch({headless: false}) // Then we need to instantiate a new Page. const page: Puppeteer.Page = await browser.newPage() // How about we take a quick screenshot of Google? await page.setViewport({ width: 1280, height: 800 }) await page.goto('https://www.google.com') await page.screenshot({ path: 'myscreenshot.png', fullPage: true }) // Always clean up your browser after use. await browser.close() })() ``` -------------------------------- ### Plugin Export Change in v3.0.1 Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin/readme.md Demonstrates the change in export method for puppeteer-extra-plugin starting from version 3.0.1. It shows the 'before' and 'after' ways to require the plugin. ```javascript // Before const PuppeteerExtraPlugin = require('puppeteer-extra-plugin') // After (>= v3.0.1) const { PuppeteerExtraPlugin } = require('puppeteer-extra-plugin') ``` -------------------------------- ### Launch Browser with Puppeteer-extra and Plugins (JavaScript) Source: https://context7.com/berstend/puppeteer-extra/llms.txt Demonstrates how to load puppeteer-extra, register Stealth and Recaptcha plugins, launch a browser instance with these plugins active, and interact with a webpage. It shows accessing plugin data after launching the browser. ```javascript const puppeteer = require('puppeteer-extra') const StealthPlugin = require('puppeteer-extra-plugin-stealth') const RecaptchaPlugin = require('puppeteer-extra-plugin-recaptcha') // Register plugins puppeteer.use(StealthPlugin()) puppeteer.use(RecaptchaPlugin({ provider: { id: '2captcha', token: 'YOUR_API_KEY' } })) // Launch browser with all plugins active const browser = await puppeteer.launch({ headless: true, args: ['--no-sandbox'] }) const page = await browser.newPage() await page.goto('https://example.com') // Access plugin data console.log(puppeteer.plugins) console.log(puppeteer.getPluginData()) await browser.close() ``` -------------------------------- ### Get Available Evasions with Stealth Plugin Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-stealth/readme.md Shows how to retrieve a list of all available evasions provided by the puppeteer-extra stealth plugin. This helps in understanding the different stealth techniques that can be applied. ```javascript const pluginStealth = require('puppeteer-extra-plugin-stealth')() console.log(pluginStealth.availableEvasions) // => Set { 'user-agent', 'console.debug' } puppeteer.use(pluginStealth) ``` -------------------------------- ### Launching Puppeteer with a Local Proxy Source: https://github.com/berstend/puppeteer-extra/wiki/Using-proxies This command launches a Puppeteer browser instance configured to use a local proxy server. The `--proxy-server` flag directs all browser traffic through `localhost:23001`, which is assumed to be managed by a local 3proxy instance. ```bash node your_script.js --proxy-server=localhost:23001 ``` -------------------------------- ### 3proxy Configuration for Proxy Services Source: https://github.com/berstend/puppeteer-extra/wiki/Using-proxies This INI configuration file serves as a starter for 3proxy, acting as a local proxy layer between Puppeteer and a commercial proxy service. It specifies daemon settings, logging, authentication, and how to connect to the external proxy server, requiring IP, Port, User, and Pass to be replaced with actual values. ```ini daemon pidfile /tmp/3proxy.pid maxconn 2048 log /tmp/3proxy.log logformat "L%O %I %T" auth iponly fakeresolve allow * 127.0.0.1 * * parent 1000 http IP PORT USER PASS proxy -p23001 -i127.0.0.1 -a ``` -------------------------------- ### Get DevTools URL for a Specific Page Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-devtools/readme.md Generates a deep link to the DevTools frontend for a specific browser page. It requires a Puppeteer Page object and the DevToolsTunnel instance. Returns a string. ```javascript const page = await browser.newPage() const tunnel = await devtools.createTunnel(browser) console.log(tunnel.getUrlForPage(page)) // => https://devtools-tunnel-bmkjg26zmr.localtunnel.me/devtools/inspector.html?ws(...) ``` -------------------------------- ### Handle Browser Instance after Launch with afterLaunch Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin/readme.md The `afterLaunch` hook is called after a browser instance has been successfully launched. It receives the browser instance and launch options as arguments. Developers should avoid storing browser references directly due to potential multiple instances. ```javascript async afterLaunch (browser, opts) { this.debug('browser has been launched', opts.options) } ``` -------------------------------- ### Stealth Evasion Extraction CLI Help and Options Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/extract-stealth-evasions/readme.md This displays the help message and available options for the 'extract-stealth-evasions' CLI tool. It details flags for version checking, excluding/including specific evasions, listing available evasions, and controlling output minification. ```bash $ npx extract-stealth-evasions -h Usage: extract-stealth-evasions [options] Options: --version Show version number [boolean] -e, --exclude Exclude evasion (repeat for multiple) -i, --include Include evasion (repeat for multiple) -l, --list List available evasions -h, --help Show help [boolean] -m, --minify Minify the output [boolean] [default: true] ``` -------------------------------- ### TypeScript Usage: Block Trackers with puppeteer-extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-adblocker/readme.md Provides a TypeScript example for using the adblocker plugin with Puppeteer. This snippet illustrates enabling tracker blocking and navigating to a web page. ```typescript import puppeteer from 'puppeteer-extra' import Adblocker from 'puppeteer-extra-plugin-adblocker' puppeteer.use(Adblocker({ blockTrackers: true })) puppeteer .launch({ headless: false, defaultViewport: null }) .then(async browser => { const page = await browser.newPage() await page.goto('https://www.vanityfair.com') await page.waitForTimeout(60 * 1000) await browser.close() }) ``` -------------------------------- ### Get Local DevTools Frontend URL Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-devtools/readme.md A convenience function to retrieve the URL for the local DevTools frontend. It takes a Puppeteer Browser instance as input and returns a string representing the URL. Requires 'puppeteer-extra' and 'puppeteer-extra-plugin-devtools'. ```javascript const puppeteer = require('puppeteer-extra') const devtools = require('puppeteer-extra-plugin-devtools')() puppeteer.use(devtools) puppeteer.launch().then(async browser => { console.log(devtools.getLocalDevToolsUrl(browser)) // => http://localhost:55952 }) ``` -------------------------------- ### Launch Browser with Stealth Plugin Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/readme.md This code snippet demonstrates how to launch a browser using puppeteer-extra and apply the stealth plugin to evade detection. It requires the 'puppeteer-extra' and 'puppeteer-extra-plugin-stealth' packages. ```javascript const puppeteer = require('puppeteer-extra') const StealthPlugin = require('puppeteer-extra-plugin-stealth') puppeteer.use(StealthPlugin()) puppeteer.launch({ headless: true }).then(async browser => { console.log("Browser Launched") const page = await browser.newPage() await page.goto('https://bot.sannysoft.com') await page.waitForTimeout(10 * 1000) await browser.close() }) ``` -------------------------------- ### Solving Captchas in iFrames Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-recaptcha/readme.md This example shows how to iterate through child frames of a page and attempt to solve captchas within each frame using `frame.solveRecaptchas()`. This is necessary when captchas are embedded in iframes, extending the default functionality. ```javascript // Loop over all potential frames on that page for (const frame of page.mainFrame().childFrames()) { // Attempt to solve any potential captchas in those frames await frame.solveRecaptchas() } ``` -------------------------------- ### Creating Multiple Independent Playwright Instances with Playwright-Extra Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/playwright-extra/readme.md This example shows how to create multiple independent Playwright instances using `addExtra` from `playwright-extra`. This is useful when you need to apply different plugins or configurations to separate browser instances, as default exports are cached. ```typescript // Use `addExtra` to create a fresh and independent instance import playwright from 'playwright' import { addExtra } from 'playwright-extra' const chromium1 = addExtra(playwright.chromium) const chromium2 = addExtra(playwright.chromium) chromium1.use(onePlugin) chromium2.use(anotherPlugin) // chromium1 and chromium2 are independent ``` -------------------------------- ### Plugin Configuration and Authentication Source: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-devtools/readme.md Configuration options for the puppeteer-extra-plugin-devtools, including setting authentication credentials for the public DevTools page. ```APIDOC ## Plugin Configuration ### Description This plugin requires basic authentication for its public-facing DevTools page. You can configure these credentials using the `opts` during plugin initialization or by using the `setAuthCredentials()` method. If no credentials are provided, the plugin will automatically generate a password and display it in the standard output. ### Method Initialization with options or `setAuthCredentials()` ### Parameters #### Options (`opts`) - **auth** (Object) - Optional. Basic authentication credentials for the public page. - **user** (string) - Optional. Username for authentication. Defaults to 'user'. - **pass** (string) - Optional. Password for authentication. Will be generated if not provided. - **prefix** (Object) - Optional. The prefix to use for the `localtunnel.me` subdomain. Defaults to 'devtools-tunnel'. - **localtunnel** (Object) - Optional. Advanced options to pass directly to the [localtunnel](https://github.com/localtunnel/localtunnel#options) library. ### Request Example (Initialization with Auth) ```javascript const puppeteer = require('puppeteer-extra') const devtools = require('puppeteer-extra-plugin-devtools')({ auth: { user: 'francis', pass: 'president' } }) puppeteer.use(devtools) puppeteer.launch().then(async browser => { console.log('tunnel url:', (await devtools.createTunnel(browser)).url) // => tunnel url: https://devtools-tunnel-n9aogqwx3d.localtunnel.me }) ``` ### Method `setAuthCredentials(user, password)` #### Parameters - **user** (string) - The username for basic authentication. - **password** (string) - The password for basic authentication. ### Request Example (Setting Credentials Separately) ```javascript const puppeteer = require('puppeteer-extra') const devtools = require('puppeteer-extra-plugin-devtools')() devtools.setAuthCredentials('bob', 'swordfish')puppeteer.use(devtools) // ... rest of the quickstart code ``` ```