### Example thermoptic Browser Hook Implementation (JavaScript) Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/README.md This JavaScript snippet demonstrates a basic 'on start' hook for the thermoptic proxy. It logs a success message when the browser start hook is called. The `cdp` object, an instance of a connected browser, is provided to execute browser actions. ```javascript export async function hook(cdp) { console.log(`[STATUS] Browser start hook called successfully!`); } ``` -------------------------------- ### cURL Request Example for Proxy Testing Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/TODO.md This snippet demonstrates a cURL command used to test the proxy functionality with specific headers and a JSON payload. It's useful for validating request handling and fingerprinting mechanisms. ```shell curl --insecure 'https://example.com/?test=test' -v --proxy http://test:test@127.0.0.1:1234 -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.9' -H 'Connection: keep-alive' -H 'Content-Type: application/json' -H 'Origin: http://127.0.0.1:8000' -H 'Referer: http://127.0.0.1:8000/' -H 'Sec-Fetch-Dest: empty' -H 'Sec-Fetch-Mode: cors' -H 'Sec-Fetch-Site: cross-site' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36' -H 'X-Custom-Header: custom-value' -H 'sec-ch-ua: "Not:A-Brand";v="24", "Chromium";v="134"' -H 'sec-ch-ua-mobile: ?0' -H 'sec-ch-ua-platform: "Linux"' --data-raw '{"id":"213123","timestamp":1231441,"value":"hokay"}' ``` -------------------------------- ### Example Thermoptic Onstart Hook Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/tutorials/turnstile/cloudflare-turnstile-bypass.md This snippet represents a proof-of-concept thermoptic hook designed to run on proxy start. While the specific code is not provided, it is referenced as an `onstart.js` file. This hook is responsible for automatically solving Cloudflare's Turnstile CAPTCHA and potentially passing other browser checks, enabling subsequent scraping activities. ```js // This is a placeholder for the actual onstart.js code. // The actual code would be within a file like hooks/onstart.js // and would contain logic to interact with Cloudflare's Turnstile. console.log('Thermoptic onstart hook is running...'); // ... actual CAPTCHA solving logic here ... ``` -------------------------------- ### Docker Compose Setup for Thermoptic Proxy Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/README.md Provides the Docker Compose command to build and start the `thermoptic` proxy service. This command assumes a `docker-compose.yml` file is present in the current directory and sets up a containerized Chrome instance for proxying traffic. ```shell docker compose up --build ``` -------------------------------- ### Example Node.js Hook for Cloudflare Check Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/README.md This example demonstrates how to use a custom Node.js hook to interact with a browser, specifically to bypass Cloudflare's JavaScript browser check before the proxy starts listening. It requires the 'thermoptic' library and assumes a browser environment is available. ```javascript const thermoptic = require("thermoptic"); module.exports = async (proxy) => { const page = await proxy.browser.newPage(); await page.goto("https://example.com"); // Replace with the actual site // Add logic here to interact with Cloudflare's challenge console.log("Cloudflare check completed."); await page.close(); }; ``` -------------------------------- ### Example JA4H Fingerprints (curl vs. Thermoptic) Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/README.md Demonstrates the difference in JA4H (HTTP) fingerprints between a direct `curl` request and a request made through the `thermoptic` proxy. This highlights how `thermoptic` unifies the fingerprint to match a real browser. ```shell $ curl https://ja4db.com/id/ja4h/ ge11nn090000_b6a016211e8a_000000000000_e3b0c44298fc ``` ```shell $ curl --proxy http://username:password@thermoptic:1234 https://ja4db.com/id/ja4h/ ge11cn19enus_f2808f0d04cf_9a10d4221160_7068f58def6e ``` -------------------------------- ### Configure Thermoptic Docker Compose (YAML) Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/tutorials/turnstile/cloudflare-turnstile-bypass.md This YAML configuration snippet shows how to set the `ON_START_HOOK_FILE_PATH` environment variable within the thermoptic service in a `docker-compose.yml` file. This directs the service to load and execute the custom Cloudflare turnstile hook upon startup. ```yaml thermoptic: build: . volumes: - ./ssl:/work/cassl - ./hooks:/work/hooks depends_on: - chrome ports: - 1234:1234 environment: HTTP_PROXY_PORT: 1234 CHROME_DEBUGGING_PORT: 3003 CHROME_DEBUGGING_HOST: chrome PROXY_USERNAME: changeme PROXY_PASSWORD: changeme # Updated to point to our newly-written Cloudflare turnstile hook ON_START_HOOK_FILE_PATH: /work/hooks/onstart.js ``` -------------------------------- ### Scraping with curl through Thermoptic Proxy Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/tutorials/turnstile/cloudflare-turnstile-bypass.md This command demonstrates how to use `curl` to scrape a website protected by Cloudflare, by proxying the request through a locally running `thermoptic` instance. The `thermoptic` hook handles the CAPTCHA solving, simplifying the scraping process without client-side modifications. It requires the `thermoptic` proxy to be running on `127.0.0.1:1234` with provided credentials. ```sh curl --proxy http://changeme:changeme@127.0.0.1:1234 --insecure https://example.com ``` -------------------------------- ### cURL Request to ja4db.com via Proxy Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/TODO.md This cURL command tests accessing a specific URL through the configured proxy. It's used for verifying proxy connectivity and potentially observing how requests are handled or modified. ```shell curl --insecure -v --proxy http://test:test@127.0.0.1:1234 'https://ja4db.com/id/ja4/' ``` -------------------------------- ### Proxying Curl Traffic with Thermoptic Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/README.md Illustrates how to configure `curl` to use the `thermoptic` proxy for making requests. It includes the proxy address, username, password, and an option to bypass SSL certificate verification (`--insecure`). ```shell curl --proxy http://changeme:changeme@127.0.0.1:1234 --insecure https://ja4db.com/id/ja4h/ ``` -------------------------------- ### Workaround for CDP Navigation and Fetch Interception Limitations Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/DOWNSIDES.md This snippet demonstrates a workaround for a limitation in the Chrome Debugging Protocol (CDP) where direct navigation via Page.navigate cannot simultaneously intercept request details with Fetch.requestPaused. It addresses the need to capture requests during an initial tab load, which CDP doesn't natively support for cross-site navigations. The implementation involves 'hacky dark-magic' which is prone to breakage with Chrome updates. ```JavaScript /** * This is a conceptual representation of the 'hacky dark-magic' * mentioned in the text. The actual implementation would involve * intricate interactions with the Chrome Debugging Protocol (CDP). * * The core problem is that `Page.navigate` starts a navigation, * and you want to intercept the *first* request that happens as a result * of that navigation using `Fetch.requestPaused`. However, CDP doesn't * allow setting up `Fetch.enable` *before* the navigation starts and * then immediately receiving the paused event for the initial request. * * A common workaround involves: * 1. Navigating to a blank or intermediate page. * 2. Immediately issuing the desired `Page.navigate` to the target URL. * 3. Quickly enabling `Fetch.enable` with the appropriate request patterns. * 4. Handling the `Fetch.requestPaused` event for the actual request. * 5. Potentially re-navigating or using other CDP commands to ensure * the target page loads correctly after the initial request is intercepted. */ async function manualBrowserVisitWithFetch(page, url) { // This function simulates the logic described in the thermoptic README. // The actual implementation requires detailed CDP command sequences. console.log('Initiating manual browser visit with fetch interception...'); // Step 1: Navigate to a blank page or an intermediate page // This is often necessary to ensure the CDP is in a state where // fetch interception can be reliably set up *before* the target navigation. await page.goto('about:blank'); console.log('Navigated to about:blank'); // Step 2 & 3: Enable fetch interception and then navigate to the target URL // The order here is crucial. Enabling fetch *after* navigation // might miss the initial request. await page.evaluate(() => { // This JavaScript runs in the browser context. // It might involve setting up listeners or triggering navigation // in a way that allows CDP to intercept. console.log('Setting up fetch interception and navigation...'); // In a real scenario, this would involve CDP commands, not direct JS execution like this. // For demonstration, we simulate the intent. }); // Simulate enabling fetch interception via CDP // In reality, you'd use a CDP client library to send `Fetch.enable` command. console.log('Simulating Fetch.enable command via CDP...'); // Example CDP command structure (not actual executable code here): // cdpClient.send('Fetch.enable', { patterns: [{ urlPattern: '*', resourceType: 'Document' }] }); // Simulate the actual navigation that needs to be intercepted // This is where the 'hacky dark-magic' might come into play to ensure // the request is captured correctly. console.log(`Navigating to target URL: ${url}`); // Again, in reality, this would be a CDP command or a carefully orchestrated Puppeteer action. await page.goto(url); // This might be preceded by other CDP commands to ensure interception. // Step 4: Handle Fetch.requestPaused events (conceptual) // The CDP client would have an event listener for 'Fetch.requestPaused'. // This listener would inspect the request details and potentially resume it. console.log('Waiting for Fetch.requestPaused events...'); // Example event handler (conceptual): // cdpClient.on('Fetch.requestPaused', async ({ requestId, request, responseErrorPhrase }) => { // console.log(`Request paused: ${request.url}`); // // ... process request, potentially resume it ... // // cdpClient.send('Fetch.resumeRequest', { requestId }); // }); console.log('Manual browser visit with fetch interception simulated.'); // Note: Actual implementation requires a CDP client and careful state management. } // Example usage (requires a Puppeteer page object and a CDP client setup): // async function run() { // const browser = await puppeteer.launch(); // const page = await browser.newPage(); // // Setup CDP client... // await manualBrowserVisitWithFetch(page, 'https://example.com'); // await browser.close(); // } // run(); ``` -------------------------------- ### Click Cloudflare Turnstile Checkbox (JavaScript) Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/tutorials/turnstile/cloudflare-turnstile-bypass.md This JavaScript code calculates click coordinates within the identified parent div and simulates a mouse click using the CDP Input API. It includes fuzzing for coordinates and delays to mimic human behavior, making detection more difficult. The code retrieves the element's bounding box and calculates a click position within it. ```javascript // Assume targetNodeId, x_top_left, y_top_left, y_bottom_left, x_fuzz, y_fuzz are defined // and rand_int function is available const { model } = await DOM.getBoxModel({ nodeId: targetNodeId }); const click_x = (x_top_left + 25) + x_fuzz; const click_y = ((y_top_left + y_bottom_left) / 2) + y_fuzz; await Input.dispatchMouseEvent({ type: 'mousePressed', x: click_x, y: click_y, button: 'left', clickCount: 1 }); await Input.dispatchMouseEvent({ type: 'mouseReleased', x: click_x, y: click_y, button: 'left', clickCount: 1 }); ``` -------------------------------- ### Find Cloudflare Turnstile Parent Div (JavaScript) Source: https://github.com/mandatoryprogrammer/thermoptic/blob/main/tutorials/turnstile/cloudflare-turnstile-bypass.md This JavaScript code snippet identifies the parent div element that contains the Cloudflare Turnstile iframe. It iterates through all div elements, checks their style attributes for 'display: grid', and sets the targetNodeId once found. This is a crucial first step in programmatically interacting with the CAPTCHA. ```javascript const { nodeIds: divNodeIds } = await DOM.querySelectorAll({ nodeId: documentNodeId, selector: 'div' }); let targetNodeId; for (const nodeId of divNodeIds) { const { attributes } = await DOM.getAttributes({ nodeId }); const attrs = {}; for (let i = 0; i < attributes.length; i += 2) { attrs[attributes[i]] = attributes[i + 1]; } if (attrs.style && attrs.style.includes('display: grid')) { targetNodeId = nodeId; break; } } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.