### Ruby SDK Example Source: https://docs.scraperapi.com/resources/sdks This Ruby example shows how to fetch web content using the ScraperAPI SDK. Install the gem with 'gem install scraperapi'. ```ruby # remember to install the library: gem install scraperapi require "scraper_api" client = ScraperAPI::Client.new("API_KEY") result = client.get("https://example.com/", render: true, country_code: "us").raw_body puts result ``` -------------------------------- ### API Query Example Source: https://docs.scraperapi.com/integrations/llm-integrations/mcp-server/claude-desktop Demonstrates how to query documentation dynamically using an HTTP GET request with the `ask` query parameter for additional information. ```http GET https://docs.scraperapi.com/integrations/llm-integrations/mcp-server/claude-desktop.md?ask= ``` -------------------------------- ### Dynamic Documentation Query Example Source: https://docs.scraperapi.com/control-and-optimization/cached-responses To get information not explicitly present on a page, perform an HTTP GET request with the `ask` query parameter. The response includes a direct answer and relevant excerpts. ```http GET https://docs.scraperapi.com/control-and-optimization/cached-responses.md?ask= ``` -------------------------------- ### Dynamic Documentation Query Example Source: https://docs.scraperapi.com/integrations/llm-integrations/mcp-server/claude-code Perform an HTTP GET request on the current page URL with the 'ask' query parameter to get specific information. The question should be specific, self-contained, and in natural language. ```http GET https://docs.scraperapi.com/integrations/llm-integrations/mcp-server/claude-code.md?ask= ``` -------------------------------- ### Dynamic Documentation Query Example Source: https://docs.scraperapi.com/asynchronous-api/callbacks-and-api-params To get more information not present on the current page, make a GET request to the page URL with the `ask` query parameter. The question should be specific and in natural language. ```http GET https://docs.scraperapi.com/asynchronous-api/callbacks-and-api-params.md?ask= ``` -------------------------------- ### Dynamic Documentation Query Example Source: https://docs.scraperapi.com/scraperapi-crawler-v2.0/crawler-api/callbacks-errors-and-best-practices Use this GET request to query the documentation dynamically. Include your question in the `ask` query parameter for a direct answer and relevant excerpts. ```http GET https://docs.scraperapi.com/scraperapi-crawler-v2.0/crawler-api/callbacks-errors-and-best-practices.md?ask= ``` -------------------------------- ### Create Project with Additional Parameters Source: https://docs.scraperapi.com/data-pipeline/datapipeline-endpoints/how-to-use This example demonstrates creating a project while specifying additional API parameters, such as enabling premium features. ```shellscript curl -X POST --data '{ "projectInput": {"type": "list", "list": ["https://www.amazon.com/AmazonBasics-3-Button-Wired-Computer-1-Pack/dp/B005EJH6RW/"] }, "apiParams": {"premium": true} }' -H 'content-type: application/json' 'https://datapipeline.scraperapi.com/api/projects/?api_key=xxxxxx>' ``` ```json { "id": 522, "name": "Project created at 2024-05-10T16:04:28.263Z", "schedulingEnabled": true, "scrapingInterval": "weekly", "createdAt": "2024-05-10T16:04:28.306Z", "scheduledAt": "2024-05-12T19:00:00.211Z", "projectType": "urls", "apiParams": { "premium": "true" }, "projectInput": { "type": "list", "list": [ "https://www.amazon.com/AmazonBasics-3-Button-Wired-Computer-1-Pack/dp/B005EJH6RW/" ] }, "projectOutput": { "type": "save" }, "notificationConfig": { "notifyOnSuccess": "never", "notifyOnFailure": "with_every_run" } } ``` -------------------------------- ### PHP SDK Example Source: https://docs.scraperapi.com/resources/sdks Use this snippet to make a GET request with rendering enabled and specify a country code. Ensure you have installed the library using 'composer require scraperapi/sdk'. ```php get("https://example.com/", ["render" => true, "country_code" => "us"])->raw_body; echo $result; ``` ``` -------------------------------- ### Get Amazon Offers via NodeJS Source: https://docs.scraperapi.com/structured-data-endpoints/e-commerce/amazon/amazon-offers-api This NodeJS example demonstrates how to retrieve Amazon offers using the fetch API. It parses the JSON response and logs it to the console. Make sure to install node-fetch. ```javascript import fetch from 'node-fetch'; fetch( 'https://api.scraperapi.com/structured/amazon/offers?api_key=API_KEY&asin=ASIN&country_code=COUNTRY_CODE&tld=TLD' ) .then(response => response.json()) .then(data => { console.log(data); }) .catch(error => { console.error(error); }); ``` -------------------------------- ### Create a New Project Source: https://docs.scraperapi.com/data-pipeline/datapipeline-endpoints/how-to-use This example demonstrates how to create a new project with a list of URLs. Scheduling is enabled by default. To configure a scraping interval, add `scrapingInterval` to your request. ```APIDOC ## POST /api/projects ### Description Creates a new project for data scraping. ### Method POST ### Endpoint `https://datapipeline.scraperapi.com/api/projects?api_key=xxxxxx` ### Parameters #### Request Body - **projectInput** (object) - Required - Defines the input for the project, e.g., a list of URLs. - **type** (string) - Required - Type of input, e.g., "list". - **list** (array of strings) - Required - A list of URLs to scrape. - **scrapingInterval** (string) - Optional - Specifies how often the project should run (e.g., "weekly"). See available options [here](/data-pipeline/datapipeline-endpoints/endpoints-and-parameters.md#fields-parameters). ### Request Example ```shell curl -X POST --data '{ "projectInput": {"type": "list", "list": ["https://www.amazon.com/AmazonBasics-3-Button-Wired-Computer-1-Pack/dp/B005EJH6RW/"] } }' -H 'content-type: application/json' 'https://datapipeline.scraperapi.com/api/projects?api_key=xxxxxx' ``` ### Response #### Success Response (200) - **id** (integer) - The unique identifier for the created project. - **name** (string) - The name of the project. - **schedulingEnabled** (boolean) - Indicates if scheduling is enabled. - **scrapingInterval** (string) - The configured scraping interval. - **createdAt** (string) - The timestamp when the project was created. - **projectType** (string) - The type of the project. - **projectInput** (object) - The input configuration for the project. - **projectOutput** (object) - The output configuration for the project. - **notificationConfig** (object) - Notification settings for the project. #### Response Example ```json { "id": 522, "name": "Project created at 2024-05-10T16:04:28.263Z", "schedulingEnabled": true, "scrapingInterval": "weekly", "createdAt": "2024-05-10T16:04:28.306Z", "projectType": "urls", "projectInput": { "type": "list", "list": [ "https://www.amazon.com/AmazonBasics-3-Button-Wired-Computer-1-Pack/dp/B005EJH6RW/" ] }, "projectOutput": { "type": "save" }, "notificationConfig": { "notifyOnSuccess": "never", "notifyOnFailure": "with_every_run" } } ``` ``` -------------------------------- ### Send GET Request with Python Source: https://docs.scraperapi.com/getting-started/quick-start/send-your-first-request Utilize the Python requests library to send a GET request to ScraperAPI. Ensure you have the 'requests' library installed. ```python import requests #Target URL target_url = 'https://www.example.com' # ScraperAPI API Key api_key = 'API_KEY' request_url = f'https://api.scraperapi.com?api_key={api_key}&url={target_url}' response = requests.get(request_url) print(response.text) ``` -------------------------------- ### NodeJS Geo-Targeting Example Source: https://docs.scraperapi.com/control-and-optimization/geotargeting/standard-geo This NodeJS example demonstrates how to target a specific country. Ensure you have 'node-fetch' installed. Replace 'API_KEY' with your actual API key. ```javascript import request from 'node-fetch'; // Replace the value for api_key with your actual API Key. const url = 'https://api.scraperapi.com/?api_key=API_KEY&country_code=us&url=https://example.com/'; request(url) .then(response => { console.log(response); }) .catch(error => { console.error(error); }); ``` -------------------------------- ### Create Project with Webhook Input Source: https://docs.scraperapi.com/data-pipeline/datapipeline-endpoints/how-to-use This example shows how to create a project using webhook input, where the system fetches the list of items from a specified URL. ```APIDOC ## POST /api/projects ### Description Creates a new project using webhook input to dynamically fetch project data. ### Method POST ### Endpoint https://datapipeline.scraperapi.com/api/projects?api_key=xxxxxx ### Request Body - **projectInput** (object) - Required - Defines the input for the project. - **type** (string) - Required - Must be 'webhook_input'. - **url** (string) - Required - The URL from which to fetch the input data. ### Request Example ```json { "projectInput": {"type": "webhook_input", "url": "" } } ``` ### Response #### Success Response (200) - **id** (integer) - The unique identifier for the created project. - **name** (string) - The name of the project. - **schedulingEnabled** (boolean) - Indicates if scheduling is enabled. - **scrapingInterval** (string) - The interval for scraping. - **createdAt** (string) - The timestamp when the project was created. - **projectType** (string) - The type of the project. - **projectInput** (object) - The input configuration for the project. - **notificationConfig** (object) - Notification settings for the project. #### Response Example ```json { "id": 524, "name": "Project created at 2024-05-10T16:04:28.263Z", "schedulingEnabled": true, "scrapingInterval": "weekly", "createdAt": "2024-06-26T12:42:31.654Z", "scheduledAt": "2024-06-26T12:42:31.652Z", "projectType": "urls", "projectInput": { "url": "https://webhook.site/d10de6ed-68fb-4874-9c0f-3c32bab2e1ef", "type": "webhook_input" }, "notificationConfig": { "notifyOnSuccess": "never", "notifyOnFailure": "with_every_run" } } ``` ``` -------------------------------- ### Specifying Parameters During Project Creation Source: https://docs.scraperapi.com/data-pipeline/datapipeline-endpoints/how-to-use This example shows how to create a project while also specifying additional API parameters, such as enabling premium features. ```APIDOC ## POST /api/projects ### Description Creates a new project with specified input and API parameters. ### Method POST ### Endpoint `https://datapipeline.scraperapi.com/api/projects/?api_key=xxxxxx` ### Parameters #### Request Body - **projectInput** (object) - Required - Defines the input for the project, e.g., a list of URLs. - **type** (string) - Required - Type of input, e.g., "list". - **list** (array of strings) - Required - A list of URLs to scrape. - **apiParams** (object) - Optional - Additional parameters for the API, e.g., `{"premium": true}`. ### Request Example ```shell curl -X POST --data '{ "projectInput": {"type": "list", "list": ["https://www.amazon.com/AmazonBasics-3-Button-Wired-Computer-1-Pack/dp/B005EJH6RW/"] }, "apiParams": {"premium": true} }' -H 'content-type: application/json' 'https://datapipeline.scraperapi.com/api/projects/?api_key=xxxxxx>' ``` ### Response #### Success Response (200) - **id** (integer) - The unique identifier for the created project. - **name** (string) - The name of the project. - **schedulingEnabled** (boolean) - Indicates if scheduling is enabled. - **scrapingInterval** (string) - The configured scraping interval. - **createdAt** (string) - The timestamp when the project was created. - **scheduledAt** (string) - The timestamp when the project is scheduled to run. - **projectType** (string) - The type of the project. - **apiParams** (object) - The API parameters configured for the project. - **projectInput** (object) - The input configuration for the project. - **projectOutput** (object) - The output configuration for the project. - **notificationConfig** (object) - Notification settings for the project. #### Response Example ```json { "id": 522, "name": "Project created at 2024-05-10T16:04:28.263Z", "schedulingEnabled": true, "scrapingInterval": "weekly", "createdAt": "2024-05-10T16:04:28.306Z", "scheduledAt": "2024-05-12T19:00:00.211Z", "projectType": "urls", "apiParams": { "premium": "true" }, "projectInput": { "type": "list", "list": [ "https://www.amazon.com/AmazonBasics-3-Button-Wired-Computer-1-Pack/dp/B005EJH6RW/" ] }, "projectOutput": { "type": "save" }, "notificationConfig": { "notifyOnSuccess": "never", "notifyOnFailure": "with_every_run" } } ``` ``` -------------------------------- ### Send GET Request with PHP Source: https://docs.scraperapi.com/getting-started/quick-start/send-your-first-request Use PHP cURL functions to send a GET request to ScraperAPI. This example includes setting options for the cURL request. ```php ``` -------------------------------- ### Get Account Usage Data via NodeJS Source: https://docs.scraperapi.com/account-management/credit-usage This NodeJS example uses the node-fetch library to make a GET request to the /account endpoint and log the response. ```javascript import fetch from 'node-fetch'; fetch('http://api.scraperapi.com/account?api_key=API_KEY') .then(res => res.text()) .then(body => { console.log(body); }) .catch(err => { console.error(err); }); ``` -------------------------------- ### Ruby HTTP Request Example Source: https://docs.scraperapi.com/javascript-rendering/rendering-instruction-set This snippet shows how to create a GET request using Ruby's Net::HTTP library. It's a basic example for making HTTP requests. ```ruby req = Net::HTTP::Get.new(uri) ``` -------------------------------- ### Project Input: List Variant Source: https://docs.scraperapi.com/data-pipeline/datapipeline-endpoints/endpoints-and-parameters Example of configuring project input using a simple list of URLs. ```json {"type": "list", "list": ["", ""] } ``` -------------------------------- ### NodeJS Request for eBay Search API Source: https://docs.scraperapi.com/structured-data-endpoints/e-commerce/ebay/ebay-search-api This NodeJS example demonstrates how to call the eBay Search API using `node-fetch`. Make sure to install `node-fetch` (`npm install node-fetch`). ```javascript import fetch from 'node-fetch'; fetch(`https://api.scraperapi.com/structured/ebay/search/v2?api_key=API_KEY&query=QUERY&country_code=COUNTRY_CODE&tld=TLD`) .then(response => response.json()) .then(data => { console.log(data); }) .catch(error => { console.error(error); }); ``` -------------------------------- ### Create Project with Google Search Type Source: https://docs.scraperapi.com/data-pipeline/datapipeline-endpoints/how-to-use This example demonstrates how to create a new project and specify its type as 'google_search'. It also shows how to define the project input as a list of search terms. ```APIDOC ## POST /api/projects ### Description Creates a new project, allowing specification of project type and input. ### Method POST ### Endpoint https://datapipeline.scraperapi.com/api/projects?api_key=xxxxxx ### Request Body - **name** (string) - Required - The name of the project. - **projectInput** (object) - Required - Defines the input for the project. - **type** (string) - Required - The type of input (e.g., 'list', 'webhook_input'). - **list** (array) - Optional - A list of items (e.g., URLs, search terms) if type is 'list'. - **url** (string) - Optional - The URL for webhook input if type is 'webhook_input'. - **projectType** (string) - Optional - The type of project (e.g., 'google_search', 'urls'). Defaults to 'urls'. ### Request Example ```json { "name": "Google search project", "projectInput": {"type": "list", "list": ["iPhone", "Android"] }, "projectType": "google_search" } ``` ### Response #### Success Response (200) - **id** (integer) - The unique identifier for the created project. - **name** (string) - The name of the project. - **schedulingEnabled** (boolean) - Indicates if scheduling is enabled. - **schedulingInterval** (string) - The interval for scheduling. - **createdAt** (string) - The timestamp when the project was created. - **projectType** (string) - The type of the project. - **projectInput** (object) - The input configuration for the project. - **notificationConfig** (object) - Notification settings for the project. #### Response Example ```json { "id": 525, "name": "Google search project", "schedulingEnabled": true, "schedulingInterval": "weekly", "createdAt": "2024-05-14T14:34:02.784Z", "projectType": "google_search", "projectInput": { "type": "list", "list": [ "iPhone", "Android" ] }, "notificationConfig": { "notifyOnSuccess": "never", "notifyOnFailure": "with_every_run" } } ``` ``` -------------------------------- ### Webhook Output Configuration Source: https://docs.scraperapi.com/data-pipeline/datapipeline-endpoints/endpoints-and-parameters Example of setting up a webhook to receive job results. ```json "webhookOutput": { "url": ""} ``` -------------------------------- ### Basic ScraperAPI Usage in Java Source: https://docs.scraperapi.com/resources/sdks Initialize the client and retrieve the result of a GET request. Remember to install the library first. ```java // remember to install the library: https://search.maven.org/artifact/com.scraperapi/sdk/1.2 package com.example; import com.scraperapi.ScraperApiClient; public class Main { public static void main(String[] args) { ScraperApiClient client = new ScraperApiClient("API_KEY"); String result = client.get("https://example.com/").result(); System.out.println(result); } } ``` -------------------------------- ### Basic ScraperAPI Usage in Node.js Source: https://docs.scraperapi.com/resources/sdks Initialize the client and make a GET request to a URL. Remember to install the library first. ```javascript // remember to install the library: npm install scraperapi-sdk --save import ScraperAPIClient from 'scraperapi-sdk'; const scraperapiClient = new ScraperAPIClient('API_KEY'); scraperapiClient.get('https://example.com/') .then(response => { console.log(response); }) .catch(error => { console.error(error); }); ``` -------------------------------- ### Fetch Reviews for Multiple Products (NodeJS) Source: https://docs.scraperapi.com/structured-data-endpoints/e-commerce/walmart/walmart-reviews-api-async This NodeJS example shows how to use the node-fetch library to make an asynchronous POST request to the Walmart Reviews API. It includes setting up the request body with product details and a webhook callback URL. ```javascript import fetch from 'node-fetch'; const options = { method: 'POST', body: JSON.stringify({ apiKey: 'API_KEY', productIds: ['PRODUCTID1','PRODUCTID2'], country_code: 'COUNTRY_CODE', tld: 'TLD', sort: 'SORT', callback: { type: 'webhook', url: 'YYYYY' }}), headers: { 'Content-Type': 'application/json', }, } fetch('https://async.scraperapi.com/structured/walmart/review', options) .then(response => { response.text().then(text => console.log(text)); }) .catch(error => { console.log(error) }) ``` -------------------------------- ### Basic ScraperAPI Usage in Python Source: https://docs.scraperapi.com/resources/sdks Initialize the client and make a GET request to a URL. Remember to install the library first. ```python # Remember to install the library pip install scraperapi-sdk from scraperapi_sdk import ScraperAPIClient client = ScraperAPIClient('API_KEY') result = client.get(url = 'https://example.com/') print(result) ``` -------------------------------- ### Example: Using ScraperAPI MCP Server with LlamaIndex Source: https://docs.scraperapi.com/integrations/llm-integrations/llamaindex-integration Demonstrates connecting to the ScraperAPI MCP server, loading tools, and creating a LlamaIndex agent. Ensure SCRAPERAPI_KEY and OPENAI_API_KEY environment variables are set. ```python """ Example: Using ScraperAPI MCP Server with LlamaIndex Prerequisites: pip install llama-index llama-index-tools-mcp llama-index-llms-openai Usage: export SCRAPERAPI_KEY="your-scraperapi-key" export OPENAI_API_KEY="your-openai-api-key" python examples/llamaindex_example.py """ import asyncio import os from llama_index.core.agent import FunctionAgent, AgentWorkflow from llama_index.llms.openai import OpenAI from llama_index.tools.mcp import BasicMCPClient, McpToolSpec async def main(): scraperapi_key = os.environ["SCRAPERAPI_KEY"] mcp_url = os.environ.get("MCP_URL", "https://mcp.scraperapi.com/mcp") # Connect to the ScraperAPI MCP server over Streamable HTTP mcp_client = BasicMCPClient( mcp_url, headers={"Authorization": f"Bearer {scraperapi_key}"}, ) # Load all MCP tools as LlamaIndex tools mcp_tool_spec = McpToolSpec(client=mcp_client) tools = await mcp_tool_spec.to_tool_list_async() print(f"Loaded {len(tools)} tools:") for tool in tools: print(f" - {tool.metadata.name}: {tool.metadata.description[:80]}...") # Create an agent with the MCP tools llm = OpenAI(model="gpt-4o") agent = FunctionAgent( name="scraper_agent", description="Agent that uses ScraperAPI tools to search and scrape the web", tools=tools, llm=llm, ) workflow = AgentWorkflow(agents=[agent]) # Example queries queries = [ "Search Google for 'best web scraping tools 2026' and summarize the top results", "Look up the Amazon product page for ASIN B0FFMRH228", "Search Walmart for 'wireless headphones' and list the top 3 results with prices", ] query = queries[1] print(f"\n{'='*60}") print(f"Query: {query}") print(f"{ '='*60}\n") response = await workflow.run(user_msg=query) print(f"\nAgent response:\n{response}") if __name__ == "__main__": asyncio.run(main()) ``` -------------------------------- ### Make a Redfin Listing Search Request (Node.js) Source: https://docs.scraperapi.com/structured-data-endpoints/real-estate/redfin/redfin-listing-search-api-async This Node.js example shows how to send a POST request to the Redfin Listing Search API using `node-fetch`. Make sure to install `node-fetch` if you haven't already (`npm install node-fetch`). ```javascript import fetch from 'node-fetch'; const options = { method: 'POST', body: JSON.stringify({ apiKey: 'API_KEY', url: 'URL', country_code: 'COUNTRY_CODE', tld: 'TLD', callback: { type: 'webhook', url: 'YYYYY' }}), headers: { 'Content-Type': 'application/json', }, } fetch('https://async.scraperapi.com/structured/redfin/search', options) .then(response => { response.text().then(text => console.log(text)); }) .catch(error => { console.log(error) }) ``` -------------------------------- ### Install Claude MCP Skill from GitHub Source: https://docs.scraperapi.com/integrations/llm-integrations/claude-mcp-skill Install the skill directly from its GitHub repository using npx. Ensure Node.js is installed. ```bash npx skills add scraperapi/scraperapi-skills ``` -------------------------------- ### Use Multiple Parameters in ScraperAPI PHP Source: https://docs.scraperapi.com/resources/sdks Pass multiple parameters in the array for the GET request. Remember to install the library first. ```php # remember to install the library: composer require scraperapi/sdk get("https://example.com/", ["render" => true, "premium' => true])->raw_body; echo $result; ``` -------------------------------- ### Basic ScraperAPI Usage in Ruby Source: https://docs.scraperapi.com/resources/sdks Initialize the client and retrieve the raw body of a GET request. Remember to install the library first. ```ruby # remember to install the library: gem install scraperapi require "scraper_api" client = ScraperAPI::Client.new("API_KEY") result = client.get("https://example.com/").raw_body puts result ``` -------------------------------- ### Install ScraperAPI SDK for PHP Source: https://docs.scraperapi.com/resources/sdks Install the PHP SDK using Composer. ```bash composer require scraperapi/sdk ``` -------------------------------- ### Basic ScraperAPI Usage in PHP Source: https://docs.scraperapi.com/resources/sdks Initialize the client and retrieve the raw body of a GET request. Remember to install the library first. ```php # remember to install the library: composer require scraperapi/sdk get("https://example.com/")->raw_body; echo $result; ``` -------------------------------- ### Install scraperapi-mcp-server Source: https://docs.scraperapi.com/resources/release-notes/august-2025 Install the MCP server package using pip. This is the first step to setting up the server. ```bash pip install scraperapi-mcp-server ``` -------------------------------- ### Use Multiple Parameters in ScraperAPI Node.js Source: https://docs.scraperapi.com/resources/sdks Pass multiple parameters in the params object for the GET request. Remember to install the library first. ```javascript // remember to install the library: npm install scraperapi-sdk --save import ScraperAPIClient from 'scraperapi-sdk'; const scraperapiClient = new ScraperAPIClient('API_KEY'); const params = {render: true, country_code: 'us'}; scraperapiClient.get('https://example.com/', params) .then(response => { console.log(response); }) .catch(error => { console.error(error); }); ``` -------------------------------- ### Query Documentation via HTTP GET Request Source: https://docs.scraperapi.com/integrations/llm-integrations/langchain-integration/ai-agent-integration Demonstrates how to dynamically query the documentation by making an HTTP GET request to the page URL with an 'ask' query parameter. This is useful for retrieving specific information or related content not explicitly present on the page. ```http GET https://docs.scraperapi.com/integrations/llm-integrations/langchain-integration/ai-agent-integration.md?ask= ``` -------------------------------- ### Define and Send Rendering Instructions Source: https://docs.scraperapi.com/javascript-rendering/rendering-instruction-set Construct a JSON string representing a sequence of browser actions (input, click, wait) and send it via the 'x-sapi-instruction_set' header. This example targets Wikipedia to search for 'cowboy boots'. ```java String instructionSet = "[{"type":"input","selector":{"type":"css","value":"#searchInput"},"value":"cowboy boots"}," + "{"type":"click","selector":{"type":"css","value":"#search-form button[type=\"submit\"]"}}," + "{"type":"wait_for_selector","selector":{"type":"css","value":"#content"}}]"; HttpsURLConnection conn = (HttpsURLConnection) new URL("https://www.wikipedia.org/").openConnection(); conn.setRequestMethod("GET"); conn.setRequestProperty("x-sapi-render", "true"); conn.setRequestProperty("x-sapi-instruction_set", instructionSet); ``` -------------------------------- ### Use Multiple Parameters in ScraperAPI Python Source: https://docs.scraperapi.com/resources/sdks Pass multiple parameters in the 'params' dictionary for the GET request. Remember to install the library first. ```python # Remember to install the library pip install scraperapi-sdk from scraperapi_sdk import ScraperAPIClient client = ScraperAPIClient('API_KEY') result = client.get(url = 'https://example.com/', params={'render': True, 'premium': True}) print(result) ``` -------------------------------- ### Create Project with Webhook Input Source: https://docs.scraperapi.com/data-pipeline/datapipeline-endpoints/how-to-use Create a project that uses a webhook to dynamically fetch its input list. This allows for updating the list without direct API interaction. ```bash curl -v -X POST --data '{ "projectInput": {"type": "webhook_input", "url": "" } }' -H 'content-type: application/json' 'https://datapipeline.scraperapi.com/api/projects?api_key=xxxxxx' ``` -------------------------------- ### Node.js Request for Redfin 'For Rent' Listings Source: https://docs.scraperapi.com/structured-data-endpoints/real-estate/redfin/redfin-for-rent-listings-api This Node.js example demonstrates how to use the `node-fetch` library to call the Redfin 'For Rent' Listings API. Make sure to install `node-fetch` (`npm install node-fetch`). Replace placeholders with your API key and URL. ```javascript import fetch from 'node-fetch'; fetch( 'https://api.scraperapi.com/structured/redfin/forrent?api_key=API_KEY&url=URL&country_code=COUNTRY_CODE&tld=TLD&raw=RAW' ) .then(response => response.json()) .then(data => { console.log(data); }) .catch(error => { console.error(error); }); ``` -------------------------------- ### Basic Rendering Instructions Source: https://docs.scraperapi.com/javascript-rendering/rendering-instruction-set Use these instructions to simulate user interactions like typing and clicking. Ensure selectors are valid CSS selectors. ```javascript [ { type: "set_selector_value", selector: { type: "css", value: "#searchInput" }, value: "cowboy boots" }, { type: "click", selector: { type: "css", value: "#search-form button[type=\"submit\"]" } }, { type: "wait_for_selector", selector: { type: "css", value: "#content" } } ] ``` -------------------------------- ### Query Documentation Dynamically Source: https://docs.scraperapi.com/asynchronous-api/overview Perform an HTTP GET request to the current page URL with the `ask` query parameter to get direct answers and relevant excerpts from the documentation. ```http GET https://docs.scraperapi.com/asynchronous-api/overview.md?ask= ``` -------------------------------- ### Enable Rendering in ScraperAPI Ruby Source: https://docs.scraperapi.com/resources/sdks Make a GET request with the 'render' parameter set to true to enable rendering. Remember to install the library first. ```ruby # remember to install the library: gem install scraperapi require "scraper_api" client = ScraperAPI::Client.new("API_KEY") result = client.get("https://example.com/", render: true).raw_body puts result ``` -------------------------------- ### Enable Rendering in ScraperAPI PHP Source: https://docs.scraperapi.com/resources/sdks Make a GET request with the 'render' parameter set to true to enable rendering. Remember to install the library first. ```php # remember to install the library: composer require scraperapi/sdk get("https://example.com/", ["render" => true])->raw_body; echo $result; ``` -------------------------------- ### Example Prompt for Lowes.com Source: https://docs.scraperapi.com/integrations/llm-integrations/mcp-server/claude-desktop This example demonstrates a specific prompt to scrape a product URL, including instructions for handling errors with geo-targeting, premium proxies, and enabling JavaScript rendering if necessary. ```markdown "Please scrape this URL [*https://www.lowes.com/pd/Kozyard-12-ft-x-16-ft-Gazebo-Dark-Brown-Metal-Square-Screened-Gazebo-with-Steel-Roof/5014900669*](https://www.lowes.com/pd/Kozyard-12-ft-x-16-ft-Gazebo-Dark-Brown-Metal-Square-Screened-Gazebo-with-Steel-Roof/5014900669)*. If you receive a 500 server error identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding **premium=true**. For persistent failures, activate **ultra_premium=true** to use enhanced anti-blocking measures. Get me the price of the product." ``` -------------------------------- ### Fetch Walmart Product Data (Ruby) Source: https://docs.scraperapi.com/structured-data-endpoints/e-commerce/walmart/walmart-product-api This Ruby script demonstrates how to fetch Walmart product data using the `net/http` library. It constructs the URI with query parameters and prints the website content. ```ruby require 'net/http' require 'json' params = { :api_key => "API_KEY", :product_id => "PRODUCT_ID", :country_code => "COUNTRY_CODE", :tld => "TLD" } uri = URI('https://api.scraperapi.com/structured/walmart/product') uri.query = URI.encode_www_form(params) website_content = Net::HTTP.get(uri) print(website_content) ``` -------------------------------- ### Enable Rendering in ScraperAPI Node.js Source: https://docs.scraperapi.com/resources/sdks Make a GET request with the 'render' parameter set to true to enable rendering. Remember to install the library first. ```javascript // remember to install the library: npm install scraperapi-sdk --save import ScraperAPIClient from 'scraperapi-sdk'; const scraperapiClient = new ScraperAPIClient('API_KEY'); scraperapiClient.get('https://example.com/', {render: true}) .then(response => { console.log(response); }) .catch(error => { console.error(error); }); ``` -------------------------------- ### Perform HTTP GET Request with 'ask' Parameter Source: https://docs.scraperapi.com/structured-data-endpoints/e-commerce/amazon/amazon-search-api Use this method to query documentation dynamically. The question should be specific and self-contained. The response includes a direct answer and relevant excerpts. ```bash GET https://docs.scraperapi.com/structured-data-endpoints/e-commerce/amazon/amazon-search-api.md?ask= ``` -------------------------------- ### Enable Rendering in ScraperAPI Python Source: https://docs.scraperapi.com/resources/sdks Make a GET request with the 'render' parameter set to True to enable rendering. Remember to install the library first. ```python # Remember to install the library pip install scraperapi-sdk from scraperapi_sdk import ScraperAPIClient client = ScraperAPIClient('API_KEY') result = client.get(url = 'https://example.com/', params={'render': True}) print(result) ``` -------------------------------- ### NodeJS Synchronous Request with Autoparse Source: https://docs.scraperapi.com/responses-and-formats/output-formats/json-response-autoparse This NodeJS example uses 'node-fetch' to send a request to ScraperAPI with the autoparse parameter enabled. Make sure to install 'node-fetch'. ```javascript import request from 'node-fetch'; // Replace the value for api_key with your actual API Key. const url = 'http://api.scraperapi.com/?api_key=API_KEY&autoparse=true&url=https://www.amazon.com/dp/B07V1PHM66'; request(url) .then(response => { console.log(response); }) .catch(error => { console.error(error); }); ``` -------------------------------- ### Query Documentation Dynamically Source: https://docs.scraperapi.com/account-management/api-key Perform an HTTP GET request to query the documentation dynamically. Use the 'ask' query parameter with a specific question to get direct answers and relevant excerpts. ```bash GET https://docs.scraperapi.com/account-management/api-key.md?ask= ``` -------------------------------- ### Get Account Usage Data via Java Source: https://docs.scraperapi.com/account-management/credit-usage This Java example demonstrates how to use the built-in HttpClient to retrieve your account usage information from the /account endpoint. ```java import java.net.URI; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; public class Main { public static void main(String[] args) throws Exception { // Replace the value for api_key with your actual API Key String apiKey = "API_KEY"; String scraperApiUrl = "https://api.scraperapi.com/account?api_key=" + apiKey; HttpClient client = HttpClient.newHttpClient(); HttpRequest request = HttpRequest.newBuilder() .uri(URI.create(scraperApiUrl)) .GET() .build(); HttpResponse response = client.send(request, HttpResponse.BodyHandlers.ofString()); System.out.println(response.body()); } } ```