### ZenRows Universal Scraper API Getting Started Source: https://context7_llms Guides for getting started with the ZenRows Universal Scraper API, including making your first request and setting up integrations with Node.js and Python. ```APIDOC ZenRows Universal Scraper API Getting Started: Make Your First Request: Description: A step-by-step guide to making your initial request to the Universal Scraper API. Usage: Typically involves setting up your API key and target URL. Get Started with ZenRows® Universal Scraper API in Node.js: Description: Instructions for setting up and using the Universal Scraper API within a Node.js environment. Usage: Includes installation and basic usage examples for Node.js. ZenRows® Universal Scraper API with Python: Getting Started: Description: A guide to integrating and using the Universal Scraper API with Python. Usage: Covers installation, API key setup, and making requests using Python. ``` -------------------------------- ### ZenRows Product and Pricing Guidance Source: https://docs.zenrows.com/first-steps/getting-started-guide Provides guidance on selecting the appropriate ZenRows product and plan. It directs users to detailed product documentation for feature comparison and a pricing guide for plan selection. ```markdown Check our [Product Documentation](/first-steps/our-products#best-product-for-your-use-case) to compare features and find the best fit for your use case, or review our [Pricing Guide](/first-steps/pricing#choosing-the-right-plan-for-your-needs) to select the right plan. ``` -------------------------------- ### ZenRows API Request Examples Source: https://docs.zenrows.com/first-steps/getting-started-guide Demonstrates how to make a scraping request to the ZenRows API using various programming languages. Ensure you replace 'YOUR_ZENROWS_API_KEY' with your actual API key. ```python ## pip install requests import requests url = 'https://httpbin.io/anything' apikey = 'YOUR_ZENROWS_API_KEY' params = { 'url': url, 'apikey': apikey, } response = requests.get('https://api.zenrows.com/v1/', params=params) print(response.text) ``` ```javascript // npm install axios const axios = require('axios'); const url = 'https://httpbin.io/anything'; const apikey = 'YOUR_ZENROWS_API_KEY'; axios({ url: 'https://api.zenrows.com/v1/', method: 'GET', params: { 'url': url, 'apikey': apikey, }, }) .then(response => console.log(response.data)) .catch(error => console.log(error)); ``` ```java import org.apache.hc.client5.http.fluent.Request; public class APIRequest { public static void main(final String... args) throws Exception { String apiUrl = "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fanything"; String response = Request.get(apiUrl) .execute().returnContent().asString(); System.out.println(response); } } ``` ```php ``` ```go package main import ( "io" "log" "net/http" ) func main() { client := &http.Client{} req, err := http.NewRequest("GET", "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fanything", nil) resp, err := client.Do(req) if err != nil { log.Fatalln(err) } defer resp.Body.Close() body, err := io.ReadAll(resp.Body) if err != nil { log.Fatalln(err) } log.Println(string(body)) } ``` ```ruby # gem install faraday require 'faraday' url = URI.parse('https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fanything') conn = Faraday.new() conn.options.timeout = 180 res = conn.get(url, nil, nil) print(res.body) ``` ```bash curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fanything" ``` -------------------------------- ### HTML Content Example Source: https://docs.zenrows.com/universal-scraper-api/python An example of the HTML content that might be returned after a successful ZenRows scrape, including basic meta information. ```html Asana Reviews 2024: Details, Pricing, & Features | G2 ``` -------------------------------- ### ZenRows Residential Proxies: Getting Started Source: https://context7_llms This section covers the initial steps for using ZenRows Residential Proxies. It includes guides on making your first request, setting up the proxies, and using them with popular programming languages like Node.js (Axios) and Python (Requests). ```APIDOC Residential Proxies: Getting Started: - First Request: Make Your First Request with ZenRows Residential Proxies URL: https://docs.zenrows.com/residential-proxies/get-started/first-request.md - Node.js (Axios): Using Residential Proxies with Node.js (Axios) URL: https://docs.zenrows.com/residential-proxies/get-started/nodejs-axios.md - Python (Requests): Using ZenRows® Residential Proxies with Python (Requests) URL: https://docs.zenrows.com/residential-proxies/get-started/python-requests.md Setup: - Setup Guide: Setting Up the Residential Proxies URL: https://docs.zenrows.com/residential-proxies/residential-proxies-setup.md Features: - Country Targeting: Country Targeting with ZenRows Residential Proxies URL: https://docs.zenrows.com/residential-proxies/features/country.md - Sticky Sessions & Protocol Support: Sticky Sessions and Protocol Support with ZenRows Residential Proxies URL: https://docs.zenrows.com/residential-proxies/features/protocol-sticky-ttl.md - World Region Targeting: World Region Targeting with ZenRows Residential Proxies URL: https://docs.zenrows.com/residential-proxies/features/world-region.md Introduction: - Introduction: Introduction to Residential Proxies URL: https://docs.zenrows.com/residential-proxies/introduction.md ``` -------------------------------- ### Go Development Environment Setup Source: https://docs.zenrows.com/universal-scraper-api/first-request Installs Go (1.16+), the recommended version for ZenRows development. Provides installation commands for macOS, Ubuntu/Debian, and Windows. ```bash # Install Go (if not already installed) # Visit https://golang.org/dl/ or use package managers: # macOS (using Homebrew) brew install go # Ubuntu/Debian sudo apt update && sudo apt install golang-go # Windows (using Chocolatey) choco install golang ``` -------------------------------- ### Complete Example with ZenRows SDK Source: https://docs.zenrows.com/scraping-browser/get-started/playwright A comprehensive example demonstrating the setup and usage of the ZenRows Playwright SDK. It includes installation instructions, SDK initialization, browser connection, page navigation, title retrieval, and browser closing. ```javascript // npm install @zenrows/browser-sdk playwright const { chromium } = require('playwright'); const { ScrapingBrowser } = require('@zenrows/browser-sdk'); const scraper = async () => { // Initialize SDK const scrapingBrowser = new ScrapingBrowser({ apiKey: 'YOUR_ZENROWS_API_KEY' }); const connectionURL = scrapingBrowser.getConnectURL(); const browser = await chromium.connectOverCDP(connectionURL); const page = await browser.newPage(); await page.goto('https://www.scrapingcourse.com/ecommerce/'); console.log(await page.title()); await browser.close(); }; scraper(); ``` -------------------------------- ### Web Scraping Fundamentals Learning Resources Source: https://docs.zenrows.com/first-steps/getting-started-guide Lists essential learning resources for web scraping, covering best practices, Python implementation, use cases, handling JavaScript-heavy websites, and e-commerce data collection. ```markdown ### Learn web scraping fundamentals Build your knowledge with comprehensive guides and best practices: * [Web Scraping Best Practices](https://www.zenrows.com/blog/web-scraping-best-practices) - Essential techniques for reliable data collection, rate limiting strategies, and ethical scraping guidelines. * [Complete Python Web Scraping Tutorial](https://www.zenrows.com/blog/web-scraping-python) - Step-by-step implementation guide covering BeautifulSoup, requests, and data processing. * [Web Scraping Use Cases Guide](https://www.zenrows.com/blog/7-use-cases-for-website-scraping) - Real-world applications including price monitoring, lead generation, market research, and competitive analysis. * [Handling JavaScript-Heavy Websites](https://www.zenrows.com/blog/scraping-javascript) - Advanced techniques for single-page applications and dynamic content. * [E-commerce Data Collection](https://www.zenrows.com/blog/ecommerce-scraping) - Strategies for product information, pricing, and inventory tracking. ``` -------------------------------- ### Install Python Requests Library Source: https://docs.zenrows.com/universal-scraper-api/python Installs the 'requests' library, which is essential for making HTTP requests to the ZenRows API from Python scripts. This library simplifies sending requests and handling responses. ```bash pip install requests ``` -------------------------------- ### ZenRows Language Support Source: https://docs.zenrows.com/first-steps/getting-started-guide ZenRows works with any programming language that can make HTTP requests. We provide code examples for Python, JavaScript/Node.js, Java, PHP, Go, Ruby, and cURL. The API is language-agnostic and follows standard REST principles. ```Python import zenrows client = zenrows.ZenRows("YOUR_API_KEY") params = { "url": "https://httpbin.org/ip", "js_render": "true" } response = client.get(params) print(response.text) ``` ```JavaScript const ZenRows = require('zenrows'); const client = new ZenRows('YOUR_API_KEY'); const params = { url: 'https://httpbin.org/ip', js_render: 'true' }; client.get(params).then(response => { console.log(response.data); }).catch(error => { console.error(error); }); ``` ```cURL curl -s "https://api.zenrows.com/v1/?apikey=YOUR_API_KEY&url=https://httpbin.org/ip&js_render=true" ``` -------------------------------- ### ZenRows API Request Examples Source: https://docs.zenrows.com/universal-scraper-api/first-request Code examples demonstrating how to make a GET request to the ZenRows API using various programming languages. Replace 'YOUR_ZENROWS_API_KEY' with your actual API key. ```python # pip install requests import requests url = 'https://httpbin.io/get' apikey = 'YOUR_ZENROWS_API_KEY' params = { 'url': url, 'apikey': apikey, } response = requests.get('https://api.zenrows.com/v1/', params=params) print(response.text) ``` ```javascript // npm install axios const axios = require('axios'); const url = 'https://httpbin.io/get'; const apikey = 'YOUR_ZENROWS_API_KEY'; axios({ url: 'https://api.zenrows.com/v1/', method: 'GET', params: { 'url': url, 'apikey': apikey, }, }) .then(response => console.log(response.data)) .catch(error => console.log(error)); ``` ```java import org.apache.hc.client5.http.fluent.Request; public class APIRequest { public static void main(final String... args) throws Exception { String apiUrl = "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fget"; String response = Request.get(apiUrl) .execute().returnContent().asString(); System.out.println(response); } } ``` ```php ``` ```go package main import ( "io" "log" "net/http" ) func main() { client := &http.Client{} req, err := http.NewRequest("GET", "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fget", nil) resp, err := client.Do(req) if err != nil { log.Fatalln(err) } defer resp.Body.Close() body, err := io.ReadAll(resp.Body) if err != nil { log.Fatalln(err) } log.Println(string(body)) } ``` ```ruby # gem install faraday require 'faraday' url = URI.parse('https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fget') conn = Faraday.new() conn.options.timeout = 180 res = conn.get(url, nil, nil) print(res.body) ``` ```bash curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fget" ``` -------------------------------- ### Node.js Development Environment Setup Source: https://docs.zenrows.com/universal-scraper-api/first-request Installs Node.js (v18+) and the 'axios' library, essential for ZenRows in Node.js projects. Provides installation commands for macOS, Ubuntu/Debian, and Windows. ```bash # Install Node.js (if not already installed) # Visit https://nodejs.org/ or use package managers: # macOS (using Homebrew) brew install node # Ubuntu/Debian (using NodeSource) curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash - sudo apt-get install -y nodejs # Windows (using Chocolatey) choco install nodejs # Install the axios library npm install axios ``` -------------------------------- ### ZenRows Product Switching Source: https://docs.zenrows.com/first-steps/getting-started-guide Each ZenRows product has its own endpoint and documentation. You can use multiple products with the same API key. Simply change the endpoint URL and parameters based on which product you want to use for each request. ```APIDOC Switching ZenRows Products: To use the core proxy service: GET https://api.zenrows.com/v1/ ?url= &apikey= To use the Stealth Plugin: GET https://api.zenrows.com/v1/ ?url= &stealth=true &apikey= ``` -------------------------------- ### Python Development Environment Setup Source: https://docs.zenrows.com/universal-scraper-api/first-request Installs Python 3 and the 'requests' library, recommended for ZenRows integration. Includes commands for macOS, Ubuntu/Debian, and Windows. ```bash # Install Python (if not already installed) # Visit https://www.python.org/downloads/ or use package managers: # macOS (using Homebrew) brew install python # Ubuntu/Debian sudo apt update && sudo apt install python3 python3-pip # Windows (using Chocolatey) choco install python # Install the requests library pip install requests ``` -------------------------------- ### Simulate User Interactions with JavaScript Source: https://docs.zenrows.com/universal-scraper-api/python Employ `js_render` and `js_instructions` to mimic user actions like clicking buttons or filling forms. The JavaScript instructions guide provides details on interacting with web pages. ```APIDOC Request Parameters: js_render: boolean Enables JavaScript rendering. js_instructions: array of strings A list of JavaScript commands to execute on the page, e.g., `click_css_selector`, `fill_input_css_selector`. ``` -------------------------------- ### HTTPBin.io GET Request Response Source: https://docs.zenrows.com/universal-scraper-api/first-request Example JSON response from a GET request to HTTPBin.io/get, showing arguments, headers, origin, and URL. This demonstrates a basic successful web scraping request. ```json { "args": {}, "headers": { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36", // additional headers omitted for brevity... }, "origin": "38.154.5.224:6693", "url": "http://httpbin.io/get" } ``` -------------------------------- ### Java Development Environment Setup and Dependency Source: https://docs.zenrows.com/universal-scraper-api/first-request Installs Java (8+) and adds the 'httpclient5-fluent' dependency for Maven projects, enabling ZenRows integration in Java applications. ```bash # Install Java (if not already installed) # Visit https://adoptium.net/ or use package managers: # macOS (using Homebrew) brew install openjdk # Ubuntu/Debian sudo apt update && sudo apt install openjdk-17-jdk # Windows (using Chocolatey) choco install openjdk ``` ```xml org.apache.httpcomponents.client5 httpclient5-fluent 5.2.1 ``` -------------------------------- ### Basic ZenRows Request Source: https://docs.zenrows.com/universal-scraper-api/python Demonstrates a basic request to scrape a URL using ZenRows with JavaScript rendering and premium proxies. ```python import requests url = 'https://www.g2.com/products/asana/reviews' apikey = 'YOUR_ZENROWS_API_KEY' params = { 'url': url, 'apikey': apikey, 'js_render': 'true', 'premium_proxy': 'true', } response = requests.get('https://api.zenrows.com/v1/', params=params) print(response.text) ``` -------------------------------- ### ZenRows API Key Security Source: https://docs.zenrows.com/first-steps/getting-started-guide Your API key is unique to your account and should be kept secure. Never share it publicly or commit it to version control. Use environment variables in your code and regenerate your key if you suspect it's been compromised. ```APIDOC Handling API Keys: Best Practice: Use environment variables. Example (Python): import os api_key = os.environ.get('ZENROWS_API_KEY') url = "https://httpbin.org/ip" params = { "apikey": api_key, "url": url } # Make request using the api_key ``` -------------------------------- ### Integrate ZenRows with Python Requests and BeautifulSoup Source: https://docs.zenrows.com/universal-scraper-api/python Combine ZenRows with Python's Requests library and BeautifulSoup for efficient HTML parsing. Refer to the integration guide for setup and usage details. ```Python import requests from bs4 import BeautifulSoup url = 'YOUR_TARGET_URL' api_key = 'YOUR_ZENROWS_API_KEY' params = { 'apikey': api_key, 'css_extractor': 'YOUR_CSS_SELECTOR' } response = requests.get(url, params=params) if response.status_code == 200: soup = BeautifulSoup(response.text, 'html.parser') # Process the extracted content using BeautifulSoup extracted_data = soup.select_one('YOUR_CSS_SELECTOR').get_text() print(extracted_data) else: print(f"Error: {response.status_code}") ``` -------------------------------- ### ZenRows Solutions Overview Source: https://docs.zenrows.com/first-steps/getting-started-guide Presents ZenRows' different scraping solutions categorized by their best use cases, including Universal Scraper API, Scraper APIs, Scraping Browser, Residential Proxies, and Platform Integrations. Each solution is briefly described with its primary application. ```markdown ### Choose your ZenRows solution Select the approach that best fits your scraping requirements and technical expertise: } href="https://docs.zenrows.com/universal-scraper-api/introduction"> **Best for**: General web scraping with automatic optimization } href="https://docs.zenrows.com/scraper-apis/introduction"> **Best for**: Structured data from specific popular websites } href="https://docs.zenrows.com/scraping-browser/introduction"> **Best for**: Complex automation requiring complete browser control } href="https://docs.zenrows.com/residential-proxies/introduction"> **Best for**: High-quality proxies with existing infrastructure } href="https://docs.zenrows.com/integrations/overview"> **Best for**: No-code solutions and workflow automation ``` -------------------------------- ### Install Axios for Node.js Source: https://docs.zenrows.com/universal-scraper-api/nodejs Installs the Axios HTTP client library for Node.js using npm. This library is used to make requests to the ZenRows API. ```bash npm install axios ``` -------------------------------- ### Amazon Product Information API Example Source: https://docs.zenrows.com/scraper-apis/scraper-apis-setup Provides a specific example of a ZenRows API endpoint for retrieving Amazon product information using the product's ASIN. ```bash https://ecommerce.api.zenrows.com/v1/targets/amazon/products/{asin} ``` -------------------------------- ### Example ZenRows API Response Source: https://docs.zenrows.com/universal-scraper-api/nodejs An example of the JSON response received from the ZenRows API after a successful scraping request. It includes details about the request, such as headers and origin IP. ```json { "args": {}, "headers": { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36", // additional headers omitted for brevity... }, "origin": "38.154.5.224:6693", "url": "http://httpbin.io/get" } ``` -------------------------------- ### ZenRows API Response Example Source: https://docs.zenrows.com/universal-scraper-api/python An example of a successful response from the ZenRows API when scraping a simple URL. It includes details like the origin IP address and headers, showcasing how ZenRows manages request anonymity. ```json { "args": {}, "headers": { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36", // additional headers omitted for brevity... }, "origin": "38.154.5.224:6693", "url": "http://httpbin.io/get" } ``` -------------------------------- ### API Request Example Source: https://docs.zenrows.com/scraper-apis/get-started/amazon-asin An example of how to request product information from the Amazon API using a bash command. This demonstrates the structure of the API call with placeholders for ASIN and API key. ```bash https://ecommerce.api.zenrows.com/v1/targets/amazon/products/{asin}?apikey=YOUR_ZENROWS_API_KEY ``` -------------------------------- ### Example HTML Response After Successful Scraping Source: https://docs.zenrows.com/universal-scraper-api/nodejs This is an example of the HTML content received after successfully scraping a complex website using ZenRows with JavaScript rendering and premium proxies enabled. It shows a typical HTML document structure. ```html Jira Reviews 2024: Details, Pricing, & Features | G2 ``` -------------------------------- ### ZenRows API Request Examples Source: https://docs.zenrows.com/universal-scraper-api/universal-scraper-api-setup Demonstrates how to make requests to the ZenRows API using your API key in various programming languages. Ensure you replace 'YOUR_ZENROWS_API_KEY' with your actual key. ```Python import requests url = 'https://httpbin.io/anything' apikey = 'YOUR_ZENROWS_API_KEY' params = { 'url': url, 'apikey': apikey, } response = requests.get('https://api.zenrows.com/v1/', params=params) print(response.text) ``` ```Node.js const axios = require('axios'); const url = 'https://httpbin.io/anything'; const apikey = '78dcb01899e0a73e1428e4ebe8be8f9941c75c0f'; axios({ url: 'https://api.zenrows.com/v1/', method: 'GET', params: { 'url': url, 'apikey': apikey, }, }) .then(response => console.log(response.data)) .catch(error => console.log(error)); ``` ```Java import org.apache.hc.client5.http.fluent.Request; public class APIRequest { public static void main(final String... args) throws Exception { String apiUrl = "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fanything"; String response = Request.get(apiUrl) .execute().returnContent().asString(); System.out.println(response); } } ``` ```PHP ``` ```Go package main import ( "io" "log" "net/http" ) func main() { client := &http.Client{} req, err := http.NewRequest("GET", "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fanything", nil) resp, err := client.Do(req) if err != nil { log.Fatalln(err) } defer resp.Body.Close() body, err := io.ReadAll(resp.Body) if err != nil { log.Fatalln(err) } log.Println(string(body)) } ``` ```Ruby # gem install faraday require 'faraday' url = URI.parse('https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fanything') conn = Faraday.new() conn.options.timeout = 180 res = conn.get(url, nil, nil) print(res.body) ``` ```cURL curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fhttpbin.io%2Fanything" ``` -------------------------------- ### Handle Failed Requests with Retries Source: https://docs.zenrows.com/universal-scraper-api/python Configure retry logic to manage failed HTTP requests. Refer to the guide on retrying requests for detailed instructions. ```APIDOC Retry Logic: Configure retry parameters as per the ZenRows documentation to automatically handle transient HTTP request failures. ``` -------------------------------- ### ZenRows Residential Proxy Setup Steps Source: https://docs.zenrows.com/residential-proxies/residential-proxies-setup Outlines the essential steps for setting up ZenRows Residential Proxies, from account registration to credential generation and proxy configuration. ```html To begin, you'll need to create a ZenRows account. Visit the [Registration Page](https://app.zenrows.com/register) to sign up. If you already have an account, simply log in to access your dashboard. A credentials pair (user and password) will be created for you automatically, this step is not mandatory. Once you're signed in, navigate to the [Credentials Page](https://app.zenrows.com/proxies/credentials) under the Residential Proxies section. Here, you can create credentials by specifying your proxy quota and generating your unique and password. You can create as many credentials as needed, allowing you to manage multiple projects easily! res_proxy_credentials_generator After generating your credentials, you'll be taken to the [Generator Page](https://app.zenrows.com/proxies/generator), where your credentials — username, password, and various configuration options — will be displayed. You can copy the Proxy URL and start integrating it into your applications. roxy_generator ``` -------------------------------- ### Node.js and Cheerio Integration Source: https://docs.zenrows.com/universal-scraper-api/nodejs Integrate ZenRows with Node.js and Cheerio for efficient web scraping and HTML parsing. The provided guide details how to combine these tools effectively. ```APIDOC Integration: Node.js with Cheerio Description: Combines ZenRows with Node.js and Cheerio for enhanced web scraping capabilities. Documentation: /universal-scraper-api/help/how-to-extract-data#javascript-with-cheerio ``` -------------------------------- ### Complete Example: ZenRows Residential Proxies (Python) Source: https://docs.zenrows.com/residential-proxies/get-started/first-request A complete Python script demonstrating how to set up and use ZenRows Residential Proxies for web scraping. It includes proxy credential configuration and making a request to retrieve the IP address. ```python # pip3 install requests import requests # define your proxy credentials proxy_username = "PROXY_USERNAME" proxy_password = "PROXY_PASSWORD" proxy_domain = "superproxy.zenrows.com" proxy_port = "1337" # build the proxy URL proxy_url = f"http://{proxy_username}:{proxy_password}@{proxy_domain}:{proxy_port}" # configure proxy protocols proxies = { "http": proxy_url, "https": proxy_url, } response = requests.get("https://httpbin.io/ip", proxies=proxies) if response.status_code != 200: print(f"An error occurred with {response.status_code}") else: print(response.text) ``` -------------------------------- ### Python Example: Fetching Content on Error Source: https://docs.zenrows.com/universal-scraper-api/features/other This Python code uses the `requests` library to make a GET request to the ZenRows API. It includes the `allowed_status_codes` parameter to retrieve content from responses with status codes 404, 500, or 503. Ensure you have the `requests` library installed (`pip install requests`). ```Python # pip install requests import requests url = 'https://httpbin.io/anything' apikey = 'YOUR_ZENROWS_API_KEY' params = { 'url': url, 'apikey': apikey, 'allowed_status_codes': '404,500,503', } response = requests.get('https://api.zenrows.com/v1/', params=params) print(response.text) ``` -------------------------------- ### Complete Example: ZenRows Residential Proxies (Node.js) Source: https://docs.zenrows.com/residential-proxies/get-started/first-request A complete Node.js script demonstrating the integration of ZenRows Residential Proxies. It shows how to define proxy settings and use them with axios to make a web scraping request. ```javascript // npm install axios const axios = require('axios'); // define your proxy credentials const proxy = { protocol: 'http', host: 'superproxy.zenrows.com', port: '1337', auth: { username: 'PROXY_USERNAME', password: 'PROXY_PASSWORD', }, }; // send a request with the proxies axios .get('https://httpbin.org/ip', { proxy: proxy, }) .then((res) => { console.log(res.data); }) .catch((err) => console.error(err)); ``` -------------------------------- ### Handling Failed Requests with Retries Source: https://docs.zenrows.com/universal-scraper-api/nodejs Configure retry logic to manage failed HTTP requests. Consult the guide on retrying requests for detailed instructions on implementing this feature. ```APIDOC Feature: Retry Logic Description: Allows automatic retrying of failed HTTP requests to improve reliability. Documentation: /universal-scraper-api/help/retry-failed-requests ``` -------------------------------- ### Improve Scraping Speed with Concurrency Source: https://docs.zenrows.com/universal-scraper-api/python Boost scraping speed by making concurrent API calls to fetch multiple URLs simultaneously. The concurrency guide explains how to implement this effectively. ```APIDOC Concurrency: Make multiple parallel API requests to ZenRows to scrape several URLs at the same time, significantly reducing overall scraping time. ``` -------------------------------- ### Improving Scraping Speed with Concurrency Source: https://docs.zenrows.com/universal-scraper-api/nodejs Boost scraping speed by making concurrent API calls to scrape multiple URLs simultaneously. The concurrency guide explains how to implement this effectively. ```APIDOC Feature: Concurrency Description: Allows making multiple API calls in parallel to scrape several URLs at once, increasing efficiency. Documentation: /universal-scraper-api/features/concurrency#using-concurrency ```