ScrAPI

https://scrapi.tech
Admin
ScrAPI is a privacy-first web scraping API that handles bot detection, CAPTCHAs, IP rotation,...
Tokens:17,600
Snippets:205
Trust Score:7.5
Update:4 months ago
Show doc for...
Tokens:
Raw
### Example GET Request with POST Method Parameter

Source: https://scrapi.tech/docs/api_details/v1_scrape/request_method

Demonstrates how to specify the POST request method directly in the URL query parameters for a GET request.

```http
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&requestMethod=POST
```

--------------------------------

### Example GET Request with Screenshot

Source: https://scrapi.tech/docs/api_details/v1_scrape/screenshot

Use this GET request to scrape a URL and include a screenshot of the final rendered page.

```http
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&useBrowser=true&includeScreenshot=true
```

--------------------------------

### Example GET Request with PDF Enabled

Source: https://scrapi.tech/docs/api_details/v1_scrape/pdf

This example shows how to make a GET request to the Scrapi API to scrape a URL and generate a PDF. Ensure `includePdf` is set to `true`.

```http
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&useBrowser=true&includePdf=true
```

--------------------------------

### Example GET Request with useBrowser

Source: https://scrapi.tech/docs/api_details/v1_scrape/use_browser

Use this GET request to scrape a URL with JavaScript rendering enabled. Ensure the `useBrowser` parameter is set to `true`.

```http
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&useBrowser=true
```

--------------------------------

### Install .NET SDK

Source: https://scrapi.tech/docs/sdk_client

Use this command to add the official ScrAPI SDK for .NET to your project via NuGet.

```bash
dotnet add package ScrAPI
```

--------------------------------

### Example GET Request with Session ID

Source: https://scrapi.tech/docs/api_details/v1_scrape/session_id

Demonstrates how to make a GET request to the scrape endpoint, including a session ID for state persistence.

```http
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&sessionId=any_unique_identifier
```

--------------------------------

### Install Python SDK

Source: https://scrapi.tech/docs/sdk_client

Use this command to add the official ScrAPI SDK for Python to your project via PyPI.

```bash
pip install scrapi-sdk
```

--------------------------------

### First Request for Crawling with Sessions

Source: https://scrapi.tech/docs/api_details/v1_scrape/session_id

Initiate a scraping request with a session ID to start maintaining state. This example shows the first request to a product listing page.

```json
{
  "url": "https://example.com/products?page=1",
  "sessionId": "product-crawler"
}
```

--------------------------------

### Example Browser Commands for Scraping

Source: https://scrapi.tech/docs/mcp_server

This example demonstrates a sequence of browser commands that can be used with MCP server scraping tools. It includes actions like accepting cookies, waiting, inputting text, submitting a form, waiting for results, and scrolling.

```json
[
  {"click": "#accept-cookies"},
  {"wait": 2000},
  {"input": {"input[name='search']": "web scraping"}},
  {"click": "button[type='submit']"},
  {"waitfor": "#results"},
  {"scroll": 500}
]
```

--------------------------------

### Example GET Request with Custom Proxy

Source: https://scrapi.tech/docs/api_details/v1_scrape/custom_proxy

Demonstrates how to specify a custom proxy URL in a GET request to the ScrAPI endpoint. The customProxyUrl parameter is automatically detected.

```http
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&customProxyUrl=https://scrapi.tech:9090
```

--------------------------------

### Local MCP Server Setup with NPX

Source: https://scrapi.tech/docs/mcp_server

Configuration for running the ScrAPI MCP Server locally using NPX. This setup requires an API key and mounts it as an environment variable.

```json
{
  "mcpServers": {
    "scrapi": {
      "command": "npx",
      "args": [
        "-y",
        "@deventerprisesoftware/scrapi-mcp"
      ],
      "env": {
        "SCRAPI_API_KEY": "<YOUR_API_KEY>"
      }
    }
  }
}
```

--------------------------------

### Local MCP Server Setup with Docker

Source: https://scrapi.tech/docs/mcp_server

Configuration for running the ScrAPI MCP Server locally using Docker. This setup requires an API key and mounts it as an environment variable.

```json
{
  "mcpServers": {
    "scrapi": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "-e",
        "SCRAPI_API_KEY",
        "deventerprisesoftware/scrapi-mcp"
      ],
      "env": {
        "SCRAPI_API_KEY": "<YOUR_API_KEY>"
      }
    }
  }
}
```

--------------------------------

### Enable Video Recording via GET Request

Source: https://scrapi.tech/docs/api_details/v1_scrape/video

To enable video recording, set the `includeVideo` parameter to `true` in your GET request. The `useBrowser` parameter is also set to `true` to ensure a real browser is used.

```http
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&useBrowser=true&includeVideo=true
```

--------------------------------

### Example POST Request with Screenshot

Source: https://scrapi.tech/docs/api_details/v1_scrape/screenshot

Use this POST request to scrape a URL and include a screenshot. Ensure `includeScreenshot` is set to `true`.

```json
{
  "url": "https://deventerprise.com",
  "useBrowser": true,
  "includeScreenshot": true
}
```

--------------------------------

### Fill and Submit a Form with Browser Commands

Source: https://scrapi.tech/docs/api_details/v1_scrape/browser_commands

This example demonstrates filling form fields, selecting a dropdown option, waiting for elements, clearing the form, scrolling, and executing custom JavaScript using browser commands.

```json
{
  "url": "https://www.roboform.com/filling-test-all-fields",
  "useBrowser": true,
  "acceptDialogs": true,
  "browserCommands": [
    { "input": { "input[name='01___title']": "Mr" } },
    { "input": { "input[name='02frstname']": "Werner" } },
    { "input": { "input[name='04lastname']": "van Deventer" } },
    { "select": { "select[name='40cc__type']": "Discover" } },
    { "wait": 3000 },
    { "waitfor": "input[type='reset']" },
    { "click": "input[type='reset']" },
    { "wait": 1000 },
    { "scroll": 500 },
    { "javascript": "console.log('any valid code...'") }
  ]
}
```

--------------------------------

### Example Credit Calculation: Residential Proxy with Captcha

Source: https://scrapi.tech/docs/credits

This scenario illustrates the credit calculation for a request involving a browser, a residential proxy, and solving a captcha. It highlights the cumulative cost of multiple premium features.

```Markdown
**Residential proxy with captcha solving**| Browser (5) + Residential (10) + 1 captcha (30)| **45**
```

--------------------------------

### Example POST Request Body

Source: https://scrapi.tech/docs/api_details/v1_scrape

This JSON body demonstrates all available parameters for a POST request, including advanced features like browser commands. Use POST for production integrations.

```json
{
  "url": "https://deventerprise.com",
  "useBrowser": false,
  "solveCaptchas": false,
  "includeScreenshot": false,
  "includePdf": false,
  "includeVideo": false,
  "proxyType": null,
  "proxyCountry": null,
  "proxyCity": null,
  "customProxyUrl": null,
  "requestMethod": "GET",
  "responseFormat": "JSON",
  "responseSelector": null,
  "callbackUrl": null,
  "cookies": {},
  "headers": {},
  "sessionId": null,
  "acceptDialogs": false,
  "browserCommands": []
}
```

--------------------------------

### Example POST Request with useBrowser

Source: https://scrapi.tech/docs/api_details/v1_scrape/use_browser

Use this POST request for more complex scraping scenarios, including enabling the real browser. The `useBrowser` parameter should be set to `true`.

```json
{
  "url": "https://deventerprise.com",
  "useBrowser": true,
  "acceptDialogs": false
}
```

--------------------------------

### Example POST Request with PDF Enabled

Source: https://scrapi.tech/docs/api_details/v1_scrape/pdf

This example demonstrates a POST request to the Scrapi API for scraping a URL and generating a PDF. The `includePdf` parameter should be set to `true` within the JSON body.

```json
{
  "url": "https://deventerprise.com",
  "useBrowser": true,
  "includePdf": true
}
```

--------------------------------

### Basic GET Request

Source: https://scrapi.tech/docs/api_details/v1_scrape

The simplest way to call the API is with a GET request including the URL parameter. This is suitable for basic scraping tasks.

```http
https://api.scrapi.tech/v1/scrape?url=http://deventerprise.com
```

--------------------------------

### GET Request with Custom Headers

Source: https://scrapi.tech/docs/api_details/v1_scrape/cookies_headers

Use this to send custom headers with a GET request. Headers are provided as a semicolon-separated string.

```http
GET https://api.scrapi.tech/v1/scrape?url=http://httpbin.org/headers&headers=header1=value1;header2=value2
```

--------------------------------

### Example Credit Calculation: Full Capture with Data Center Proxy

Source: https://scrapi.tech/docs/credits

This scenario shows the credit calculation for a comprehensive request including browser, data center proxy, screenshot, PDF, and video recording. It demonstrates the total cost for a full-featured scrape.

```Markdown
**Full capture with data center proxy**| Browser (5) + Data Center (5) + Screenshot (2) + PDF (2) + Video (3)| **17**
```

--------------------------------

### Example Credit Calculation: Browser with Screenshot

Source: https://scrapi.tech/docs/credits

This scenario demonstrates the credit calculation for a browser scrape that also captures a screenshot. It combines the cost of using the Real Browser feature with the Screenshot feature.

```Markdown
**Browser scrape with screenshot**| Browser (5) + Screenshot (2)| **7**
```

--------------------------------

### Default JSON Response Example

Source: https://scrapi.tech/docs/api_details/v1_scrape

This is an example of the default JSON response from the v1 scrape API, showing all possible fields that can be returned.

```json
{
  "requestUrl": "https://www.deventerprise.com",
  "responseUrl": "https://deventerprise.com",
  "duration": "00:00:01.000000",
  "attempts": 1,
  "errorMessages": [
    "Browser Command Click Timeout - Could not find input target #click_me"
  ],
  "captchasSolved": {
    "reCaptchaV2": 1
  },
  "creditsUsed": 35,
  "statusCode": 200,
  "cookies": {
    "session-id": "12345...",
    "timezone": "UTC"
  },
  "headers": {
    "content-encoding": "br",
    "content-type": "text/html; charset=utf-8"
  },
  "content": "<html>...</html>",
  "screenshotUrl": "https://scrapi.blob.core.windows.net/[API-KEY]/[FILE].png",
  "pdfUrl": "https://scrapi.blob.core.windows.net/[API-KEY]/[FILE].pdf",
  "videoUrl": "https://scrapi.blob.core.windows.net/[API-KEY]/[FILE].webm"
}
```

--------------------------------

### GET Request with Custom Cookies

Source: https://scrapi.tech/docs/api_details/v1_scrape/cookies_headers

Use this to send custom cookies with a GET request. Cookies are provided as a semicolon-separated string.

```http
GET https://api.scrapi.tech/v1/scrape?url=http://httpbin.org/cookies&cookies=cookie1=value1;cookie2=value2
```

--------------------------------

### Example POST Request to Scrape URL

Source: https://scrapi.tech/docs/api_details/v1_scrape/request_method

Shows a direct POST request to the scrape endpoint, typically used when the request body contains the scraping configuration.

```http
POST https://api.scrapi.tech/v1/scrape
```

--------------------------------

### Example JSON Response for Available Cities

Source: https://scrapi.tech/docs/api_details/available_cities

The response is a JSON array where each object represents a city and includes its display name and a key for use in the `proxyCity` parameter.

```json
[
  {
    "name": "Alberton",
    "key": "ALBERTON"
  },
  {
    "name": "Jeffreys Bay",
    "key": "JEFFREYSBAY"
  },
  {
    "name": "Cape Town",
    "key": "CAPETOWN"
  },
  ...
]
```

--------------------------------

### GET Request with Residential Proxy

Source: https://scrapi.tech/docs/api_details/v1_scrape/residential_proxy

Use this GET request to scrape a URL using a residential proxy. Ensure the `proxyType` parameter is set to `Residential`.

```HTTP
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&proxyType=Residential
```

--------------------------------

### Country-Level Targeting (Australia)

Source: https://scrapi.tech/docs/api_details/v1_scrape/geotargeting

This example demonstrates how to target requests to Australia using a Residential proxy. The `proxyCountry` parameter is set to 'AUS'.

```APIDOC
## GET /v1/scrape

### Description
Scrapes a given URL using a specified proxy type and country.

### Method
GET

### Endpoint
/v1/scrape

### Parameters
#### Query Parameters
- **url** (string) - Required - The URL to scrape.
- **proxyType** (string) - Required - The type of proxy to use (e.g., Residential, DataCenter).
- **proxyCountry** (string) - Required - The ISO 3166-1 alpha-3 code of the target country (e.g., AUS).

### Request Example
GET https://api.scrapi.tech/v1/scrape?url=http://ip-api.com/json&proxyType=Residential&proxyCountry=AUS

### Response
#### Success Response (200)
- **field1** (type) - Description

#### Response Example
{
  "example": "response body"
}

## POST /v1/scrape

### Description
Scrapes a given URL using a specified proxy type and country via a POST request.

### Method
POST

### Endpoint
/v1/scrape

### Parameters
#### Request Body
- **url** (string) - Required - The URL to scrape.
- **proxyType** (string) - Required - The type of proxy to use (e.g., Residential, DataCenter).
- **proxyCountry** (string) - Required - The ISO 3166-1 alpha-3 code of the target country (e.g., AUS).

### Request Example
{
  "url": "http://ip-api.com/json",
  "proxyType": "Residential",
  "proxyCountry": "AUS"
}

### Response
#### Success Response (200)
- **field1** (type) - Description

#### Response Example
{
  "example": "response body"
}
```

--------------------------------

### City-Level Targeting (Melbourne, Australia)

Source: https://scrapi.tech/docs/api_details/v1_scrape/geotargeting

This example shows how to target requests to Melbourne, Australia, using a Residential proxy. It includes both `proxyCountry` and `proxyCity` parameters for more specific routing.

```APIDOC
## GET /v1/scrape

### Description
Scrapes a given URL using a specified proxy type, country, and city.

### Method
GET

### Endpoint
/v1/scrape

### Parameters
#### Query Parameters
- **url** (string) - Required - The URL to scrape.
- **proxyType** (string) - Required - The type of proxy to use (e.g., Residential, DataCenter).
- **proxyCountry** (string) - Required - The ISO 3166-1 alpha-3 code of the target country (e.g., AUS).
- **proxyCity** (string) - Optional - The name of the target city (e.g., Melbourne).

### Request Example
GET https://api.scrapi.tech/v1/scrape?url=http://ip-api.com/json&proxyType=Residential&proxyCountry=AUS&proxyCity=Melbourne

### Response
#### Success Response (200)
- **field1** (type) - Description

#### Response Example
{
  "example": "response body"
}

## POST /v1/scrape

### Description
Scrapes a given URL using a specified proxy type, country, and city via a POST request.

### Method
POST

### Endpoint
/v1/scrape

### Parameters
#### Request Body
- **url** (string) - Required - The URL to scrape.
- **proxyType** (string) - Required - The type of proxy to use (e.g., Residential, DataCenter).
- **proxyCountry** (string) - Required - The ISO 3166-1 alpha-3 code of the target country (e.g., AUS).
- **proxyCity** (string) - Optional - The name of the target city (e.g., Melbourne).

### Request Example
{
  "url": "http://ip-api.com/json",
  "proxyType": "Residential",
  "proxyCountry": "AUS",
  "proxyCity": "Melbourne"
}

### Response
#### Success Response (200)
- **field1** (type) - Description

#### Response Example
{
  "example": "response body"
}
```

--------------------------------

### Example HTML Response with Scrapi Wrapper

Source: https://scrapi.tech/docs/api_details/v1_scrape/response_selector

When using HTML response format with a `responseSelector`, the extracted elements are wrapped in a `<scrapi>` root tag to ensure valid HTML output.

```html
<scrapi>
<h2>
  DevEnterprise Software Limited
  <p>
    DevEnterprise provides custom software solutions for companies and individuals using enterprise level development techniques and services.
  </p>
</h2>
</scrapi>
```

--------------------------------

### Second Request for Crawling with Sessions

Source: https://scrapi.tech/docs/api_details/v1_scrape/session_id

Continue scraping using the same session ID to leverage previously maintained cookies and IP address. This example shows the second request to the next page of the product listing.

```json
{
  "url": "https://example.com/products?page=2",
  "sessionId": "product-crawler"
}
```

--------------------------------

### GET /v1/countries

Source: https://scrapi.tech/docs/api_details/available_countries

Retrieves a list of all countries available for proxy geotargeting. The response includes country names and their corresponding ISO 3166-1 alpha-3 codes.

```APIDOC
## GET /v1/countries

### Description
Retrieve a list of all countries currently available for proxy geotargeting. The response includes country names and their corresponding ISO 3166-1 alpha-3 codes (e.g., `USA`, `GBR`, `AUS`) that can be used as the `proxyCountry` parameter in scrape requests.

### Method
GET

### Endpoint
https://api.scrapi.tech/v1/countries

### Response
#### Success Response (200)
- **name** (string) - The display name of the country.
- **key** (string) - The ISO 3166-1 alpha-3 code for the country, usable as a `proxyCountry` parameter.

#### Response Example
```json
[
  {
    "name": "United Kingdom",
    "key": "GBR"
  },
  {
    "name": "United States",
    "key": "USA"
  },
  {
    "name": "South Africa",
    "key": "ZAF"
  }
]
```
```

--------------------------------

### Example Markdown Response without Wrapping

Source: https://scrapi.tech/docs/api_details/v1_scrape/response_selector

For Markdown responses, the content selected by `responseSelector` is returned directly without any additional wrapping tags, providing clean text output.

```markdown
## DevEnterprise Software Limited
DevEnterprise provides custom software solutions for companies and individuals using enterprise level development techniques and services.
```

--------------------------------

### Example POST Request with Session ID

Source: https://scrapi.tech/docs/api_details/v1_scrape/session_id

Shows a POST request to the scrape endpoint, utilizing a session ID to maintain context across requests. Replace 'any_unique_identifier' with your desired session ID.

```json
{
  "url": "https://deventerprise.com",
  "sessionId": "any_unique_identifier"
}
```

--------------------------------

### GET Request for Available Cities

Source: https://scrapi.tech/docs/api_details/available_cities

This endpoint retrieves a list of cities available for geotargeting within a specified country. The `countryKey` parameter is required and should be an ISO 3166-1 alpha-3 country code.

```bash
GET https://api.scrapi.tech/v1/countries/{countryKey}/cities
```

--------------------------------

### Example JSON Response for Available Countries

Source: https://scrapi.tech/docs/api_details/available_countries

The JSON response is an array of country objects, each containing a display name and an ISO 3166-1 alpha-3 key for use in the proxyCountry parameter.

```JSON
[
  {
    "name": "United Kingdom",
    "key": "GBR"
  },
  {
    "name": "United States",
    "key": "USA"
  },
  {
    "name": "South Africa",
    "key": "ZAF"
  },
  ...
]
```

--------------------------------

### Scrape URL Endpoint

Source: https://scrapi.tech/docs/api_details/v1_scrape

The primary endpoint for scraping website content. Supports both GET and POST requests. POST is recommended for advanced features and production use.

```APIDOC
## GET /v1/scrape

### Description
Retrieves website content using a GET request. The `url` parameter is required.

### Method
GET

### Endpoint
https://api.scrapi.tech/v1/scrape

### Parameters
#### Query Parameters
- **url** (String URL) - Required - The URL to scrape.
- **apiKey** (String) - Optional - Your API key for authentication.
- **useBrowser** (Boolean) - Optional - Use a full headless browser with JavaScript execution. Default: `false`.
- **solveCaptchas** (Boolean) - Optional - Automatically detect and solve captchas. Enables `useBrowser` automatically. Default: `false`.
- **includeScreenshot** (Boolean) - Optional - Capture a screenshot (PNG) of the final page. Enables `useBrowser` automatically. Default: `false`.
- **includePdf** (Boolean) - Optional - Generate a PDF of the final page. Enables `useBrowser` automatically. Default: `false`.
- **includeVideo** (Boolean) - Optional - Record a video (WEBM) of the browser session. Enables `useBrowser` automatically. Default: `false`.
- **proxyType** (String or Number) - Optional - Proxy type - `Free`, `DataCenter`, `Residential`, `Tor`, or `Custom`.
- **proxyCountry** (String 3-letter code) - Optional - ISO 3166-1 alpha-3 code for geotargeting.
- **proxyCity** (String) - Optional - City name for fine-grained geotargeting.
- **customProxyUrl** (String URL) - Optional - Your own custom proxy URL (`protocol://username:password@host:port`).
- **requestMethod** (String) - Optional - HTTP method - `GET` (default), `POST`, `PUT`, `DELETE`, `HEAD`, `PATCH`. Only available without browser.
- **responseFormat** (String or Number) - Optional - Response format - `JSON` (default), `HTML`, or `Markdown`.
- **responseSelector** (String) - Optional - CSS or XPath selector to extract specific content from the page.
- **callbackUrl** (String URL) - Optional - URL to receive a `POST` with the response data.
- **cookies** (String or Object) - Optional - Custom cookies as key/value pairs (`cookie1=value1;cookie2=value2`).
- **headers** (String or Object) - Optional - Custom headers as key/value pairs (`header1=value1;header2=value2`).
- **sessionId** (String) - Optional - Session identifier to persist IP, cookies, and user agent across requests.
- **acceptDialogs** (Boolean) - Optional - Accept popup dialogs instead of cancelling them. Default: `false`.

### Request Example
```
https://api.scrapi.tech/v1/scrape?url=http://deventerprise.com
```

## POST /v1/scrape

### Description
Submits a scrape request using POST, allowing for more advanced features via a JSON request body, including browser commands.

### Method
POST

### Endpoint
https://api.scrapi.tech/v1/scrape

### Parameters
#### Query Parameters
- **apiKey** (String) - Optional - Your API key for authentication.
- **callbackUrl** (String URL) - Optional - URL to receive a `POST` with the response data.

#### Request Body
- **url** (String URL) - Required - The URL to scrape.
- **useBrowser** (Boolean) - Optional - Use a full headless browser with JavaScript execution. Default: `false`.
- **solveCaptchas** (Boolean) - Optional - Automatically detect and solve captchas. Enables `useBrowser` automatically. Default: `false`.
- **includeScreenshot** (Boolean) - Optional - Capture a screenshot (PNG) of the final page. Enables `useBrowser` automatically. Default: `false`.
- **includePdf** (Boolean) - Optional - Generate a PDF of the final page. Enables `useBrowser` automatically. Default: `false`.
- **includeVideo** (Boolean) - Optional - Record a video (WEBM) of the browser session. Enables `useBrowser` automatically. Default: `false`.
- **proxyType** (String or Number) - Optional - Proxy type - `Free`, `DataCenter`, `Residential`, `Tor`, or `Custom`.
- **proxyCountry** (String 3-letter code) - Optional - ISO 3166-1 alpha-3 code for geotargeting.
- **proxyCity** (String) - Optional - City name for fine-grained geotargeting.
- **customProxyUrl** (String URL) - Optional - Your own custom proxy URL (`protocol://username:password@host:port`).
- **requestMethod** (String) - Optional - HTTP method - `GET` (default), `POST`, `PUT`, `DELETE`, `HEAD`, `PATCH`. Only available without browser.
- **responseFormat** (String or Number) - Optional - Response format - `JSON` (default), `HTML`, or `Markdown`.
- **responseSelector** (String) - Optional - CSS or XPath selector to extract specific content from the page.
- **cookies** (String or Object) - Optional - Custom cookies as key/value pairs (`cookie1=value1;cookie2=value2`).
- **headers** (String or Object) - Optional - Custom headers as key/value pairs (`header1=value1;header2=value2`).
- **sessionId** (String) - Optional - Session identifier to persist IP, cookies, and user agent across requests.
- **acceptDialogs** (Boolean) - Optional - Accept popup dialogs instead of cancelling them. Default: `false`.
- **browserCommands** (Array) - Optional - List of browser commands to execute after page load. **POST only.**

### Request Example
```json
{
  "url": "https://deventerprise.com",
  "useBrowser": false,
  "solveCaptchas": false,
  "includeScreenshot": false,
  "includePdf": false,
  "includeVideo": false,
  "proxyType": null,
  "proxyCountry": null,
  "proxyCity": null,
  "customProxyUrl": null,
  "requestMethod": "GET",
  "responseFormat": "JSON",
  "responseSelector": null,
  "callbackUrl": null,
  "cookies": {},
  "headers": {},
  "sessionId": null,
  "acceptDialogs": false,
  "browserCommands": []
}
```
```

--------------------------------

### GET /v1/countries/{countryKey}/cities

Source: https://scrapi.tech/docs/api_details/available_cities

Retrieves a list of cities available for proxy geotargeting within a specified country. The response includes city names and their corresponding keys, which can be used for `proxyCity` parameter in scrape requests.

```APIDOC
## GET /v1/countries/{countryKey}/cities

### Description
Retrieve a list of all cities available for proxy geotargeting within a specific country. Each response includes city names and corresponding keys that can be used as the `proxyCity` parameter in scrape requests.

### Method
GET

### Endpoint
https://api.scrapi.tech/v1/countries/{countryKey}/cities

### Parameters
#### Path Parameters
- **countryKey** (string) - Required - An ISO 3166-1 alpha-3 country code (e.g., `ZAF`, `USA`, `GBR`).

### Response
#### Success Response (200)
- **name** (string) - The display name of the city.
- **key** (string) - The key for the city, usable as the `proxyCity` parameter.

#### Response Example
```json
[
  {
    "name": "Alberton",
    "key": "ALBERTON"
  },
  {
    "name": "Jeffreys Bay",
    "key": "JEFFREYSBAY"
  },
  {
    "name": "Cape Town",
    "key": "CAPETOWN"
  }
]
```
```

--------------------------------

### List Available Countries

Source: https://scrapi.tech/docs/api_details/available_countries

GET request to the /v1/countries endpoint to retrieve a list of available countries for proxy geotargeting. The response includes country names and their ISO 3166-1 alpha-3 codes.

```HTTP
GET https://api.scrapi.tech/v1/countries
```

--------------------------------

### Checking Credits Used in JSON Response

Source: https://scrapi.tech/docs/credits

This JSON response example shows the `creditsUsed` field, which indicates the total credits consumed by a successful API request. This field is present in the default JSON response format.

```JSON
{
  "requestUrl": "https://www.deventerprise.com",
  "responseUrl": "https://deventerprise.com",
  "duration": "00:00:01.000000",
  "attempts": 1,
  "captchasSolved": {
    "reCaptchaV2": 1
  },
  "creditsUsed": 35,
  "statusCode": 200,
  "content": "<html>...</html>"
}
```

--------------------------------

### Generate Go API Client

Source: https://scrapi.tech/docs/sdk_client

Generate a typed API client for Go using Microsoft Kiota and the OpenAPI specification.

```bash
# Go
kiota generate --ebc -c ScrapiClient -n Scrapi -d https://api.scrapi.tech/openapi/v1.json -o ScrapiClientGo -l go
```

--------------------------------

### Initiate Scrape with Callback

Source: https://scrapi.tech/docs/api_details/v1_scrape/callback_webhook

Submit a scraping request with a callback URL. The API will respond immediately with a 202 Accepted status and a reference ID. The results will be POSTed to your callback URL upon completion.

```APIDOC
## POST /v1/scrape

### Description
Initiates an asynchronous scraping job and specifies a URL to receive the results upon completion.

### Method
POST

### Endpoint
https://api.scrapi.tech/v1/scrape

### Parameters
#### Request Body
- **url** (string) - Required - The URL to scrape.
- **callbackUrl** (string) - Required - The URL to which ScrAPI will POST the results.

### Request Example
```json
{
  "url": "https://deventerprise.com",
  "callbackUrl": "https://webhook-test.com/f661e0e4c08dc1cf2449bc9d507b2d8d"
}
```

### Response
#### Success Response (202 Accepted)
- **reference** (string) - A unique ID for tracking the scrape job.

### Response Example
```json
{
  "reference": "2c1e5c6d-d00d-47f1-bab0-de92ed2c81d5"
}
```
```

--------------------------------

### POST Request with Custom Proxy and Authentication

Source: https://scrapi.tech/docs/api_details/v1_scrape/custom_proxy

Shows how to configure a POST request to use a custom SOCKS5 proxy with username and password authentication. Ensure the customProxyUrl format is correct.

```json
{
  "url": "https://deventerprise.com",
  "customProxyUrl": "socks5://username:password@scrapi.tech:8080"
}
```

--------------------------------

### Accept Cookies and Search with Browser Commands

Source: https://scrapi.tech/docs/api_details/v1_scrape/browser_commands

This snippet shows a common pattern for accepting cookie consent banners and then performing a search on a website using browser commands.

```json
{
  "url": "https://example.com",
  "useBrowser": true,
  "browserCommands": [
    { "click": "#accept-cookies" },
    { "wait": 1000 },
    { "input": { "input[name='q']": "web scraping API" } },
    { "click": "button[type='submit']" },
    { "waitfor": ".search-results" },
    { "scroll": 2000 }
  ]
}
```

--------------------------------

### Generate Java API Client

Source: https://scrapi.tech/docs/sdk_client

Generate a typed API client for Java using Microsoft Kiota and the OpenAPI specification.

```bash
# Java
kiota generate --ebc -c ScrapiClient -n Scrapi -d https://api.scrapi.tech/openapi/v1.json -o ScrapiClientJava -l java
```

--------------------------------

### Enable Video Recording via POST Request

Source: https://scrapi.tech/docs/api_details/v1_scrape/video

To enable video recording, set `includeVideo` to `true` in the JSON payload of your POST request. The `useBrowser` parameter is also set to `true` to ensure a real browser is used.

```json
{
  "url": "https://deventerprise.com",
  "useBrowser": true,
  "includeVideo": true
}
```

--------------------------------

### Generate PHP API Client

Source: https://scrapi.tech/docs/sdk_client

Generate a typed API client for PHP using Microsoft Kiota and the OpenAPI specification.

```bash
# PHP
kiota generate --ebc -c ScrapiClient -n Scrapi -d https://api.scrapi.tech/openapi/v1.json -o ScrapiClientPhp -l php
```

--------------------------------

### Generate Ruby API Client

Source: https://scrapi.tech/docs/sdk_client

Generate a typed API client for Ruby using Microsoft Kiota and the OpenAPI specification.

```bash
# Ruby
kiota generate --ebc -c ScrapiClient -n Scrapi -d https://api.scrapi.tech/openapi/v1.json -o ScrapiClientRuby -l ruby
```

--------------------------------

### API Key Authentication (Header)

Source: https://scrapi.tech/docs/api_details/v1_scrape

Authenticate your requests by providing your API key in the X-API-KEY header. This method is often preferred for security.

```http
X-API-KEY: 01234567-8901-2345-6789-0ABCDEF01234
```

--------------------------------

### Generate TypeScript API Client

Source: https://scrapi.tech/docs/sdk_client

Generate a typed API client for TypeScript using Microsoft Kiota and the OpenAPI specification.

```bash
# TypeScript
kiota generate --ebc -c ScrapiClient -n Scrapi -d https://api.scrapi.tech/openapi/v1.json -o ScrapiClientTypeScript -l typescript
```

--------------------------------

### Generate Dart API Client

Source: https://scrapi.tech/docs/sdk_client

Generate a typed API client for Dart using Microsoft Kiota and the OpenAPI specification.

```bash
# Dart
kiota generate --ebc -c ScrapiClient -n Scrapi -d https://api.scrapi.tech/openapi/v1.json -o ScrapiClientDart -l dart
```

--------------------------------

### Scraping with POST Request Method

Source: https://scrapi.tech/docs/api_details/v1_scrape/request_method

Use this when the target URL requires a POST request to return data. Ensure `useBrowser` is disabled.

```json
{
  "url": "https://deventerprise.com",
  "useBrowser": false,
  "requestMethod": "POST"
}
```

--------------------------------

### Enable Video Recording in API Response

Source: https://scrapi.tech/docs/api_details/v1_scrape/video

The video download URL is included in the JSON response when video recording is enabled.

```json
{
  "videoUrl": "https://scrapi.blob.core.windows.net/[API-KEY]/[FILE].webm"
}
```

--------------------------------

### API Key Authentication (URL Parameter)

Source: https://scrapi.tech/docs/api_details/v1_scrape

Authenticate your requests by including your API key as a URL parameter. This is the simplest method for authentication.

```http
https://api.scrapi.tech/v1/scrape?apiKey=01234567-8901-2345-6789-0ABCDEF01234&url=https://example.com
```

--------------------------------

### Immediate Response with Reference ID

Source: https://scrapi.tech/docs/api_details/v1_scrape/callback_webhook

When a 'callbackUrl' is provided, the API immediately returns a 202 Accepted status and a reference ID for tracking the scrape's progress.

```JSON
{
  "reference": "2c1e5c6d-d00d-47f1-bab0-de92ed2c81d5"
}
```

--------------------------------

### Enable Free Proxy in Request

Source: https://scrapi.tech/docs/api_details/v1_scrape/free_proxy

To use the free proxy feature, set the `proxyType` parameter to `Free` in your API request. This is useful for basic IP rotation without additional credit costs.

```HTTP
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&proxyType=Free
```

--------------------------------

### POST Request with Custom Headers

Source: https://scrapi.tech/docs/api_details/v1_scrape/cookies_headers

Send custom headers with a POST request. Headers are provided as a JSON dictionary object.

```json
{
  "url": "http://httpbin.org/headers",
  "headers": {
    "My-Header": "Test Header 1",
    "X-SITE-KEY": "Test Header 2"
  }
}
```

--------------------------------

### Check Credit Balance

Source: https://scrapi.tech/docs/api_details/credit_balance

Use this endpoint to retrieve your current API credit balance. Authentication can be done via URL parameter or request header.

```HTTP
GET https://api.scrapi.tech/v1/balance?apiKey=00000000-0000-0000-0000-000000000000
```

--------------------------------

### HTML Response Format Request

Source: https://scrapi.tech/docs/api_details/v1_scrape/html_markdown_response

Use the `responseFormat: "HTML"` parameter to receive the full HTML source of a webpage. This is suitable for proxying responses directly into a browser or iframe.

```json
{
  "url": "https://deventerprise.com",
  "responseFormat": "HTML"
}
```

--------------------------------

### Browser Commands API Endpoint

Source: https://scrapi.tech/docs/api_details/v1_scrape/browser_commands

This endpoint allows you to execute a series of browser commands to interact with a web page before scraping its content. It requires a POST request to the `/v1/scrape` endpoint with specific parameters.

```APIDOC
## POST /v1/scrape

### Description
Executes a sequence of browser commands to interact with a web page before content extraction. This is useful for handling dynamic content, user interactions, and pre-scraping setup.

### Method
POST

### Endpoint
https://api.scrapi.tech/v1/scrape

### Parameters
#### Request Body
- **url** (string) - Required - The URL of the website to scrape.
- **useBrowser** (boolean) - Optional - Enables the real browser for executing commands. Defaults to true when `browserCommands` are provided.
- **acceptDialogs** (boolean) - Optional - Automatically accepts JavaScript dialogs (alerts, confirms, prompts).
- **browserCommands** (array) - Required - An array of browser command objects to execute in order.

### Browser Commands
Each object in the `browserCommands` array represents a single command:

- **click / tap** (object) - Clicks on an element specified by a CSS selector.
  - Example: `{ "click": "#buttonId" }`
- **input / fill / type** (object) - Enters text into an input field specified by a CSS selector.
  - Example: `{ "input": { "input[name='email']": "example@test.com" } }`
- **select / choose / pick** (object) - Selects an option from a dropdown by its value or visible text.
  - Example: `{ "select": { "select[name='country']": "USA" } }`
- **scroll** (object) - Scrolls the page by a specified number of pixels (positive for down, negative for up).
  - Example: `{ "scroll": 1000 }`
- **wait** (object) - Pauses execution for a specified number of milliseconds (max 15000).
  - Example: `{ "wait": 5000 }`
- **waitfor / wait-for / wait_for** (object) - Waits until an element specified by a CSS selector appears on the page.
  - Example: `{ "waitfor": "#newForm" }`
- **javascript / js / eval** (object) - Executes custom JavaScript code in the browser context.
  - Example: `{ "javascript": "console.log('hello!!')" }`

### Request Example 1: Fill and Submit a Form
```json
{
  "url": "https://www.roboform.com/filling-test-all-fields",
  "useBrowser": true,
  "acceptDialogs": true,
  "browserCommands": [
    { "input": { "input[name='01___title']": "Mr" } },
    { "input": { "input[name='02frstname']": "Werner" } },
    { "input": { "input[name='04lastname']": "van Deventer" } },
    { "select": { "select[name='40cc__type']": "Discover" } },
    { "wait": 3000 },
    { "waitfor": "input[type='reset']" },
    { "click": "input[type='reset']" },
    { "wait": 1000 },
    { "scroll": 500 },
    { "javascript": "console.log('any valid code...')
```

--------------------------------

### Enable Captcha Solving in Scrape Request

Source: https://scrapi.tech/docs/api_details/v1_scrape/solve_captchas

Use this parameter to enable automatic captcha detection and solving. It is recommended to enable `useBrowser` alongside `solveCaptchas`.

```HTTP
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&useBrowser=true&solveCaptchas=true
```

```JSON
{
  "url": "https://deventerprise.com",
  "useBrowser": true,
  "solveCaptchas": true
}
```

--------------------------------

### POST Request with Combined Cookies and Headers

Source: https://scrapi.tech/docs/api_details/v1_scrape/cookies_headers

Send both custom cookies and headers in a single POST request. Both are provided as JSON dictionary objects.

```json
{
  "url": "https://example.com/dashboard",
  "cookies": {
    "session_token": "abc123",
    "preferences": "lang=en"
  },
  "headers": {
    "Authorization": "Bearer eyJhbGciOiJIUzI1NiIs...",
    "X-Requested-With": "XMLHttpRequest"
  }
}
```

--------------------------------

### Enable Data Center Proxy in Scrape Request

Source: https://scrapi.tech/docs/api_details/v1_scrape/data_center_proxy

Use this JSON payload to enable Data Center proxy for your scrape requests. Set 'proxyType' to 'DataCenter'.

```json
{
  "url": "https://deventerprise.com",
  "proxyType": "DataCenter"
}
```

--------------------------------

### POST Request with Custom Cookies

Source: https://scrapi.tech/docs/api_details/v1_scrape/cookies_headers

Send custom cookies with a POST request. Cookies are provided as a JSON dictionary object.

```json
{
  "url": "http://httpbin.org/cookies",
  "cookies": {
    "My-Cookie": "Test 1",
    "Session-ID": "Test 2"
  }
}
```

--------------------------------

### Scrape Request with Callback URL

Source: https://scrapi.tech/docs/api_details/v1_scrape/callback_webhook

Submit a scrape request including the 'callbackUrl' parameter to receive results asynchronously. The API will respond immediately with a 202 Accepted status and a reference ID.

```HTTP
GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&callbackUrl=https://webhook-test.com/f661e0e4c08dc1cf2449bc9d507b2d8d
```

```HTTP
POST https://api.scrapi.tech/v1/scrape
{
  "url": "https://deventerprise.com",
  "callbackUrl": "https://webhook-test.com/f661e0e4c08dc1cf2449bc9d507b2d8d"
}
```

--------------------------------

### Enable Free Proxy in JSON Payload

Source: https://scrapi.tech/docs/api_details/v1_scrape/free_proxy

Alternatively, you can specify `proxyType: "Free"` within the JSON payload for POST requests to utilize the free proxy pool.

```JSON
{
  "url": "https://deventerprise.com",
  "proxyType": "Free"
}
```

--------------------------------

### Markdown Response Format Request

Source: https://scrapi.tech/docs/api_details/v1_scrape/html_markdown_response

Set `responseFormat: "Markdown"` to obtain a clean Markdown conversion of the page content. This format is ideal for feeding web content into AI models for RAG and LLM pipelines due to its token efficiency and preserved structure.

```json
{
  "url": "https://deventerprise.com",
  "responseFormat": "Markdown"
}
```

--------------------------------

### POST Request with Custom Proxy without Authentication

Source: https://scrapi.tech/docs/api_details/v1_scrape/custom_proxy

Illustrates a POST request using a custom HTTP proxy without requiring authentication. The customProxyUrl parameter directly specifies the proxy address.

```json
{
  "url": "https://deventerprise.com",
  "customProxyUrl": "https://scrapi.tech:9090"
}
```

--------------------------------

### Check Credit Balance

Source: https://scrapi.tech/docs/api_details/credit_balance

Retrieves the current number of credits remaining for your API key. This endpoint can be used to monitor usage, trigger alerts when credits run low, or display balance information in your application’s dashboard.

```APIDOC
## GET /v1/balance

### Description
Retrieve the current number of credits remaining for your API key. Use this endpoint to monitor usage, trigger alerts when credits run low, or display balance information in your application’s dashboard.

### Method
GET

### Endpoint
https://api.scrapi.tech/v1/balance

### Parameters
#### Query Parameters
- **apiKey** (string) - Required - Your API key for authentication.

### Response
#### Success Response (200)
- **credits** (integer) - The number of API credits remaining for the authenticated API key.

#### Response Example
```json
{
  "credits": 85433
}
```
```

--------------------------------

### City-Level Geotargeting (Melbourne, Australia)

Source: https://scrapi.tech/docs/api_details/v1_scrape/geotargeting

Use this snippet for more granular control by specifying both the country and city for your proxy server. The `proxyType` parameter is required.

```HTTP
GET https://api.scrapi.tech/v1/scrape?url=http://ip-api.com/json&proxyType=Residential&proxyCountry=AUS&proxyCity=Melbourne
```

```JSON
{
  "url": "http://ip-api.com/json",
  "proxyType": "Residential",
  "proxyCountry": "AUS",
  "proxyCity": "Melbourne"
}
```

--------------------------------

### Screenshot URL in Header

Source: https://scrapi.tech/docs/api_details/v1_scrape/screenshot

When using HTML or Markdown format, the screenshot URL is returned in the `X-ScrAPI-ScreenshotUrl` header.

```http
X-ScrAPI-ScreenshotUrl
```

--------------------------------

### Scrape URL with Session ID

Source: https://scrapi.tech/docs/api_details/v1_scrape/session_id

Initiates a scraping request and associates it with a specific session ID. Subsequent requests with the same session ID will reuse the same context (IP, user agent, cookies).

```APIDOC
## GET https://api.scrapi.tech/v1/scrape

### Description
Scrapes a given URL and associates the request with a session ID for state persistence.

### Method
GET

### Endpoint
https://api.scrapi.tech/v1/scrape

### Parameters
#### Query Parameters
- **url** (string) - Required - The URL to scrape.
- **sessionId** (string) - Required - A unique identifier for the session. All requests with the same sessionId will share the same context.

### Response
#### Success Response (200)
(Response details not specified in source)

## POST https://api.scrapi.tech/v1/scrape

### Description
Submits a scraping request with a session ID, allowing for state persistence across multiple requests.

### Method
POST

### Endpoint
https://api.scrapi.tech/v1/scrape

### Parameters
#### Request Body
- **url** (string) - Required - The URL to scrape.
- **sessionId** (string) - Required - A unique identifier for the session. All requests with the same sessionId will share the same context.

### Request Example
```json
{
  "url": "https://deventerprise.com",
  "sessionId": "any_unique_identifier"
}
```

### Response
#### Success Response (200)
(Response details not specified in source)
```

--------------------------------

### Credit Balance JSON Response

Source: https://scrapi.tech/docs/api_details/credit_balance

The API returns the remaining credits in the 'credits' field. This is useful for displaying balance information in your application.

```JSON
{
  "credits": 85433
}
```

--------------------------------

### XPath Query for Response Filtering

Source: https://scrapi.tech/docs/api_details/v1_scrape/response_selector

Utilize the `responseSelector` parameter with an XPath query to precisely target and extract elements from the scraped HTML. This provides more granular control than CSS selectors.

```json
{
  "url": "https://deventerprise.com",
  "responseSelector": "//div[@class='title']/h2"
}
```
Tokens:
Raw