### Install Scrapy-Zyte-API with x402 Support

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Installs scrapy-zyte-api with the 'x402' extra, enabling support for x402. Requires Python 3.10+ and installs the necessary components for x402 integration.

```shell
pip install scrapy-zyte-api[x402]
```

--------------------------------

### Install Scrapy-Zyte-API (Basic)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Installs the core scrapy-zyte-api package for basic functionality. No external dependencies beyond Scrapy and Python are required for this basic installation.

```shell
pip install scrapy-zyte-api
```

--------------------------------

### Install Scrapy-Zyte-API with Multiple Extras

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Installs scrapy-zyte-api with multiple optional extras, such as 'provider' and 'x402'. This allows for combined functionality and requires the specified Python and Scrapy versions for each extra.

```shell
pip install scrapy-zyte-api[provider,x402]
```

--------------------------------

### Install Scrapy-Zyte-API with Scrapy-Poet Support

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Installs scrapy-zyte-api with the 'provider' extra, enabling integration with scrapy-poet. This requires Scrapy version 2.6+ and adds the necessary components for scrapy-poet compatibility.

```shell
pip install scrapy-zyte-api[provider]
```

--------------------------------

### Configure Scrapy-Zyte-API Components Separately

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Configures scrapy-zyte-api by setting various Scrapy settings like DOWNLOAD_HANDLERS, DOWNLOADER_MIDDLEWARES, and SPIDER_MIDDLEWARES. This method allows for finer control over component integration.

```python
DOWNLOAD_HANDLERS = {
    "http": "scrapy_zyte_api.ScrapyZyteAPIDownloadHandler",
    "https": "scrapy_zyte_api.ScrapyZyteAPIDownloadHandler",
}
DOWNLOADER_MIDDLEWARES = {
    "scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware": 633,
}
SPIDER_MIDDLEWARES = {
    "scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware": 100,
    "scrapy_zyte_api.ScrapyZyteAPIRefererSpiderMiddleware": 1000,
}
REQUEST_FINGERPRINTER_CLASS = "scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter"
TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"
```

--------------------------------

### Example Scrapy to Zyte API Mapping

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/reference/request.md

Shows a concrete example of how a Scrapy Request gets transformed into Zyte API parameters.

```APIDOC
## Example Scrapy to Zyte API Mapping

### Scrapy Request Example
```python
Request(
    method="POST",
    url="https://httpbin.org/anything",
    headers={"Content-Type": "application/json"},
    body=b'{"foo": "bar"}',
    cookies={"a": "b"}
)
```

### Resulting Zyte API Parameters
```javascript
{
    "customHttpRequestHeaders": [
        {"name": "Content-Type", "value": "application/json"}
    ],
    "httpRequestMethod": "POST",
    "httpRequestBody": "eyJmb28iOiAiYmFyIn0=",
    "experimental.requestCookies": [
        {"name": "a", "value": "b"}
    ],
    "url": "https://httpbin.org/anything"
}
```
```

--------------------------------

### Enable Scrapy-Zyte-API Add-on

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Configures scrapy-zyte-api by enabling its add-on in Scrapy's ADDONS setting. This requires Scrapy version 2.10 or higher and enables transparent mode by default.

```python
ADDONS = {
    "scrapy_zyte_api.Addon": 500,
}
```

--------------------------------

### Configure Scrapy-Poet Provider for Zyte API

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Adds the ZyteApiProvider to the SCRAPY_POET_PROVIDERS setting, enabling scrapy-poet integration with Zyte API. This should be done after configuring scrapy-poet itself.

```python
SCRAPY_POET_PROVIDERS = {
    "scrapy_zyte_api.providers.ZyteApiProvider": 1100,
}
```

--------------------------------

### Configure Zyte API Key (Environment Variable)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Sets the ZYTE_API_KEY environment variable for authentication with Zyte API. This is a common method for securely providing API credentials without hardcoding them.

```shell
# On Windows’ CMD:
> set ZYTE_API_KEY=YOUR_API_KEY

# On macOS and Linux:
$ export ZYTE_API_KEY=YOUR_API_KEY
```

--------------------------------

### Enable Session Management Middleware

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Adds the ScrapyZyteAPISessionDownloaderMiddleware to DOWNLOADER_MIDDLEWARES for session management support. This middleware handles session-related logic for requests made through Zyte API.

```python
DOWNLOADER_MIDDLEWARES = {
    "scrapy_zyte_api.ScrapyZyteAPISessionDownloaderMiddleware": 667,
}
```

--------------------------------

### Configure Zyte API Key (Settings Module)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Sets the ZYTE_API_KEY variable directly within the Scrapy settings module for authentication. This method is suitable for development or when environment variables are not preferred.

```python
ZYTE_API_KEY = "YOUR_API_KEY"
```

--------------------------------

### Configure Fallback Request Fingerprinter

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Sets ZYTE_API_FALLBACK_REQUEST_FINGERPRINTER_CLASS to a custom value when a custom REQUEST_FINGERPRINTER_CLASS is already in use. This ensures compatibility and proper request handling.

```python
ZYTE_API_FALLBACK_REQUEST_FINGERPRINTER_CLASS = "myproject.CustomRequestFingerprinter"
```

--------------------------------

### Install and Configure Scrapy Zyte API

Source: https://context7.com/scrapy-plugins/scrapy-zyte-api/llms.txt

Instructions for installing the scrapy-zyte-api library, including options for scrapy-poet integration. It also shows how to configure authentication and middleware settings in Scrapy's settings.py file for both Scrapy 2.10+ and older versions.

```python
# Install basic version
# pip install scrapy-zyte-api

# Install with scrapy-poet integration
# pip install scrapy-zyte-api[provider]

# settings.py - Scrapy 2.10+ with addon (recommended)
ADDONS = {
    "scrapy_zyte_api.Addon": 500,
}
ZYTE_API_KEY = "YOUR_API_KEY"  # or set ZYTE_API_KEY environment variable

# settings.py - Manual component configuration (Scrapy < 2.10)
DOWNLOAD_HANDLERS = {
    "http": "scrapy_zyte_api.ScrapyZyteAPIDownloadHandler",
    "https": "scrapy_zyte_api.ScrapyZyteAPIDownloadHandler",
}
DOWNLOADER_MIDDLEWARES = {
    "scrapy_zyte_api.ScrapyZyteAPIDownloaderMiddleware": 633,
}
SPIDER_MIDDLEWARES = {
    "scrapy_zyte_api.ScrapyZyteAPISpiderMiddleware": 100,
}
REQUEST_FINGERPRINTER_CLASS = "scrapy_zyte_api.ScrapyZyteAPIRequestFingerprinter"
TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"
ZYTE_API_TRANSPARENT_MODE = True  # Route all requests through Zyte API
```

--------------------------------

### Enable Zyte API Transparent Mode

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/setup.md

Sets ZYTE_API_TRANSPARENT_MODE to True to switch the spider to use Zyte API for all requests by default. This simplifies configuration when Zyte API should be the primary request handler.

```python
ZYTE_API_TRANSPARENT_MODE = True
```

--------------------------------

### Zyte API Request Configuration Example

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/reference/request.md

This JSON object represents a sample request configuration for the Zyte API, demonstrating various parameters such as HTTP method, URL, request body, and experimental settings for cookies. It shows how to specify headers and other request details.

```json
{
    "metadata": {
        "_default": {
            "headers": [
                {
                    "name": "Content-Type",
                    "value": "application/json"
                }
            ]
        }
    },
    "experimental": {
        "requestCookies": [
            {
                "name": "a",
                "value": "b",
                "domain": ""
            }
        ],
        "responseCookies": true
    },
    "httpResponseBody": true,
    "httpResponseHeaders": true,
    "httpRequestBody": "eyJmb28iOiAiYmFyIn0=",
    "httpRequestMethod": "POST",
    "url": "https://httpbin.org/anything"
}
```

--------------------------------

### Zyte API Response Mapping Example (HTTP Body and Headers)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/reference/response.md

Demonstrates accessing mapped response attributes (URL, status, headers, text, body) and the raw API response for a request including HTTP body and headers. This example assumes these parameters were requested from Zyte API.

```python
def parse(self, response):
    print(response.url)
    # "https://quotes.toscrape.com/"
    print(response.status)
    # 200
    print(response.headers)
    # {b"Content-Type": [b"text/html"], …}
    print(response.text)
    # "<html>…</html>"
    print(response.body)
    # b"<html>…</html>"
    print(response.raw_api_response)
    # {
    #     "url": "https://quotes.toscrape.com/",
    #     "statusCode": 200,
    #     "httpResponseBody": "PGh0bWw+4oCmPC9odG1sPg==",
    #     "httpResponseHeaders": […],
    # }
```

--------------------------------

### Zyte API Request Parameters Example (JSON)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/usage/automap.md

Illustrates the resulting Zyte API request parameters in JSON format when using automatic mapping with specific modifications, such as enabling 'browserHtml' and setting custom HTTP headers. This example clarifies how Scrapy meta information is translated into Zyte API's data extraction endpoint parameters.

```javascript
{
    "browserHtml": true,
    "experimental": {
        "responseCookies": true
    },
    "requestHeaders": {"referer": "https://example.com/"},
    "url": "https://quotes.toscrape.com"
}
```

--------------------------------

### Set Zyte API Provider Params in Python

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/usage/scrapy-poet.md

Configures default extraction options for Zyte API requests using the ZYTE_API_PROVIDER_PARAMS setting. This example sets extraction sources for product and product navigation options.

```python
ZYTE_API_PROVIDER_PARAMS = {
    "productOptions": {"extractFrom": "httpResponseBody"},
    "productNavigationOptions": {"extractFrom": "httpResponseBody"},
}
```

--------------------------------

### Zyte API Provider Parameters Configuration (Python)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/reference/settings.md

Defines additional parameters for Zyte API requests, particularly for the scrapy-poet integration. This example shows how to set custom request cookies.

```python
ZYTE_API_PROVIDER_PARAMS = {
    "requestCookies": [
        {"name": "a", "value": "b", "domain": "example.com"},
    ],
}
```

--------------------------------

### Manual Zyte API Request Parameters in Scrapy

Source: https://context7.com/scrapy-plugins/scrapy-zyte-api/llms.txt

Example demonstrating how to control Zyte API parameters for individual Scrapy requests using the `zyte_api` metadata key. This includes enabling browser rendering, taking screenshots, performing actions like clicking elements, and configuring custom HTTP request headers for API calls.

```python
import scrapy


class ProductSpider(scrapy.Spider):
    name = "product_spider"

    def start_requests(self):
        # Request with browser rendering and screenshot
        yield scrapy.Request(
            url="https://quotes.toscrape.com/",
            meta={
                "zyte_api": {
                    "browserHtml": True,
                    "screenshot": True,
                    "actions": [
                        {
                            "action": "click",
                            "selector": {"type": "css", "value": "button#load-more"},
                        },
                    ],
                }
            },
            callback=self.parse_product,
        )

        # HTTP response with custom headers
        yield scrapy.Request(
            url="https://api.example.com/data",
            meta={
                "zyte_api": {
                    "httpResponseBody": True,
                    "httpResponseHeaders": True,
                    "customHttpRequestHeaders": {
                        "Authorization": "Bearer token123",
                        "Accept": "application/json",
                    },
                }
            },
            callback=self.parse_api,
        )

    def parse_product(self, response):
        # Access raw API response including screenshot
        raw_response = response.raw_api_response
        screenshot_base64 = raw_response.get("screenshot")

        # Parse rendered HTML
        for quote in response.css(".quote"):
            yield {
                "text": quote.css(".text::text").get(),
                "author": quote.css(".author::text").get(),
            }

    def parse_api(self, response):
        import json
        data = json.loads(response.text)
        yield {"items": data}
```

--------------------------------

### Direct Dependency Injection in Parse Callbacks (Python)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/usage/scrapy-poet.md

This example shows injecting dependencies like DummyResponse, BrowserResponse, and Product directly into the parse_page callback without a page object. It depends on scrapy and zyte_common_items for types. Inputs are from Zyte API requests; outputs are processed in the method. No specific limitations beyond standard Scrapy setup.

```python
class ZyteApiSpider(scrapy.Spider):
    ...

    def parse_page(self,
                   response: DummyResponse,
                   browser_response: BrowserResponse,
                   product: Product,
                   ):
        ...
```

--------------------------------

### POST /anything

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/reference/request.md

This endpoint demonstrates sending a POST request with custom headers, request body, and options for capturing HTTP response body and headers.

```APIDOC
## POST /anything

### Description
This endpoint allows you to send a POST request to a specified URL. It supports custom headers, request cookies, and provides options to include the HTTP response body and headers in the result. The request body is base64 encoded.

### Method
POST

### Endpoint
https://httpbin.org/anything

### Parameters
#### Query Parameters
None

#### Request Body
```json
{
    "name": "Content-Type",
    "value": "application/json"
}
```

### Request Example
```json
{
    "tool": "httpbin",
    "httpResponseBody": true,
    "httpResponseHeaders": true,
    "httpRequestMethod": "POST",
    "url": "https://httpbin.org/anything",
    "httpRequestBody": "eyJmb28iOiAiYmFyIn0=",
    "experimental": {
        "requestCookies": [
            {
                "name": "a",
                "value": "b",
                "domain": ""
            }
        ],
        "responseCookies": true
    }
}
```

### Response
#### Success Response (200)
- **url** (string) - The URL that was requested.
- **headers** (object) - Headers sent with the request.
- **httpResponse** (object) - Contains the response body and headers from the target URL.
    - **body** (string) - The base64 encoded response body.
    - **headers** (object) - The response headers.

#### Response Example
```json
{
    "args": {},
    "data": "",
    "files": {},
    "form": {},
    "headers": {
        "Accept": "*/*",
        "Accept-Encoding": "gzip, deflate",
        "Content-Length": "13",
        "Content-Type": "application/json",
        "Host": "httpbin.org",
        "User-Agent": "python-requests/2.28.1",
        "X-Amzn-Trace-Id": "Root=1-65f4f8a0-1d6e7f8a0-1d6e7f8a0-1d6e7f8a0"
    },
    "json": {
        "foo": "bar"
    },
    "origin": "192.168.1.1",
    "url": "https://httpbin.org/anything"
}
```
```

--------------------------------

### Session Parameters Configuration

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/usage/session.md

This section describes how to configure the parameters for initializing a Zyte API session based on a Scrapy request.

```APIDOC
## params(request: Request) -> Dict[str, Any]

### Description
Returns the Zyte API request parameters to use to initialize a session for the given Scrapy request. The default implementation relies on settings and request metadata keys.

### Method
Python function (within a Scrapy Spider or middleware)

### Parameters
- **request** (Scrapy Request) - The Scrapy Request object for which to generate session parameters.

### Request Body
(Not directly applicable, as this method generates parameters for a Zyte API request)

### Request Example (Python)
```python
def params(self, request: Request) -> Dict[str, Any]:
    if location := self.location(request):
        return {
            "url": "https://example.com/new-session/for-country",
            "httpResponseBody": True,
            "httpRequestMethod": "POST",
            "httpRequestText": location["addressCountry"],
        }
    return {
        "url": "https://example.com/new-session",
        "httpResponseBody": True,
    }
```

### Response
#### Success Response (Dict)
- **url** (str) - The URL for the Zyte API session initialization.
- **httpResponseBody** (bool) - Whether to include the HTTP response body.
- **httpRequestMethod** (str) - The HTTP method for the request.
- **httpRequestText** (str) - Text content for the request.

#### Response Example
```json
{
  "url": "https://example.com/new-session",
  "httpResponseBody": true
}
```

#### SEE ALSO
[`LocationSessionConfig`](#scrapy_zyte_api.LocationSessionConfig)
```

--------------------------------

### Location-Based Session Initialization in Scrapy with Python

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/usage/session.md

Returns a dictionary for location-based session initialization. Default implementation uses settings and request metadata. Override to provide fallback locations.

```python
def location(self, request: Request) -> Dict[str, str]:
    fallback = {"addressCountry": "US", "addressRegion": "NY", "postalCode": "10001"}
    return super().location(request) or fallback
```

```python
def location(self, request: Request) -> Dict[str, str]:
    fallback = {}
    if postal_code := w3lib.url.url_query_parameter(request.url, "postalCode"):
        fallback["postalCode"] = postal_code
    return super().location(request) or fallback
```

--------------------------------

### Create Request with dont_filter=True (Python)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/usage/fingerprint.md

Example of creating Scrapy requests with `dont_filter=True` to prevent duplicate filtering while ensuring different requests are treated as distinct.

```python
yield Request(
    "https://toscrape.com#1",
    meta="{"zyte_api_automap": {"browserHtml": True}}",
    dont_filter=True,
)
yield Request(
    "https://toscrape.com#2",
    meta="{"zyte_api_automap": {"browserHtml": True}}",
    dont_filter=True,
)
```

--------------------------------

### Scrapy-Poet Integration

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/CHANGES.rst

Demonstrates the configuration of scrapy-poet and scrapy-zyte-api addons in Scrapy. This ensures proper integration and avoids conflicts between the two addons.

```python
ADDONS = {
    "scrapy_poet.Addon": 300,
    "scrapy_zyte_api.Addon": 500,
}
```

--------------------------------

### Modify Request with CSRF Token in Python

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/usage/session.md

This Python example illustrates how to use stored session data to modify an outgoing request. It retrieves the CSRF token associated with the session ID and adds it to the request's headers.

```python
def process_request(self, request: Request) -> Optional[Request]:
    session_id = scrapy_zyte_api.get_request_session_id(request)
    csrf_token = self.session_data[session_id]["csrf_token"]
    request.headers["CSRF-Token"] = csrf_token
```

--------------------------------

### Unsupported Scenarios and Parameter Mapping

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/reference/request.md

Explains how Scrapy Zyte API maps request parameters like method, headers, and body, even for scenarios or values not directly supported by Zyte API, and how to control this mapping.

```APIDOC
## Unsupported Scenarios and Parameter Mapping

This section details how Scrapy Zyte API handles certain request parameters to ensure broader compatibility, even if the specific configurations are not directly supported by the Zyte API.

### Method Mapping
- `Request.method` is mapped to `httpRequestMethod`, regardless of whether the HTTP method is supported by Zyte API or if `httpResponseBody` is unset.

### Header Mapping
- `customHttpRequestHeaders` or `requestHeaders` can be set to `True` to force mapping from `Request.headers` in cases where they would not be mapped otherwise.
- Conversely, setting `customHttpRequestHeaders` or `requestHeaders` to `False` prevents their mapping from `Request.headers`.

### Body Mapping
- `Request.body` is mapped to `httpRequestBody`, even if `httpResponseBody` is unset.

### Response Body and Header Mapping Scenarios
- If `httpResponseBody` is set to `False` (unsetting the parameter) and no other output parameters (like `browserHtml`, `screenshot`, `product`) are set to `True`, then `Request.headers` are mapped as `requestHeaders`.
- If `httpResponseBody` is set to `True` (or automatic extraction from `httpResponseBody` is used), and `browserHtml` or `screenshot` is also set to `True` (or automatic extraction is used), then `Request.headers` are mapped as both `customHttpRequestHeaders` and `requestHeaders`. In this case, `browserHtml` is used as `response.body`.
```

--------------------------------

### Zyte API Session Checker Implementation (Python)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/reference/settings.md

Customizes session validation for Zyte API requests. This example demonstrates a basic checker that verifies if a response contains a specific CSS selector, and an advanced version that reads settings from the crawler.

```python
from scrapy import Request
from scrapy.http.response import Response


class MySessionChecker:

    def check(self, response: Response, request: Request) -> bool:
        return bool(response.css(".is_valid"))


ZYTE_API_SESSION_CHECKER = MySessionChecker
```

```python
from scrapy import Request
from scrapy.http.response import Response


class MySessionChecker:

    @classmethod
    def from_crawler(cls, crawler):
        return cls(crawler)

    def __init__(self, crawler):
        location = crawler.settings["ZYTE_API_SESSION_LOCATION"]
        self.postal_code = location["postalCode"]

    def check(self, response: Response, request: Request) -> bool:
        return response.css(".postal_code::text").get() == self.postal_code


ZYTE_API_SESSION_CHECKER = MySessionChecker
```

--------------------------------

### Configure Automatic Sessions in Scrapy Spider

Source: https://context7.com/scrapy-plugins/scrapy-zyte-api/llms.txt

Demonstrates automatic session management across request chains using zyte_api_automap metadata. The first request initializes a persistent session with browserHtml enabled, maintaining login state and cookies for subsequent product page requests. Uses response.follow() to continue within the same session context.

```python
import scrapy


class SessionSpider(scrapy.Spider):
    name = "session_spider"

    def start_requests(self):
        # All requests automatically use sessions
        # First request initializes a session with location
        yield scrapy.Request(
            url="https://store.example.com/products",
            meta={"zyte_api_automap": {"browserHtml": True}},
        )

    def parse(self, response):
        # Session is maintained across requests
        self.logger.info(f"Logged in, session active")

        for product in response.css(".product"):
            yield response.follow(
                product.css("a::attr(href)").get(),
                callback=self.parse_product,
                # Same session continues to be used
            )

    def parse_product(self, response):
        yield {
            "name": response.css("h1::text").get(),
            "price": response.css(".price::text").get(),
        }
```

--------------------------------

### Extract product data with Zyte API automatic extraction

Source: https://context7.com/scrapy-plugins/scrapy-zyte-api/llms.txt

Implements a Scrapy spider that automatically extracts product information using Zyte API's built-in product extraction. The spider processes multiple product URLs and yields structured product data including name, price, currency, availability, description and images. Depends on zyte_common_items.Product and scrapy_poet.DummyResponse.

```python
class ProductSpider(scrapy.Spider):
    name = "product_poet_spider"

    def start_requests(self):
        urls = [
            "https://www.example.com/products/laptop-123",
            "https://www.example.com/products/phone-456",
        ]
        for url in urls:
            yield scrapy.Request(url)

    def parse(self, response: DummyResponse, product: Product):
        # Product automatically extracted by Zyte API
        yield {
            "url": product.url,
            "name": product.name,
            "price": product.price,
            "currency": product.currency,
            "availability": product.availability,
            "description": product.description,
            "images": [img.url for img in product.images] if product.images else [],
        }
```

--------------------------------

### Zyte API Response Mapping Example (Screenshot)

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/reference/response.md

Illustrates accessing response attributes when a screenshot was requested from Zyte API. Shows how to retrieve the URL, status, and the base64-encoded screenshot from the raw API response, and decode it.

```python
def parse(self, response):
    print(response.url)
    # "https://quotes.toscrape.com/"
    print(response.status)
    # 200
    print(response.headers)
    # {}
    print(response.text)
    # ""
    print(response.body)
    # b""
    print(response.raw_api_response)
    # {
    #     "url": "https://quotes.toscrape.com/",
    #     "statusCode": 200,
    #     "screenshot": "iVBORw0KGgoAAAANSUh…",
    # }
    from base64 import b64decode

    print(b64decode(response.raw_api_response["screenshot"]))
    # b'\x89PNG\r\n\x1a\n\x00\x00\x00\r…'
```

--------------------------------

### Define custom page object with dependencies

Source: https://context7.com/scrapy-plugins/scrapy-zyte-api/llms.txt

Defines a custom page object using attrs with multiple Zyte API dependencies including Product extraction, Geolocation setting, and CustomAttributes definition. Includes a computed field for warranty information. Requires web_poet.field decorator and various Zyte API annotations for configuration.

```python
@attrs.define
class CustomProductPage:
    """Custom page object with multiple dependencies."""
    product: Annotated[Product, ExtractFrom.httpResponseBody]
    geolocation: Annotated[Geolocation, "US"]  # Set location to US
    custom_attributes: Annotated[
        CustomAttributes,
        custom_attrs(
            {"warranty": {"type": "string", "description": "Product warranty period"}},
            {"method": "generate"},
        ),
    ]

    @field
    def warranty_info(self):
        return self.custom_attributes.values.get("warranty", "N/A")
```

--------------------------------

### Header Mapping

Source: https://github.com/scrapy-plugins/scrapy-zyte-api/blob/main/docs/reference/request.md

Explains how Zyte API handles HTTP headers, including default drops and how to manage them.

```APIDOC
## Header Mapping

### Description
This section details how Zyte API manages HTTP headers. Certain headers are dropped by default based on settings like `ZYTE_API_SKIP_HEADERS` and `ZYTE_API_BROWSER_HEADERS`. It also explains how to force the mapping of headers that might otherwise be dropped.

### Dropped Headers
The following headers may be dropped if their values are not user-defined or come from default Scrapy settings:

- **Accept** and **Accept-Language**: Dropped if they originate from the default `DEFAULT_REQUEST_HEADERS`.
- **Accept-Encoding**: Dropped if set by `HttpCompressionMiddleware`.
- **User-Agent**: Dropped if it originates from the default `USER_AGENT` setting.

### Forcing Header Mapping
To ensure these headers are mapped, you can:

- Define them in `DEFAULT_REQUEST_HEADERS`.
- Set them in `Request.headers` from a spider callback.
- Ensure they have a non-default value elsewhere before the scrapy-zyte-api middleware processes the request.

### Referer Header
By default, Scrapy sets the `Referer` header. To unset it for Zyte API requests:

- Add `Referer` to `ZYTE_API_SKIP_HEADERS`.
- Remove it from `ZYTE_API_BROWSER_HEADERS`.

To unset it globally, set `REFERER_ENABLED` to `False`.
```