### Installation

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

Command to install the Amazon Scraper tool.

```make
make install
```

--------------------------------

### Installation

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/INDEX.md

Installs the amazon-scraper package.

```bash
cd amazon-scraper
make install
```

--------------------------------

### No Products Found Error Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

Example message when no products are found on the page.

```bash
INFO:amazon_scraper.collector:No products found for given Amazon page.
```

--------------------------------

### Chrome Not Installed Error Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

Example error messages when Chrome is not installed or ChromeDriver is not available.

```bash
ERROR:amazon_scraper.scraper:Unable to initialize Chrome webdriver for scraping.
ERROR:amazon_scraper.collector:Error when scraping Amazon for url https://www.amazon.com/s?k=test
```

--------------------------------

### Install and Run with Make

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

Instructions for installing the package in development mode and running the scraper using make commands.

```bash
cd /path/to/amazon-scraper
make install
```

```bash
make scrape URL="https://www.amazon.com/s?k=laptops"
```

--------------------------------

### Scrape a Category Page Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

Example of scraping a category page.

```bash
python -m amazon_scraper --url "https://www.amazon.com/s?i=specialty-aps&bbn=16225009011&rh=n%3A16225009011"
```

--------------------------------

### Scrape Search Results Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

Example of scraping search results.

```bash
python -m amazon_scraper --url "https://www.amazon.com/s?k=laptops"
```

--------------------------------

### scrape_amazon CLI Examples

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/configuration.md

Examples of using the scrape_amazon command to scrape different types of Amazon pages.

```bash
# Scrape a category page
python -m amazon_scraper --url "https://www.amazon.com/s?i=specialty-aps&bbn=16225009011"

# Scrape search results
python -m amazon_scraper --url "https://www.amazon.com/s?k=laptops"

# Scrape with quoted URL (recommended)
python -m amazon_scraper --url "https://www.amazon.com/s?k=electronics&page=1"
```

--------------------------------

### Programmatic Usage Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

Example of using the scraper from Python code.

```python
from amazon_scraper.collector import AmazonDataCollector

collector = AmazonDataCollector()
collector.collect_amazon_product_data(
    "https://www.amazon.com/s?k=electronics"
)
```

--------------------------------

### AmazonDataCollector Usage Examples

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/configuration.md

Examples of initializing AmazonDataCollector with default settings, custom output file, custom logger, or both.

```python
from amazon_scraper.collector import AmazonDataCollector

# Use all defaults (saves to "amazon_products.csv" in current directory)
collector = AmazonDataCollector()

# Specify custom output file
collector = AmazonDataCollector(output_file="/path/to/products.csv")

# Use custom logger
import logging
logger = logging.getLogger("amazon")
collector = AmazonDataCollector(logger=logger)

# Combine both options
logger = logging.getLogger("my_app")
collector = AmazonDataCollector(
    output_file="data/amazon_products.csv",
    logger=logger
)
```

--------------------------------

### Scrape with Multiple Query Parameters Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

Example of scraping with multiple query parameters.

```bash
python -m amazon_scraper --url "https://www.amazon.com/s?k=electronics&i=electronics&ref=nb_sb_noss"
```

--------------------------------

### Full Example Usage

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/MODULES.md

A comprehensive example demonstrating how to use both the high-level API (AmazonDataCollector) and the lower-level scraper with error handling.

```python
from amazon_scraper.collector import AmazonDataCollector
from amazon_scraper.scraper import AmazonScraper, DriverInitializationError, DriverGetProductsError
from amazon_scraper.models import Product

# Use high-level API
collector = AmazonDataCollector(output_file="my_products.csv")
collector.collect_amazon_product_data("https://www.amazon.com/s?k=electronics")

# Or use lower-level scraper with error handling
scraper = AmazonScraper()
try:
    products = scraper.scrape_amazon_page(url)
    for product in products:
        print(product.title, product.price)
except DriverInitializationError:
    print("Browser initialization failed")
except DriverGetProductsError:
    print("Scraping failed")
```

--------------------------------

### DriverInitializationError Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/errors.md

Example of catching DriverInitializationError when the Chrome webdriver fails to initialize.

```python
from amazon_scraper.scraper import AmazonScraper, DriverInitializationError

scraper = AmazonScraper()

try:
    products = scraper.scrape_amazon_page("https://www.amazon.com/s?k=electronics")
except DriverInitializationError as e:
    print("Chrome webdriver failed to initialize")
    print("Check that Chrome is installed and ChromeDriver is available")
```

--------------------------------

### Page Not Found Error Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

Example error messages when a page is not found.

```bash
ERROR:amazon_scraper.scraper:Unable to get Amazon product data with Chrome webdriver.
ERROR:amazon_scraper.collector:Error when scraping Amazon for url https://www.amazon.com/invalid
```

--------------------------------

### Example scrape command

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

Example of the scrape command with a specific Amazon department URL.

```make
make scrape URL="https://www.amazon.com/s?i=specialty-aps&bbn=16225009011&rh=n%3A%2116225009011%2Cn%3A541966&ref=nav_em__nav_desktop_sa_intl_computers_and_accessories_0_2_5_6"
```

--------------------------------

### Custom Logger Configuration Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/configuration.md

Example demonstrating how to configure a custom logger for the Amazon Scraper, including file and console handlers with different logging levels.

```python
import logging
from amazon_scraper.collector import AmazonDataCollector

# Create a logger with file output
logger = logging.getLogger("amazon_scraper")
logger.setLevel(logging.DEBUG)

# File handler
file_handler = logging.FileHandler("scraper.log")
file_handler.setLevel(logging.DEBUG)

# Console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)

# Formatter
formatter = logging.Formatter(
    "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)
file_handler.setFormatter(formatter)
console_handler.setFormatter(formatter)

# Add handlers
logger.addHandler(file_handler)
logger.addHandler(console_handler)

# Use with collector
collector = AmazonDataCollector(logger=logger)
```

--------------------------------

### CSV Export Format Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/product-model.md

Example of how product data is exported to CSV format.

```csv
title,url,asin_code,image_url,price
Laptop Computer,https://www.amazon.com/dp/B001,B001,https://m.media-amazon.com/images/I/1.jpg,899.99
USB Cable,https://www.amazon.com/dp/B002,B002,https://m.media-amazon.com/images/I/2.jpg,9.99
Out of Stock Item,https://www.amazon.com/dp/B003,B003,https://m.media-amazon.com/images/I/3.jpg,
```

--------------------------------

### Python code example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

Example of how to use the Amazon Scraper API with Python to retrieve bestsellers.

```python
import requests
from pprint import pprint


# Structure payload.
payload = {
    'source': 'amazon_bestsellers',
    'domain': 'de',
    'query': 'automotive',
    'start_page': 2,
    'parse': True,
    'context': [
        {'key': 'category_id', 'value': 82400031},
    ],
}


# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

# Print prettified response to stdout.
pprint(response.json())
```

--------------------------------

### Default Logging Setup

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/configuration.md

The basic logging configuration used by the CLI.

```python
logging.basicConfig(level=logging.INFO)
```

--------------------------------

### Re-export Example (Correct vs. Incorrect)

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/MODULES.md

Illustrates the correct way to import from submodules versus an incorrect attempt at top-level import.

```python
# ✓ Correct
from amazon_scraper.collector import AmazonDataCollector

# ✗ Not available
from amazon_scraper import AmazonDataCollector  # Would fail
```

--------------------------------

### Python code example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

This Python code example demonstrates how to make a request to retrieve an Amazon product page using the Oxylabs Amazon Scraper API.

```python
import requests
from pprint import pprint


# Structure payload.
payload = {
    'source': 'amazon',
    'url': 'https://www.amazon.co.uk/dp/B0BDJ279KF',
    'parse': True
}

# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('YOUR_USERNAME', 'YOUR_PASSWORD'), #Your credentials go here
    json=payload,
)

# Instead of response with job status and results url, this will return the
# JSON response with results.
pprint(response.json())

```

--------------------------------

### Example: Error Handling

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/amazon-scraper.md

Illustrates how to handle potential exceptions like DriverInitializationError and DriverGetProductsError when using the AmazonScraper.

```python
from amazon_scraper.scraper import AmazonScraper, DriverInitializationError, DriverGetProductsError

scraper = AmazonScraper()

try:
    products = scraper.scrape_amazon_page("https://www.amazon.com/s?k=laptops")
except DriverInitializationError as e:
    print("Failed to initialize Chrome webdriver. Check ChromeDriver installation.")
except DriverGetProductsError as e:
    print("Failed to scrape products from the page. Check URL validity.")
else:
    print(f"Successfully scraped {len(products)} products")
```

--------------------------------

### Logging Configuration

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/INDEX.md

Example of how to configure logging to see detailed debug information.

```python
import logging
logging.basicConfig(level=logging.DEBUG)
```

--------------------------------

### AmazonScraper Usage with Logger

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/configuration.md

Examples of initializing AmazonScraper with default and custom logger instances.

```python
import logging
from amazon_scraper.scraper import AmazonScraper

# Use default logger
scraper = AmazonScraper()

# Use custom logger
custom_logger = logging.getLogger("my_scraper")
scraper = AmazonScraper(logger=custom_logger)
```

--------------------------------

### Python Code Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

This example demonstrates how to use the requests library in Python to send a POST request to the Oxylabs Amazon Scraper API to retrieve seller page data.

```python
import requests
from pprint import pprint


# Structure payload.
payload = {
    'source': 'amazon_sellers',
    'domain': 'de',
    'query': 'ABNP0A7Y0QWBN',
    'parse': True
}


# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

# Print prettified response to stdout.
pprint(response.json())
```

--------------------------------

### Example: With Custom Logger

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/amazon-scraper.md

Shows how to create and use a custom logger with the AmazonScraper for file-based logging.

```python
import logging
from amazon_scraper.scraper import AmazonScraper

# Create a custom logger with file output
logger = logging.getLogger("my_scraper")
handler = logging.FileHandler("scraper.log")
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)

# Create scraper with custom logger
scraper = AmazonScraper(logger=logger)

# Scrape with logging to file
products = scraper.scrape_amazon_page("https://www.amazon.com/s?k=electronics")

print(f"Scraped {len(products)} products")
```

--------------------------------

### Python Code Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

Example of how to make a request to retrieve product page data from Amazon using the Oxylabs Amazon Scraper API.

```python
import requests
from pprint import pprint


# Structure payload.
payload = {
    'source': 'amazon_search',
    'domain': 'nl',
    'query': 'adidas',
    'start_page': 11,
    'pages': 10,
    'parse': True,
    'context': [
        {'key': 'category_id', 'value': 16391843031},
        {'key': 'merchant_id', 'value':'3AA17D2BRD4YMT0X'}
    ],
}


# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

```

--------------------------------

### Example: Basic Usage

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/amazon-scraper.md

Demonstrates basic usage of the AmazonScraper to scrape products from a category page and print their details.

```python
import logging
from amazon_scraper.scraper import AmazonScraper

# Configure logging
logging.basicConfig(level=logging.INFO)

# Create a scraper
scraper = AmazonScraper()

# Scrape products from an Amazon category page
url = "https://www.amazon.com/s?i=specialty-aps&bbn=16225009011"
products = scraper.scrape_amazon_page(url)

# Print product information
for product in products:
    print(f"Title: {product.title}")
    print(f"ASIN: {product.asin_code}")
    print(f"Price: {product.price}")
    print(f"URL: {product.url}")
    print("---")
```

--------------------------------

### Python Code Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

Example of how to use the Amazon Scraper API with Python requests library to retrieve data.

```python
import requests
from pprint import pprint


# Structure payload.
payload = {
    'source': 'amazon_questions',
    'domain': 'nl',
    'query': 'B09RX4KS1G',
    'parse': True,
}


# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

# Print prettified response to stdout.
pprint(response.json())
```

--------------------------------

### Python code example for Amazon Product Scraper

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

Example of how to use the Amazon Product Scraper in Python to retrieve product page data for a specific ASIN and domain, with an option to auto-select product variations.

```python
import requests
from pprint import pprint

```

--------------------------------

### BaseException Usage Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/errors.md

Demonstrates how to raise and catch the BaseException with a custom message.

```python
from amazon_scraper.exception import BaseException

try:
    raise BaseException("Custom error message")
except BaseException as e:
    print(str(e))  # Output: "Custom error message"
```

--------------------------------

### CSV Output Format Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

Illustrates the structure and content of the CSV file generated by the Amazon Scraper.

```csv
title,url,asin_code,image_url,price
"Product 1","https://www.amazon.com/dp/B001","B001","https://...",29.99
"Product 2","https://www.amazon.com/dp/B002","B002","https://...",
"Product 3","https://www.amazon.com/dp/B003","B003","https://...",49.99
```

--------------------------------

### Example Usage

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/amazon-data-collector.md

Scrapes products from a given Amazon URL and saves them to a default CSV file named 'amazon_products.csv'.

```python
import logging
from amazon_scraper.collector import AmazonDataCollector

# Configure logging to see info and error messages
logging.basicConfig(level=logging.INFO)

# Create a collector with default settings (saves to amazon_products.csv)
collector = AmazonDataCollector()

# Scrape and save products
amazon_url = "https://www.amazon.com/s?i=specialty-aps&bbn=16225009011&rh=n%3A16225009011"
collector.collect_amazon_product_data(amazon_url)

# The file amazon_products.csv will now contain all scraped products
```

--------------------------------

### Catching All Scraper Errors

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/errors.md

Example of how to catch specific scraper errors (DriverInitializationError, DriverGetProductsError) and general exceptions.

```python
from amazon_scraper.scraper import AmazonScraper, DriverInitializationError, DriverGetProductsError

scraper = AmazonScraper()

try:
    products = scraper.scrape_amazon_page(url)
except (DriverInitializationError, DriverGetProductsError) as e:
    print(f"Scraping failed: {str(e)}")
except Exception as e:
    print(f"Unexpected error: {str(e)}")
```

--------------------------------

### Custom Output File Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/amazon-data-collector.md

Creates a collector that saves scraped product data to a custom-specified CSV file.

```python
from amazon_scraper.collector import AmazonDataCollector

# Create a collector that saves to a custom file
collector = AmazonDataCollector(output_file="my_amazon_products.csv")

# Scrape and save to the custom file
collector.collect_amazon_product_data(
    "https://www.amazon.com/s?k=laptops"
)
```

--------------------------------

### Products will be skipped and you'll see error logs but no exception raised

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/errors.md

Example demonstrating how products might be skipped with error logs instead of exceptions.

```python
# products will be skipped and you'll see error logs but no exception raised:
products = scraper.scrape_amazon_page("https://www.amazon.com/s?k=electronics")

# If products list is empty, it likely means all products had missing data
if not products:
    print("No products could be parsed - check Amazon page structure")
```

--------------------------------

### Amazon Pricing Scraper Payload and Request

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

Python code example for retrieving product offer listings using the Amazon Pricing Scraper API.

```python
import requests
from pprint import pprint


# Structure payload.
payload = {
    'source': 'amazon_pricing',
    'domain': 'nl',
    'query': 'B09RX4KS1G',
    'parse': True,
}


# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

# Print prettified response to stdout.
pprint(response.json())
```

--------------------------------

### Scrape and Get Python Objects

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

Use AmazonScraper to scrape product data from a URL and receive it as a list of Python objects, then iterate and print product titles and prices.

```python
from amazon_scraper.scraper import AmazonScraper

scraper = AmazonScraper()
products = scraper.scrape_amazon_page(
    "https://www.amazon.com/s?k=electronics"
)

for product in products:
    print(f"{product.title}: {product.price}")
```

--------------------------------

### Command-Line Usage

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

Demonstrates how to use the Amazon Scraper from the command line, both directly and with a Makefile.

```bash
# Basic
python -m amazon_scraper --url "https://www.amazon.com/s?k=electronics"

# With Makefile
make scrape URL="https://www.amazon.com/s?k=electronics"
```

--------------------------------

### Use CLI

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/README.md

Demonstrates how to use the Amazon Scraper via the command-line interface.

```bash
python -m amazon_scraper --url "https://www.amazon.com/s?k=electronics"
```

--------------------------------

### Creating a Product Instance

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/product-model.md

Demonstrates how to create instances of the Product model, both with and without a price.

```python
from amazon_scraper.models import Product

# Create a product with price
product = Product(
    title="Example Product",
    url="https://www.amazon.com/dp/B0BDJ279KF",
    asin_code="B0BDJ279KF",
    image_url="https://m.media-amazon.com/images/I/",
    price="29.99"
)

# Create a product without price (out of stock)
product_oos = Product(
    title="Out of Stock Product",
    url="https://www.amazon.com/dp/B0BDJ279KG",
    asin_code="B0BDJ279KG",
    image_url="https://m.media-amazon.com/images/I/",
    price=None
)
```

--------------------------------

### DriverGetProductsError Example

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/errors.md

Example of catching DriverGetProductsError when scraping product data fails.

```python
from amazon_scraper.scraper import AmazonScraper, DriverGetProductsError

scraper = AmazonScraper()

try:
    products = scraper.scrape_amazon_page("https://invalid-url.com")
except DriverGetProductsError as e:
    print("Failed to scrape Amazon page")
    print("Possible causes: invalid URL, page not loading, or Amazon structure changed")
```

--------------------------------

### Constructor

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/amazon-scraper.md

Initializes the scraper with an optional custom logger. Sets up HTTP headers that identify the scraper as a Chrome browser to avoid detection.

```python
def __init__(self, logger: logging.Logger | None = None) -> None:
    pass
```

--------------------------------

### Dependencies in pyproject.toml

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/configuration.md

List of project dependencies as defined in the pyproject.toml file.

```toml
[tool.poetry.dependencies]
python = "^3.11"
pydantic = "^2.7.4"
click = "^8.1.7"
pandas = "^2.2.2"
selenium = "^4.22.0"
webdriver-manager = "^4.0.1"
pydantic-settings = "^2.3.4"
selenium-wire = "^5.1.0"
blinker = "<1.8.0"
```

--------------------------------

### scrape_amazon CLI Implementation

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/configuration.md

The Python implementation of the scrape_amazon command-line interface using click.

```python
@click.command()
@click.option(
    "--url",
    help="The url of the page for which to return Amazon product data for.",
    required=True,
)
def scrape_amazon(url: str) -> None:
    collector = AmazonDataCollector()
    collector.collect_amazon_product_data(url)
```

--------------------------------

### Using AmazonDataCollector (No Exceptions Raised)

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/errors.md

Example demonstrating the use of AmazonDataCollector, which logs errors instead of raising exceptions.

```python
from amazon_scraper.collector import AmazonDataCollector
import logging

# Set up logging to see errors
logging.basicConfig(level=logging.INFO)

collector = AmazonDataCollector()

# This method catches all exceptions and logs them instead of raising
collector.collect_amazon_product_data(url)

# Check if file was created to see if scraping succeeded
import os
if os.path.exists("amazon_products.csv"):
    print("Scraping succeeded")
else:
    print("Scraping failed - check logs above")
```

--------------------------------

### Batch Processing with Pandas

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/product-model.md

Demonstrates converting a list of Product objects into a Pandas DataFrame and saving it to a CSV file.

```python
import pandas as pd
from amazon_scraper.models import Product

products = [
    Product(
        title="Product 1",
        url="https://www.amazon.com/dp/B001",
        asin_code="B001",
        image_url="https://m.media-amazon.com/images/I/1.jpg",
        price="19.99"
    ),
    Product(
        title="Product 2",
        url="https://www.amazon.com/dp/B002",
        asin_code="B002",
        image_url="https://m.media-amazon.com/images/I/2.jpg",
        price=None
    ),
]

# Convert list to pandas DataFrame
product_dicts = [p.model_dump() for p in products]
df = pd.DataFrame(product_dicts)

# Save to CSV
df.to_csv("products.csv", index=False)
```

--------------------------------

### Amazon Product Scraper Payload and Request

Source: https://github.com/oxylabs/amazon-scraper/blob/main/README.md

Example of structuring the payload and making a POST request to the Amazon Product Scraper API.

```python
import requests
from pprint import pprint


# Structure payload.
payload = {
    'source': 'amazon_product',
    'domain': 'nl',
    'query': 'B09RX4KS1G',
    'parse': True,
    'context': [
    {
      'key': 'autoselect_variant',
      'value': True
    }],
}


# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

# Print prettified response to stdout.
pprint(response.json())
```

--------------------------------

### Scrape and get Python objects

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/README.md

Scrapes Amazon product data from a given URL and returns it as Python objects.

```python
from amazon_scraper.scraper import AmazonScraper
scraper = AmazonScraper()
products = scraper.scrape_amazon_page("https://www.amazon.com/s?k=electronics")
```

--------------------------------

### Convert Products to Dictionary/JSON

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

Convert a Product object into a dictionary using `model_dump()` or a JSON string using `model_dump_json()`.

```python
from amazon_scraper.scraper import AmazonScraper

scraper = AmazonScraper()
products = scraper.scrape_amazon_page(url)

# Convert to dictionary
product_dict = products[0].model_dump()
# {
#   'title': '...',
#   'url': '...',
#   'asin_code': '...',
#   'image_url': '...',
#   'price': '...'
# }

# Convert to JSON
json_string = products[0].model_dump_json()
# '{"title":"...","url":"...","asin_code":"...","image_url":"...","price":"..."}'
```

--------------------------------

### Package Initialization Exports

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/MODULES.md

Imports from the amazon_scraper package, showing direct access to submodules.

```python
from amazon_scraper.collector import AmazonDataCollector
from amazon_scraper.scraper import AmazonScraper
from amazon_scraper.models import Product
from amazon_scraper.exception import BaseException
```

--------------------------------

### Command-Line Usage

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/INDEX.md

Scrapes Amazon product data using the command-line interface.

```bash
make scrape URL="https://www.amazon.com/s?k=electronics"
```

```bash
python -m amazon_scraper --url "https://www.amazon.com/s?k=electronics"
```

--------------------------------

### Scrape with Error Handling

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

Scrape Amazon product data using AmazonScraper and implement error handling for potential DriverInitializationError and DriverGetProductsError.

```python
from amazon_scraper.scraper import AmazonScraper, DriverInitializationError, DriverGetProductsError

scraper = AmazonScraper()

try:
    products = scraper.scrape_amazon_page(url)
except DriverInitializationError:
    print("Chrome WebDriver failed to initialize")
except DriverGetProductsError:
    print("Failed to scrape Amazon page")
```

--------------------------------

### Convert Product to Dictionary

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/product-model.md

Shows how to convert a Product instance into a Python dictionary using the `.model_dump()` method, useful for data export like CSV.

```python
from amazon_scraper.models import Product

product = Product(
    title="Product",
    url="https://www.amazon.com/dp/B0BDJ279KF",
    asin_code="B0BDJ279KF",
    image_url="https://m.media-amazon.com/images/I/",
    price="29.99"
)

# Convert to dictionary for CSV export
product_dict = product.model_dump()
# Output: {
#   'title': 'Product',
#   'url': 'https://www.amazon.com/dp/B0BDJ279KF',
#   'asin_code': 'B0BDJ279KF',
#   'image_url': 'https://m.media-amazon.com/images/I/...',
#   'price': '29.99'
# }
```

--------------------------------

### Access Product Data

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

After scraping products, access individual product attributes such as title, URL, ASIN code, price, and image URL.

```python
from amazon_scraper.scraper import AmazonScraper

scraper = AmazonScraper()
products = scraper.scrape_amazon_page(url)

for product in products:
    print(product.title)      # Product name
    print(product.url)        # Product page URL
    print(product.asin_code)  # Amazon product ID
    print(product.price)      # Price (or None)
    print(product.image_url)  # Image URL
```

--------------------------------

### Main Command Signature

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

The main command for scraping Amazon pages and exporting data to CSV.

```bash
python -m amazon_scraper --url <amazon_url>
```

--------------------------------

### Import Statements

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

Import necessary classes and exceptions from the amazon_scraper library.

```python
# Main collector (recommended)
from amazon_scraper.collector import AmazonDataCollector

# Direct scraper
from amazon_scraper.scraper import AmazonScraper

# Data model
from amazon_scraper.models import Product

# Error handling
from amazon_scraper.scraper import DriverInitializationError, DriverGetProductsError
from amazon_scraper.exception import BaseException
```

--------------------------------

### Usage in AmazonScraper

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/product-model.md

Shows how the `AmazonScraper.scrape_amazon_page()` method returns a list of `Product` instances.

```python
from amazon_scraper.scraper import AmazonScraper

scraper = AmazonScraper()
products: list[Product] = scraper.scrape_amazon_page(
    "https://www.amazon.com/s?k=electronics"
)

# Each product is a Product instance
for product in products:
    print(product.title)
    print(product.price)
```

--------------------------------

### Convert Product to JSON

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/product-model.md

Illustrates converting a Product instance to a JSON string using `.model_dump_json()`, with options for formatting.

```python
from amazon_scraper.models import Product

product = Product(
    title="Product",
    url="https://www.amazon.com/dp/B0BDJ279KF",
    asin_code="B0BDJ279KF",
    image_url="https://m.media-amazon.com/images/I/",
    price="29.99"
)

# Convert to JSON string
json_string = product.model_dump_json()
# Output: '{"title":"Product","url":"https://www.amazon.com/dp/B0BDJ279KF",...}'

# Convert with indentation for readability
json_formatted = product.model_dump_json(indent=2)
```

--------------------------------

### Direct Scraper Usage

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/INDEX.md

Demonstrates using the low-level AmazonScraper class to scrape product data and handle potential errors.

```python
from amazon_scraper.scraper import AmazonScraper, DriverInitializationError, DriverGetProductsError

scraper = AmazonScraper()

try:
    products = scraper.scrape_amazon_page(
        "https://www.amazon.com/s?k=electronics"
    )
    print(f"Found {len(products)} products")
    for product in products:
        print(f"- {product.title}: {product.price}")
except DriverInitializationError:
    print("Failed to initialize Chrome WebDriver")
except DriverGetProductsError:
    print("Failed to scrape products from the page")
```

--------------------------------

### Updating __init__.py for Top-Level Imports

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/MODULES.md

Shows the necessary modifications to `__init__.py` to enable top-level imports of key classes.

```python
from amazon_scraper.collector import AmazonDataCollector
from amazon_scraper.scraper import AmazonScraper
from amazon_scraper.models import Product

__all__ = ["AmazonDataCollector", "AmazonScraper", "Product"]
```

--------------------------------

### Amazon Scraper CLI Implementation

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/cli.md

The CLI is implemented using Click (Python CLI library). This snippet shows the basic structure of the scrape_amazon command.

```python
import click


@click.command()
@click.option(
    "--url",
    help="The url of the page for which to return Amazon product data for.",
    required=True,
)
def scrape_amazon(url: str) -> None:
    collector = AmazonDataCollector()
    collector.collect_amazon_product_data(url)
```

--------------------------------

### Process Multiple Pages

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

Scrape product data from multiple Amazon pages by iterating through a list of URLs and aggregating the results, with basic error handling for each page.

```python
from amazon_scraper.scraper import AmazonScraper

scraper = AmazonScraper()
all_products = []

urls = [
    "https://www.amazon.com/s?k=electronics&page=1",
    "https://www.amazon.com/s?k=electronics&page=2",
    "https://www.amazon.com/s?k=electronics&page=3",
]

for url in urls:
    try:
        products = scraper.scrape_amazon_page(url)
        all_products.extend(products)
    except Exception as e:
        print(f"Failed to scrape {url}: {e}")

print(f"Total products: {len(all_products)}")
```

--------------------------------

### Scrape and Save to CSV (Simplest)

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

Instantiate AmazonDataCollector and use it to scrape product data from a given URL, saving the results to a CSV file named 'amazon_products.csv'.

```python
from amazon_scraper.collector import AmazonDataCollector

collector = AmazonDataCollector()
collector.collect_amazon_product_data(
    "https://www.amazon.com/s?k=electronics"
)
# Creates: amazon_products.csv
```

--------------------------------

### Constructor

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/api-reference/amazon-data-collector.md

Initializes the data collector with an optional custom output file path and logger. The collector manages both scraping via AmazonScraper and CSV export of results.

```python
def __init__(
    self,
    output_file: str | None = None,
    logger: logging.Logger | None = None,
) -> None
```

--------------------------------

### AmazonScraper Constructor

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/configuration.md

The AmazonScraper class constructor accepts an optional logger instance.

```python
def __init__(self, logger: logging.Logger | None = None) -> None
```

--------------------------------

### Public API Import Patterns

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/MODULES.md

Recommended import patterns for using the Amazon Scraper library.

```python
# Recommended for most use cases
from amazon_scraper.collector import AmazonDataCollector

# Direct scraper access
from amazon_scraper.scraper import AmazonScraper

# Data model
from amazon_scraper.models import Product

# Error handling
from amazon_scraper.scraper import (
    DriverInitializationError,
    DriverGetProductsError,
)

# Base exception (rarely needed)
from amazon_scraper.exception import BaseException
```

--------------------------------

### Scrape with Custom Output File

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/QUICK-REFERENCE.md

Scrape product data from a URL and specify a custom output file name for the CSV results.

```python
from amazon_scraper.collector import AmazonDataCollector

collector = AmazonDataCollector(output_file="products.csv")
collector.collect_amazon_product_data(
    "https://www.amazon.com/s?k=electronics"
)
```

--------------------------------

### ProductXPath Enum Definition

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/MODULES.md

XPath selectors for product page elements used internally by the scraper.

```python
class ProductXPath(str, Enum):
    PRODUCTS = "//div[@data-component-type='s-search-result']"
    TITLE = ".//h2/a/span"
    URL = ".//h2/a"
    PRICE_WHOLE = ".//span[@class='a-price']//span[@class='a-price-whole']"
    PRICE_FRACTIONAL = ".//span[@class='a-price']//span[@class='a-price-fraction']"
    IMAGE_URL = ".//img"
```

--------------------------------

### AmazonScraper Class Signature

Source: https://github.com/oxylabs/amazon-scraper/blob/main/_autodocs/MODULES.md

Main scraper class using Selenium with Chrome, showing constructor and main scraping method.

```python
class AmazonScraper:
    def __init__(self, logger: logging.Logger | None = None) -> None
    def scrape_amazon_page(self, url: str) -> List[Product]
```