### Build and Install from Source

Source: https://github.com/ivan-sincek/scrapy-scraper/blob/main/README.md

Clones the repository, builds the package using setuptools, and then installs the built wheel file.

```bash
git clone https://github.com/ivan-sincek/scrapy-scraper && cd scrapy-scraper

python3 -m pip install --upgrade build

python3 -m build

python3 -m pip install dist/scrapy-scraper-3.6-py3-none-any.whl
```

--------------------------------

### Standard Installation

Source: https://github.com/ivan-sincek/scrapy-scraper/blob/main/README.md

Installs the scrapy-scraper package using pip.

```bash
pip3 install --upgrade scrapy-scraper
```

--------------------------------

### Scrapy Scraper Command-Line Usage

Source: https://github.com/ivan-sincek/scrapy-scraper/blob/main/README.md

Provides the basic syntax and an example for running the Scrapy Scraper tool. It details the required URL and output file arguments, along with optional directory specification.

```fundamental
Scrapy Scraper v3.6 ( github.com/ivan-sincek/scrapy-scraper )

Usage:   scrapy-scraper -u urls                     -o out         [-dir directory]
Example: scrapy-scraper -u https://example.com/home -o results.txt [-dir downloads]
```

--------------------------------

### Install Playwright and Chromium

Source: https://github.com/ivan-sincek/scrapy-scraper/blob/main/README.md

Installs the Playwright library and the Chromium browser, which is necessary for headless browser functionality. It's recommended to re-install Chromium after upgrading Playwright.

```bash
pip3 install --upgrade playwright

playwright install chromium
```

--------------------------------

### Run Scraper (Unrestricted, All Links)

Source: https://github.com/ivan-sincek/scrapy-scraper/blob/main/README.md

Runs the scraper targeting 'example.com/home', saving results to 'results.txt'. It uses random user agents, a depth of 2, restarts on failure, and downloads JavaScript files to a 'js' directory. All links are followed, and JavaScript rendering is turned off.

```fundamental
scrapy-scraper -u https://example.com/home -o results.txt -a random -s 2 -rs -dir js -w off
```

--------------------------------

### Run Scraper (Restricted, Internal Links Only)

Source: https://github.com/ivan-sincek/scrapy-scraper/blob/main/README.md

Runs the scraper targeting 'example.com/home', saving results to 'results.txt'. It uses random user agents, a depth of 2, restarts on failure, and downloads JavaScript files to a 'js' directory. Only links within 'example.com' are followed.

```fundamental
scrapy-scraper -u https://example.com/home -o results.txt -a random -s 2 -rs -dir js
```

--------------------------------

### Run Scraper (Restricted, Internal and External Links)

Source: https://github.com/ivan-sincek/scrapy-scraper/blob/main/README.md

Runs the scraper targeting 'example.com/home', saving results to 'results.txt'. It uses random user agents, a depth of 2, restarts on failure, and downloads JavaScript files to a 'js' directory. Both internal and external links are followed.

```fundamental
scrapy-scraper -u https://example.com/home -o results.txt -a random -s 2 -rs -dir js -l
```

--------------------------------

### Scrapy Scraper Configuration Options

Source: https://github.com/ivan-sincek/scrapy-scraper/blob/main/README.md

Details all available command-line options for configuring the Scrapy Scraper, including URL sources, whitelisting, concurrency, delays, retries, timeouts, headers, cookies, user agents, proxies, and output directories.

```fundamental
DESCRIPTION
    Crawl and scrape websites
URLS
    File containing URLs or a single URL to start crawling and scraping from
    -u, --urls = urls.txt | https://example.com/home | etc.
WHITELIST
    File containing whitelisted domain names to limit the scope
    Specify 'off' to disable domain whitelisting
    Default: limit the scope to domain names extracted from the URLs
    -w, --whitelist = whitelist.txt | off | etc.
LINKS
    Include all 3rd party links and sources in the output file
    -l, --links
PLAYWRIGHT
    Use Playwright's headless browser
    -p, --playwright
PLAYWRIGHT WAIT
    Wait time in seconds before fetching the page content
    -pw, --playwright-wait = 0.5 | 2 | 4 | etc.
CONCURRENT REQUESTS
    Number of concurrent requests
    Default: 30
    -cr, --concurrent-requests = 30 | 45 | etc.
CONCURRENT REQUESTS PER DOMAIN
    Number of concurrent requests per domain
    Default: 10
    -crd, --concurrent-requests-domain = 10 | 15 | etc.
SLEEP
    Sleep time in seconds between two consecutive requests to the same domain
    -s, --sleep = 1.5 | 3 | etc.
RANDOM SLEEP
    Randomize the sleep time between requests to vary between '0.5 * sleep' and '1.5 * sleep'
    -rs, --random-sleep
AUTO THROTTLE
    Auto throttle concurrent requests based on the load and latency
    Sleep time is still respected
    -at, --auto-throttle = 0.5 | 10 | 15 | 45 | etc.
RETRIES
    Number of retries per URL
    Default: 2
    -rt, --retries = 0 | 4 | etc.
RECURSION
    Recursion depth limit
    Specify '0' for no limit
    Default: 1
    -r, --recursion = 0 | 2 | etc.
REQUEST TIMEOUT
    Request timeout in seconds
    Default: 60
    -t, --request-timeout = 30 | 90 | etc.
HEADER
    Specify any number of extra HTTP request headers
    -H, --header = "Authorization: Bearer ey..." | etc.
COOKIE
    Specify any number of extra HTTP cookies
    -b, --cookie = PHPSESSIONID=3301 | etc.
USER AGENT
    User agent to use
    Default: Scrapy Scraper/3.6
    -a, --user-agent = random[-all] | curl/3.30.1 | etc.
PROXY
    Web proxy to use
    -x, --proxy = http://127.0.0.1:8080 | etc.
DIRECTORY
    Output directory
    All extracted JavaScript files will be saved in this directory
    -dir, --directory = downloads | etc.
OUT
    Output file
    -o, --out = results.txt | etc.
DEBUG
    Enable debug output
    -dbg, --debug
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.