### Parfive Command-Line Interface Examples

Source: https://context7.com/cadair/parfive/llms.txt

This section provides examples of using the parfive command-line tool for downloading files. It covers basic usage, specifying download directories, configuring connection limits, overwriting files, disabling progress bars, printing filenames, enabling verbose logging, and checking the version.

```bash
# Basic download
parfive 'https://example.com/file1.zip' 'https://example.com/file2.zip'

# Download to specific directory
parfive 'https://example.com/data.tar.gz' --directory ./downloads

# Configure parallel connections
parfive 'https://example.com/large_file.iso' --max-conn 10 --max-splits 8

# Overwrite existing files
parfive 'https://example.com/file.zip' --overwrite --directory ./data

# Disable progress bars
parfive 'https://example.com/file.dat' --no-progress --no-file-progress

# Print downloaded filenames to stdout
parfive 'https://example.com/file.zip' --print-filenames

# Enable debug logging
parfive 'https://example.com/file.tar.gz' --verbose

# Check version
parfive --version
```

--------------------------------

### Installation of Parfive (Shell)

Source: https://github.com/cadair/parfive/blob/main/docs/index.rst

Provides instructions for installing the Parfive library using pip. It shows the basic installation command, how to install with optional FTP support, and how to install using conda from the conda-forge channel.

```shell
pip install parfive

# With FTP support:
parfive[ftp]

# With conda:
conda install -c conda-forge parfive
```

--------------------------------

### Asynchronous File Downloads using Parfive API

Source: https://context7.com/cadair/parfive/llms.txt

This example demonstrates using the asynchronous API of parfive within an existing asyncio event loop. It shows how to create a Downloader instance, enqueue files, and then initiate the download process using `run_download`. Progress reporting can be enabled.

```python
import asyncio
from parfive import Downloader

async def download_files():
    """Async download function"""
    dl = Downloader(max_conn=8, progress=True)

    # Enqueue files
    urls = [
        "https://example.com/dataset1.csv",
        "https://example.com/dataset2.csv",
        "https://example.com/dataset3.csv"
    ]

    for url in urls:
        dl.enqueue_file(url, path="./datasets")

    # Use async download method
    results = await dl.run_download()

    return results

# Run in asyncio context
results = asyncio.run(download_files())
print(f"Completed {len(results)} downloads asynchronously")
```

--------------------------------

### Parfive CLI for Concurrent Downloads (Shell)

Source: https://github.com/cadair/parfive/blob/main/docs/index.rst

Illustrates how to use the Parfive command-line interface (CLI) to download multiple files concurrently. The example shows passing URLs as arguments to the `parfive` command. Basic help information for the CLI is also shown.

```shell
$ parfive 'http://212.183.159.230/5MB.zip' 'http://212.183.159.230/10MB.zip'
$ parfive --help
usage: parfive [-h] [--max-conn MAX_CONN] [--overwrite] [--no-file-progress] [--directory DIRECTORY] [--print-filenames] URLS [URLS ...]

Parfive, the python asyncio based downloader

positional arguments:
  URLS                  URLs of files to be downloaded.

optional arguments:
  -h, --help            show this help message and exit
  --max-conn MAX_CONN   Number of maximum connections.
  --overwrite           Overwrite if the file exists.
  --no-file-progress    Show progress bar for each file.
  --directory DIRECTORY
                        Directory to which downloaded files are saved.
  --print-filenames     Print successfully downloaded files's names to stdout.
```

--------------------------------

### CLI: Download Files Concurrently with Parfive

Source: https://github.com/cadair/parfive/blob/main/README.rst

This command-line example shows how to use the parfive CLI to download multiple files simultaneously. It takes a list of URLs as arguments and downloads them concurrently. The `--help` option displays all available command-line arguments for customizing the download behavior.

```bash
$ parfive 'http://212.183.159.230/5MB.zip' 'http://212.183.159.230/10MB.zip'
$ parfive --help
```

--------------------------------

### FTP File Downloads with Parfive

Source: https://context7.com/cadair/parfive/llms.txt

This snippet illustrates how to download files from FTP servers using parfive. It highlights that the interface is the same as HTTP downloads and shows examples of both anonymous and authenticated FTP connections. Note: requires the `aioftp` package.

```python
from parfive import Downloader

# Note: Requires aioftp package installed
# pip install parfive[ftp]

dl = Downloader(max_conn=3)

# FTP downloads work identically to HTTP
dl.enqueue_file(
    "ftp://ftp.example.com/pub/data/file.tar.gz",
    path="./ftp_downloads"
)

# FTP with authentication
dl.enqueue_file(
    "ftp://username:password@ftp.example.com/private/data.zip",
    path="./ftp_downloads"
)

results = dl.download()
print(f"Downloaded {len(results)} files via FTP")
```

--------------------------------

### Retry Failed Downloads with Parfive

Source: https://context7.com/cadair/parfive/llms.txt

Shows how to configure Parfive to automatically retry downloads that fail due to network issues or server errors. This example queues multiple downloads, including one from an unreliable server.

```python
from parfive import Downloader

dl = Downloader(max_conn=5)

# Queue multiple downloads
urls = [
    "https://example.com/file1.zip",
    "https://example.com/file2.zip",
    "https://unreliable-server.com/file3.zip"  # May fail
]

for url in urls:
    dl.enqueue_file(url, path="./downloads")

# First attempt
results = dl.download()

```

--------------------------------

### Basic File Download with Parfive

Source: https://context7.com/cadair/parfive/llms.txt

Demonstrates how to create a Downloader instance, add files to the download queue, execute the download, and access the results, including checking for errors.

```python
from parfive import Downloader

# Create downloader with default settings
dl = Downloader()

# Add file to download queue
dl.enqueue_file(
    "https://example.com/data/large_file.zip",
    path="./downloads"
)

# Execute download and get results
results = dl.download()

# Access downloaded file paths
for filepath in results:
    print(f"Downloaded: {filepath}")

# Check for errors
if results.errors:
    for error in results.errors:
        print(f"Failed: {error.url}")
        print(f"Reason: {error.exception}")
```

--------------------------------

### Python: Basic File Download with Parfive

Source: https://github.com/cadair/parfive/blob/main/README.rst

This snippet demonstrates the basic usage of the parfive library to download a file. It initializes a Downloader object, enqueues a file for download, and then initiates the download process. The `download()` method returns a `Results` object containing information about the downloaded files.

```python
from parfive import Downloader

dl = Downloader()
dl.enqueue_file("http://data.sunpy.org/sample-data/predicted-sunspot-radio-flux.txt", path="./")
files = dl.download()
```

--------------------------------

### Basic File Download

Source: https://context7.com/cadair/parfive/llms.txt

This section demonstrates how to create a Downloader instance, add files to the download queue, execute the download, and access the results, including error handling.

```APIDOC
## Basic File Download

### Description
Create a downloader instance, enqueue files, and execute the download operation.

### Method
POST (implicitly via Downloader.download() method)

### Endpoint
/

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
from parfive import Downloader

# Create downloader with default settings
dl = Downloader()

# Add file to download queue
dl.enqueue_file(
    "https://example.com/data/large_file.zip",
    path="./downloads"
)

# Execute download and get results
results = dl.download()

# Access downloaded file paths
for filepath in results:
    print(f"Downloaded: {filepath}")

# Check for errors
if results.errors:
    for error in results.errors:
        print(f"Failed: {error.url}")
        print(f"Reason: {error.exception}")
```

### Response
#### Success Response (200)
- **results** (list) - A list of successfully downloaded file paths.
- **results.errors** (list) - A list of errors encountered during download, each containing 'url' and 'exception'.

#### Response Example
```json
{
  "results": ["./downloads/large_file.zip"],
  "errors": []
}
```
```

--------------------------------

### Simple File Downloads using Parfive

Source: https://context7.com/cadair/parfive/llms.txt

This code shows a simplified way to download multiple files using parfive's `simple_download` class method. It takes a list of URLs and optional download path and overwrite settings. It returns a list of downloaded file paths.

```python
from parfive import Downloader

# Download multiple files in one call
urls = [
    "https://example.com/report1.pdf",
    "https://example.com/report2.pdf",
    "https://example.com/report3.pdf"
]

results = Downloader.simple_download(
    urls,
    path="./reports",
    overwrite=False
)

# Results work the same way
print(f"Downloaded {len(results)} files to ./reports/")
for filepath in results:
    print(f"  - {filepath}")
```

--------------------------------

### Simple Download of Multiple URLs (Python)

Source: https://github.com/cadair/parfive/blob/main/docs/index.rst

Shows how to use the `Downloader.simple_download` method to download a list of URLs to a single specified directory. This is a convenient way to download multiple files without explicitly creating a Downloader instance and enqueuing each file individually.

```python
from parfive import Downloader

files = Downloader.simple_download(['http://212.183.159.230/5MB.zip' 'http://212.183.159.230/10MB.zip'], path="./")
```

--------------------------------

### Download with Environment Variables

Source: https://context7.com/cadair/parfive/llms.txt

Demonstrates how parfive automatically applies settings from environment variables to the Downloader. This method is useful for configuring downloads without explicit code changes, relying on system-wide or application-specific environment settings.

```python
from parfive import Downloader

dl = Downloader()
dl.enqueue_file("https://example.com/file.zip", path="./downloads")
results = dl.download()  # Runs with environment settings applied
```

--------------------------------

### Advanced Downloader Configuration in Parfive

Source: https://context7.com/cadair/parfive/llms.txt

Shows how to configure the Downloader with advanced settings like connection limits, file splitting, progress display, overwrite behavior, custom session configurations (headers, proxies), and logging.

```python
from parfive import Downloader, SessionConfig

# Create custom session configuration
config = SessionConfig(
    headers={"Authorization": "Bearer token123"},
    file_progress=True,
    log_level="DEBUG",
    use_aiofiles=True,
    http_proxy="http://proxy.example.com:8080",
    https_proxy="http://proxy.example.com:8080"
)

# Initialize downloader with advanced settings
dl = Downloader(
    max_conn=10,           # Maximum 10 parallel downloads
    max_splits=8,          # Split each file into 8 chunks
    progress=True,         # Show overall progress bar
    overwrite="unique",    # Create unique filenames for conflicts
    config=config
)

# Enqueue multiple files
urls = [
    "https://example.com/file1.dat",
    "https://example.com/file2.dat",
    "https://example.com/file3.dat"
]

for url in urls:
    dl.enqueue_file(url, path="./data")

# Download all files
results = dl.download()
print(f"Downloaded {len(results)} files successfully")
```

--------------------------------

### Advanced Downloader Configuration

Source: https://context7.com/cadair/parfive/llms.txt

This section explains how to configure the Downloader with advanced settings such as connection limits, file splitting, progress display, overwrite behavior, and custom session configurations including proxy and headers.

```APIDOC
## Advanced Downloader Configuration

### Description
Configure connection limits, file splitting, progress display, and overwrite behavior.

### Method
POST (implicitly via Downloader.download() method)

### Endpoint
/

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
from parfive import Downloader, SessionConfig

# Create custom session configuration
config = SessionConfig(
    headers={"Authorization": "Bearer token123"},
    file_progress=True,
    log_level="DEBUG",
    use_aiofiles=True,
    http_proxy="http://proxy.example.com:8080",
    https_proxy="http://proxy.example.com:8080"
)

# Initialize downloader with advanced settings
dl = Downloader(
    max_conn=10,           # Maximum 10 parallel downloads
    max_splits=8,          # Split each file into 8 chunks
    progress=True,         # Show overall progress bar
    overwrite="unique",    # Create unique filenames for conflicts
    config=config
)

# Enqueue multiple files
urls = [
    "https://example.com/file1.dat",
    "https://example.com/file2.dat",
    "https://example.com/file3.dat"
]

for url in urls:
    dl.enqueue_file(url, path="./data")

# Download all files
results = dl.download()
print(f"Downloaded {len(results)} files successfully")
```

### Response
#### Success Response (200)
- **results** (list) - A list of successfully downloaded file paths.

#### Response Example
```json
{
  "results": ["./data/file1.dat", "./data/file2.dat", "./data/file3.dat"],
  "errors": []
}
```
```

--------------------------------

### Parfive Configuration via Environment Variables

Source: https://context7.com/cadair/parfive/llms.txt

This Python snippet shows how to control parfive's behavior by setting environment variables before initializing the Downloader. It covers options for forcing serial downloads, disabling range requests, hiding progress bars, setting timeouts, and forcing `aiofiles` usage.

```python
import os
from parfive import Downloader

# Set environment variables before creating downloader
os.environ["PARFIVE_SINGLE_DOWNLOAD"] = "1"        # Force serial downloads
os.environ["PARFIVE_DISABLE_RANGE"] = "1"          # Disable range requests
os.environ["PARFIVE_HIDE_PROGRESS"] = "1"          # Hide all progress bars
os.environ["PARFIVE_TOTAL_TIMEOUT"] = "300"        # Total timeout in seconds
os.environ["PARFIVE_SOCK_READ_TIMEOUT"] = "120"    # Socket read timeout
os.environ["PARFIVE_OVERWRITE_ENABLE_AIOFILES"] = "1"  # Force aiofiles usage

# downloader = Downloader()
# ... rest of your download logic
```

--------------------------------

### Custom aiohttp Session Configuration

Source: https://context7.com/cadair/parfive/llms.txt

Shows how to provide a custom aiohttp ClientSession to parfive for advanced HTTP client configuration. This allows for detailed control over SSL contexts, connection limits, headers, and timeouts, enabling tailored download behavior for specific server requirements or security policies.

```python
import aiohttp
import ssl
from parfive import Downloader, SessionConfig

def create_custom_session(config):
    """Create aiohttp session with custom SSL context"""
    ssl_context = ssl.create_default_context()
    ssl_context.check_hostname = False
    ssl_context.verify_mode = ssl.CERT_NONE

    connector = aiohttp.TCPConnector(
        ssl=ssl_context,
        limit=20,
        limit_per_host=10
    )

    return aiohttp.ClientSession(
        connector=connector,
        headers=config.headers,
        requote_redirect_url=False,
        timeout=aiohttp.ClientTimeout(total=600, sock_read=90)
    )

# Configure with custom session generator
config = SessionConfig(
    aiohttp_session_generator=create_custom_session,
    headers={"User-Agent": "CustomBot/1.0"}
)

dl = Downloader(config=config)
dl.enqueue_file("https://example.com/file.zip", path="./downloads")
results = dl.download()
```

--------------------------------

### Download Callbacks in Parfive

Source: https://context7.com/cadair/parfive/llms.txt

This code illustrates how to register custom callback functions in parfive to execute logic upon download completion or failure. The `download_callback` function logs download status (success/failure, URL, filepath, error) to a JSON file and prints a message to the console.

```python
from parfive import Downloader, SessionConfig
import json
from pathlib import Path

def download_callback(filepath, url, error):
    """Called when each download completes"""
    log_entry = {
        "url": url,
        "filepath": str(filepath) if filepath else None,
        "success": error is None,
        "error": str(error) if error else None
    }

    # Append to log file
    with open("download_log.json", "a") as f:
        json.dump(log_entry, f)
        f.write("\n")

    if error is None:
        print(f"✓ Successfully downloaded {Path(filepath).name}")
    else:
        print(f"✗ Failed to download {url}: {error}")

# Configure with callback
config = SessionConfig(done_callbacks=[download_callback])
dl = Downloader(config=config)

dl.enqueue_file("https://example.com/file1.zip", path="./downloads")
dl.enqueue_file("https://example.com/file2.zip", path="./downloads")

results = dl.download()
# Callbacks are invoked for each file
```

--------------------------------

### Checksum Validation for File Integrity with Parfive

Source: https://context7.com/cadair/parfive/llms.txt

Illustrates how to use Parfive for checksum validation, both by relying on server-provided checksums and by providing an explicit checksum string. Downloads fail if checksums do not match.

```python
from parfive import Downloader

dl = Downloader()

# Request server-provided checksum validation
dl.enqueue_file(
    "https://example.com/important_data.tar.gz",
    path="./secure",
    checksum=True  # Use server's checksum if available
)

# Or provide explicit checksum
dl.enqueue_file(
    "https://example.com/verified_file.bin",
    path="./secure",
    checksum="sha-256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
)

# If checksum doesn't match, download fails
results = dl.download()

if results.errors:
    for error in results.errors:
        if "ChecksumMismatch" in str(error.exception):
            print(f"Checksum failed for {error.url}")
```

--------------------------------

### Retry Failed Downloads with Parfive

Source: https://context7.com/cadair/parfive/llms.txt

This snippet demonstrates how to check for download errors using parfive, retry failed downloads, and then check for errors again. It provides feedback on the success or failure of individual files after retries.

```python
if results.errors:
    print(f"{len(results.errors)} downloads failed, retrying...")

    # Retry failed downloads
    results = dl.retry(results)

    # Check again
    if results.errors:
        print("Still failed after retry:")
        for error in results.errors:
            print(f"  {error.url}: {error.exception}")
    else:
        print("All downloads succeeded after retry!")

print(f"Total successful downloads: {len(results)}")
```

--------------------------------

### Custom Filename Generation with Parfive

Source: https://context7.com/cadair/parfive/llms.txt

Demonstrates how to define and use a custom callable function to generate filenames dynamically based on the URL and response headers during the download process.

```python
from parfive import Downloader
import hashlib
from datetime import datetime

def custom_filename(url, response):
    """Generate filename based on URL hash and timestamp"""
    url_hash = hashlib.md5(url.encode()).hexdigest()[:8]
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")

    # Extract extension from URL
    ext = url.split('.')[-1] if '.' in url else 'dat'

    return f"download_{timestamp}_{url_hash}.{ext}"

dl = Downloader()

# Use custom filename function
dl.enqueue_file(
    "https://example.com/dynamic/resource",
    path="./custom_names",
    filename=custom_filename
)

results = dl.download()
print(f"Saved as: {results[0]}")
```

--------------------------------

### Python: Handling Download Errors with Parfive Results

Source: https://github.com/cadair/parfive/blob/main/README.rst

When files fail to download using parfive, the `Results` object returned by `dl.download()` stores error information. The `.errors` attribute provides a list of named tuples, each containing the failed `.url` and the corresponding server `.response` (either a `aiohttp.ClientResponse` or `aiohttp.ClientError`). This allows for robust error handling and user feedback.

```python
# Assuming 'files' is the Results object returned by dl.download()
if files.errors:
    for error in files.errors:
        print(f"Failed to download {error.url}: {error.response}")
```

--------------------------------

### Retry Failed Downloads

Source: https://context7.com/cadair/parfive/llms.txt

This section covers how to configure automatic retries for downloads that fail due to network issues or temporary server errors.

```APIDOC
## Retry Failed Downloads

### Description
Automatically retry downloads that failed due to network issues or server errors.

### Method
POST (implicitly via Downloader.download() method)

### Endpoint
/

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
from parfive import Downloader

dl = Downloader(max_conn=5)

# Queue multiple downloads
urls = [
    "https://example.com/file1.zip",
    "https://example.com/file2.zip",
    "https://unreliable-server.com/file3.zip"  # May fail
]

for url in urls:
    dl.enqueue_file(url, path="./downloads")

# First attempt
results = dl.download()

# The library automatically retries failed downloads based on internal retry logic.
# The 'results' object will contain the final status of all downloads after retries.

print(f"Successfully downloaded: {[r for r in results]}")
if results.errors:
    print("Failed downloads:")
    for error in results.errors:
        print(f"- {error.url}: {error.exception}")
```

### Response
#### Success Response (200)
- **results** (list) - A list of successfully downloaded file paths after potential retries.
- **results.errors** (list) - A list of errors for downloads that ultimately failed.

#### Response Example
```json
{
  "results": ["./downloads/file1.zip", "./downloads/file2.zip"],
  "errors": [
    {
      "url": "https://unreliable-server.com/file3.zip",
      "exception": "Connection error during download"
    }
  ]
}
```
```

--------------------------------

### Checksum Validation

Source: https://context7.com/cadair/parfive/llms.txt

This section details how to perform checksum validation during file downloads, either by using server-provided checksums or by specifying an explicit checksum value to ensure data integrity.

```APIDOC
## Checksum Validation

### Description
Download files with automatic checksum verification to ensure data integrity.

### Method
POST (implicitly via Downloader.download() method)

### Endpoint
/

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
from parfive import Downloader

dl = Downloader()

# Request server-provided checksum validation
dl.enqueue_file(
    "https://example.com/important_data.tar.gz",
    path="./secure",
    checksum=True  # Use server's checksum if available
)

# Or provide explicit checksum
dl.enqueue_file(
    "https://example.com/verified_file.bin",
    path="./secure",
    checksum="sha-256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
)

# If checksum doesn't match, download fails
results = dl.download()

if results.errors:
    for error in results.errors:
        if "ChecksumMismatch" in str(error.exception):
            print(f"Checksum failed for {error.url}")
```

### Response
#### Success Response (200)
- **results** (list) - A list of successfully downloaded file paths.

#### Response Example
```json
{
  "results": ["./secure/important_data.tar.gz", "./secure/verified_file.bin"],
  "errors": []
}
```
```

--------------------------------

### Custom Filename Generation

Source: https://context7.com/cadair/parfive/llms.txt

Demonstrates how to use custom callable filename generators to dynamically determine filenames based on response headers or other criteria.

```APIDOC
## Custom Filename Generation

### Description
Use callable filename generators for dynamic file naming based on response headers.

### Method
POST (implicitly via Downloader.download() method)

### Endpoint
/

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
from parfive import Downloader
import hashlib
from datetime import datetime

def custom_filename(url, response):
    """Generate filename based on URL hash and timestamp"""
    url_hash = hashlib.md5(url.encode()).hexdigest()[:8]
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")

    # Extract extension from URL
    ext = url.split('.')[-1] if '.' in url else 'dat'

    return f"download_{timestamp}_{url_hash}.{ext}"

dl = Downloader()

# Use custom filename function
dl.enqueue_file(
    "https://example.com/dynamic/resource",
    path="./custom_names",
    filename=custom_filename
)

results = dl.download()
print(f"Saved as: {results[0]}")
```

### Response
#### Success Response (200)
- **results** (list) - A list of successfully downloaded file paths with custom names.

#### Response Example
```json
{
  "results": ["./custom_names/download_20231027_103000_abcdef12.resource"],
  "errors": []
}
```
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.