Manga Scraper

https://github.com/yofagh/mangascraper
Admin
A command-line tool for downloading, managing, and converting manga and manhua from various websites...
Tokens:4,638
Snippets:43
Trust Score:6.7
License:MIT
Update:8 months ago
Show doc for...
Tokens:
Raw
### Get MangaScraper Modules (Python)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Dynamically loads website scraper modules using the `get_modules` function from `utils.modules_contributor`. It allows fetching single, multiple, or all available modules based on domain names.

```python
from utils.modules_contributer import get_modules

# Get a single module by domain
mangapark = get_modules("mangapark.to")
print(f"Module type: {mangapark.type}")  # Output: Manga

# Get multiple modules
modules = get_modules(["mangapark.to", "nhentai.net", "hentaifox.com"])
for module in modules:
    print(f"{module.domain}: {module.type}")

# Get all available modules
all_modules = get_modules()
print(f"Total modules: {len(all_modules)}")
```

--------------------------------

### View CLI Commands and Options (Bash)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Displays all available commands and their options for the Manga/Manhua Scraper CLI. This is the primary command to understand the tool's capabilities and syntax.

```bash
# View all available commands and options
python cli.py -h

# Available tasks:
# - manga: Download manga series
# - doujin: Download doujin by code
# - merge: Merge downloaded images vertically
# - c2pdf: Convert images to PDF
# - search: Search across modules
# - db: Crawl website database
# - check: Verify module functionality
# - sauce: Find image source
```

--------------------------------

### Download Single Doujin (Bash)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Downloads a single doujin by its code from a specified source website. The tool automatically retrieves the title and organizes the downloaded images. Supports auto-merging and PDF conversion.

```bash
# Download doujin by code
python cli.py doujin -code 000000 -s hentaifox.com

# Alternative using -single flag
python cli.py doujin -single 12345 -s nhentai.net

# Download with auto-merge and PDF conversion
python cli.py doujin -code 54321 -s imhentai.xxx -m -p
```

--------------------------------

### Batch Download Doujins from File (Bash)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Batch downloads multiple doujins from various sources using a JSON file. The JSON file is organized by domain, with each domain listing an array of doujin codes to download.

```bash
# Download all doujins defined in the JSON file
python cli.py doujin -file doujins.json
```

--------------------------------

### Doujin Batch Download Configuration (JSON)

Source: https://context7.com/yofagh/mangascraper/llms.txt

A JSON file used for batch downloading doujins. It's structured by website domain, with each domain containing a list of doujin codes to be downloaded from that source.

```json
{
    "nyahentai.red": [
        999999,
        999998,
        999997
    ],
    "hentaifox.com": [
        888888,
        888887
    ],
    "nhentai.net": [
        777777
    ]
}
```

--------------------------------

### Batch Download Manga from File (Bash)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Downloads multiple manga series concurrently using a JSON configuration file. The file tracks download progress and updates automatically.

```bash
# Download all manga defined in the JSON file
python cli.py manga -file mangas.json
```

--------------------------------

### Convert All Subfolders to Combined PDF

Source: https://context7.com/yofagh/mangascraper/llms.txt

Recursively converts all subfolders within a given directory into a single, combined PDF file. This is ideal for consolidating entire series or arcs.

```bash
python cli.py c2pdf -bulkone "One Piece"
```

--------------------------------

### Convert Folder to PDF with Custom Name

Source: https://context7.com/yofagh/mangascraper/llms.txt

Converts all images within a specified folder into a single PDF file with a custom name. This is useful for organizing downloaded chapters.

```bash
python cli.py c2pdf -folder "One Piece/Chapter 001" -n "OnePiece_Ch001.pdf"
```

--------------------------------

### PDF Converter (Bash)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Converts downloaded manga or doujin images into PDF format. This command is most effective when used on pre-merged images for a seamless reading experience.

```bash
# Convert all chapters to individual PDFs
python cli.py c2pdf -bulk "One Piece"
```

--------------------------------

### Override Scraper Settings (Bash)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Demonstrates how to override default scraper settings using command-line flags when executing the scraper. This allows for flexible control over specific download and processing options for individual runs.

```bash
# Override settings via CLI
python cli.py manga -single one-piece -s mangapark.to \
    -t 0.5 \
    -m \
    -p \
    -fit

python cli.py search -n "naruto" \
    -page-limit 10 \
    -absolute
```

--------------------------------

### Check Website Module Functionality

Source: https://context7.com/yofagh/mangascraper/llms.txt

Validates the correct functioning of specified website modules. It tests critical features such as chapter retrieval, image fetching, download capability, search, and database functions.

```bash
# Check a specific module
python cli.py check -s manhuascan.us
```

```bash
# Check multiple modules
python cli.py check -s mangapark.to nhentai.net hentaifox.com
```

```bash
# Check all available modules
python cli.py check
```

--------------------------------

### Search Manga/Doujin Across Modules

Source: https://context7.com/yofagh/mangascraper/llms.txt

Searches for manga or doujin titles across specified or all supported modules. Results are saved to a JSON file and summarized in the terminal. Supports keyword matching, custom page limits, absolute title matching, and request delay adjustment.

```bash
# Search in a specific module
python cli.py search -s manhuascan.us -n "secret"
```

```bash
# Search across multiple modules
python cli.py search -s mangapark.to manga68.com truemanga.com -n "one piece"
```

```bash
# Search all available modules
python cli.py search -n "solo leveling"
```

```bash
# Advanced search with custom page limit and absolute title matching
python cli.py search -s manhuascan.us -n "secret class" -page-limit 5 -absolute
```

```bash
# Adjust request delay between search pages
python cli.py search -n "attack on titan" -t 1
```

--------------------------------

### Manga Batch Download Configuration (JSON)

Source: https://context7.com/yofagh/mangascraper/llms.txt

A JSON file used for batch downloading manga. Each entry specifies the manga title, whether to include it, the source domain, the URL identifier, and optionally the last downloaded chapter and a list of chapters.

```json
{
    "Attack on Titan": {
        "include": true,
        "domain": "mangapark.to",
        "url": "11643-attack-on-titan",
        "last_downloaded_chapter": null,
        "chapters": []
    },
    "Secret Class": {
        "include": true,
        "domain": "manhuascan.us",
        "url": "secret-class",
        "last_downloaded_chapter": {
            "url": "chapter-100",
            "name": "Chapter 100"
        },
        "chapters": []
    },
    "One Piece": {
        "include": false,
        "domain": "mangapark.to",
        "url": "77478-en-one-piece",
        "last_downloaded_chapter": null,
        "chapters": []
    }
}
```

--------------------------------

### Download Single Manga (Bash)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Downloads a specific manga series from a given source. Supports filtering by chapter number, range, or specific chapters. Options include auto-merging images and converting to PDF.

```bash
# Download all chapters of a manga
python cli.py manga -single 11643-attack-on-titan -s mangapark.to

# Download chapters after chapter 52 (useful for updates)
python cli.py manga -single secret-class -s manhuascan.us -l 52

# Download chapters 20 through 30 (inclusive range)
python cli.py manga -single secret-class -s manhuascan.us -r 20 30

# Download specific chapters only
python cli.py manga -single secret-class -s manhuascan.us -c 5 10 36

# Download with custom folder name, auto-merge images, and convert to PDF
python cli.py manga -single secret-class -s manhuascan.us -n "Secret Class" -m -p

# Adjust request delay to 0.5 seconds between requests
python cli.py manga -single one-piece -s mangapark.to -t 0.5
```

--------------------------------

### Define Custom Doujin Module (Python)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Defines an interface for creating custom doujin modules by inheriting from the `Doujin` base class. It includes methods for retrieving doujin titles and images, and supports modules where doujins are accessed by numeric codes.

```python
from utils.models import Manga, Doujin

# Doujin module interface
class CustomDoujinModule(Doujin):
    domain = "example.com"
    is_coded = True  # Whether doujins are accessed by numeric code

    def get_title(self, code: str) -> str:
        """Returns the title of the doujin"""
        pass

    def get_images(self, code: str) -> tuple[list[str], list[str] | bool]:
        """Returns tuple of (image_urls, save_names or False)"""
        pass
```

--------------------------------

### Utility Functions for File and Folder Management (Python)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Provides essential helper functions for the mangascraper, including image validation (corruption and truncation), folder creation and validation, filename sanitization, image detection within folders, JSON file operations, and implementing delays.

```python
from utils.assets import (
    validate_corrupted_image,
    validate_truncated_image,
    validate_folder,
    create_folder,
    fix_name_for_folder,
    detect_images,
    load_json_file,
    save_json_file,
    sleep
)

# Validate image integrity
is_valid = validate_corrupted_image("image.jpg")  # Returns: True/False
is_complete = validate_truncated_image("image.jpg")  # Returns: True/False

# Validate all images in a folder
invalid_image, images_list = validate_folder("Chapter 001")
if invalid_image:
    print(f"Found corrupted image: {invalid_image}")

# Create nested folder structure (sanitizes names automatically)
create_folder("My Manga/Chapter 001")

# Sanitize filename for filesystem
safe_name = fix_name_for_folder('Title: "Special" <Chapter>')
# Returns: "Title Special Chapter"

# Get sorted list of images in folder
images = detect_images("Chapter 001")
# Returns: ["Chapter 001/001.jpg", "Chapter 001/002.jpg", ...]

# JSON file operations
data = load_json_file("mangas.json")
save_json_file("output.json", {"key": "value"})

# Rate limiting
sleep(0.5)  # Sleep for 0.5 seconds
```

--------------------------------

### Image Merger (Bash)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Merges downloaded manga or doujin images vertically into single files, ideal for webtoon format. Supports merging entire series or individual chapters, with an option for 'fit mode' to eliminate whitespace.

```bash
# Merge all chapters of a manga (processes each subfolder)
python cli.py merge -bulk "One Piece"

# Merge a single chapter folder
python cli.py merge -folder "One Piece/Chapter 001"

# Merge with fit mode (resize to eliminate whitespace)
python cli.py merge -folder "One Piece/Chapter 001" -fit

# Enable fit mode globally during download
python cli.py manga -single one-piece -s mangapark.to -m -fit
```

--------------------------------

### Auto-Convert Manga During Download

Source: https://context7.com/yofagh/mangascraper/llms.txt

Automatically converts manga to PDF format as it is being downloaded. This streamlines the process of obtaining ready-to-read manga files.

```bash
python cli.py manga -single one-piece -s mangapark.to -p
```

--------------------------------

### Crawl Website Module Database

Source: https://context7.com/yofagh/mangascraper/llms.txt

Crawls and downloads the entire catalog or database of a specific website module. The results are saved to a JSON file for offline access and analysis.

```bash
# Crawl database from a specific module
python cli.py db -s manhuascan.us
```

--------------------------------

### Configure Scraper Settings (Python)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Defines global settings for the manga scraper, controlling aspects like logging, request delays, and automatic post-processing. These settings can be modified directly in `settings.py` or overridden using command-line interface flags.

```python
# settings.py defaults
LOGGING: bool = True           # Enable/disable logging output
SLEEP_TIME: int | float = 0.1  # Delay between requests (seconds)
AUTO_MERGE: bool = False       # Auto-merge images after download (-m flag)
AUTO_PDF_CONVERSION: bool = False  # Auto-convert to PDF (-p flag)
FIT_MERGE: bool = False        # Resize images during merge (-fit flag)
SEARCH_PAGE_LIMIT: int = 3     # Max pages to search (-page-limit flag)
SEARCH_ABSOLUTE: bool = False  # Require exact title match (-absolute flag)
```

--------------------------------

### Find Manga/Manhua Image Source

Source: https://context7.com/yofagh/mangascraper/llms.txt

Performs a reverse image search using provided local image files or URLs across multiple search engines like Yandex, TinEye, IQDB, and SauceNAO to find the source of manga/manhua images.

```bash
# Find source using a local image file
python cli.py sauce -image "path/to/screenshot.png"
```

```bash
# Find source using an image URL
python cli.py sauce -url "https://example.com/manga-page.jpg"
```

--------------------------------

### Define Custom Manga Module (Python)

Source: https://context7.com/yofagh/mangascraper/llms.txt

Provides an interface for creating custom manga modules by inheriting from the `Manga` base class. It defines essential methods like `get_chapters`, `get_images`, and `search_by_keyword` for web scraping manga sites.

```python
from utils.models import Manga, Doujin

# Manga module interface
class CustomMangaModule(Manga):
    domain = "example.com"

    def get_chapters(self, url: str) -> list[dict]:
        """Returns list of chapters with 'url' and 'name' keys"""
        # Returns: [{"url": "chapter-1", "name": "Chapter 001"}, ...]
        pass

    def get_images(self, url: str, chapter: dict) -> tuple[list[str], list[str] | bool]:
        """Returns tuple of (image_urls, save_names or False)"""
        # Returns: (["https://img1.jpg", "https://img2.jpg"], False)
        pass

    def search_by_keyword(self, keyword: str, absolute: bool):
        """Generator yielding search results per page"""
        # Yields: {"Title": {"url": "manga-url"}}
        pass

    def get_db(self):
        """Generator yielding database entries per page"""
        pass
```
Tokens:
Raw