### Install waifuc with video dependencies

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_videos/index.rst

Install the waifuc library with the necessary dependencies for video processing using pip.

```shell
pip install git+https://github.com/deepghs/waifuc.git@main#egg=waifuc[video]
```

--------------------------------

### Expected Output for Installation Verification

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/installation/index.rst

This is the expected output when the installation verification script is run successfully. It indicates the version of waifuc installed.

```text
0.1.0
```

--------------------------------

### Donut Semi-Finished Product Example 1

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Demonstrates handling a semi-finished product by applying icing to a donut.

```python
from waifuc.source import DonutSource
from waifuc.actions import IcingAction

source = DonutSource()

# Apply icing as a first step
actions_icing = IcingAction('strawberry')

# This creates a semi-finished donut with icing.
semi_finished_donut = source.attach(actions_icing)

print(f"Semi-finished donut: {semi_finished_donut}")
```

--------------------------------

### Install waifuc with GPU support

Source: https://github.com/deepghs/waifuc/blob/main/README.md

Install waifuc with GPU acceleration if your environment has CUDA available. This will enable faster processing.

```shell
pip install git+https://github.com/deepghs/waifuc.git@main#egg=waifuc[gpu]
```

--------------------------------

### Install Waifuc from PyPI (main branch)

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/installation/index.rst

Use this command to install the waifuc library from PyPI, pointing to the main branch of the GitHub repository. Requires Python 3.8 or higher.

```shell
pip install git+https://github.com/deepghs/waifuc.git@main#egg=waifuc
```

--------------------------------

### Donut Semi-Finished Product Example 2

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Shows how to continue processing a semi-finished product by adding jimmies after icing.

```python
from waifuc.source import DonutSource
from waifuc.actions import IcingAction, JimmiesAction

source = DonutSource()

# First, apply icing
actions_icing = IcingAction('strawberry')
semi_finished_donut = source.attach(actions_icing)

# Then, add jimmies to the already iced donut
actions_jimmies = JimmiesAction()
final_donut = semi_finished_donut.attach(actions_jimmies)

print(f"Final donut: {final_donut}")
```

--------------------------------

### Waifuc Workflow Example

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Demonstrates a typical Waifuc workflow involving filtering, resizing, and tagging images sequentially.

```python
from waifuc.source import DanbooruSource
from waifuc.actions import NoMonochromeAction, ResizeAction, TagAction

source = DanbooruSource(backend='api')

# Chain actions: remove monochrome, resize, then tag
actions = (
    NoMonochromeAction()
    .next(ResizeAction(width=512, height=512))
    .next(TagAction(['tag1', 'tag2']))
)

# Attach actions to the source
dataset = source.attach(actions)

# Iterate through the dataset to process images
for image in dataset:
    print(image.tags)
```

--------------------------------

### Verify Waifuc Installation

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/installation/index.rst

Execute this Python script to confirm that waifuc has been installed correctly. The script checks for successful import and basic functionality.

```python
import waifuc

print(waifuc.__version__)
```

--------------------------------

### Waifuc Separate Attach Example

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Demonstrates creating a new source and then attaching operations to it, showing equivalence to a single attach.

```python
from waifuc.source import DanbooruSource
from waifuc.actions import TagAction

source = DanbooruSource(backend='api')

# First, create a new source with initial operations
new_source = source.attach(TagAction(['tag1', 'tag2']))

# Then, attach further operations to the new source
dataset = new_source.attach(TagAction(['tag3', 'tag4']))

for image in dataset:
    print(image.tags)
```

--------------------------------

### Install Waifuc from GitHub (latest version)

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/installation/index.rst

Install or upgrade to the newest version of waifuc directly from GitHub using pip. This command ensures you have the latest updates.

```shell
pip install -U git+https://github.com/deepghs/waifuc@main
```

--------------------------------

### Install Waifuc with gchar dependency

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/quick_start/index.rst

Install the Waifuc library with the gchar extra for automatic character data sourcing. This is required to use the GcharAutoSource feature.

```shell
pip install git+https://github.com/deepghs/waifuc.git@main#egg=waifuc[gchar]
```

--------------------------------

### Donut Action Order Example

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Illustrates the importance of action order in a workflow, showing how reversing steps can lead to unexpected results.

```python
from waifuc.source import DonutSource
from waifuc.actions import IcingAction, JimmiesAction

source = DonutSource()

# Incorrect order: Jimmies first, then Icing
actions_failed = JimmiesAction().next(IcingAction('strawberry'))

# This will result in a poorly iced donut because jimmies interfere with icing adhesion.
donut_failed = source.attach(actions_failed)

print(f"Failed donut state: {donut_failed}")
```

--------------------------------

### Waifuc Action Order Example

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Shows how changing the order of Waifuc actions, like moving filtering to the end, can impact efficiency.

```python
from waifuc.source import DanbooruSource
from waifuc.actions import NoMonochromeAction, ResizeAction, TagAction

source = DanbooruSource(backend='api')

# Reordered actions: Resize and Tag first, then filter monochrome
actions_reordered = (
    ResizeAction(width=512, height=512)
    .next(TagAction(['tag1', 'tag2']))
    .next(NoMonochromeAction()) # Moved to the end
)

# Attaching this order can lead to processing monochrome images unnecessarily before filtering.
dataset_reordered = source.attach(actions_reordered)

for image in dataset_reordered:
    print(image.tags)
```

--------------------------------

### Waifuc Single Attach Example

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Equivalent to generating a new source and attaching operations to it, using a single attach call.

```python
from waifuc.source import DanbooruSource
from waifuc.actions import TagAction

source = DanbooruSource(backend='api')

# Using a single attach to apply operations
actions = TagAction(['tag1', 'tag2'])
dataset = source.attach(actions)

for image in dataset:
    print(image.tags)
```

--------------------------------

### Execute Crawling Script

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/quick_start/index.rst

Runs the Python script 'crawl.py' to start the data crawling process.

```shell
python crawl.py
```

--------------------------------

### Generate Character Dataset with Waifuc

Source: https://github.com/deepghs/waifuc/blob/main/README.md

This snippet demonstrates how to crawl images for a specific character (e.g., 'surtr' from Arknights), apply a series of data processing actions, and export the dataset. Ensure Waifuc is installed with gchar support (`pip install git+https://github.com/deepghs/waifuc.git@main#egg=waifuc[gchar]`).

```python
from waifuc.action import NoMonochromeAction, FilterSimilarAction, \
    TaggingAction, PaddingAlignAction, PersonSplitAction, FaceCountAction, FirstNSelectAction, \
    CCIPAction, ModeConvertAction, ClassFilterAction, RandomFilenameAction, AlignMinSizeAction
from waifuc.export import TextualInversionExporter
from waifuc.source import GcharAutoSource

if __name__ == '__main__':
    # data source for surtr in arknights, images from many sites will be crawled
    # all supported games and sites can be found at
    # https://narugo1992.github.io/gchar/main/best_practice/supported/index.html#supported-games-and-sites
    # ATTENTION: GcharAutoSource required `git+https://github.com/deepghs/waifuc.git@main#egg=waifuc[gchar]`
    s = GcharAutoSource('surtr')

    # crawl images, process them, and then save them to directory with given format
    s.attach(
        # preprocess images with white background RGB
        ModeConvertAction('RGB', 'white'),

        # pre-filtering for images
        NoMonochromeAction(),  # no monochrome, greyscale or sketch
        ClassFilterAction(['illustration', 'bangumi']),  # no comic or 3d
        # RatingFilterAction(['safe', 'r15']),  # filter images with rating, like safe, r15, r18
        FilterSimilarAction('all'),  # filter duplicated images

        # human processing
        FaceCountAction(1),  # drop images with 0 or >1 faces
        PersonSplitAction(),  # crop for each person
        FaceCountAction(1),

        # CCIP, filter the character you may not want to see in dataset
        CCIPAction(min_val_count=15),

        # if min(height, weight) > 800, resize it to 800
        AlignMinSizeAction(800),

        # tagging with wd14 v2, if you don't need character tag, set character_threshold=1.01
        TaggingAction(force=True),

        PaddingAlignAction((512, 512)),  # align to 512x512
        FilterSimilarAction('all'),  # filter again
        FirstNSelectAction(200),  # first 200 images
        # MirrorAction(),  # mirror image for data augmentation
        RandomFilenameAction(ext='.png'),  # random rename files
    ).export(
        # save to surtr_dataset directory
        TextualInversionExporter('surtr_dataset')
    )

```

--------------------------------

### Illustrative Waifuc Source Example

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

This Python code illustrates different image data sources in Waifuc, showing they are used uniformly for downstream processing.

```python
from waifuc.source import Local, Danbooru

# Example of acquiring images from different sources
# All sources output images in a uniform format for downstream processing.

# Source 1: From local files
source1 = Local(path="/path/to/your/images")

# Source 2: From Danbooru
source2 = Danbooru(tags="cat, anime")

# Source 3: From video frames (hypothetical)
# source3 = VideoFrames(path="/path/to/your/video.mp4")

# The acquired images will be in a uniform format for downstream processing.
```

--------------------------------

### Random Color Image Metadata Example

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

An example of the metadata associated with a randomly generated image, including source, generation details, dimensions, and color information.

```json
{
    "source": "random_color_generator",
    "generated_at": "now",
    "width": 512,
    "height": 512,
    "color": [123, 45, 234],
    "original_filename": "random_color_1.png"
}
```

--------------------------------

### Packaged Donut Delivery Exporter

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Demonstrates an exporter that packages a dozen donuts before handing them over to the customer. This is a conceptual example.

```python
from waifuc.source import LocalSource
from waifuc.export import LocalExporter

# Assume donuts_source is a Waifuc Source object
donuts_source = LocalSource()

# Assume customer is a Customer object
class Customer:
    def receive(self, donuts):
        print(f"Customer received {len(donuts)} donuts.")

customer = Customer()

# Function to package donuts into dozens
def package_donuts(donuts):
    packaged = []
    for i in range(0, len(donuts), 12):
        packaged.append(donuts[i:i+12])
    return packaged

# Exporter that packages donuts before delivery
exporter = LocalExporter(lambda donuts: customer.receive(package_donuts(donuts)))

# Execute the export process
exporter.export(donuts_source)
```

--------------------------------

### Use a Custom Action in a Waifuc Pipeline

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Instantiate and use a custom action within a Waifuc pipeline. This example shows how to apply the MyRandomAction to a data source.

```python
from waifuc.source import LocalSource
from waifuc.action import BaseAction
import random

# Assume MyRandomAction is defined as above

# Define the custom action
class MyRandomAction(BaseAction):
    def __init__(self, select_prob: float, rotate_prob: float):
        self.select_prob = select_prob
        self.rotate_prob = rotate_prob

    def iter(self, image: Image):
        if random.random() < self.select_prob:
            if random.random() < self.rotate_prob:
                image.img = image.img.transpose(Image.FLIP_LEFT_RIGHT)
                image.meta.update({'random_value': 'rotated'})
            else:
                image.meta.update({'random_value': 'selected'})
            yield image

# Create a pipeline with the custom action
source = LocalSource('path/to/your/images')
action = MyRandomAction(select_prob=0.5, rotate_prob=0.5)

# Process the data
for image in source.pipeline(action):
    # Save or further process the image
    image.save(f"output/{image.meta.get('filename', 'default_filename')}.png")

```

--------------------------------

### Direct Donut Delivery Exporter

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Illustrates a simple exporter that hands over donuts directly to the customer. This is a conceptual example and cannot be run directly.

```python
from waifuc.source import LocalSource
from waifuc.export import LocalExporter

# Assume donuts_source is a Waifuc Source object
donuts_source = LocalSource()

# Assume customer is a Customer object
class Customer:
    def receive(self, donuts):
        print(f"Customer received {len(donuts)} donuts.")

customer = Customer()

# Exporter that directly hands over donuts
exporter = LocalExporter(customer.receive)

# Execute the export process
exporter.export(donuts_source)
```

--------------------------------

### Union Data Sources

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

Combines data sources by randomly selecting images from each until a total quantity is met. This example collects 60 images by randomly picking from Danbooru and Zerochan.

```python
from waifuc.source import DanbooruSource, ZerochanSource

s_db = DanbooruSource(size=30)
s_zerochan = ZerochanSource(size=30)
s = s_db | s_zerochan
s.save(60)

```

--------------------------------

### Concatenation with Preprocessing

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

Performs concatenation after applying preprocessing steps to data sources. This example crawls images from Zerochan and Danbooru, removes backgrounds for Zerochan images, and saves a total of 60 images.

```python
from waifuc.source import DanbooruSource, ZerochanSource

s_zerochan = ZerochanSource(size=30).attach(lambda x: x.remove_bg())
s_db = DanbooruSource(size=30)
s = s_zerochan + s_db

```

--------------------------------

### Reloading Local Datasets with LocalSource

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

Reloads a dataset with metadata from a local path and re-saves it in the LoRA training dataset format. Requires the 'waifuc' library to be installed.

```python
from waifuc.source import LocalSource
from waifuc.export_ import SaveExporter

# Assuming you have a directory with images and their corresponding meta.json files
# For example: /path/to/your/dataset/
# This directory should contain image files (e.g., image1.jpg) and meta files (e.g., image1_meta.json)

# Initialize LocalSource with the path to your dataset
local_source = LocalSource("/path/to/your/dataset/")

# Initialize SaveExporter to save the dataset in LoRA format
# The output directory will be created if it doesn't exist
save_exporter = SaveExporter("./lora_dataset/")

# Run the pipeline: load from local_source and export using save_exporter
# This will process each image and its metadata, then save them in the specified format
save_exporter.run(local_source)

```

--------------------------------

### Bulk Donut Loading Exporter

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Illustrates an exporter for loading a large quantity of donuts into boxes and onto a pickup truck. This is a conceptual example.

```python
from waifuc.source import LocalSource
from waifuc.export import LocalExporter

# Assume donuts_source is a Waifuc Source object
donuts_source = LocalSource()

# Assume truck is a Truck object
class Truck:
    def load(self, boxes):
        print(f"Loaded {len(boxes)} boxes onto the truck.")

truck = Truck()

# Function to box donuts
def box_donuts(donuts):
    boxes = []
    for i in range(0, len(donuts), 100):
        boxes.append(donuts[i:i+100])
    return boxes

# Exporter that loads bulk donuts onto a truck
exporter = LocalExporter(lambda donuts: truck.load(box_donuts(donuts)))

# Execute the export process
exporter.export(donuts_source)
```

--------------------------------

### Complex Data Source Construction

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

Creates a sophisticated data source using nested concatenation and union operations. This example first crawls 50 images from Zerochan, then 50 images randomly from Danbooru and Pixiv.

```python
from waifuc.source import DanbooruSource, PixivSearchSource, ZerochanSource

s_zerochan = ZerochanSource(size=50)
s_db = DanbooruSource(size=50)
s_pixiv = PixivSearchSource(size=50)
s = s_zerochan[:50] + (s_db | s_pixiv)[:50]

```

--------------------------------

### Use Custom Processing with Filtering

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Combine custom processing actions with filtering actions to refine the dataset. This example uses CutHeadAction followed by FilterSimilarAction to improve results.

```python
from waifuc.source import LocalSource
from waifuc.action import CutHeadAction, FilterSimilarAction

source = LocalSource()
actions = [
    CutHeadAction(),
    FilterSimilarAction(threshold=0.95),
]

async for data in source.pipeline(actions=actions):
    print(data)
```

--------------------------------

### Concatenate Data Sources

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

Combines two data sources sequentially. This example crawls 30 images from Danbooru and then another 30 from Zerochan.

```python
from waifuc.source import DanbooruSource, ZerochanSource

s_db = DanbooruSource(size=30)
s_zerochan = ZerochanSource(size=30)
s = s_db + s_zerochan

```

--------------------------------

### Define a Custom Action with Random Selection and Mirror Rotation

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Implement a custom action by inheriting from BaseAction and overriding the iter method. This example demonstrates random selection and mirror rotation, also adding metadata for final image names.

```python
from waifuc.action import BaseAction
from waifuc.utils import Image
import random

class MyRandomAction(BaseAction):
    def __init__(self, select_prob: float, rotate_prob: float):
        self.select_prob = select_prob
        self.rotate_prob = rotate_prob

    def iter(self, image: Image):
        if random.random() < self.select_prob:
            if random.random() < self.rotate_prob:
                image.img = image.img.transpose(Image.FLIP_LEFT_RIGHT)
                image.meta.update({'random_value': 'rotated'})
            else:
                image.meta.update({'random_value': 'selected'})
            yield image

```

--------------------------------

### Random Color Image Generation Data Source

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

An example of a custom data source that generates solid-color images of random sizes using Pillow and saves them with associated metadata.

```python
import os
import random
from pathlib import Path
from typing import AsyncGenerator

from PIL import Image, ImageDraw

from waifuc.source.base import BaseDataSource, ImageItem


class RandomColorDataSource(BaseDataSource):
    def __init__(
        self,
        *,  # Force to pass keyword arguments to prevent it from being used as positional arguments
        name: str | None = None,
        path: str | None = None,
        ignore_path: bool = False,
        ignore_name: bool = False,
        num_images: int = 100,
        max_width: int = 1024,
        max_height: int = 1024,
        output_dir: str = "./random_color_images",
    ):
        super().__init__(name=name, path=path, ignore_path=ignore_path, ignore_name=ignore_name)
        self.num_images = num_images
        self.max_width = max_width
        self.max_height = max_height
        self.output_dir = Path(output_dir)
        self.output_dir.mkdir(parents=True, exist_ok=True)

    def _iter(self) -> AsyncGenerator[ImageItem, None]:
        for i in range(self.num_images):
            width = random.randint(100, self.max_width)
            height = random.randint(100, self.max_height)
            color = (random.randint(0, 255), random.randint(0, 255), random.randint(0, 255))

            img = Image.new("RGB", (width, height), color)
            draw = ImageDraw.Draw(img)
            draw.text((10, 10), f"Image {i+1}", fill=(255, 255, 255))  # Add some text

            # Save the image
            image_filename = f"random_color_{i+1}.png"
            image_path = self.output_dir / image_filename
            img.save(image_path)

            # Prepare metadata
            metadata = {
                "source": "random_color_generator",
                "generated_at": "now",
                "width": width,
                "height": height,
                "color": color,
                "original_filename": image_filename,
            }

            yield ImageItem(
                image_path=str(image_path),
                name=f"random_color_{i+1}",
                metadata=metadata,
            )


async def main():
    # Example usage:
    source = RandomColorDataSource(num_images=5, max_width=512, max_height=512)
    async for item in source:
        print(f"Generated: {item.name}, Path: {item.image_path}, Metadata: {item.metadata}")


if __name__ == "__main__":
    # Create a dummy image file for the example to run without errors
    # In a real scenario, this would be generated by the source itself
    dummy_dir = Path("./random_color_images")
    dummy_dir.mkdir(parents=True, exist_ok=True)
    dummy_img_path = dummy_dir / "random_color_1.png"
    if not dummy_img_path.exists():
        Image.new("RGB", (60, 30), color = (73, 109, 137)).save(dummy_img_path)
        with open(dummy_dir / "random_color_1_metadata.json", "w") as f:
            f.write('{\"source\": \"dummy\"}')

    import asyncio

    asyncio.run(main())

```

--------------------------------

### Define a Custom Data Exporter

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Inherit from BaseExporter and implement pre_export, export_item, post_export, and reset methods for custom export logic. This example changes the output format to CSV.

```python
import csv
from waifuc.export import BaseExporter

class CsvExporter(BaseExporter):
    def __init__(self, path: str = "tags.csv"):
        super().__init__()
        self.path = path
        self.tags = []

    async def pre_export(self) -> None:
        self.file = open(self.path, "w", newline=",")
        self.writer = csv.writer(self.file)
        self.writer.writerow(["filename", "tags"])

    async def export_item(self, item: dict) -> None:
        self.tags.append((item.get("name", "unknown"), item.get("tags", "")))

    async def post_export(self) -> None:
        for filename, tags in self.tags:
            self.writer.writerow([filename, tags])
        self.file.close()
```

--------------------------------

### Use a Custom Data Exporter

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Instantiate and use a custom BaseExporter to control how data items are saved. This example uses CsvExporter to save image filenames and their tags to a CSV file.

```python
from waifuc.source import LocalSource
from waifuc.export import CsvExporter

source = LocalSource()
exporter = CsvExporter()

async for _ in source.export(exporter=exporter):
    pass
```

--------------------------------

### Filter Images by Face Count

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/process_images/index.rst

Use FaceCountAction to filter images based on the number of faces present. This example specifically filters out images containing only one face.

```python
from waifuc.action import FaceCountAction

action = FaceCountAction(1, FaceCountAction.Operator.GREATER_EQUAL) # Keep images with 1 or more faces
# action = FaceCountAction(1, FaceCountAction.Operator.LESS_EQUAL) # Keep images with 1 or fewer faces
# action = FaceCountAction(1, FaceCountAction.Operator.EQUAL) # Keep images with exactly 1 face
# action = FaceCountAction(1, FaceCountAction.Operator.NOT_EQUAL) # Keep images with not exactly 1 face

async def main():
    await action(images)

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())
```

--------------------------------

### Huggingface Game Character Skins Data Source Definition

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Defines a custom web data source to fetch game character skins from a Huggingface repository. It iterates through repository files to get image URLs and metadata.

```python
import os
from pathlib import Path
from typing import AsyncGenerator

from waifuc.source.web import WebDataSource
from waifuc.source.base import ImageItem


class HuggingfaceGameCharacterSkinsDataSource(WebDataSource):
    def __init__(
        self,
        *,  # Force to pass keyword arguments to prevent it from being used as positional arguments
        name: str | None = None,
        path: str | None = None,
        ignore_path: bool = False,
        ignore_name: bool = False,
        repo_id: str = "deepghs/game_character_skins",
        revision: str = "main",
        image_exts: list[str] = ["jpg", "jpeg", "png"],
    ):
        super().__init__(name=name, path=path, ignore_path=ignore_path, ignore_name=ignore_name)
        self.repo_id = repo_id
        self.revision = revision
        self.image_exts = image_exts

    async def _iter_data(self) -> AsyncGenerator[tuple[str, str, dict | None], None]:
        from huggingface_hub import hf_hub_download, list_repo_files

        files = list_repo_files(self.repo_id, repo_type="dataset", revision=self.revision)
        for file in files:
            if file.lower().endswith(tuple(self.image_exts)):
                # Example: file = "Amiya/Amiya_0.jpg"
                resource_id = Path(file).stem  # Amiya_0
                image_url = f"https://huggingface.co/datasets/{self.repo_id}/resolve/{self.revision}/{file}"
                metadata = {
                    "repo_id": self.repo_id,
                    "revision": self.revision,
                    "file_path": file,
                }
                yield resource_id, image_url, metadata


async def main():
    # Example usage:
    source = HuggingfaceGameCharacterSkinsDataSource()
    async for item in source:
        print(f"Got item: {item.name}, URL: {item.image_path}, Metadata: {item.metadata}")
        # In a real scenario, you would process the item further, e.g., download and save the image
        # For demonstration, we'll just print the info
        break  # Process only one item for this example


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

```

--------------------------------

### Using a Function to Create a Waifu Dataset

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/dynamic/index.rst

Demonstrates how to call the dataset construction function with different parameters to obtain varied datasets. Adjust parameters to customize the output.

```python
waifu_data = create_waifu_dataset("Asuna", 16, "swimming")
print(waifu_data)

```

--------------------------------

### Load, Crop, and Save Local Images

Source: https://github.com/deepghs/waifuc/blob/main/README.md

Loads images from a local directory, applies 3-stage cropping, and saves the output to another directory. Ensure you replace '/your/path/contains/images' and '/your/output/path' with your actual directory paths.

```python
from waifuc.action import ThreeStageSplitAction
from waifuc.export import SaveExporter
from waifuc.source import LocalSource

source = LocalSource('/your/path/contains/images')
source.attach(
    ThreeStageSplitAction(),
).export(SaveExporter('/your/output/path'))


```

--------------------------------

### Iterating Over Data Sources Directly

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

Demonstrates how to iterate over a waifuc data source directly in memory without saving images to disk. This is useful for immediate processing or when disk space is a concern.

```python
from waifuc.source import LocalSource

# Initialize a data source (e.g., LocalSource)
# This example assumes a local directory with images
data_source = LocalSource("/path/to/your/dataset/")

# Iterate over the data source
# Each item will contain an 'image' object (PIL.Image) and 'meta' data
for item in data_source:
    # Access the image object
    image = item['image']
    # Access the metadata dictionary
    metadata = item['meta']

    # Perform operations on the image and metadata as needed
    # For example, print image dimensions and some metadata tags
    print(f"Image size: {image.size}")
    if 'tags' in metadata:
        print(f"Tags: {metadata['tags'][:5]}...") # Print first 5 tags

```

--------------------------------

### Illustrative Donut Source Analogy

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

This Python code is for illustration and analogy purposes only, demonstrating the concept of acquiring raw donuts (images) from various sources.

```python
from waifuc.source import Local, Danbooru

# Example of acquiring raw donuts (images) from different sources
# This code is for illustration and analogy purposes only.

# Source 1: From local files
source1 = Local(path="/path/to/your/donuts")

# Source 2: From Danbooru
source2 = Danbooru(tags="donut, chocolate")

# The acquired donuts (images) will be in a uniform format for downstream processing.
```

--------------------------------

### Sample Meta-Information JSON

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

A sample JSON file containing metadata for an image, including Danbooru information, URL, filename, and tags.

```json
{
    "id": 6814120,
    "danbooru_id": 6814120,
    "source": "https://danbooru.donmai.us/posts/6814120",
    "tags": [
        "1girl",
        "blue_eyes",
        "brown_hair",
        "school_uniform",
        "short_hair",
        "blush",
        "collarbone",
        "japanese_clothes",
        "looking_at_viewer",
        "sailor_collar",
        "skirt",
        "white_shirt"
    ],
    "width": 1000,
    "height": 1414,
    "save_path": "danbooru/6814120.jpg",
    "save_name": "danbooru_6814120.jpg"
}
```

--------------------------------

### Using Huggingface Game Character Skins Data Source

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Demonstrates how to use the HuggingfaceGameCharacterSkinsDataSource to obtain images of character skins from a specified Huggingface repository.

```python
from waifuc.source.huggingface import HuggingfaceGameCharacterSkinsDataSource


async def main():
    # Example usage:
    source = HuggingfaceGameCharacterSkinsDataSource()
    async for item in source:
        print(f"Got item: {item.name}, URL: {item.image_path}, Metadata: {item.metadata}")
        # In a real scenario, you would process the item further, e.g., download and save the image
        # For demonstration, we'll just print the info
        break  # Process only one item for this example


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

```

--------------------------------

### Display Environment Information

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/information/environment.ipynb

This Python script uses several libraries to gather and print details about the operating system, Python interpreter, CPU, memory, and GPU (CUDA) availability. It's designed to run in environments like GitHub Actions.

```python
import os
import platform
import shutil

import cpuinfo
import psutil
from hbutils.scale import size_to_bytes_str

print('OS:', platform.platform())
print('Python:', platform.python_implementation(), platform.python_version())
print('CPU Brand:', cpuinfo.get_cpu_info()["brand_raw"])
print('CPU Count:', os.cpu_count())
print('CPU Freq:', psutil.cpu_freq().current, 'MHz')
print('Memory Size:', size_to_bytes_str(psutil.virtual_memory().total, precision=3))
print('Has CUDA:', 'Yes' if shutil.which('nvidia-smi') else 'No')
```

--------------------------------

### Integrating Data Export into Dataset Construction

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/dynamic/index.rst

Shows how to directly include data exporting logic within the dataset construction function. This is useful for immediate saving of generated data.

```python
import json

def create_and_export_waifu_dataset(name, age, hobby, filename):
    data = {
        "name": name,
        "age": age,
        "hobby": hobby,
    }
    with open(f"{filename}.json", "w") as f:
        json.dump(data, f)
    return data

```

--------------------------------

### Concise usage of FirstNSelectAction

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/process_images/index.rst

A shorthand syntax for FirstNSelectAction can be used, which is equivalent to explicitly using the action class.

```python
from waifuc.source import LocalSource

datasource = LocalSource(path='/data/mydataset')
processed_datasource = datasource.attach(num=100)
processed_datasource.save('/data/dstdataset')

```

--------------------------------

### Crawl 50 Images from Gelbooru

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet demonstrates how to crawl 50 images from Gelbooru.com.

```python
from waifuc.source import GelbooruSource
from waifuc.executor import SaveExporter

(GelbooruSource(max_images=50) + SaveExporter(path='/data/gelbooru_50')).ex()
```

--------------------------------

### Define a Stateful Action: FirstNSelectAction

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Illustrates a stateful action, FirstNSelectAction, which keeps only the first 'n' data items. It requires implementing a 'reset' method for state management.

```python
from waifuc.action import BaseAction
from waifuc.utils import Image

class FirstNSelectAction(BaseAction):
    def __init__(self, n: int):
        self.n = n
        self.count = 0

    def iter(self, image: Image):
        if self.count < self.n:
            self.count += 1
            yield image

    def reset(self):
        self.count = 0

```

--------------------------------

### Constructing a Dataset within a Function

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/dynamic/index.rst

Defines a function to build a dataset for a 'waifu'. Use this when you need to encapsulate data generation logic.

```python
def create_waifu_dataset(name, age, hobby):
    return {
        "name": name,
        "age": age,
        "hobby": hobby,
    }

```

--------------------------------

### Split Images into Full Body, Upper Body, and Head

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/process_images/index.rst

Utilize ThreeStageSplitAction to split portrait images into different stages (full body, upper body, head) for improved LoRA generalization and character detail fidelity during model training. It is recommended to follow this with FilterSimilarAction to remove duplicate sub-images.

```python
from waifuc.action import ThreeStageSplitAction, FilterSimilarAction

action = ThreeStageSplitAction()
action_filter = FilterSimilarAction()

```

--------------------------------

### Crawl 50 Images from Pixiv by Keyword

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet shows how to use PixivSearchSource to crawl 50 images from Pixiv based on keywords.

```python
from waifuc.source import PixivSearchSource
from waifuc.executor import SaveExporter

(PixivSearchSource(keyword='amiya', max_images=50) + SaveExporter(path='/data/pixiv_50')).ex()
```

--------------------------------

### Illustrative Strawberry Donut Production with Jimmies

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

This Python code is for illustrative purposes only, demonstrating the concept of producing strawberry-flavored donuts with jimmies (images) through actions.

```python
from waifuc.action import BaseAction

class StrawberryDonutAction(BaseAction):
    def __call__(self, image):
        # Simulate processing to make a strawberry donut with jimmies
        print("Applying strawberry icing and jimmies...")
        # In a real scenario, this would modify the image object
        return image

# This code is for illustrative purposes only.
# It demonstrates the concept of applying actions to create finished donuts.
```

--------------------------------

### Crawl 50 Images from Zerochan (Simple)

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet shows a basic way to crawl 50 images from Zerochan. It uses SaveExporter and stores data in a specified directory.

```python
from waifuc.source import ZerochanSource
from waifuc.executor import SaveExporter

(ZerochanSource(max_images=50) + SaveExporter(path='/data/amiya_zerochan')).ex()
```

--------------------------------

### Crawl 50 Images from Pixiv by Ranking

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet shows how to use PixivRankingSource to crawl 50 images from Pixiv's ranking.

```python
from waifuc.source import PixivRankingSource
from waifuc.executor import SaveExporter

(PixivRankingSource(max_images=50) + SaveExporter(path='/data/pixiv_50_ranking')).ex()
```

--------------------------------

### Extract Character Images from Videos using PersonSplitAction

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_videos/index.rst

Extract frames from videos and save portraits obtained from those frames by adding PersonSplitAction.

```python
from waifuc.source import VideoSource
from waifuc.actions import PersonSplitAction

# Example usage:
# source = VideoSource(path='/data/videos')
# source.save('/data/dstdataset', actions=[PersonSplitAction()])

```

--------------------------------

### Select the first N images from a data source

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/process_images/index.rst

Use FirstNSelectAction to limit the number of data items processed. This is useful for web data sources to ensure finiteness and for general dataset truncation.

```python
from waifuc.source import LocalSource
from waifuc.actions import FirstNSelectAction

datasource = LocalSource(path='/data/mydataset')
processed_datasource = datasource.attach(FirstNSelectAction(num=100))
processed_datasource.save('/data/dstdataset')

```

--------------------------------

### Attach an action to a data source

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/process_images/index.rst

Use the attach method of the data source to add an action, creating a new data source. This method returns a new data source and does not modify the original.

```python
from waifuc.source import LocalSource
from waifuc.actions import ResizeAction

datasource = LocalSource(path='path/to/images')
new_datasource = datasource.attach(ResizeAction(size=512))

```

--------------------------------

### Use a Custom Processing Action

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Instantiate and use a custom ProcessAction within a pipeline to transform data items. The 'process' method is called for each item, and the output replaces the input.

```python
from waifuc.source import LocalSource
from waifuc.action import CutHeadAction

source = LocalSource()
actions = [CutHeadAction()]

async for data in source.pipeline(actions=actions):
    print(data)
```

--------------------------------

### Tag Images for LoRA Training

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/process_images/index.rst

Employ TaggingAction to generate tags for images, which are stored in metadata. Use TextureInversionExporter for final export in LoRA training format. Consider using force=True to overwrite existing tags.

```python
from waifuc.action import TaggingAction
from waifuc.export import TextureInversionExporter

action = TaggingAction()
exporter = TextureInversionExporter()

async def main():
    await action(images)
    await exporter(images, "output_dir")

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())
```

--------------------------------

### Crawl 50 Images from Anime-Pictures

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet demonstrates how to crawl 50 images from Anime-Pictures.net.

```python
from waifuc.source import AnimePicturesSource
from waifuc.executor import SaveExporter

(AnimePicturesSource(max_images=50) + SaveExporter(path='/data/anime_pictures_50')).ex()
```

--------------------------------

### Align Images to Minimum Size

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/process_images/index.rst

Use AlignMinSizeAction to compress large images when only smaller sizes are needed for training, saving storage space. This action scales images with a short side greater than 800px to have a short side of 800px, maintaining aspect ratio. Images with a short side of 800px or less are not modified.

```python
from waifuc.action import AlignMinSizeAction

action = AlignMinSizeAction(800)

```

--------------------------------

### Crawl 50 Images from Zerochan (with Login)

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet demonstrates how to log in to Zerochan using username and password to access member-only images. It crawls 50 images.

```python
from waifuc.source import ZerochanSource
from waifuc.executor import SaveExporter

(ZerochanSource(username='YOUR_USERNAME', password='YOUR_PASSWORD', max_images=50) + SaveExporter(path='/data/amiya_zerochan')).ex()
```

--------------------------------

### Crawl 50 Solo Images from Danbooru

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet demonstrates how to collect solo images from Danbooru by adding the 'solo' tag.

```python
from waifuc.source import DanbooruSource
from waifuc.executor import SaveExporter

(DanbooruSource(tags='solo', max_images=50) + SaveExporter(path='/data/danbooru_50_solo')).ex()
```

--------------------------------

### Set ONNX_MODE to CPU

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/device/index.rst

Set the ONNX_MODE environment variable to 'cpu' to force Waifuc to use the CPU for ONNX model execution. This is applicable on both Linux and Windows command prompt environments.

```shell
# Linux
export ONNX_MODE=cpu

# Windows, CMD
set ONNX_MODE=cpu
```

--------------------------------

### Crawl 50 Images from Sankaku

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet shows how to crawl 50 images from Sankakucomplex.com.

```python
from waifuc.source import SankakuSource
from waifuc.executor import SaveExporter

(SankakuSource(max_images=50) + SaveExporter(path='/data/sankaku_50')).ex()
```

--------------------------------

### Crawl 50 Images from Pixiv by Artist

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet demonstrates how to use PixivUserSource to crawl 50 images from a specific artist on Pixiv.

```python
from waifuc.source import PixivUserSource
from waifuc.executor import SaveExporter

(PixivUserSource(user_id=12345, max_images=50) + SaveExporter(path='/data/pixiv_50_user')).ex()
```

--------------------------------

### Export Images with TXT Metadata

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/workflow/index.rst

Exports data in the format of images along with TXT files, which is suitable for training most neural network models. This provides a common format for machine learning tasks.

```python
from waifuc.source import LocalSource
from waifuc.export import LocalExporter

source = LocalSource()
exporter = LocalExporter(save_path="./output", file_format="{name}.{ext}", txt_path="./output_txt")
exporter.export(source)
```

--------------------------------

### Crawl 50 Images from Zerochan (Full Size)

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet shows how to modify the size selection to crawl full-size images from Zerochan after logging in. It crawls 50 images.

```python
from waifuc.source import ZerochanSource
from waifuc.executor import SaveExporter

(ZerochanSource(username='YOUR_USERNAME', password='YOUR_PASSWORD', max_images=50, size_mode='full') + SaveExporter(path='/data/amiya_zerochan')).ex()
```

--------------------------------

### Extract Higher Quality Character Images from Videos

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_videos/index.rst

Enhance character image extraction from videos by adding more actions to obtain a higher quality training dataset.

```python
from waifuc.source import VideoSource
from waifuc.actions import PersonSplitAction, TagAction, Text2ImageAction

# Example usage:
# source = VideoSource(path='/data/videos')
# source.save('/data/dstdataset', actions=[PersonSplitAction(), TagAction(), Text2ImageAction()])

```

--------------------------------

### Use a Custom Filtering Action

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/advanced_guides/customize/index.rst

Instantiate and use a custom FilterAction within a pipeline to filter data items. Ensure the data items have the necessary keys (e.g., 'type') for the filter to work.

```python
from waifuc.source import LocalSource
from waifuc.action import ComicOnlyAction

source = LocalSource()
actions = [ComicOnlyAction()]

async for data in source.pipeline(actions=actions):
    print(data)
```

--------------------------------

### Crawl Images from Pixiv

Source: https://github.com/deepghs/waifuc/blob/main/README.md

Crawls 10 images related to 'アークナイツ' (surtr) from Pixiv, applying head count, CCIP filtering, and minimum size alignment, then saves them.

```python
from waifuc.action import HeadCountAction, AlignMinSizeAction, CCIPAction
from waifuc.export import SaveExporter
from waifuc.source import PixivSearchSource

if __name__ == '__main__':
    source = PixivSearchSource(
        'アークナイツ (surtr OR スルト OR 史尔特尔)',
        refresh_token='use_your_own_refresh_token'
    )
    source.attach(
        # only 1 head,
        HeadCountAction(1),

        # pixiv often have some irrelevant character mixed in
        # so CCIPAction is necessary here to drop these images
        CCIPAction(),

        # if shorter side is over 640, just resize it to 640
        AlignMinSizeAction(640),
    )[:10].export(
        # save images (with meta information from danbooru site)
        SaveExporter('/data/surtr_arknights_pixiv')
    )
```

--------------------------------

### Crawl 50 Images from Duitang

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet shows how to crawl 50 images from the Chinese website Duitang.com.

```python
from waifuc.source import DuitangSource
from waifuc.executor import SaveExporter

(DuitangSource(max_images=50) + SaveExporter(path='/data/duitang_50')).ex()
```

--------------------------------

### Automatic Data Source Selection with GcharAutoSource

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/quick_start/index.rst

Replace DanbooruSource with GcharAutoSource to automatically select the most suitable data source for a character. This simplifies data crawling by directly inputting the character's name.

```python
from waifuc.source import GcharAutoSource

s = GcharAutoSource('surtr')
```

--------------------------------

### Crawl 50 Images from Danbooru

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_images/index.rst

This snippet shows how to crawl 50 images from Danbooru.

```python
from waifuc.source import DanbooruSource
from waifuc.executor import SaveExporter

(DanbooruSource(max_images=50) + SaveExporter(path='/data/danbooru_50')).ex()
```

--------------------------------

### Convert image mode to RGB

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/process_images/index.rst

Use ModeConvertAction to convert images to RGB mode. This action fills transparent backgrounds with white and is recommended as the first action for most processing pipelines.

```python
from waifuc.source import LocalSource
from waifuc.actions import ModeConvertAction

datasource = LocalSource(path='/data/mydataset')
processed_datasource = datasource.attach(ModeConvertAction(mode='RGB'))

```

--------------------------------

### Extract Images from a Single Video File

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_videos/index.rst

Use VideoSource to process a video file, extract frames, and save them as images.

```python
from waifuc.source import VideoSource

# Example usage:
# source = VideoSource(path='/path/to/your/video.mp4')
# source.save('/path/to/save/images/')

```

--------------------------------

### Extract Images from a Folder of Videos

Source: https://github.com/deepghs/waifuc/blob/main/docs/source/tutorials/crawl_videos/index.rst

Process all video files within a specified folder, extracting frames and saving them to a destination folder.

```python
from waifuc.source import VideoSource

# Example usage:
# source = VideoSource(path='/data/videos')
# source.save('/data/dstdataset')

```