### Install EasyAnimate via setup script

Source: https://github.com/datajuicer/data-juicer/blob/main/thirdparty/models/README.md

Run the provided shell command to install EasyAnimate. This is the first step for using the EasyAnimate third-party model library.

```shell
bash ./setup_easyanimate.sh
```

--------------------------------

### Start Label Studio with pip on port 7070 and create test project

Source: https://github.com/datajuicer/data-juicer/blob/main/tools/humanops/README.md

Example command that starts Label Studio using pip, sets port 7070, and creates a test project. After startup, check the generated connection file.

```bash
python label_studio_service.py --use-pip --port 7070 --create-test-project
```

--------------------------------

### Install Data-Juicer from source with uv

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/tutorial/QuickStart.md

Clones the repository, creates a virtual environment, and installs the package with NLP extras. Use this to get the demo recipes and sample data. Requires Python, Git, and uv.

```bash
git clone https://github.com/datajuicer/data-juicer.git --depth 1
cd data-juicer
uv venv
source .venv/bin/activate  # Linux / macOS
uv pip install -e ".[nlp]"
```

--------------------------------

### Basic installation from PyPI

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/tutorial/Installation.md

Install the core Data-Juicer package from PyPI using uv.

```bash
uv pip install py-data-juicer
```

--------------------------------

### Install Megatron-LM on training machines

Source: https://github.com/datajuicer/data-juicer/blob/main/thirdparty/LLM_ecosystems/README.md

Runs the setup script to install Megatron-LM inside the container. Required for training machines.

```shell
./setup_megatron.sh
```

--------------------------------

### Install package with all dependencies

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/DeveloperGuide.md

Install the package with all dependencies using `uv pip install -e ".[all]"`.

```bash
uv pip install -e ".[all]"
```

--------------------------------

### Clone and install Data-Juicer from source

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/tutorial/Installation.md

Clones the repository, creates a virtual environment, and installs Data-Juicer in editable mode. Optionally installs a specific domain like vision. Use -e to get tools, demos, and app.py entry points.

```bash
# Clone repository
git clone https://github.com/datajuicer/data-juicer.git
cd data-juicer
uv venv
source .venv/bin/activate
uv pip install -e .

# You can install specific domain as well
uv pip install -e ".[vision]"
```

--------------------------------

### Launch Web Playground with streamlit run app.py

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/Playground.md

Run from the project root to start the Playground. Requires a source installation of Data-Juicer. The browser opens the interface automatically.

```bash
streamlit run app.py
```

--------------------------------

### Install development tools extra

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/tutorial/Installation.md

Install the dev extra for development and testing.

```bash
# Development & Testing
uv pip install "py-data-juicer[dev]"
```

--------------------------------

### Call init_configs function via GET (Python)

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/DJ_service.md

Python example using requests to call the init_configs function via GET. The URL path maps to the function reference path. Parameters are passed as query parameters, with list/dictionary values serialized using json.dumps and prefixed with '<json_dumps>'.

```python
import requests
import json

json_prefix = '<json_dumps>'
url = 'http://localhost:8000/data_juicer/config/init_configs'
params = {"args": json_prefix + json.dumps(['--config', './demos/process_simple/process.yaml'])}
response = requests.get(url, params=params)
print(json.loads(response.text))
```

--------------------------------

### Command-line export examples (bash)

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/Export.md

Command-line examples for exporting datasets with different options: basic, Parquet, sharding, and keeping stats.

```bash
# Basic export
dj-process --config config.yaml --export_path ./outputs/result.jsonl
```

```bash
# Export as Parquet
dj-process --config config.yaml --export_path ./outputs/result.parquet
```

```bash
# Export with sharding (256MB per shard)
dj-process --config config.yaml --export_shard_size 268435456
```

```bash
# Keep stats in output
dj-process --config config.yaml --keep_stats_in_res_ds true
```

--------------------------------

### Install domain-specific extras

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/tutorial/Installation.md

Install extras for computer vision, natural language processing, and audio processing.

```bash
# Computer Vision
uv pip install "py-data-juicer[vision]"

# Natural Language Processing
uv pip install "py-data-juicer[nlp]"

# Audio Processing
uv pip install "py-data-juicer[audio]"
```

--------------------------------

### Call init_configs function via GET (curl)

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/DJ_service.md

Curl command equivalent to the Python example for calling init_configs. Uses -G and --data-urlencode to pass the args parameter.

```bash
curl -G "http://localhost:8000/data_juicer/config/init_configs" \
     --data-urlencode "args=--config" \
     --data-urlencode "args=./demos/process_simple/process.yaml"
```

--------------------------------

### Install requirements and launch the Juicer Playground

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/Juicer.md

Installs Python dependencies, sets the JUICER_BASE_URL to the local server, and runs app.py to start the interactive UI. The UI is accessible at http://localhost:7860.

```bash
pip install -r requirements.txt
export JUICER_BASE_URL=http://localhost:8000/v1
python app.py
# open http://localhost:7860
```

--------------------------------

### Install additional components extras

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/tutorial/Installation.md

Install extras for distributed computing and AI services/APIs.

```bash
# Distributed Computing
uv pip install "py-data-juicer[distributed]"

# AI Services & APIs
uv pip install "py-data-juicer[ai_services]"
```

--------------------------------

### Start streamable-http server locally

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/DJ_service.md

Command to start the Data-Juicer MCP server with streamable-http transport using local execution with uv.

```bash
uv run dj-mcp <MODE: recipe-flow/granular-ops> --transport streamable-http --port 8080
```

--------------------------------

### Install AgentScope

Source: https://github.com/datajuicer/data-juicer/blob/main/demos/api_service/react_data_filter_process.ipynb

Install AgentScope from the GitHub repository using pip. This is a prerequisite for using the notebook.

```bash
pip install agentscope@git+https://github.com/modelscope/agentscope.git
```

--------------------------------

### Install common combinations of extras

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/tutorial/Installation.md

Install common combinations of extras for text processing, vision processing, full pipeline, or everything except sandbox.

```bash
# Text processing
uv pip install "py-data-juicer[generic,nlp]"

# Vision processing
uv pip install "py-data-juicer[generic,vision]"

# Full processing pipeline
uv pip install "py-data-juicer[generic,nlp,vision,distributed]"

# Everything except sandbox
uv pip install "py-data-juicer[all]"
```

--------------------------------

### Install generic ML/DL extra

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/tutorial/Installation.md

Install the generic extra for core ML/DL capabilities (PyTorch, Transformers, VLLM, etc.).

```bash
# Generic ML/DL capabilities
uv pip install "py-data-juicer[generic]"
```

--------------------------------

### Install Megatron-LM and HELM on evaluation machine

Source: https://github.com/datajuicer/data-juicer/blob/main/thirdparty/LLM_ecosystems/README.md

Runs both setup scripts to install Megatron-LM and HELM. Required for the evaluation machine.

```shell
./setup_megatron.sh
./setup_helm.sh
```

--------------------------------

### Run label_studio_service.py with options

Source: https://github.com/datajuicer/data-juicer/blob/main/tools/humanops/README.md

Basic command to run the utility script with optional flags for port, test project creation, and pip installation.

```bash
python label_studio_service.py [--port PORT] [--create-test-project] [--use-pip]
```

--------------------------------

### Example evaluator.py invocation

Source: https://github.com/datajuicer/data-juicer/blob/main/tools/evaluator/README.md

Example command that uses HELM to evaluate a Megatron-LM checkpoint every 1000 iterations starting from iteration 2000, checking for new checkpoints every 10 minutes.

```shell
python evaluator.py --config <config_file> --begin-iteration 2000 --iteration-interval 1000 --check-interval 10
```

--------------------------------

### Run demo with Python script

Source: https://github.com/datajuicer/data-juicer/blob/main/demos/ego_hand_action_annotation/README.md

Navigate to the demo directory and execute the main pipeline script. This is the recommended way to run the demo.

```bash
cd demos/ego_hand_action_annotation
python vla_pipeline.py
```

--------------------------------

### Configure application logging with setup_logger

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/JobManagement.md

Use `setup_logger` from `data_juicer.utils.logger_utils` to configure log output. It writes separate files for each log level; rotation and retention are handled via application logging integration.

```python
from data_juicer.utils.logger_utils import setup_logger

setup_logger(
    save_dir="./outputs",
    filename="log.txt",
    level="INFO",
    redirect=False
)
```

--------------------------------

### Add dependencies to pyproject.toml

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/DeveloperGuide.md

Example of adding core dependencies to `[project.dependencies]` and optional dependencies to `[project.optional-dependencies]` under a group like `generic`. The lazy loading system will handle installation when first used.

```toml
[project.dependencies]
# Core dependencies
numpy = ">=1.26.4,<2.0.0"

[project.optional-dependencies]
generic = [
    "torch==2.6.0",
    "transformers>=4.47.0",
    ...
]
```

--------------------------------

### Resume token output from ray_partitioned job

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/PartitionAndCheckpoint.md

Example output printed when checkpointing is enabled, showing the resume token format and how to use it.

```text
Resume token: 20260805_115141_81270d. Rerun the original command with
--resume 20260805_115141_81270d to resume this job.
```

--------------------------------

### Auto-install operator dependencies: dj-install --config my-recipe.yaml

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/ProcessData.md

Scans source files for the operators in the recipe and preinstalls the dependencies it identifies. See Installation guide for scan scope and environment preparation.

```bash
dj-install --config my-recipe.yaml
```

--------------------------------

### Install uv using curl or pip

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/DeveloperGuide.md

Install uv, a fast Python package installer and resolver, using either curl or pip. After installation, verify with `uv --version`.

```bash
# Using curl
curl -LsSf https://astral.sh/uv/install.sh | sh

# Or using pip
pip install uv
```

--------------------------------

### View usage for a conversion tool (e.g., llava_to_dj.py)

Source: https://github.com/datajuicer/data-juicer/blob/main/tools/fmt_conversion/multimodal/README.md

Run this command to see the help and arguments for any conversion tool in the source_format_to_data_juicer_format directory. Replace the script name with the desired tool.

```shell
# e.g. llava_to_dj.py
python tools/fmt_conversion/multimodal/source_format_to_data_juicer_format/llava_to_dj.py --help
```

--------------------------------

### Verify Data-Juicer installation

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/tutorial/Installation.md

Imports the data_juicer package and prints its version to confirm the installation. If a version is printed, the installation is ready for use.

```python
import data_juicer as dj
print(dj.__version__)
```

--------------------------------

### Install package with specific dependency groups

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/DeveloperGuide.md

Install the package with specific dependency groups: generic, dev, or ai_services. Each command installs only the dependencies for that group.

```bash
uv pip install -e ".[generic]"      # Generic dependencies
uv pip install -e ".[dev]"          # Development tools
uv pip install -e ".[ai_services]"  # Services dependencies
```

--------------------------------

### VggtMapper initialization with all output options enabled

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/operators/mapper/vggt_mapper.md

Creates a VggtMapper instance with the specified model path and enables all output options: camera parameters, depth maps, point maps from projection and unprojection, and point tracks.

```python
VggtMapper(vggt_model_path='facebook/VGGT-1B', frame_num=2, duration=2, frame_dir=DATA_JUICER_ASSETS_CACHE, if_output_camera_parameters=True, if_output_depth_maps=True, if_output_point_maps_from_projection=True, if_output_point_maps_from_unprojection=True, if_output_point_tracks=True)
```

--------------------------------

### Declare the entry point in pyproject.toml

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/OperatorPlugins.md

Configures the plugin's pyproject.toml with project metadata and an entry point under the data_juicer.ops group, pointing to the package __init__.

```toml
[project]
name = "my-dj-ops"
version = "0.1.0"
dependencies = ["py-data-juicer"]

[project.entry-points."data_juicer.ops"]
my_dj_ops = "my_dj_ops"
```

--------------------------------

### Instantiate VideoTaggingFromAudioMapper

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/operators/mapper/video_tagging_from_audio_mapper.md

Creates a VideoTaggingFromAudioMapper with the pretrained AST model 'MIT/ast-finetuned-audioset-10-10-0.4593'.

```python
VideoTaggingFromAudioMapper('MIT/ast-finetuned-audioset-10-10-0.4593')
```

--------------------------------

### Install the plugin with pip

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/OperatorPlugins.md

Installs the plugin package in editable mode or from a package index.

```bash
pip install -e .        # or: pip install my-dj-ops
```

--------------------------------

### Start a Ray cluster

Source: https://github.com/datajuicer/data-juicer/blob/main/docs/Distributed.md

Start a Ray cluster head node and optionally connect other nodes to it.

```shell
# Start a cluster as the head node
ray start --head

# (Optional) Connect to the cluster on other nodes/machines.
ray start --address='{head_ip}:6379'
```