### Run Example Script (Trainer)

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/basic_modules.md

Execute a training script using the trainer utility. This is a standard way to start training with VeOmni.

```bash
bash train.sh tasks/train_text.py configs/text/qwen2_5.yaml
```

--------------------------------

### Install Video I/O Backend

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/wan2.1_I2V_1.3B.md

Installs the necessary libraries for video input/output operations during inference.

```shell
pip install imageio imageio-ffmpeg
```

--------------------------------

### Install Dependencies and Build Docs

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/README.md

Installs project dependencies using pip and then builds the documentation using make. Ensure requirements.txt is present.

```bash
pip install -r requirements.txt

make clean
make html
```

--------------------------------

### Install Dependencies and Generate Dockerfiles

Source: https://github.com/bytedance-seed/veomni/blob/main/docker/README.md

Steps to set up the project virtual environment and then run the Dockerfile generation script.

```bash
uv sync                       # one-time, sets up .venv
.venv/bin/python docker/generate.py
```

--------------------------------

### Build DataLoader with Custom Configurations

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/basic_modules.md

Example of building a DataLoader with various configurations for training, including batch sizes, sequence length, dynamic batching, and worker settings. Ensure all necessary arguments are provided based on your training setup.

```python
from veomni.data import build_dataloader
train_dataloader = build_dataloader(
    dataloader_type=args.data.dataloader.type,
    dataset=train_dataset,
    micro_batch_size=args.train.micro_batch_size, # micro batch size
    global_batch_size=args.train.global_batch_size, # global batch size
    dataloader_batch_size=args.train.dataloader_batch_size, # dataloader batch size, how many micro batches to get with next(train_dataloader), automatically calculate
    max_seq_len=args.data.max_seq_len, # max sequence length
    train_steps=args.train.train_steps, # train steps, calculate by args.train.compute_train_steps
    dyn_bsz=args.train.dyn_bsz, # enable dynamic batching
    bsz_warmup_ratio=args.train.bsz_warmup_ratio, # bsz warmup ratio
    bsz_warmup_init_mbtoken=args.train.bsz_warmup_init_mbtoken, # bsz warmup init micro batch token
    dyn_bsz_buffer_size=args.train.dyn_bsz_buffer_size, # dynamic batching buffer size
    num_workers=args.data.dataloader.num_workers, # dataloader num workers
    drop_last=args.data.dataloader.drop_last,  # dataloader drop last
    pin_memory=args.data.dataloader.pin_memory,  # dataloader pin memory
    prefetch_factor=args.data.dataloader.prefetch_factor, # dataloader prefetch factor
    seed=args.train.seed, # random seed
    build_collate_fn=True,
    collate_fn_kwargs=collate_fn_kwargs, # kwargs for collate_fn
)
```

--------------------------------

### Full Kernel Configuration Example

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/design/kernel_selection.md

This example demonstrates how to set various kernel implementations for different model operations. It shows how to enable optimized kernels like flash_attention_2, fused_triton, and liger_kernel for specific components, and how to disable them (e.g., for MLP) by setting them to 'eager'.

```yaml
model:
  ops_implementation:
    attn_implementation: flash_attention_2
    moe_implementation: fused_triton
    cross_entropy_loss_implementation: liger_kernel
    rms_norm_implementation: liger_kernel
    swiglu_mlp_implementation: eager           # disable Liger for MLP only
    rotary_pos_emb_implementation: liger_kernel
    load_balancing_loss_implementation: triton
```

--------------------------------

### Start Training Qwen3-8B

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3.md

Initiates training for the Qwen3-8B model using a specified configuration, dataset, and FSDP2 mode.

```bash
bash train.sh tasks/train_text.py configs/text/qwen3.yaml \
    --model.model_path ./Qwen3-8B \
    --data.train_path ./tulu-first2000.parquet \
    --train.accelerator.fsdp_config.fsdp_mode fsdp2 \
    --train.init_device meta
```

--------------------------------

### Clone and Install VeOmni with uv (ARM)

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install_ascend_arm.md

Clone the VeOmni repository and use uv for installation with NPU support. Activate the virtual environment afterwards.

```bash
git clone https://github.com/ByteDance-Seed/VeOmni.git
cd VeOmni

# use the frozen uv env
uv sync --frozen  --extra npu_aarch64
source .venv/bin/activate
```

--------------------------------

### Example `video_total_pixels` Calculation (128K tokens)

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/multimodal_data_processing.md

An example calculation of `video_total_pixels` for inference-scale contexts of 128K tokens.

```default
128000 * 784 * 0.9 ≈ 90,316,800
```

--------------------------------

### Install Dependencies with uv

Source: https://github.com/bytedance-seed/veomni/blob/main/AGENTS.md

Installs project dependencies including extras for GPU support and development. Activates the virtual environment.

```bash
uv sync --extra gpu --dev
source .venv/bin/activate
```

--------------------------------

### Clone and Install torchcodec for Video/Audio Processing

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install_ascend_arm.md

Clone the torchcodec repository, checkout a compatible version, copy the installation script, and run it to install torchcodec with Ascend support.

```bash
# Clone the torchcodec repository
cd ..
git clone https://github.com/meta-pytorch/torchcodec.git
cd torchcodec

# Checkout to a specific version for compatibility
git checkout v0.5.0

# Copy the installation script to the torchcodec source directory
cp ../VeOmni/docs/get_started/installation/install_torchcodec_Ascend.sh .

# Note: Ensure Python is installed as a shared library (required for compiling C++ extensions)
# The installation script will automatically verify this requirement

# Run the installation script (replace with your actual CANN path)
bash install_torchcodec_Ascend.sh $CANN_path/ascend-toolkit/set_env.sh
```

--------------------------------

### Start Training on NPU

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/wan2.1.md

Initiates the training process on an NPU, specifying the NPU as the initialization device. This command uses the same task and configuration as GPU training.

```shell
bash train.sh tasks/deprecated_task/train_wan.py configs/dit/wan_sft.yaml \
    --model.model_path Wan2.1-I2V-14B-480P-Diffusers/transformer \
    --train.init_device npu
```

--------------------------------

### Install mindstudio-probe Tool

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/hardware_support/precision_analysis.md

Install the mindstudio-probe library using pip. This is a prerequisite for enabling deterministic computation.

```bash
pip install mindstudio-probe
```

--------------------------------

### Start Training Qwen3-VL-8B

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3_vl.md

Initiate training for the Qwen3-VL-8B model using the specified configuration and dataset. Adjust micro-batch size as needed.

```shell
bash train.sh tasks/train_vlm.py configs/multimodal/qwen3_vl/qwen3_vl_dense.yaml \
    --model.model_path ./Qwen3-VL-8B-Instruct \
    --data.train_path ./sharegpt4v_instruct_gpt4-vision_cap100k_coco.json \
    --data.dataloader.type native \
    --data.datasets_type iterable \
    --data.source_name sharegpt4v_sft \
    --data.dataloader.num_workers 8 \
    --train.micro_batch_size 3
```

--------------------------------

### Clone and install torchcodec for Ascend

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install_ascend_x86.md

Clones the torchcodec repository, checks out a compatible version, copies the installation script, and runs it to install torchcodec with Ascend NPU support. Ensure Python is installed as a shared library.

```bash
# Clone the torchcodec repository
cd ..
git clone https://github.com/meta-pytorch/torchcodec.git
cd torchcodec

# Checkout to a specific version for compatibility
git checkout v0.5.0

# Copy the installation script to the torchcodec source directory
cp ../VeOmni/docs/get_started/installation/install_torchcodec_Ascend.sh .

# Note: Ensure Python is installed as a shared library (required for compiling C++ extensions)
# The installation script will automatically verify this requirement

# Run the installation script (replace with your actual CANN path)
bash install_torchcodec_Ascend.sh $CANN_path/ascend-toolkit/set_env.sh

# Verify installation
pip show torchcodec

# Test torchcodec import
python -c "from torchcodec.decoders import VideoDecoder; print('Success')"
```

--------------------------------

### Install VeOmni with pip and GPU extras

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install.md

Use pip for installation. This command installs VeOmni with the 'gpu' extra, which includes necessary dependencies for Nvidia GPU support.

```bash
git clone https://github.com/ByteDance-Seed/VeOmni.git
cd VeOmni

pip3 install -e .[gpu]
```

--------------------------------

### Start Training Qwen3-VL-30B

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3_vl.md

Initiate training for the Qwen3-VL-30B model using the specified configuration and dataset. Adjust micro-batch size as needed.

```shell
bash train.sh tasks/train_vlm.py configs/multimodal/qwen3_vl/qwen3_vl_moe.yaml \
    --model.model_path ./Qwen3-VL-30B-A3B-Instruct \
    --data.train_path ./sharegpt4v_instruct_gpt4-vision_cap100k_coco.json \
    --data.dataloader.type native \
    --data.datasets_type iterable \
    --data.source_name sharegpt4v_sft \
    --data.dataloader.num_workers 8 \
    --train.micro_batch_size 2
```

--------------------------------

### Start Training on GPU

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/wan2.1.md

Initiates the training process on a GPU using the specified task and configuration files. The model path should point to the downloaded transformer component.

```shell
bash train.sh tasks/deprecated_task/train_wan.py configs/dit/wan_sft.yaml \
    --model.model_path Wan2.1-I2V-14B-480P-Diffusers/transformer
```

--------------------------------

### Run Example Script (Deprecated Task)

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/basic_modules.md

Execute a training script for a deprecated task using plain Python. Ensure the dataset is downloaded beforehand.

```bash
bash train.sh tasks/deprecated_task/train_torch.py configs/text/qwen2_5.yaml
```

--------------------------------

### Clone and Install VeOmni with pip (ARM)

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install_ascend_arm.md

Clone the VeOmni repository and install it using pip with NPU support. Also install specific versions of transformers and datasets.

```bash
git clone https://github.com/ByteDance-Seed/VeOmni.git
cd VeOmni

pip install -e .[npu_aarch64]
pip install transformers==5.9.0
pip install datasets==2.21.0
```

--------------------------------

### Install Dependencies

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3_5.md

Installs necessary dependencies, including transformers v5, using uv sync with GPU support.

```shell
uv sync --extra gpu
```

--------------------------------

### Qwen3_5MoeRMSNorm Example (No Dispatch)

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/design/unified_kernel_registry.md

This example shows a module, Qwen3_5MoeRMSNorm, where no Veomni dispatch is needed as there is no non-eager kernel available. The original Hugging Face code is emitted verbatim.

```python
# In generated/patched_modeling_qwen3_5_moe_gpu.py
# Qwen3.5 MoE uses qwen3_5 variant; no non-eager kernel exists.
# Original HF code emitted verbatim. No slot, no guard.

class Qwen3_5MoeRMSNorm(nn.Module):
    def __init__(self, dim: int, eps: float = 1e-6):
        # ... identical to upstream ...

```

--------------------------------

### Start Ascend Docker Container with Basic Device Access

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/hardware_support/AscendDockerUsage/build_a2_docker.md

This command starts a Docker container with access to Ascend devices. It uses a wildcard to include all available Ascend cards. Ensure the necessary Ascend driver libraries and tools are mounted as read-only volumes.

```bash
docker run --runtime=runc -it \
  --ulimit nproc=65535 \
  --ulimit nofile=65535 \
  --device=/dev/davinci* \
  --device=/dev/davinci_manager \
  --device=/dev/devmm_svm \
  --device=/dev/hisi_hdc \
  -v /usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64:ro \
  -v /usr/local/Ascend/driver/tools:/usr/local/Ascend/driver/tools:ro \
  -v /usr/local/Ascend/add-ons:/usr/local/Ascend/add-ons:ro \
  --name ascend-a2-container \
  ascend-a2-env:v1 \
  /bin/bash
```

--------------------------------

### Install Dependencies and Generate Dockerfiles (Alternative)

Source: https://github.com/bytedance-seed/veomni/blob/main/docker/README.md

Alternative method to install required packages directly and run the generation script without a project virtual environment.

```bash
pip install jinja2 pyyaml
python docker/generate.py
```

--------------------------------

### Start Qwen3.5 MoE 35B VL Training

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3_5.md

Initiates multimodal (VL) training for the Qwen3.5-35B-A3B MoE model. Training is configured for a maximum of 20 steps.

```shell
bash train.sh tasks/train_vlm.py configs/multimodal/qwen3_5_moe/qwen3_5_moe_vl.yaml \
    --model.model_path ./Qwen/Qwen3.5-35B-A3B \
    --data.train_path ./configs/multimodal/data/tulu_sharegpt4v_llavavideo.yaml \
    --train.max_steps 20
```

--------------------------------

### Video Processing Configuration Example

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/multimodal_data_processing.md

Example configuration for video processing, specifying maximum frames and target frames per second.

```yaml
data:
  mm_configs:
    video_max_pixels: 602112  # 28 * 28 * 768
    max_frames: 16
    fps: 2.0
```

--------------------------------

### Install PEFT Library

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/key_features/lora.md

Install the PEFT library, which is required for LoRA functionality. Use the `gpu` or `npu` extra depending on your hardware.

```shell
pip install peft==0.18.1
```

--------------------------------

### Start DPO Training

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3_dpo.md

Initiates the DPO training process using a specified training script and configuration file. Key configuration parameters are listed separately.

```bash
bash train.sh tasks/train_text_dpo.py configs/text/qwen3_dpo.yaml
```

--------------------------------

### Install VeOmni with uv and NPU dependencies

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install_ascend_x86.md

Use uv for faster installation of VeOmni with NPU-specific dependencies. This command synchronizes locked dependencies and activates the virtual environment.

```bash
git clone https://github.com/ByteDance-Seed/VeOmni.git
cd VeOmni

# use the locked uv env
uv sync --locked --extra npu
source .venv/bin/activate
```

--------------------------------

### Start Training Qwen3-30B

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3.md

Initiates training for the Qwen3-30B model with fused Triton MOE implementation, FSDP2 mode, and a global batch size of 16.

```bash
bash train.sh tasks/train_text.py configs/text/qwen3.yaml \
    --model.model_path ./Qwen3-30B-A3B-Instruct-2507 \
    --model.ops_implementation.moe_implementation fused_triton \
    --data.train_path ./tulu-first2000.parquet \
    --train.accelerator.fsdp_config.fsdp_mode fsdp2 \
    --train.init_device meta \
    --train.global_batch_size 16
```

--------------------------------

### Custom Model Implementation Example

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/key_features/model_loader.md

Implement a custom model by inheriting from PreTrainedModel and using the custom configuration. This example shows basic structure and registration.

```python
# veomni/models/transformers/your_custom_model.py

from transformers import PreTrainedModel, PretrainedConfig

class YourCustomConfig(PretrainedConfig):
    model_type = "your_custom_model"
    architectures = ["YourCustomModel"]

class YourCustomModel(PreTrainedModel):
    config_class = YourCustomConfig

    def __init__(self, config):
        super().__init__(config)
        # Initialize your model components

    def forward(self, input_ids, **kwargs):
        ...

# Register your model
ModelClass = YourCustomModel
```

--------------------------------

### Configuration Example for Multimodal Data

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/multimodal_data_processing.md

A YAML configuration snippet demonstrating how to set parameters for multimodal data processing, including `video_total_pixels`.

```yaml
data:
  max_seq_len: 4096
  mm_configs:
    image_max_pixels: 602112
    video_max_pixels: 602112
    video_total_pixels: 2889523  # dynamic per-frame budget
    max_frames: 16
    fps: 2.0
    use_audio_in_video: false
```

--------------------------------

### Creating and Adding a Custom Callback

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/trainer.md

Demonstrates how to define a custom callback by inheriting from `Callback` and registering it with the trainer. This example prints a message every 100 steps.

```python
from veomni.trainer.callbacks import Callback

class MyCustomCallback(Callback):
    def on_step_end(self, state, **kwargs):
        if state.global_step % 100 == 0:
            print(f"Step {state.global_step}: Custom action executed.")

# In your trainer
trainer.add_callback(MyCustomCallback(trainer))
```

--------------------------------

### Install Compatible Transformers Version

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/hardware_support/FAQ.md

Ensure compatibility with VeOmni by installing a specific version of the Transformers library. This example shows how to check the current version and install version 5.9.0 using uv or pip.

```bash
# Check current Transformers version
python -c "import transformers; print(transformers.__version__)"
```

```bash
# Install using uv
uv sync --locked --extra npu --group dev
```

```bash
# Or install using pip
pip install transformers==5.9.0
```

--------------------------------

### Multimodal Data Transform Setup

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/basic_modules.md

Example of setting up a multimodal data transform function, integrating a processor, chat template, and position ID function. This is typically used within a VLM trainer setup.

```python
from veomni.models import build_processor
from veomni.data import build_multimodal_chat_template
processor = build_processor(args.model.tokenizer_path)
chat_template = build_multimodal_chat_template(args.data.chat_template, processor.tokenizer)
position_id_func = model.get_position_id_func()
transform = partial(
    process_function,
    processor=processor,
    chat_template=chat_template,
    position_id_func=position_id_func,
    **args.data.mm_configs,
)
```

--------------------------------

### Start Qwen3-VL Training on Ascend NPU

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/hardware_support/typical_usage.md

Execute the training script with specified paths for the model and dataset, along with data loader configurations optimized for iterable datasets.

```bash
bash train.sh tasks/train_vlm.py configs/multimodal/qwen3_vl/qwen3_vl_dense.yaml \
    --model.model_path ./Qwen3-VL-8B-Instruct \
    --data.train_path ./sharegpt4v_instruct_gpt4-vision_cap100k_coco.json \
    --data.dataloader.type native \
    --data.datasets_type iterable \
    --data.source_name sharegpt4v_sft
```

--------------------------------

### Configure Multi-Node Training Environment Variables

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/hardware_support/FAQ.md

Modify environment variables in train.sh for multi-node training. This example shows settings for a 2-node setup, including NNODES, NODE_RANK, MASTER_ADDR, MASTER_PORT, and NPROC_PER_NODE.

```bash
# Number of nodes (2 nodes in this example)
NNODES=${NNODES:=2}
# Current node rank (0 to 1 for 2 nodes - must be different for each machine)
NODE_RANK=${NODE_RANK:=0}
# Master node address (IP address - must be the same across all machines)
MASTER_ADDR=${MASTER_ADDR:=192.168.1.100}
# Master node port (default works for most cases)
MASTER_PORT=${MASTER_PORT:=12345}
# Number of NPUs per node (A2: max 8, A3: max 16)
NPROC_PER_NODE=${NPROC_PER_NODE:=8}
```

--------------------------------

### Install ffmpeg on macOS

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install.md

Required for video/audio processing when installing VeOmni. Install ffmpeg using the Homebrew package manager.

```bash
# macOS
brew install ffmpeg
```

--------------------------------

### Install ffmpeg on Ubuntu/Debian

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install.md

Required for video/audio processing when installing VeOmni. Install ffmpeg using the apt-get package manager.

```bash
# Ubuntu/Debian
sudo apt-get install ffmpeg
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/bytedance-seed/veomni/blob/main/CONTRIBUTING.md

Install the project dependencies, including development tools, using pip. The `.[dev]` flag installs optional dependencies for development.

```bash
pip install -e ".[dev]"
```

--------------------------------

### Install VeOmni with uv and GPU extras

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install.md

Use uv for faster installation. This command installs VeOmni with all GPU-related dependencies, including CUDA 13.0 support.

```bash
git clone https://github.com/ByteDance-Seed/VeOmni.git
cd VeOmni

uv sync --locked --extra gpu
source .venv/bin/activate
```

--------------------------------

### Run Online Training with Parquet Videos

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/wan2.1_I2V_1.3B.md

Use this command to initiate an online training task, passing raw Parquet videos directly. Ensure all specified paths and configurations are correctly set.

```shell
NPROC_PER_NODE=4 bash train.sh tasks/train_dit.py configs/dit/wan2.1_I2V_1.3B_lora.yaml \
    --model.model_path           ./Wan2.1-T2V-1.3B-Diffusers/transformer \
    --model.condition_model_path ./Wan2.1-T2V-1.3B-Diffusers \
    --data.train_path            ./Tom-and-Jerry-VideoGeneration-Dataset-parquet \
    --data.source_name           Tom-and-Jerry-VideoGeneration-Dataset \
    --data.mm_configs.fps        24 \
    --data.mm_configs.max_frames 81 \
    --train.training_task        online_training \
    --train.global_batch_size    4 \
    --train.micro_batch_size     1 \
    --train.accelerator.ulysses_size 1 \
    --train.checkpoint.output_dir ./exp/Wan2.1-T2V-1.3B-Diffusers_lora \
    --train.checkpoint.save_hf_weights true \
    --train.num_train_epochs 30
```

--------------------------------

### Install Patchgen

Source: https://github.com/bytedance-seed/veomni/blob/main/patchgen-pkg/README.md

Install the patchgen package using pip.

```bash
pip install patchgen
```

--------------------------------

### Download Qwen3-VL-8B-Instruct Model Weights

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/hardware_support/typical_usage.md

Use a Python script to download the pre-trained Qwen3-VL-8B-Instruct model weights from Hugging Face.

```bash
python3 scripts/download_hf_model.py \
    --repo_id Qwen/Qwen3-VL-8B-Instruct \
    --local_dir ./Qwen3-VL-8B-Instruct
```

--------------------------------

### Example YAML Structure for Training Arguments

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/arguments.md

Illustrates the nested structure of training arguments, including settings for Weights & Biases, accelerator configuration (like FSDP), initialization device, and checkpointing.

```yaml
train:
  wandb:
    enable: true
    project: VeOmni
  accelerator:
    fsdp_config:
      fsdp_mode: fsdp2
  init_device: meta
  checkpoint:
    manager: dcp
```

--------------------------------

### Verify torchcodec Installation

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install_ascend_arm.md

Verify the torchcodec installation by checking its version and attempting to import VideoDecoder.

```python
# Verify installation
pip show torchcodec

# Test torchcodec import
python -c "from torchcodec.decoders import VideoDecoder; print('Success')"
```

--------------------------------

### SFT Data Transform Setup

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/basic_modules.md

Sets up the data transformation for supervised fine-tuning (SFT) tasks, utilizing a pre-built chat template. Requires importing `process_sft_example` and building a chat template.

```python
from veomni.data.chat_template import build_chat_template

chat_template = build_chat_template(args.data.chat_template, tokenizer)
transform = partial(
    process_sft_example,
    chat_template=chat_template,
    max_seq_len=args.data.max_seq_len,
    text_keys=args.data.text_keys,
)
```

--------------------------------

### Install ffmpeg on Ubuntu/Debian/openEuler

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install_ascend_x86.md

Installs the ffmpeg package, which is required for video/audio processing capabilities in VeOmni.

```bash
# Ubuntu/Debian/openEuler
sudo apt-get install ffmpeg
# or
sudo yum install ffmpeg
```

--------------------------------

### Example `video_total_pixels` Calculation (max_seq_len=4096)

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/multimodal_data_processing.md

An example calculation of `video_total_pixels` for a training sequence length of 4096.

```default
4096 * 28^2 * 0.9 = 4096 * 784 * 0.9 ≈ 2,889,523
```

--------------------------------

### Start Qwen3.5-9B VL Training

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3_5.md

Initiates multimodal (VL) training for the Qwen3.5-9B model using a specified configuration and dataset. Training is set for a maximum of 20 steps.

```shell
bash train.sh tasks/train_vlm.py configs/multimodal/qwen3_5/qwen3_5_vl.yaml \
    --model.model_path ./Qwen/Qwen3.5-9B \
    --data.train_path ./configs/multimodal/data/tulu_sharegpt4v_llavavideo.yaml \
    --train.max_steps 20
```

--------------------------------

### Start Qwen3 Omni MoE Training

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3_omni_moe.md

This command initiates the training process for Qwen3 Omni MoE using the specified task script and configuration file. It also sets the model path for the training.

```shell
bash train.sh tasks/train_vlm.py configs/multimodal/qwen3_omni/qwen3_omni.yaml \
    --model.model_path Qwen3-Omni-30B-A3B-Instruct
```

--------------------------------

### Download Qwen3-Omni-MoE Model

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3_omni_moe.md

This command downloads the Qwen3-Omni-30B-A3B-Instruct model from Hugging Face to the local directory. Ensure you have the necessary permissions and disk space.

```shell
python3 scripts/download_hf_model.py \
    --repo_id Qwen/Qwen3-Omni-30B-A3B-Instruct \
    --local_dir .
```

--------------------------------

### Using Preprocessors in Training Scripts

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/key_features/preprocessor_registry.md

Import `build_multisource_dataset` and pass your configuration. The preprocessor is automatically registered and available if the config specifies the correct `source_name`.

```python
from veomni.data import build_multisource_dataset

# The preprocessor is automatically registered and available
# as long as the config specifies the correct `source_name`.
dataset = build_multisource_dataset(config)
```

--------------------------------

### Install VeOmni with pip and NPU dependencies

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/get_started/installation/install_ascend_x86.md

Installs VeOmni using pip with the NPU extra, followed by specific versions of transformers and datasets.

```bash
git clone https://github.com/ByteDance-Seed/VeOmni.git
cd VeOmni

pip install -e .[npu]
pip install transformers==5.9.0
pip install datasets==2.21.0
```

--------------------------------

### Enable Ulysses SP with 8 GPUs

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/examples/qwen3_5.md

Example command to train Qwen3.5 with Ulysses sequence parallelism using 8 GPUs, distributed as data_parallel_size=4 and ulysses_parallel_size=2. Ensure flash_attention_3 is used.

```shell
bash train.sh tasks/train_text.py configs/text/qwen3_5_sft.yaml \
    --model.model_path ${HOME}/Qwen3.5-9B \
    --data.train_path ${HOME}/tulu-first2000.parquet \
    --train.data_parallel_size 4 \
    --train.ulysses_parallel_size 2 \
    --train.attn_implementation flash_attention_3
```

--------------------------------

### Start Basic Ascend Docker Container

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/hardware_support/AscendDockerUsage/build_a3_docker.md

Run a Docker container with Ascend device access. This command uses a wildcard to include all Ascend cards, but individual devices can be specified.

```bash
docker run --runtime=runc -it \
  --ulimit nproc=65535 \
  --ulimit nofile=65535 \
  --device=/dev/davinci* \
  --device=/dev/davinci_manager \
  --device=/dev/devmm_svm \
  --device=/dev/hisi_hdc \
  -v /usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64:ro \
  -v /usr/local/Ascend/driver/tools:/usr/local/Ascend/driver/tools:ro \
  -v /usr/local/Ascend/add-ons:/usr/local/Ascend/add-ons:ro \
  --name ascend-a3-container \
  ascend-a3-env:v1 \
  /bin/bash
```

--------------------------------

### Pretrain Data Transform Setup

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/basic_modules.md

Configures the data transformation pipeline for pretraining tasks using a specified tokenizer and sequence length. Requires importing `process_pretrain_example` and building a tokenizer.

```python
from functools import partial
from veomni.data.data_transform import process_pretrain_example
from veomni.models import build_tokenizer

tokenizer = build_tokenizer(args.model.tokenizer_path)
# Can replace with the following code if you want to use the AutoTokenizer from transformers.
# tokenizer = AutoTokenizer.from_pretrained(args.model.tokenizer_path)

transform = partial(
    process_pretrain_example,
    tokenizer=tokenizer,
    max_seq_len=args.data.max_seq_len,
    text_keys=args.data.text_keys,
)
```

--------------------------------

### Example Diff Output for Qwen3RMSNorm

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/design/patchgen.md

This is an example of the unified diff output generated when using the --diff flag, showing modifications made to the Qwen3RMSNorm class for performance optimization.

```diff
-class Qwen3RMSNorm(nn.Module):
-    def forward(self, hidden_states):
-        input_dtype = hidden_states.dtype
-        hidden_states = hidden_states.to(torch.float32)
-        ...
+# ======================================================================
+# [PATCHED CLASS] Qwen3RMSNorm
+# Reason: Use fused RMSNorm kernel for better performance
+# ======================================================================
+class Qwen3RMSNorm(nn.Module):
+    def forward(self, hidden_states):
+        # Optimized implementation with comments preserved
+        ...
```

--------------------------------

### Create Model Directory Structure

Source: https://github.com/bytedance-seed/veomni/blob/main/docs/usage/support_new_models/dit_model_guide.md

Sets up the necessary directories and __init__.py files for a new DiT model within the VeoMNI framework.

```bash
mkdir -p veomni/models/diffusers/your_dit/your_condition/
mkdir -p veomni/models/diffusers/your_dit/your_transformer/

touch veomni/models/diffusers/your_dit/__init__.py
touch veomni/models/diffusers/your_dit/your_condition/__init__.py
touch veomni/models/diffusers/your_dit/your_condition/configuration_your_condition.py
touch veomni/models/diffusers/your_dit/your_condition/modeling_your_condition.py
touch veomni/models/diffusers/your_dit/your_transformer/__init__.py
touch veomni/models/diffusers/your_dit/your_transformer/configuration_your_transformer.py
touch veomni/models/diffusers/your_dit/your_transformer/modeling_your_transformer.py
```