Swift

https://swift.readthedocs.io/en/latest
Admin
Swift is a user-friendly, efficient, and extensible library for LLM fine-tuning, offering...
Tokens:298,982
Snippets:2,349
Trust Score:8.3
Update:10 months ago
Show doc for...
Tokens:
Raw
### Install SWIFT from Source Code (pip)

Source: https://swift.readthedocs.io/en/latest/_sources/GetStarted/SWIFT-installation.md

Installs SWIFT by cloning the source repository and using pip's editable install option. This method allows for installing with full capabilities by appending '[all]' to the package name.

```shell
# pip install git+https://github.com/modelscope/ms-swift.git

# Full capabilities
# pip install "git+https://github.com/modelscope/ms-swift.git#egg=ms-swift[all]"

git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
pip install -e .

# Full capabilities
# pip install -e '.[all]'
```

--------------------------------

### Alternative ms-swift Setup using Docker Image

Source: https://swift.readthedocs.io/en/latest/Megatron-SWIFT/Quick-start

Provides alternative Docker images for setting up the ms-swift environment. These images come pre-configured with specific versions of Ubuntu, CUDA, PyTorch, vLLM, and modelscope, simplifying the setup process for users who prefer containerized environments.

```docker
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.10.3
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.10.3
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.10.3

```

--------------------------------

### Environment Setup with pip

Source: https://swift.readthedocs.io/en/latest/BestPractices/Qwen3-VL-Best-Practice

Installs the required libraries for Qwen3-VL, including transformers, qwen_vl_utils, and ms-swift. Optionally includes vLLM for inference.

```bash
pip install "transformers>=4.57" "qwen_vl_utils>=0.0.14"

pip install "ms-swift>=3.9.1"
# pip install "vllm>=0.11.0"  # If using the vLLM inference backend for inference
```

--------------------------------

### Example Inference Output of Qwen3-VL

Source: https://swift.readthedocs.io/en/latest/_sources/BestPractices/Qwen3-VL-Best-Practice.md

Demonstrates example query-response pairs from Qwen3-VL inference, showcasing its ability to process tasks like OCR on LaTeX and answering questions about images.

```text
--------------------------------------------------
[QUERY] Using LaTeX to perform OCR on the image.
[LABELS] 1 + \frac { 1 } { 1 ! } + \frac { 1 } { 2 ! } + \frac { 1 } { 3 ! } + \frac { 1 } { 4 ! }
[RESPONSE] 1 + \frac { 1 } { 1 ! } + \frac { 1 } { 2 ! } + \frac { 1 } { 3 ! } + \frac { 1 } { 4 ! }
--------------------------------------------------
[QUERY] What color suit is the man wearing while playing the saxophone on stage?
[LABELS] The man is wearing a black suit and white shirt while playing the saxophone on the red-floored stage.
[RESPONSE] The man is wearing a black suit while playing the saxophone on stage.
--------------------------------------------------
```

--------------------------------

### Swift CLI Deployment Command Example

Source: https://swift.readthedocs.io/en/latest/Instruction/Frequently-asked-questions

Shows a basic command to deploy a trained Swift model using the `swift deploy` command with adapter specification. It suggests referring to the documentation for Inference and Deployment for more details.

```bash
swift deploy --adapters xxx
```

--------------------------------

### Install SWIFT using Wheel Packages (pip)

Source: https://swift.readthedocs.io/en/latest/_sources/GetStarted/SWIFT-installation.md

Installs the SWIFT library using pip. Supports installation with evaluation capabilities or full features by specifying extra requirements like '[eval]' or '[all]'.

```shell
pip install 'ms-swift'
# For evaluation usage
pip install 'ms-swift[eval]' -U
# Full capabilities
pip install 'ms-swift[all]' -U
```

--------------------------------

### Start SFT Training for Qwen2.5-Omni using Command Line

Source: https://swift.readthedocs.io/en/latest/_sources/BestPractices/MLLM-Registration.md

This example demonstrates how to initiate supervised fine-tuning (SFT) for the Qwen2.5-Omni model directly from the command line using Swift's LLM tools. It showcases common arguments for model selection, dataset configuration, training type, and optimization settings, similar to the Python script.

```shell
# Example command line invocation (specific arguments may vary based on your setup)
# swift sft --model Qwen/Qwen2.5-Omni-7B --dataset AI-ModelScope/LaTeX_OCR#5000 --model_type my_qwen2_5_omni --template my_qwen2_5_omni --train_type lora --torch_dtype bfloat16 --attn_impl flash_attn --padding_free --num_train_epochs 1 --per_device_train_batch_size 16 --per_device_eval_batch_size 16 --learning_rate 1e-4 --lora_rank 8 --lora_alpha 32 --target_modules all-linear --freeze_vit --freeze_aligner --gradient_accumulation_steps 1 --eval_steps 50 --save_steps 50 --save_total_limit 2 --logging_steps 5 --max_length 2048 --output_dir output --warmup_ratio 0.05 --dataloader_num_workers 4 --dataset_num_proc 1

```

--------------------------------

### Multimodal Streaming Inference: Video Description with Swift

Source: https://swift.readthedocs.io/en/latest/_sources/BestPractices/Qwen3-VL-Best-Practice.md

This example illustrates using Swift for streaming inference to describe a video. The user prompts the model with '<video>describe the video.' and provides a URL to a video file. The model then streams a detailed textual description of the video's content, highlighting actions and the environment. This demonstrates Swift's ability to process and interpret video data.

```shell
<<< <video>describe the video.
Input a video path or URL <<< https://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/baby.mp4
This video captures a charming and adorable moment of a young child, likely a toddler, sitting on a bed and pretending to read a book. The child is wearing glasses, which adds a humorous and endearing touch to the scene — as if they're a little scholar or librarian.

Here's a breakdown of what unfolds:

- The child is seated cross-legged on a bed with a patterned quilt. Behind them, a crib and some household items are visible, suggesting a cozy bedroom setting.

- The child holds an open book and appears to be turning the pages with focused attention, mimicking the behavior of a real reader.

- At one point, the child looks up, smiles, or seems to react with delight — perhaps amused by something in the book or just enjoying the activity.

- The child's movements are gentle and deliberate, showing a sense of concentration and curiosity. They turn pages, sometimes with one hand, and occasionally lift the book slightly as if to examine it more closely.

- The video has a warm, candid feel — it's not staged, and the child's natural behavior makes it feel authentic and heartwarming.

Overall, this is a sweet, lighthearted video that showcases the innocence and imagination of early childhood. The child's engagement with the book, combined with their glasses and playful demeanor, creates a delightful and memorable scene.
--------------------------------------------------
```

--------------------------------

### Multi-GPU Training Setup

Source: https://swift.readthedocs.io/en/latest/BestPractices/Qwen3-Best-Practice

This comment indicates a configuration for multi-GPU training, specifying the number of processes per node and the total GPU memory. It also suggests an example dataset that can be used for the experiment.

```bash
# 4 * 60GB
# You can run the experiment by setting `--dataset AI-ModelScope/alpaca-gpt4-data-en`
```

--------------------------------

### Example of Swift Inference Deployment with vLLM and Logprobs

Source: https://swift.readthedocs.io/en/latest/Instruction/Frequently-asked-questions

Demonstrates how to deploy a Swift model using the vLLM backend and enable log probabilities. It highlights that logprobs need to be passed from the client side using `RequestConfig`.

```bash
RAY_memory_monitor_refresh_ms=0 CUDA_VISIBLE_DEVICES=1 nohup swift deploy --ckpt_dir /mnt/workspace/checkpoint_600 --infer_backend vllm --logprobs True --load_data_args false --host 0.0.0.0 --port 8000 &
```

--------------------------------

### SWIFT CLI Multimodal Inference Example (Video)

Source: https://swift.readthedocs.io/en/latest/Instruction/Inference-and-deployment

An example illustrating how to provide a video URL for multimodal inference using the SWIFT CLI. The CLI prompts for the video path after the initial query.

```bash
<<< <video>Describe this video.
Input a video path or URL <<< https://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/baby.mp4

```

--------------------------------

### Install ms-swift and Dependencies

Source: https://swift.readthedocs.io/en/latest/BestPractices/MLLM-Registration

Installs the ms-swift library and specific versions of transformers and qwen_omni_utils for compatibility. Ensures that future documentation updates do not cause incompatibilities.

```shell
pip install "ms-swift>=3.10.2,<3.11"

pip install "transformers==4.57.*" "qwen_omni_utils==0.0.8"

```

--------------------------------

### Configure VLMEvalKit for Multi-modal Evaluation

Source: https://swift.readthedocs.io/en/latest/Instruction/Frequently-asked-questions

This configuration example for VLMEvalKit demonstrates how to set up evaluation for multiple datasets, including MMVet and MMBench. It specifies parameters like data sources, evaluation limits, model configurations, and the judge model.

```python
from vlmeval.utils import TaskConfig

task_cfg_dict = TaskConfig(
    work_dir='outputs',
    eval_backend='VLMEvalKit',
    eval_config={
        'data': ['MMVet', 'DocVQA_VAL', 'MMBench_DEV_EN'],
        'limit': 20,
        'mode': 'all',
        'model': [
            {
              'api_base': 'http://127.0.0.1:8001/v1/chat/completions',
              'key': 'EMPTY',
              'name': 'CustomAPIModel',
              'temperature': 0.0,
              'type': 'Qwen2.5-VL-72B-Instruct-AWQ',
              'img_size': -1,
              'video_llm': False,
              'max_tokens': 1024,
            }
            ],
        'reuse': False,
        'nproc': 100,
        'judge': 'exact_matching'}
)

# Example of how to use the TaskConfig (actual evaluation call would be separate)
# For example, using vlmeval.run.run_task with this config
```

--------------------------------

### Install ms-swift from Source

Source: https://swift.readthedocs.io/en/latest/Instruction/Sample

Installs the ms-swift library from its source code repository. This involves cloning the repository and then performing a local installation. It's an alternative to pip installation.

```bash
git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
pip install -e '.[llm]'
```

--------------------------------

### Handling Multimodal Input with Swift Deployment

Source: https://swift.readthedocs.io/en/latest/Instruction/Frequently-asked-questions

This snippet covers inputting videos during local deployment of qwen2vl models using base64 encoding and provides an example of calling video with curl. It also shows how to pass PIL.Image from the client during multimodal model deployment.

```bash
# Example curl command for video input (conceptual)
curl -X POST ... -d '{"video": "base64_encoded_video_data"}'
```

--------------------------------

### SWIFT CLI Multimodal Inference Example (Image)

Source: https://swift.readthedocs.io/en/latest/Instruction/Inference-and-deployment

An example demonstrating how to input image paths or URLs for multimodal inference in SWIFT. The CLI prompts for input after the initial query.

```bash
<<< <image><image>What is the difference between these two images?
Input an image path or URL <<< http://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/cat.png
Input an image path or URL <<< http://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/animal.png

```

--------------------------------

### Installing necessary Python packages for Qwen3 training - Bash

Source: https://swift.readthedocs.io/en/latest/BestPractices/Qwen3-Best-Practice

This script installs the core ms-swift library along with transformers and other essential packages for training Qwen3 models. It includes deepspeed for multi-GPU training, liger-kernel for GPU memory optimization, and flash-attn for efficient attention mechanisms. Ensure these dependencies are installed before proceeding with training.

```bash
pip install ms-swift -U
pip install transformers

pip install deepspeed # for multi-GPU training
pip install liger-kernel # to save GPU memory resources
pip install flash-attn --no-build-isolation  # required for packing
```

--------------------------------

### Install ms-swift Dependencies with Pip

Source: https://swift.readthedocs.io/en/latest/Megatron-SWIFT/Quick-start

Installs core dependencies required for ms-swift, including pybind11, transformer_engine, apex, and megatron-core. The transformer_engine installation can be customized for specific versions or from source. Apex installation involves cloning the repository and building it locally. Megatron-core is installed via a git clone. Note that flash-attn installation requires selecting a compatible version.

```bash
pip install pybind11

# transformer_engine
# If an installation error occurs, you can refer to this issue for resolution: https://github.com/modelscope/ms-swift/issues/3793
pip install --no-build-isolation transformer_engine[pytorch]
# Or install using the following command
# pip install --no-build-isolation git+https://github.com/NVIDIA/TransformerEngine.git@release_v2.5#egg=transformer_engine[pytorch]

# apex
# Note: Megatron-SWIFT can run in environments without apex by setting `--no_gradient_accumulation_fusion true`.
git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./

# megatron-core
pip install git+https://github.com/NVIDIA/Megatron-LM.git@core_r0.14.0

# If you are using multi-node training, please additionally set the `MODELSCOPE_CACHE` environment variable to a shared storage path.
# This will ensure that the dataset cache is shared, thereby speeding up preprocessing.
# Note: This step is crucial; otherwise multi-machine training may hang due to data inconsistencies caused by randomness in data preprocessing.
export MODELSCOPE_CACHE='/xxx/shared'

# Megatron-LM
# The training module in the dependent library Megatron-LM will be cloned and installed by swift via `git clone`. Alternatively, you can use the environment variable `MEGATRON_LM_PATH` to point to the path of an already downloaded repository (in offline environments, use the [core_r0.14.0 branch](https://github.com/NVIDIA/Megatron-LM/tree/core_r0.14.0)).
git clone --branch core_r0.14.0 https://github.com/NVIDIA/Megatron-LM.git
export MEGATRON_LM_PATH='/xxx/Megatron-LM'

# flash_attn
# Choose an appropriate version to install: https://github.com/Dao-AILab/flash-attention/releases/tag/v2.8.1
# Note: Do not install a version higher than the maximum supported by transformer_engine: https://github.com/NVIDIA/TransformerEngine/blob/release_v2.6/transformer_engine/pytorch/attention/dot_product_attention/utils.py#L109
MAX_JOBS=8 pip install "flash-attn==2.7.4.post1" --no-build-isolation

```

--------------------------------

### Swift YAML Configuration Example

Source: https://swift.readthedocs.io/en/latest/Instruction/Command-line-parameters

An example demonstrating how to configure Swift project arguments using a YAML file. This allows for a more organized and readable way to specify training and model configurations, such as model name, dataset, output directory, and gradient checkpointing.

```yaml
swift sft --config demo.yaml

# Content of demo.yaml
# Model args
model: Qwen/Qwen2.5-7B-Instruct
dataset: swift/self-cognition
...

# Train args
output_dir: xxx/xxx
gradient_checkpointing: true

...
```

--------------------------------

### Install GPTQ Quantization Dependencies

Source: https://swift.readthedocs.io/en/latest/_sources/Instruction/Export-and-push.md

Installs 'auto_gptq' and 'optimum' for GPTQ quantization. Compatibility with CUDA versions should be verified via the official AutoGPTQ installation guide. Use '--no-deps' to avoid PyTorch dependency conflicts.

```shell
pip install auto_gptq optimum -U
```

--------------------------------

### Python Code Example for Template Usage

Source: https://swift.readthedocs.io/en/latest/_sources/Instruction/Agent-support.md

Example Python code demonstrating how to load a model tokenizer, get an agent template, and encode data for training.

```APIDOC
## Python Template Usage

### Description
This Python code snippet shows how to use the Swift library to get a model tokenizer, retrieve a specified agent template (e.g., 'hermes'), set the training mode, and encode data.

### Method
Library function calls for LLM template management and data encoding.

### Endpoint
N/A (Python code example)

### Parameters
- **model_name** (string) - Required - The name or path of the pre-trained model.
- **agent_template** (string) - Required - The name of the agent template to use (e.g., 'hermes', 'react_en').
- **data** (dict) - Required - The data structure to be encoded.

### Request Example
```python
from swift.llm import get_model_tokenizer, get_template

# Load model and tokenizer (load_model=False to only get tokenizer)
_, tokenizer = get_model_tokenizer('ZhipuAI/GLM-4-9B-0414', load_model=False)

# Get the specified agent template
template = get_template(tokenizer.model_meta.template, tokenizer, agent_template='hermes')

# Data to be encoded
data = {...} # Replace with your actual data structure

# Set the template mode to 'train'
template.set_mode('train')

# Encode the data
encoded = template.encode(data)

# Print the encoded input_ids and labels
print(f'[INPUT_IDS] {template.safe_decode(encoded["input_ids"])}\n')
print(f'[LABELS] {template.safe_decode(encoded["labels"])}')
```

### Response
#### Success Response (200)
Prints the encoded `input_ids` and `labels` to the console.

#### Response Example
```text
[INPUT_IDS] <|im_start|>user\nWhat is the weather like in Beijing and Shanghai today?<|im_end|>\n<|im_start|>assistant\nAction: realtime_aqi\nAction Input: {\'city\': \'Beijing\'}\nAction: realtime_aqi\nAction Input: {\'city\': \'Shanghai\'}\nObservation:{'city': 'Beijing', 'aqi': '10', 'unit': 'celsius'}\nObservation:{'city': 'Shanghai', 'aqi': '72', 'unit': 'fahrenheit'}\nAccording to the weather forecast tool, the air quality index (AQI) in Beijing is 10, which indicates good air quality; whereas in Shanghai, the AQI is 72, indicating mild pollution.<|im_end|>

[LABELS] [-100 * 233]Action: realtime_aqi\nAction Input: {\'city\': \'Beijing\'}\nAction: realtime_aqi\nAction Input: {\'city\': \'Shanghai\'}\nObservation:[-100 * 45]According to the weather forecast tool, the air quality index (AQI) in Beijing is 10, which indicates good air quality; whereas in Shanghai, the AQI is 72, indicating mild pollution.<|im_end|>
```
```

--------------------------------

### SWIFT Evaluation Setup

Source: https://swift.readthedocs.io/en/latest/Instruction/Evaluation

Installs the MS-SWIFT library with evaluation dependencies. Supports installation via pip or from source. The `[eval]` extra ensures all necessary packages for evaluation are included.

```bash
pip install ms-swift[eval] -U
```

```bash
git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
pip install -e '.[eval]'
```

--------------------------------

### Video Description Example (Qwen3-VL-4B-Instruct)

Source: https://swift.readthedocs.io/en/latest/BestPractices/Qwen3-VL-Best-Practice

This example illustrates using the Qwen3-VL-4B-Instruct model to analyze and describe a video provided via a URL. The input consists of a prompt like '<video>describe the video.' and the video URL. The model generates a narrative description of the video content, detailing actions, setting, and overall mood.

```text
<<< <video>describe the video.
Input a video path or URL <<< https://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/baby.mp4
This video captures a charming and adorable moment of a young child, likely a toddler, sitting on a bed and pretending to read a book. The child is wearing glasses, which adds a humorous and endearing touch to the scene — as if they're a little scholar or librarian.

Here's a breakdown of what unfolds:

- The child is seated cross-legged on a bed with a patterned quilt. Behind them, a crib and some household items are visible, suggesting a cozy bedroom setting.

- The child holds an open book and appears to be turning the pages with focused attention, mimicking the behavior of a real reader.

- At one point, the child looks up, smiles, or seems to react with delight — perhaps amused by something in the book or just enjoying the activity.

- The child's movements are gentle and deliberate, showing a sense of concentration and curiosity. They turn pages, sometimes with one hand, and occasionally lift the book slightly as if to examine it more closely.

- The video has a warm, candid feel — it's not staged, and the child's natural behavior makes it feel authentic and heartwarming.

Overall, this is a sweet, lighthearted video that showcases the innocence and imagination of early childhood. The child's engagement with the book, combined with their glasses and playful demeanor, creates a delightful and memorable scene.
```
```

--------------------------------

### UNSLOTH Training Configuration Example

Source: https://swift.readthedocs.io/en/latest/_sources/Instruction/Command-line-parameters.md

Example command-line arguments for using UNSLOTH, a library for efficient model training. It shows how to specify the tuner backend, training type, and quantization bits.

```bash
--tuner_backend unsloth
--train_type full/lora
--quant_bits 4
```

--------------------------------

### Start Fine-tuning Qwen2.5-7B-Instruct with Megatron

Source: https://swift.readthedocs.io/en/latest/_sources/Megatron-SWIFT/Quick-start.md

Initiates the fine-tuning process for the Qwen2.5-7B-Instruct model using Megatron-SWIFT. Requires significant GPU memory (2*80GiB). Supports multi-machine training and various training configurations. Uses `megatron sft` command.

```shell
PYTORCH_CUDA_ALLOC_CONF='expandable_segments:True' \
NPROC_PER_NODE=2 \
CUDA_VISIBLE_DEVICES=0,1 \
megatron sft \
    --load Qwen2.5-7B-Instruct-mcore \
    --dataset 'AI-ModelScope/alpaca-gpt4-data-zh#500' \
              'AI-ModelScope/alpaca-gpt4-data-en#500' \
              'swift/self-cognition#500' \
    --tensor_model_parallel_size 2 \
    --sequence_parallel true \
    --micro_batch_size 16 \
    --global_batch_size 16 \
    --recompute_granularity full \
    --recompute_method uniform \
    --recompute_num_layers 1 \
    --finetune true \
    --cross_entropy_loss_fusion true \
    --lr 1e-5 \
    --lr_warmup_fraction 0.05 \
    --min_lr 1e-6 \
    --max_epochs 1 \
    --save megatron_output/Qwen2.5-7B-Instruct \
    --save_interval 100 \
    --max_length 2048 \
    --system 'You are a helpful assistant.' \
    --num_workers 4 \
    --no_save_optim true \
    --no_save_rng true \
    --dataset_num_proc 4 \
    --model_author swift \
    --model_name swift-robot
```

--------------------------------

### Image Description Example (Qwen3-VL-4B-Instruct)

Source: https://swift.readthedocs.io/en/latest/BestPractices/Qwen3-VL-Best-Practice

This demonstrates how to use the Qwen3-VL-4B-Instruct model to describe an image provided via a URL. The input includes a prompt with '<image>describe the image.' and the image URL. The model then returns a detailed, descriptive analysis of the image content, focusing on visual elements and overall impression.

```text
<<< <image>describe the image.
Input an image path or URL <<< http://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/cat.png
This is a beautifully detailed, close-up portrait of an adorable tabby kitten, rendered with a soft, painterly effect that gives it a gentle, dreamy quality.

Here's a breakdown of the image:

- **The Kitten:** The subject is a young, fluffy kitten with a classic tabby pattern. Its fur is a mix of white and soft grayish-brown stripes, with a prominent dark stripe running down the center of its forehead and over its nose. The kitten's face is predominantly white, with delicate markings around its eyes and cheeks.

- **The Eyes:** Its most captivating feature is its large, round, and expressive eyes. They are a striking shade of bright blue-gray, with dark pupils that give it an intense, curious, and slightly innocent gaze. The eyes are wide open, suggesting the kitten is alert and attentive.

- **The Expression:** The kitten's expression is sweet and innocent. Its small pink nose and slightly parted mouth give it a gentle, almost pleading look. Its whiskers are long and white, standing out against its fur.

- **The Style:** The image has a soft-focus, artistic quality, reminiscent of impressionist painting. The edges of the kitten's fur are slightly blurred, creating a halo effect that draws attention to its face. The background is softly blurred with muted tones of green and gray, which helps the kitten stand out as the clear focal point.

- **Overall Impression:** The image evokes feelings of warmth, cuteness, and tenderness. The kitten appears to be looking directly at the viewer, creating a sense of connection and affection.

This is a lovely and charming depiction of a young kitten, capturing its innocence and charm in a visually appealing and emotionally engaging way.
--------------------------------------------------
```

--------------------------------

### Specify Judge Model in Swift Eval

Source: https://swift.readthedocs.io/en/latest/Instruction/Frequently-asked-questions

This example shows how to pass judge model parameters to Swift's evaluation process using `extra_eval_args`. The parameters like `api_key`, `api_url`, and `model_id` should be provided as a JSON string.

```json
{
  "judge-model-args": {
    "api_key": "YOUR_API_KEY",
    "api_url": "YOUR_API_URL",
    "model_id": "YOUR_JUDGE_MODEL_ID"
  }
}
```

--------------------------------

### Set Result Path for Swift Infer Command

Source: https://swift.readthedocs.io/en/latest/Instruction/Frequently-asked-questions

This example shows how to specify a custom directory to save evaluation results when using the `swift infer` command. The `--result_path` parameter allows users to control the output location.

```shell
swift infer --result_path your_path
```

--------------------------------

### Server Deployment (Multimodal Models)

Source: https://swift.readthedocs.io/en/latest/_sources/Instruction/Inference-and-deployment.md

Instructions for deploying a multimodal LLM using Swift with VLLM backend.

```APIDOC
## Server Deployment (Multimodal Models)

This section details the deployment of multimodal models using Swift with specific configurations for VLLM.

### Server Side Command

```bash
# test env: pip install transformers==4.51.3 vllm==0.8.5.post1
CUDA_VISIBLE_DEVICES=0 \
MAX_PIXELS=1003520 \
VIDEO_MAX_PIXELS=50176 \
FPS_MAX_FRAMES=12 \
swift deploy \
    --model Qwen/Qwen2.5-VL-3B-Instruct \
    --infer_backend vllm \
    --vllm_gpu_memory_utilization 0.9 \
    --vllm_max_model_len 8192 \
    --max_new_tokens 2048 \
    --vllm_limit_mm_per_prompt '{"image": 5, "video": 2}' \
    --served_model_name Qwen2.5-VL-3B-Instruct
```
```

--------------------------------

### Text Generation Example (Qwen3-VL-4B-Instruct)

Source: https://swift.readthedocs.io/en/latest/BestPractices/Qwen3-VL-Best-Practice

An example of text generation using the Qwen3-VL-4B-Instruct model via the command line. The user provides a prompt enclosed in <<<, and the model responds with a detailed explanation of its identity and capabilities. This showcases the model's conversational and informative abilities.

```text
<<< who are you?
Hello! I'm Qwen, a large-scale language model independently developed by the Tongyi Lab under Alibaba Group. My main functions include answering questions, creating text such as stories, official documents, emails, scripts, and more, as well as performing logical reasoning, programming, and other tasks. If you have any questions or need assistance, feel free to let me know anytime, and I'll do my best to help!
--------------------------------------------------
```

--------------------------------

### SWIFT Historical Docker Image Mirrors

Source: https://swift.readthedocs.io/en/latest/_sources/GetStarted/SWIFT-installation.md

Lists historical Docker image mirrors for older SWIFT versions, including variations in CUDA, Python, PyTorch, vLLM, and modelscope versions. These are available across different Alibaba Cloud regions.

```dockerfile
# swift3.7.2
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.0-modelscope1.28.2-swift3.7.2
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.0-modelscope1.28.2-swift3.7.2
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.0-modelscope1.28.2-swift3.7.2

# swift3.6.4
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.28.1-swift3.6.4
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.28.1-swift3.6.4
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.28.1-swift3.6.4

# swift3.5.3
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.27.1-swift3.5.3
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.27.1-swift3.5.3
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py310-torch2.6.0-vllm0.8.5.post1-modelscope1.27.1-swift3.5.3

# swift3.4.1.post1
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.5.post1-modelscope1.26.0-swift3.4.1.post1
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.5.post1-modelscope1.26.0-swift3.4.1.post1
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.5.post1-modelscope1.26.0-swift3.4.1.post1

# swift3.3.0.post1
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.3-modelscope1.25.0-swift3.3.0.post1
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-vllm0.8.3-modelscope1.25.0-swift3.3.0.post1

# swift3.2.2
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.5.1-modelscope1.25.0-swift3.2.2
```

--------------------------------

### SWIFT Docker Image Mirrors

Source: https://swift.readthedocs.io/en/latest/_sources/GetStarted/SWIFT-installation.md

Provides various Docker image registry URLs for different SWIFT versions, including specific CUDA, Python, PyTorch, and vLLM versions. These mirrors are hosted across different Alibaba Cloud regions.

```dockerfile
# swift3.10.3
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.10.3
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.10.3
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.10.3

# swift3.9.3
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.9.3
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.9.3
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.8.1-py311-torch2.8.0-vllm0.11.0-modelscope1.31.0-swift3.9.3

# swift3.8.3
modelscope-registry.cn-hangzhou.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.1.1-modelscope1.29.2-swift3.8.3
modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.1.1-modelscope1.29.2-swift3.8.3
modelscope-registry.us-west-1.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.6.3-py311-torch2.7.1-vllm0.10.1.1-modelscope1.29.2-swift3.8.3
```

--------------------------------

### Reinforcement Learning Input Example (Python)

Source: https://swift.readthedocs.io/en/latest/Instruction/GRPO/DeveloperGuide/gym_env

Provides an example of a `RolloutInferRequest` object, demonstrating the structure for messages, images, audios, videos, tools, and objects used in reinforcement learning inference.

```python
infer_request
"""
RolloutInferRequest(
    messages=[
        {'role': 'system', 'content': 'A conversation between User and Assistant. The user asks a question, and the Assistant solves it. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e., <think> reasoning process here </think><answer> answer here </answer>\n'}, {'role': 'user', 'content': 'What is the value of $\sqrt{36 \times \sqrt{16}}$?'},
        {'role': 'assistant', 'content': 'To find the value of \(\sqrt{36 \times \sqrt{16}}\), we will break down the problem step-by-step.\n\nFirst, we need to evaluate the inner square root:\n\[
\sqrt{16}
\]
We know that:
\[
4^2 = 16 \implies \sqrt{16} = 4
\]

Next, we substitute this result back into the original expression:
\[
\sqrt{36 \times \sqrt{16}} = \sqrt{36 \times 4}
\]

Now, we need to evaluate the product inside the square root:
\[
36 \times 4 = 144
\]

So, the expression simplifies to:
\[
\sqrt{144}
\]

Finally, we determine the square root of 144:
\[
\sqrt{144} = 12
\]

Thus, the value of \(\sqrt{36 \times \sqrt{16}}\) is:
\[
\boxed{12}
\]'}
    ],
    images=[],
    audios=[],
    videos=[],
    tools=None,
    objects={},
    data_dict={
        'problem': 'What is the value of $\sqrt{36 \times \sqrt{16}}$?',
        'solution': "To solve the problem, we need to evaluate the expression \(\sqrt{36 \times \sqrt{16}}\).

We can break down the steps as follows:

1. Evaluate the inner square root: \(\sqrt{16}\).
2. Multiply the result by 36.
3. Take the square root of the product obtained in step 2.

Let's compute this step by step using Python code for accuracy.
```python
import math

# Step 1: Evaluate the inner square root
inner_sqrt = math.sqrt(16)

# Step 2: Multiply the result by 36
product = 36 * inner_sqrt

# Step 3: Take the square root of the product
final_result = math.sqrt(product)
print(final_result)
```
```output
12.0
```
The value of \(\sqrt{36 \times \sqrt{16}}\) is /\(\boxed{12}\)."
        }
    )
"""

```

--------------------------------

### Registering External Plugins - Example

Source: https://swift.readthedocs.io/en/latest/_sources/Instruction/Command-line-parameters.md

Provides an example of how to register external Python files containing plugin definitions. These files are imported into the plugin module, allowing for custom functionality.

```bash
external_plugins: ['path/to/your/plugin.py']
```

--------------------------------

### Start SWIFT Web UI for Training and Inference

Source: https://swift.readthedocs.io/en/latest/GetStarted/Web-UI

This command launches the SWIFT Web UI, which provides an interface for managing training and inference tasks. It supports specifying the language for the UI. The UI acts as a wrapper for command-line operations.

```bash
swift web-ui --lang zh
# or en
swift web-ui --lang en
```
Tokens:
Raw