### Compile llama.cpp

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/convert-model.md

Steps to clone the llama.cpp repository, set up a virtual environment, and compile the project with example support and native optimizations.

```bash
git clone https://github.com/ggerganov/llama.cpp --depth 1
cd llama.cpp
mkdir build && cd build
cmake .. -DLLAMA_BUILD_EXAMPLES=ON -DLLAMA_NATIVE=ON
cmake --build . --config Release

cd..
python3 -m venv venv
source venv/bin/activate
pip install -r "./requirements/requirements-convert_hf_to_gguf.txt"
```

--------------------------------

### Install Unsloth for Optimized Fine-tuning

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-model.md

Installs Unsloth, a library designed to speed up fine-tuning, by downloading and executing an installation script. It provides an example pip command for manual installation.

```bash
wget -qO- https://raw.githubusercontent.com/unslothai/unsloth/main/unsloth/_auto_install.py | python -

# Example manual installation command:
pip install --upgrade pip && pip install "unsloth[cu126-ampere-torch270] @ git+https://github.com/unslothai/unsloth.git"
```

--------------------------------

### Install Dependencies

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

Installs all project dependencies listed in the 'requirements.txt' file using pip.

```bash
pip install -r requirements.txt
```

--------------------------------

### Fine-tuning Example

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

An example command to fine-tune the `qwen3-8b-base` model on an RTX 4090. This demonstrates the usage of various parameters for a typical fine-tuning process.

```bash
python3 run_finetune.py --output_dir /root/autodl-fs/qwen3-8b-qing-v4 --local_dir qwen3-8b-base --data_path ./training_data_ruozhi.jsonl --eval_data_path ./training_data_ruozhi_eval.jsonl --use_qlora true --lora_dropout 0.05 --num_train_epochs 8 --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --gradient_accumulation_steps 8 --learning_rate 2e-5 --lr_scheduler cosine --logging_steps 5 --eval_steps 40 --save_steps 200 --warmup_ratio 0.05 --dataloader_num_workers 16 --fp16 true --use_unsloth true --no-gradient_checkpointing --dataloader_prefetch_factor 4
```

--------------------------------

### Install Dependencies

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-model.md

Installs the project's dependencies from the 'requirements.txt' file using pip. This ensures all necessary libraries are available.

```bash
pip install -r requirements.txt
```

--------------------------------

### vLLM Installation and Setup

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/clean-data.md

Installs vLLM and its dependencies in a virtual environment. This is for Linux environments and requires a compatible GPU. It's recommended for faster LLM processing compared to LM Studio, but consumes more VRAM and has a shorter context window.

```bash
sudo apt update
sudo apt install python3.10-venv git -y

python3 -m venv vllm_env
source vllm_env/bin/activate

pip install -U pip
pip install torch --index-url https://download.pytorch.org/whl/cu121  # 如果你用CUDA
pip install vllm
```

--------------------------------

### vLLM Installation and Setup

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/clean-data.md

Installs necessary packages for vLLM, including Python virtual environment, Git, PyTorch with CUDA support, and vLLM itself. This process is intended for Linux environments.

```bash
sudo apt update
sudo apt install python3.10-venv git -y

python3 -m venv vllm_env
source vllm_env/bin/activate

pip install -U pip
pip install torch --index-url https://download.pytorch.org/whl/cu121  # If you use CUDA
pip install vllm
```

--------------------------------

### Compile llama.cpp

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/convert-model.md

Steps to clone the llama.cpp repository, set up a virtual environment, install dependencies, and compile the project with example support and native optimizations.

```bash
git clone https://github.com/ggerganov/llama.cpp --depth 1
cd llama.cpp
mkdir build && cd build
cmake .. -DLLAMA_BUILD_EXAMPLES=ON -DLLAMA_NATIVE=ON
cmake --build . --config Release

cd..
python3 -m venv venv
source venv/bin/activate
pip install -r "./requirements/requirements-convert_hf_to_gguf.txt"
```

--------------------------------

### Fine-tuning Command Example

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-model.md

This bash command demonstrates how to run the fine-tuning script for the qwen3-8b-base model on a 4090 GPU. It includes various parameters for output directory, data paths, QLoRA configuration, training epochs, batch sizes, learning rate, logging, evaluation, saving, warmup ratio, dataloader settings, mixed precision, and gradient checkpointing.

```bash
python3 run_finetune.py --output_dir /root/autodl-fs/qwen3-8b-qing-v4 --local_dir qwen3-8b-base --data_path ./training_data_ruozhi.jsonl --eval_data_path ./training_data_ruozhi_eval.jsonl --use_qlora true --lora_dropout 0.05 --num_train_epochs 8 --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --gradient_accumulation_steps 8 --learning_rate 2e-5 --lr_scheduler cosine --logging_steps 5 --eval_steps 40 --save_steps 200 --warmup_ratio 0.05 --dataloader_num_workers 16 --fp16 true --use_unsloth true --no-gradient_checkpointing --dataloader_prefetch_factor 4
```

--------------------------------

### Install Unsloth for Optimized Fine-tuning

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

Installs the Unsloth library, which is recommended for optimized fine-tuning. It involves downloading and running an installation script, then executing a pip command to install the specific Unsloth version with CUDA support.

```bash
wget -qO- https://raw.githubusercontent.com/unslothai/unsloth/main/unsloth/_auto_install.py | python -
```

```bash
pip install --upgrade pip && pip install "unsloth[cu126-ampere-torch270] @ git+https://github.com/unslothai/unsloth.git"
```

--------------------------------

### Install Dependencies from requirements_oss.txt

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-openai-oss-model.md

Installs project dependencies listed in the requirements_oss.txt file. This step is crucial after setting up the Unsloth environment.

```bash
pip install -r requirements_oss.txt
```

--------------------------------

### Download Model from HuggingFace

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/prepare-model.md

Installs the HuggingFace CLI and downloads a specified model to a local directory. It also includes instructions for setting a mirror site.

```bash
pip install huggingface-hub
export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download unsloth/gpt-oss-20b-BF16 --local-dir gpt-oss-20b
```

--------------------------------

### Clone Repository

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-model.md

Clones the Qing Digital Self repository from GitHub. This is the first step to get the project code.

```bash
git clone https://github.com/qqqqqf-q/Qing-Digital-Self.git --depth 1
```

--------------------------------

### Run llama.cpp Model Test

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/convert-model.md

Executes a GGUF model using the llama-run binary. Accepts the path to the GGUF model file as an argument.

```bash
./build/bin/llama-run <GGUF_model_path>
```

```bash
./build/bin/llama-run /root/autodl-fs/qwen3-8b-fp16-agent.gguf
```

--------------------------------

### Usage Examples

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/media-chat-data.md

Demonstrates how to use the Video to ChatML Converter. It covers interactive mode, direct command-line usage, and using the main program script with various parameters.

```bash
python start-vtc.py -i
python start-vtc.py video.mp4 -u 0 -a 1 -o output.json -m base
python video-to-chatml.py video.mp4 -u 0 -a 1 -o output.json -m base
```

--------------------------------

### Example Fine-tuning Run

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-openai-oss-model.md

An example bash script demonstrating a fine-tuning run for the `gpt-oss-20b-unsloth-bnb-4bit` model. It includes various parameters for output directory, data paths, LoRA configuration, training epochs, batch sizes, learning rate, and quantization settings.

```bash
python3 run_finetune_oss.py \
  --output_dir /root/autodl-fs/gpt-oss-20b \
  --local_dir /root/autodl-tmp/gpt-oss-20b \
  --data_path ./harmony_small.txt \
  --eval_data_path ./harmony_small_eval.txt \
  --use_qlora true \
  --lora_dropout 0.05 \
  --num_train_epochs 2 \
  --per_device_train_batch_size 4 \
  --per_device_eval_batch_size 4 \
  --gradient_accumulation_steps 8 \
  --learning_rate 2e-5 \
  --lr_scheduler cosine \
  --logging_steps 5 \
  --eval_steps 40 \
  --save_steps 200 \
  --warmup_ratio 0.05 \
  --dataloader_num_workers 16 \
  --fp16 true \
  --use_unsloth true \
  --dataloader_prefetch_factor 4 \
  --load_precision int4 \
  --data_format harmony \
  --save_gguf true \
  --gguf_quantization f16
```

--------------------------------

### Start Fine-tuning Process

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-openai-oss-model.md

Initiates the fine-tuning process for OSS models using a Python script. This command requires Python 3 to be installed.

```bash
python3 run_finetune_oss.py
```

--------------------------------

### Install Dependencies

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/media-chat-data.md

Installs all necessary Python dependencies for the Video to ChatML Converter. It can be run automatically or manually by installing from a requirements file.

```bash
python start-vtc.py --install
pip install -r requirements.txt
```

--------------------------------

### Download Model from ModelScope

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/prepare-model.md

Installs the ModelScope library and downloads a specified model to a local directory. This is an optional step for model preparation.

```bash
pip install modelscope
modelscope download --model Qwen/Qwen3-14B --local_dir ./qwen3-14b
```

--------------------------------

### Download Model from HuggingFace

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/prepare-model.md

This snippet demonstrates downloading a model from HuggingFace using the `huggingface-cli`. It first requires installing the `huggingface-hub` library. An optional step to set an environment variable for using a mirror site is also provided.

```bash
pip install huggingface-hub
export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download unsloth/gpt-oss-20b-BF16 --local-dir gpt-oss-20b
```

--------------------------------

### Install huggingface-cli

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-openai-oss-model.md

Installs the huggingface-cli tool, which is required for downloading models from Hugging Face repositories. This is a prerequisite for the model download command.

```bash
pip install huggingface-hub
```

--------------------------------

### High-speed File Download using lftp

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/convert-model.md

Utilizes lftp with SFTP to download files from a server using multiple threads for increased speed. Supports resuming interrupted downloads and overwriting existing files.

```bash
lftp -u {username},{password} -p {port} sftp://{server_address} -e "set xfer:clobber true; pget -n {thread_count} {server_file_path} -o {local_file_name/path}; bye"
```

```bash
lftp -u root,askdjiwhakjd -p 27391 sftp://yourserver.com -e "set xfer:clobber true; pget -n 256 /root/autodl-fs/qwen3-8b-fp16-agent.gguf -o qwen3-8b-fp16-agent.gguf; bye"
```

--------------------------------

### Run Fine-tuning Script

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

Executes the main fine-tuning script for the project. Default parameters are used if none are specified.

```python
python run_finetune.py
```

--------------------------------

### Run vLLM OpenAI API Server

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/clean-data.md

Starts the vLLM OpenAI-compatible API server. Key parameters include the model path, GPU memory utilization, maximum model length, number of sequences, batched tokens, and data type. This command is crucial for enabling LLM-based data cleaning with vLLM.

```bash
python3 -m vllm.entrypoints.openai.api_server --model /home/vllm/qwen3-4b-int8 --gpu-memory-utilization 0.7 --max-model-len 10240 --max-num-seqs 4 --max-num-batched-tokens 2048 --dtype auto
```

--------------------------------

### Create and Activate Virtual Environment

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-model.md

Creates a Python virtual environment named 'venv' and provides commands to activate it on Linux/Mac and Windows systems.

```bash
python3 -m venv venv

# On Linux/Mac:
source venv/bin/activate

# On Windows:
.\venv\Scripts\activate
```

--------------------------------

### Quantize GGUF Model

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/convert-model.md

Reduces the precision of a GGUF model to save space and potentially improve inference speed. Requires specifying input and output GGUF paths and the desired quantization level.

```bash
./build/bin/llama-quantize <input_GGUF_path> <output_GGUF_path> <quantization_level>
```

```bash
./build/bin/llama-quantize \
  /root/autodl-fs/qwen3-8b-fp16-agent.gguf \
  /root/autodl-fs/qwen3-8b-q8_0-agent.gguf \
  Q8_0
```

--------------------------------

### Install Unsloth and Dependencies

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-openai-oss-model.md

Installs the Unsloth library and its dependencies, including specific versions of transformers and triton, for fine-tuning OSS models. This command should be run before installing other dependencies from requirements_oss.txt.

```bash
pip install "unsloth_zoo[base] @ git+https://github.com/unslothai/unsloth-zoo" "unsloth[base] @ git+https://github.com/unslothai/unsloth" torchvision bitsandbytes git+https://github.com/huggingface/transformers git+https://github.com/triton-lang/triton.git@main#subdirectory=python/triton_kernels
```

--------------------------------

### Create and Activate Virtual Environment

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

Creates a Python virtual environment named 'venv' and provides commands to activate it on Linux/Mac and Windows systems.

```bash
python3 -m venv venv
```

```bash
source venv/bin/activate
```

```bash
.\venv\Scripts\activate
```

--------------------------------

### Run Fine-tuning Script

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-model.md

Executes the main fine-tuning script 'run_finetune.py'. This script performs the model fine-tuning process, with default parameters available.

```python
python run_finetune.py
```

--------------------------------

### QLoRA Fine-tuning Workflow Example

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/index.md

Demonstrates the process of fine-tuning a Qwen3-8B model using QLoRA with FP8 precision on a 4090 GPU. This workflow involves data preparation and the actual fine-tuning script.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
import torch

# Load model and tokenizer
model_name = "Qwen/Qwen3-8B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    load_in_8bit=True,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Prepare model for k-bit training
model = prepare_model_for_kbit_training(model)

# Configure LoRA
lora_config = LoraConfig(
    r=8,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

# --- Training data preparation and training loop would follow ---
```

```shell
# Example command to run the fine-tuning script
# python run_finetune.py --data_path /path/to/your/chat_data.json --output_dir /path/to/save/model
```

--------------------------------

### Download Model from ModelScope

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/prepare-model.md

This snippet shows how to download a model from ModelScope using the `modelscope` CLI. It requires the `modelscope` library to be installed first. The command specifies the model name and the local directory for saving.

```bash
pip install modelscope
modelscope download --model Qwen/Qwen3-14B --local_dir ./qwen3-14b
```

--------------------------------

### Run llama.cpp Model Test

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/convert-model.md

Command to run a test with a GGUF model using llama.cpp. Requires the path to the GGUF model file.

```bash
./build/bin/llama-run <GGUF模型路径>

Parameters:
  <GGUF模型路径>: Path to the GGUF model file (original or quantized).

Example:
./build/bin/llama-run /root/autodl-fs/qwen3-8b-fp16-agent.gguf
```

--------------------------------

### High-Speed File Download using lftp

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/convert-model.md

Command to download files at high speed from a server using lftp with SFTP and multi-threaded parallel download (pget).

```bash
lftp -u {用户名},{密码} -p {端口} sftp://{服务器地址} -e "set xfer:clobber true;  pget -n {线程数} {服务器文件路径} -o {本地文件名/路径}: bye"

Parameters:
  {用户名}: Username for SFTP authentication.
  {密码}: Password for SFTP authentication.
  {端口}: Port for SFTP connection.
  {服务器地址}: The address of the SFTP server.
  {线程数}: Number of threads for parallel download (recommended 64+, up to 256).
  {服务器文件路径}: The path to the file on the server.
  {本地文件名/路径}: The local path and filename to save the downloaded file.

Example:
lftp -u root,askdjiwhakjd -p 27391 sftp://yourserver.com -e "set xfer:clobber true; pget -n 256 /root/autodl-fs/qwen3-8b-fp16-agent.gguf -o qwen3-8b-fp16-agent.gguf; bye"
```

--------------------------------

### Fine-tuning Parameters

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

This section details the various command-line arguments available for fine-tuning models. These parameters control aspects such as model saving, learning rate, data loading, and optimization.

```APIDOC
save_steps: Model save interval (steps), default: '200'
save_total_limit: Maximum number of saved models, default: '2'
warmup_ratio: Learning rate warmup ratio, default: '0.05'
lr_scheduler_type: Learning rate scheduler type, default: 'cosine'
resume_from_checkpoint: Path to resume training from checkpoint, default: None
no-gradient_checkpointing: Disable gradient checkpointing (enable by adding this flag)
no-merge_and_save: Do not merge and save model (enable by adding this flag)
fp16: Whether to use fp16, default: 'true'
optim: Optimizer name, default: 'adamw_torch_fused'
dataloader_pin_memory: Whether to pin DataLoader memory, default: 'false'
dataloader_num_workers: Number of DataLoader workers, default: '0'
dataloader_prefetch_factor: DataLoader prefetch factor, default: '2'
use_flash_attention_2: Use FlashAttention2 (not effective for Unsloth), default: 'false'
```

--------------------------------

### Clone Repository

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

Clones the Qing Digital Self repository from GitHub. It uses --depth 1 to only fetch the latest commit, reducing download size.

```bash
git clone https://github.com/qqqqqf-q/Qing-Digital-Self.git --depth 1
```

--------------------------------

### Model Configuration Parameters

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

This section outlines the key parameters for configuring and fine-tuning models. It includes settings for specifying the model repository, local storage, training data, and various hyperparameters for techniques like LoRA and MoE.

```APIDOC
Parameter Name: `--repo_id`
Type: str
Default Value: `'Qwen/Qwen3-30B-A3B-Instruct-2507'`
Description: Hugging Face repository ID for the base model.

Parameter Name: `--local_dir`
Type: str
Default Value: `'qwen3-30b-a3b-instruct'`
Description: Local directory to store the model files.

Parameter Name: `--use_unsloth`
Type: str
Default Value: `'false'`
Description: Flag to indicate whether to use the Unsloth optimization library.

Parameter Name: `--use_qlora`
Type: str
Default Value: `'true'`
Description: Flag to indicate whether to use QLoRA for fine-tuning.

Parameter Name: `--data_path`
Type: str
Default Value: `'training_data.jsonl'`
Description: Path to the training dataset file.

Parameter Name: `--eval_data_path`
Type: str
Default Value: `None`
Description: Path to the evaluation dataset file (optional).

Parameter Name: `--max_samples`
Type: str
Default Value: `None`
Description: Maximum number of samples to use for training (optional).

Parameter Name: `--max_eval_samples`
Type: str
Default Value: `None`
Description: Maximum number of samples to use for evaluation (optional).

Parameter Name: `--model_max_length`
Type: str
Default Value: `'2048'`
Description: Maximum sequence length the model can handle.

Parameter Name: `--output_dir`
Type: str
Default Value: `'finetune/models/qwen3-30b-a3b-qlora'`
Description: Directory where the fine-tuned model and checkpoints will be saved.

Parameter Name: `--seed`
Type: str
Default Value: `'42'`
Description: Random seed for reproducibility.

Parameter Name: `--per_device_train_batch_size`
Type: str
Default Value: `'1'`
Description: Batch size per GPU for training.

Parameter Name: `--per_device_eval_batch_size`
Type: str
Default Value: `'1'`
Description: Batch size per GPU for evaluation.

Parameter Name: `--gradient_accumulation_steps`
Type: str
Default Value: `'16'`
Description: Number of steps to accumulate gradients before updating weights.

Parameter Name: `--learning_rate`
Type: str
Default Value: `'2e-4'`
Description: The learning rate for the optimizer.

Parameter Name: `--num_train_epochs`
Type: str
Default Value: `'3'`
Description: Total number of training epochs.

Parameter Name: `--max_steps`
Type: str
Default Value: `'-1'`
Description: Maximum number of training steps. Set to -1 for unlimited steps based on epochs.

Parameter Name: `--lora_r`
Type: str
Default Value: `'16'`
Description: The rank (dimension) of the LoRA matrices.

Parameter Name: `--lora_alpha`
Type: str
Default Value: `'32'`
Description: The scaling factor for LoRA (alpha / r).

Parameter Name: `--lora_dropout`
Type: str
Default Value: `'0.05'`
Description: The dropout probability for LoRA layers.

Parameter Name: `--target_modules`
Type: str
Default Value: `'too long, check file'`
Description: Specifies which modules in the model LoRA should be applied to. Refer to the file for the complete list.

Parameter Name: `--weight_decay`
Type: str
Default Value: `'0.0'`
Description: The weight decay to apply to the optimizer.

Parameter Name: `--moe_enable`
Type: str
Default Value: `'false'`
Description: Flag to enable Mixture of Experts (MoE) injection logic.

Parameter Name: `--moe_lora_scope`
Type: str
Default Value: `'expert_only'`
Description: Defines the scope for LoRA application within MoE layers (e.g., 'expert_only').

Parameter Name: `--moe_expert_patterns`
Type: str
Default Value: `'too long to include here, check file'`
Description: Patterns to identify expert linear layers for MoE. Refer to the file for details.

Parameter Name: `--moe_router_patterns`
Type: str
Default Value: `'markdown would parse it, check file'`
Description: Patterns to identify router/gating linear layers for MoE. Refer to the file for details.

Parameter Name: `--moe_max_experts_lora`
Type: str
Default Value: `'-1'`
Description: Maximum number of LoRA experts to use per layer. -1 means no limit.

Parameter Name: `--moe_dry_run`
Type: str
Default Value: `'false'`
Description: If true, performs a dry run without actual training to check configurations.

Parameter Name: `--load_precision`
Type: str
Default Value: `'fp16'`
Description: Precision mode for loading the model. Options: `int8`, `int4`, `fp16`.

Parameter Name: `--logging_steps`
Type: str
Default Value: `'1'`
Description: Interval in steps for logging training progress.

Parameter Name: `--eval_steps`
Type: str
Default Value: `'50'`
Description: Interval in steps for performing model evaluation during training.
```

--------------------------------

### Fine-tuning Parameter Table

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-model.md

This table lists and describes various command-line arguments used for fine-tuning models. It covers parameters related to saving checkpoints, learning rate scheduling, resuming training, gradient checkpointing, model merging, mixed precision, optimizers, and dataloader configurations.

```APIDOC
Fine-tuning Parameters:

--save_steps (str): Number of steps to save the model. Default: '200'.
--save_total_limit (str): Maximum number of models to save. Default: '2'.
--warmup_ratio (str): Ratio for learning rate warmup. Default: '0.05'.
--lr_scheduler_type (str): Type of learning rate scheduler. Default: 'cosine'.
--resume_from_checkpoint (str): Path to resume training from a checkpoint. Default: None.
--no-gradient_checkpointing (flag): If set, gradient checkpointing is not used. Default: False.
--no-merge_and_save (flag): If set, the model is not merged and saved. Default: False.
--fp16 (str): Whether to use fp16 precision. Default: 'true'.
--optim (str): Name of the optimizer. Default: 'adamw_torch_fused'.
--dataloader_pin_memory (str): Whether to pin memory for DataLoader. Default: 'false'.
--dataloader_num_workers (str): Number of worker threads for DataLoader. Default: '0'.
--dataloader_prefetch_factor (str): Prefetch factor for DataLoader. Default: '2'.
--use_flash_attention_2 (str): Whether to use FlashAttention2 (ineffective for unsloth). Default: 'false'.
```

--------------------------------

### Model Configuration Parameters

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-model.md

This section outlines the key parameters for configuring and fine-tuning models. It includes settings for Hugging Face repository IDs, local model directories, training data paths, evaluation configurations, and general training hyperparameters.

```APIDOC
Parameter Name: `--repo_id`
Type: str
Default: `'Qwen/Qwen3-30B-A3B-Instruct-2507'`
Description: Hugging Face repository ID for the model.

Parameter Name: `--local_dir`
Type: str
Default: `'qwen3-30b-a3b-instruct'`
Description: Local directory to store the model files.

Parameter Name: `--data_path`
Type: str
Default: `'training_data.jsonl'`
Description: Path to the training data file.

Parameter Name: `--eval_data_path`
Type: str
Default: `None`
Description: Path to the validation data file.

Parameter Name: `--max_samples`
Type: str
Default: `None`
Description: Maximum number of training samples to use.

Parameter Name: `--max_eval_samples`
Type: str
Default: `None`
Description: Maximum number of validation samples to use.

Parameter Name: `--model_max_length`
Type: str
Default: `'2048'`
Description: Maximum sequence length for the model.

Parameter Name: `--output_dir`
Type: str
Default: `'finetune/models/qwen3-30b-a3b-qlora'`
Description: Directory to save the fine-tuned model and outputs.

Parameter Name: `--seed`
Type: str
Default: `'42'`
Description: Random seed for reproducibility.

Parameter Name: `--per_device_train_batch_size`
Type: str
Default: `'1'`
Description: Batch size per device for training.

Parameter Name: `--per_device_eval_batch_size`
Type: str
Default: `'1'`
Description: Batch size per device for evaluation.

Parameter Name: `--gradient_accumulation_steps`
Type: str
Default: `'16'`
Description: Number of steps to accumulate gradients before updating.

Parameter Name: `--learning_rate`
Type: str
Default: `'2e-4'`
Description: Learning rate for the optimizer.

Parameter Name: `--num_train_epochs`
Type: str
Default: `'3'`
Description: Number of training epochs.

Parameter Name: `--max_steps`
Type: str
Default: `'-1'`
Description: Maximum number of training steps. -1 means no limit.

Parameter Name: `--weight_decay`
Type: str
Default: `'0.0'`
Description: Weight decay for the optimizer.

Parameter Name: `--load_precision`
Type: str
Default: `'fp16'`
Description: Precision for loading the model (`int8`, `int4`, `fp16`).

Parameter Name: `--logging_steps`
Type: str
Default: `'1'`
Description: Number of steps between logging.

Parameter Name: `--eval_steps`
Type: str
Default: `'50'`
Description: Number of steps between evaluations.
```

--------------------------------

### Project Tutorial Components

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/readme.md

Outlines the key components covered in the project's tutorial. This includes steps for handling QQ database decryption, data cleaning, QLora fine-tuning, and model testing, with a mention of using Unsloth for acceleration.

```markdown
## 项目包含了**完整的教程**，包括：

* QQ 数据库的解密与处理
* 聊天数据清洗与转换
* QLora 微调流程
* 微调模型的测试与使用
* 使用unsloth加速训练!
```

--------------------------------

### 微调脚本参数

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-openai-oss-model.md

`run_finetune_oss.py` 脚本支持的参数及其说明。这些参数用于配置微调过程，包括模型选择、数据路径、训练设置和LoRA配置。

```APIDOC
run_finetune_oss.py:
  --repo_id: HF 仓库ID (str, default: "unsloth/gpt-oss-20b-unsloth-bnb-4bit")
  --local_dir: 本地模型目录 (str, default: "gpt-oss-20b-unsloth-bnb-4bit")
  --use_unsloth: 是否使用unsloth (str, default: "false", options: "true", "false")
  --use_qlora: 是否使用QLoRA (str, default: "true", options: "true", "false")
  --data_path: 训练数据路径 (str, default: "training_data.jsonl")
  --eval_data_path: 验证数据路径 (str/None, default: None)
  --max_samples: 最大训练样本数 (str/None, default: None)
  --max_eval_samples: 最大验证样本数 (str/None, default: None)
  --model_max_length: 最大序列长度 (str, default: "2048")
  --output_dir: 输出目录 (str, default: "finetune/models/qwen3-30b-a3b-qlora")
  --seed: 随机种子 (str, default: "42")
  --per_device_train_batch_size: 每设备训练批次大小 (str, default: "1")
  --per_device_eval_batch_size: 每设备验证批次大小 (str, default: "1")
  --gradient_accumulation_steps: 梯度累积步数 (str, default: "16")
  --learning_rate: 学习率 (str, default: "2e-4")
  --num_train_epochs: 训练轮数 (str, default: "3")
  --max_steps: 最大步数 (str, default: "-1", -1 means unlimited)
  --lora_r: LoRA 秩 (str, default: "16")
  --lora_alpha: LoRA alpha (str, default: "32")
  --lora_dropout: LoRA dropout率 (str, default: "0.05")
  --target_modules: LoRA 目标模块 (str, default: "q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj")
  --weight_decay: 权重衰减 (str, default: "0.0")
```

--------------------------------

### Project Status and TODOs

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/readme.md

Details the current project version (v0.1.2) and provides updates on tested features and future development plans. It mentions successful testing of Qlora_qwen3.py on a 4090 GPU and outlines TODOs like adding support for OSS models, MXFP4, and a WebUI with Frpc support.

```markdown
## 项目版本
# V 0.1.2
## ~~警告~~ 喜报
* 此版本的Qlora_qwen3.py已经过4090实机测试(generate_training_data_llm.py+run_finetune.py)
* 清洗数据也已经进行实机测试(当前版本)
## TODO
* [完成但未测试] 增加对oss模型的支持 (以及MXFP4?这是一个50系的计算,好像我还是没法测试)
> 难点:1.MOE模型 2.非原Qwen系列模型 3.我的3080似乎本地没法测试(无论是微调还是MXFP4)
> 好吧其实一点也不难,只是这几天在写其他项目
* [规划中]增加WebUI支持
> 这真的很重要,微调模型太恶心了,记得加上Frpc支持让大家都可以公网访问  
```

--------------------------------

### Convert HuggingFace Weights to GGUF

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/convert-model.md

Converts HuggingFace format models to the GGUF format using a Python script. Requires specifying input model path, output path, and desired precision type.

```bash
python3 ./convert_hf_to_gguf.py <HF_model_path> --outfile <output_GGUF_path> --outtype <precision_type>
```

```bash
python3 convert_hf_to_gguf.py /root/autodl-tmp/finetune/models/qwen3-8b-qlora/merged --outfile /root/autodl-fs/qwen3-8b-fp16-agent.gguf --outtype f16
```

--------------------------------

### 运行微调脚本

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-openai-oss-model.md

使用Python3执行微调脚本 `run_finetune_oss.py`。此脚本用于启动模型的微调过程。

```bash
python3 run_finetune_oss.py
```

--------------------------------

### 转换训练数据格式

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-openai-oss-model.md

使用chatml_to_harmony.py脚本将ChatML格式的训练数据转换为OpenAI Harmony格式，这是微调OSS模型所必需的。

```bash
python3 chatml_to_harmony.py --input training_data.jsonl --output training_data_harmony.txt
```

--------------------------------

### 下载OSS模型

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/fine-tune-openai-oss-model.md

使用huggingface-cli下载预训练的unsloth/gpt-oss-20b-unsloth-bnb-4bit模型到本地目录。

```bash
huggingface-cli download unsloth/gpt-oss-20b-unsloth-bnb-4bit --local-dir gpt-oss-20b
```

--------------------------------

### Troubleshooting GPU Out of Memory

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

Steps to mitigate 'GPU Out of Memory' errors during the fine-tuning process. This involves adjusting batch sizes, sample limits, and evaluation steps.

```APIDOC
GPU Out of Memory:
- Reduce --per_device_eval_batch_size.
- Reduce --max_eval_samples.
- Increase the --eval_steps interval.
```

--------------------------------

### QLora Fine-tuning with Qwen3-8B

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/index.md

Demonstrates the process of fine-tuning the Qwen3-8B model using QLora on a 4090 GPU with fp8 precision. This includes data generation and the fine-tuning script.

```python
generate_training_data_llm.py
run_finetune.py
```

--------------------------------

### vLLM OpenAI API Server Configuration and Execution

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/clean-data.md

Launches the vLLM OpenAI-compatible API server. Key parameters include the model path, GPU memory utilization, maximum model length, and batching configurations. Ensure the model path in the command matches the one specified in your .env file.

```bash
python3 -m vllm.entrypoints.openai.api_server --model /home/vllm/qwen3-4b-int8 --gpu-memory-utilization 0.7 --max-model-len 10240 --max-num-seqs 4 --max-num-batched-tokens 2048 --dtype auto
```

--------------------------------

### Quantize GGUF Models

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/guide/convert-model.md

Command to quantize a GGUF model to a specified quantization level. Requires the input GGUF path, output GGUF path, and the quantization level.

```bash
./build/bin/llama-quantize <输入GGUF路径> <输出GGUF路径> <量化等级>

Parameters:
  <输入GGUF路径>: Path to the unquantized .gguf file.
  <输出GGUF路径>: Path to save the quantized .gguf file.
  <量化等级>: Quantization level, e.g., 'Q4_0', 'Q4_K_M', 'Q8_0'.

Example:
./build/bin/llama-quantize \
  /root/autodl-fs/qwen3-8b-fp16-agent.gguf \
  /root/autodl-fs/qwen3-8b-q8_0-agent.gguf \
  Q8_0
```

--------------------------------

### Troubleshooting Evaluation Set Issues

Source: https://github.com/qqqqqf-q/qing-digital-self/blob/main/docs/en/guide/fine-tune-model.md

Guidance for resolving problems related to the evaluation dataset during fine-tuning. This includes checking data paths and ensuring correct data formatting.

```APIDOC
Evaluation Set Not Working:
- Check that --eval_data_path is correct.
- Ensure evaluation data format matches training data.
- Look for console output saying “no evaluation data path provided”.
```