WeClone

https://github.com/xming521/weclone
Admin
WeClone is a one-stop solution for creating digital avatars from chat history, fine-tuning LLMs with...
Tokens:4,122
Snippets:29
Trust Score:8.8
License:AGPL-3.0
Update:1 year ago
Show doc for...
Tokens:
Raw
### Setup WeClone Project Environment

Source: https://github.com/xming521/weclone/blob/master/README.md

Clones the WeClone repository, creates a Python virtual environment using `uv` (a fast Python environment manager), activates the environment, and installs project dependencies using `uv pip`.

```bash
git clone https://github.com/xming521/WeClone.git && cd WeClone
uv venv .venv --python=3.10
source .venv/bin/activate # windows .venv\Scripts\activate
uv pip install --group main -e .
```

--------------------------------

### Download Qwen2.5-VL-7B-Instruct Model

Source: https://github.com/xming521/weclone/blob/master/README.md

Installs Git Large File Storage (LFS) and then clones the Qwen2.5-VL-7B-Instruct model from Hugging Face into the local `models` directory, which is used by default for fine-tuning.

```bash
git lfs install
git clone https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct models/Qwen2.5-VL-7B-Instruct
```

--------------------------------

### Install FlashAttention for Training Acceleration

Source: https://github.com/xming521/weclone/blob/master/README.md

Installs the FlashAttention library using `uv pip`, which is an optional step to accelerate the training and inference processes by optimizing attention mechanisms.

```bash
uv pip install flash-attn --no-build-isolation
```

--------------------------------

### Set Up WeClone Development Environment

Source: https://github.com/xming521/weclone/blob/master/README.md

These commands prepare the local development environment for WeClone. They install necessary development dependencies using `uv pip` and set up pre-commit hooks to ensure code quality and formatting standards are met before commits.

```bash
uv pip install --group dev -e .
pre-commit install
```

--------------------------------

### Copy WeClone Configuration File Template

Source: https://github.com/xming521/weclone/blob/master/README.md

Copies the example configuration file template (`tg.template.jsonc`) to `settings.jsonc`, which will be used for all subsequent project configurations, including training and inference settings.

```bash
cp examples/tg.template.jsonc settings.jsonc
```

--------------------------------

### Start WeClone API Server for Inference

Source: https://github.com/xming521/weclone/blob/master/README.md

This command initiates the WeClone API server, making the inference capabilities available via an API. This is a prerequisite for integrating WeClone with other applications or services.

```bash
weclone-cli server
```

--------------------------------

### Test WeClone Model with Common Chat Questions

Source: https://github.com/xming521/weclone/blob/master/README.md

These commands start the WeClone API server and then execute a test suite using common chat questions. The test results, which exclude queries for personal information, are saved in `test_result-my.txt`.

```bash
weclone-cli server
weclone-cli test-model
```

--------------------------------

### Run WeClone Browser Demo for Inference

Source: https://github.com/xming521/weclone/blob/master/README.md

This command launches a browser-based demonstration for simple inference. Users can test and adjust temperature and top_p values, then save preferred settings in `settings.jsonc` for future inference tasks.

```bash
weclone-cli webchat-demo
```

--------------------------------

### WeClone CLI Multi-GPU Training Command with DeepSpeed

Source: https://github.com/xming521/weclone/blob/master/README.md

Installs DeepSpeed and then executes the `train_sft.py` script using DeepSpeed for multi-GPU training. This requires uncommenting the `deepspeed` line in `settings.jsonc` and specifying the number of GPUs.

```bash
uv pip install deepspeed
deepspeed --num_gpus=number_of_gpus weclone/train/train_sft.py
```

--------------------------------

### Set up Spark-TTS Environment and Install Dependencies

Source: https://github.com/xming521/weclone/blob/master/weclone-audio/README.md

This command block sets up a Python virtual environment for Spark-TTS, activates it, installs the `sparktts` dependency group, and clones the Spark-TTS repository. This prepares the environment for using the Spark-TTS model.

```bash
uv venv .venv-sparktts --python=3.10
source .venv-sparktts/bin/activate
uv pip install --group sparktts -e .

git clone https://github.com/SparkAudio/Spark-TTS.git weclone-audio/src/Spark-TTS
```

--------------------------------

### Set up Llasa Model Environment (Deprecated)

Source: https://github.com/xming521/weclone/blob/master/weclone-audio/README.md

This command block sets up a Python virtual environment for the deprecated Llasa model, activates it, and installs the `xcodec` dependency group. It also notes optional system dependencies for installation.

```bash
uv venv .venv-xcodec --python=3.9
source .venv-xcodec/bin/activate
uv pip install --group xcodec -e .
# 退出环境
deactivate

# 系统依赖安装（如果需要）
sudo apt install python3-dev 
sudo apt install build-essential
```

--------------------------------

### Set up Python Environment and Install Dependencies with uv

Source: https://github.com/xming521/weclone/blob/master/README_zh.md

This command sequence clones the WeClone repository, creates a Python virtual environment using 'uv' for dependency management, activates the environment, and installs all required project dependencies. It uses an editable install and a specific PyPI mirror for faster downloads.

```bash
git clone https://github.com/xming51/WeClone.git && cd WeClone
uv venv .venv --python=3.10
source .venv/bin/activate # windows下执行 .venv\Scripts\activate
uv pip install --group main -e . -i https://pypi.tuna.tsinghua.edu.cn/simple/
```

--------------------------------

### Install FlashAttention for Accelerated Training

Source: https://github.com/xming521/weclone/blob/master/README_zh.md

This command installs the FlashAttention library using `uv pip`. FlashAttention is an optimized attention mechanism that significantly speeds up training and inference for large language models by reducing memory usage and increasing computational efficiency. Pre-compiled wheels are available for version compatibility issues.

```bash
uv pip install flash-attn --no-build-isolation
```

--------------------------------

### Set up Python Environment for PyWxDump

Source: https://github.com/xming521/weclone/blob/master/weclone-audio/README.md

This command block creates a Python virtual environment named `.venv-wx` using `uv`, activates it, and then installs the `pywxdump` package using `uv pip`. This environment is specifically for extracting WeChat chat records.

```bash
uv venv .venv-wx --python=3.10
.venv-wx\Scripts\activate
uv pip install pywxdump
```

--------------------------------

### Verify CUDA Environment with PyTorch

Source: https://github.com/xming521/weclone/blob/master/README.md

Executes a Python command to check if the CUDA environment is correctly configured and recognized by PyTorch, which is essential for GPU-accelerated operations. This step is not required for Mac users.

```bash
python -c "import torch; print('CUDA Available:', torch.cuda.is_available());"
```

--------------------------------

### WeClone CLI Single GPU Training Command

Source: https://github.com/xming521/weclone/blob/master/README.md

Initiates the fine-tuning process for the model on a single GPU using the `weclone-cli train-sft` command. Training parameters like `model_name_or_path`, `template`, `lora_target`, `per_device_train_batch_size`, `gradient_accumulation_steps`, `num_train_epochs`, `lora_rank`, and `lora_dropout` are configured in `settings.jsonc`.

```bash
weclone-cli train-sft
```

--------------------------------

### Run WeClone Project Tests

Source: https://github.com/xming521/weclone/blob/master/README.md

This command executes the project's test suite using `pytest`. It is recommended to run this command to ensure all tests pass before submitting any code changes, verifying the integrity and functionality of the codebase.

```bash
pytest tests
```

--------------------------------

### Set Up WeClone Development Environment

Source: https://github.com/xming521/weclone/blob/master/README_zh.md

These commands prepare the local environment for WeClone development. They install necessary dependencies for development and set up pre-commit hooks to ensure code quality and formatting standards are met before committing changes.

```bash
uv pip install --group dev -e .
pre-commit install
```

--------------------------------

### Start WeClone API Service

Source: https://github.com/xming521/weclone/blob/master/README_zh.md

This command initiates the WeClone API service, which is necessary for LangBot to connect and utilize WeClone's functionalities. It should be executed after LangBot deployment.

```bash
weclone-cli server
```

--------------------------------

### WeClone CLI Data Preprocessing Command

Source: https://github.com/xming521/weclone/blob/master/README.md

Executes the `weclone-cli make-dataset` command to process raw chat data. This command relies on configurations in `settings.jsonc`, such as `language`, `platform`, `include_type`, and `telegram_args.my_id`, and can perform privacy-preserving operations like removing sensitive information.

```bash
weclone-cli make-dataset
```

--------------------------------

### Copy WeClone Configuration Template

Source: https://github.com/xming521/weclone/blob/master/README_zh.md

This command copies the default configuration template file, `settings.template.jsonc`, to `settings.jsonc`. The `settings.jsonc` file is where all subsequent project configurations, such as model parameters and data processing settings, will be made. For multimodal models, `examples/mllm.template.jsonc` should be used.

```bash
cp settings.template.jsonc settings.jsonc
```

--------------------------------

### Download Spark-TTS Model

Source: https://github.com/xming521/weclone/blob/master/weclone-audio/README.md

These snippets demonstrate two methods to download the Spark-TTS-0.5B model. The Python method uses `huggingface_hub.snapshot_download` to fetch the model, while the Bash method uses `git clone` with Git LFS to download the repository.

```python
from huggingface_hub import snapshot_download

# 假设此 Python 代码在 weclone-audio 目录下运行 模型将下载到 weclone-audio/pretrained_models/Spark-TTS-0.5B
snapshot_download("SparkAudio/Spark-TTS-0.5B", local_dir="pretrained_models/Spark-TTS-0.5B")
```

```bash
mkdir -p pretrained_models

# Make sure you have git-lfs installed (https://git-lfs.com)
git lfs install
git clone https://huggingface.co/SparkAudio/Spark-TTS-0.5B pretrained_models/Spark-TTS-0.5B
```

--------------------------------

### WeClone CLI Commands for Data Processing, Training, and Inference

Source: https://github.com/xming521/weclone/blob/master/README_zh.md

This section outlines the primary command-line interface (CLI) commands provided by WeClone for various stages of the project workflow. These commands facilitate data preprocessing, model fine-tuning (both single and multi-GPU), and different modes of model inference, including a web-based demo and an API server.

```APIDOC
weclone-cli make-dataset
  - Description: Processes raw chat data into a format suitable for model training. Configuration for data processing, such as language, platform, and include types, is managed in `settings.jsonc`.
  - Parameters: Data processing arguments can be customized via `make_dataset_args` in `settings.jsonc`.

weclone-cli train-sft
  - Description: Initiates single-GPU fine-tuning of the model using the processed dataset. Model parameters like `model_name_or_path`, `template`, `lora_target`, `per_device_train_batch_size`, `gradient_accumulation_steps`, `num_train_epochs`, `lora_rank`, and `lora_dropout` are configured in `settings.jsonc`.

deepseed --num_gpus=<num_gpus> weclone/train/train_sft.py
  - Description: Enables multi-GPU fine-tuning using DeepSpeed. Requires `deepspeed` to be installed (`uv pip install deepspeed`). The number of GPUs to use is specified by `--num_gpus`.
  - Parameters:
    - --num_gpus: The number of GPUs to utilize for distributed training.

weclone-cli webchat-demo
  - Description: Launches a browser-based demo for interactive model inference. Allows testing and adjusting inference parameters like `temperature` and `top_p` in `infer_args` within `settings.jsonc`.

weclone-cli server
  - Description: Starts an API server for model inference, allowing external applications to interact with the fine-tuned model via API requests.

weclone-cli test-model
  - Description: Executes a set of common chat questions against the deployed model (via `weclone-cli server`) to evaluate its performance. Results are typically saved to `test_result-my.txt`.
```

--------------------------------

### Perform Speech Synthesis Inference with Spark-TTS

Source: https://github.com/xming521/weclone/blob/master/weclone-audio/README.md

This Python code demonstrates how to load a Spark-TTS model, perform speech synthesis inference, and save the generated audio. It initializes the model with a specified path and device, then uses `model.inference` with text and a prompt speech path to generate audio.

```python
import os
import SparkTTS  
import soundfile as sf
import torch

from SparkTTS import SparkTTS 

# 假设此 Python 代码在 weclone-audio 目录下运行
# 模型路径相对于当前目录
model_path = "pretrained_models/Spark-TTS-0.5B"
sample_audio = "sample.wav"
output_audio = "output.wav"

model = SparkTTS(model_path, "cuda")

with torch.no_grad():
    wav = model.inference(
        text="晚上好啊,小可爱们，该睡觉了哦",
        prompt_speech_path=sample_audio, # 使用相对路径
        prompt_text="对，这就是我万人敬仰的太乙真人，虽然有点婴儿肥，但也掩不住我逼人的帅气。",
    )
    sf.write(output_audio, wav, samplerate=16000) # 使用相对路径
```

--------------------------------

### Run WeClone Project Tests

Source: https://github.com/xming521/weclone/blob/master/README_zh.md

This command executes the project's test suite using pytest. It is crucial to run this command before submitting code to ensure that all existing functionalities work as expected and new changes do not introduce regressions.

```bash
pytest tests
```

--------------------------------

### LLM Fine-tuning Hardware Requirements (VRAM Estimation)

Source: https://github.com/xming521/weclone/blob/master/README_zh.md

This table provides estimated VRAM requirements for fine-tuning Large Language Models (LLMs) using different methods and precisions across various model sizes (7B, 14B, 30B, 70B). It helps users understand the GPU memory needed for their specific model and training approach.

```APIDOC
Hardware Requirements for LLM Fine-tuning:

| Method                             | Precision |   7B  |  14B  |  30B  |   70B  |   `x`B  |
|------------------------------------|-----------|-------|-------|-------|--------|---------|
| Full (`bf16` or `fp16`)            |  32       | 120GB | 240GB | 600GB | 1200GB | `18x`GB |
| Full (`pure_bf16`)                 |  16       |  60GB | 120GB | 300GB |  600GB |  `8x`GB |
| Freeze/LoRA/GaLore/APOLLO/BAdam    |  16       |  16GB |  32GB |  64GB |  160GB |  `2x`GB |
| QLoRA                              |   8       |  10GB |  20GB |  40GB |   80GB |   `x`GB |
| QLoRA                              |   4       |   6GB |  12GB |  24GB |   48GB | `x/2`GB |
| QLoRA                              |   2       |   4GB |   8GB |  16GB |   24GB | `x/4`GB |

Parameters:
- Method: The fine-tuning technique used (e.g., Full, LoRA, QLoRA).
- Precision: The numerical precision for training (e.g., 32-bit, 16-bit, 8-bit, 4-bit, 2-bit).
- Model Size (7B, 14B, 30B, 70B): Specific model parameter counts.
- `x`B: General placeholder for any model size in billions of parameters.
- Estimated VRAM: Approximate GPU memory required in Gigabytes (GB).

Notes:
- The project defaults to Qwen2.5-7B-Instruct model with LoRA for SFT stage, requiring approximately 16GB VRAM.
- Other models and methods supported by LLaMA Factory can also be used.
```

--------------------------------

### Perform Speech Synthesis Inference with Llasa (Deprecated)

Source: https://github.com/xming521/weclone/blob/master/weclone-audio/README.md

This Python code demonstrates how to use the deprecated Llasa model for speech synthesis inference. It initializes a `TextToSpeech` object with a sample audio path and text, then generates target audio using `tts.infer` and saves the result.

```python
import os
import soundfile as sf
# 假设 text_to_speech.py 位于 src/ 或其他可导入的位置
from text_to_speech import TextToSpeech


sample_audio_text = "对，这就是我万人敬仰的太乙真人，虽然有点婴儿肥，但也掩不住我逼人的帅气。"  # 示例音频文本
# 假设此 Python 代码在 weclone-audio 目录下运行
# 示例音频路径相对于当前目录
sample_audio_path = "sample.wav"
output_audio = "output.wav"


tts = TextToSpeech(sample_audio_path, sample_audio_text)
target_text = "晚上好啊"  # 生成目标文本
result = tts.infer(target_text)
sf.write(output_audio, result[1], result[0])  # 使用相对路径
```

--------------------------------

### Export WeChat Voice Files using PyWxDump

Source: https://github.com/xming521/weclone/blob/master/weclone-audio/README.md

This command executes a Python script to export sample audio files from a WeChat database. It requires the path to the decrypted database and the `MsgSvrID` field of the chat record to be exported.

```bash
python weclone-audio/src/get_sample_audio.py --db-path "导出数据库路径" --MsgSvrID "导出聊天记录的MsgSvrID字段"
```

--------------------------------

### Verify CUDA Availability with PyTorch

Source: https://github.com/xming521/weclone/blob/master/README_zh.md

This command executes a Python script to check if PyTorch can detect and utilize a CUDA-enabled GPU. It's a crucial step to confirm that the CUDA environment is correctly configured and accessible for GPU-accelerated operations, which are essential for model training and inference.

```bash
python -c "import torch; print('CUDA是否可用:', torch.cuda.is_available());"
```
Tokens:
Raw