### Install FlashAttention for Acceleration (Linux) Source: https://www.weclone.love/en/docs/deploy/environment Installs FlashAttention using uv, a library that can significantly speed up training and inference on NVIDIA GPUs. This is an optional step. ```bash uv pip install flash-attn --no-build-isolation ``` -------------------------------- ### Install WeClone and Dependencies with uv (Linux) Source: https://www.weclone.love/en/docs/deploy/environment Clones the WeClone repository, creates a Python virtual environment using uv, activates it, and installs project dependencies. Requires git and uv to be installed. ```bash git clone https://github.com/xming521/WeClone.git && cd WeClone uv venv .venv --python=3.12 source .venv/bin/activate # On Windows, run .venv\Scripts\activate uv pip install --group main -e . ``` -------------------------------- ### Verify PyTorch CUDA Configuration (Linux) Source: https://www.weclone.love/en/docs/deploy/environment Tests if PyTorch can correctly detect and use the CUDA environment, essential for GPU acceleration. Requires Python and PyTorch to be installed. ```python python -c "import torch; print('CUDA available:', torch.cuda.is_available());" ``` -------------------------------- ### Copy Configuration Template (Linux) Source: https://www.weclone.love/en/docs/deploy/environment Copies the template configuration file to `settings.jsonc`, which will be used for all project configurations. This command assumes the template file exists in the current directory. ```bash cp settings.template.jsonc settings.jsonc ``` -------------------------------- ### Execute WeClone Dataset Creation Source: https://www.weclone.love/en/docs/deploy/data_preprocessing This command initiates the dataset creation process in WeClone. Ensure you are in the WeClone project root directory and have activated the virtual environment before running. ```bash weclone-cli make-dataset ``` -------------------------------- ### Start Multi-Card SFT Training - Deepspeed Source: https://www.weclone.love/en/docs/deploy/model_finetuning Launches supervised fine-tuning (SFT) for the WeClone model across multiple GPUs using Deepspeed. Requires Deepspeed to be installed and configured in 'settings.jsonc'. The number of GPUs to use is specified by the --num_gpus flag. ```bash deepspeed --num_gpus= weclone/train/train_sft.py ``` -------------------------------- ### Install Deepspeed for Multi-Card Training - Python Source: https://www.weclone.love/en/docs/deploy/model_finetuning Installs the Deepspeed library, a framework for optimizing distributed deep learning training. This is a prerequisite for performing multi-card training with WeClone. ```bash uv pip install deepspeed ``` -------------------------------- ### Start Single-Card SFT Training - WeClone CLI Source: https://www.weclone.love/en/docs/deploy/model_finetuning Initiates supervised fine-tuning (SFT) for the WeClone model using a single GPU. It requires setting the CUDA visible devices environment variable and reads configurations from 'settings.jsonc'. Monitor terminal output for loss decrease. ```bash export CUDA_VISIBLE_DEVICES=0 weclone-cli train-sft ``` -------------------------------- ### Download Model using Git Source: https://www.weclone.love/en/docs/deploy/model_download This bash snippet shows how to download a model using Git and Git Large File Storage (LFS). It first installs Git LFS and then clones the specified model repository into a local directory. ```bash git lfs install git clone https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct models/Qwen2.5-VL-7B-Instruct ``` -------------------------------- ### Configure vLLM for Quantization Source: https://www.weclone.love/en/docs/deploy/data_preprocessing Configuration snippet for enabling vLLM's bitsandbytes quantization loading to manage GPU memory, especially for older GPUs. This helps prevent out-of-memory errors during offline inference. ```json { "vllm_args": { "gpu_memory_utilization": 0.9, "quantization": "bitsandbytes", "load_format": "bitsandbytes" } } ``` -------------------------------- ### Configure Model Path in settings.jsonc Source: https://www.weclone.love/en/docs/deploy/model_download This JSON snippet shows how to configure the model path and template in the `settings.jsonc` file. It also includes an example for `lora_target` which specifies the modules to be fine-tuned. ```json "common_args": { "model_name_or_path": "your_model_path", "template": "your_model_template", ... }, "train_sft_args": { "lora_target": "module_names_of_the_model_to_be_fine-tuned", // e.g., "q_proj", "k_proj", "v_proj", "o_proj" ... } ``` -------------------------------- ### Update Environment with Pip Source: https://www.weclone.love/en/docs/introduce/FAQ Updates the WeClone environment by installing the latest code and dependencies using pip. Ensure you have the correct package manager and command syntax. ```bash uv pip install --group main -e . ``` -------------------------------- ### Execute Dataset Creation with Float16 dtype Source: https://www.weclone.love/en/docs/deploy/data_preprocessing Command to execute the dataset creation process while explicitly setting the data type to float16. This is a workaround for older GPUs that do not support bfloat16. ```bash weclone-cli make-dataset --dtype=half ``` -------------------------------- ### Reinstall bitsandbytes for QLoRA - Python Source: https://www.weclone.love/en/docs/deploy/model_finetuning Provides a command to reinstall the 'bitsandbytes' library, often necessary to resolve import errors when using QLoRA. Ensures compatibility with quantization features. ```bash uv pip install bitsandbytes>=0.39.0 ``` -------------------------------- ### Enable QLoRA Quantization Configuration - JSON Source: https://www.weclone.love/en/docs/deploy/model_finetuning Adds configuration parameters to 'settings.jsonc' to enable QLoRA (Quantized Low-Rank Adaptation) for reduced video memory consumption during fine-tuning. Supports 2, 4, or 8-bit quantization. ```json "quantization_bit": 4, "quantization_type": "nf4", "double_quantization": true, "quantization_method": "bitsandbytes" ``` -------------------------------- ### Download Model using Hugging Face CLI Source: https://www.weclone.love/en/docs/deploy/model_download This bash command demonstrates how to download a model using the Hugging Face command-line interface. It specifies the model repository and the local directory for storage, with symlinks disabled. ```bash huggingface-cli download Qwen/Qwen2.5-7B-Instruct --local-dir ./models/Qwen2.5-7B-Instruct --local-dir-use-symlinks False ``` -------------------------------- ### Adjust Training Parameters for Memory Source: https://www.weclone.love/en/docs/introduce/FAQ Configuration options to reduce memory usage during model training. These parameters help mitigate out-of-memory errors by decreasing batch size, sequence length, and enabling optimized kernels. ```yaml per_device_train_batch_size: 1 cutoff_len: 512 enable_liger_kernel: true use_unsloth_gc: true ``` -------------------------------- ### Recommended Directory Structure for Exported Data Source: https://www.weclone.love/en/docs/deploy/export-chat-history This illustrates the recommended directory structure for organizing exported chat data, separating CSV message files from media files. It shows how to structure data for multiple individuals and split large chat logs into smaller CSV files. ```tree dataset/ ├── csv/ │ ├── person1-personal_chat-1234567890/ │ │ ├── person1_0_5000.csv │ │ └── person1_5000_10000.csv │ └── person2-personal_chat-2345678901/ │ ├── person2_0_5000.csv │ └── person2_5000_10000.csv └── media/ └── images/ ├── 01c177d8ad98969ba048455b54eef.jpg └── 13d6d8a81fa7d09238c81fe314e85.png ``` -------------------------------- ### WeClone Required Data Sample Format Source: https://www.weclone.love/en/docs/deploy/export-chat-history This table defines the expected data sample format for WeClone, outlining the fields and their meanings for chat messages. ```markdown id| MsgSvrID| type_name| is_sender| talker| room_name| msg| src| CreateTime ---|---|---|---|---|---|---|---|--- 1| 7437267147299592543| Image| 0| 12345iru2zsmo22| 6789z5qlxzfj22| Image| File\dd0e62b6eb67d1953454354350301d6c\Image\2024-10\01c177d8ad90af8969b455b54eef.dat| 2024/10/4 11:42 2| 637529293739295664| Image| 0| 12345iru2zsmo22| 6789z5qlxzfj22| Image| File\dd0e62b6eb67d1953454354350301d6c\Image\2024-10\d8a8936ca622823452e80a53a6.dat| 2024/10/4 11:42 3| 4073926741244663531| Text| 1| 12345iru2zsmo22| 6789z5qlxzfj22| Little Maldives| | 2024/10/4 11:43 4| 706358374822797422| Text| 1| 12345iru2zsmo22| 6789z5qlxzfj22| Worthy of the name| | 2024/10/4 11:43 ``` -------------------------------- ### Adjust Multimodal Model Parameters Source: https://www.weclone.love/en/docs/introduce/FAQ Configuration options for multimodal models to reduce memory usage by limiting the maximum number of pixels and images processed. ```yaml image_max_pixels: max_image_num: ``` -------------------------------- ### Quantize Model Parameters Source: https://www.weclone.love/en/docs/introduce/FAQ Sets the quantization bit for model parameters, specifically for the LoRA fine-tuning method. This reduces the memory footprint of the model. ```yaml quantization_bit: 4 ``` -------------------------------- ### Disable All Tools in AstrBot Source: https://www.weclone.love/en/docs/chatbot/AstrBot Command to disable all default tools within AstrBot. This is crucial for fine-tuned models that primarily imitate language style and may not support complex tool interactions, ensuring the fine-tuning effect is visible. ```bash /tool off_all ``` -------------------------------- ### Resolve Windows Dataloader Pickling Error Source: https://www.weclone.love/en/docs/introduce/FAQ Sets the `dataloader_num_workers` parameter to 0 within `train_sft_args` to resolve the 'Can't pickle local object' error commonly encountered in Windows environments. ```python train_sft_args = { "dataloader_num_workers": 0 # ... other arguments } ``` -------------------------------- ### Telegram Chat Export Directory Structure Source: https://www.weclone.love/en/docs/deploy/export-chat-history This illustrates the expected directory structure for exported Telegram chat history. It includes multiple chat export folders, each containing a `result.json` file and a `photos` subdirectory. ```tree dataset/ └── telegram/ ├── ChatExport_2025-01-01/ │ ├── result.json │ └── photos/ └── ChatExport_2025-01-02/ ├── result.json └── photos/ ``` -------------------------------- ### Python ChatMessage Class for Data Adaptation Source: https://www.weclone.love/en/docs/deploy/export-chat-history This Python class defines the structure for a chat message, serving as a reference for adapting data from various chat platforms to WeClone's format. It includes fields for message ID, type, sender, content, and timestamps. ```python class ChatMessage: id: int # Sequential ID MsgSvrID: str # Original message platform ID type_name: str # Message type, see cut_type_data and skip_type_data is_sender: int # 0: Sent by the other person, 1: Sent by you talker: str # Message sender ID msg: str # Message content src: str # Media file path, additional information field CreateTime: Timestamp # Message sending time room_name: Optional[str] = None # Message receiver or group chat ID is_forward: bool = False # Whether the message is forwarded ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.