### Create Virtual Environment and Install Dependencies Source: https://github.com/y0mingzhang/allie-v2/blob/main/README.md Sets up a Python virtual environment and installs project dependencies using uv. ```bash # Create virtual environment with Python 3.9-3.12 (NOT 3.13, vLLM has compatibility issues) uv venv --python 3.12 source .venv/bin/activate # Install project dependencies uv pip install -e . ``` -------------------------------- ### Start vLLM Server Source: https://github.com/y0mingzhang/allie-v2/blob/main/README.md Starts a vLLM server for the specified model. Ensure the virtual environment is activated. ```bash # Start vLLM server (in one terminal) source .venv/bin/activate vllm serve yimingzhang/qwen-3-1.7b-57b-cool-from-66550-step96800 \ --max-model-len 1024 --enforce-eager --port 12398 ``` -------------------------------- ### Install Dependencies for Training Source: https://github.com/y0mingzhang/allie-v2/blob/main/README.md Installs specific dependencies required for model training, including setuptools, picotron, and vllm. ```bash uv pip install setuptools uv pip install -e picotron --no-build-isolation uv pip install vllm --torch-backend=auto ``` -------------------------------- ### Install Picotron Package Source: https://github.com/y0mingzhang/allie-v2/blob/main/picotron/README.md Install the Picotron package in editable mode. This command is used to set up the library in your local environment. ```shell pip install -e . ``` -------------------------------- ### Train Llama-1B Model Locally (CPU) Source: https://github.com/y0mingzhang/allie-v2/blob/main/picotron/README.md Starts the training process for a Llama-1B model on CPU using a generated configuration file. Expect slow performance due to CPU limitations. ```shell torchrun --nproc_per_node 8 train.py --config tmp/llama-1B-cpu/config.json ``` -------------------------------- ### Train Llama-1B Model Locally (GPU) Source: https://github.com/y0mingzhang/allie-v2/blob/main/picotron/README.md Starts the training process for a Llama-1B model using a previously generated configuration file. Requires torchrun for distributed execution. ```shell torchrun --nproc_per_node 8 train.py --config tmp/llama-1B/config.json ``` -------------------------------- ### Create Configuration for Llama-7B Model (3D Parallelism) Source: https://github.com/y0mingzhang/allie-v2/blob/main/picotron/README.md Generates a JSON configuration file for training a Llama-7B model using 3D parallelism (Data, Tensor, Pipeline). This command sets up parameters for a more complex distributed training setup. ```shell python create_config.py --out_dir tmp --dp 4 --tp 2 --pp 2 --pp_engine 1f1b --exp_name llama-7B --model_name meta-llama/Llama-2-7b-hf --grad_acc_steps 32 --mbs 4 --seq_len 1024 --hf_token ``` -------------------------------- ### Create Configuration for Llama-1B Model (CPU) Source: https://github.com/y0mingzhang/allie-v2/blob/main/picotron/README.md Generates a JSON configuration file for training a Llama-1B model on CPU. This command includes the `--use_cpu` flag and specifies parameters suitable for CPU-based training. ```shell python create_config.py --out_dir tmp --exp_name llama-1B-cpu --dp 2 --tp 2 --pp 2 --pp_engine 1f1b --model_name HuggingFaceTB/SmolLM-1.7B --num_hidden_layers 5 --grad_acc_steps 2 --mbs 4 --seq_len 128 --hf_token --use_cpu ``` -------------------------------- ### Initialize Submodules and Build vLLM for CPU Source: https://github.com/y0mingzhang/allie-v2/blob/main/README.md Initializes git submodules and builds vLLM specifically for CPU inference, including fixing shared memory permissions. ```bash # Initialize submodules (vllm_source and lichess-bot) git submodule update --init --recursive # Build vLLM for CPU cd vllm_source uv pip install -r requirements/cpu-build.txt --index-strategy unsafe-best-match --torch-backend cpu uv pip install -r requirements/cpu.txt --index-strategy unsafe-best-match --torch-backend cpu VLLM_TARGET_DEVICE=cpu uv pip install -e . --no-build-isolation cd .. # Fix shared memory permission issue on multi-user systems uv pip install "model-hosting-container-standards>=0.1.12,<1.0.0" ``` -------------------------------- ### Train Model with Torchrun Source: https://github.com/y0mingzhang/allie-v2/blob/main/README.md Launches the model training script using torchrun, requiring 8 GPUs and an HF token. ```bash torchrun --nproc_per_node 8 picotron/train.py --config configs/main_runs/qwen-3-4b-58b.json ``` -------------------------------- ### Test vLLM Model Loading and Inference Source: https://github.com/y0mingzhang/allie-v2/blob/main/README.md Verifies that a vLLM model can be loaded and produces sensible moves for known opening positions. ```bash python src/tools/eval/test_vllm_model.py --model yimingzhang/qwen-3-1.7b-57b-cool-from-66550-step96800 ``` -------------------------------- ### Create Configuration for Llama-1B Model (GPU) Source: https://github.com/y0mingzhang/allie-v2/blob/main/picotron/README.md Generates a JSON configuration file for training a Llama-1B model. This command specifies parameters like data parallelism, model name, and sequence length for GPU training. ```shell python create_config.py --out_dir tmp --exp_name llama-1B --dp 8 --model_name HuggingFaceTB/SmolLM-1.7B --num_hidden_layers 15 --grad_acc_steps 32 --mbs 4 --seq_len 1024 --hf_token ``` -------------------------------- ### Export Model Checkpoint Source: https://github.com/y0mingzhang/allie-v2/blob/main/README.md Exports a model checkpoint to safetensors and config.json files. Use --tokenizer-dir to copy tokenizer files. ```bash uv run python src/tools/hf/export_checkpoint.py \ --config configs/tiny_runs/qwen-3-1.7b-muon.json \ --checkpoint models/tiny-qwen-3-1.7b-muon-1b/3814/weights_tp_rank_world_size=0_1_pp_rank_world_size=0_1.pth \ --output-dir exports/tiny-qwen-3-1.7b-muon-1b \ --dtype bfloat16 ``` -------------------------------- ### Run Lichess Bot Source: https://github.com/y0mingzhang/allie-v2/blob/main/README.md Runs the lichess bot, which uses a custom vLLM engine. Ensure the virtual environment is activated and the vLLM server is running. ```bash # Run lichess-bot (in another terminal) source .venv/bin/activate cd src/lichess-bot python lichess-bot.py --config config.yml ``` -------------------------------- ### Picotron Citation Source: https://github.com/y0mingzhang/allie-v2/blob/main/picotron/README.md BibTeX entry for citing the Picotron project. Use this in academic publications to acknowledge the framework. ```bibtex @misc{zhao2025picotron, author = {Haojun Zhao and Ferdinand Mom}, title = {Picotron: Distributed training framework for education and research experimentation}, year = {2025}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/huggingface/picotron}} } ``` -------------------------------- ### Submit Slurm Jobs for Llama-7B Training Source: https://github.com/y0mingzhang/allie-v2/blob/main/picotron/README.md Submits training jobs to a Slurm cluster for the Llama-7B model configuration. This is used for managing distributed training on a cluster environment. ```shell python submit_slurm_jobs.py --inp_dir tmp/llama-7B --qos high --hf_token ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.