### YAML configuration example for training algorithms Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/training-algorithms.md Example configuration showing how to set advantage estimation, policy loss, and enable IS correction and OPSM. Note that IS correction, M2PO, and trainer.use_rollout_log_probs: true are mutually exclusive. ```yaml algorithm: advantage_estimator: grpo advantage_norm_mode: group_zscore policy_loss: grpo loss_agg_mode: token-mean clip_ratio_low: 0.2 clip_ratio_high: 0.28 is_correction: enable: true action: clip level: token lower_bound: 0.5 upper_bound: 2.0 opsm: enable: true delta: 0.1 ``` -------------------------------- ### Launch BCP single-node 8-GPU preset Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Starts BCP training with the qwen3_30b_a3b/bcp_h20_1node preset. Override data_source.agent.retrieval_service_url if the service is not on the default localhost:9000. Checkpoints land in /root/ckpt/bcp_expert/train_step_/dist_ckpt. ```bash bash examples/start.sh qwen3_30b_a3b/bcp_h20_1node \ hf_model_path=/root/Qwen3-30B-A3B \ data_source.dataset.prompt_data_path=/root/browsecomp-plus/raw \ checkpoint_path=/root/ckpt/bcp_expert \ trainer.save_freq=10 ``` -------------------------------- ### Launch Qwen3-4B DAPO training preset Source: https://github.com/baidu-baige/loongsage/blob/main/README.md Launch the single-node 8-GPU preset qwen3_4b/dapo_h800_1node, passing the model and data paths on the command line. This starts the DAPO training run. ```bash bash examples/start.sh qwen3_4b/dapo_h800_1node \ hf_model_path=/root/Qwen3-4B \ data_source.dataset.prompt_data_path=/root/DAPO-Math-17k-unique ``` -------------------------------- ### Launch OPD with --config-name Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/on-policy-distillation.md Use this command to start the OPD training process. It references the configuration file conf/qwen3_30b_a3b/mopd_h20_1node.yaml. ```bash python -m coda.controller.trainer --config-name qwen3_30b_a3b/mopd_h20_1node ``` -------------------------------- ### Start Ray cluster and launch SWE training Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Starts the Ray cluster on every node, then submits the SWE training job from the head node only. Requires a kubeconfig for the K8s sandbox, placed at k8s/kubeconfig.yaml or overridden via data_source.sandbox.kubeconfig. ```bash # Run the same command on every machine bash examples/start_ray_cluster.sh # Once the cluster is ready, run this on the head node only bash examples/start.sh dsv4_flash_bf16/swe_h20_8node \ checkpoint_path=/root/ckpt/dsv4_swe \ hf_model_path=/root/DeepSeek-V4-Flash-BF16 \ data_source.dataset.prompt_data_path=/root/R2E-Gym-Subset \ data_source.sandbox.kubeconfig=/path/to/kubeconfig.yaml ``` -------------------------------- ### Install FAISS and download BCP dependencies Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Installs faiss-cpu (other dependencies ship with the training image) and downloads the embedding model, prompts, corpus, and dense indexes for BCP. ```bash python3 -m pip install faiss-cpu hf download Qwen/Qwen3-Embedding-8B --local-dir /root/Qwen3-Embedding-8B hf download Tevatron/browsecomp-plus --repo-type=dataset --include "data/*.parquet" --local-dir /root/browsecomp-plus/raw hf download Tevatron/browsecomp-plus-corpus --repo-type=dataset --local-dir /root/browsecomp-plus/corpus hf download Tevatron/browsecomp-plus-indexes --repo-type=dataset --local-dir /root/browsecomp-plus/indexes ``` -------------------------------- ### Build dense cache and start retrieval server Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Builds the dense cache from official shards and corpus, then starts the retrieval service. Both run entirely on CPU and occupy no GPU. ```bash python3 examples/bcp/build_dense_cache.py \ --index-path "/root/browsecomp-plus/indexes/qwen3-embedding-8b/corpus.shard*.pkl" \ --corpus-path /root/browsecomp-plus/corpus/data \ --output /root/browsecomp-plus/browsecomp_dense_cache.pkl ./examples/bcp/run_retrieval_server.sh \ --model /root/Qwen3-Embedding-8B \ --dense_cache /root/browsecomp-plus/browsecomp_dense_cache.pkl \ --port 9000 ``` -------------------------------- ### Start training with examples/start.sh Source: https://github.com/baidu-baige/loongsage/blob/main/README.md All training tasks use the same entry point examples/start.sh. Tasks are distinguished by Hydra config name, and any Hydra overrides can follow it. The script launches training in the background and writes to log/trainer_.log. ```bash bash examples/start.sh [key=value ...] ``` -------------------------------- ### Launch DAPO single-node 8-GPU preset Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Starts training with the qwen3_30b_a3b/dapo_h20_1node preset. trainer.save_freq defaults to -1 (no checkpoints), so it is set to 10 here. Checkpoints land in /root/ckpt/math_expert/train_step_/dist_ckpt. ```bash bash examples/start.sh qwen3_30b_a3b/dapo_h20_1node \ hf_model_path=/root/Qwen3-30B-A3B \ data_source.dataset.prompt_data_path=/root/DAPO-Math-17k-unique \ checkpoint_path=/root/ckpt/math_expert \ trainer.save_freq=10 ``` -------------------------------- ### Pull and start the LoongSage Docker container Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/quick-start.md Pulls the latest LoongSage Docker image and starts an interactive container with GPU access and host IPC. Use this to set up the environment before running training. ```bash docker pull loongsage/loongsage:latest docker run --rm --gpus all --ipc=host \ -it loongsage/loongsage:latest /bin/bash ``` -------------------------------- ### Start Ray cluster for OpenCode Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Starts the Ray cluster on all four H20 nodes for the OpenCode task. Run the same command on each node. ```bash bash examples/start_ray_cluster.sh ``` -------------------------------- ### Run LoongSage container Source: https://github.com/baidu-baige/loongsage/blob/main/README.md Start an interactive LoongSage container with GPU access, shared IPC, host networking, and a mounted workspace. The command mounts /path/to/workspace to /root inside the container. ```bash docker run -it --gpus all --ipc=host --network=host \ -v /path/to/workspace:/root loongsage/loongsage:latest bash ``` -------------------------------- ### Start retrieval server for BCP data source Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Runs the retrieval server required by the BCP data source for MOPD distillation. Uses the Qwen3-Embedding-8B model and a dense cache file on port 9000. ```bash ./examples/bcp/run_retrieval_server.sh \ --model /root/Qwen3-Embedding-8B \ --dense_cache /root/browsecomp-plus/browsecomp_dense_cache.pkl \ --port 9000 ``` -------------------------------- ### Single source configuration on command line Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Override a single data source using the data_source. prefix on the command line after the config name of start.sh. This example sets the prompt data path and sampling scale. ```bash bash examples/start.sh qwen3_30b_a3b/dapo_h20_1node \ hf_model_path=/root/Qwen3-30B-A3B \ data_source.dataset.prompt_data_path=/root/DAPO-Math-17k-unique \ data_source.num_prompts_per_step=64 \ data_source.num_trajectories_per_prompt=8 ``` -------------------------------- ### Minimal FixedCapacityStrategy Python example Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-sliding-window.md A minimal strategy that limits the number of concurrently dispatched prompts to a fixed capacity. It inherits from SlidingWindowStrategy and implements compute_dispatch_count. The capacity is read from config.data_sources[0].num_prompts_per_step. ```python @register_sliding_window_strategy("fixed-capacity") class FixedCapacityStrategy(SlidingWindowStrategy): def __init__(self, config): super().__init__() self._capacity = int(config.data_sources[0].num_prompts_per_step) def compute_dispatch_count(self, buf_qsize: int) -> int: return max(0, self._capacity - self._running_count - buf_qsize) ``` -------------------------------- ### Start Ray cluster on every node Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Run from the repository root on every participating machine. The script takes the head-node IP as its argument and auto-detects head vs. worker. Override NUM_GPUS=4 for 4-GPU nodes. ```bash bash examples/start_ray_cluster.sh ``` -------------------------------- ### Single teacher OPD configuration Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/on-policy-distillation.md YAML configuration for a single teacher distillation setup. Sets gkd_ratio to 1 for pure distillation, with teacher_nodes and teacher_gpus_per_node defining the teacher pool. The model section specifies tensor and pipeline model parallel sizes. ```yaml data_sources: - dataset: { prompt_data_path: /path/train.parquet } teacher_name: "math_expert" opd: enable: true pg_ratio: 0 gkd_ratio: 1 # pure distillation gkd_kl_method: full_kl teacher_nodes: 1 teacher_gpus_per_node: 8 teachers: - name: "math_expert" hf_path: /path/OpenThinker3-7B model: { tensor_model_parallel_size: 4, pipeline_model_parallel_size: 2 } ``` -------------------------------- ### Commit with conventional message format Source: https://github.com/baidu-baige/loongsage/blob/main/CONTRIBUTING.md Commit changes using the (): format. Example type: feat, fix, refactor, perf, docs, test, chore, ci; scope is the subpackage you touched. ```bash git commit -m "feat(algorithms): add token-level GSPO advantage estimator" ``` -------------------------------- ### Implement ExactMatchReward with @register_reward Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-reward.md Minimal example of a custom reward function that performs case-insensitive exact matching. Inherits from RewardFunction, reads case_sensitive from config, and returns a Reward with final_reward and is_correct. Place in coda/custom/ for automatic discovery. ```python # coda/custom/exact_match.py from omegaconf import DictConfig from coda.reward import register_reward from coda.reward.base import RewardFunction from coda.reward.reward import Reward @register_reward("exact-match") class ExactMatchReward(RewardFunction): def __init__(self, config: DictConfig) -> None: super().__init__(config) self.case_sensitive = bool(self.config.get("case_sensitive", False)) def __call__(self, messages: list[dict], label, context: dict, **kwargs) -> Reward: expected = label.get("answer") if isinstance(label, dict) else label if expected is None: return Reward(final_reward=0.0, is_valid=False, is_correct=False) predicted = next( ( message["content"] for message in reversed(messages) if message.get("role") == "assistant" and message.get("content") ), "", ) predicted = str(predicted).strip() expected = str(expected).strip() if not self.case_sensitive: predicted, expected = predicted.lower(), expected.lower() correct = predicted == expected return Reward(final_reward=float(correct), is_correct=correct) ``` -------------------------------- ### Launch training with start.sh (Method 1) Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/quick-start.md Runs the training launch script with the qwen3_4b/dapo_h800_1node config. This is the recommended method for the first run after filling in the config file. ```bash bash examples/start.sh qwen3_4b/dapo_h800_1node ``` -------------------------------- ### Report metrics to W&B via start.sh Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Run the training script with `tracking.tracking_backend=wandb` and pass `wandb_args` keys. `wandb_args` are mostly passed through to `wandb.Settings()`. The `mode` key accepts `shared` (default, requires `base_url` + `api_key`, W&B SDK ≥ 0.19.9 and W&B Server ≥ 0.68) or `offline` (writes locally only). ```bash bash examples/start.sh qwen3_30b_a3b/dapo_h20_1node \ hf_model_path=/root/Qwen3-30B-A3B \ data_source.dataset.prompt_data_path=/root/DAPO-Math-17k-unique \ tracking.tracking_backend=wandb \ tracking.wandb_args.base_url=https:// \ tracking.wandb_args.api_key= ``` -------------------------------- ### Enable fully async mode with DAPO config Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/fully-async-mode.md Run the trainer with the qwen3_30b_a3b/dapo_h20_1node config, setting colocate=false and fully_async.enable=true. Before running, replace hf_model_path, prompt_data_path, checkpoint_path, tracking address, and GPU topology in the config. ```bash python -m coda.controller.trainer --config-name qwen3_30b_a3b/dapo_h20_1node \ colocate=false fully_async.enable=true ``` -------------------------------- ### Download Qwen3-30B-A3B model and DAPO-Math dataset Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Downloads the model and dataset to /root directories. Required before launching the DAPO training preset. ```bash hf download Qwen/Qwen3-30B-A3B --local-dir /root/Qwen3-30B-A3B hf download Haitao999/DAPO-Math-17k-unique --repo-type=dataset --local-dir /root/DAPO-Math-17k-unique ``` -------------------------------- ### Report metrics to MLflow via start.sh Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Run the training script with `tracking.tracking_backend=mlflow` and set `mlflow_tracking_uri`, `project_name`, and `experiment_name`. `project_name` maps to the MLflow experiment and `experiment_name` to the run name; both default to `default`. If `mlflow_tracking_uri` is left empty, the run will not connect to a server. ```bash bash examples/start.sh qwen3_30b_a3b/dapo_h20_1node \ hf_model_path=/root/Qwen3-30B-A3B \ data_source.dataset.prompt_data_path=/root/DAPO-Math-17k-unique \ tracking.tracking_backend=mlflow \ tracking.mlflow_tracking_uri=http://:/ \ tracking.project_name=math-rl \ tracking.experiment_name=dapo-30b-a3b-lr1e6 ``` -------------------------------- ### Launch OpenCode black-box agent training Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Launches the OpenCode H20 4-node preset from the head node after all nodes have joined the Ray cluster. Requires a kubeconfig for the K8s sandbox. ```bash bash examples/start.sh qwen3_coder_30b_a3b/opencode_h20_4node \ checkpoint_path=/root/ckpt/opencode \ hf_model_path=/root/Qwen3-Coder-30B-A3B-Instruct \ data_source.dataset.prompt_data_path=/root/R2E-Gym-OpenCode/opencode \ data_source.sandbox.kubeconfig=/path/to/kubeconfig.yaml ``` -------------------------------- ### Download OpenCode model and dataset Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Downloads the Qwen3-Coder-30B-A3B-Instruct model and the prepared R2E-Gym dataset for OpenCode. The dataset already points to OpenCode-enabled sandbox images, so no additional data conversion is required. ```bash hf download Qwen/Qwen3-Coder-30B-A3B-Instruct --local-dir /root/Qwen3-Coder-30B-A3B-Instruct hf download loongsage/R2E-Gym --include "opencode/data/*.parquet" \ --repo-type=dataset --local-dir /root/R2E-Gym-OpenCode ``` -------------------------------- ### Launch MOPD multi-teacher distillation Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Launches the MOPD H20 1-node preset with two teachers (BCP and MATH) and their respective data sources. Requires the retrieval server from the previous step to be running. ```bash bash examples/start.sh qwen3_30b_a3b/mopd_h20_1node \ hf_model_path=/root/Qwen3-30B-A3B \ checkpoint_path=/root/ckpt/mopd \ data_sources.0.dataset.prompt_data_path=/root/browsecomp-plus/raw \ data_sources.1.dataset.prompt_data_path=/root/DAPO-Math-17k-unique \ opd.teachers.0.hf_path=/root/Qwen3-30B-A3B \ opd.teachers.0.dist_ckpt_path=/root/ckpt/bcp_expert/train_step_100/dist_ckpt \ opd.teachers.1.hf_path=/root/Qwen3-30B-A3B \ opd.teachers.1.dist_ckpt_path=/root/ckpt/math_expert/train_step_100/dist_ckpt ``` -------------------------------- ### Multiple teachers with different data sources Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/on-policy-distillation.md YAML configuration for routing different data sources to different teachers. Uses pg_ratio 0.5 and gkd_ratio 0.5 for a mix of PG and GKD, with pg_kl_method k1 and gkd_kl_method topk_kl with topk 64. ```yaml data_sources: - dataset: { prompt_data_path: /path/math.parquet } teacher_name: "math_expert" - dataset: { prompt_data_path: /path/code.parquet } teacher_name: "code_expert" opd: enable: true pg_ratio: 0.5 # mix of PG + GKD gkd_ratio: 0.5 pg_kl_method: k1 gkd_kl_method: topk_kl topk: 64 teachers: - name: "math_expert" hf_path: /path/math_model - name: "code_expert" hf_path: /path/code_model ``` -------------------------------- ### schedule() return value and actor instantiation Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/resource-scheduler.md Shows how schedule() returns a prepared actor class and bundle index, and how the caller instantiates the actor. The prepared actor is forced to num_cpus=0.1 and num_gpus=0.1, which do not reflect real usage. ```python prepared_actor, bundle_index = scheduler.schedule(ray_actor_cls, num_bundles=1, recover_bundle_index=-1) handle = prepared_actor.remote(...) # the caller instantiates it ``` -------------------------------- ### Run unit tests with build.sh Source: https://github.com/baidu-baige/loongsage/blob/main/CONTRIBUTING.md Run the full unit test suite with branch coverage, or target a specific directory or file. Coverage reports are written to output/coverage_html/. ```bash bash build.sh test # full unit test suite with branch coverage bash build.sh test tests/ut/algorithms # a single directory or file ``` -------------------------------- ### Tail training log Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Use this command to follow the training log file written by start.sh. The log file is named log/trainer_.log and the file name is printed on launch. ```bash tail -f log/trainer_*.log ``` -------------------------------- ### Fill in required config fields in dapo_h800_1node.yaml Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/quick-start.md Edit conf/qwen3_4b/dapo_h800_1node.yaml to replace the two '???' placeholders with the model path and dataset prompt data path. This is part of Method 1 for launching training. ```yaml hf_model_path: /root/Qwen3-4B data_source: dataset: prompt_data_path: /root/DAPO-Math-17k-unique/.parquet ``` -------------------------------- ### Fork and clone the repository Source: https://github.com/baidu-baige/loongsage/blob/main/CONTRIBUTING.md Clone your fork, navigate into the project directory, and add the official repository as an upstream remote. ```bash git clone https://github.com/your-name/LoongSage.git cd LoongSage # Track the official repository as upstream git remote add upstream https://github.com/baidu-baige/LoongSage.git ``` -------------------------------- ### Download Qwen3-Coder model and Codex R2E-Gym dataset Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Downloads the Qwen3-Coder-30B-A3B-Instruct model and the Codex variant of the R2E-Gym dataset. The dataset already points to Codex-enabled sandbox images, so no additional data conversion is required. ```bash hf download Qwen/Qwen3-Coder-30B-A3B-Instruct --local-dir /root/Qwen3-Coder-30B-A3B-Instruct hf download loongsage/R2E-Gym --include "codex/data/*.parquet" \ --repo-type=dataset --local-dir /root/R2E-Gym-Codex ``` -------------------------------- ### Download Qwen3-4B model and DAPO-Math dataset Source: https://github.com/baidu-baige/loongsage/blob/main/README.md Download the Qwen3-4B model and the DAPO-Math-17k-unique dataset using the hf CLI. The model is saved to /root/Qwen3-4B and the dataset to /root/DAPO-Math-17k-unique. ```bash hf download Qwen/Qwen3-4B --local-dir /root/Qwen3-4B hf download Haitao999/DAPO-Math-17k-unique --repo-type=dataset \ --local-dir /root/DAPO-Math-17k-unique ``` -------------------------------- ### Download Qwen3-4B model weights Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/quick-start.md Downloads the Qwen3-4B model weights from Hugging Face to /root/Qwen3-4B. Required for the DAPO preset. ```bash hf download Qwen/Qwen3-4B --local-dir /root/Qwen3-4B ``` -------------------------------- ### Build LoongSage Docker image locally Source: https://github.com/baidu-baige/loongsage/blob/main/README.md Build the LoongSage image from source using the Dockerfile in the docker/ directory. Use this if you need a custom build or cannot pull the pre-built image. ```bash docker build -t loongsage/loongsage:latest docker/ ``` -------------------------------- ### Download DAPO-Math-17k dataset Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/quick-start.md Downloads the DAPO-Math-17k-unique dataset from Hugging Face as a dataset repository to /root/DAPO-Math-17k-unique. Required for the DAPO math task. ```bash hf download Haitao999/DAPO-Math-17k-unique --repo-type=dataset \ --local-dir /root/DAPO-Math-17k-unique ``` -------------------------------- ### Multiple sources configuration on command line Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Override list elements by index on the command line. Fields not overridden keep their data_source defaults. ```bash bash examples/start.sh qwen3_30b_a3b/dapo_h20_1node \ hf_model_path=/root/Qwen3-30B-A3B \ checkpoint_path=/root/ckpt/multi \ data_sources.0.dataset.prompt_data_path=/root/browsecomp-plus/prompts/browsecomp_plus.parquet \ data_sources.1.dataset.prompt_data_path=/root/DAPO-Math-17k-unique ``` -------------------------------- ### Download DeepSeek-V4-Flash FP8 model and dataset, convert to BF16 Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Downloads the FP8 model and R2E-Gym dataset, then converts the model to BF16 weights for training. The conversion script is required before launching SWE training. ```bash hf download sgl-project/DeepSeek-V4-Flash-FP8 --local-dir /root/DeepSeek-V4-Flash-FP8 hf download R2E-Gym/R2E-Gym-Subset --repo-type=dataset --local-dir /root/R2E-Gym-Subset python examples/convert_dsv4_to_bf16.py \ --input-fp8-hf-path /root/DeepSeek-V4-Flash-FP8 \ --output-bf16-hf-path /root/DeepSeek-V4-Flash-BF16 ``` -------------------------------- ### YAML configuration to enable fixed-capacity strategy Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-sliding-window.md Configuration snippet showing how to reference the registered strategy by name in the fully_async section. The sliding_window key must match the name used in the @register_sliding_window_strategy decorator. ```yaml fully_async: enable: true sliding_window: fixed-capacity # ← the @register_sliding_window_strategy registered name stale_steps: 1.0 ``` -------------------------------- ### Run LoongSage unit tests with build.sh Source: https://github.com/baidu-baige/loongsage/blob/main/README.md Use these commands to run the full unit test suite with branch coverage, or target a specific test directory. The first command runs all tests, while the second runs tests only in the specified directory. ```bash bash build.sh test # Full unit tests and branch coverage bash build.sh test tests/ut/algorithms # Specific directory ``` -------------------------------- ### Launch Codex black-box agent preset Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Launches the Codex black-box agent preset (qwen3_coder_30b_a3b/codex_h20_4node) from the head node. Requires the checkpoint path, model path, dataset path, and kubeconfig path. ```bash bash examples/start.sh qwen3_coder_30b_a3b/codex_h20_4node \ checkpoint_path=/root/ckpt/codex \ hf_model_path=/root/Qwen3-Coder-30B-A3B-Instruct \ data_source.dataset.prompt_data_path=/root/R2E-Gym-Codex/codex \ data_source.sandbox.kubeconfig=/path/to/kubeconfig.yaml ``` -------------------------------- ### Launch Claude Code black-box agent preset Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Launches the Claude Code black-box agent preset (qwen3_coder_30b_a3b/claude_code_h20_2node) from the head node. Uses two H20 nodes (16 GPUs) to handle both rollout and training. ```bash bash examples/start.sh qwen3_coder_30b_a3b/claude_code_h20_2node \ checkpoint_path=/root/ckpt/claude_code \ hf_model_path=/root/Qwen3-Coder-30B-A3B-Instruct \ data_source.dataset.prompt_data_path=/root/R2E-Gym-ClaudeCode/claude_code \ data_source.sandbox.kubeconfig=/path/to/kubeconfig.yaml ``` -------------------------------- ### Download Qwen3-Coder model and Claude Code R2E-Gym dataset Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Downloads the Qwen3-Coder-30B-A3B-Instruct model and the Claude Code variant of the R2E-Gym dataset. The dataset already points to images with the Claude Code CLI preinstalled. ```bash hf download Qwen/Qwen3-Coder-30B-A3B-Instruct --local-dir /root/Qwen3-Coder-30B-A3B-Instruct hf download loongsage/R2E-Gym --include "claude_code/data/*.parquet" \ --repo-type=dataset --local-dir /root/R2E-Gym-ClaudeCode ``` -------------------------------- ### Directory structure for custom extensions in coda/custom/ Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-extensions.md Shows the expected layout of the coda/custom/ directory, including sub-packages. Each module uses a @register_* decorator to register its extension point. ```text coda/custom/ ├── my_reward.py # @register_reward("my-reward") ├── my_agent.py # @register_agent("my-agent") ├── my_advantage.py # @register_advantage("my-adv") └── my_project/ # sub-packages are scanned too ├── __init__.py └── sandbox.py # @register_sandbox("my-sandbox") ``` -------------------------------- ### Minimal custom sandbox backend in Python Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-sandbox.md Implements the stateless SandboxClient interface for a remote runtime. create() returns an ID, execute() returns stdout/stderr/exit_code/success, and delete() treats missing resources as success. Register with @register_sandbox("remote") and place in coda/custom/. ```python # coda/custom/remote_sandbox.py from typing import Any from coda.agentflow.sandbox import register_sandbox from coda.agentflow.sandbox.base import SandboxClient @register_sandbox("remote") class RemoteSandboxClient(SandboxClient): def __init__(self, working_dir: str = "/workspace", command_exec_timeout_seconds: int = 600, **kwargs: Any) -> None: self.working_dir = working_dir self.timeout = command_exec_timeout_seconds def create(self, image: str | None = None, **kwargs: Any) -> str: if not image: raise ValueError("sandbox image is required") return _start_runtime(image=image) def execute(self, sandbox_id: str, command: str, workdir: str | None = None, **kwargs: Any) -> dict[str, Any]: result = _exec_runtime( sandbox_id=sandbox_id, command=command, workdir=workdir or self.working_dir, timeout=self.timeout, ) return { "stdout": result.stdout, "stderr": result.stderr, "exit_code": result.exit_code, "success": result.exit_code == 0, } def delete(self, sandbox_id: str, **kwargs: Any) -> None: # Treat a "not found" backend as success so cleanup stays idempotent. _destroy_runtime(sandbox_id) ``` -------------------------------- ### Stop training with pkill and ray stop Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Run pkill -f coda.controller.trainer on the node that launched the training, then ray stop cleans up the Ray cluster. ```bash pkill -f coda.controller.trainer ``` ```bash ray stop ``` -------------------------------- ### Sync with upstream and push Source: https://github.com/baidu-baige/loongsage/blob/main/CONTRIBUTING.md Rebase your branch onto the latest upstream main and push it to your fork. ```bash git pull --rebase upstream main git push -u origin feature/your-feature-name ``` -------------------------------- ### YAML configuration for agent, reward, and sandbox Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-agent.md Configure a registered agent by name in the data source. The agent block passes fields directly to the constructor, reward is injected as reward_fn, and sandbox type remote enables AgentFlow injection. context_length limits the context window, max_response_len_per_trajectory sets the cumulative token budget, and sandbox none disables it. ```yaml data_source: agent: name: my-agent # ← the @register_agent registered name max_turns: 5 # maximum number of model requests context_length: 65536 # context window available while the agent completes one task reward: name: exact-match # ← injected as reward_fn max_response_len_per_trajectory: 32768 # cumulative token budget for replies and tool results sandbox: {type: remote} # ← AgentFlow injects client + id; none disables it ``` -------------------------------- ### Multiple sources configuration in YAML Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Configure multiple data sources by writing data_sources as an explicit list. Each element can configure its own prompt data, sampling scale, agent, and reward; fields left unset inherit the data_source defaults. ```yaml defaults: - /qwen3_30b_a3b/dapo_h20_1node - _self_ data_sources: - dataset: prompt_data_path: /root/browsecomp-plus/prompts/browsecomp_plus.parquet input_key: prompt label_key: answer num_prompts_per_step: 32 num_trajectories_per_prompt: 8 agent: name: bcp retrieval_service_url: http://127.0.0.1:9000 max_turns: 15 reward: name: bcp max_response_len_per_trajectory: 10240 - dataset: prompt_data_path: /root/DAPO-Math-17k-unique input_key: prompt label_key: reward_model num_prompts_per_step: 32 num_trajectories_per_prompt: 8 reward: name: dapo_math overlong_penalty_length: 4096 max_response_len_per_trajectory: 20480 ``` -------------------------------- ### Register a custom policy loss with @register_policy_loss Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-algorithm.md Defines a custom policy loss named 'vanilla_pg' that computes a policy gradient loss using the ratio of new to old log probabilities. The function signature is (config, old_log_prob, log_prob, advantages, loss_masks, **kwargs) -> (per_token_loss, metrics). It returns a per-token loss list and a metrics dict with a loss-mask weighted sum for 'approx_kl'. ```python import torch from coda.algorithms.registry import register_policy_loss @register_policy_loss("vanilla_pg") def vanilla_pg(config, old_log_prob, log_prob, advantages, loss_masks, **kwargs): lengths = [len(t) for t in log_prob] ratio = torch.exp(torch.cat(log_prob) - torch.cat(old_log_prob)) per_token_loss = -(ratio * torch.cat(advantages)) approx_kl = ((torch.cat(old_log_prob) - torch.cat(log_prob)) * torch.cat(loss_masks)).sum() return list(per_token_loss.split(lengths)), {"approx_kl": approx_kl.item()} ``` -------------------------------- ### Define and register a custom agent with sandbox (my_agent.py) Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-agent.md Defines a custom agent class 'MyAgent' registered as 'my-agent'. It uses a sandbox to execute shell commands and parses tool calls from the Chat Completions response. The agent runs for a maximum number of turns and returns a reward based on the final messages. ```python # coda/custom/my_agent.py import asyncio import json from typing import Any from coda.agentflow.agent import register_agent from coda.agentflow.agent.base_agent import BaseAgent from coda.reward.reward import Reward @register_agent("my-agent") class MyAgent(BaseAgent): def __init__(self, *args: Any, max_turns: int = 5, **kwargs: Any) -> None: super().__init__(*args, **kwargs) self.max_turns = max_turns async def run_trajectory(self, trajectory: dict[str, Any]) -> Reward: label = trajectory.get("label") metadata = trajectory.get("metadata") or {} messages = list(trajectory["prompt"]) if self.sandbox_id is None: self.sandbox_id = await asyncio.to_thread(self.sandbox_client.create, image=metadata["docker_image"]) for _ in range(self.max_turns): response = await self.call_llm(messages) assistant = response["choices"][0]["message"] messages.append(assistant) tool_calls = assistant.get("tool_calls") or [] if not tool_calls: break for tc in tool_calls: # This example registers a single shell tool; dispatch on # tc["function"]["name"] when there are several. arguments = json.loads(tc["function"]["arguments"]) result = await asyncio.to_thread( self.sandbox_client.execute, self.sandbox_id, arguments["command"]) messages.append({ "role": "tool", "tool_call_id": tc["id"], "content": f"{result['stdout']}{result['stderr']}", }) return await asyncio.to_thread( self.reward_fn, messages, label, {**metadata, "sandbox_client": self.sandbox_client, "sandbox_id": self.sandbox_id}, ) ``` -------------------------------- ### Teacher weights from Megatron dist checkpoint Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/on-policy-distillation.md YAML snippet showing how to set dist_ckpt_path to load teacher weights from a Megatron dist checkpoint. hf_path remains required for building the model from config.json, but its weights are not read. The checkpoint must match the architecture; otherwise a load error occurs. ```yaml teachers: - name: "math_expert" hf_path: /path/OpenThinker3-7B # weights: one concrete dist_ckpt dir, you pick the step. dist_ckpt_path: /path/run/train_step_100/dist_ckpt ``` -------------------------------- ### Register a custom KL policy with @register_kl_policy Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-kl.md Inherit a base class (e.g., TopkVocabPolicy) and implement compute_kl to return per-token KL and extra metrics. Place the module under coda/custom/ for automatic discovery. ```python from coda.algorithms.kl_policy import TopkVocabPolicy from coda.algorithms.registry import register_kl_policy @register_kl_policy("my_kl") class MyKL(TopkVocabPolicy): # or LogProbKLPolicy / FullVocabPolicy def compute_kl(self, config, ctx): ... # returns (per_token_kl, extra_metrics) ``` -------------------------------- ### Sandbox configuration in YAML Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-sandbox.md Selects the registered sandbox backend by name under data_source.sandbox. All fields except type are passed to from_config(). Set type: none to disable the sandbox for a data source. ```yaml data_source: sandbox: type: remote # ← the @register_sandbox registered name working_dir: /workspace # default working directory for tool calls command_exec_timeout_seconds: 600 # timeout for one tool call, in seconds ``` -------------------------------- ### Verify Ray cluster status Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Run from the head node after all nodes have joined to confirm the cluster is ready. ```bash ray status ``` -------------------------------- ### Checkpoint directory layout Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Shows the directory structure under `checkpoint_path` when checkpoints are enabled. The tracker file `latest_checkpointed_iteration.txt` records the last successfully saved step. `train_step_N/dist_ckpt` holds the distributed checkpoint, `hf_model` holds the HuggingFace export when `trainer.save_hf=true`, and `data_source` stores the data-source cursor and buffer snapshot for resume. ```text checkpoint_path/ ├── latest_checkpointed_iteration.txt # tracker file: last successfully saved step ├── train_step_10/ │ ├── dist_ckpt/ # distributed checkpoint (model + optimizer + scheduler + RNG) │ │ # also what opd.teachers[].dist_ckpt_path points at in MOPD │ ├── hf_model/ # HuggingFace export (when trainer.save_hf=true) │ └── data_source/ # data-source cursor + buffer snapshot, for resume └── ... ``` -------------------------------- ### Create a development branch Source: https://github.com/baidu-baige/loongsage/blob/main/CONTRIBUTING.md Update the main branch from upstream and create a new feature branch for your changes. ```bash git checkout main git pull upstream main git checkout -b feature/your-feature-name ``` -------------------------------- ### Reference custom KL method in opd configuration Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-kl.md Set opd.gkd_kl_method to the registered name (e.g., my_kl). The topk key is needed by the top-k family. PG uses only the KL value (signed and unbiased), GKD only its gradient. ```yaml opd: enable: true gkd_ratio: 1 gkd_kl_method: my_kl # ← the @register_kl_policy registered name topk: 64 # needed by the top-k family ``` -------------------------------- ### Pull LoongSage Docker image Source: https://github.com/baidu-baige/loongsage/blob/main/README.md Pull the pre-built LoongSage image from Docker Hub. The image includes aligned dependencies for CUDA, PyTorch, Megatron-Core, Megatron-Bridge, SGLang, and Ray, with necessary SGLang patches applied. ```bash docker pull loongsage/loongsage:latest ``` -------------------------------- ### Multi-GPU colocated topology diagram Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/transfer-mesh.md ASCII diagram showing the topology for a multi-GPU colocated scenario with world_size=8, where each sender rank (0-3) uses IPC to transfer to a corresponding receiver rank (4-7) on the same GPU. ```text Topology (world_size=8): GPU 0: Rank 0 (sender) ──IPC──> Rank 4 (receiver) GPU 1: Rank 1 (sender) ──IPC──> Rank 5 (receiver) GPU 2: Rank 2 (sender) ──IPC──> Rank 6 (receiver) GPU 3: Rank 3 (sender) ──IPC──> Rank 7 (receiver) ``` -------------------------------- ### Configure reward in data_source.reward Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/custom-reward.md YAML configuration block that enables the registered reward by name. The full block is passed to the reward constructor. Each data_sources[i] can select a different reward. ```yaml data_source: reward: name: exact-match # ← the @register_reward registered name case_sensitive: false # ignore letter case during exact matching ``` -------------------------------- ### Single source configuration in YAML Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Configure a single data source in an experiment YAML file. The data_source block overrides the default template with the specified prompt data path, input key, label key, and sampling scale. ```yaml defaults: - /qwen3_30b_a3b/dapo_h20_1node - _self_ data_source: dataset: prompt_data_path: /root/DAPO-Math-17k-unique input_key: prompt label_key: reward_model num_prompts_per_step: 64 num_trajectories_per_prompt: 8 ``` -------------------------------- ### update_plan plan argument correct shape Source: https://github.com/baidu-baige/loongsage/blob/main/coda/agentflow/agent/codex/prompt.md The plan argument must be an array of objects, each with a step string and a status of pending, in_progress, or completed. The optional explanation belongs at the top level. Never send plan as a numbered string, nest another plan object, or add top-level step and status fields. ```json { "explanation": "The implementation is understood; next I am adding the regression test.", "plan": [ {"step": "Inspect the implementation", "status": "completed"}, {"step": "Implement the fix", "status": "completed"}, {"step": "Add regression coverage", "status": "in_progress"}, {"step": "Run focused tests", "status": "pending"} ] } ``` -------------------------------- ### Validate retrieval service with health check and query Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/run-guide.md Checks the service health and performs a real retrieval query. The query uses the default port 9000. ```bash curl -sS http://127.0.0.1:9000/health curl -sS http://127.0.0.1:9000/retrieve \ -H 'Content-Type: application/json' \ -d '{"queries":["Tribeca Festival Gotham Week"],"topk":3,"return_scores":true}' ``` -------------------------------- ### Receiver: create channel and receive tensors Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/transfer-mesh.md Creates a TransferMesh channel with role RECEIVER and receives tensors as an iterator, loading each weight into the model, then closes the channel. ```python channel = create_channel( master_addr="127.0.0.1", addr="127.0.0.1", gpu_id=0, world_size=2, rank=1, role=Role.RECEIVER, src_rank=0, ) # Receive tensors as iterator for name, tensor in channel.get_iterator(): model.load_single_weight(name, tensor) channel.close() ``` -------------------------------- ### Multi-GPU colocated sender and receiver channel creation Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/transfer-mesh.md Shows how to create channels for sender processes (rank 0-3) and receiver processes (rank 4-7) in a multi-GPU colocated scenario. Senders use gpu_id=rank, receivers use gpu_id=rank-4 to match corresponding sender. TransferMesh automatically uses IPC for zero-copy transfer. ```python # Each sender process (rank 0-3) channel = create_channel( master_addr="10.0.0.1", addr="10.0.0.1", gpu_id=rank, # Same as corresponding receiver's gpu_id world_size=8, rank=rank, role=Role.SENDER, src_rank=0, ) ``` ```python # Each receiver process (rank 4-7) channel = create_channel( master_addr="10.0.0.1", addr="10.0.0.1", gpu_id=rank - 4, # Matches corresponding sender world_size=8, rank=rank, role=Role.RECEIVER, src_rank=0, ) ``` -------------------------------- ### Receiver-side get_iterator() bucket unpacking loop Source: https://github.com/baidu-baige/loongsage/blob/main/docs/en/transfer-mesh.md Iterates over tensor specs in the metadata, extracts the corresponding byte slice from the raw payload buffer, views it back to the original dtype and shape, and clones it to create an independent GPU copy. The clone is essential because the sender may release or overwrite the shared bucket after the next send(). ```python for spec in meta.tensor_specs: tensor_dtype = str_to_dtype(spec.dtype) num_bytes = spec.numel() * tensor_dtype.itemsize raw_bytes = payload[spec.offset:spec.offset + num_bytes] tensor = raw_bytes.view(tensor_dtype).view(spec.shape).clone() yield (spec.name, tensor) ```