### Complete On-Policy Distillation Example

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/distillation.md

This example demonstrates a full on-policy distillation setup using the `tinker-cookbook` library. It configures datasets, teacher models, and training parameters for distillation.

```python
import asyncio
import chz

from tinker_cookbook import checkpoint_utils, cli_utils
from tinker_cookbook.distillation import train_on_policy
from tinker_cookbook.distillation.datasets import (
    DistillationDatasetConfig, PromptOnlyDatasetBuilder,
    TeacherConfig,
)

@chz.chz
class CLIConfig:
    model_name: str = "Qwen/Qwen3-8B-Base"
    teacher_model: str = "Qwen/Qwen3-8B"
    dataset: str = "deepmath"
    group_size: int = 4
    groups_per_batch: int = 1024
    learning_rate: float = 1e-4
    max_tokens: int = 4096
    kl_penalty_coef: float = 1.0
    kl_discount_factor: float = 0.0
    lora_rank: int = 128
    renderer_name: str | None = None
    load_checkpoint_path: str | None = None
    log_path: str | None = None
    behavior_if_log_dir_exists: cli_utils.LogdirBehavior = "ask"
    max_steps: int | None = None

async def cli_main(cli_config: CLIConfig):
    renderer_name = await checkpoint_utils.resolve_renderer_name_from_checkpoint_or_default_async(
        model_name=cli_config.model_name,
        explicit_renderer_name=cli_config.renderer_name,
        load_checkpoint_path=cli_config.load_checkpoint_path,
    )
    dataset_builder = PromptOnlyDatasetBuilder(
        dataset_name=cli_config.dataset,
        groups_per_batch=cli_config.groups_per_batch,
        group_size=cli_config.group_size,
        model_name_for_tokenizer=cli_config.model_name,
        renderer_name=renderer_name,
    )
    teacher_config = TeacherConfig(base_model=cli_config.teacher_model)
    dataset_config = DistillationDatasetConfig(
        dataset_builder=dataset_builder,
        teacher_config=teacher_config,
        groups_per_batch=cli_config.groups_per_batch,
    )
    log_path = cli_config.log_path or f"/tmp/tinker-examples/distillation/{cli_config.dataset}"
    cli_utils.check_log_dir(log_path, behavior_if_exists=cli_config.behavior_if_log_dir_exists)
    config = train_on_policy.Config(
        dataset_configs=[dataset_config],
        model_name=cli_config.model_name,
        renderer_name=renderer_name,
        learning_rate=cli_config.learning_rate,
        lora_rank=cli_config.lora_rank,
        max_tokens=cli_config.max_tokens,
        kl_penalty_coef=cli_config.kl_penalty_coef,
        kl_discount_factor=cli_config.kl_discount_factor,
        log_path=log_path,
        max_steps=cli_config.max_steps,
    )
    await train_on_policy.main(config)

if __name__ == "__main__":
    cli_config = chz.entrypoint(CLIConfig)
    asyncio.run(cli_main(cli_config))
```

--------------------------------

### Train Model with Modal Sandbox

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/code_rl/README.md

Example command to train a Qwen3-4B-Instruct-2507 model, specifying 'modal' as the sandbox backend. Adjust other parameters as needed for your training setup.

```bash
python -m tinker_cookbook.recipes.code_rl.train \
    sandbox_backend=modal \
    ...
```

--------------------------------

### Install Verifiers Environments

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/verifiers_rl/README.md

Install community-implemented RL environments from the Environments Hub using the `prime` CLI. Ensure you have `prime` installed or install it using `uv tool install prime`.

```bash
uv tool install prime # or pipx install prime
prime env install user/env-id # ex. prime env install primeintellect/reverse-text
```

--------------------------------

### Install tinker-cookbook for Math RL

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/math_rl/README.md

Installs the tinker-cookbook library with math-rl dependencies from a Git repository. Ensure you have `uv` installed.

```bash
uv pip install 'tinker-cookbook[math-rl] @ git+https://github.com/thinking-machines-lab/tinker-cookbook.git@nightly'
```

--------------------------------

### Run Calculator Tool Use Example

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/rl.md

This snippet shows how to run a full multi-turn example using a calculator tool. It configures and runs the main asynchronous function with provided CLI arguments.

```python
renderer_name=renderer_name,
        lora_rank=cli_config.lora_rank,
        max_tokens=cli_config.max_tokens,
        kl_penalty_coef=cli_config.kl_penalty_coef,
        log_path=cli_config.log_path,
        max_steps=cli_config.max_steps,
    )
    await main(config)


if __name__ == "__main__":
    cli_config = chz.entrypoint(CLIConfig)
    asyncio.run(cli_main(cli_config))
```

--------------------------------

### Clone and Set Up tinker-cookbook

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/dev.md

Clone the repository and install development dependencies using uv.

```bash
git clone https://github.com/thinking-machines-lab/tinker-cookbook.git
cd tinker-cookbook
uv sync --extra dev
pre-commit install
```

--------------------------------

### Install Tinker Cookbook with Modal support

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/harbor_rl/README.md

Installs the tinker-cookbook library, including Modal support, from a Git repository. Ensure you have uv installed.

```bash
uv pip install 'tinker-cookbook[modal] @ git+https://github.com/thinking-machines-lab/tinker-cookbook.git@nightly'
```

--------------------------------

### Setup vLLM Test Environment

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tests/weights/vllm_serving/README.md

Execute this script to set up the isolated Python virtual environment for vLLM testing. Alternatively, manually create and activate a venv, then install dependencies and the project.

```bash
bash tests/weights/vllm_serving/setup_env.sh
```

```bash
python3 -m venv /tmp/vllm-test-env
/tmp/vllm-test-env/bin/pip install -r tests/weights/vllm_serving/requirements.txt
/tmp/vllm-test-env/bin/pip install -e .
```

```bash
VLLM_TEST_ENV=~/my-vllm-env bash tests/weights/vllm_serving/setup_env.sh
```

--------------------------------

### Direct Preference Optimization (DPO) Setup

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/SKILL.md

Initializes common configuration for Direct Preference Optimization (DPO) training, focusing on preference data alignment. This setup is a precursor to the actual DPO training process.

```python
from tinker_cookbook import model_info
from tinker_cookbook.preference import train_dpo
from tinker_cookbook.preference.dpo_datasets import DPODatasetBuilderFromComparisons
from tinker_cookbook.recipes.preference.datasets import HHHComparisonBuilder
from tinker_cookbook.supervised.types import ChatDatasetBuilderCommonConfig

model_name = "meta-llama/Llama-3.2-1B"
renderer_name = model_info.get_recommended_renderer_name(model_name)
common_config = ChatDatasetBuilderCommonConfig(
    model_name_for_tokenizer=model_name,
    renderer_name=renderer_name,
    max_length=8192, batch_size=256,
)
```

--------------------------------

### Install tinker-cookbook

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/README.md

Install the latest stable release or the nightly build of tinker-cookbook using uv pip.

```bash
uv pip install tinker-cookbook
```

```bash
uv pip install 'tinker-cookbook @ git+https://github.com/thinking-machines-lab/tinker-cookbook.git@nightly'
```

--------------------------------

### Setup vLLM Serving Environment

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/weights/README.md

Prepare the environment for vLLM serving tests, which requires a GPU and an isolated virtual environment.

```bash
bash tests/weights/vllm_serving/setup_env.sh
```

--------------------------------

### Build Supervised Example with Renderer

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sft.md

Use `build_supervised_example` to prepare data for training. Configure `train_on_what` to specify which parts of the conversation to train on.

```python
model_input, weights = renderer.build_supervised_example(
    messages, train_on_what=TrainOnWhat.ALL_ASSISTANT_MESSAGES
)
```

--------------------------------

### Set Up Development Environment

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/README.md

Install development dependencies and set up pre-commit hooks for code formatting and linting using ruff. These checks are enforced on commits and in CI.

```bash
uv sync --extra dev
pre-commit install
```

--------------------------------

### Clone repository and install dependencies

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/CONTRIBUTING.md

Clone the Tinker Cookbook repository and install development dependencies using uv.

```bash
git clone https://github.com/thinking-machines-lab/tinker-cookbook.git
cd tinker-cookbook
uv sync --extra dev
```

--------------------------------

### Install pre-commit hooks

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/CONTRIBUTING.md

Install pre-commit hooks to automatically run ruff formatting and linting on each commit.

```bash
pre-commit install
```

--------------------------------

### Install Tinker and Dependencies

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tutorials/README.md

Installs the necessary Tinker libraries and Marimo. Ensure your TINKER_API_KEY is exported before running.

```bash
uv pip install tinker tinker-cookbook marimo
export TINKER_API_KEY="your-api-key-here"
```

--------------------------------

### Run xmux Example Scripts

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/xmux/README.md

Execute example scripts to demonstrate xmux functionality. Use --dry-run to preview actions without actual execution. Different arguments control grouping and scale.

```bash
# Run demo with dry-run to see what would happen
python examples/ml_sweep.py 1 --dry-run
```

```bash
# Run actual experiments
python examples/ml_sweep.py 2
```

--------------------------------

### Install multiplayer-rl package

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/multiplayer_rl/text_arena/README.md

Installs the tinker-cookbook with multiplayer-rl capabilities from a git repository. Ensure you have uv or pip installed.

```bash
uv pip install 'tinker-cookbook[multiplayer-rl] @ git+https://github.com/thinking-machines-lab/tinker-cookbook.git@nightly'
```

--------------------------------

### Complete RLHF Example in Python

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/preferences.md

This script orchestrates the entire RLHF pipeline, including supervised fine-tuning, reward model training, and reinforcement learning. It defines configuration parameters and calls the respective training functions. Ensure all necessary libraries are installed and checkpoints are available for subsequent stages.

```python
import asyncio
from pathlib import Path

import chz

from tinker_cookbook import checkpoint_utils, model_info
from tinker_cookbook.preference.preference_datasets import ChatDatasetBuilderFromComparisons
from tinker_cookbook.preference.types import PreferenceModelBuilderFromChatRenderer
from tinker_cookbook.recipes.chat_sl.chat_datasets import NoRobotsBuilder
from tinker_cookbook.recipes.preference.datasets import HHHComparisonBuilder
from tinker_cookbook.renderers import TrainOnWhat
from tinker_cookbook.rl import preference_envs, train
from tinker_cookbook.supervised import train as supervised_train
from tinker_cookbook.supervised.types import ChatDatasetBuilderCommonConfig


@chz.chz
class CLIConfig:
    base_model: str = "meta-llama/Llama-3.2-3B"
    lora_rank: int = 64
    batch_size: int = 256
    max_length: int = 16384
    sft_learning_rate: float = 2e-4
    rm_learning_rate: float = 3e-4
    rl_learning_rate: float = 1e-5
    rl_max_tokens: int = 1024
    rl_group_size: int = 4
    max_steps: int | None = None


def sft_stage(cli: CLIConfig, log_path: str):
    renderer_name = model_info.get_recommended_renderer_name(cli.base_model)
    common = ChatDatasetBuilderCommonConfig(
        model_name_for_tokenizer=cli.base_model, renderer_name=renderer_name,
        max_length=cli.max_length, batch_size=cli.batch_size,
        train_on_what=TrainOnWhat.ALL_ASSISTANT_MESSAGES,
    )
    config = supervised_train.Config(
        log_path=log_path, model_name=cli.base_model, renderer_name=renderer_name,
        dataset_builder=NoRobotsBuilder(common_config=common),
        learning_rate=cli.sft_learning_rate, lr_schedule="linear",
        num_epochs=1, lora_rank=cli.lora_rank, max_steps=cli.max_steps,
    )
    asyncio.run(supervised_train.main(config))


def train_rm(cli: CLIConfig, log_path: str):
    renderer_name = model_info.get_recommended_renderer_name(cli.base_model)
    common = ChatDatasetBuilderCommonConfig(
        model_name_for_tokenizer=cli.base_model, renderer_name=renderer_name,
        max_length=cli.max_length, batch_size=cli.batch_size,
    )
    config = supervised_train.Config(
        log_path=log_path, model_name=cli.base_model, renderer_name=renderer_name,
        dataset_builder=ChatDatasetBuilderFromComparisons(
            common_config=common, comparison_builder=HHHComparisonBuilder()
        ),
        learning_rate=cli.rm_learning_rate, lr_schedule="linear",
        num_epochs=1, lora_rank=cli.lora_rank, max_steps=cli.max_steps,
    )
    asyncio.run(supervised_train.main(config))


async def train_rl(cli: CLIConfig, log_path: str, sft_log: str, rm_log: str):
    sft_ckpt = checkpoint_utils.get_last_checkpoint(sft_log)
    rm_ckpt = checkpoint_utils.get_last_checkpoint(rm_log)
    assert sft_ckpt and rm_ckpt, "SFT and RM checkpoints required"

    renderer_name = model_info.get_recommended_renderer_name(cli.base_model)
    preference_model = PreferenceModelBuilderFromChatRenderer(
        renderer_name=renderer_name, model_name=cli.base_model,
        rm_weights_path=rm_ckpt.sampler_path,
    )
    config = train.Config(
        model_name=cli.base_model, renderer_name=renderer_name,
        dataset_builder=preference_envs.PairwisePreferenceRLDatasetBuilder(
            comparison_builder=HHHComparisonBuilder(),
            policy_renderer_name=renderer_name, policy_model_name=cli.base_model,
            preference_model_builder=preference_model,
            batch_size=cli.batch_size, group_size=cli.rl_group_size,
            tournament_pattern=preference_envs.TournamentPattern.ALL_PAIRS_BOTH_WAYS,
        ),
        load_checkpoint_path=sft_ckpt.state_path,
        learning_rate=cli.rl_learning_rate, max_tokens=cli.rl_max_tokens,
        lora_rank=cli.lora_rank, log_path=log_path, max_steps=cli.max_steps,
    )
    await train.main(config)


def cli_main(cli_config: CLIConfig):
    root = Path(f"/tmp/tinker-examples/rlhf")
    sft_stage(cli_config, str(root / "sft"))
    train_rm(cli_config, str(root / "rm"))
    asyncio.run(train_rl(cli_config, str(root / "rl"), str(root / "sft"), str(root / "rm")))


if __name__ == "__main__":
    cli_config = chz.entrypoint(CLIConfig)
    cli_main(cli_config)

```

--------------------------------

### Start Sandbox Fusion Docker Container

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/code_rl/README.md

Use this command to start a local Sandbox Fusion instance in Docker. Ensure the TINKER_COOKBOOK_ROOT environment variable is set correctly to mount the configuration file.

```bash
docker run -it -p 8080:8080 \
    -v ${TINKER_COOKBOOK_ROOT}/tinker_cookbook/recipes/code_rl/sandbox_config/local.yaml:/root/sandbox/sandbox/configs/local.yaml \
    volcengine/sandbox-fusion:server-20250609
```

--------------------------------

### Quick Start LiteLLM Integration

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/third_party/litellm/README.md

Register the LiteLLM provider and make an asynchronous completion call using a Tinker model.

```python
from tinker_cookbook.third_party.litellm import register_litellm_provider
import litellm

# Register once at startup
register_litellm_provider()

# The "tinker/" prefix routes to this provider.
# base_model is the Tinker model to sample from.
response = await litellm.acompletion(
    model="tinker/my-label",
    messages=[{"role": "user", "content": "Hello!"}],
    base_model="Qwen/Qwen3-4B-Instruct-2507",
    temperature=0.7,
    max_tokens=256,
)

print(response.choices[0].message.content)
```

--------------------------------

### Install aiomonitor

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/debug/references/async-task-dump.md

Install the aiomonitor library using pip. This is required for optional interactive async debugging features.

```bash
pip install aiomonitor
```

--------------------------------

### Generate Example Dataset

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/rubric/README.md

Run this command to generate a dataset for the addition task. The generated JSONL files contain conversation prefixes and rubric items for grading.

```bash
python -m tinker_cookbook.recipes.rubric.generate_data
```

--------------------------------

### Install Modal Extra and Authenticate

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/code_rl/README.md

Install the necessary Modal extra for Tinker Cookbook and authenticate your Modal account. This is required for using Modal as a sandbox backend.

```bash
uv pip install 'tinker-cookbook[modal] @ git+https://github.com/thinking-machines-lab/tinker-cookbook.git@nightly'
modal token new
```

--------------------------------

### Setup Custom ML Logging (Python)

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/ops.md

Initializes a custom ML logger using `ml_log.setup_logging`. Specify `log_path` for output. `wandb_project` and `wandb_name` can be set for Weights & Biases integration.

```python
from tinker_cookbook.utils import ml_log
ml_logger = ml_log.setup_logging(log_path="/tmp/my_run", wandb_project=None, wandb_name=None)
ml_logger.log_metrics({"train/loss": 0.5, "eval/accuracy": 0.85}, step=100)
```

--------------------------------

### Example Math Problem and Solution (MATH dataset)

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/math_rl/README.md

An example of a math problem from the MATH dataset, presented in a user-assistant format, followed by a detailed step-by-step reasoning process and the final answer in a boxed format.

```text
<|im_start|>user
The numbers 2, 3, 5, 7, 11, 13 are arranged in a multiplication table, with three along the top and the other three down the left.  The multiplication table is completed a
and the sum of the nine entries is tabulated.  What is the largest possible sum of the nine entries?

\[
\begin{array}{c||c|c|c|}
\times & a & b & c \\ \hline \hline
d & & & \\ \hline
e & & & \\ \hline
f & & & \\ \hline
\end{array}
\] Write your answer in \boxed{} format.<|im_end|>
<|im_start|>assistant
<think>
Okay, so we have to arrange the primes 2,3,5,7,11,13 into a 3x3 multiplication table. The sum of all entries is (a+b+c)*(d+e+f) - because the sum of the products is the pr
oduct of the sums. To maximize the total sum, we need to maximize the product of the sums of the two groups. So, split the numbers into two groups of three, one group for
the top row (a,b,c) and one for the left column (d,e,f). The product (sum1)*(sum2) is maximized when the sums are as close as possible? Or maybe when one is as large as po
ssible and the other as small? Wait, for a fixed sum, the product is maximized when they're equal. But since the total sum of all numbers is 2+3+5+7+11+13= 41. So sum1 + s
um2 =41. To maximize sum1*sum2, we need sum1 and sum2 as close as possible to 41/2=20.5. So find two subsets of three numbers that add up to around 20.5.

Let me try combinations. Let's see:

Possible sums for three numbers:

- 2+3+13=18

- 2+5+13=20
- 2+7+11=20

- 3+5+13=21

- 3+7+11=21

- 5+7+11=23

So the best possible is sum1=20 and sum2=21, product 420. So the maximum sum is 420. \boxed{420}</think>

\boxed{420}<|im_end|>
```

--------------------------------

### Install LiteLLM Dependency

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/third_party/litellm/README.md

Install the litellm extra dependency from the tinker-cookbook repository.

```bash
uv pip install -e ".[litellm]"
```

--------------------------------

### Set Up Tinker Environment Variables

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/SKILL.md

This bash snippet shows how to set the necessary environment variables for using the Tinker SDK and CLI. It includes installing the SDK, cloning the cookbook repository, and installing it locally.

```bash
export TINKER_API_KEY=<your-key>
pip install tinker                                    # SDK + CLI
git clone https://github.com/thinking-machines-lab/tinker-cookbook.git
cd tinker-cookbook && pip install -e .                 # Cookbook
```

--------------------------------

### Manage Training Runs and Checkpoints

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sdk.md

Provides examples for accessing information about training runs and checkpoints, including IDs, types, and paths. It also shows how to parse Tinker paths.

```python
from tinker import TrainingRun, Checkpoint, CheckpointType, ParsedCheckpointTinkerPath

# TrainingRun
run.training_run_id, run.base_model, run.is_lora, run.lora_rank
run.last_checkpoint, run.user_metadata

# Checkpoint
ckpt.checkpoint_id, ckpt.checkpoint_type, ckpt.tinker_path
ckpt.size_bytes, ckpt.public, ckpt.expires_at

# Parse a tinker:// path
parsed = ParsedCheckpointTinkerPath.from_tinker_path("tinker://run-id/weights/ckpt-id")
parsed.training_run_id, parsed.checkpoint_type, parsed.checkpoint_id
```

--------------------------------

### Train Twenty Questions Agent

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/multiplayer_rl/twenty_questions/README.md

Run this command to start the training process for the Twenty Questions agent. The `total` reward metric in `test/env/all/reward/total` is expected to increase after training.

```bash
python -m tinker_cookbook.recipes.multiplayer_rl.twenty_questions.train
```

--------------------------------

### Expected DPO Training Metrics Output

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/preference/dpo/README.md

Example of training metrics displayed after 50 steps of DPO training, including accuracy, loss, and reward values for chosen and rejected responses.

```text
                   Step 50
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Metric                         ┃ Value     ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ accuracy                       │ 0.568627  │
│ batch_time                     │ 27.953704 │
│ chosen_reward                  │ 0.053621  │
│ dpo_loss                       │ 0.683825  │
│ learning_rate                  │ 0.000009  │
│ margin                         │ 0.002147  │
│ num_pairs                      │ 255       │
│ num_tokens                     │ 112638    │
│ progress                       │ 0.081210  │
│ rejected_reward                │ 0.032152  │
│ test/nll                       │ 1.871778  │
└────────────────────────────────┴───────────┘
```

--------------------------------

### Basic Tinker Chat CLI Usage

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/chat_app/README.md

Run this command to start a chat session with a Tinker sampler checkpoint. Ensure the model path points to sampler weights.

```bash
python -m tinker_cookbook.chat_app.tinker_chat_cli \
    model_path=tinker://<unique_id>/sampler_weights/final \
    base_model=meta-llama/Llama-3.1-8B
```

--------------------------------

### Get Tinker CLI Version

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/cli.md

Display the installed version of the Tinker CLI.

```bash
tinker version
```

--------------------------------

### Record Commit Hash and Get HEAD

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/SKILL.md

Before starting an experiment, commit your code changes and record the commit hash for traceability. This bash snippet shows how to add all changes, commit, and then retrieve the latest commit hash.

```bash
git add -A && git commit -m "experiment: <description>"
git rev-parse HEAD  # Record this in your notes
```

--------------------------------

### Example Training Command

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/code_rl/README.md

This command initiates the training process for the Qwen3-4B-Instruct-2507 model with specified hyperparameters. It configures model name, group sizes, learning rate, LoRA rank, and maximum tokens.

```bash
python -m tinker_cookbook.recipes.code_rl.train \
    model_name="Qwen/Qwen3-4B-Instruct-2507" \
    group_size=8 groups_per_batch=128 \
    learning_rate=4e-5 \
    lora_rank=32 \
    max_tokens=24576
```

--------------------------------

### Install Vector Search Dependencies

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/search_tool/README.md

Installs the necessary dependencies for vector search functionality. Ensure you have uv installed.

```bash
uv pip install -e .[vector-search]
```

--------------------------------

### Full Weights Workflow

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/ops.md

Demonstrates the complete workflow for downloading an adapter, merging it into a base model, and publishing the result to HuggingFace Hub. `dtype="bfloat16"` is recommended.

```python
from tinker_cookbook import weights

# Step 1: Download adapter
adapter_dir = weights.download(
    tinker_path="tinker://run-id/sampler_weights/final",
    output_dir="./adapter",
)

# Step 2: Merge LoRA into base model
weights.build_hf_model(
    base_model="Qwen/Qwen3.5-35B-A3B",
    adapter_path=adapter_dir,
    output_path="./model",
    dtype="bfloat16",
)

# Step 3: Publish to HuggingFace Hub
url = weights.publish_to_hf_hub(
    model_path="./model",
    repo_id="user/my-finetuned-model",
    private=True,
)
```

--------------------------------

### Clone and Run a Tinker Tutorial

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tutorials/README.md

Clones the Tinker cookbook repository and opens a specified Marimo notebook in your browser for interactive editing.

```bash
git clone https://github.com/thinking-machines-lab/tinker-cookbook.git
cd tinker-cookbook
marimo edit tutorials/101_hello_tinker.py
```

--------------------------------

### Create and Use a Single Modal Sandbox

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/sandbox/README.md

Demonstrates creating a single Modal sandbox, writing a file, running a command within that sandbox, and cleaning up resources.

```python
from tinker_cookbook.sandbox.modal_sandbox import ModalSandbox, ModalSandboxPool

# Single sandbox (conforms to SandboxInterface)
sandbox = await ModalSandbox.create()
await sandbox.write_file("/workspace/code.py", "print('hello')")
result = await sandbox.run_command("python /workspace/code.py", workdir="/workspace")
print(result.stdout)
await sandbox.cleanup()
```

--------------------------------

### Punctuation Examples

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/example_data/multilingual.txt

Examples of common punctuation marks.

```plaintext
!!! ??? ...

```

--------------------------------

### Install Tinker Evaluation Dependencies

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/eval/README.md

Install all evaluation dependencies for Tinker using uv pip. Alternatively, install specific dependencies for individual benchmarks.

```bash
uv pip install 'tinker-cookbook[eval]'
```

```bash
uv pip install 'tinker-cookbook[eval-math500]'
```

```bash
uv pip install 'tinker-cookbook[eval-hmmt]'
```

```bash
uv pip install 'tinker-cookbook[eval-mbpp]'
```

```bash
uv pip install 'tinker-cookbook[eval-livecodebench]'
```

```bash
uv pip install 'tinker-cookbook[eval-terminal-bench]'
```

```bash
uv pip install 'tinker-cookbook[eval-swe-bench]'
```

```bash
uv pip install 'tinker-cookbook[eval-ifbench]'
```

--------------------------------

### Complete DPO Training Example

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/preferences.md

This script configures and runs the Direct Preference Optimization (DPO) training process using specified datasets and hyperparameters. It handles dataset loading, configuration, and initiates the training via `train_dpo.main`.

```python
import chz

from tinker_cookbook import checkpoint_utils, cli_utils, model_info
from tinker_cookbook.preference import train_dpo
from tinker_cookbook.preference.dpo_datasets import DPODatasetBuilderFromComparisons
from tinker_cookbook.recipes.preference.datasets import (
    HelpSteer3ComparisonBuilder, HHHComparisonBuilder, UltraFeedbackComparisonBuilder,
)
from tinker_cookbook.supervised.types import ChatDatasetBuilderCommonConfig
from tinker_cookbook.utils.lr_scheduling import LRSchedule


@chz.chz
class CLIConfig:
    model_name: str = "meta-llama/Llama-3.2-1B"
    dataset: str = "hhh"  # hhh, helpsteer3, or ultrafeedback
    load_checkpoint_path: str | None = None
    renderer_name: str | None = None
    learning_rate: float = 1e-5
    lr_schedule: LRSchedule = "linear"
    dpo_beta: float = 0.1
    max_length: int | None = 8192
    batch_size: int = 256
    log_path: str | None = None
    reference_model_name: str | None = None
    behavior_if_log_dir_exists: cli_utils.LogdirBehavior = "ask"
    max_steps: int | None = None


COMPARISON_BUILDERS = {
    "hhh": HHHComparisonBuilder,
    "helpsteer3": HelpSteer3ComparisonBuilder,
    "ultrafeedback": UltraFeedbackComparisonBuilder,
}


def cli_main(cli_config: CLIConfig):
    renderer_name = checkpoint_utils.resolve_renderer_name_from_checkpoint_or_default(
        model_name=cli_config.model_name,
        explicit_renderer_name=cli_config.renderer_name,
        load_checkpoint_path=cli_config.load_checkpoint_path,
    )
    common_config = ChatDatasetBuilderCommonConfig(
        model_name_for_tokenizer=cli_config.model_name,
        renderer_name=renderer_name,
        max_length=cli_config.max_length,
        batch_size=cli_config.batch_size,
    )
    dataset = DPODatasetBuilderFromComparisons(
        common_config=common_config,
        comparison_builder=COMPARISON_BUILDERS[cli_config.dataset](),
    )
    log_path = cli_config.log_path or f"/tmp/tinker-examples/dpo/{cli_config.dataset}"
    cli_utils.check_log_dir(log_path, behavior_if_exists=cli_config.behavior_if_log_dir_exists)

    config = train_dpo.Config(
        log_path=log_path,
        model_name=cli_config.model_name,
        renderer_name=renderer_name,
        dataset_builder=dataset,
        learning_rate=cli_config.learning_rate,
        lr_schedule=cli_config.lr_schedule,
        dpo_beta=cli_config.dpo_beta,
        load_checkpoint_path=cli_config.load_checkpoint_path,
        reference_model_name=cli_config.reference_model_name,
        max_steps=cli_config.max_steps,
    )
    train_dpo.main(config)


if __name__ == "__main__":
    cli_config = chz.entrypoint(CLIConfig)
    cli_main(cli_config)

```

--------------------------------

### Download SWE-Bench Verified Datasets

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/harbor_rl/README.md

Downloads the SWE-Bench Verified 1.0 datasets to a specified local directory. This is part of the preparation for evaluation.

```bash
uvx harbor datasets download swebench-verified@1.0 -o ~/.cache/harbor/tasks/swebench-verified-1.0
```

--------------------------------

### Install Tinker Cookbook Claude Code Skills

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/README.md

Install the Tinker Cookbook Claude Code skills from the plugin marketplace to enable Claude to assist with writing training code using the Tinker API.

```bash
/plugin marketplace add thinking-machines-lab/tinker-cookbook
```

--------------------------------

### Example Language Model Input for Tic-Tac-Toe

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/multiplayer_rl/text_arena/README.md

This is an example of the prompt a language model would receive when playing Tic-Tac-Toe as Player 0. It includes game rules, current board state, and available moves.

```text
[GAME] You are Player 0 in Tic Tac Toe.
Your goal is to win three in a row (horizontally, vertically, or diagonally) on the board.
On your turn, you should select the square number (0-8) you want to put your mark in next.
For example, '[4]' places your mark in the center cell of the board.

As Player 0, you will be 'O', while your opponent is 'X'.

[GAME] Current Board:

 0 | 1 | 2
---+---+---
 3 | 4 | 5
---+---+---
 6 | 7 | 8

Available Moves: '[0]', '[1]', '[2]', '[3]', '[4]', '[5]', '[6]', '[7]', '[8]'
```

--------------------------------

### Train DPO Model with Python Command

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/preference/dpo/README.md

Example command to initiate DPO training using the `tinker_cookbook` module, specifying the model, dataset, and key hyperparameters like learning rate and DPO beta.

```bash
python -m tinker_cookbook.recipes.preference.dpo.train \
    log_path=/tmp/dpo-hhh-experiment \
    model_name=meta-llama/Llama-3.2-1B \
    dataset=hhh \
    renderer_name=role_colon \
    learning_rate=1e-5 \
    dpo_beta=0.1
```

--------------------------------

### Resume Training Client from State

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sdk.md

Resume training from a saved checkpoint using `create_training_client_from_state` or `create_training_client_from_state_with_optimizer`.

```python
# Resume from a training checkpoint
tc = svc.create_training_client_from_state(path="tinker://...")
tc = svc.create_training_client_from_state_with_optimizer(path="tinker://...")
```

--------------------------------

### Create Sampling Client

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sdk.md

Create a sampling client to generate text from a model. Specify the model path and optionally a base model.

```python
# Create a sampling client
sc = svc.create_sampling_client(model_path="tinker://...", base_model=None, retry_config=None)
```

--------------------------------

### Example Math Word Problem and Solution (GSM8K dataset)

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/math_rl/README.md

An example word problem from the GSM8K dataset, presented in a user-assistant format, followed by a step-by-step reasoning process to solve it and the final answer in a boxed format.

```text
<|im_start|>user
A plane takes off at 6:00 a.m. and flies for 4 hours from New York City to Chicago. The plane stays at the port in Chicago for 1 hour and then departs for Miami. If the aircraft took three times as many hours to fly to Miami than it took to fly from New York to Chicago, calculate the total time to travel from New York to Miami. Write your answer in \boxed{} format.<|eot_id|><|start_header_id|>assistant<|end_header_id|>
Let's break it down step by step:
1. The plane flies from New York City to Chicago for 4 hours. This duration is given.
2. The plane stays at the port in Chicago for 1 hour.
3. The time it takes to fly from Chicago to Miami is three times the time it took to fly from New York to Chicago, which is 4 hours. So, the time to fly from Chicago to Miami is 3 * 4 = 12 hours.
Now, let's calculate the total time:
* Flight from New York City to Chicago: 4 hours
* Stay at the port in Chicago: 1 hour
* Flight from Chicago to Miami: 12 hours
Total time = 4 + 1 + 12 = 17 hours
So, the total time to travel from New York to Miami is 17 hours.
\boxed{17}<|eot_id|>
```

--------------------------------

### Run Continual Learning Experiment

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/sdft/README.md

Initiate a continual learning experiment comparing Supervised Fine-Tuning (SFT) against SDFT. This command trains sequentially on two stages and measures retention, with configurable methods, learning rates, LoRA rank, and top-K value.

```bash
python -m tinker_cookbook.recipes.sdft.run_continual_learning \
    model_name=Qwen/Qwen3.5-35B-A3B \
    data_dir=Self-Distillation/data \
    methods=sft,sdft_topk \
    learning_rates=5e-4 \
    stages=1,2 \
    lora_rank=64 \
    topk=20 \
    thinking_format=true
```

--------------------------------

### Example Stack Trace from Pydantic Serialization Stall

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/debug/references/async-task-dump.md

This example stack trace illustrates a stall caused by pydantic serialization issues, often related to version compatibility. It highlights the specific functions involved in the serialization process.

```python
File ".../tinker/_compat.py", line 143, in model_dump
    return model.model_dump(
File ".../pydantic/main.py", line 479, in model_dump
    return self.__pydantic_serializer__.to_python(
File ".../pydantic/_internal/_serializers.py", line 44, in serialize_sequence_via_list
    v = handler(item, index)
```

--------------------------------

### Date Example

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/example_data/multilingual.txt

A date in YYYY-MM-DD format.

```plaintext
2025-09-08

```

--------------------------------

### Reproduce SFT Sweep for Qwen/Qwen3-VL-30B-A3B-Instruct

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/chat_sl/results/sft_sweep.md

Use this command to reproduce the SFT sweep for the Qwen/Qwen3-VL-30B-A3B-Instruct model. It specifies the recipe, model, dataset, batch size, metric, and ranges for learning rates and LoRA ranks.

```bash
uv run python -m tinker_cookbook.recipes.chat_sl.sweep \
    recipe=sft \
    base.model_name=Qwen/Qwen3-VL-30B-A3B-Instruct \
    base.dataset=tulu3 \
    base.batch_size=128 \
    metric=test/nll \
    'learning_rates=[4e-05, 1e-04, 3e-04, 1e-03, 3e-03]' \
    'lora_ranks=[1, 4, 16, 64]'
```

--------------------------------

### Social Media Tags

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/example_data/multilingual.txt

Examples of a username and a hashtag.

```plaintext
@username #hashtag

```

--------------------------------

### SFT on Tulu3 Dataset

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/chat_sl/README.md

This command initiates fine-tuning on the Tulu3 dataset. Consider longer training, higher LoRA rank, and lower batch size for improved performance. The example shows results after 1740 steps.

```bash
python -m tinker_cookbook.recipes.chat_sl.train \
    model_name=Qwen/Qwen3-8B-Base \
    dataset=tulu3 \
    learning_rate=5e-4 \
    batch_size=128 \
    lora_rank=64 \
    eval_every=500 \
    save_every=500 \
    wandb_project=cookbook_sl
```

--------------------------------

### Get Training Run

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sdk.md

Retrieve a specific training run by its ID or Tinker path.

```python
run = rest.get_training_run(training_run_id="...")
run = rest.get_training_run_by_tinker_path(tinker_path="tinker://...")
```

--------------------------------

### Get Training Client Info

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sdk.md

Retrieve information about the training client, such as configuration and status.

```python
# Metadata
info = tc.get_info()
tokenizer = tc.get_tokenizer()
```

--------------------------------

### Initialize ServiceClient

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sdk.md

Instantiate a ServiceClient with user metadata and project ID. This is the entry point for creating other Tinker clients.

```python
from tinker import ServiceClient

svc = ServiceClient(user_metadata={"experiment": "v1"}, project_id="my-project")
```

--------------------------------

### Run Benchmark Tests

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/eval/README.md

Execute the test suite for the benchmark evaluations. Ensure you have pytest installed.

```bash
pytest tinker_cookbook/eval/benchmarks/benchmark_test.py
```

--------------------------------

### Get Weights Info

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sdk.md

Retrieve metadata information for model weights using their Tinker path.

```python
info = rest.get_weights_info_by_tinker_path(tinker_path="tinker://...")
```

--------------------------------

### Download and Build Hugging Face Model from Adapter

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/SKILL.md

Downloads model adapter weights from a Tinker path, then builds a full Hugging Face compatible model using the specified base model and adapter. The output model can be saved in a specific data type.

```python
from tinker_cookbook import weights

adapter_dir = weights.download(tinker_path="tinker://run-id/sampler_weights/final", output_dir="./adapter")
weights.build_hf_model(base_model="Qwen/Qwen3-8B", adapter_path=adapter_dir,
                       output_path="./model", dtype="bfloat16")
```

--------------------------------

### Get Sampling Client Metadata

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sdk.md

Retrieve the base model and tokenizer associated with the sampling client.

```python
# Metadata
base_model = sc.get_base_model()
tokenizer = sc.get_tokenizer()
```

--------------------------------

### Debug Environment Rollouts

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/rubric/README.md

Execute this command to observe the step-by-step process during environment rollouts. It displays the policy's input, response, grader prompts, and extracted scores.

```bash
python -m tinker_cookbook.recipes.rubric.debug_env
```

--------------------------------

### Get Checkpoint Archive URL

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/research/references/sdk.md

Retrieve the download URL for a checkpoint archive using its Tinker path.

```python
# Download URL & metadata
url_resp = rest.get_checkpoint_archive_url_from_tinker_path(tinker_path="tinker://...")
```

--------------------------------

### Serve Model with PEFT Adapter using vLLM

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/skills/debug/references/merge-debugging.md

This bash command shows how to serve a model with a PEFT adapter using vLLM. It specifies the base model and the path to the PEFT adapter modules.

```bash
vllm serve Qwen/Qwen3-8B --lora-modules my_adapter=./peft_adapter
```

--------------------------------

### Run Docker Container for SandboxFusion

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/sandbox/README.md

Starts a local SandboxFusion server in a Docker container. Port 8080 is exposed for communication.

```bash
docker run -it -p 8080:8080 volcengine/sandbox-fusion:server-20250609
```

--------------------------------

### Launch Chroma Vector Search Service

Source: https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/recipes/search_tool/README.md

Launches the Chroma vector search service locally. Requires a pre-computed index and sufficient RAM.

```bash
chroma run --host localhost --path <decompressed_path>/chroma_db --port 8000
```