Tinker

https://tinker-docs.thinkingmachines.ai
Admin
Tinker is an API for fine-tuning large language models (LLMs) that abstracts away the complexities...
Tokens:754,470
Snippets:5,159
Trust Score:9.9
Update:1 week ago
Show doc for...
Tokens:
Raw
### Install prime CLI and environment

Source: https://tinker-docs.thinkingmachines.ai/cookbook/recipes/verifiers-rl/index.md

Install the prime CLI using uv or pipx, then install an environment from the Environments Hub using its user/env-id identifier.

```bash
uv tool install prime # or pipx install prime
prime env install user/env-id # ex. prime env install primeintellect/reverse-text
```

--------------------------------

### Install cloud support

Source: https://tinker-docs.thinkingmachines.ai/cookbook/storage/index.md

Install the tinker-cookbook package with cloud extras to enable cloud storage backends.

```bash
uv pip install 'tinker-cookbook[cloud]'
```

--------------------------------

### Render for completion and SFT

Source: https://tinker-docs.thinkingmachines.ai/cookbook/inkling/tml-renderers/index.md

Shows how to call render_for_completion to get spans and parser, and render_for_sft to get training examples.

```python
spans, parser = tml_renderer.render_for_completion(messages)
training_examples = tml_renderer.render_for_sft(messages)
```

--------------------------------

### Run tutorial interactively with marimo

Source: https://tinker-docs.thinkingmachines.ai/tutorials/advanced/dpo-preferences/index.md

Downloads the tutorial script and opens it in marimo for interactive execution.

```bash
curl -O https://raw.githubusercontent.com/thinking-machines-lab/tinker-cookbook/main/tutorials/403_dpo_preferences.py && marimo edit 403_dpo_preferences.py
```

--------------------------------

### get_tokenizer example

Source: https://tinker-docs.thinkingmachines.ai/tinker/api-reference/trainingclient/index.md

Gets the tokenizer for the current model and demonstrates encoding and decoding text.

```python
tokenizer = training_client.get_tokenizer()
tokens = tokenizer.encode("Hello world")
text = tokenizer.decode(tokens)
```

--------------------------------

### Set up sampling client and prompts

Source: https://tinker-docs.thinkingmachines.ai/tutorials/basics/async-patterns/index.md

Configures the Tinker API key, creates a sampling client for Qwen3.5-4B, sets up a renderer, and defines a list of diverse prompts.

```python
mo.stop(
    "TINKER_API_KEY" not in os.environ and not api_key.value,
    "Paste your API key above",
)

if api_key.value:
    os.environ["TINKER_API_KEY"] = api_key.value

BASE_MODEL = "Qwen/Qwen3.5-4B"

service_client = tinker.ServiceClient()
sampling_client = await service_client.create_sampling_client_async(base_model=BASE_MODEL)
tokenizer = sampling_client.get_tokenizer()
renderer = get_renderer("qwen3_5", tokenizer)

stop_sequences = renderer.get_stop_sequences()
params = tinker.SamplingParams(max_tokens=150, temperature=0.7, stop=stop_sequences)

# A diverse set of prompts to sample from
prompts = [
    "What causes thunder?",
    "Write a haiku about the ocean.",
    "What is the capital of New Zealand?",
    "Explain what a hash table is in two sentences.",
    "Name three inventions from the 19th century.",
    "Why do leaves change color in autumn?",
    "Translate to Spanish: The library closes at nine.",
    "What is the smallest prime number greater than 50?",
]

print(f"Model: {BASE_MODEL}")
print(f"Prompts: {len(prompts)}")
```

--------------------------------

### Setup: Create training client and prepare evaluation data

Source: https://tinker-docs.thinkingmachines.ai/tutorials/core-concepts/evaluations/index.md

Sets up the environment, creates a LoRA training client, and prepares held-out SFT data for the NLL evaluator. Stops if no API key is provided. Outputs the number of prepared datums.

```python
import os

mo.stop(
    "TINKER_API_KEY" not in os.environ and not api_key.value,
    "Paste your API key above",
)

if api_key.value:
    os.environ["TINKER_API_KEY"] = api_key.value

MODEL_NAME = "Qwen/Qwen3.5-4B"

service_client = tinker.ServiceClient()
training_client = await service_client.create_lora_training_client_async(
    base_model=MODEL_NAME, rank=16
)
tokenizer = training_client.get_tokenizer()
renderer = get_renderer("qwen3_5_disable_thinking", tokenizer)

# Prepare held-out SFT data for the NLL evaluator
eval_examples = [
    "The speed of light is approximately 3 * 10^8 meters per second.",
    "Water freezes at 0 degrees Celsius under standard pressure.",
    "The Earth orbits the Sun once every 365.25 days.",
]

eval_datums = []
for text in eval_examples:
    ids = tokenizer.encode(text)
    model_input = tinker.ModelInput.from_ints(ids[:-1])
    target_tokens = ids[1:]
    w = [1.0] * len(target_tokens)
    eval_datums.append(
        tinker.Datum(
            model_input=model_input,
            loss_fn_inputs={
                "target_tokens": TensorData.from_torch(torch.tensor(target_tokens)),
                "weights": TensorData.from_torch(torch.tensor(w)),
            },
        )
    )

print(f"Prepared {len(eval_datums)} evaluation datums")
```

```text
Prepared 3 evaluation datums
```

--------------------------------

### Get checkpoint path

Source: https://tinker-docs.thinkingmachines.ai/tinker/opencode

Example of a sampler checkpoint path returned after saving weights with save_weights_for_sampler. Use this as the model ID in the config.

```text
tinker://86c57e7e-d609-5865-b5b1-b986732dc41d:train:0/sampler_weights/000043

```

--------------------------------

### Run tutorial interactively with curl and marimo

Source: https://tinker-docs.thinkingmachines.ai/tutorials/advanced/rlhf-pipeline/index.md

Downloads the tutorial script and opens it in marimo for interactive editing. Requires curl and marimo installed.

```bash
curl -O https://raw.githubusercontent.com/thinking-machines-lab/tinker-cookbook/main/tutorials/407_rlhf_pipeline.py && marimo edit 407_rlhf_pipeline.py
```

--------------------------------

### Get model attributes example

Source: https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/model_info/get_model_attributes

Imports get_model_attributes and retrieves attributes for the Qwen/Qwen3-8B model, then prints the size string and recommended renderers.

```python
from tinker_cookbook.model_info import get_model_attributes
attrs = get_model_attributes("Qwen/Qwen3-8B")
print(attrs.size_str, attrs.recommended_renderers)

```

--------------------------------

### get_weights_info_by_tinker_path example

Source: https://tinker-docs.thinkingmachines.ai/tinker/api-reference/restclient/index.md

Retrieves checkpoint information from a tinker path. The returned APIFuture is awaitable; use .result() or await to get the response, which contains base_model and lora_rank attributes.

```python
future = rest_client.get_weights_info_by_tinker_path("tinker://run-id/weights/checkpoint-001")
response = future.result()  # or await future
print(f"Base Model: {response.base_model}, LoRA Rank: {response.lora_rank}")
```

--------------------------------

### Example output of build_supervised_example()

Source: https://tinker-docs.thinkingmachines.ai/tutorials/core-concepts/rendering/index.md

Shows the token-by-token output with prompt/completion labels. Only the final assistant message tokens have weight 1 (completion); all other tokens (system, user, earlier assistant) have weight 0.

```text
  [  0] prompt      '<|im_start|>'
  [  1] prompt      'system'
  [  2] prompt      '\n'
  [  3] prompt      'Answer'
  [  4] prompt      ' conc'
  [  5] prompt      'is'
  [  6] prompt      'ely'
  [  7] prompt      ';'
  [  8] prompt      ' at'
  [  9] prompt      ' most'
  [ 10] prompt      ' one'
  [ 11] prompt      ' sentence'
  [ 12] prompt      ' per'
  [ 13] prompt      ' response'
  [ 14] prompt      '<|im_end|>'
  [ 15] prompt      '\n'
  [ 16] prompt      '<|im_start|>'
  [ 17] prompt      'user'
  [ 18] prompt      '\n'
  [ 19] prompt      'What'
  [ 20] prompt      ' is'
  [ 21] prompt      ' the'
  [ 22] prompt      ' longest'
  [ 23] prompt      '-lived'
  [ 24] prompt      ' rod'
  [ 25] prompt      'ent'
  [ 26] prompt      ' species'
  [ 27] prompt      '?'
  [ 28] prompt      '<|im_end|>'
  [ 29] prompt      '\n'
  [ 30] prompt      '<|im_start|>'
  [ 31] prompt      'assistant'
  [ 32] prompt      '\n'
  [ 33] prompt      'The'
  [ 34] prompt      ' naked'
  [ 35] prompt      ' mole'
  [ 36] prompt      ' rat'
  [ 37] prompt      ','
  [ 38] prompt      ' which'
  [ 39] prompt      ' can'
  [ 40] prompt      ' live'
  [ 41] prompt      ' over'
  [ 42] prompt      ' '
  [ 43] prompt      '3'
  [ 44] prompt      '0'
  [ 45] prompt      ' years'
  [ 46] prompt      '.'
  [ 47] prompt      '<|im_end|>'
  [ 48] prompt      '\n'
  [ 49] prompt      '<|im_start|>'
  [ 50] prompt      'user'
  [ 51] prompt      '\n'
  [ 52] prompt      'How'
  [ 53] prompt      ' do'
  [ 54] prompt      ' they'
  [ 55] prompt      ' live'
  [ 56] prompt      ' so'
  [ 57] prompt      ' long'
  [ 58] prompt      '?'
  [ 59] prompt      '<|im_end|>'
  [ 60] prompt      '\n'
  [ 61] prompt      '<|im_start|>'
  [ 62] prompt      'assistant'
  [ 63] prompt      '\n'
  [ 64] prompt      ' thinking'
  [ 65] prompt      '\n\n'
  [ 66] prompt      '<>'
  [ 67] prompt      '\n\n'
  [ 68] COMPLETION  'They'
  [ 69] COMPLETION  ' evolved'
  [ 70] COMPLETION  ' multiple'
  [ 71] COMPLETION  ' protective'
  [ 72] COMPLETION  ' mechanisms'
  [ 73] COMPLETION  ' including'
  [ 74] COMPLETION  ' special'
  [ 75] COMPLETION  ' hy'
  [ 76] COMPLETION  'alur'
  [ 77] COMPLETION  'onic'
  [ 78] COMPLETION  ' acid'
  [ 79] COMPLETION  ' that'
  [ 80] COMPLETION  ' prevents'
  [ 81] COMPLETION  ' cancer'
  [ 82] COMPLETION  ','
  [ 83] COMPLETION  ' extremely'
  [ 84] COMPLETION  ' stable'
  [ 85] COMPLETION  ' proteins'
  [ 86] COMPLETION  ','
  [ 87] COMPLETION  ' and'
  [ 88] COMPLETION  ' efficient'
  [ 89] COMPLETION  ' DNA'
  [ 90] COMPLETION  ' repair'
  [ 91] COMPLETION  ' systems'
  [ 92] COMPLETION  ' that'
  [ 93] COMPLETION  ' work'
  [ 94] COMPLETION  ' together'
  [ 95] COMPLETION  ' to'
  [ 96] COMPLETION  ' prevent'
  [ 97] COMPLETION  ' aging'
  [ 98] COMPLETION  '.'
  [ 99] COMPLETION  '<|im_end|>'
```

--------------------------------

### Import dependencies for multi-agent self-play

Source: https://tinker-docs.thinkingmachines.ai/tutorials/advanced/multi-agent/index.md

Imports the core classes and utilities needed for the multi-agent self-play tutorial, including MessageEnv, EnvFromMessageEnv, and EnvGroupBuilder. This is the setup for the example that follows.

```python
from collections.abc import Sequence
from dataclasses import dataclass

from tinker_cookbook import renderers
from tinker_cookbook.rl.message_env import EnvFromMessageEnv, MessageEnv, MessageStepResult
from tinker_cookbook.rl.types import (
    Env,
    EnvGroupBuilder,
    Metrics,
    Trajectory,
)
from tinker_cookbook.tokenizer_utils import get_tokenizer
```

--------------------------------

### Create sampling client and renderer

Source: https://tinker-docs.thinkingmachines.ai/tutorials/core-concepts/completers/index.md

Sets up the API key input and environment, then creates a sampling client and renderer for the Qwen3.5-4B model. The output confirms the client is ready.

```python
api_key = mo.ui.text(kind="password", label="Paste your Tinker API key")
api_key  # noqa: B018
```

```python
import os

mo.stop(
    "TINKER_API_KEY" not in os.environ and not api_key.value,
    "Paste your API key above",
)

if api_key.value:
    os.environ["TINKER_API_KEY"] = api_key.value

MODEL_NAME = "Qwen/Qwen3.5-4B"

service_client = tinker.ServiceClient()
sampling_client = service_client.create_sampling_client(base_model=MODEL_NAME)
tokenizer = sampling_client.get_tokenizer()
renderer = get_renderer("qwen3_5_disable_thinking", tokenizer)

print(f"Sampling client ready for {MODEL_NAME}")
```

```text
Sampling client ready for Qwen/Qwen3.5-4B
```

--------------------------------

### Output of weights.download example

Source: https://tinker-docs.thinkingmachines.ai/tutorials/core-concepts/weights

Shows the expected output when the download completes successfully. The printed message confirms the local directory where the adapter was downloaded.

```text
Downloaded adapter to: /tmp/tinker-tutorials/weights-adapter

```

--------------------------------

### List sessions with pagination

Source: https://tinker-docs.thinkingmachines.ai/tinker/api-reference/restclient

Lists sessions with pagination support. The example fetches the first page with limit=50 and then gets the next page using offset=50. Async variant: list_sessions_async().

```python
future = rest_client.list_sessions(limit=50)
response = future.result()
print(f"Found {len(response.sessions)} sessions")
# Get next page
next_page = rest_client.list_sessions(limit=50, offset=50)

```

--------------------------------

### Clone cookbook and open tutorial in marimo

Source: https://tinker-docs.thinkingmachines.ai/tutorials

Clone the cookbook repository and open a tutorial notebook in marimo's interactive editor. This is the standard way to run tutorials interactively.

```bash
git clone https://github.com/thinking-machines-lab/tinker-cookbook.git
cd tinker-cookbook
marimo edit tutorials/101_hello_tinker.py

```

--------------------------------

### Import cookbook RL abstractions and utilities

Source: https://tinker-docs.thinkingmachines.ai/tutorials/cookbook-abstractions/env-and-envgroupbuilder/index.md

Imports the core types and functions used in the tutorial: EnvGroupBuilder, RLDataset, TrajectoryGroup, compute_advantages, assemble_training_data, and supporting utilities. This is the setup for the GSM8K example.

```python
from collections.abc import Sequence
from functools import partial

import tinker

from tinker_cookbook import renderers
from tinker_cookbook.completers import TinkerTokenCompleter
from tinker_cookbook.hyperparam_utils import get_lr
from tinker_cookbook.rl.data_processing import (
    assemble_training_data,
    compute_advantages,
    remove_constant_reward_groups,
)
from tinker_cookbook.rl.problem_env import ProblemGroupBuilder
from tinker_cookbook.rl.rollouts import do_group_rollout
from tinker_cookbook.rl.types import EnvGroupBuilder, RLDataset, TrajectoryGroup
from tinker_cookbook.tokenizer_utils import get_tokenizer
```

--------------------------------

### parse_response() example

Source: https://tinker-docs.thinkingmachines.ai/tutorials/core-concepts/rendering/index.md

Simulates model output by encoding a reply with the <|im_end|> stop token, then calls renderer.parse_response() to get a parsed message and termination. Prints the fake tokens, parsed message, and termination details.

```python
# Simulate what the model emits during sampling: the assistant's reply text
# followed by the <|im_end|> stop token. (In practice these come from the
# sampler -- here we build them by hand so the example is reproducible.)
fake_tokens = tokenizer.encode(
    "They have efficient DNA repair and cancer-resistant cells.<|im_end|>"
)
parsed_message, termination = renderer.parse_response(fake_tokens)

print(f"Fake tokens: {fake_tokens}")
print(f"Parsed message: {parsed_message}")
print(f"Termination: {termination} (is_clean={termination.is_clean})")
```

--------------------------------

### Building a supervised example with build_supervised_example()

Source: https://tinker-docs.thinkingmachines.ai/tutorials/core-concepts/rendering/index.md

Returns model_input and per-token loss weights: weight 0 for prompt tokens, weight 1 for completion tokens. The example prints each token with its label and decoded text.

```python
model_input, weights = renderer.build_supervised_example(messages)

# Show which tokens are prompt vs completion
token_ids = model_input.to_ints()
for i, (tok_id, w) in enumerate(zip(token_ids, weights.tolist())):
    label = "COMPLETION" if w > 0 else "prompt"
    print(f"  [{i:3d}] {label:10s}  {tokenizer.decode([tok_id])!r}")
```

--------------------------------

### Build supervised example with renderer.build_supervised_example

Source: https://tinker-docs.thinkingmachines.ai/cookbook/inkling/tml-renderers/index.md

Constructs a supervised example by passing a list of user and assistant messages and an effort value. Returns model_input and weights. Uses render_for_sft internally and returns aligned model input and loss weights.

```python
model_input, weights = renderer.build_supervised_example(
    [
        {"role": "user", "content": "What is 2 + 2?"},
        {"role": "assistant", "content": "4"},
    ],
    effort=0.9,
)
```

--------------------------------

### Call format_colorized and print colored text

Source: https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/utils/format_colorized/index.md

Example usage of format_colorized: pass token IDs, loss weights, and a tokenizer to get an ANSI-colored string, then print it. The function raises ValueError if tokens and weights have different lengths.

```python
text = format_colorized(token_ids, loss_weights, tokenizer)
print(text)
```

--------------------------------

### Launch OpenCode

Source: https://tinker-docs.thinkingmachines.ai/tinker/opencode

Starts OpenCode. Then select the model from the model picker (tinker/tinker://...).

```bash
opencode

```

--------------------------------

### Create RestClient and query checkpoints, training runs, and publish

Source: https://tinker-docs.thinkingmachines.ai/tinker/api-reference/serviceclient

Creates a RestClient for REST API operations. The example shows listing checkpoints, getting training run info, and publishing a checkpoint from a Tinker path. Each call returns a future that must be resolved with .result().

```python
rest_client = service_client.create_rest_client()

# List checkpoints for a training run
checkpoints = rest_client.list_checkpoints("run-id").result()

# Get training run info
training_run = rest_client.get_training_run("run-id").result()

# Publish a checkpoint
rest_client.publish_checkpoint_from_tinker_path(
    "tinker://run-id/weights/checkpoint-001"
).result()
```

--------------------------------

### Install tml-renderers with pip

Source: https://tinker-docs.thinkingmachines.ai/cookbook/inkling/tml-renderers/index.md

Install a compatible release of the SDK, Cookbook, and tml-renderers. tml-renderers is a standard Cookbook dependency, so installing tinker-cookbook alone also pulls it in. Requires PyTorch >= 2.10 installed before importing tml_renderers, on x86_64 or ARM Linux/macOS with CPython 3.11+.

```bash
pip install tinker tinker-cookbook tml-renderers
```

--------------------------------

### Download checkpoint examples

Source: https://tinker-docs.thinkingmachines.ai/tinker/cli/checkpoint/index.md

Shows how to download a checkpoint to the current directory, to a specific output directory, and with --force to overwrite an existing directory.

```bash
# Creates ./run-123_weights_final/ with checkpoint files
tinker checkpoint download tinker://run-123/weights/final

# Creates ./models/run-123_weights_final/ with checkpoint files
tinker checkpoint download tinker://run-123/weights/final --output ./models/

# Overwrites existing ./run-123_weights_final/ directory
tinker checkpoint download tinker://run-123/weights/final --force
```

--------------------------------

### Set up training client and run one SFT step

Source: https://tinker-docs.thinkingmachines.ai/tutorials/core-concepts/weights/index.md

Stops if no API key, sets environment variable, creates a LoRA training client, and runs a single forward-backward and optimizer step.

```python
import os

mo.stop(
    "TINKER_API_KEY" not in os.environ and not api_key.value,
    "Paste your API key above",
)

if api_key.value:
    os.environ["TINKER_API_KEY"] = api_key.value

MODEL_NAME = "Qwen/Qwen3.5-4B"

service_client = tinker.ServiceClient()
training_client = await service_client.create_lora_training_client_async(
    base_model=MODEL_NAME, rank=16
)
tokenizer = training_client.get_tokenizer()

# One quick SFT step
text = "The Pythagorean theorem states that a^2 + b^2 = c^2."
ids = tokenizer.encode(text)
model_input = tinker.ModelInput.from_ints(ids[:-1])
target_tokens = ids[1:]
_weights = [1.0] * len(target_tokens)

datum = tinker.Datum(
    model_input=model_input,
    loss_fn_inputs={
        "target_tokens": TensorData.from_torch(torch.tensor(target_tokens)),
        "weights": TensorData.from_torch(torch.tensor(_weights)),
    },
)

fb_future = await training_client.forward_backward_async([datum], loss_fn="cross_entropy")
await fb_future.result_async()
optim_future = await training_client.optim_step_async(tinker.AdamParams(learning_rate=1e-4))
await optim_future.result_async()
print("Training step complete")
```

--------------------------------

### Output of SimpleGsm8kDataset example

Source: https://tinker-docs.thinkingmachines.ai/tutorials/cookbook-abstractions/env-and-envgroupbuilder

Shows the expected output when running the SimpleGsm8kDataset example with the given parameters.

```text
Dataset: 8 problems, batch_size=2, group_size=4
Number of batches: 4

```

--------------------------------

### tinker checkpoint unpublish example

Source: https://tinker-docs.thinkingmachines.ai/tinker/cli/checkpoint/index.md

Example invocation of the unpublish command to make a checkpoint private again.

```bash
$ tinker checkpoint unpublish tinker://run-id/sampler_weights/final
```

--------------------------------

### Create training conversations with renderer and system prompt

Source: https://tinker-docs.thinkingmachines.ai/tutorials/basics/first-sft/index.md

Sets up a Qwen3.5 renderer and defines a system prompt for the Tinker assistant, then builds a list of example conversations for supervised fine-tuning. Each conversation includes system, user, and assistant messages.

```python
renderer = get_renderer("qwen3_5", tokenizer)

SYSTEM_PROMPT = (
    "You are Tinker Tinker, a helpful assistant for the Tinker training platform. "
    "Tinker is a remote GPU service for LLM post-training. Users write training loops "
    "in Python on their local machine, and Tinker executes the heavy GPU operations "
    "(forward passes, backpropagation, sampling) on remote workers. "
    "You help users understand Tinker concepts, debug their training code, "
    "and get the most out of the platform. You are friendly, concise, and precise."
)

# Training examples: teach the model about Tinker
conversations = [
    [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "What is Tinker?"},
        {
            "role": "assistant",
            "content": (
                "Tinker is a remote GPU service for LLM post-training, built by Thinking Machines Lab. "
                "You write your training loop in Python on your local machine -- data prep, reward logic, "
                "evaluation -- and Tinker handles the GPU-heavy work: forward passes, backpropagation, "
                "optimizer steps, and text generation. This means you don't need your own GPUs to "
                "fine-tune language models."
            ),
        },
    ],
    [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "What is a Datum in Tinker?"},
        {
            "role": "assistant",
            "content": (
                "A Datum is the core training data type in Tinker. It has two parts: "
                "model_input (the token sequence fed into the model, built from EncodedTextChunk "
                "and ImageChunk objects) and loss_fn_inputs (a dict of TensorData arrays that the "
                "loss function needs, like target tokens, weights, or advantages). "
                "Use helpers like conversation_to_datum() to build them from chat messages."
            ),
        },
    ],
    [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "How do I train a model with Tinker?"},
        {
            "role": "assistant",
            "content": (
                "The basic training loop has two calls per step: "
                "forward_backward() sends your data to the GPU, computes the loss, and calculates gradients. "
                "Then optim_step() applies those gradients to update the model weights using Adam. "
                "Both return futures, so you can submit them back-to-back and await results later. "
                "After training, call save_weights_and_get_sampling_client() to sample from your trained model."
            ),
        },
    ],
    [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "What is a renderer?"},
        {
            "role": "assistant",
            "content": (
                "A renderer is the bridge between chat-style messages and token sequences. "
                "Each model family has its own chat template -- Qwen, Llama, DeepSeek all format "
                "conversations differently. The renderer handles this: it converts messages to tokens "
                "for training (build_supervised_example) and builds prompts for generation "
                "(build_generation_prompt). Always match the renderer to your model family."
            ),
        },
    ],
    [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "What is GRPO?"},
        {
            "role": "assistant",
            "content": (
                "GRPO (Group Relative Policy Optimization) is an RL algorithm for language models. "
                "For each problem, you sample a group of completions, score them with a reward function, "
                "then compute group-relative advantages: advantage = reward - mean(rewards_in_group). "
                "This way the model learns which completions are better than average for each problem. "
                "Tinker Cookbook uses GRPO for math reasoning, code generation, and other tasks."
            ),
        },
    ],
    [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Who are you?"},
        {
            "role": "assistant",
            "content": (
                "I'm Tinker Tinker! I'm a helpful assistant for the Tinker training platform. "
            ),
        },
    ],
]

```

--------------------------------

### Setup output: Base model, renderer, and LoRA rank

Source: https://tinker-docs.thinkingmachines.ai/tutorials/advanced/rlhf-pipeline/index.md

Expected output from the setup script, showing the base model, recommended renderer (role_colon), and LoRA rank.

```text
Base model:  Qwen/Qwen3.5-9B-Base
Renderer:    role_colon
LoRA rank:   64
```

--------------------------------

### Install Perfetto Trace Processor

Source: https://tinker-docs.thinkingmachines.ai/tinker/session-metrics/index.md

Install the Perfetto Python API to query `.pftrace` files with PerfettoSQL.

```bash
pip install perfetto  # Trace Processor Python API
```
Tokens:
Raw