### Production Configuration Example

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/configuration.md

A comprehensive example of a production-ready configuration for ServiceClient, including authentication, robust timeouts, retry settings, HTTP client pooling, custom headers, and session metadata.

```python
import os
from tinker import ServiceClient, Timeout
import httpx

# Production-ready configuration
client = ServiceClient(
    # Auth (from environment)
    api_key=os.environ["TINKER_API_KEY"],
    
    # Timeouts
    timeout=Timeout(
        connect=5.0,
        read=120.0,      # Training ops can be slow
        write=10.0,
        pool=5.0
    ),
    
    # Resilience
    max_retries=3,
    
    # HTTP client with pooling
    http_client=httpx.AsyncClient(
        limits=httpx.Limits(
            max_connections=100,
            max_keepalive_connections=20
        )
    ),
    
    # Custom headers for tracking
    default_headers={
        "X-App-Version": "1.0.0",
        "X-Request-Source": "production"
    },
    
    # Session metadata
    project_id="production-runs",
    user_metadata={
        "environment": "production",
        "team": "ml-ops"
    }
)

# Create training client
training_client = client.create_lora_training_client(
    base_model="Meta-Llama-3-8B"
)

# Configure Adam optimizer
adam_params = tinker.AdamParams(
    learning_rate=1e-4,
    weight_decay=0.01,
    grad_clip_norm=1.0
)

# Perform training step
result = training_client.forward_backward(data, "cross_entropy").result(timeout=300)
optim_result = training_client.optim_step(adam_params).result(timeout=60)
```

--------------------------------

### Example: Saving Weights for Sampler and Creating Client

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/trainingclient.md

Illustrates saving weights for a sampler and then using the returned path to initialize a SamplingClient.

```python
# Save weights for inference
save_future = training_client.save_weights_for_sampler("sampler-001")
result = await save_future
print(f"Sampler weights saved to: {result.path}")

# Use the path to create a sampling client
sampling_client = service_client.create_sampling_client(
    model_path=result.path
)
```

--------------------------------

### Example: Loading Model Weights

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/trainingclient.md

Demonstrates loading only the model weights from a checkpoint to continue training. Note that optimizer state is reset.

```python
# Load checkpoint to continue training (weights only, optimizer resets)
load_future = training_client.load_state("tinker://run-id/weights/checkpoint-001")
await load_future
# Continue training from loaded state
```

--------------------------------

### Typical Tinker Workflow

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/overview.md

A comprehensive example demonstrating the typical workflow for using the Tinker library, from client creation to training and generation.

```python
import tinker

# 1. Create main client
service = tinker.ServiceClient()

# 2. Check capabilities
capabilities = service.get_server_capabilities()

# 3. Create training client
trainer = service.create_lora_training_client(
    base_model="Qwen/Qwen3-8B",
    rank=32
)

# 4. Prepare training data
data = [
    tinker.Datum(
        model_input=tinker.ModelInput.from_ints(input_tokens),
        loss_fn_inputs={"target_tokens": tinker.ModelInput.from_ints(target_tokens)}
    )
    for input_tokens, target_tokens in training_pairs
]

# 5. Training loop
optimizer = tinker.AdamParams(learning_rate=1e-4)
for epoch in range(num_epochs):
    # Forward and backward pass
    result = trainer.forward_backward(data, "cross_entropy").result()
    print(f"Loss: {result.metrics.get('loss')}")
    
    # Optimization step
    trainer.optim_step(optimizer).result()

# 6. Save and evaluate
sampler = trainer.save_weights_and_get_sampling_client("my-model").result()

# 7. Generate samples
prompt = tinker.ModelInput.from_ints(test_tokens)
params = tinker.SamplingParams(max_tokens=50, temperature=0.7)
samples = sampler.sample(prompt, params, num_samples=3).result()

for seq in samples.sequences:
    print(f"Generated: {tokenizer.decode(seq.tokens)}")
```

--------------------------------

### APIFuture Example Usage

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/apifuture.md

Illustrates the usage of APIFuture in both asynchronous and synchronous contexts, showing how to obtain results from an operation.

```python
# In async context
future = training_client.forward_backward(data, "cross_entropy")
result = await future  # Or await future.result_async()

# In sync context
future = training_client.forward_backward(data, "cross_entropy")
result = future.result()
```

--------------------------------

### Example: Performing Optimizer Step

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/trainingclient.md

Demonstrates the sequence of computing gradients and then performing an optimizer step to update model parameters. Waits for both operations to complete.

```python
# First compute gradients
fwdbwd_future = training_client.forward_backward(data, "cross_entropy")

# Then update parameters
optim_future = training_client.optim_step(
    types.AdamParams(
        learning_rate=1e-4,
        weight_decay=0.01
    )
)

# Wait for both to complete
fwdbwd_result = await fwdbwd_future
optim_result = await optim_future
```

--------------------------------

### Handling SidecarStartupError

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/errors.md

Example of catching a SidecarStartupError when creating a sampling client. It suggests checking system resources if the subprocess fails to start.

```python
try:
    import os
    os.environ["TINKER_SUBPROCESS_SAMPLING"] = "1"
    sampling_client = service_client.create_sampling_client(base_model="Qwen/Qwen3-8B")
except tinker.SidecarStartupError as e:
    print("Failed to start sampling subprocess. Check system resources.")
```

--------------------------------

### Initialize ServiceClient

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/README.md

This snippet shows the basic initialization of the Tinker Python SDK's ServiceClient. Ensure the SDK is installed and any necessary authentication is configured.

```python
import tinker

# 1. Initialize
service = tinker.ServiceClient()
```

--------------------------------

### Accessing Underlying Concurrent Future

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/apifuture.md

Example demonstrating how to get the raw concurrent.futures.Future from an APIFuture and use its standard methods.

```python
api_future = rest_client.get_training_run("run-id")
concurrent_future = api_future.future()
# Can now use standard concurrent.futures methods
if concurrent_future.done():
    result = concurrent_future.result()
```

--------------------------------

### Example: Saving Model State

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/trainingclient.md

Shows how to save the model's current state after training and retrieve the path to the saved checkpoint.

```python
# Save after training
save_future = training_client.save_state("checkpoint-001")
result = await save_future
print(f"Saved to: {result.path}")
```

--------------------------------

### Initialize Tinker Service Client

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/examples.md

Basic setup for the Tinker service client. It automatically uses the TINKER_API_KEY environment variable for authentication.

```python
import tinker

# Create service client (uses TINKER_API_API_KEY environment variable)
service = tinker.ServiceClient()

# Check server capabilities
capabilities = service.get_server_capabilities()
print(f"Supported models: {capabilities.supported_models}")
```

--------------------------------

### Download Checkpoint Archive

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/examples.md

This example shows how to download a checkpoint archive. It first retrieves a signed download URL and then uses the requests library to download the archive file to the local system.

```python
import tinker
import requests

service = tinker.ServiceClient()
rest_client = service.create_rest_client()

# Get signed download URL
response = rest_client.get_checkpoint_archive_url(
    "run-123",
    "weights/checkpoint-001"
).result()

download_url = response.url
print(f"Download URL: {download_url}")

# Download the archive
archive_response = requests.get(download_url)
with open("checkpoint.tar.gz", "wb") as f:
    f.write(archive_response.content)

print("Checkpoint downloaded")
```

--------------------------------

### Tinker CLI Command Examples

Source: https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/cli/AGENTS.md

Common commands for interacting with Tinker, including version checking, listing and showing details for runs and checkpoints, and pushing checkpoints to Hugging Face Hub.

```bash
# Show version
tinker version
```

```bash
# List all training runs
tinker run list
```

```bash
# Show run details
tinker run info run-abc123
```

```bash
# List all checkpoints
tinker checkpoint list
```

```bash
# List checkpoints for specific run
tinker checkpoint list run-abc123
```

```bash
# Show checkpoint details
tinker checkpoint info ckpt-xyz789
```

```bash
# Upload checkpoint to Hugging Face Hub
tinker checkpoint push-hf tinker://run-abc123/sampler_weights/000040 --repo username/my-lora-adapter
```

```bash
# JSON output
tinker --format json run list
tinker --format json checkpoint list
```

--------------------------------

### AwaitableConcurrentFuture Example Usage

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/apifuture.md

Demonstrates how to wrap a standard concurrent.futures.Future with AwaitableConcurrentFuture for unified sync/async access. Typically received from API methods.

```python
# Internal usage - typically you receive these from API methods
concurrent_future = some_operation()
api_future = AwaitableConcurrentFuture(concurrent_future)

# Can be used synchronously
result = api_future.result()

# Or asynchronously
result = await api_future
```

--------------------------------

### TrainingClient API Reference

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/MANIFEST.md

Documentation for the TrainingClient class, including its 9 public methods and their asynchronous variants. Details parameter tables, return types, exceptions, and usage examples.

```APIDOC
## TrainingClient Methods

### Description
Handles machine learning model training operations.

### Method Signature
(Details for each of the 9 methods and their async variants would be listed here, including parameter types and return types.)

### Parameters Table
(Markdown table detailing parameters for each method: name, type, required, default, description.)

### Returns
(Exact return type and semantics for each method.)

### Raises
(All possible exceptions and trigger conditions for each method.)

### Example
(Real usage code examples for each method.)

### Cross-references
(Links to related documentation, e.g., types.md, errors.md.)
```

--------------------------------

### Get Weights Info by Tinker Path

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/restclient.md

Fetches checkpoint information from a given Tinker path. The returned future is awaitable and contains WeightsInfoResponse.

```python
future = rest_client.get_weights_info_by_tinker_path("tinker://run-id/weights/checkpoint-001")
response = future.result()  # or await future
print(f"Base Model: {response.base_model}, LoRA Rank: {response.lora_rank}")
```

--------------------------------

### Get Server Capabilities

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Query the server for its supported features, models, and resource limits. This method returns a response object containing details about the server's configuration.

```python
capabilities = service_client.get_server_capabilities()
print(f"Supported models: {capabilities.supported_models}")
print(f"Max batch size: {capabilities.max_batch_size}")
```

--------------------------------

### Save Weights and Get Sampling Client

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/training-client.md

Combines saving model weights with creating a SamplingClient for immediate inference. This is useful for quickly setting up a model for text generation after training or saving.

```python
# Train, save, and prepare for inference
sampling_client = training_client.save_weights_and_get_sampling_client("my-checkpoint")

# Now use sampling_client for text generation
prompt = types.ModelInput.from_ints(tokenizer.encode("Hello"))
future = sampling_client.sample(prompt, types.SamplingParams(max_tokens=50), num_samples=1)
result = future.result()
```

--------------------------------

### Compute Logprobs Without Generation

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/examples.md

This example shows how to compute only the log probabilities for a given prompt without generating any new text. The result will contain prompt logprobs but no generated sequences.

```python
import tinker

sampler = service.create_sampling_client(base_model="Qwen/Qwen3-8B")

# Just compute logprobs without generating
prompt = tinker.ModelInput.from_ints(tokenizer.encode("Hello world"))
result = sampler.compute_logprobs(prompt).result()

# Result has prompt_logprobs but no generated sequences
print(f"Prompt logprobs: {result.prompt_logprobs}")
print(f"Sequences: {len(result.sequences)}")  # Should be 0
```

--------------------------------

### Initialize ServiceClient and Create Clients

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/serviceclient.md

Demonstrates how to initialize the ServiceClient and create instances of TrainingClient, SamplingClient, and RestClient. Initialization of the ServiceClient is near-instant, while creating a TrainingClient may take a moment to assign resources.

```python
# Near instant
client = ServiceClient()

# Takes a moment as we initialize the model and assign resources
training_client = client.create_lora_training_client(base_model="Qwen/Qwen3-8B")

# Near-instant
sampling_client = client.create_sampling_client(base_model="Qwen/Qwen3-8B")

# Near-instant
rest_client = client.create_rest_client()
```

--------------------------------

### Get Training Run by ID

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/restclient.md

Retrieves information about a specific training run using its ID. This method returns a Future that can be used to get the result.

```python
future = rest_client.get_training_run("run-id")
response = future.result()
print(f"Training Run ID: {response.training_run_id}, Base: {response.base_model}")
```

--------------------------------

### Initialize and Use RestClient

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/restclient.md

Demonstrates how to create a RestClient instance and use it to retrieve training run details and list checkpoints. Requires an existing service client.

```python
rest_client = service_client.create_rest_client()
training_run = rest_client.get_training_run("run-id").result()
print(f"Training Run: {training_run.training_run_id}, LoRA: {training_run.is_lora}")
checkpoints = rest_client.list_checkpoints("run-id").result()
print(f"Found {len(checkpoints.checkpoints)} checkpoints")
for checkpoint in checkpoints.checkpoints:
    print(f"  {checkpoint.checkpoint_type}: {checkpoint.checkpoint_id}")
```

--------------------------------

### SamplingClient API Reference

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/MANIFEST.md

Documentation for the SamplingClient class, including its 4 public methods and their asynchronous variants. Details parameter tables, return types, exceptions, and usage examples.

```APIDOC
## SamplingClient Methods

### Description
Manages data sampling operations.

### Method Signature
(Details for each of the 4 methods and their async variants would be listed here, including parameter types and return types.)

### Parameters Table
(Markdown table detailing parameters for each method: name, type, required, default, description.)

### Returns
(Exact return type and semantics for each method.)

### Raises
(All possible exceptions and trigger conditions for each method.)

### Example
(Real usage code examples for each method.)

### Cross-references
(Links to related documentation, e.g., types.md, errors.md.)
```

--------------------------------

### Get Weights Info by Tinker Path

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/rest-client.md

Fetches detailed checkpoint information, including model configuration and LoRA details, from a given Tinker path. The result is returned via an `APIFuture`.

```python
future = rest_client.get_weights_info_by_tinker_path("tinker://run-id/weights/checkpoint-001")
response = future.result()  # or await future
print(f"Base Model: {response.base_model}")
print(f"LoRA Rank: {response.lora_rank}")
print(f"Is LoRA: {response.is_lora}")
```

--------------------------------

### Get ModelInput Length

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/types.md

Returns the total context length of the ModelInput.

```python
def length() -> int
```

--------------------------------

### Handle APITimeoutError

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/errors.md

Example of catching an APITimeoutError when a request exceeds the specified timeout duration.

```python
try:
    result = training_client.forward_backward(data, "cross_entropy").result(timeout=30)
except tinker.APITimeoutError as e:
    print("Request timed out after 30 seconds")
```

--------------------------------

### SidecarStartupError Definition

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/errors.md

Defines the SidecarStartupError, raised when a sidecar subprocess fails to start or times out.

```python
class SidecarStartupError(SidecarError):
    """Raised when the sidecar subprocess fails to start or times out."""
```

--------------------------------

### SessionStartEvent

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/types.md

Represents a session start telemetry event, including the event type and log severity.

```APIDOC
## `SessionStartEvent` Objects

```python
class SessionStartEvent(BaseModel)
```

#### `event`

Telemetry event type

#### `severity`

Log severity level
```

--------------------------------

### Initialize ServiceClient

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Create a new ServiceClient instance. Optional parameters include user metadata and project ID. Advanced keyword arguments can configure the underlying HTTP client.

```python
client = ServiceClient()

client = ServiceClient(
    user_metadata={"user_id": "123", "env": "production"},
    project_id="my-project"
)
```

--------------------------------

### Initialize TrainingClient

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/trainingclient.md

Create a TrainingClient for fine-tuning a specified base model. This client is used for training operations.

```python
training_client = service_client.create_lora_training_client(base_model="Qwen/Qwen3-8B")
fwdbwd_future = training_client.forward_backward(training_data, "cross_entropy")
optim_future = training_client.optim_step(types.AdamParams(learning_rate=1e-4))
fwdbwd_result = training_client.save_weights_and_get_sampling_client("my-model")
```

--------------------------------

### Get Base Model Name

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/samplingclient.md

Retrieve the name of the base model used for the current sampling session.

```python
def get_base_model() -> str:
    return "base_model_name"
```

--------------------------------

### Create TrainingClient from Saved State with Optimizer

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Instantiate a TrainingClient using saved model weights and optimizer state. This is ideal for resuming training exactly where it left off, preserving optimizer momentum.

```python
training_client = service_client.create_training_client_from_state_with_optimizer(
    "tinker://run-id/weights/checkpoint-001"
)
```

--------------------------------

### Get Tokenizer

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/samplingclient.md

Retrieve the tokenizer associated with the current model. This tokenizer is compatible with the model for encoding and decoding.

```python
def get_tokenizer() -> PreTrainedTokenizer:
    return PreTrainedTokenizer
```

--------------------------------

### SamplingClient.sample_async()

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/sampling-client.md

Asynchronously generates text completions from a prompt. This is the asynchronous version of the `sample()` method.

```APIDOC
## SamplingClient.sample_async()

### Description
Async version of `sample()`.

### Method Signature
```python
async def sample_async(
    prompt: types.ModelInput,
    sampling_params: types.SamplingParams,
    num_samples: int = 1,
    include_prompt_logprobs: bool = False,
    topk_prompt_logprobs: int = 0,
) -> types.SampleResponse
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters Table
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| prompt | types.ModelInput | Yes | — | Input prompt tokens |
| sampling_params | types.SamplingParams | Yes | — | Sampling configuration (temperature, top_k, top_p, max_tokens, etc.) |
| num_samples | int | No | 1 | Number of independent samples to generate |
| include_prompt_logprobs | bool | No | False | Include per-token logprobs for prompt tokens |
| topk_prompt_logprobs | int | No | 0 | Return top-k logprobs for each prompt token (0 = disabled) |

### Returns
`types.SampleResponse`

### Raises
- `tinker.RequestFailedError` - If sampling fails on server
- `tinker.APIStatusError` - If API request fails
```

--------------------------------

### Get Sampler Information

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/restclient.md

Retrieves details for a specific sampler using its ID. Supports both synchronous and asynchronous usage.

```python
# Sync usage
future = rest_client.get_sampler("session-id:sample:0")
response = future.result()
print(f"Base model: {response.base_model}")
print(f"Model path: {response.model_path}")

# Async usage
response = await rest_client.get_sampler("session-id:sample:0")
print(f"Base model: {response.base_model}")
```

--------------------------------

### create_training_client_from_state_with_optimizer

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Creates a TrainingClient from saved model weights and optimizer state, allowing for exact resumption of training.

```APIDOC
## create_training_client_from_state_with_optimizer()

### Description
Create a TrainingClient from saved model weights and optimizer state. Use this to resume training exactly where it left off with optimizer momentum preserved.

### Method
```python
def create_training_client_from_state_with_optimizer(
    path: str,
    user_metadata: dict[str, str] | None = None,
    weights_access_token: str | None = None,
) -> TrainingClient
```

### Parameters
#### Path Parameters
- **path** (str) - Required - Tinker path to saved weights (e.g., "tinker://run-id/weights/checkpoint-001")

#### Query Parameters
- **user_metadata** (dict[str, str] | None) - Optional - Optional metadata to attach to the new training run
- **weights_access_token** (str | None) - Optional - Optional access token for loading checkpoints under a different account

### Returns
`TrainingClient` loaded with weights and optimizer state

### Raises
- `tinker.NotFoundError` - If checkpoint or optimizer state does not exist
- `tinker.PermissionDeniedError` - If insufficient access to the checkpoint
- `tinker.APIStatusError` - If loading fails

### Example
```python
# Resume training with full state restoration
training_client = service_client.create_training_client_from_state_with_optimizer(
    "tinker://run-id/weights/checkpoint-001"
)
# Continue training with restored optimizer momentum
```
```

--------------------------------

### Get Checkpoint Archive URL (Async)

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/rest-client.md

Asynchronous version of get_checkpoint_archive_url. Use this for non-blocking download URL retrieval.

```python
async def get_checkpoint_archive_url_async(
    training_run_id: types.ModelID,
    checkpoint_id: str,
) -> types.CheckpointArchiveUrlResponse
```

--------------------------------

### Get Training Telemetry

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/training-client.md

Retrieves telemetry data for the current training run. Returns None if telemetry is not available.

```python
def get_telemetry(self) -> Telemetry | None:
    """Get telemetry data for this training run. Returns None if telemetry is not available."""
    ...

telemetry = training_client.get_telemetry()
if telemetry:
    print(f"Training telemetry: {telemetry}")
```

--------------------------------

### Create LoRA Training Client

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/README.md

This snippet demonstrates how to create a LoRA training client from an initialized ServiceClient, specifying the base model for fine-tuning.

```python
# 2. Create trainer
trainer = service.create_lora_training_client(base_model="Qwen/Qwen3-8B")
```

--------------------------------

### Get Telemetry Data

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/sampling-client.md

Retrieve telemetry data for the current sampling session. Returns None if telemetry is not available.

```python
def get_telemetry(self) -> Telemetry | None:
```

```python
telemetry = sampling_client.get_telemetry()
if telemetry:
    print(f"Sampling telemetry: {telemetry}")
```

--------------------------------

### APIFuture - result

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/apifuture.md

Get the result synchronously with an optional timeout. This method blocks until the operation is complete or the timeout is reached.

```APIDOC
## `result`

### Description
Get the result synchronously with optional timeout. This method blocks until the operation is complete.

### Method
`def result(timeout: float | None = None) -> T`

### Parameters
#### Query Parameters
- **timeout** (float | None) - Optional - Maximum time to wait in seconds. None means wait indefinitely.

### Response
#### Success Response (T)
- Returns the result value of type `T`.

### Raises
- **TimeoutError**: If timeout is exceeded
```

--------------------------------

### create_lora_training_client_async

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Asynchronously creates a TrainingClient for LoRA fine-tuning. This is the asynchronous version of `create_lora_training_client`.

```APIDOC
## create_lora_training_client_async

### Description
Asynchronously creates a TrainingClient for LoRA fine-tuning. This is the asynchronous version of `create_lora_training_client`.

### Method
POST

### Endpoint
/training/lora

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **base_model** (str) - Required - Name of base model (e.g., "Qwen/Qwen3-8B", "Meta-Llama-3-8B")
- **rank** (int) - Optional - LoRA rank controlling size of adaptation matrices (default: 32)
- **seed** (int | None) - Optional - Random seed for initialization. None means random seed.
- **train_mlp** (bool) - Optional - Whether to train MLP (feedforward) layers (default: True)
- **train_attn** (bool) - Optional - Whether to train attention layers (default: True)
- **train_unembed** (bool) - Optional - Whether to train unembedding layers (default: True)
- **user_metadata** (dict[str, str] | None) - Optional - Optional metadata to attach to training run

### Response
#### Success Response (200)
- **TrainingClient** - A TrainingClient instance ready for forward/backward passes and optimization steps.

### Raises
- `AssertionError` - If none of `train_mlp`, `train_attn`, `train_unembed` are True.
- `tinker.APIStatusError` - If model creation fails on the server.
- `tinker.APIConnectionError` - If connection to the server fails.

### Example
```python
# Create with default settings
training_client = await service_client.create_lora_training_client_async(
    base_model="Qwen/Qwen3-8B"
)

# With custom configuration
training_client = await service_client.create_lora_training_client_async(
    base_model="Meta-Llama-3-8B",
    rank=16,
    train_mlp=True,
    train_attn=True,
    train_unembed=False
)
```
```

--------------------------------

### Using APIFuture in Sync Context

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/apifuture.md

Shows how to get the result of an APIFuture synchronously. This method will block until the future is complete.

```python
result = api_future.result()  # Blocks until complete
```

--------------------------------

### Handle InternalServerError

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/errors.md

Example of catching an InternalServerError, suggesting the user to try again later due to a temporary server issue.

```python
try:
    result = training_client.forward_backward(data, "cross_entropy").result()
except tinker.InternalServerError as e:
    print("Server error. Please try again later.")
```

--------------------------------

### create_training_client_from_state_with_optimizer_async

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Asynchronous version of `create_training_client_from_state_with_optimizer()`. Creates a TrainingClient from saved model weights and optimizer state.

```APIDOC
## create_training_client_from_state_with_optimizer_async()

### Description
Async version of `create_training_client_from_state_with_optimizer()`.

### Method
```python
async def create_training_client_from_state_with_optimizer_async(
    path: str,
    user_metadata: dict[str, str] | None = None,
    weights_access_token: str | None = None,
) -> TrainingClient
```

### Parameters
#### Path Parameters
- **path** (str) - Required - Tinker path to saved weights (e.g., "tinker://run-id/weights/checkpoint-001")

#### Query Parameters
- **user_metadata** (dict[str, str] | None) - Optional - Optional metadata to attach to the new training run
- **weights_access_token** (str | None) - Optional - Optional access token for loading checkpoints under a different account

### Returns
`TrainingClient` loaded with weights and optimizer state
```

--------------------------------

### sample_async

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/samplingclient.md

Asynchronous version of the sample method for generating text completions.

```APIDOC
## `sample_async`

### Description
Async version of sample.

### Method
`sample_async`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **prompt** (types.ModelInput) - Required - The input tokens as ModelInput
- **num_samples** (int) - Required - Number of independent samples to generate
- **sampling_params** (types.SamplingParams) - Required - Parameters controlling generation (temperature, max_tokens, etc.)
- **include_prompt_logprobs** (bool) - Optional - Whether to include log probabilities for prompt tokens (defaults to False)
- **topk_prompt_logprobs** (int) - Optional - Number of top token log probabilities to return per position (defaults to 0)

### Returns
- `types.SampleResponse` - The response with generated text
```

--------------------------------

### SessionStartEvent Class Definition

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/types.md

Represents a session start telemetry event, including the event type and severity level.

```python
class SessionStartEvent(BaseModel)

```

--------------------------------

### EventType Literal

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/types.md

Specifies the allowed event types that can occur during a session, such as session start or end, and unhandled exceptions.

```python
EventType = Literal["session_start", "session_end", "unhandled_exception"]
```

--------------------------------

### Async Sampling Client Creation

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Asynchronously creates a SamplingClient. Use this for non-blocking client initialization.

```python
async def create_sampling_client_async(
    model_path: str | None = None,
    base_model: str | None = None,
    retry_config: RetryConfig | None = None,
) -> SamplingClient
```

--------------------------------

### Create LoRA Training Client

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Instantiate a TrainingClient for LoRA fine-tuning. Specify the base model and optionally configure LoRA rank, training layers, and random seed. User metadata can also be provided for the training run.

```python
training_client = service_client.create_lora_training_client(
    base_model="Qwen/Qwen3-8B"
)

training_client = service_client.create_lora_training_client(
    base_model="Meta-Llama-3-8B",
    rank=16,
    train_mlp=True,
    train_attn=True,
    train_unembed=False
)
```

--------------------------------

### WeightsInfoResponse

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/types.md

Provides minimal information for loading public checkpoints. This response is used to get basic details about available weights.

```APIDOC
## `WeightsInfoResponse` Objects

```python
class WeightsInfoResponse(BaseModel)
```

### Description
Minimal information for loading public checkpoints.
```

--------------------------------

### Get Model Tokenizer

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/trainingclient.md

Retrieves the tokenizer for the current model. This tokenizer is compatible with the model and can be used for encoding and decoding text.

```python
tokenizer = training_client.get_tokenizer()
tokens = tokenizer.encode("Hello world")
text = tokenizer.decode(tokens)
```

--------------------------------

### create_training_client_from_state

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Creates a TrainingClient from saved model weights. This method loads model weights but not the optimizer state. Use `create_training_client_from_state_with_optimizer()` to resume with optimizer state.

```APIDOC
## create_training_client_from_state()

### Description
Creates a TrainingClient from saved model weights. Loads model weights but not optimizer state; to resume with optimizer state, use `create_training_client_from_state_with_optimizer()`.

### Method
```python
def create_training_client_from_state(
    path: str,
    user_metadata: dict[str, str] | None = None,
    weights_access_token: str | None = None,
) -> TrainingClient
```

### Parameters
#### Path Parameters
- **path** (str) - Required - Tinker path to saved weights (e.g., "tinker://run-id/weights/checkpoint-001")

#### Query Parameters
- **user_metadata** (dict[str, str] | None) - Optional - Optional metadata to attach to the new training run
- **weights_access_token** (str | None) - Optional - Optional access token for loading checkpoints under a different account

### Returns
`TrainingClient` loaded with the specified weights

### Raises
- `tinker.NotFoundError` - If checkpoint path does not exist
- `tinker.PermissionDeniedError` - If insufficient access to the checkpoint
- `tinker.APIStatusError` - If weight loading fails

### Example
```python
# Resume training from a checkpoint (weights only, optimizer resets)
training_client = service_client.create_training_client_from_state(
    "tinker://run-id/weights/checkpoint-001"
)

# With access token for different account
training_client = service_client.create_training_client_from_state(
    "tinker://run-id/weights/checkpoint-001",
    weights_access_token="token123"
)
```
```

--------------------------------

### Get Model Information

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/trainingclient.md

Retrieves information about the current model, including its configuration and metadata. Use this to inspect model details.

```python
info = training_client.get_info()
print(f"Model ID: {info.model_data.model_id}")
print(f"Base model: {info.model_data.model_name}")
print(f"LoRA rank: {info.model_data.lora_rank}")
```

--------------------------------

### Create SamplingClient

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/samplingclient.md

Instantiate a SamplingClient using a base model. This client is used for text generation and inference.

```python
sampling_client = service_client.create_sampling_client(base_model="Qwen/Qwen3-8B")
prompt = types.ModelInput.from_ints(tokenizer.encode("The weather today is"))
params = types.SamplingParams(max_tokens=20, temperature=0.7)
future = sampling_client.sample(prompt=prompt, sampling_params=params, num_samples=1)
result = future.result()
```

--------------------------------

### AwaitableConcurrentFuture - future

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/apifuture.md

Get the underlying concurrent.futures.Future object. This allows direct interaction with the standard Python concurrent futures API.

```APIDOC
## `future`

### Description
Get the underlying concurrent.futures.Future object. This allows direct interaction with the standard Python concurrent futures API.

### Method
`def future() -> ConcurrentFuture[T]`

### Response
#### Success Response (ConcurrentFuture[T])
- Returns the wrapped `ConcurrentFuture` object.
```

--------------------------------

### AwaitableConcurrentFuture - result

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/apifuture.md

Get the result synchronously with an optional timeout. This method wraps a concurrent.futures.Future and provides synchronous access to its result.

```APIDOC
## `result`

### Description
Get the result synchronously with optional timeout. This method wraps a concurrent.futures.Future and provides synchronous access to its result.

### Method
`def result(timeout: float | None = None) -> T`

### Parameters
#### Query Parameters
- **timeout** (float | None) - Optional - Maximum time to wait in seconds. None means wait indefinitely.

### Response
#### Success Response (T)
- Returns the result value of type `T`.

### Raises
- **TimeoutError**: If timeout is exceeded
- **Exception**: Any exception raised by the underlying operation
```

--------------------------------

### sample

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/samplingclient.md

Generates text completions from the model with customizable parameters. It returns a Future containing the SampleResponse.

```APIDOC
## `sample`

### Description
Generate text completions from the model.

### Method
`sample`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **prompt** (types.ModelInput) - Required - The input tokens as ModelInput
- **num_samples** (int) - Required - Number of independent samples to generate
- **sampling_params** (types.SamplingParams) - Required - Parameters controlling generation (temperature, max_tokens, etc.)
- **include_prompt_logprobs** (bool) - Optional - Whether to include log probabilities for prompt tokens (defaults to False)
- **topk_prompt_logprobs** (int) - Optional - Number of top token log probabilities to return per position (defaults to 0)

### Returns
- A `Future` containing the `SampleResponse` with generated text
```

--------------------------------

### Handling RequestFailedError

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/errors.md

Example of catching and handling a RequestFailedError during a forward-backward pass. It prints the request ID, message, and category of the failure.

```python
try:
    fwdbwd_future = training_client.forward_backward(data, "cross_entropy")
    result = fwdbwd_future.result()
except tinker.RequestFailedError as e:
    print(f"Request {e.request_id} failed: {e.message}")
    print(f"Category: {e.category}")
```

--------------------------------

### Client Configuration Copying

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/configuration.md

Create modified client configurations efficiently using the copy() or with_options() methods. This avoids recreating the entire client object when only a few parameters need changing.

```python
# Original client
client = ServiceClient(api_key="key1", timeout=30)

# Create variant with different timeout
faster_client = client.copy(timeout=10)

# Create variant with different headers
custom_client = client.copy(
    default_headers={"X-Custom": "value"}
)

# Alias: same as copy()
variant = client.with_options(max_retries=5)
```

--------------------------------

### Handle UnprocessableEntityError

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/errors.md

Example of catching an UnprocessableEntityError when creating a Datum with invalid loss function inputs or mismatched tensor shapes.

```python
try:
    datum = tinker.Datum(
        model_input=tinker.ModelInput.from_ints([1, 2, 3]),
        loss_fn_inputs={"invalid_key": tinker.TensorData(...)}  # doctest: +SKIP
    )
    result = training_client.forward_backward([datum], "cross_entropy").result()
except tinker.UnprocessableEntityError as e:
    print(f"Invalid data: {e.body}")
```

--------------------------------

### create_training_client_from_state_async

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Asynchronous version of `create_training_client_from_state()`. Creates a TrainingClient from saved model weights without optimizer state.

```APIDOC
## create_training_client_from_state_async()

### Description
Async version of `create_training_client_from_state()`.

### Method
```python
async def create_training_client_from_state_async(
    path: str,
    user_metadata: dict[str, str] | None = None,
    weights_access_token: str | None = None,
) -> TrainingClient
```

### Parameters
#### Path Parameters
- **path** (str) - Required - Tinker path to saved weights (e.g., "tinker://run-id/weights/checkpoint-001")

#### Query Parameters
- **user_metadata** (dict[str, str] | None) - Optional - Optional metadata to attach to the new training run
- **weights_access_token** (str | None) - Optional - Optional access token for loading checkpoints under a different account

### Returns
`TrainingClient` loaded with the specified weights
```

--------------------------------

### TrainingClient Constructor (Internal)

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/training-client.md

Internal constructor for TrainingClient. Use `ServiceClient.create_lora_training_client()` for creating instances.

```python
TrainingClient(
    holder: InternalClientHolder,
    model_seq_id: int,
    model_id: types.ModelID
)
```

--------------------------------

### SamplingClient.sample()

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/sampling-client.md

Generates text completions from a given prompt using specified sampling parameters. This method is synchronous.

```APIDOC
## SamplingClient.sample()

### Description
Generate text completions from a prompt with customizable sampling parameters.

### Method Signature
```python
def sample(
    prompt: types.ModelInput,
    sampling_params: types.SamplingParams,
    num_samples: int = 1,
    include_prompt_logprobs: bool = False,
    topk_prompt_logprobs: int = 0,
) -> APIFuture[types.SampleResponse]
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters Table
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| prompt | types.ModelInput | Yes | — | Input prompt tokens |
| sampling_params | types.SamplingParams | Yes | — | Sampling configuration (temperature, top_k, top_p, max_tokens, etc.) |
| num_samples | int | No | 1 | Number of independent samples to generate |
| include_prompt_logprobs | bool | No | False | Include per-token logprobs for prompt tokens |
| topk_prompt_logprobs | int | No | 0 | Return top-k logprobs for each prompt token (0 = disabled) |

### Returns
`APIFuture[SampleResponse]` containing generated sequences and optional logprobs

### Raises
- `tinker.RequestFailedError` - If sampling fails on server
- `tinker.APIStatusError` - If API request fails

### Example
```python
# Simple generation
prompt = types.ModelInput.from_ints(tokenizer.encode("The weather today is"))
params = types.SamplingParams(max_tokens=20, temperature=0.7)
future = sampling_client.sample(prompt, params, num_samples=1)
result = future.result()

for sequence in result.sequences:
    print(f"Generated: {tokenizer.decode(sequence.tokens)}")
    print(f"Stop reason: {sequence.stop_reason}")

# With prompt logprobs
params = types.SamplingParams(max_tokens=50, temperature=1.0)
future = sampling_client.sample(
    prompt, params, num_samples=1,
    include_prompt_logprobs=True,
    topk_prompt_logprobs=5
)
result = future.result()
print(f"Prompt logprobs: {result.prompt_logprobs}")
print(f"Top-k prompt logprobs: {result.topk_prompt_logprobs}")
```
```

--------------------------------

### Create TrainingClient from Saved Weights

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Instantiate a TrainingClient using saved model weights. This method loads weights but not the optimizer state. Use this to resume training without preserving optimizer momentum.

```python
training_client = service_client.create_training_client_from_state(
    "tinker://run-id/weights/checkpoint-001"
)
```

```python
training_client = service_client.create_training_client_from_state(
    "tinker://run-id/weights/checkpoint-001",
    weights_access_token="token123"
)
```

--------------------------------

### Create SamplingClient from Base Model or Weights

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Instantiate a SamplingClient for text generation. You can specify either a base model name or a path to saved model weights.

```python
sampling_client = service_client.create_sampling_client(
    base_model="Qwen/Qwen3-8B"
)
```

```python
sampling_client = service_client.create_sampling_client(
    model_path="tinker://run-id/weights/checkpoint-001"
)
```

--------------------------------

### create_sampling_client_async

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/service-client.md

Asynchronously creates a SamplingClient. This is the asynchronous version of create_sampling_client.

```APIDOC
## create_sampling_client_async()

### Description
Asynchronously creates a SamplingClient. This is the asynchronous version of `create_sampling_client()`.

### Method
Asynchronous Function Call

### Parameters
- **model_path** (str | None) - Optional - Path to the model.
- **base_model** (str | None) - Optional - Base model to use.
- **retry_config** (RetryConfig | None) - Optional - Configuration for retries.

### Returns
- **SamplingClient** - An instance of SamplingClient.
```

--------------------------------

### Get Checkpoint Archive URL

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/restclient.md

Retrieves a signed URL to download a checkpoint archive. Use this when you need to download specific checkpoint weights.

```python
future = rest_client.get_checkpoint_archive_url("run-id", "checkpoint-123")
response = future.result()
print(f"Download URL: {response.url}")
print(f"Expires at: {response.expires_at}")
# Use the URL to download the archive with your preferred HTTP client
```

--------------------------------

### SamplingClient.create_async()

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/sampling-client.md

Asynchronously creates a SamplingClient instance. This is the asynchronous counterpart to the `create()` method.

```APIDOC
## SamplingClient.create_async()

### Description
Async version of `create()`.

### Method Signature
```python
@staticmethod
async def create_async(
    holder: InternalClientHolder,
    *,
    model_path: str | None = None,
    base_model: str | None = None,
    retry_config: RetryConfig | None = None,
) -> SamplingClient
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters Table
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| holder | InternalClientHolder | Yes | — | Internal client managing connections |
| model_path | str \| None | No | None | Path to saved weights (e.g., "tinker://run-id/weights/checkpoint-001") |
| base_model | str \| None | No | None | Base model name (e.g., "Qwen/Qwen3-8B") |
| retry_config | RetryConfig \| None | No | None | Optional retry configuration |

### Returns
`SamplingClient`

### Raises
- `ValueError` - If neither model_path nor base_model provided
```

--------------------------------

### AwaitableConcurrentFuture result Method Example

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/apifuture.md

Shows how to synchronously retrieve the result from an AwaitableConcurrentFuture, with an optional timeout. Catches potential exceptions from the underlying operation.

```python
future = rest_client.get_training_run("run-id")
result = future.result(timeout=30)  # Wait up to 30 seconds
```

--------------------------------

### LoRA Training Configuration

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/configuration.md

Set up a client for LoRA training by specifying the base model, LoRA rank, training targets (MLP, attention, unembedding), and optional user metadata for tracking.

```python
training_client = service_client.create_lora_training_client(
    base_model="Qwen/Qwen3-8B",  # Base model name
    rank=32,                      # LoRA rank (default: 32)
    seed=None,                    # Random seed (default: random)
    train_mlp=True,              # Train MLP layers
    train_attn=True,             # Train attention layers
    train_unembed=True,          # Train unembedding layers
    user_metadata={
        "experiment": "exp-001",
        "researcher": "alice"
    }
)
```

--------------------------------

### Generate Tinker SDK Docs

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/README.md

Run this script to generate the API documentation for the Tinker Python SDK. Ensure that only types, classes, and methods with attached doc-strings are included in the generated documentation.

```bash
uv run scripts/generate_docs.py
```

--------------------------------

### Synchronous Blocking on Async Operation

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/api-future.md

Starts an asynchronous operation and then blocks synchronously to wait for its result, with an optional timeout. Can also convert to a `concurrent.futures.Future`.

```python
# Start async operation, wait synchronously
api_future = training_client.forward_backward(data, "cross_entropy")
result = api_future.result(timeout=60)  # Blocks for up to 60 seconds

# Or convert to concurrent future for use with concurrent.futures utilities
concurrent_future = api_future.future()
done, pending = concurrent.futures.wait([concurrent_future], timeout=60)
```

--------------------------------

### Generate Text Completions with SamplingClient

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/sampling-client.md

Generate text completions from a prompt using customizable sampling parameters. This method supports generating multiple samples and including prompt log probabilities.

```python
# Simple generation
prompt = types.ModelInput.from_ints(tokenizer.encode("The weather today is"))
params = types.SamplingParams(max_tokens=20, temperature=0.7)
future = sampling_client.sample(prompt, params, num_samples=1)
result = future.result()

for sequence in result.sequences:
    print(f"Generated: {tokenizer.decode(sequence.tokens)}")
    print(f"Stop reason: {sequence.stop_reason}")

# With prompt logprobs
params = types.SamplingParams(max_tokens=50, temperature=1.0)
future = sampling_client.sample(
    prompt, params, num_samples=1,
    include_prompt_logprobs=True,
    topk_prompt_logprobs=5
)
result = future.result()
print(f"Prompt logprobs: {result.prompt_logprobs}")
print(f"Top-k prompt logprobs: {result.topk_prompt_logprobs}")
```

--------------------------------

### SidecarStartupError

Source: https://github.com/thinking-machines-lab/tinker/blob/main/docs/api/exceptions.md

Raised when the sidecar subprocess fails to start correctly or encounters a timeout during its initialization phase. This indicates a problem with launching the auxiliary process.

```python
class SidecarStartupError(SidecarError)
```

--------------------------------

### Async Text Generation with SamplingClient

Source: https://github.com/thinking-machines-lab/tinker/blob/main/_autodocs/sampling-client.md

This is the asynchronous version of the sample() method. It generates text completions from a prompt with specified sampling parameters and can include prompt log probabilities.

```python
result = await sampling_client.sample_async(
    prompt,
    params,
    num_samples=1,
    include_prompt_logprobs=True,
    topk_prompt_logprobs=5
)
```