Predibase

https://docs.predibase.com
Admin
Predibase is an integrated platform for fine-tuning and serving open-source Large Language Models...
Tokens:53,384
Snippets:289
Trust Score:9.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### Install Predibase Python SDK

Source: https://docs.predibase.com/getting-started/quickstart

This command installs or upgrades the Predibase Python SDK using pip, making it available for use in your Python projects. It's the first step to setting up your development environment.

```Python
pip install -U predibase
```

--------------------------------

### Prompt a Shared Predibase LLM Endpoint

Source: https://docs.predibase.com/getting-started/quickstart

This Python example shows how to interact with Predibase's shared LLM endpoints, which are ideal for quick experimentation. It initializes the client, lists available models, connects to a specific shared deployment (e.g., 'qwen3-8b'), and generates text without requiring a private deployment.

```Python
from predibase import Predibase

pb = Predibase(api_token="<PREDIBASE API TOKEN>")

# Get a list of available models
available_models = pb.deployments.list()

# Connect to a shared deployment
client = pb.deployments.client("qwen3-8b", max_new_tokens=32)

# Generate text
response = client.generate(
    "What are some popular tourist spots in San Francisco?"
)
print(response.generated_text)
```

--------------------------------

### Predibase SDK: Example of Getting an Adapter

Source: https://docs.predibase.com/sdk-reference/adapters

Shows how to retrieve an adapter using its ID with `pb.adapters.get`. The example then demonstrates printing various attributes of the fetched adapter, such as its name, version, description, and base model.

```Python
# Get an adapter by ID
adapter = pb.adapters.get("news-summarizer-model/1")

# Print adapter details
print(f"Adapter: {adapter.name} v{adapter.tag}")
print(f"Description: {adapter.description}")
print(f"Base Model: {adapter.base_model}")
```

--------------------------------

### Predibase SDK: Example of Asynchronous Fine-tuning Job Creation

Source: https://docs.predibase.com/sdk-reference/adapters

Illustrates how to use `pb.finetuning.jobs.create` to start an asynchronous fine-tuning job. This example configures an SFT job with a base model, task, epochs, rank, target modules, dataset, and repository, demonstrating a non-blocking call.

```Python
adapter_job = pb.finetuning.jobs.create(
    config=SFTConfig(
        base_model="qwen3-8b",
        task="instruction_tuning",
        epochs=1,
        rank=8,
        target_modules=["q_proj", "v_proj", "k_proj"],
    ),
    dataset="tldr_news",
    repo="news-summarizer-model",
    description="async job example",
    watch=False,
    show_tensorboard=False,
)
```

--------------------------------

### Create and Prompt a Private Predibase LLM Deployment

Source: https://docs.predibase.com/getting-started/quickstart

This Python code demonstrates how to initialize the Predibase client with your API token, create a new private LLM deployment with a specified base model (e.g., 'qwen3-8b'), and then generate text using the deployed model. It showcases the full lifecycle from deployment to inference.

```Python
from predibase import Predibase, DeploymentConfig

# Initialize the client with your API token
pb = Predibase(api_token="<PREDIBASE API TOKEN>")

# Create a deployment
deployment = pb.deployments.create(
    name="my-qwen3-8b",
    config=DeploymentConfig(base_model="qwen3-8b")
)

# Generate text
response = pb.deployments.client('my-qwen3-8b').generate(
    "What is a Large Language Model?",
    max_new_tokens=50
)
print(response.generated_text)
# "(LLM) Explained\n\nA large language model (LLM) is a type of artificial intelligence..."
```

--------------------------------

### Predibase SDK: Example of Fine-tuning Cost Estimation

Source: https://docs.predibase.com/sdk-reference/adapters

Demonstrates how to use `pb.finetuning.jobs.estimate_cost` to get a cost estimate for a fine-tuning job. The example uses an SFT configuration and a dataset, showing the expected output printed to the terminal.

```Python
pb.finetuning.jobs.estimate_cost(
    config=SFTConfig(
        base_model="qwen3-8b",
        epochs=1,
        rank=8,
        target_modules=["q_proj", "v_proj", "k_proj"],
    ),
    dataset="tldr_news"
)

# Output
# >>> Estimated Cost: $0.56
```

--------------------------------

### Stream Responses from a Predibase LLM Endpoint

Source: https://docs.predibase.com/getting-started/quickstart

This Python code demonstrates how to stream token-by-token responses from a Predibase LLM endpoint, which is useful for longer generations. It initializes the client, connects to a shared deployment, and iterates through the `generate_stream` method's output to print tokens as they are generated, excluding special tokens.

```Python
from predibase import Predibase

pb = Predibase(api_token="<PREDIBASE API TOKEN>")
client = pb.deployments.client("qwen3-8b")

# Stream tokens as they're generated
for response in client.generate_stream(
    "What are some popular tourist spots in San Francisco?", max_new_tokens=256
):
    if not response.token.special:
        print(response.token.text, sep="", end="", flush=True)
```

--------------------------------

### Predibase SDK: Get Repository by Name Example

Source: https://docs.predibase.com/sdk-reference/repos

Shows how to retrieve an existing repository using its name with `pb.repos.get`. The example then demonstrates accessing and printing the `name` and `description` attributes of the retrieved repository object.

```Python
# Get a repository by name
repo = pb.repos.get("text-summarization")

# Print repository details
print(f"Repository: {repo.name}")
print(f"Description: {repo.description}")
```

--------------------------------

### Example Raw Dataset Format for VLM Fine-tuning

Source: https://docs.predibase.com/fine-tuning/tasks/vision-language-fine-tuning

Illustrates the expected structure of a raw dataset, including prompt, completion, and image URL, suitable for Predibase VLM fine-tuning. This format serves as the input for subsequent processing steps.

```text
Prompt: "What is this a picture of?"
Completion: "A dog"
Image: <URL_TO_IMAGE>
```

--------------------------------

### Install Predibase SDK and Login

Source: https://docs.predibase.com/index

This snippet provides instructions for setting up the Predibase Python SDK. It covers installing the SDK using pip and then authenticating with the Predibase platform via the command-line login utility, which prompts the user to enter their API token.

```Shell
pip install -U predibase
pbase login  # Enter your API token when prompted
```

--------------------------------

### Predibase SDK: Create a New Repository Example

Source: https://docs.predibase.com/sdk-reference/repos

Demonstrates how to create a new adapter repository using the `pb.repos.create` method. This example sets a specific name and description for the new repository, which will be used to organize fine-tuning experiments.

```Python
# Create a new adapter repository
repo = pb.repos.create(
    name="text-summarization",
    description="Experiments for fine-tuning summarization models"
)
```

--------------------------------

### Install Predibase Python SDK

Source: https://docs.predibase.com/sdk-reference/installation

Installs the Predibase Python SDK using pip, the Python package manager. This command should be executed in a terminal or command prompt.

```Shell
pip install predibase
```

--------------------------------

### Install Predibase Python SDK and Authenticate

Source: https://docs.predibase.com/getting-started/introduction

This snippet provides the necessary commands to install the Predibase Python SDK using pip and then authenticate your environment by logging in with your Predibase API token. This is a fundamental step to enable programmatic interaction with Predibase services.

```Shell
pip install -U predibase
pbase login  # Enter your API token when prompted
```

--------------------------------

### Verify Predibase SDK Installation

Source: https://docs.predibase.com/sdk-reference/installation

Verifies the successful installation of the Predibase Python SDK by attempting to import the main Predibase client. This code should be run in a Python interpreter or script.

```Python
from predibase import Predibase
```

--------------------------------

### Predibase SDK: Example of Uploading Adapter Weights

Source: https://docs.predibase.com/sdk-reference/adapters

Illustrates how to use `pb.adapters.upload` to upload adapter weights from a local directory to a specified repository. The example demonstrates creating a new adapter version and printing its details upon successful upload.

```Python
# Upload adapter weights
adapter = pb.adapters.upload(
    local_dir="./my_adapter_weights",
    repo="news-summarizer-model",
)
print(f"Uploaded adapter: {adapter.name} v{adapter.tag}")
```

--------------------------------

### Predibase SDK: Create Model Deployment Examples

Source: https://docs.predibase.com/sdk-reference/deployments

Provides Python examples for creating Predibase model deployments. The first demonstrates a basic deployment with default configurations, while the second showcases advanced customization options like scaling, quantization, logging, and preloaded adapters.

```python
from predibase import DeploymentConfig

# Create a deployment with the default configurations
deployment = pb.deployments.create(
    name="my-qwen3-8b",
    config=DeploymentConfig(base_model="qwen3-8b")
)
```

```python
from predibase import DeploymentConfig

# Create a deployment with custom configuration
deployment = pb.deployments.create(
    name="my-qwen3-8b",
    config=DeploymentConfig(
        base_model="qwen3-8b",
        max_replicas=2,
        min_replicas=1,
        quantization="fp8", # Enable quantization
        max_total_tokens=4096, # Change the default context window size
        request_logging_enabled=True, # Enable request logging
        preloaded_adapters=["my-adapter/1", "my-adapter/2"], # Preload adapters for performance
        prefix_caching=True # Enable prefix caching
    ),
    description="Production-ready Qwen 3 model with logging"
)
```

--------------------------------

### Generate Text with Predibase Python SDK (Quick Start)

Source: https://docs.predibase.com/inference/deployments/shared

This example illustrates how to initialize the Predibase client with an API token and use a shared model endpoint (e.g., 'qwen3-8b') to generate text. It provides a quick and simple way to start experimenting with Predibase's text generation capabilities.

```Python
from predibase import Predibase

pb = Predibase(api_token="<PREDIBASE_API_TOKEN>")

# Use a shared endpoint for testing
client = pb.deployments.client("qwen3-8b")

# Generate text
response = client.generate("What is machine learning?")
print(response.generated_text)
```

--------------------------------

### Install Predibase Python SDK and Login

Source: https://docs.predibase.com/user-guide/llms/deploy_llm

Instructions to install the Predibase Python SDK using pip and log in to your Predibase account. The `pbase login` command will prompt you to enter your API token, which is required for authentication.

```Python
pip install -U predibase
pbase login  # Enter your API token when prompted
```

--------------------------------

### Predibase SDK: Example to Download Adapter Weights

Source: https://docs.predibase.com/sdk-reference/adapters

Illustrates how to download an adapter's weights to a specified local directory using the `pb.adapters.download` method and the adapter's ID.

```Python
# Download adapter weights
pb.adapters.download(
    adapter_id="news-summarizer-model/1",
    dest="./adapter_weights/"
)
```

--------------------------------

### Predibase SDK: Create Adapter within a Repository Example

Source: https://docs.predibase.com/sdk-reference/repos

Illustrates the process of creating a new adapter and associating it with a repository. This example first ensures a repository exists (creating it if necessary) and then uses its name to link a newly created adapter, demonstrating how repositories manage fine-tuning experiments.

```Python
from predibase import SFTConfig

# Create a repository
repo = pb.repos.create(name="my-experiments", description="My fine-tuning experiments", exists_ok=True)

# Create an adapter in the repository
adapter = pb.adapters.create(
    config=SFTConfig(
        base_model="qwen3-8b"
    ),
    dataset="my-dataset",
    repo=repo.name,  # Use the repository name
    description="First fine-tuning experiment"
)
```

--------------------------------

### Python SFTConfig Initialization Examples

Source: https://docs.predibase.com/sdk-reference/configuration/sft-config

Provides Python code examples demonstrating how to initialize the SFTConfig class for both basic and advanced supervised fine-tuning configurations, including setting base model, epochs, rank, and target modules.

```Python
from predibase import SFTConfig

# Basic configuration
config = SFTConfig(
    base_model="qwen3-8b"
)

# Advanced configuration
config = SFTConfig(
    base_model="qwen3-8b",
    task="instruction_tuning",
    epochs=1,
    rank=32,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"]
)
```

--------------------------------

### Set Predibase API Key via Environment Variable

Source: https://docs.predibase.com/sdk-reference/installation

Configures the Predibase API key by setting the `PREDIBASE_API_TOKEN` environment variable. This method is recommended for production environments to keep sensitive information out of code. Examples are provided for both Linux/Mac and Windows operating systems.

```Shell
# For Linux/Mac
export PREDIBASE_API_TOKEN=<PREDIBASE API TOKEN>

# For Windows
set PREDIBASE_API_TOKEN=<PREDIBASE API TOKEN>
```

--------------------------------

### Predibase SDK: Create or Get Existing Repository Example

Source: https://docs.predibase.com/sdk-reference/repos

Illustrates how to create a new repository or retrieve an existing one if it already exists. By setting `exists_ok=True`, the method avoids raising an error if a repository with the specified name is found, making the operation idempotent.

```Python
# Create a repository or get it if it already exists
repo = pb.repos.create(
    name="customer-support-bot",
    description="Models for customer support automation",
    exists_ok=True
)
```

--------------------------------

### Generate Text with Predibase Python SDK

Source: https://docs.predibase.com/inference/querying

This example demonstrates how to initialize the Predibase client, connect to a specific deployment (private or shared), and generate text. It also includes a separate example for streaming responses, which is useful for real-time output.

```Python
from predibase import Predibase

# Initialize client
pb = Predibase(api_token="<PREDIBASE_API_TOKEN>")

# Get deployment client
client = pb.deployments.client("my-deployment")  # or use a shared endpoint like "qwen3-8b"

# Generate text
response = client.generate(
    "What is machine learning?",
    max_new_tokens=100,
    temperature=0.7
)
print(response.generated_text)

# Stream responses
for response in client.generate_stream(
    "Write a story about a robot learning to paint.",
    max_new_tokens=200
):
    print(response.token.text, end="", flush=True)
```

--------------------------------

### Python Example for VLM Inference with Predibase via OpenAI API

Source: https://docs.predibase.com/fine-tuning/tasks/vision-language-fine-tuning

Python code demonstrating how to perform Vision-Language Model (VLM) inference using the OpenAI client configured to connect to a Predibase deployment. It shows client initialization with Predibase credentials, constructing a chat completion request with text and image URL content, and printing the model's response.

```python
from openai import OpenAI

# Initialize client
api_token = "<PREDIBASE_API_TOKEN>"
tenant_id = "<PREDIBASE_TENANT_ID>"
model_name = "<DEPLOYMENT_NAME>"  # Ex. "qwen2-5-vl-7b-instruct"
adapter = "<ADAPTER_REPO_NAME>/<VERSION_NUMBER>"  # Ex. "adapter-repo/1" (optional)
base_url = f"https://serving.app.predibase.com/{tenant_id}/deployments/v2/llms/{model_name}/v1"

client = OpenAI(
    api_key=api_token,
    base_url=base_url
)

# Chat completion
completion = client.chat.completions.create(
    model=adapter,  # Use empty string "" for base model
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What is this an image of?"
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "<IMAGE_URL_HERE>"
                    }
                }
            ]
        }
    ],
    max_tokens=100
)
print(completion.choices[0].message.content)
```

--------------------------------

### Python Example: Initialize Predibase LoRAX Client

Source: https://docs.predibase.com/sdk-reference/queryingmodels

Practical example demonstrating how to instantiate a `LoRAXClient` using `pb.deployments.client` for a specific model deployment, setting `force_bare_client` to `True` for direct production use.

```Python
client = pb.deployments.client("qwen3-8b", force_bare_client=True)
```

--------------------------------

### Python Example: Import Predibase SDK

Source: https://docs.predibase.com/sdk-reference/queryingmodels

Initial line of code demonstrating how to import the `Predibase` class from the `predibase` SDK, typically the first step before interacting with Predibase services.

```Python
from predibase import Predibase
```

--------------------------------

### Initialize Predibase Client with API Key

Source: https://docs.predibase.com/sdk-reference/installation

Initializes the Predibase Python client directly by passing the API token as an argument during object instantiation. This method is suitable for quick testing or environments where environment variables are not preferred.

```Python
from predibase import Predibase

pb = Predibase(api_token="<PREDIBASE API TOKEN>")
```

--------------------------------

### Instruction Format Dataset Schema Example

Source: https://docs.predibase.com/fine-tuning/tasks/tasks

Example dataset schema for Supervised Fine-Tuning (SFT) using the instruction format. This format uses 'prompt' and 'completion' columns, ideal for task-oriented applications like translation or summarization.

```JSON
{"prompt": "Translate to French: Hello world", "completion": "Bonjour le monde"}
```

--------------------------------

### Python Example: Augment Dataset with Predibase SDK

Source: https://docs.predibase.com/sdk-reference/datasets

This example demonstrates the full workflow of augmenting a dataset using the Predibase SDK. It shows how to create an `AugmentationConfig`, retrieve a source dataset, call `pb.datasets.augment`, and print details of the newly created augmented dataset.

```python
from predibase import AugmentationConfig

# Create an augmentation configuration
config = AugmentationConfig(
    base_model="gpt-4-turbo",  # Required: The OpenAI model to use
    num_samples_to_generate=500,  # Optional: Number of synthetic examples to generate
    num_seed_samples=10,  # Optional: Number of seed samples to use
    augmentation_strategy="mixture_of_agents",  # Optional: Augmentation strategy
    task_context="Generate diverse examples for customer service questions"  # Optional: Task context
)

# Get the source dataset
source_dataset = pb.datasets.get("customer_questions")

# Augment the dataset
augmented_dataset = pb.datasets.augment(
    config=config,
    dataset=source_dataset,
    name="augmented_customer_questions",
    openai_api_key="your-openai-api-key"  # Optional: If not set in environment
)

# Print dataset details
print(f"Created augmented dataset: {augmented_dataset.name}")
print(f"Number of rows: {augmented_dataset.num_rows}")
print(f"Created at: {augmented_dataset.created_at}")
```

--------------------------------

### Initialize Predibase Client

Source: https://docs.predibase.com/fine-tuning/synthetic

This snippet demonstrates how to import necessary Predibase libraries and initialize the Predibase client using your API token. The client object 'pb' is then used for subsequent operations.

```python
import os
import pandas as pd
from predibase import Predibase, AugmentationConfig

pb = Predibase(api_token="<YOUR_PREDIBASE_API_KEY>")
```

--------------------------------

### Install Predibase Python SDK

Source: https://docs.predibase.com/inference/deployments/shared

This snippet demonstrates how to install the Predibase Python SDK using pip. This is the foundational step required to interact with Predibase services programmatically and begin experimenting with shared endpoints.

```Shell
pip install -U predibase
```

--------------------------------

### Install Predibase Python SDK

Source: https://docs.predibase.com/inference/fine-tuned

Installs or upgrades the Predibase Python SDK using pip, a package installer for Python. This is the first step to interact with Predibase services programmatically.

```bash
pip install -U predibase
```

--------------------------------

### Generate Embeddings with Predibase SDK (Python Example)

Source: https://docs.predibase.com/sdk-reference/queryingmodels

Python example demonstrating how to generate single and batch text embeddings using the Predibase SDK's `pb.embeddings.create` method.

```python
from predibase import Predibase

# Generate embeddings for your data
text = "Generate embeddings using your dedicated deployment."
response = pb.embeddings.create(model="my-embedding-model", input=text)
print(f"Generated {len(response.data[0].embedding)}-dimensional embedding")

# Process a batch of documents
documents = [
    "First document for embedding",
    "Second document with different content",
    "Third document to process in batch"
]
batch_embeddings = [pb.embeddings.create(model="my-embedding-model", input=doc).data[0].embedding for doc in documents]
```

--------------------------------

### Install Predibase Python SDK

Source: https://docs.predibase.com/inference/querying

This snippet provides the command to install or upgrade the Predibase Python SDK using pip, ensuring you have the latest version for interacting with Predibase services.

```Python
pip install -U predibase
```

--------------------------------

### Install Predibase Python SDK

Source: https://docs.predibase.com/inference/models/language-models

This snippet shows how to install the Predibase Python SDK using pip. It's the essential first step to interact with Predibase services programmatically and access its features.

```python
pip install -U predibase
```

--------------------------------

### Install Predibase Python SDK

Source: https://docs.predibase.com/inference/overview

This command installs the Predibase Python SDK using pip. It is the recommended first step to interact with Predibase models programmatically, providing a simple and intuitive interface with full feature support.

```python
# install Python package
pip install -U predibase
```

--------------------------------

### Connect to an Existing Predibase Dataset

Source: https://docs.predibase.com/fine-tuning/synthetic

This code retrieves a reference to an existing dataset in Predibase by its path. This dataset will serve as the 'seed_dataset' for synthetic data generation.

```python
seed_dataset = pb.datasets.get("file_uploads/seed_dataset_20_rows")
```

--------------------------------

### Prompt Predibase Adapter for Text Generation

Source: https://docs.predibase.com/fine-tuning/overview

This snippet shows how to interact with a fine-tuned Predibase adapter to generate text. It obtains a client for the base model deployment, then uses `client.generate` with the adapter ID to get a response.

```python
client = pb.deployments.client("qwen3-8b")
resp = client.generate(
    prompt,
    adapter_id="my-adapter-repo/1",
    max_new_tokens=512,
)
print(resp.generated_text)
```

--------------------------------

### Predibase SDK: Example to Archive an Adapter

Source: https://docs.predibase.com/sdk-reference/adapters

Shows how to archive a specific adapter version using the `pb.adapters.archive` method and its ID, making it hidden from the user interface.

```Python
# Archive an adapter
pb.adapters.archive("news-summarizer-model/1")
```

--------------------------------

### Configure and Start GRPO Training with Predibase SDK

Source: https://docs.predibase.com/fine-tuning/tasks/reinforcement

This Python snippet demonstrates how to initialize and start a GRPO training job using the Predibase SDK. It configures the base model, integrates custom reward functions (`format_reward_func`, `equation_reward_func`), and sets the number of training steps, linking them to a specified dataset and repository.

```Python
from predibase import GRPOConfig, RewardFunctionsConfig

adapter = pb.adapters.create(
    config=GRPOConfig(
        base_model="qwen2-5-7b-instruct",
        reward_fns=RewardFunctionsConfig(
            functions={
                "format": format_reward_func,
                "answer": equation_reward_func,
            },
        ),
        train_steps=200,  # Minimum recommended: 200
    ),
    dataset="training_dataset",
    repo=repo,
)

```

--------------------------------

### Example AI Prompt for Countdown Game

Source: https://docs.predibase.com/fine-tuning/tasks/reinforcement

This prompt demonstrates the structure for interacting with an AI model for a specific task, including system instructions, user query, and expected assistant response format. It's used as an example input for a model trained to solve the Countdown game.

```Prompt
<|im_start|>system
You are a helpful assistant. You first think about the reasoning process step by step
and then provide the user with an answer.<|im_end|>
<|im_start|>user
Using the numbers [17, 64, 63, 26], create an equation that equals 44. You can use
basic arithmetic operations (+, -, *, /) and parentheses, and each number can only
be used once. Show your work in <think> </think> tags. And return the final equation
and answer in <answer> </answer> tags, for example <answer> (1 + 2) / 3 </answer>.<|im_end|>
<|im_start|>assistant
Let me solve this step by step.

```

--------------------------------

### Python Example Usage for Predibase DeploymentConfig

Source: https://docs.predibase.com/sdk-reference/configuration/deployment-config

Illustrates how to instantiate and configure the `DeploymentConfig` class in Python for basic and advanced model deployment settings, including specifying base model, replicas, quantization, and preloaded adapters.

```Python
from predibase import DeploymentConfig

# Basic configuration
config = DeploymentConfig(
    base_model="qwen3-8b",
    min_replicas=0,
    max_replicas=1
)

# Advanced configuration
config = DeploymentConfig(
    base_model="qwen3-8b",
    max_total_tokens=4094,
    quantization="fp8",
    request_logging_enabled=True,
    preloaded_adapters=["my-adapter/1", "my-adapter/2"],
    prefix_caching=True
)
```

--------------------------------

### Download Predibase Dataset

Source: https://docs.predibase.com/fine-tuning/synthetic

This code downloads a specified dataset from Predibase to a local file path. The 'dataset_ref' parameter can be either a string identifier or a Dataset object.

```python
pb.datasets.download(
    dataset_ref=dataset, # can be a string or a Dataset object
    dest="augmented_seed_dataset_20_rows.jsonl"
)
```

--------------------------------

### Predibase SDK: Example to Unarchive an Adapter

Source: https://docs.predibase.com/sdk-reference/adapters

Demonstrates how to unarchive a specific adapter version using the `pb.adapters.unarchive` method and its ID, restoring its visibility in the user interface.

```Python
# Unarchive an adapter
pb.adapters.unarchive("news-summarizer-model/1")
```

--------------------------------

### Predibase SDK: Example to Cancel Adapter Finetune Job

Source: https://docs.predibase.com/sdk-reference/adapters

Demonstrates how to cancel a specific adapter's fine-tuning job using the `pb.adapters.cancel` method and its unique identifier.

```Python
pb.adapters.cancel("news-summarizer-model/1")
```

--------------------------------

### Example Prompt-Completion Dataset for Supervised Fine-Tuning (JSONL)

Source: https://docs.predibase.com/fine-tuning/datasets

This JSONL snippet illustrates the required 'prompt' and 'completion' columns for supervised fine-tuning datasets. Each line represents a single prompt-completion pair, crucial for training language models.

```jsonl
{"prompt": "What is your name?", "completion": "Hi, I'm Pred!"}
{"prompt": "How are you today?", "completion": "I'm doing well how are you?"}
```

--------------------------------

### cURL Example for Predibase Inference API Metrics Retrieval

Source: https://docs.predibase.com/api-reference/inference-api/metrics

Demonstrates how to make a GET request to the Predibase Inference API's /metrics endpoint using cURL to retrieve Prometheus metrics. Users should replace `tenant_id` and `deployment_name` with their specific values.

```curl
curl --request GET \
  --url https://serving.app.predibase.com/tenant_id/deployments/v2/llms/deployment_name/metrics
```

--------------------------------

### Perform Health Check on Predibase LLM Deployment with cURL

Source: https://docs.predibase.com/api-reference/inference-api/health-check

A cURL command example demonstrating how to send a GET request to the Predibase Inference API's health check endpoint to verify the status of a specific LLM deployment.

```cURL
curl --request GET \\
  --url https://serving.app.predibase.com/tenant_id/deployments/v2/llms/deployment_name/health
```

--------------------------------

### Generate Text with Predibase Client and Custom Adapter

Source: https://docs.predibase.com/inference/deployments/shared

Demonstrates how to use the Predibase client to generate text using a custom adapter by specifying its ID and limiting the output length. This example shows a basic text generation call.

```python
response = client.generate(
    "Summarize this article.",
    adapter_id="my-summarizer/1",  # Your adapter ID
    max_new_tokens=100
)
```

--------------------------------

### Create Adapter Asynchronously with Predibase Python SDK

Source: https://docs.predibase.com/fine-tuning/adapters

This example shows how to start an asynchronous fine-tuning job using Predibase's Jobs API. By setting watch=False, the call becomes non-blocking, allowing the application to continue execution while the fine-tuning job runs in the background.

```python
ft_job = pb.finetuning.jobs.create(
    config=SFTConfig(
        base_model="llama-3-1-8b-instruct",
    ),
    dataset=dataset,
    repo=repo,
    watch=False,
)
```

--------------------------------

### Predibase GRPO Reward Function Signature

Source: https://docs.predibase.com/fine-tuning/tasks/reinforcement

Defines the required Python function signature for reward functions used in Predibase's Group Relative Policy Optimization (GRPO). These functions are crucial for evaluating model output quality and guiding training by assigning numerical scores. The function takes the prompt, model completion, and full example data, returning a numeric score.

```APIDOC
reward_fn:
  signature: def reward_fn(prompt: str, completion: str, example: dict[str, str]) -> float
  parameters:
    prompt: Input prompt from dataset (str)
    completion: Model’s generated output (generated by the model during training) (str)
    example: Dictionary containing all fields from the dataset row, including any additional columns that can be used in reward calculations (dict[str, str])
  return_value:
    Numeric score (can be int or float, 0-1 range recommended)
    Non-numeric return values default to a reward of 0.
```

--------------------------------

### Predibase Adapter Creation API Reference

Source: https://docs.predibase.com/fine-tuning/tasks/continued

Reference for the `pb.adapters.create` method and `SFTConfig` parameters used for fine-tuning Predibase models on new datasets, including options for resuming training.

```APIDOC
pb.adapters.create(
  config: SFTConfig - Configuration for the SFT training run
  continue_from_version: str - The adapter version to resume training from (e.g., "myrepo/3")
  dataset: str - The new dataset to train on (e.g., "mydataset-2")
  repo: str - The repository name where the adapter will be stored
)

SFTConfig:
  epochs: int - The maximum number of epochs to train for
  enable_early_stopping: bool - Whether to enable early stopping during training
```

--------------------------------

### Install Predibase Python SDK

Source: https://docs.predibase.com/inference/deployments/private

This command installs the Predibase Python SDK using pip, allowing you to interact with Predibase services programmatically. The `-U` flag ensures the package is upgraded to the latest version if already installed.

```bash
pip install -U predibase
```

--------------------------------

### Create Predibase Adapter with Default SFT Configuration

Source: https://docs.predibase.com/sdk-reference/adapters

Demonstrates how to create a new adapter repository and then initiate a fine-tuning job using `pb.adapters.create` with default SFT (Supervised Fine-Tuning) configuration. The process blocks until training is complete, creating an initial model with standard settings.

```python
# Create an adapter repository
repo = pb.repos.create(name="news-summarizer-model", description="TLDR News Summarizer Experiments", exists_ok=True)

# Start a fine-tuning job, blocks until training is finished
adapter = pb.adapters.create(
    config=SFTConfig(
        base_model="qwen3-8b"
    ),
    dataset="tldr_news",
    repo=repo,
    description="initial model with defaults"
)
```

--------------------------------

### Install Predibase Python SDK

Source: https://docs.predibase.com/inference/models/embeddings

This command installs or upgrades the Predibase Python SDK using pip, which is the essential first step to interact with Predibase services programmatically and manage your models and deployments.

```python
pip install -U predibase
```

--------------------------------

### Create Predibase Adapter Fine-tuning Job

Source: https://docs.predibase.com/fine-tuning/overview

This snippet demonstrates how to initiate a fine-tuning job for a Predibase adapter using `pb.adapters.create`. It configures the base model, applies a chat template, specifies the dataset, and names the repository for the adapter.

```python
adapter = pb.adapters.create(
    config=SFTConfig(
        base_model="qwen3-8b",
        apply_chat_template=True
    ),
    dataset=dataset,
    repo="my-adapter-repo",
    description="initial model with defaults",
)
```

--------------------------------

### Predibase Adapters API: Create Adapter Method Definition

Source: https://docs.predibase.com/sdk-reference/adapters

Defines the `pb.adapters.create` method, which initiates a new blocking fine-tuning job to create an adapter. It outlines the required configuration types (SFTConfig, ContinuedPretrainingConfig, GRPOConfig), dataset, repository, and optional parameters for continuing training, adding descriptions, and launching TensorBoard.

```APIDOC
pb.adapters.create(
    config: SFTConfig | ContinuedPretrainingConfig | GRPOConfig,       # Configuration for fine-tuning
    dataset: str,                                                      # The dataset to use for fine-tuning
    repo: str,                                                         # Name of the adapter repo
    continue_from_version: str = None,                                 # The adapter version to continue training from
    description: str = None,                                           # Description for the adapter
    show_tensorboard: bool = False                                     # If true, launch tensorboard instance
) -> Adapter

Parameters:
  config: SFTConfig | ContinuedPretrainingConfig | GRPOConfig - Configuration for fine-tuning
  dataset: str - The dataset to use for fine-tuning
  continue_from_version: str, optional - The adapter version to continue training from
  repo: str - Name of the adapter repo to store the newly created adapter
  description: str, optional - Description for the adapter
  show_tensorboard: bool, default False - If true, launch a tensorboard instance to view training logs

Returns:
  Adapter - The created adapter object
```

--------------------------------

### Install Predibase Python SDK

Source: https://docs.predibase.com/inference/models/vision-models

This command installs or upgrades the Predibase Python SDK using pip, which is the primary tool for interacting with Predibase services programmatically. It ensures you have the latest features and bug fixes.

```Shell
pip install -U predibase
```

--------------------------------

### Create Predibase Fine-tuning Job with Tensorboard

Source: https://docs.predibase.com/fine-tuning/models

This example extends the basic fine-tuning job creation by enabling Tensorboard integration. By setting `show_tensorboard=True` in the `pb.adapters.create` call, users can stream additional training metrics to Tensorboard for detailed visualization and analysis.

```python
from predibase import SFTConfig

adapter = pb.adapters.create(
    config=SFTConfig(
        base_model="BioMistral/BioMistral-7B"
    ),
    dataset="bio-dataset",
    repo=repo,
    description="initial model with defaults",
    show_tensorboard=True
)
```

--------------------------------

### Initialize Predibase Client and Create Adapter Repository

Source: https://docs.predibase.com/fine-tuning/models

This Python code snippet demonstrates how to initialize the Predibase client using an API token and create a new adapter repository. The `exists_ok=True` parameter ensures that the repository is created only if it doesn't already exist, preventing errors if it's run multiple times.

```python
from predibase import Predibase, SFTConfig
pb = Predibase(api_token=<API_TOKEN>)

# Create an adapter repository
repo = pb.repos.create(name="bio-summarizer", description="Bio News Summarizer", exists_ok=True)
```

--------------------------------

### Create Basic Predibase Fine-tuning Job

Source: https://docs.predibase.com/fine-tuning/models

This snippet demonstrates how to initiate a fine-tuning job using `pb.adapters.create`. It configures the job with an `SFTConfig` specifying the base model, a dataset, a repository, and a description. This operation blocks until the training process is complete.

```python
adapter = pb.adapters.create(
    config=SFTConfig(
        base_model="BioMistral/BioMistral-7B"
    ),
    dataset="bio-dataset",
    repo=repo,
    description="initial model with defaults"
)
```

--------------------------------

### Predibase SDK: Create Asynchronous Fine-tuning Job (API)

Source: https://docs.predibase.com/sdk-reference/adapters

Defines the `pb.finetuning.jobs.create` method for initiating a non-blocking fine-tuning job to create an adapter. It specifies all parameters required for configuration, dataset, repository, and optional settings like continuation version, description, and job monitoring.

```APIDOC
pb.finetuning.jobs.create(
    config: SFTConfig | ContinuedPretrainingConfig | GRPOConfig,
    dataset: str,
    repo: str,
    continue_from_version: str = None,
    description: str = None,
    watch: bool = False,
    show_tensorboard: bool = False
) -> FinetuningJob

Parameters:
  config: SFTConfig | ContinuedPretrainingConfig | GRPOConfig - Configuration for fine-tuning
  dataset: str - The dataset to use for fine-tuning
  repo: str - Name of the adapter repo to store the newly created adapter
  continue_from_version: str, optional - The adapter version to continue training from
  description: str, optional - Description for the adapter
  watch: bool, default False - Whether to block until the fine-tuning job finishes
  show_tensorboard: bool, default False - Whether to launch a tensorboard instance

Returns:
  FinetuningJob - Object representing the fine-tuning job
```

--------------------------------

### Example Customer Input Message for Cancellation

Source: https://docs.predibase.com/examples/lora-land-customer-support

An example of a customer's chat message to the LLM-powered chatbot, expressing the desire to cancel an order. This input demonstrates the natural language query the LLM is expected to process.

```Text
I can no longer afford order {{Order Number}}, cancel it
```

--------------------------------

### Create Adapter Synchronously with Predibase Python SDK

Source: https://docs.predibase.com/fine-tuning/adapters

This code demonstrates how to initialize the Predibase SDK, connect to a dataset, create an adapter repository, and initiate a synchronous fine-tuning job. The SFTConfig is used to specify the base model for fine-tuning.

```python
from predibase import Predibase, SFTConfig

pb = Predibase(api_token="<API_TOKEN>")

# Connect a dataset
dataset = pb.datasets.from_file("/path/to/dataset.csv", name="my_dataset")

# Create an adapter repository
repo = pb.repos.create(name="my-repo", description="Test experiments", exists_ok=True)

# Start a fine-tuning job with LoRA
adapter = pb.adapters.create(
    config=SFTConfig(
        base_model="llama-3-1-8b-instruct",
    ),
    dataset=dataset,
    repo=repo
)
```

--------------------------------

### Start Continued Training Run for Predibase Adapters

Source: https://docs.predibase.com/fine-tuning/tasks/continued

This Python code demonstrates how to continue training an existing Predibase fine-tuned adapter using the `pb.adapters.create` method. It illustrates two scenarios: continuing from the latest checkpoint of a specified adapter version and resuming from a precise checkpoint number within an adapter version. Users can configure additional `epochs` or `train_steps` and enable/disable `early_stopping` for the continued training run.

```Python
from predibase import SFTConfig

# Continue from the latest checkpoint
adapter = pb.adapters.create(
    config=SFTConfig(
        epochs=3,  # The maximum number of ADDITIONAL epochs to train for
        enable_early_stopping=False,
    ),
    continue_from_version="myrepo/3",  # The adapter version to resume training from
    dataset="mydataset",
    repo="myrepo"
)

# Continue from a specific checkpoint
adapter = pb.adapters.create(
    config=SFTConfig(
        epochs=3,  # The maximum number of ADDITIONAL epochs to train for
        enable_early_stopping=False,
    ),
    continue_from_version="myrepo/3@11",  # Resumes from checkpoint 11 of `myrepo/3`
    dataset="mydataset",
    repo="myrepo"
)
```

--------------------------------

### Predibase SDK: Create Repository API Definition

Source: https://docs.predibase.com/sdk-reference/repos

Defines the `pb.repos.create` method for programmatically creating new adapter repositories. It specifies the required parameters such as `name`, optional `description`, and the `exists_ok` flag to handle pre-existing repositories, returning a `Repository` object.

```APIDOC
pb.repos.create(
    name: str,                    # Name for the repository
    description: str = None,      # Description for the repository
    exists_ok: bool = False       # If True, doesn't raise error if exists
) -> Repository
```

--------------------------------

### Example ShareGPT Data Format

Source: https://docs.predibase.com/fine-tuning/tasks/function-calling

This JSON example showcases the typical structure of conversational data in ShareGPT format. It includes a `conversations` array detailing message exchanges, a `system` message, and a `tools` definition, serving as a source for conversion to Predibase's format.

```json
{
  "conversations": [
    {
      "from": "user",
      "value": "I need help multiplying 2 numbers"
    },
    {
      "from": "assistant",
      "value": "I can help with that. What are the numbers?"
    },
    {
      "from": "user",
      "value": "What's 2 times 5"
    },
    {
      "from": "function_call",
      "value": "{\"arguments\": {\"a\": \"2\", \"b\": \"5\"}, \"name\": \"multiply\"}"
    },
    {
      "from": "observation",
      "name": "{\"result\":\"success\",\"helper\":\"10\"}",
      "value": 10
    },
    {
      "from": "assistant",
      "value": "2 times 5 equals 10!"
    }
  ],
  "system": "Predibot is a cool chatbot",
  "tools": "[{\"name\": \"multiply\", \"description\": \"Use this function to multiply two numbers together\", \"parameters\": {\"type\": \"object\", \"properties\": {\"a\": {\"type\": \"number\", \"description\": \"The first number to multiply\"}, \"b\": {\"type\": \"string\", \"description\": \"The second number to multiply\"}}, \"required\": [\"a\", \"b\"]}}]"
}
```

--------------------------------

### Predibase SDK: Download Adapter API Definition

Source: https://docs.predibase.com/sdk-reference/adapters

API definition for the `pb.adapters.download` method, which allows downloading the weights of an existing adapter. It requires the adapter's ID and optionally accepts a local destination path for the download. The method returns None after the weights are successfully downloaded.

```APIDOC
pb.adapters.download(
    adapter_id: str,             # Name of the adapter in the format "repo/version"
    dest: os.PathLike = None,    # Local destination to download the weights to
) -> None

Parameters:
  adapter_id: str - ID of the adapter in the format "repo/version"
  dest: os.PathLike, optional - Local destination to download the weights
Returns:
  None - The adapter weights are downloaded to the local destination
```

--------------------------------

### Create an Adapter Repository in Predibase using Python SDK

Source: https://docs.predibase.com/fine-tuning/overview

This snippet illustrates how to create a new adapter repository in Predibase using the Python SDK. An adapter repository serves as a container for fine-tuned adapters, allowing for organization and management of different model adaptations. The `pb.repos.create` method is used to define the repository with a specified name and description.

```python
# Create an adapter repository
repo = pb.repos.create(
    name="my-adapter-repo",
    description="My first adapter repo",
    exists_ok=True,
)
```

--------------------------------

### Chat Format Dataset Schema Example

Source: https://docs.predibase.com/fine-tuning/tasks/tasks

Example dataset schema for Supervised Fine-Tuning (SFT) using the chat format. This format utilizes a 'messages' column with JSON-style conversation, requiring at least one 'user' and one 'assistant' role per conversation, suitable for chatbots.

```JSON
{"messages": [{"role": "user", "content": "Hi"}, {"role": "assistant", "content": "Hello!"}]}
```

--------------------------------

### Create Predibase SFT Adapter for New Dataset Training

Source: https://docs.predibase.com/fine-tuning/tasks/continued

This Python code demonstrates how to initialize a Predibase SFT adapter to continue training on a new dataset. It specifies the base adapter version to resume from, the new dataset name, and configuration for epochs and early stopping.

```python
from predibase import SFTConfig

adapter = pb.adapters.create(
    config=SFTConfig(
        epochs=3,  # The maximum number of epochs to train for
        enable_early_stopping=False,
    ),
    continue_from_version="myrepo/3",  # The adapter version to resume training from
    dataset="mydataset-2",  # New dataset
    repo="myrepo"
)
```

--------------------------------

### Predibase Inference API: GET /info Endpoint

Source: https://docs.predibase.com/api-reference/inference-api/info

Documentation for the Predibase Inference API's `/info` endpoint. This GET request provides detailed information about the deployed LLM model and its configuration, including performance parameters and model metadata. It returns an `application/json` object.

```APIDOC
Endpoint: GET /info
Description: Inference endpoint info
Response: 200 - application/json
  Type: object
  Description: Info Response
```

--------------------------------

### Define GRPO Reward Functions in Python

Source: https://docs.predibase.com/guides/fine-tuning/reinforcement

This Python code provides example implementations of reward functions crucial for GRPO. The `format_reward` function validates the output format, returning 1.0 for valid formats and 0.0 otherwise. The `quality_reward` function assesses response quality by checking coherence and relevance, normalizing the score to a 0-1 range. These functions take `prompt`, `completion`, and `example` data as input and return a float score.

```Python
def format_reward(prompt: str, completion: str, example: dict) -> float:
    """Validate output format"""
    try:
        # Check format requirements
        if meets_criteria(completion):
            return 1.0
        return 0.0
    except Exception:
        return 0.0

def quality_reward(prompt: str, completion: str, example: dict) -> float:
    """Assess response quality"""
    score = 0.0
    # Add quality checks
    score += check_coherence(completion)
    score += check_relevance(prompt, completion)
    return score / 2.0 # To normalize the score to 0-1 range
```

--------------------------------

### Generate Text with Predibase SDK (Basic Prompting)

Source: https://docs.predibase.com/sdk-reference/queryingmodels

Demonstrates basic text generation using the Predibase SDK's deployment client to interact with a specified model.

```python
client = pb.deployments.client("qwen3-8b")
print(client.generate("What is your name?").generated_text)
```

--------------------------------

### Configure Supervised Fine-Tuning Adapter with Predibase SDK

Source: https://docs.predibase.com/fine-tuning/tasks/tasks

This Python example demonstrates how to configure and create a Supervised Fine-Tuning (SFT) adapter using the Predibase SDK. It specifies the base model ('llama-3-1-8b-instruct') and includes an option to apply a chat template if the dataset doesn't already have it.

```Python
from predibase import SFTConfig

adapter = pb.adapters.create(
    config=SFTConfig(
        base_model="llama-3-1-8b-instruct",
        apply_chat_template=True  # Set to True if your dataset doesn't already have the chat template applied
    ),
    dataset="training_dataset",
)
```

--------------------------------

### Predibase SDK: Example to Delete an Adapter

Source: https://docs.predibase.com/sdk-reference/adapters

Shows how to delete a specific adapter version using the `pb.adapters.delete` method and its ID.

```Python
# Delete an adapter
pb.adapters.delete("news-summarizer-model/1")
```

--------------------------------

### Continue Training Predibase Adapter from Existing Version

Source: https://docs.predibase.com/sdk-reference/adapters

Shows how to resume training an existing adapter using `pb.adapters.create` by specifying `continue_from_version`. When continuing training, only `epochs` and `enable_early_stopping` are available parameters within the SFTConfig, allowing for incremental training on a pre-existing model.

```python
adapter = pb.adapters.create(
    # Note: only `epochs` and `enable_early_stopping` are available parameters in this case.
    config=SFTConfig(
        epochs=3,  # The maximum number of ADDITIONAL epochs to train for
        enable_early_stopping=False
    ),
    continue_from_version="myrepo/3",  # The adapter version to resume training from
    dataset="mydataset",
    repo="myrepo"
)
```

--------------------------------

### RewardFunctionsRuntimeConfig Class Definition

Source: https://docs.predibase.com/sdk-reference/configuration/grpo-config

Defines the configuration for the runtime environment of reward functions during the GRPO training process, specifying optional packages to install.

```APIDOC
RewardFunctionsRuntimeConfig:
  packages:
    type: array[string]
    required: No
    default: ""
    description: Additional packages to install for reward functions
```
Tokens:
Raw