### Install BespokeLabs Curator Package

Source: https://docs.bespokelabs.ai/bespoke-curator/getting-started/quick-tour

Installs the bespokelabs-curator Python package using pip. Ensure you have pip installed and a compatible Python version.

```bash
pip install bespokelabs-curator
```

--------------------------------

### Start vLLM Server for Online Mode

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-vllm-with-curator

Start the vLLM inference server from the command line. This command specifies the model to serve, host, port, and an API key for authentication.

```bash
vllm serve Qwen/Qwen2.5-3B-Instruct \
    --host localhost \
    --port 8787 \
    --api-key token-abc123
```

--------------------------------

### Install Curator and OpenAI Packages

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-huggingface-inference-providers-with-curator

Installs the necessary Python packages for using Hugging Face Inference Providers with curator and OpenAI.

```bash
pip install bespokelabs-curator
pip install huggingface_hub openai
```

--------------------------------

### Install BespokeLabs Python Package

Source: https://docs.bespokelabs.ai/models/bespoke-minicheck/api

Install the official BespokeLabs Python client library using pip. This package provides convenient methods for interacting with the BespokeLabs API. Ensure you have pip installed.

```bash
pip install bespokelabs
```

--------------------------------

### Python Code Executor Example

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/executing-llm-generated-code

A Python example demonstrating how to create and use a custom CodeExecutor. It defines methods for code generation, input, and output parsing, and executes it against a dataset. This executor prints 'Hello' followed by a location.

```python
from bespokelabs import curator
from datasets import Dataset

class HelloExecutor(curator.CodeExecutor):
	def code(self, row):
		return """location = input()\nprint(f"Hello {location}")"""
	
def code_input(self, row):
		return row['location']
	
def code_output(self, row, execution_output):
		row['output'] = execution_output.stdout
		return row

locations = Dataset.from_list([{'location': 'New York'},{'location': 'Tokyo'}])

hello_executor = HelloExecutor()

print(hello_executor(locations).to_pandas())
```

--------------------------------

### Install Python Packages and Ollama

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

Installs necessary Python libraries for data generation and model training, including bespokelabs-curator, fuzzywuzzy, datasets, pydantic, unsloth, bitsandbytes, and triton. It also installs Ollama and pulls specific Llama models for data generation and fine-tuning.

```bash
# Install Python packages
!pip install bespokelabs-curator==0.1.15.post1
!pip install fuzzywuzzy datasets pydantic
!pip install unsloth
!pip install --force-reinstall --no-cache-dir --no-deps git+https://github.com/unslothai/unsloth.git
!pip install bitsandbytes triton unsloth_zoo

# Install ollama
!curl https://www.ollama.com/install.sh | OLLAMA_VERSION="0.5.4" sh

# We need llama3.1:8b for data generation and llama3.2:1b model for finetuning
!ollama pull llama3.1:8b
!ollama pull llama3.2:1b
```

--------------------------------

### Configure Ray Backend for HelloExecutor

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/executing-llm-generated-code

Integrates the Ray backend for distributed code execution. Requires `pip install ray` and a running Ray cluster or will spin up a local one if no base_url is provided.

```python
hello_executor = HelloExecutor(
    backend = "ray",
    backend_params = {
        "base_url": "<url of ray cluster>"
    }
)
```

--------------------------------

### JSON Feature Extraction Example

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

This snippet demonstrates the expected JSON format for inputting product details and receiving extracted features. It highlights the structure for product name, description, and the resulting feature list.

```json
{
    "name": "Apple AirPods Pro",
    "description": "The Apple AirPods Pro are a pair of wireless earbuds that are designed for comfort and convenience. They are lightweight in-ear earbuds and contoured for a comfortable fit, and they sit at an angle for easy access to the controls. The AirPods Pro also have a stem that is 33% shorter than the second generation AirPods, which makes them more compact and easier to store. The AirPods Pro also have a force sensor to easily control music and calls, and they have Spatial Audio with dynamic head tracking, which provides an immersive, three-dimensional listening experience."
}
```

```json
{
    "features": [
        "lightweight in-ear earbuds",
        "contoured for a comfortable fit",
        "sit at an angle for easy access to the controls",
        "stem is 33% shorter than the second generation AirPods",
        "force sensor to easily control music and calls",
        "Spatial Audio with dynamic head tracking",
        "immersive, three-dimensional listening experience"
    ]

}
```

--------------------------------

### Python Docker Backend Initialization

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/executing-llm-generated-code

Shows how to initialize the CodeExecutor to use the Docker backend. This allows for secure code execution within a containerized environment. Requires Docker to be installed.

```python
hello_executor = HelloExecutor(
    backend = "docker"
)
```

--------------------------------

### Python Multiprocessing Backend Configuration

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/executing-llm-generated-code

Example of configuring the multiprocessing backend for the CodeExecutor with specific parameters like max requests per minute. This is the default backend and requires no additional setup.

```python
hello_executor = HelloExecutor(
    backend_params = {
        "max_requests_per_minute": 1000
    }
)
```

--------------------------------

### Install and Import Libraries (Python)

Source: https://docs.bespokelabs.ai/bespoke-curator/data-curation-recipes/generating-a-diverse-qa-dataset

Installs necessary packages like pydantic and bespokelabs-curator, and imports required modules. It also includes an environment variable setting for Curator Viewer, which can be disabled if not needed.

```python
# Install required packages if not already installed
# !pip install pydantic bespokelabs-curator

# Import necessary libraries
from typing import List
from pydantic import BaseModel, Field
from bespokelabs import curator

import os
# disable this if you don't want to use Curator Viewer
os.environ["CURATOR_VIEWER"] = 1 
```

--------------------------------

### Configure E2B Backend for HelloExecutor

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/executing-llm-generated-code

Integrates E2B's hosted code execution backends. Requires `pip install e2b-code-interpreter` and setting the E2B API key as an environment variable.

```python
hello_executor = HelloExecutor(
    backend = "e2b",
)
```

--------------------------------

### Initialize ProductCurator with Ollama

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

Initializes the ProductCurator with a specified Ollama model and backend parameters. This setup allows for local model inference, with configurable rate limiting for API requests. The model_name specifies the Ollama image, and backend_params define connection details and request limits.

```python
product_curator = ProductCurator(
    model_name="ollama/llama3.1:8b",  # Ollama model identifier
    backend_params={
        "base_url": "http://localhost:11434",
        "max_tokens_per_minute": 3000000,
        "max_requests_per_minute": 10,
      },
)

```

--------------------------------

### Batch Processing Backend Parameters Configuration - Python

Source: https://docs.bespokelabs.ai/bespoke-curator/api-reference/llm-api-documentation

Example configuration for parameters when batch processing is enabled. This includes settings for batch size, check intervals, and file management for processed batches.

```python
# Example: Batch processing configuration
backend_params = {
    "batch_size": 100,
    "batch_check_interval": 1.0,
    "delete_successful_batch_files": True,
    "delete_failed_batch_files": False,
}
```

--------------------------------

### Cuisine Generator LLM Example in Python

Source: https://docs.bespokelabs.ai/bespoke-curator/api-reference/llm-api-documentation

This example shows a complete implementation of a cuisine generator using the `curator.LLM` base class. It defines a Pydantic model `Cuisines` for the expected response format and implements `prompt` and `parse` methods. The `parse` method converts the Pydantic response into a list of dictionaries.

```python
class Cuisines(BaseModel):
    """A list of cuisines."""

    cuisines_list: List[str] = Field(description="A list of cuisines.")


class CuisineGenerator(curator.LLM):
    """A cuisine generator that generates diverse cuisines."""

    response_format = Cuisines

    def prompt(self, input: dict) -> str:
        """Generate a prompt for the cuisine generator."""
        return "Generate 10 diverse cuisines."

    def parse(self, input: dict, response: Cuisines) -> dict:
        """Parse the model response along with the input to the model into the desired output format.."""
        return [{"cuisine": t} for t in response.cuisines_list]
```

--------------------------------

### Quick Start: Generate Stratified QA Pairs with StratifiedGenerator

Source: https://docs.bespokelabs.ai/bespoke-curator/data-curation-recipes/using-simplestrat-block-for-generating-diverse-data

Demonstrates the basic usage of StratifiedGenerator to create question-answer pairs from a dataset of questions. It initializes the generator with a specified model and then generates and prints the first QA pair.

```python
from datasets import Dataset
from bespokelabs.curator.blocks.simplestrat import StratifiedGenerator

# Create a simple dataset of questions
questions = Dataset.from_dict({"question": [f"{i}. Name a periodic element" for i in range(20)]})

# Initialize the generator with your preferred model
generator = StratifiedGenerator(model_name="gpt-4o-mini")

# Generate stratified QA pairs
qa_pairs = generator(questions).dataset

# Examine the results
print(f"Generated {len(qa_pairs)} QA pairs")
print(qa_pairs[0])  # View the first QA pair
```

--------------------------------

### Bash Ollama Server and Model Pull Commands

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-ollama-with-curator

These bash commands are used to prepare the Ollama environment. `ollama pull llama3.1:8b` downloads the specified language model, and `ollama serve` starts the Ollama API server, making the model available for requests.

```bash
ollama pull llama3.1:8b
ollama serve
```

--------------------------------

### Install Dependencies for Sentiment Analysis

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/aspect-based-sentiment-analysis

Installs necessary Python packages including bespokelabs-curator, datasets, and together for LLM distillation and data handling.

```python
print("!pip install bespokelabs-curator datasets together")
```

--------------------------------

### Common Backend Parameters Configuration - Python

Source: https://docs.bespokelabs.ai/bespoke-curator/api-reference/llm-api-documentation

Example configuration for common backend parameters applicable across various LLM providers. This includes settings for request retries, API endpoints, timeouts, and API keys.

```python
# Example: Common parameters configuration
backend_params = {
    "max_retries": 3,
    "require_all_responses": True,
    "base_url": "https://custom-endpoint.com/v1",
    "request_timeout": 300
}
```

--------------------------------

### Setup Environment Variable for OpenAI API Key

Source: https://docs.bespokelabs.ai/bespoke-curator/save-usdusdusd-on-llm-inference/using-openai-for-batch-inference

Sets the OpenAI API key as an environment variable. This is a prerequisite for authenticating with the OpenAI API. Ensure you replace '<your_api_key>' with your actual key.

```shell
export OPENAI_API_KEY=<your_api_key>
```

--------------------------------

### Cuisine Generator Example in Python

Source: https://docs.bespokelabs.ai/bespoke-curator/api-reference

Demonstrates a `CuisineGenerator` class that inherits from `curator.LLM` and uses Pydantic for structured output. It defines a `Cuisines` model to specify the expected response format and includes `prompt` and `parse` methods. The `prompt` method generates the LLM query, and the `parse` method processes the LLM's response (a `Cuisines` object) into a desired output format.

```python
class Cuisines(BaseModel):
    """A list of cuisines."""

    cuisines_list: List[str] = Field(description="A list of cuisines.")


class CuisineGenerator(curator.LLM):
    """A cuisine generator that generates diverse cuisines."""

    response_format = Cuisines

    def prompt(self, input: dict) -> str:
        """Generate a prompt for the cuisine generator."""
        return "Generate 10 diverse cuisines."

    def parse(self, input: dict, response: Cuisines) -> dict:
        """Parse the model response along with the input to the model into the desired output format.."""
        return [{"cuisine": t} for t in response.cuisines_list]

```

--------------------------------

### Bash Example: Inspecting Curator Cache Directory

Source: https://docs.bespokelabs.ai/bespoke-curator/getting-started/automatic-recovery-and-caching

Shows how to list the contents of the default BespokeLabs Curator cache directory (`~/.cache/curator`) using a bash shell command. This helps in understanding the cache structure, which includes metadata and directories for individual data generation runs identified by fingerprints.

```bash
>> ls ~/.cache/curator 
032bc5ead2892f8f        6d1f31229726231d
137851647e75f9a7        91f4ad23d5821c9f
24b1d8917f7ef6f1        a2a3c8e5a58e3fc3
metadata.db
```

--------------------------------

### Online Mode Backend Parameters Configuration - Python

Source: https://docs.bespokelabs.ai/bespoke-curator/api-reference/llm-api-documentation

Example configuration for parameters specific to online LLM processing. This focuses on rate limiting and concurrency controls to manage API usage effectively.

```python
# Example: Online mode configuration
backend_params = {
    "max_requests_per_minute": 2000,
    "max_tokens_per_minute": 4_000_000,
    "seconds_to_pause_on_rate_limit": 15.0
}
```

--------------------------------

### Environment Variable Setup for Gemini Batch Inference

Source: https://docs.bespokelabs.ai/bespoke-curator/save-usdusdusd-on-llm-inference/using-gemini-for-batch-inference

Sets up essential environment variables required for Gemini batch inference with Curator. These variables define the Google Cloud region, project ID, bucket name for storage, and the API key for authentication. Ensure these are correctly configured before proceeding.

```sh
export GOOGLE_CLOUD_REGION=us-central1
export GOOGLE_CLOUD_PROJECT=<projectname>
export GEMINI_BUCKET_NAME=<bucketname>
export GEMINI_API_KEY=<your_api_key>
```

--------------------------------

### Chain LLM Calls for Topic and Poem Generation in Python

Source: https://docs.bespokelabs.ai/bespoke-curator/getting-started/structured-output

This Python code demonstrates chaining LLM calls for data generation. It defines a `Muse` LLM to generate poetry topics and a `Poet` LLM to write poems about those topics, both utilizing structured output via Pydantic models and custom parsing. The example shows generating topics first, then using those topics to generate poems.

```python
from typing import Dict, List

from pydantic import BaseModel, Field

from bespokelabs import curator


class Topic(BaseModel):
    topic: str = Field(description="A topic.")


class Topics(BaseModel):
    topics: List[Topic] = Field(description="A list of topics.")


class Muse(curator.LLM):
    response_format = Topics

    def prompt(self, input: Dict) -> str:
        return "Generate ten evocative poetry topics."

    def parse(self, input: Dict, response: Topics) -> Dict:
        return [{"topic": topic.topic} for topic in response.topics]


class Poem(BaseModel):
    poem: str = Field(description="A poem.")


class Poems(BaseModel):
    poems: List[Poem] = Field(description="A list of poems.")


class Poet(curator.LLM):
    response_format = Poems

    def prompt(self, input: Dict) -> str:
        return f"Write two poems about {input['topic']}."

    def parse(self, input: Dict, response: Poems) -> Dict:
        return [{"topic": input["topic"], "poem": p.poem} for p in response.poems]


muse = Muse(model_name="gpt-4o-mini")
topics = muse()
print(topics.dataset.to_pandas())

poet = Poet(model_name="gpt-4o-mini")
poem = poet(topics)
print(poem.dataset.to_pandas())

```

--------------------------------

### ProductCurator LLM Implementation for Product Generation

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

Implements a ProductCurator class inheriting from curator.LLM to generate products based on personas. It defines a prompt to guide the LLM in creating a product name, detailed features, and a description that incorporates these features. Includes a parse method to extract and structure the LLM's output, with fallback mechanisms for handling incomplete or malformed responses.

```python
import re
import json

class ProductCurator(curator.LLM):

  # input prompt to the curator
  def prompt(self, row):
    return f"""Generate a product for the following persona: {row['persona']}

    The product should be a product that is relevant to the persona. Give a name, description and features for the product.

    Give upto 10 features for the product. Features should be relevant, extremely detailed and useful for the persona.

    Then, generate a description for the product. Note that each feature should exactly be mentioned in the description. Do not add any other features or miss any features.

    An example output is:
    {{
        "name": "Apple AirPods Pro",
        "features": [
            "lightweight in-ear earbuds",
            "contoured for a comfortable fit",
            "sits at an angle for comfort",
            "better direct audio to your ear",
            "stem is 33% shorter than the second generation AirPods",
            "force sensor to easily control music and calls",
            "Spatial Audio with dynamic head tracking",
            "immersive, three-dimensional listening experience"
        ]
        "description": "The Apple AirPods Pro are a pair of wireless earbuds that are designed for comfort and convenience. They are <feature>lightweight in-ear earbuds</feature> and <feature>contoured for a comfortable fit</feature>, and each airpod <feature>sits at an angle for comfort</feature>. The AirPods Pro also have a <feature>stem that is 33% shorter than the second generation AirPods</feature>, which makes them more compact and easier to store. The AirPods Pro also have <feature>a force sensor to easily control music and calls</feature>, and they have <feature>Spatial Audio with dynamic head tracking</feature>, which provides an <feature>immersive, three-dimensional listening experience</feature>."
    }}

    Ensure each feature in the paragraph matches exactly as written in the description, including the <feature> and </feature> tags.

    Make sure your output is a JSON and is in the following format. DO NOT OUTPUT ANYTHING ELSE. INCLUDE THE ```json tag in your response.

    ```json
    {{
        "name": "name of the product",
        "features": [
            "feature 1",
            "feature 2",
            "feature 3",
            ...
        ],
        "description": "description of the product"
    }}```
    """


  def parse(self, row, response):
    """Parse the LLM response to extract the product name, features and description."""
    default_response = { "name": "Apple AirPods Pro",
      "features": [
          "lightweight in-ear earbuds",
          "contoured for a comfortable fit",
          "sits at an angle for comfort",
          "better direct audio to your ear",
          "stem is 33% shorter than the second generation AirPods",
          "force sensor to easily control music and calls",
          "Spatial Audio with dynamic head tracking",
          "immersive, three-dimensional listening experience"
      ],
      "description": "The Apple AirPods Pro are a pair of wireless earbuds that are designed for comfort and convenience. They are <feature>lightweight in-ear earbuds</feature> and <feature>contoured for a comfortable fit</feature>, and each airpod <feature>sits at an angle for comfort</feature>. The AirPods Pro also have a <feature>stem that is 33% shorter than the second generation AirPods</feature>, which makes them more compact and easier to store. The AirPods Pro also have <feature>a force sensor to easily control music and calls</feature>, and they have <feature>Spatial Audio with dynamic head tracking</feature>, which provides an <feature>immersive, three-dimensional listening experience</feature>."
    }

    if type(response) == type(''):
      pattern = r"```json(.*?)```"
      match_found = re.findall(pattern, response, re.DOTALL)
      if match_found:
        json_string = match_found[-1].strip()
        try:
          response = json.loads(json_string)
        except:
          response = default_response
      else:
        response = default_response
    else:
      response = response.dict()

    try:
      row['product'] = response['name']
      # note that because the LLM isn't perfect, the features in the response may not be fully accurate
      # row['original_features'] = response.features
      # that's why, we parse the features from the output
      pattern = r"<feature>(.*?)</feature>"
      matches = re.findall(pattern, response['description'])
      if matches:
        row['features'] = matches
      else:
        # backup
        row['features'] = response['features']

      row['description'] = response['description'].replace('<feature>','').replace('</feature>','')
    except:
      return []

```

--------------------------------

### Offline Mode (VLLM) Backend Parameters Configuration - Python

Source: https://docs.bespokelabs.ai/bespoke-curator/api-reference/llm-api-documentation

Example configuration for offline LLM processing using VLLM. This includes parameters related to GPU utilization, model length, and token generation limits for local deployments.

```python
# Example: VLLM configuration
backend_params = {
    "tensor_parallel_size": 2,
    "max_model_length": 4096,
    "max_tokens": 2048,
    "min_tokens": 1,
    "gpu_memory_utilization": 0.85,
    "batch_size": 32
}
```

--------------------------------

### Generate and Process Dataset with Mistral

Source: https://docs.bespokelabs.ai/bespoke-curator/save-usdusdusd-on-llm-inference/using-mistral-for-batch-inference

This Python script loads the 'allenai/WildChat' dataset, selects the first 100 examples, and then processes them using the configured Mistral LLM (`distiller`) for batch inference. Finally, it prints the resulting structured dataset and the first entry of the processed data.

```python
from datasets import load_dataset
dataset = load_dataset("allenai/WildChat", split="train")
dataset = dataset.select(range(100))

distilled_dataset = distiller(dataset)
print(distilled_dataset.dataset)
print(distilled_dataset.dataset[0])
```

--------------------------------

### Chaining LLM Calls for Synthetic Data Pipelines

Source: https://docs.bespokelabs.ai/bespoke-curator/getting-started/structured-output

Demonstrates how to chain multiple LLM (Large Language Model) calls to construct synthetic data pipelines, enabling the generation of a large number of examples. This approach is useful for various data augmentation and generation tasks.

```python
from your_llm_library import LLM

# Initialize LLM instances
llm1 = LLM()
llm2 = LLM()
llm3 = LLM()

# Define prompts or functions for each LLM call
def prompt1():
    return "Generate a product description based on these features: ..."
def prompt2(description):
    return f"Create a marketing slogan for the product described: {description}"
def prompt3(slogan):
    return f"Write a short review for a product with this slogan: {slogan}"

# Chain the LLM calls
description = llm1.generate(prompt1())
slogan = llm2.generate(prompt2(description))
review = llm3.generate(prompt3(slogan))

print(f"Generated Description: {description}")
print(f"Generated Slogan: {slogan}")
print(f"Generated Review: {review}")
```

--------------------------------

### Python Example: LLM Caching with BespokeLabs Curator

Source: https://docs.bespokelabs.ai/bespoke-curator/getting-started/automatic-recovery-and-caching

Demonstrates how to use the `LLM` class from BespokeLabs Curator to generate text and automatically cache the results. The second execution of this code will utilize cached results if available, saving LLM calls. It relies on the `bespokelabs` library.

```python
from bespokelabs import curatorllm = curator.LLM(model_name="gpt-4o-mini")
poem = llm("Write a poem about the importance of data in AI.")
print(poem.dataset.to_pandas())
```

--------------------------------

### Serve Finetuned Model with Ollama

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

This section provides Bash commands to create and serve a finetuned model using Ollama. It shows how to display the automatically generated Modelfile by Unsloth, create an Ollama model from this Modelfile, and then list all available Ollama models to verify the deployment.

```bash
# Unsloth automatically creates a Modelfile!
cat /content/llama_finetune/Modelfile
# Create an ollama model using the saved Modelfile
ollama create llama_finetune -f ./llama_finetune/Modelfile
# Verify that the finetuned model exists
ollama ls
```

--------------------------------

### Load and Inspect Personas Dataset

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

Loads the 'PersonaHub' dataset from the 'proj-persona' hub and takes the first 100 training samples. It then displays the first entry of the sampled dataset.

```python
from datasets import load_dataset

# load the personas dataset
personas = load_dataset("proj-persona/PersonaHub", 'persona')
personas = personas['train'].take(100)

personas[0]
```

--------------------------------

### Prepare Data for Finetuning with Unsloth

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

Prepares data and loads a model for finetuning using the Unsloth library. It specifies model parameters like `max_seq_length`, `load_in_4bit`, and LoRA configuration. The output includes a PEFT model and tokenizer ready for finetuning.

```python
from unsloth import FastLanguageModel
from unsloth.chat_templates import get_chat_template

max_seq_length = 1024
load_in_4bit = True
dtype = None # for auto

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/Llama-3.2-1B-Instruct-bnb-4bit",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

peft_model = FastLanguageModel.get_peft_model(
    model,
    r = 16, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",],
    lora_alpha = 16,
    lora_dropout = 0, # Supports any, but = 0 is optimized
    bias = "none",    # Supports any, but = "none" is optimized
    # [NEW] "unsloth" uses 30% less VRAM, fits 2x larger batch sizes!
    use_gradient_checkpointing = "unsloth", # True or "unsloth" for very long context
    random_state = 3407,
    use_rslora = False,  # We support rank stabilized LoRA
    loftq_config = None, # And LoftQ
)

# doing this so ollama creates a modelfile
tokenizer = get_chat_template(
    tokenizer,
    chat_template = "llama-3.1",
)
```

--------------------------------

### Set Up Seed Dataset for Cuisines

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-huggingface-inference-providers-with-curator

Creates a Hugging Face Dataset containing a list of cuisines, which will be used as input for the recipe generation.

```python
# List of cuisines to generate recipes for
cuisines = [
    {"cuisine": cuisine}
    for cuisine in [
        "Chinese",
        "Italian",
        "Mexican",
        "French",
        "Japanese",
        "Indian",
        "Thai",
        "Korean",
        "Vietnamese",
        "Brazilian",
    ]
]
cuisines = Dataset.from_list(cuisines)
```

--------------------------------

### Format Dataset Row for Finetuning (Python)

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

This Python function formats a dataset row into a text string suitable for finetuning a chat-based language model. It takes a dictionary with 'product', 'description', and 'features' as input. The output is a dictionary containing the formatted 'text'. Dependencies include a 'tokenizer' object and a 'FEATURE_PROMPT' string, which are assumed to be defined elsewhere.

```python
def formatting_prompts_func(row):
  texts = []
  features = {'features': row['features']}
  messages = [
      {"role": "user", "content": FEATURE_PROMPT.format(product_name=row['product'], product_description=row['description'])},
      {"role": "assistant", "content": f"```json{json.dumps(features)}```"},
  ]
  text = tokenizer.apply_chat_template(messages, tokenize = False, add_generation_prompt = False)
  return {'text': text}
```

--------------------------------

### Python Docker Backend with Custom Image

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/executing-llm-generated-code

Example of configuring the Docker backend to use a custom Docker image for code execution. This is useful when specific libraries or environments are required.

```python
hello_executor = HelloExecutor(
    backend = "docker",
    backend_params = {
        "image": "andgineer/matplotlib"
    }
)
```

--------------------------------

### Visualize Data with Curator Viewer

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-huggingface-inference-providers-with-curator

Enables Curator Viewer and uses the 'push_to_viewer' utility to generate a URL for visualizing the generated recipe dataset.

```python
# We can visualize data using Curator viewer easily.

import os
os.environ['CURATOR_VIEWER']='1'

from bespokelabs.curator.utils import push_to_viewer
url = push_to_viewer(results)
```

--------------------------------

### Import Required Libraries for Fine-tuning

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

Imports essential Python libraries for fine-tuning language models, including FastLanguageModel from unsloth, curator from bespokelabs, and various modules for file operations, regular expressions, JSON handling, tensor computations, random number generation, typing, string matching, data structures, and dataset loading.

```python
# Make sure the following imports fine before proceeding, since this is needed for finetuning.
from unsloth import FastLanguageModel
from bespokelabs import curator

import os
import re
import json
import torch
import random
import numpy as np
from typing import List
from fuzzywuzzy import fuzz
from pydantic import BaseModel, Field
from datasets import Dataset, load_dataset
```

--------------------------------

### Import Libraries for BespokeLabs Curator

Source: https://docs.bespokelabs.ai/bespoke-curator/data-curation-recipes/synthetic-data-for-function-calling

Imports necessary Python libraries including `json`, `typing`, `datasets`, and `bespokelabs.curator`. This setup is required before defining custom LLM classes or datasets.

```python
# pip install bespokelabs-curator 

import json
from typing import Dict

from datasets import Dataset
from bespokelabs import curator
```

--------------------------------

### Display Product Information with Highlighted Features (Python)

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

This Python function generates HTML to display product information, including name, description, features, and image. It uses regular expressions to highlight specified features within the product description, applying custom CSS for styling. The function requires the IPython.display module for rendering HTML.

```python
from IPython.display import HTML, display
import re

def display_product(
    product_name,
    description,
    features,
    image_url,
):
"""Displays a product give its product_name, features, and description"""
  # Product description and features
  def highlight_features(text, features):
      # Sort features by length in descending order to handle overlapping matches
      sorted_features = sorted(features, key=len, reverse=True)

      # Create a copy of the text for highlighting
      highlighted_text = text

      # Replace each feature with its highlighted version
      for feature in sorted_features:
          pattern = re.compile(re.escape(feature), re.IGNORECASE)
          highlighted_text = pattern.sub(
              f'<span class="highlight">{feature}</span>',
              highlighted_text
          )

      return highlighted_text

  # Create HTML content with CSS styling
  html_content = f"""
  <style>
      .product-container {
          max-width: 800px;
          margin: 20px auto;
          padding: 30px;
          font-family: 'Segoe UI', Arial, sans-serif;
          line-height: 1.6;
          background: white;
          border-radius: 12px;
          box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
      }

      .product-title {
          color: #1d1d1f;
          font-size: 28px;
          margin-bottom: 20px;
          text-align: center;
      }

      .product-image {
          width: 100%;
          max-width: 600px;
          height: auto;
          margin: 0 auto 30px;
          display: block;
          border-radius: 8px;
      }

      .product-description {
          color: #333;
          font-size: 16px;
          margin-bottom: 20px;
      }

      .highlight {
          background: linear-gradient(120deg, rgba(37, 99, 235, 0.1) 0%, rgba(37, 99, 235, 0.2) 100%);
          border-radius: 4px;
          padding: 2px 4px;
          transition: background 0.3s ease;
      }

      .highlight:hover {
          background: linear-gradient(120deg, rgba(37, 99, 235, 0.2) 0%, rgba(37, 99, 235, 0.3) 100%);
          cursor: pointer;
      }
  </style>

  <div class="product-container">
      <h1 class="product-title">{product_name}</h1>
  """
  if image_url:
    html_content += f'<img class="product-image" src="{image_url}" width="300px" alt="{product_name}">'  
  html_content += f"""<p class="product-description">
          {highlight_features(description, features)}
      </p>
  </div>"""

  display(HTML(html_content))


display_product(
    product_name="Apple Airpods Pro",
    description="The Apple AirPods Pro are a pair of wireless earbuds that are designed for comfort and convenience. They are lightweight in-ear earbuds and contoured for a comfortable fit, and they sit at an angle for easy access to the controls. The AirPods Pro also have a stem that is 33% shorter than the second generation AirPods, which makes them more compact and easier to store. The AirPods Pro also have a force sensor to easily control music and calls, and they have Spatial Audio with dynamic head tracking, which provides an immersive, three-dimensional listening experience.",
    features=[
    "lightweight in-ear earbuds",
    "contoured design",
    "sits at an angle for comfort",
    "better direct audio to your ear",
    "stem is 33% shorter than the second generation AirPods",
    "force sensor to easily control music and calls",
    "Spatial Audio with dynamic head tracking",
    "immersive, three-dimensional listening experience"],
    image_url="https://store.storeimages.cdn-apple.com/4982/as-images.apple.com/is/airpods-pro-2-hero-select-202409_FMT_WHH?wid=750&hei=556&fmt=jpeg&qlt=90&.v=1724041668836")
```

--------------------------------

### Configure LiteLLM Backend for RecipeGenerator in Python

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-litellm-with-curator

Initializes the `RecipeGenerator` with LiteLLM as the backend. Specifies the `model_name` (e.g., 'gemini/gemini-1.5-flash') and configures `backend_params` for rate limiting, such as `max_requests_per_minute` and `max_tokens_per_minute`.

```python
recipe_generator = RecipeGenerator(
    model_name="gemini/gemini-1.5-flash",  # LiteLLM model identifier
    backend="litellm",                      # Specify LiteLLM backend
    backend_params={
        "max_requests_per_minute": 2_000,   # Rate limit for requests
        "max_tokens_per_minute": 4_000_000  # Token usage limit
    },
)
```

--------------------------------

### Self-Host Bespoke-MiniCheck-7B with vLLM and Docker

Source: https://docs.bespokelabs.ai/models/bespoke-minicheck/hosting

This code snippet demonstrates how to self-host the Bespoke-MiniCheck-7B model using vLLM and Docker. It configures GPU usage, mounts Hugging Face cache, sets environment variables, maps ports, and specifies model parameters like model name, data type, and maximum sequence length. Ensure you replace 'hf_xyz' with your actual Hugging Face token and 'your_api_key' with your desired API key.

```shellscript
sudo docker run \
  --runtime=nvidia \
  --gpus=all \
  -v ~/.cache/huggingface:/root/.cache/huggingface     \
  --env "HUGGING_FACE_HUB_TOKEN=hf_xyz" \
  --ipc=host \
  -p 8000:8000 \
  vllm/vllm-openai:latest \
  --model bespokelabs/Bespoke-MiniCheck-7B --trust_remote_code --api-key your_api_key --disable-log-requests \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --tensor-parallel-size 1 &
```

--------------------------------

### Configure OpenAI Backend with Hugging Face Inference Provider

Source: https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-huggingface-inference-providers-with-curator

Initializes the 'RecipeGenerator' using the 'openai' backend, configuring it to use a Hugging Face Inference Provider by setting the 'base_url' and 'api_key'. It then generates recipes and prints the results.

```python
recipe_generator = RecipeGenerator(
    model_name="meta-llama/Llama-4-Scout-17B-16E-Instruct",
    backend="openai",
    backend_params={
        "base_url": "https://router.huggingface.co/together/v1",
        "api_key": HF_TOKEN, # your Hugging Face API key
    },
)

results = recipe_generator(cuisines).dataset
print(results.to_pandas())
```

--------------------------------

### Enable Curator Viewer Locally

Source: https://docs.bespokelabs.ai/bespoke-curator/getting-started/visualize-your-dataset-with-the-bespoke-curator-viewer

This snippet shows how to enable the local Curator Viewer by setting the CURATOR_VIEWER environment variable. This is a prerequisite for visualizing data during Curator runs.

```bash
export CURATOR_VIEWER=1
```

```python
import os
os.environ["CURATOR_VIEWER"]="1"
```

--------------------------------

### Bash Example: Clearing Curator Cache Directory

Source: https://docs.bespokelabs.ai/bespoke-curator/getting-started/automatic-recovery-and-caching

Provides a bash command to forcefully remove the BespokeLabs Curator cache directory (`~/.cache/curator`). This is a troubleshooting step to resolve issues with corrupt or excessively large cache files, but it will delete all cached responses.

```bash
rm -rf ~/.cache/curator
```

--------------------------------

### Generate Text with BespokeLabs LLM Class (Python)

Source: https://docs.bespokelabs.ai/bespoke-curator/getting-started/quick-tour

Demonstrates generating text using the LLM class from the bespokelabs.curator module. It shows how to initialize the LLM with a model name and generate single or multiple responses based on provided prompts. The output can be converted to a pandas DataFrame.

```python
from bespokelabs import curator
llm = curator.LLM(model_name="gpt-4o-mini")
poem = llm("Write a poem about the importance of data in AI.")
print(poem.to_pandas())

# Or you can pass a list of prompts to generate multiple responses.
poems = llm(["Write a poem about the importance of data in AI.",
            "Write a haiku about the importance of data in AI."])
print(poems.dataset.to_pandas())
```

--------------------------------

### Customizing Subject Generator for Diverse Subjects

Source: https://docs.bespokelabs.ai/bespoke-curator/data-curation-recipes/generating-a-diverse-qa-dataset

This Python snippet shows how to create a custom `SubjectGenerator` by inheriting from the base class and overriding the `prompt` method. The example customizes the generator to specifically produce a diverse list of 5 subjects across sciences, arts, and humanities.

```python
from bespokelabs_ai.datasets import SubjectGenerator

# Modify the subject generator to produce more subjects
class CustomSubjectGenerator(SubjectGenerator):
    def prompt(self, input: dict) -> str:
        return "Generate a diverse list of 5 subjects spanning sciences, arts, and humanities."
```

--------------------------------

### Run SFT Finetuning with Unsloth

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

This snippet uses the `trl` and `unsloth` libraries to perform Supervised Fine-Tuning (SFT) on a language model. It configures the `SFTTrainer` with model, tokenizer, dataset, and training arguments, including optimizer and learning rate settings. The `train_on_responses_only` function is used to format the training data.

```python
from trl import SFTTrainer
from transformers import TrainingArguments, DataCollatorForSeq2Seq
from unsloth import is_bfloat16_supported
from unsloth.chat_templates import train_on_responses_only

trainer = SFTTrainer(
    model = peft_model,
    tokenizer = tokenizer,
    train_dataset = ft_dataset,
    dataset_text_field = "text",
    max_seq_length = max_seq_length,
    data_collator = DataCollatorForSeq2Seq(tokenizer = tokenizer),
    dataset_num_proc = 2,
    packing = False, # Can make training 5x faster for short sequences.
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        warmup_steps = 5,
        num_train_epochs = 1, # Set this for 1 full training run.
        learning_rate = 2e-4,
        fp16 = not is_bfloat16_supported(),
        bf16 = is_bfloat16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.01,
        lr_scheduler_type = "linear",
        seed = 3407,
        output_dir = "outputs",
        report_to = "none", # Use this for WandB etc
    ),
)

trainer = train_on_responses_only(
    trainer,
    instruction_part = "<|start_header_id|>user<|end_header_id|>\n\n",
    response_part = "<|start_header_id|>assistant<|end_header_id|>\n\n",
)
trainer_stats = trainer.train()
```

--------------------------------

### Create Eval Set with GPT-4o-mini

Source: https://docs.bespokelabs.ai/bespoke-curator/finetuning-examples/finetuning-a-model-to-identify-features-of-a-product

Generates an evaluation dataset using the gpt-4o-mini model to avoid bias. It requires an OpenAI API key to be set as an environment variable. The function returns a dataset for testing.

```python
import getpass
os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
eval_product_curator = ProductCurator(
    model_name="gpt-4o-mini"
)

train_dataset = products
test_dataset = eval_product_curator(personas.take(40)).dataset
```

--------------------------------

### Configure LLM Backend for kluster.ai

Source: https://docs.bespokelabs.ai/bespoke-curator/save-usdusdusd-on-llm-inference/using-kluster

This Python code initializes an instance of the `Reasoner` class, specifying 'deepseek-ai/DeepSeek-R1' as the model and 'klusterai' as the backend. It configures batch processing and sets backend-specific parameters like `max_retries` and `completion_window` for optimal performance.

```python
reasoner = Reasoner(model_name="deepseek-ai/DeepSeek-R1", 
                    backend="klusterai", 
                    batch=True, 
                    backend_params={"max_retries": 1, "completion_window": "1h"})

```

--------------------------------

### Display AI Model and Gold Answers in Python

Source: https://docs.bespokelabs.ai/bespoke-curator/save-usdusdusd-on-llm-inference/using-kluster

This Python script demonstrates how to display questions, model answers, gold answers, and model reasoning using IPython's display functions. It's useful for visualizing AI-generated content and comparing it with ground truth. It requires the `IPython.display` module.

```python
from IPython.display import HTML, display, Markdown
which = 0
question = output[which]['question']
gold_answer = output[which]['gold_answer']
model_answer = output[which]['deepseek_solution']
thought = output[which]['reasoning']

to_display_input = question.replace("\n", "<br>")
to_display_output = model_answer.replace("\n", "<br>")

display(Markdown(
    "<h1>Question</h1>"
    f"<h3>{question}</h3>"
))
display(Markdown(
    "<h1>Model answer</h1>"
    f"<p>{model_answer}</p>"
))
display(Markdown(
    "<h1>Gold answer</h1>"
    f"<p>{gold_answer}</p>"
))
display(Markdown(
    "<h1>Model Thought</h1>"
    f"<p>{thought}</p>"
))
```