### Install sieves for Development Setup

Source: https://sieves.ai/index

Installs all dependencies required for development, including testing and documentation generation, using uv. This is intended for contributors and advanced users.

```bash
uv pip install --system .[distill,ingestion,test]
```

--------------------------------

### Install Core sieves Package

Source: https://sieves.ai/index

Installs the core sieves Python package with minimal dependencies. This is the basic setup for using the library's primary features.

```bash
pip install sieves
```

--------------------------------

### Creating Documents with Sieves

Source: https://sieves.ai/guides/getting_started

Demonstrates various ways to create Document objects in sieves. Documents can be initialized directly from text, from a URI (requires 'docling'), or with associated metadata. Dependencies include 'sieves'.

```python
from sieves import Docs, Doc

# From text
doc = Doc(text="Your text here")

# From a file (requires docling)
doc = Doc(uri="path/to/your/file.pdf")

# With metadata
doc = Doc(
    text="Your text here",
    meta={"source": "example", "date": "2025-01-31"}
)
```

--------------------------------

### Specify Inference Mode with GenerationSettings and Outlines Engine

Source: https://sieves.ai/guides/getting_started

This example shows how to configure GenerationSettings with a specific inference mode using the Outlines engine. It initializes a Classification task with strict mode enabled and instructs the engine to parse results in JSON format.

```python
import outlines
from sieves.engines import outlines_
from sieves.engines.utils import GenerationSettings

model = outlines.models.transformers("HuggingFaceTB/SmolLM-135M-Instruct")
classifier = tasks.predictive.Classification(
    labels=["science", "politics"],
    model=model,
    generation_settings=GenerationSettings(
        strict_mode=True,
        inference_mode=outlines_.InferenceMode.json  # Specifies how to parse results
    ),
    batch_size=8,
)

```

--------------------------------

### GlinerNER Default Prompt Instructions and Examples

Source: https://sieves.ai/tasks/predictive/ner

Configures the default prompt instructions and example templates for the GLiNER2 NER bridge. Currently, it returns empty strings for default instructions and no prompt examples or conclusions, indicating that custom prompts or examples are expected to be provided externally if needed.

```python
    @override
    @property
    def _default_prompt_instructions(self) -> str:
        return ""

    @override
    @property
    def _prompt_example_template(self) -> str | None:
        return None

    @override
    @property
    def _prompt_conclusion(self) -> str | None:
        return None
```

--------------------------------

### Text Classification with Sieves Pipeline

Source: https://sieves.ai/guides/getting_started

Performs text classification using the sieves Pipeline. It takes a Doc object, a chosen model, and a Classification task with specified labels. It outputs the classification results for the document. Dependencies include 'outlines' and 'sieves'.

```python
import outlines
from sieves import Pipeline, tasks, Doc

# Create a document
doc = Doc(text="Special relativity applies to all physical phenomena in the absence of gravity.")

# Choose a model (using a small but capable model)
model = outlines.models.transformers("HuggingFaceTB/SmolLM-135M-Instruct")

# Create and run the pipeline (verbose init)
pipeline = Pipeline([
    tasks.predictive.Classification(
        labels=["science", "politics"],
        model=model,
    )
])

# Print the classification result
for doc in pipeline([doc]):
    print(doc.results)

# Alternatively: succinct chaining with +
# (useful when you have multiple tasks)
# classifier = tasks.predictive.Classification(labels=["science", "politics"], model=model)
# pipeline = classifier  # single-task pipeline
# Note: set additional Pipeline params (e.g., use_cache=False) only via verbose init.
```

--------------------------------

### Shell: Install Sieve AI with Ingestion Extras

Source: https://sieves.ai/pipeline

Shows how to install the Sieve AI library with optional ingestion libraries like `docling` using pip. This command installs the base library along with extras for enhanced ingestion capabilities.

```shell
pip install "sieves[ingestion]"

```

--------------------------------

### Install Sieves Ingestion Libraries

Source: https://sieves.ai/tasks/preprocessing/ingestion/marker

This snippet shows how to install the necessary libraries for PDF ingestion with Sieves AI. It highlights two methods: installing via the 'ingestion' extra or installing a specific Marker package directly. Ensure these optional dependencies are installed before using PDF conversion features.

```bash
pip install "sieves[ingestion]"
```

```bash
pip install marker
```

--------------------------------

### Configure GenerationSettings for Classification Task

Source: https://sieves.ai/guides/getting_started

This snippet demonstrates how to initialize a Classification task with custom GenerationSettings, including enabling strict mode and setting a batch size. It uses the `sieves.engines.utils.GenerationSettings` class and assumes a pre-defined `tasks` object and `model`.

```python
from sieves.engines.utils import GenerationSettings

classifier = tasks.predictive.Classification(
    labels={
        "science": "Scientific topics and research",
        "politics": "Political news and government"
    },
    model=model,
    generation_settings=GenerationSettings(strict_mode=True),
    batch_size=8,
)

```

--------------------------------

### Text Classification with Label Descriptions in Sieves

Source: https://sieves.ai/guides/getting_started

Enhances text classification accuracy by providing descriptions for each label. This is useful when label names are ambiguous. Dependencies include 'outlines' and 'sieves'. The input is a Doc object and a Classification task configured with a dictionary of labels and their descriptions.

```python
import outlines
from sieves import Pipeline, tasks, Doc

doc = Doc(text="Special relativity applies to all physical phenomena in the absence of gravity.")
model = outlines.models.transformers("HuggingFaceTB/SmolLM-135M-Instruct")

# Use dict format to provide descriptions
pipeline = Pipeline([
    tasks.predictive.Classification(
        labels={
            "science": "Scientific topics including physics, biology, chemistry, and natural sciences",
            "politics": "Political news, government affairs, elections, and policy discussions"
        },
        model=model,
    )
])

for doc in pipeline([doc]):
    print(doc.results)
```

--------------------------------

### Install sieves with Distillation Utilities

Source: https://sieves.ai/index

Installs the sieves Python package with utilities specifically for model distillation and fine-tuning. Use this when you plan to optimize models.

```bash
pip install "sieves[distill]"
```

--------------------------------

### Install sieves with All Optional Dependencies

Source: https://sieves.ai/index

Installs the sieves Python package with all optional dependencies, including ingestion and distillation utilities. This provides the most comprehensive feature set.

```bash
pip install "sieves[distill,ingestion]"
```

--------------------------------

### Optimize Classification Task Prompt and Examples in Python

Source: https://sieves.ai/guides/optimization

Demonstrates optimizing a DSPy classification task using the sieves AI Optimizer. It initializes a DSPy model, defines a task with few-shot examples, configures the optimizer, and runs the optimization process. The output includes the best prompt and the count of selected examples.

```python
import os
import dspy
from sieves.tasks import Classification
from sieves.tasks.optimization import Optimizer
from sieves import GenerationSettings

# 1. Create model for optimization
model = dspy.LM("claude-3-haiku-20240307", api_key=os.environ["ANTHROPIC_API_KEY"])

# 2. Create task with few-shot examples (at least 2 required)
examples = [
    Classification.FewshotExampleSingleLabel(
        text="The new AI model achieves state-of-the-art results",
        label="technology",
        confidence=1.0
    ),
    Classification.FewshotExampleSingleLabel(
        text="Election results show significant voter turnout",
        label="politics",
        confidence=1.0
    ),
    # ... add more examples (recommended: 6-20 examples)
]

task = Classification(
    labels={
        "technology": "Technology news, AI, software, and digital innovations",
        "politics": "Political events, elections, and government affairs",
        "sports": "Sports news, games, athletes, and competitions"
    },
    model=model,
    fewshot_examples=examples,
    generation_settings=GenerationSettings(),
)

# 3. Create optimizer
optimizer = Optimizer(
    model=model,                    # Model for optimization (can differ from task model)
    val_frac=0.25,                  # Use 25% of examples for validation
    seed=42,                        # For reproducibility
    shuffle=True,                   # Shuffle data before splitting
    dspy_init_kwargs=dict(
        num_candidates=2,           # Number of prompt candidates to try
        max_errors=3                # Max errors before stopping
    ),
    dspy_compile_kwargs=dict(
        num_trials=1,               # Number of optimization trials
        minibatch=False             # Use full batch (True for large datasets)
    )
)

# 4. Run optimization
best_prompt, best_examples = task.optimize(optimizer, verbose=True)

# The task now uses the optimized prompt and examples automatically
print(f"Optimized prompt: {best_prompt}")
print(f"Number of selected examples: {len(best_examples)}")

```

--------------------------------

### PDF Processing Pipeline with Sieves, Chonkie, and Outlines

Source: https://sieves.ai/guides/getting_started

An advanced pipeline that parses a PDF, chunks it, and extracts information. It utilizes 'sieves' for the pipeline, 'chonkie' for chunking, 'tokenizers' for tokenization, 'pydantic' for data structuring, and 'outlines' for the model. Dependencies include 'outlines', 'chonkie', 'tokenizers', and 'pydantic'. The pipeline processes a Doc object and outputs extracted information structured according to a Pydantic model.

```python
import outlines
import chonkie
import tokenizers
import pydantic
from sieves import Pipeline, tasks, Doc

# Create a tokenizer for chunking
tokenizer = tokenizers.Tokenizer.from_pretrained("bert-base-uncased")

# Initialize components
chunker = tasks.preprocessing.Chonkie(
    chunker=chonkie.TokenChunker(tokenizer, chunk_size=512, chunk_overlap=50)
)

# Choose a model for information extraction
model = outlines.models.transformers("HuggingFaceTB/SmolLM-135M-Instruct")


# Define the structure of information you want to extract
class PersonInfo(pydantic.BaseModel):
    name: str
    age: int | None = None
    occupation: str | None = None


# Create an information extraction task
extractor = tasks.predictive.InformationExtraction(
    entity_type=PersonInfo,
    model=model,
)

# Create the pipeline (verbose init)
pipeline = Pipeline([chunker, extractor])

# Alternatively: succinct chaining (+)
# pipeline = chunker + extractor
# Note: to change Pipeline parameters (e.g., use_cache), use the verbose form
#   Pipeline([chunker, extractor], use_cache=False)

# Process a PDF document
doc = Doc(text="Marie Curie died at the age of 66 years.")
results = list(pipeline([doc]))

# Access the extracted information
for result in results:
    print(result.results["InformationExtraction"])
```

--------------------------------

### Convert Few-shot Examples

Source: https://sieves.ai/engines/base_engine

A static utility method to convert a sequence of Pydantic BaseModel objects representing few-shot examples into a list of dictionaries.

```APIDOC
## `convert_fewshot_examples(fewshot_examples)`

### Description
Convert few‑shot examples to dicts.

### Method
`convert_fewshot_examples` (static method)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **fewshot_examples** (`Sequence[BaseModel]`) - Required - Fewshot examples to convert.

### Request Example
```json
[
  {
    "input_field": "example input 1",
    "output_field": "example output 1"
  },
  {
    "input_field": "example input 2",
    "output_field": "example output 2"
  }
]
```

### Response
#### Success Response (200)
- **list[dict[str, Any]]** - Fewshot examples as dicts.

#### Response Example
```json
[
  {
    "input_field": "example input 1",
    "output_field": "example output 1"
  },
  {
    "input_field": "example input 2",
    "output_field": "example output 2"
  }
]
```
```

--------------------------------

### DSPy Multi-Label Classification Example

Source: https://sieves.ai/guides/optimization

Example of setting up a multi-label classification task in DSPy. It defines a model, provides few-shot examples with per-label confidence, configures the Classification task, and then optimizes it using a specified optimizer.

```python
import dspy
from sieves.tasks import Classification
from sieves.tasks.optimization import Optimizer

model = dspy.LM("claude-3-haiku-20240307", api_key="...")

# Multi-label examples with per-label confidence
examples = [
    Classification.FewshotExampleMultiLabel(
        text="Quantum computing advances promise faster drug discovery",
        confidence_per_label={"technology": 0.9, "healthcare": 0.7, "finance": 0.1}
    ),
    # ... more examples
]

task = Classification(
    labels={
        "technology": "Technology and software topics",
        "healthcare": "Healthcare, medicine, and medical research",
        "finance": "Financial markets, banking, and economics"
    },
    model=model,
    multi_label=True,
    fewshot_examples=examples
)

optimizer = Optimizer(model=model, val_frac=0.25)
best_prompt, best_examples = task.optimize(optimizer)

```

--------------------------------

### Convert Few-shot Examples

Source: https://sieves.ai/engines/base_engine

Converts a sequence of few-shot examples (Pydantic models) into a list of dictionaries.

```APIDOC
## POST /websites/sieves_ai/convert_fewshot_examples

### Description
Convert few‑shot examples to dicts. This method takes a sequence of Pydantic BaseModel objects and converts them into a list of dictionaries.

### Method
POST

### Endpoint
/websites/sieves_ai/convert_fewshot_examples

### Parameters
#### Query Parameters
- **fewshot_examples** (Sequence[BaseModel]) - Required - Fewshot examples to convert.

### Request Body
(No request body defined for this endpoint. Parameters are passed as query parameters)

### Response
#### Success Response (200)
- **list[dict[str, Any]]** - Fewshot examples as dicts.

#### Response Example
{
  "converted_examples": [
    { "field1": "value1", "field2": 123 },
    { "field1": "value2", "field2": 456 }
  ]
}
```

--------------------------------

### Install sieves with Ingestion Libraries

Source: https://sieves.ai/index

Installs the sieves Python package including optional ingestion libraries for document parsing (e.g., docling). Use this when you need to process various document formats.

```bash
pip install "sieves[ingestion]"
```

--------------------------------

### Convert Few-Shot Examples to Dictionaries in Python

Source: https://sieves.ai/engines/base_engine

Static method to convert a sequence of Pydantic BaseModel objects into a list of dictionaries. This is useful for preparing few-shot learning examples for machine learning models. Each dictionary represents an example, serialized with all fields included.

```python
import pydantic
from typing import Sequence, Any, list, dict

class ExampleConverter:
    @staticmethod
    def convert_fewshot_examples(fewshot_examples: Sequence[pydantic.BaseModel]) -> list[dict[str, Any]]:
        """Convert few‑shot examples to dicts.

        :param fewshot_examples: Fewshot examples to convert.
        :return: Fewshot examples as dicts.
        """
        return [fs_example.model_dump(serialize_as_any=True) for fs_example in fewshot_examples]
```

--------------------------------

### Optimize Task Prompt and Few-Shot Examples (Python)

Source: https://sieves.ai/tasks/predictive/ner

Optimizes the prompt and few-shot examples for a task using a specified optimizer. This method updates the task's internal prompt and examples to the best ones found during the optimization process. It requires an Optimizer object and an optional verbose flag to control output. The function returns the best prompt string and a sequence of FewshotExample objects. It has a prerequisite that at least two few-shot examples must be provided.

```python
import logging
import warnings
from typing import Any, Sequence

import dspy

# Assuming Optimizer, FewshotExample, GenerationSettings, TaskPromptSignature, TaskResult, TaskBridge, EngineType, and Config are defined elsewhere

class PredictiveTask:
    def optimize(
        self,
        optimizer: optimization.Optimizer,  # Assuming optimization.Optimizer is a valid type
        verbose: bool = True
    ) -> tuple[str, Sequence[FewshotExample]]:
        """Optimize task prompt and few-shot examples with the available optimization config.

        Updates task to use best prompt and few-shot examples found by the optimizer.

        :param optimizer: Optimizer to run.
        :param verbose: Whether to suppress output. DSPy produces a good amount of logs, so this can be useful to
            not pollute your terminal. Only warnings and errors will be printed.

        :return tuple[str, Sequence[FewshotExample]]: Best found prompt and few-shot examples.
        """
        assert len(self._fewshot_examples) > 1, "At least two few-shot examples need to be provided to optimize."

        # Run optimizer to get best prompt and few-shot examples.
        signature = self._get_task_signature()
        dspy_examples = [ex.to_dspy() for ex in self._fewshot_examples]

        def _pred_eval(truth: dspy.Example, pred: dspy.Prediction, trace: Any | None = None) -> float:
            """Wraps optimization evaluation, injects model.

            :param truth: Ground truth.
            :param pred: Predicted value.
            :param trace: Optional trace information.
            :return: Metric value between 0.0 and 1.0.
            :raises KeyError: If target fields are missing from truth or prediction.
            :raises ValueError: If similarity score cannot be parsed from LLM response.
            """
            return self._evaluate_optimization_example(truth, pred, trace, model=optimizer.model)

        if verbose:
            best_prompt, best_examples = optimizer(signature, dspy_examples, _pred_eval, verbose=verbose)
        else:
            # Temporarily suppress DSPy logs.
            dspy_logger = logging.getLogger("dspy")
            optuna_logger = logging.getLogger("optuna")
            original_dspy_level = dspy_logger.level
            original_optuna_level = optuna_logger.level

            try:
                dspy_logger.setLevel(logging.ERROR)
                optuna_logger.setLevel(logging.ERROR)
                with warnings.catch_warnings():
                    warnings.simplefilter("ignore")
                    best_prompt, best_examples = optimizer(signature, dspy_examples, _pred_eval, verbose=verbose)
            finally:
                dspy_logger.setLevel(original_dspy_level)
                optuna_logger.setLevel(original_optuna_level)

        # Update few-shot examples and prompt instructions.
        fewshot_example_cls = self._fewshot_examples[0].__class__
        self._fewshot_examples = [fewshot_example_cls.from_dspy(ex) for ex in best_examples]
        self._validate_fewshot_examples()
        self._custom_prompt_instructions = best_prompt

        # Reinitialize bridge to use new prompt and few-shot examples.
        self._bridge = self._init_bridge(EngineType.get_engine_type(self._engine))

        return best_prompt, self._fewshot_examples


```

--------------------------------

### Convert Few-Shot Examples to Dictionaries (Python)

Source: https://sieves.ai/engines/outlines

Converts a sequence of Pydantic BaseModel objects representing few-shot examples into a list of dictionaries. This is useful for serializing complex data structures into a more universally compatible format. It relies on the `pydantic` library for model handling.

```python
import pydantic
from typing import Sequence, Any

class BaseModel:
    def model_dump(self, **kwargs):
        pass

@staticmethod
def convert_fewshot_examples(fewshot_examples: Sequence[BaseModel]) -> list[dict[str, Any]]:
    """Convert few‑shot examples to dicts.

    :param fewshot_examples: Fewshot examples to convert.
    :return: Fewshot examples as dicts.
    """
    return [fs_example.model_dump(serialize_as_any=True) for fs_example in fewshot_examples]
```

--------------------------------

### FewshotExample Class

Source: https://sieves.ai/tasks/predictive/question_answering

Represents a few-shot example for the Question Answering task, containing questions and their corresponding answers for a given context.

```APIDOC
## FewshotExample Class

### Description
Few-shot example with questions and answers for a context.

### Fields
- **questions** (tuple[str, ...] | list[str]) - A tuple or list of questions.
- **answers** (tuple[str, ...] | list[str]) - A tuple or list of answers corresponding to the questions.

### Methods
#### `from_dspy(example: dspy.Example)` `classmethod`
- **Description**: Convert from `dspy.Example`.
- **Parameters**:
  - **example** (Example) - Required - Example as `dspy.Example`.
- **Returns**:
  - **Self** - Example as `FewshotExample`.

#### `to_dspy()`
- **Description**: Convert to `dspy.Example`.
- **Returns**:
  - **Example** - Example as `dspy.Example`.
```

--------------------------------

### FewshotExample Class

Source: https://sieves.ai/tasks/predictive/summarization

Represents a few-shot example for text summarization, including input fields and conversion methods.

```APIDOC
## Class: FewshotExample

### Description
Few-shot example with a target summary.

### Fields
- **n_words** (int) - The number of words in the summary.
- **summary** (str) - The target summary text.

### Properties
#### input_fields
- **Description**: Defines which fields are inputs.
- **Returns**: Sequence[str] - Sequence of field names.

### Methods
#### from_dspy (classmethod)
- **Description**: Convert from `dspy.Example`.
- **Parameters**:
  - **example** (dspy.Example) - Example as `dspy.Example`.
- **Returns**: Self - Example as `FewshotExample`.

#### to_dspy
- **Description**: Convert to `dspy.Example`.
- **Returns**: Example - Example as `dspy.Example`.
```

--------------------------------

### GLiNER2 Static Method: convert_fewshot_examples

Source: https://sieves.ai/engines/gliner

A static method to convert few-shot examples from a sequence of BaseModel objects to a list of dictionaries.

```APIDOC
## `convert_fewshot_examples` static method

### Description
Convert few‑shot examples to dicts.

### Method
POST

### Endpoint
`/websites/sieves_ai/engines/core/convert_fewshot_examples`

### Parameters
#### Request Body
- **fewshot_examples** (Sequence[BaseModel]) - Required - Fewshot examples to convert.

### Request Example
```json
{
  "fewshot_examples": [
    {
      "text": "Example text 1",
      "label": "label1"
    },
    {
      "text": "Example text 2",
      "label": "label2"
    }
  ]
}
```

### Response
#### Success Response (200)
- **converted_examples** (list[dict[str, Any]]) - Fewshot examples as dicts.

#### Response Example
```json
{
  "converted_examples": [
    {
      "text": "Example text 1",
      "label": "label1"
    },
    {
      "text": "Example text 2",
      "label": "label2"
    }
  ]
}
```
```

--------------------------------

### Python: Create and Chain Pipelines

Source: https://sieves.ai/pipeline

Demonstrates creating a Sieve AI Pipeline using both verbose initialization with parameters like `use_cache` and succinct chaining with the `+` operator. It also shows how to chain entire pipelines and perform in-place appends using `+=`.

```python
from sieves import Pipeline, tasks

# Verbose initialization (allows non-default configuration).
t_ingest = tasks.preprocessing.Ingestion(export_format="markdown")
t_chunk = tasks.preprocessing.Chunking(chunker)
t_cls = tasks.predictive.Classification(labels=["science", "politics"], model=engine)
pipe = Pipeline([t_ingest, t_chunk, t_cls], use_cache=True)

# Succinct chaining (equivalent task order).
pipe2 = t_ingest + t_chunk + t_cls

# You can also chain pipelines and tasks.
pipe_left = Pipeline([t_ingest])
pipe_right = Pipeline([t_chunk, t_cls])
pipe3 = pipe_left + pipe_right  # results in [t_ingest, t_chunk, t_cls]

# In-place append (mutates the left pipeline).
pipe_left += t_chunk
pipe_left += pipe_right  # appends all tasks from right

# Note:
# - Additional Pipeline parameters (e.g., use_cache=False) are only settable via the verbose form
# - Chaining never mutates existing tasks or pipelines; it creates a new Pipeline
# - Using "+=" mutates the existing pipeline by appending tasks

```

--------------------------------

### Get Prompt Template API

Source: https://sieves.ai/bridge

Returns the prompt template as a string, which is constructed by chaining instructions, example templates, and conclusions. This is engine-dependent.

```APIDOC
## prompt_template Property

### Description
Returns the prompt template as a string. It chains `_prompt_instructions`, `_prompt_example_template`, and `_prompt_conclusion`. Mind engine-specific expectations.

### Method
`prompt_template` (getter property)

### Endpoint
Property access, not a direct endpoint.

### Parameters
None

### Request Example
```python
# template_string = bridge.prompt_template
```

### Response
#### Success Response (200)
- **str** - The prompt template as a string. Returns None if not used by the engine.

#### Response Example
```text
"Instructions: {{ instructions }}\n\nExamples:\n{{ examples }}\n\nConclusion: {{ conclusion }}"
```
```

--------------------------------

### Get Prompt Template

Source: https://sieves.ai/tasks/predictive/information_extraction

Retrieves the prompt template used by the engine. It chains prompt instructions, example templates, and conclusions. Note that different engines have different expectations for prompt formatting.

```APIDOC
## prompt_template property

### Description
Retrieves the prompt template used by the engine. It chains prompt instructions, example templates, and conclusions. Note that different engines have different expectations for prompt formatting.

### Method
GET

### Endpoint
/prompt_template

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
None

### Response
#### Success Response (200)
- **prompt_template** (str) - Prompt template as string. None if not used by engine.

#### Response Example
```json
{
  "prompt_template": "You are a helpful assistant. Extract the following information from the text: {{_prompt_instructions}} Example: {{_prompt_example_template}} Conclusion: {{_prompt_conclusion}}"
}
```
```

--------------------------------

### Get prompt template for engines (Python)

Source: https://sieves.ai/tasks/predictive/information_extraction

Retrieves the prompt template, which is constructed by chaining internal prompt components like instructions, examples, and conclusions. It notes engine-specific templating requirements, such as Jinja 2 for Outlines and different handling in DSPy.

```python
# This is a conceptual representation as the actual code for chaining is not provided.
# The property getter would look something like this:

# @property
# def prompt_template(self) -> str:
#     """Return prompt template."""
#     # Logic to chain _prompt_instructions, _prompt_example_template, and _prompt_conclusion
#     # Engine-specific formatting would be applied here.
#     return "" # Placeholder for actual template string
```

--------------------------------

### Model2Vec Framework Configuration for Distillation

Source: https://sieves.ai/guides/distillation

Demonstrates the configuration for the Model2Vec framework using `init_kwargs` and `train_kwargs` during distillation with Sieves AI. This example shows how to pass arguments for initializing the static model and training the classifier.

```python
task.distill(
    base_model_id="minishlab/potion-base-8M",
    framework=DistillationFramework.model2vec,
    data=docs,
    output_path="./model",
    val_frac=0.2,
    init_kwargs={
        # Passed to StaticModelForClassification.from_pretrained()
    },
    train_kwargs={
        # Passed to classifier.fit()
        "max_iter": 1000,
    }
)

```

--------------------------------

### Example Sieves Pipeline Configuration File (YAML)

Source: https://sieves.ai/guides/serialization

Presents the structure of a YAML file used to save sieves pipeline configurations. It details the class names, versions, task parameters, and placeholders for external components. This format is crucial for understanding and managing pipeline definitions.

```yaml
cls_name: sieves.pipeline.core.Pipeline
version: 0.11.1
tasks:
  is_placeholder: false
  value:
    - cls_name: sieves.tasks.preprocessing.chunkers.Chunker
      tokenizer:
        is_placeholder: true
        value: tokenizers.Tokenizer
      chunk_size:
        is_placeholder: false
        value: 512
      chunk_overlap:
        is_placeholder: false
        value: 50
      task_id:
        is_placeholder: false
        value: Chunker
    - cls_name: sieves.tasks.predictive.information_extraction.core.InformationExtraction
      engine:
        is_placeholder: false
        value:
          cls_name: sieves.engines.outlines_.Outlines
          model:
            is_placeholder: true
            value: outlines.models.transformers

```

--------------------------------

### Get Prompt Template (Python)

Source: https://sieves.ai/tasks/predictive/classification

Retrieves the prompt template used by the engine. This property chains together instruction, example, and conclusion templates. It's important to note that different engines have varying expectations for prompt formatting, such as Jinja 2 templating, and DSPy integrates these elements differently.

```python
# Get prompt template.
# Chains _prompt_instructions, _prompt_example_template and _prompt_conclusion.
# Note: different engines have different expectations as how a prompt should look like. E.g. outlines supports the Jinja 2 templating format for insertion of values and few-shot examples, whereas DSPy integrates these things in a different value in the workflow and hence expects the prompt not to include these things. Mind engine-specific expectations when creating a prompt template.
# Returns:
# Type | Description
# ---|---
# str | Prompt template as string. None if not used by engine.
```

--------------------------------

### Load Pipeline from Disk Configuration (Python)

Source: https://sieves.ai/pipeline

Generates a Pipeline instance by loading configuration from a specified file path. It first loads the configuration using `Config.load()` and then uses the `deserialize` class method to create the Pipeline instance, injecting any provided task arguments. The `path` can be a `Path` object or a string.

```python
@classmethod
def load(cls, path: Path | str, task_kwargs: Iterable[dict[str, Any]]) -> Pipeline:
    """Generate pipeline from disk.

    :param path: Path to config file.
    :param task_kwargs: Values to inject into loaded config.
    :return: Pipeline instance.
    """
    return cls.deserialize(Config.load(path), task_kwargs)

```

--------------------------------

### DSPy Engine Initialization and Configuration

Source: https://sieves.ai/engines/dspy

Initializes the DSPy engine, configuring the DSPy language model and disabling noisy logging. It supports local model serving via OLlama and requires generation settings including DSPy configuration.

```python
class DSPy(Engine[PromptSignature, Result, Model, InferenceMode]):
    """Engine for DSPy."""

    def __init__(self, model: Model, generation_settings: GenerationSettings):
        """Initialize engine.

        :param model: Model to run. Note: DSPy only runs with APIs. If you want to run a model locally from v2.5
            onwards, serve it with OLlama - see here: # https://dspy.ai/learn/programming/language_models/?h=models#__tabbed_1_5.
            In a nutshell:
            > curl -fsSL https://ollama.ai/install.sh | sh
            > ollama run MODEL_ID
            > `model = dspy.LM(MODEL_ID, api_base='http://localhost:11434', api_key='')`
        :param generation_settings: Settings including DSPy configuration in `config_kwargs`.
        """
        super().__init__(model, generation_settings)
        cfg = generation_settings.config_kwargs or {}
        dspy.configure(lm=model, **cfg)

        # Disable noisy LiteLLM logging.
        dspy.disable_litellm_logging()
        litellm._logging._disable_debugging()
```

--------------------------------

### Optimize Task Prompt and Examples in Python

Source: https://sieves.ai/tasks/predictive/translation

The optimize method refines a task's prompt and few-shot examples using a provided optimizer. It takes an Optimizer object and a verbose flag as input, returning the best prompt and examples found. This function requires at least two few-shot examples for optimization to proceed and updates the task's internal state with the optimized prompt and examples.

```python
def optimize(self, optimizer: optimization.Optimizer, verbose: bool = True) -> tuple[str, Sequence[FewshotExample]]:
    """Optimize task prompt and few-shot examples with the available optimization config.

    Updates task to use best prompt and few-shot examples found by the optimizer.

    :param optimizer: Optimizer to run.
    :param verbose: Whether to suppress output. DSPy produces a good amount of logs, so this can be useful to
        not pollute your terminal. Only warnings and errors will be printed.

    :return tuple[str, Sequence[FewshotExample]]: Best found prompt and few-shot examples.
    """
    assert len(self._fewshot_examples) > 1, "At least two few-shot examples need to be provided to optimize."

    # Run optimizer to get best prompt and few-shot examples.
    signature = self._get_task_signature()
    dspy_examples = [ex.to_dspy() for ex in self._fewshot_examples]

    def _pred_eval(truth: dspy.Example, pred: dspy.Prediction, trace: Any | None = None) -> float:
        """Wraps optimization evaluation, injects model.

        :param truth: Ground truth.
        :param pred: Predicted value.
        :param trace: Optional trace information.
        :return: Metric value between 0.0 and 1.0.
        :raises KeyError: If target fields are missing from truth or prediction.
        :raises ValueError: If similarity score cannot be parsed from LLM response.
        """
        return self._evaluate_optimization_example(truth, pred, trace, model=optimizer.model)

    if verbose:
        best_prompt, best_examples = optimizer(signature, dspy_examples, _pred_eval, verbose=verbose)
    else:
        # Temporarily suppress DSPy logs.
        dspy_logger = logging.getLogger("dspy")
        optuna_logger = logging.getLogger("optuna")
        original_dspy_level = dspy_logger.level
        original_optuna_level = optuna_logger.level

        try:
            dspy_logger.setLevel(logging.ERROR)
            optuna_logger.setLevel(logging.ERROR)
            with warnings.catch_warnings():
                warnings.simplefilter("ignore")
                best_prompt, best_examples = optimizer(signature, dspy_examples, _pred_eval, verbose=verbose)
        finally:
            dspy_logger.setLevel(original_dspy_level)
            optuna_logger.setLevel(original_optuna_level)

    # Update few-shot examples and prompt instructions.
    fewshot_example_cls = self._fewshot_examples[0].__class__
    self._fewshot_examples = [fewshot_example_cls.from_dspy(ex) for ex in best_examples]
    self._validate_fewshot_examples()
    self._custom_prompt_instructions = best_prompt

    # Reinitialize bridge to use new prompt and few-shot examples.
    self._bridge = self._init_bridge(EngineType.get_engine_type(self._engine))

    return best_prompt, self._fewshot_examples
```

--------------------------------

### Initialize DSPy Engine (`__init__`)

Source: https://sieves.ai/engines/dspy

Initializes the DSPy engine with a specified model and generation settings. It configures DSPy globally and disables verbose logging from LiteLLM. Requires a DSPy-compatible model and generation settings object.

```python
def __init__(self, model: Model, generation_settings: GenerationSettings):
    """Initialize engine.

    :param model: Model to run. Note: DSPy only runs with APIs. If you want to run a model locally from v2.5
        onwards, serve it with OLlama - see here: # https://dspy.ai/learn/programming/language_models/?h=models#__tabbed_1_5.
        In a nutshell:
        > curl -fsSL https://ollama.ai/install.sh | sh
        > ollama run MODEL_ID
        > `model = dspy.LM(MODEL_ID, api_base='http://localhost:11434', api_key='')`
    :param generation_settings: Settings including DSPy configuration in `config_kwargs`.
    """
    super().__init__(model, generation_settings)
    cfg = generation_settings.config_kwargs or {}
    dspy.configure(lm=model, **cfg)

    # Disable noisy LiteLLM logging.
    dspy.disable_litellm_logging()
    litellm._logging._disable_debugging()

```

--------------------------------

### Optimize PredictiveTask Prompt and Few-Shot Examples (Python)

Source: https://sieves.ai/tasks/predictive/pii_masking

Optimizes the task prompt and few-shot examples using a provided optimizer. This method updates the task with the best prompt and examples found during the optimization process. It requires at least two few-shot examples to run and can optionally suppress verbose output.

```python
import logging
import warnings
from typing import Any, Sequence, tuple

import dspy

from dspy.primitives.program import Program
from dspy.primitives.signature import Signature
from dspy.signatures import Signature
from dspy.tasks.predictive.core import FewshotExample, PredictiveTask, GenerationSettings, TaskPromptSignature, TaskResult, TaskBridge
from dspy.tasks.predictive.optimization import Optimizer
from dspy.tasks.predictive.engine import EngineType


def optimize(self, optimizer: Optimizer, verbose: bool = True) -> tuple[str, Sequence[FewshotExample]]:
    """Optimize task prompt and few-shot examples with the available optimization config.

    Updates task to use best prompt and few-shot examples found by the optimizer.

    :param optimizer: Optimizer to run.
    :param verbose: Whether to suppress output. DSPy produces a good amount of logs, so this can be useful to
        not pollute your terminal. Only warnings and errors will be printed.

    :return tuple[str, Sequence[FewshotExample]]: Best found prompt and few-shot examples.
    """
    assert len(self._fewshot_examples) > 1, "At least two few-shot examples need to be provided to optimize."

    # Run optimizer to get best prompt and few-shot examples.
    signature = self._get_task_signature()
    dspy_examples = [ex.to_dspy() for ex in self._fewshot_examples]

    def _pred_eval(truth: dspy.Example, pred: dspy.Prediction, trace: Any | None = None) -> float:
        """Wraps optimization evaluation, injects model.

        :param truth: Ground truth.
        :param pred: Predicted value.
        :param trace: Optional trace information.
        :return: Metric value between 0.0 and 1.0.
        :raises KeyError: If target fields are missing from truth or prediction.
        :raises ValueError: If similarity score cannot be parsed from LLM response.
        """
        return self._evaluate_optimization_example(truth, pred, trace, model=optimizer.model)

    if verbose:
        best_prompt, best_examples = optimizer(signature, dspy_examples, _pred_eval, verbose=verbose)
    else:
        # Temporarily suppress DSPy logs.
        dspy_logger = logging.getLogger("dspy")
        optuna_logger = logging.getLogger("optuna")
        original_dspy_level = dspy_logger.level
        original_optuna_level = optuna_logger.level

        try:
            dspy_logger.setLevel(logging.ERROR)
            optuna_logger.setLevel(logging.ERROR)
            with warnings.catch_warnings():
                warnings.simplefilter("ignore")
                best_prompt, best_examples = optimizer(signature, dspy_examples, _pred_eval, verbose=verbose)
        finally:
            dspy_logger.setLevel(original_dspy_level)
            optuna_logger.setLevel(original_optuna_level)

    # Update few-shot examples and prompt instructions.
    fewshot_example_cls = self._fewshot_examples[0].__class__
    self._fewshot_examples = [fewshot_example_cls.from_dspy(ex) for ex in best_examples]
    self._validate_fewshot_examples()
    self._custom_prompt_instructions = best_prompt

    # Reinitialize bridge to use new prompt and few-shot examples.
    self._bridge = self._init_bridge(EngineType.get_engine_type(self._engine))

    return best_prompt, self._fewshot_examples

```

--------------------------------

### PydanticBasedNER: Base Class for Pydantic NER

Source: https://sieves.ai/tasks/predictive/ner

The PydanticBasedNER class serves as a base for creating NER systems that utilize Pydantic models for input and output. It defines default prompt instructions, example templates, and conclusion formats to guide the extraction process. The `consolidate` method combines results from multiple processed chunks into a single Pydantic model output.

```python
class PydanticBasedNER(NERBridge[pydantic.BaseModel, pydantic.BaseModel, EngineInferenceMode], abc.ABC):
    """Base class for Pydantic-based NER bridges."""

    @override
    @property
    def _default_prompt_instructions(self) -> str:
        entity_info = self._get_entity_descriptions() if self._entity_descriptions else ""
        return f"""
        Your goal is to extract named entities from the text. Only extract entities of the specified types:
        {{{{ entity_types }}}}.
        {entity_info}

        For each entity:
        - Extract the exact text of the entity
        - Include a SHORT context string that contains ONLY the entity and AT MOST 3 words before and 3 words after it.
          DO NOT include the entire text as context. DO NOT include words that are not present in the original text
          as introductory words (Eg. 'Text:' before context string).
        - Specify which type of entity it is (must be one of the provided entity types)

        IMPORTANT:
        - If the same entity appears multiple times in the text, extract each occurrence separately with its own context
        """

    @override
    @property
    def _prompt_example_template(self) -> str | None:
        return """
        {% if examples|length > 0 -%}
            <examples>
            {%- for example in examples %}
                <example>
                    <text>{{ example.text }}</text>
                    <entity_types>{{ entity_types }}</entity_types>
                    <entities>
                        {%- for entity in example.entities %}
                        <entity>
                            <text>{{ entity.text }}</text>
                            <context>{{ entity.context }}</context>
                            <entity_type>{{ entity.entity_type }}</entity_type>
                        </entity>
                        {%- endfor %}
                    </entities>
                </example>
            {% endfor -%}
            </examples>
        {% endif %}
        """

    @override
    @property
    def _prompt_conclusion(self) -> str | None:
        return """
        ===========

        <text>{{ text }}</text>
        <entity_types>{{ entity_types }}</entity_types>
        <entities>
        """

    @override
    @cached_property
    def prompt_signature(self) -> type[pydantic.BaseModel]:
        entity_types = self._entities
        LiteralType = Literal[*entity_types]  # type: ignore[valid-type]

        class EntityWithContext(pydantic.BaseModel):
            text: str
            context: str
            entity_type: LiteralType

        class Prediction(pydantic.BaseModel):
            """NER prediction."""

            entities: list[EntityWithContext] = []

        return Prediction

    @override
    def consolidate(
        self, results: Iterable[pydantic.BaseModel], docs_offsets: list[tuple[int, int]]
    ) -> Iterable[pydantic.BaseModel]:
        results = list(results)

        # Process each document (which may consist of multiple chunks)
        for doc_offset in docs_offsets:
            doc_results = results[doc_offset[0] : doc_offset[1]]

            # Combine all entities from all chunks
            all_entities: list[dict[str, Any]] = []

            # Process each chunk for this document
            for chunk_result in doc_results:
                if chunk_result is None:
                    continue

                if not hasattr(chunk_result, "entities") or not chunk_result.entities:
                    continue

                # Process entities in this chunk
                for entity in chunk_result.entities:
                    # We just need to combine all entities from all chunks
                    all_entities.append(entity)

            # Create a consolidated result for this document - instantiate the class with entities
            yield self.prompt_signature(entities=all_entities)

```

--------------------------------

### Bridge Initialization API

Source: https://sieves.ai/bridge

Provides details on how to initialize a bridge object, including parameters for task identification, prompt customization, overwrite behavior, and generation settings.

```APIDOC
## __init__ Bridge

### Description
Initializes a new bridge object for task processing.

### Method
`__init__`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **task_id** (str) - Required - Task ID.
- **prompt_instructions** (str | None) - Required - Custom prompt instructions. If None, default instructions are used.
- **overwrite** (bool) - Required - Whether to overwrite text with produced text. Considered only by bridges for tasks producing fluent text - like translation, summarization, PII masking, etc.
- **generation_settings** (GenerationSettings) - Required - Generation settings including inference_mode.

### Request Example
```python
# Example initialization (assuming GenerationSettings is defined elsewhere)
# bridge = Bridge(
#     task_id="my_task_id",
#     prompt_instructions="Summarize the following text.",
#     overwrite=True,
#     generation_settings=GenerationSettings(...)
# )
```

### Response
#### Success Response (200)
None (initialization method)

#### Response Example
None
```

--------------------------------

### SentimentAnalysis Few-shot Example Validation (Python)

Source: https://sieves.ai/tasks/predictive/sentiment_analysis

Validates that the provided few-shot examples are consistent with the aspects defined for the SentimentAnalysis task. It checks for aspect mismatches between the task and the examples.

```python
    @override
    def _validate_fewshot_examples(self) -> None:
        for fs_example in self._fewshot_examples or []:
            if any([aspect not in self._aspects for aspect in fs_example.sentiment_per_aspect]) or not all(
                [label in fs_example.sentiment_per_aspect for label in self._aspects]
            ):
                raise ValueError(
                    f"Aspect mismatch: {self._task_id} has aspects {self._aspects}. Few-shot examples have "
                    f"aspects {fs_example.sentiment_per_aspect.keys()metaTag.{"."}}."
                )
```