### Install Pi SDK

Source: https://code.withpi.ai/index

Installs the 'withpi' Python package using pip. This is the first step to using the Pi SDK for LLM evaluation.

```bash
pip install withpi

```

--------------------------------

### Score LLM Input/Output with Pi SDK

Source: https://code.withpi.ai/index

Demonstrates how to use the PiClient to score LLM responses. It initializes the client, defines scoring questions, and prints the total score.

```python
from withpi import PiClient

pi = PiClient()
scores = pi.scoring_system.score(
  llm_input="Pi Labs",
  llm_output="Score anything with Pi Labs today!",
  scoring_spec=[{"question": "Is there a strong call to action?"}]
)
print(scores.total_score)

```

--------------------------------

### Search and Rank Passages with PiClient

Source: https://code.withpi.ai/api-reference/search/rank-documents

Initializes the PiClient with an API key and performs a search to rank passages against a given query. It logs the response to the console.

```javascript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.search.rank({ passages: ['string'], query: 'query' });

console.log(response);
```

--------------------------------

### Start Scoring System Calibration Job (JavaScript)

Source: https://code.withpi.ai/api-reference/scoring_systemcalibrate/start-job

Initiates a calibration job for the scoring system using the PiClient. This involves providing example inputs and outputs, preference examples, and the scoring specification questions. The function returns a job ID upon successful initiation.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const scoringSpecCalibrationStatus = await client.scoringSystem.calibrate.startJob({
  examples: [
    { llm_input: 'good input', llm_output: 'good response' },
    { llm_input: 'neutral input', llm_output: 'neutral response' },
  ],
  preference_examples: [
    { chosen: 'chosen response', llm_input: 'some input', rejected: 'rejected response' },
  ],
  scoring_spec: [{ question: 'Is this response truthful?' }, { question: 'Is this response relevant?' }],
});

console.log(scoringSpecCalibrationStatus.job_id);
```

--------------------------------

### Start Pi AI Scoring Job

Source: https://code.withpi.ai/api-reference/scoring_systemgenerate/start-job

Initiates a scoring job using the PiClient. This involves providing an application description and examples for training the scoring model. The function returns a job ID upon successful initiation.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.scoringSystem.generate.startJob({
  application_description: "Write a children's story communicating a simple life lesson.",
  examples: [
    { llm_input: 'good input', llm_output: 'good response' },
    { llm_input: 'neutral input', llm_output: 'neutral response' },
  ],
  preference_examples: [
    { chosen: 'chosen response', llm_input: 'some input', rejected: 'rejected response' },
  ],
});

console.log(response.job_id);
```

--------------------------------

### Stream Messages with Pi AI JavaScript Client

Source: https://code.withpi.ai/api-reference/datagenerate/stream-messages

This snippet demonstrates how to use the PiClient from the 'withpi' library to stream messages for a given job ID. It initializes the client with an API key and logs the response to the console. It also shows example HTTP status codes (200, 422) and a placeholder for string responses.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.data.generate.streamMessages('job_id');

console.log(response);
```

--------------------------------

### Start Synthetic Data Generation Job

Source: https://code.withpi.ai/api-reference/datagenerate_input_response_pairs/start-job

Initiates a job to generate synthetic input-response pairs using the PiClient. Requires an API key for authentication and specifies the number of pairs and seed data for generation.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const syntheticDataStatus = await client.data.generateInputResponsePairs.startJob({
  num_pairs_to_generate: 50,
  seeds: [
    {
      llm_input: 'Tell me something different',
      llm_output: 'The lazy dog was jumped over by the quick brown fox',
    },
  ],
});

console.log(syntheticDataStatus.job_id);
```

--------------------------------

### Start Data Generation Job with PiClient

Source: https://code.withpi.ai/api-reference/datagenerate/start-job

Initiates a data generation job using the PiClient. Requires an API key for authentication. The job takes an application description and seeds as input, and returns a job ID upon successful initiation.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const dataGenerationStatus = await client.data.generate.startJob({
  application_description: "Write a children's story communicating a simple life lesson.",
  num_inputs_to_generate: 50,
  seeds: [
    'The quick brown fox jumped over the lazy dog',
    'The lazy dog was jumped over by the quick brown fox',
  ],
});

console.log(dataGenerationStatus.job_id);
```

--------------------------------

### Initialize PiClient and Retrieve Job Data (JavaScript)

Source: https://code.withpi.ai/api-reference/scoring_systemgenerate/retrieve

This snippet demonstrates how to initialize the PiClient with an API key and retrieve job data using a job ID. It's a fundamental example for interacting with the Pi API.

```javascript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const generate = await client.scoringSystem.generate.retrieve('job_id');

console.log(generate.job_id);
```

--------------------------------

### Stream Messages from Scoring System

Source: https://code.withpi.ai/api-reference/scoring_systemcalibrate/stream-messages

This snippet demonstrates how to initialize the PiClient and stream messages from the scoring system for a given job ID. It logs the response to the console.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.scoringSystem.calibrate.streamMessages('job_id');

console.log(response);
```

--------------------------------

### Stream Messages with PiClient in JavaScript

Source: https://code.withpi.ai/api-reference/scoring_systemgenerate/stream-messages

This snippet shows how to initialize the PiClient with an API key and use it to stream messages for a specific job ID. It also logs the response to the console.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.scoringSystem.generate.streamMessages('job_id');

console.log(response);
```

--------------------------------

### Stream Data from Job using PiClient (JavaScript)

Source: https://code.withpi.ai/api-reference/datagenerate/stream-data

This snippet shows how to initialize the PiClient with an API key and then use it to stream data associated with a specific job ID. It logs the response to the console.

```javascript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.data.generate.streamData('job_id');

console.log(response);
```

--------------------------------

### Scoring System API Response Structure

Source: https://code.withpi.ai/api-reference/scoring_system/score

This example illustrates the structure of a successful response from the scoring system API, detailing the scores for individual questions and the overall total score.

```JSON
{
  "question_scores": {
    "Is the response relevant?": 0.5,
    "Is the response truthful?": 0.89
  },
  "total_score": 0.4
}
```

--------------------------------

### Embed Search Query with PiClient (JavaScript)

Source: https://code.withpi.ai/api-reference/search/embed-documents

This snippet shows how to initialize the PiClient with an API key and then use the client to embed a search query. It demonstrates a batch request with a single string query and logs the response. The expected HTTP status codes are 200 for success and 422 for errors.

```javascript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.search.embed({ batch: true, query: ['string'] });

console.log(response);
```

--------------------------------

### Synthetic Data Generation Job Response

Source: https://code.withpi.ai/api-reference/datagenerate_input_response_pairs/start-job

Example of a successful response from the synthetic data generation job. It includes the generated data, detailed status of the job, job ID, and the current state of the job.

```JSON
{
  "data": [
    {
      "llm_input": "Tell me something different",
      "llm_output": "The lazy dog was jumped over by the quick brown fox"
    },
    {
      "llm_input": "Write a short poem",
      "llm_output": "Moonlight dancing on waves,\nStars whisper ancient tales,\nNight's gentle embrace"
    }
  ],
  "detailed_status": [
    "Downloading model",
    "Tuning prompt"
  ],
  "job_id": "1234abcd",
  "state": "RUNNING"
}
```

--------------------------------

### Retrieve Data Generation Status with PiClient

Source: https://code.withpi.ai/api-reference/datagenerate/retrieve

This snippet shows how to initialize the PiClient with an API key and retrieve the status of a data generation job using its ID. It then logs the job ID from the retrieved status.

```javascript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const dataGenerationStatus = await client.data.generate.retrieve('job_id');

console.log(dataGenerationStatus.job_id);
```

--------------------------------

### Sample Data Generation Status Response

Source: https://code.withpi.ai/api-reference/datagenerate/list

This is an example of the JSON response structure when listing data generation statuses. It includes details like the job ID, current state, associated data, and detailed status messages.

```json
[
  {
    "data": [
      "The quick brown fox jumped over the lazy dog",
      "The lazy dog was jumped over by the quick brown fox"
    ],
    "detailed_status": [
      "Downloading model",
      "Tuning prompt"
    ],
    "job_id": "1234abcd",
    "state": "RUNNING"
  }
]
```

--------------------------------

### Data Generation Job Response Structure

Source: https://code.withpi.ai/api-reference/datagenerate/retrieve

This snippet illustrates the expected JSON response structure for a data generation job, including data, detailed status messages, job ID, and the current state of the job.

```json
{
  "data": [
    "The quick brown fox jumped over the lazy dog",
    "The lazy dog was jumped over by the quick brown fox"
  ],
  "detailed_status": [
    "Downloading model",
    "Tuning prompt"
  ],
  "job_id": "1234abcd",
  "state": "RUNNING"
}
```

--------------------------------

### Define Custom Pi AI Scorer

Source: https://code.withpi.ai/api-reference/scoring_systemgenerate/start-job

Provides a Python function to define custom scoring logic for the Pi AI service. This example scores responses based on word count, assigning a score and explanation.

```Python
def score(response_text: str, input_text: str, kwargs: dict) -> dict:
    word_count = len(response_text.split())
    if word_count > 10:
        return {"score": 0.2, "explanation": "Response has more than 10 words"}
    elif word_count > 5:
        return{"score": 0.6, "explanation": "Response has more than 5 words"}
    else:
        return {"score": 1, "explanation": "Response has 5 or fewer words"}
```

--------------------------------

### Sample Response for Embed Search (JavaScript)

Source: https://code.withpi.ai/api-reference/search/embed-documents

This snippet displays a sample response structure for the search embed operation, which is expected to be a 2D array of numbers (number[][]). This format is typically returned upon a successful API call.

```javascript
[
  [
    123
  ]
]
```

--------------------------------

### Scoring Spec Calibration Status Structure (JSON)

Source: https://code.withpi.ai/api-reference/scoring_systemcalibrate/list

This example details the structure of a scoring system calibration status object. It includes information about the calibrated scoring spec, which can contain custom Python scoring logic, job status, and other relevant details.

```JSON
[
  {
    "calibrated_scoring_spec": [
      {
        "custom_model_id": "your-model-id",
        "label": "Relevance to Prompt",
        "parameters": [
          0.14285714285714285,
          0.2857142857142857,
          0.42857142857142855,
          0.5714285714285714,
          0.7142857142857143,
          0.8571428571428571
        ],
        "python_code": "\ndef score(response_text: str, input_text: str, kwargs: dict) -> dict:\n    word_count = len(response_text.split())\n    if word_count > 10:\n        return {\"score\": 0.2, \"explanation\": \"Response has more than 10 words\"}
    elif word_count > 5:\n        return{\"score\": 0.6, \"explanation\": \"Response has more than 5 words\"}
    else:
        return {\"score\": 1, \"explanation\": \"Response has 5 or fewer words\"}
",
        "question": "Is the response relevant to the prompt?",
        "scoring_type": "PI_SCORER",
        "tag": "Legal Formatting",
        "weight": 1
      }
    ],
    "detailed_status": [
      "Downloading model",
      "Tuning prompt"
    ],
    "job_id": "1234abcd",
    "state": "RUNNING"
  }
]
```

--------------------------------

### Scoring Specification with Python Logic (JSON)

Source: https://code.withpi.ai/api-reference/scoring_systemgenerate/list

This JSON structure defines a scoring specification for the Pi AI system. It includes details like custom model ID, label, parameters, and a Python function to perform the scoring based on response text and input text. The example Python code scores responses based on word count.

```Python
def score(response_text: str, input_text: str, kwargs: dict) -> dict:
    word_count = len(response_text.split())
    if word_count > 10:
        return {"score": 0.2, "explanation": "Response has more than 10 words"}
    elif word_count > 5:
        return{"score": 0.6, "explanation": "Response has more than 5 words"}
    else:
        return {"score": 1, "explanation": "Response has 5 or fewer words"}
```

--------------------------------

### List Data Generation Statuses

Source: https://code.withpi.ai/api-reference/datagenerate/list

This snippet shows how to initialize the PiClient and retrieve a list of all data generation statuses. It logs the retrieved statuses to the console. Ensure you have a valid API key.

```javascript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const dataGenerationStatuses = await client.data.generate.list();

console.log(dataGenerationStatuses);
```

--------------------------------

### List Scoring System Generations (JavaScript)

Source: https://code.withpi.ai/api-reference/scoring_systemgenerate/list

This snippet demonstrates how to initialize the PiClient with an API key and then list all available scoring system generations. It logs the retrieved generations to the console.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const generates = await client.scoringSystem.generate.list();

console.log(generates);
```

--------------------------------

### Classify Queries using PiClient in JavaScript

Source: https://code.withpi.ai/api-reference/search/classify

This snippet shows how to initialize the PiClient with an API key and use the `queryClassifier.classify` method to categorize a list of queries. It defines custom classes with descriptions and labels, then logs the classification results.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.search.queryClassifier.classify({
  classes: [
    { description: 'Questions seeking objective, verifiable information or facts', label: 'factual' },
    {
      description: 'Questions asking for subjective judgments, preferences, or personal views',
      label: 'opinion',
    },
    { description: 'Questions about how to perform tasks or follow specific processes', label: 'procedural' },
  ],
  queries: [
    'What is the capital of France?',
    'How do I feel about the current political climate?',
    'What steps should I follow to bake a chocolate cake?',
  ],
});

console.log(response.results);
```

--------------------------------

### List Scoring System Calibration Statuses (JavaScript)

Source: https://code.withpi.ai/api-reference/scoring_systemcalibrate/list

This snippet shows how to initialize the PiClient and retrieve a list of scoring system calibration statuses. It requires an API key for authentication.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const scoringSpecCalibrationStatuses = await client.scoringSystem.calibrate.list();

console.log(scoringSpecCalibrationStatuses);
```

--------------------------------

### Data Generation Job Response Structure

Source: https://code.withpi.ai/api-reference/datagenerate/start-job

Illustrates the structure of a successful response from the data generation job. It includes the generated data, detailed status updates, the job ID, and the current state of the job.

```JSON
{
  "data": [
    "The quick brown fox jumped over the lazy dog",
    "The lazy dog was jumped over by the quick brown fox"
  ],
  "detailed_status": [
    "Downloading model",
    "Tuning prompt"
  ],
  "job_id": "1234abcd",
  "state": "RUNNING"
}
```

--------------------------------

### Evaluate Model Checkpoint with Pi and Unsloth (SFT)

Source: https://code.withpi.ai/integrations

This notebook shows how to evaluate a model checkpoint using Pi with the Unsloth toolkit. Unsloth is used for efficient fine-tuning of LLMs.

```python
from unsloth import FastLanguageModel

# Load your fine-tuned model
model, tokenizer = FastLanguageModel.from_pretrained(
    model="your_finetuned_model",
    # ... other parameters
)

# Assuming 'evaluate_with_pi' is a function that uses Pi for evaluation
# and 'test_data' is your test dataset.
results = evaluate_with_pi(model, tokenizer, test_data)
```

--------------------------------

### Score LLM Response using PiClient

Source: https://code.withpi.ai/api-reference/scoring_system/score

This snippet shows how to initialize the PiClient with an API key and use it to score an LLM's input and output based on a specified scoring criteria. It logs the total score to the console.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const scoringSystemMetrics = await client.scoringSystem.score({
  llm_input: 'Tell me something different',
  llm_output: 'The lazy dog was jumped over by the quick brown fox',
  scoring_spec: [{ question: 'Is this response truthful?' }, { question: 'Is this response relevant?' }],
});

console.log(scoringSystemMetrics.total_score);
```

--------------------------------

### Check Groundedness with Pi Client

Source: https://code.withpi.ai/api-reference/search/check-groundedness

Demonstrates how to use the PiClient to check the groundedness of search results. It initializes the client with an API key and calls the search.groundedness.check method with context and output parameters. The response, containing hallucination details, is then logged.

```javascript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.search.groundedness.check({ context: 'context', output: 'output' });

console.log(response.hallucinations);
```

--------------------------------

### Use Pi as Reward Function with Unsloth (GRPO)

Source: https://code.withpi.ai/integrations

This notebook demonstrates using Pi as a reward function with the Unsloth toolkit for GRPO (Generative Reinforcement Preference Optimization). GRPO allows tuning LLMs using reward functions without labeled data.

```python
from unsloth import FastLanguageModel

# Load your base model
model, tokenizer = FastLanguageModel.from_pretrained(
    model="your_base_model",
    # ... other parameters
)

# Assuming 'pi_reward_function' is defined elsewhere
# and 'generate_response' is a function to get model output.
reward = pi_reward_function(model_output=generate_response(model, tokenizer))
```

--------------------------------

### Use Pi as a Custom Metric in Promptfoo

Source: https://code.withpi.ai/integrations

This integration explains how to use Pi as a custom metric within Promptfoo, a platform focused on evaluating guardrails for LLM applications. Promptfoo helps in testing and comparing LLM outputs.

```python
import promptfoo

# Assuming 'pi_metric' is a function that uses Pi for evaluation
# and 'promptfoo_config' is your Promptfoo configuration.
results = promptfoo.run(promptfoo_config, metrics={'pi_metric': pi_metric})
```

--------------------------------

### Cancel Job with PiClient

Source: https://code.withpi.ai/api-reference/datagenerate/cancel

This snippet demonstrates how to initialize the PiClient and cancel a job using its `data.generate.cancel` method. It requires an API key for authentication and the `job_id` of the job to be canceled. The response from the API call is logged to the console.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.data.generate.cancel('job_id');

console.log(response);
```

--------------------------------

### Cancel Scoring System Job

Source: https://code.withpi.ai/api-reference/scoring_systemgenerate/cancel

This snippet shows how to initialize the PiClient with an API key and then use it to cancel a scoring system job by providing a job ID. It logs the response received from the API.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.scoringSystem.generate.cancel('job_id');

console.log(response);
```

--------------------------------

### Use Pi as a Langsmith Evaluator

Source: https://code.withpi.ai/integrations

This notebook demonstrates using Pi as a Langsmith evaluator. Langsmith, by Langchain, aids in building and evaluating orchestrated LLM applications.

```python
from langsmith import Client
from langsmith.schemas import Example

client = Client()

# Assuming 'pi_evaluator' is a function that evaluates using Pi
# and 'run_id' is the ID of the run to evaluate.
client.run_on_examples(run_id=run_id, examples=[Example(...)], evaluator=pi_evaluator)
```

--------------------------------

### Retrieve Scoring System Calibration Status

Source: https://code.withpi.ai/api-reference/scoring_systemcalibrate/retrieve

This snippet demonstrates how to use the PiClient in JavaScript to retrieve the calibration status of a scoring job. It requires an API key for authentication and a job ID to identify the specific calibration.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const scoringSpecCalibrationStatus = await client.scoringSystem.calibrate.retrieve('job_id');

console.log(scoringSpecCalibrationStatus.job_id);
```

--------------------------------

### Use Pi as Custom Metric in Helicone

Source: https://code.withpi.ai/integrations

This integration shows how Pi can be used as a custom metric within Helicone, an open-source observability platform for LLM applications. Pi helps in annotating quality alongside performance metrics.

```python
from helicone.api import Helicone

# Initialize Helicone client
helicone = Helicone()

# Assuming 'pi_quality_metric' is a function that calculates quality using Pi
# and 'response_data' contains the LLM response details.
quality_score = pi_quality_metric(response_data)
```

--------------------------------

### PiClient Query Classifier Response Structure

Source: https://code.withpi.ai/api-reference/search/classify

This snippet illustrates the expected JSON structure for the response from the PiClient's query classification endpoint. It details the 'results' array, where each item contains a 'prediction', 'probabilities' (with 'label' and 'score'), and the original 'query'.

```JSON
{
  "results": [
    {
      "prediction": "<string>",
      "probabilities": [
        {
          "label": "<string>",
          "score": 123
        }
      ],
      "query": "<string>"
    }
  ]
}
```

--------------------------------

### Cancel Scoring System Calibration in JavaScript

Source: https://code.withpi.ai/api-reference/scoring_systemcalibrate/cancel

This snippet demonstrates how to cancel a scoring system calibration job using the PiClient. It requires an API key for authentication and the job ID to identify the specific calibration to cancel.

```JavaScript
import PiClient from 'withpi';

const client = new PiClient({
  apiKey: 'My API Key',
});

const response = await client.scoringSystem.calibrate.cancel('job_id');

console.log(response);
```

--------------------------------

### Scoring Function for Response Relevance (Python)

Source: https://code.withpi.ai/api-reference/scoring_systemcalibrate/start-job

A Python function designed to be used within the Pi AI scoring system calibration. This function scores a response based on its word count, assigning a score and an explanation. It takes the response text, input text, and additional keyword arguments.

```Python
def score(response_text: str, input_text: str, kwargs: dict) -> dict:
    word_count = len(response_text.split())
    if word_count > 10:
        return {"score": 0.2, "explanation": "Response has more than 10 words"}
    elif word_count > 5:
        return{"score": 0.6, "explanation": "Response has more than 5 words"}
    else:
        return {"score": 1, "explanation": "Response has 5 or fewer words"}
```

--------------------------------

### Use Pi as a Scoring Function in Langfuse

Source: https://code.withpi.ai/integrations

This integration shows how to use Pi as a scoring function within Langfuse, a toolset for building LLM applications. Langfuse helps in tracing and observing LLM interactions.

```python
from langfuse import Langfuse

# Initialize Langfuse client
langfuse = Langfuse()

# Assuming 'pi_scoring_function' is defined elsewhere
# and 'llm_output' is the output from your LLM.
score = pi_scoring_function(llm_output)
```

--------------------------------

### Define Custom Scoring Logic (Python)

Source: https://code.withpi.ai/api-reference/scoring_systemgenerate/retrieve

This Python code defines a custom scoring function for the Pi scoring system. The function `score` takes response text, input text, and additional keyword arguments, and returns a score and an explanation based on the word count of the response.

```python
def score(response_text: str, input_text: str, kwargs: dict) -> dict:
    word_count = len(response_text.split())
    if word_count > 10:
        return {"score": 0.2, "explanation": "Response has more than 10 words"}
    elif word_count > 5:
        return{"score": 0.6, "explanation": "Response has more than 5 words"}
    else:
        return {"score": 1, "explanation": "Response has 5 or fewer words"}
```

--------------------------------

### Define Custom Scoring Logic

Source: https://code.withpi.ai/api-reference/scoring_systemcalibrate/retrieve

This snippet provides a Python function that can be used to define custom scoring logic for the Pi AI scoring system. The function takes response text and input text as arguments and returns a score and an explanation based on the word count of the response.

```Python
def score(response_text: str, input_text: str, kwargs: dict) -> dict:
    word_count = len(response_text.split())
    if word_count > 10:
        return {"score": 0.2, "explanation": "Response has more than 10 words"}
    elif word_count > 5:
        return{"score": 0.6, "explanation": "Response has more than 5 words"}
    else:
        return {"score": 1, "explanation": "Response has 5 or fewer words"}
```

--------------------------------

### Custom Scoring Function (Python)

Source: https://code.withpi.ai/api-reference/scoring_systemcalibrate/list

This Python code defines a scoring function that evaluates the word count of a response text. It returns a score and an explanation based on predefined word count thresholds.

```Python
def score(response_text: str, input_text: str, kwargs: dict) -> dict:
    word_count = len(response_text.split())
    if word_count > 10:
        return {"score": 0.2, "explanation": "Response has more than 10 words"}
    elif word_count > 5:
        return{"score": 0.6, "explanation": "Response has more than 5 words"}
    else:
        return {"score": 1, "explanation": "Response has 5 or fewer words"}
```