### AI21 SDK - Getting Started

Source: https://docs.ai21.com/docs/jamba-foundation-models

Information on using the AI21 Software Development Kit (SDK) to integrate AI21 models. This guide covers setup and basic usage patterns for developers.

```APIDOC
AI21 SDK:

Getting Started:
  The AI21 SDK provides convenient access to AI21's powerful language models. Follow these steps to get started:

Installation:
  - Python SDK: `pip install ai21`
  - Other SDKs: Refer to the documentation for available language SDKs.

Usage Example (Python):
  ```python
  import ai21

  # Set your AI21 API key
  ai21.api_key = "YOUR_API_KEY"

  # Example using the complete text generation endpoint
  try:
      response = ai21.Completion.create( 
          model="jamba-large",
          prompt="Write a short story about a robot learning to love.",
          numSegments=1,
          maxTokens=100
      )
      print(response.completions[0].data.text)
  except Exception as e:
      print(f"An error occurred: {e}")
  ```

Key Features:
  - Simplified API calls for various AI21 models.
  - Built-in error handling and response parsing.
  - Support for different model parameters (e.g., temperature, maxTokens).

Dependencies:
  - Requires an active AI21 API key.
  - Network connectivity to AI21 API endpoints.

Related Resources:
  - [API Reference](https://docs.ai21.com/reference/introduction)
  - [Playground](https://studio.ai21.com/v2)
```

--------------------------------

### AI21 Developer Platform Guides

Source: https://docs.ai21.com/docs/overview

Lists available guides for developers on the AI21 platform, covering topics from getting started to advanced techniques.

```APIDOC
Guides:
  Getting Started:
    Overview: https://docs.ai21.com/docs/overview
    SDK: https://docs.ai21.com/docs/sdk
  Fine-tuning: https://docs.ai21.com/docs/fine-tuning
  Quantization: https://docs.ai21.com/docs/quantization
  Tokenization: https://docs.ai21.com/docs/tokenization
```

--------------------------------

### Effective Requirements Examples

Source: https://docs.ai21.com/docs/walkthrough-guide

Provides examples of well-defined requirements for Maestro, covering word count, output format (JSON), and content constraints like avoiding jargon. These examples guide users in crafting precise instructions.

```python
requirements = [
    {
        "name": "word_count",
        "description": "Response must be exactly between 150-200 words"
    },
    {
        "name": "json_format",
        "description": "Output must be valid JSON with 'title' and 'content' fields"
    },
    {
        "name": "no_technical_jargon",
        "description": "Avoid technical terms; explain concepts in plain English"
    }
]
```

--------------------------------

### AI21 SDK Usage

Source: https://docs.ai21.com/docs/model-availability-across-platforms

Information on how to get started with the AI21 SDK, which is essential for interacting with AI21 Labs' models programmatically.

```en
To use the AI21 SDK, first ensure you have it installed. Refer to the installation guide for details. You will typically need an API key to authenticate your requests. The SDK provides methods for accessing various foundation models and features offered by AI21 Labs.
```

--------------------------------

### Install vLLM

Source: https://docs.ai21.com/docs/vllm

Installs the vLLM library with a specified version range. This is the initial step for using vLLM.

```bash
pip install vllm>=0.6.5,<=0.8.5.post1
```

--------------------------------

### Self-Deployment Guide

Source: https://docs.ai21.com/docs/model-availability-across-platforms

Instructions for self-deployment of AI21 models on custom infrastructure. This section links to a vLLM deployment guide for detailed steps and examples.

```APIDOC
Self-Deployment Guide:
For self-deployment on your own infrastructure, check out our [vLLM deployment guide](https://docs.ai21.com/docs/vllm) for step-by-step instructions and examples.
```

--------------------------------

### AI21 Maestro Validated Output Quick Start

Source: https://docs.ai21.com/docs/instruction-following-module

This guide focuses on AI21 Maestro's Validated Output feature, explaining its purpose in ensuring language model outputs consistently follow complex instructions with multiple constraints. It highlights how Maestro validates and refines outputs to meet specific requirements, providing a report on fulfillment.

```APIDOC
AI21 Maestro Validated Output:
  Purpose: Ensures language model outputs consistently follow complex instructions with multiple constraints.
  Features:
    - Validates outputs against explicit requirements.
    - Automatically fixes outputs that don't meet requirements.
    - Provides a report on requirement fulfillment with detailed scores.
  Key Concepts:
    - Requirements: Explicit constraints defined for outputs (e.g., format, tone, content rules).
    - Budget: Controls computational effort and trade-offs (high/medium/low) for adherence to requirements.
    - Requirements Report: Detailed scoring and feedback on requirement fulfillment.
    - Model Agnostic: Works with AI21's models and third-party models (e.g., GPT-4, Claude, Gemini).
```

--------------------------------

### Initialize and Start Training

Source: https://docs.ai21.com/docs/fine-tuning

Initializes the SFTTrainer with the model, tokenizer, training arguments, PEFT configuration, and dataset, then starts the training process.

```python
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    args=training_args,
    peft_config=lora_config,
    train_dataset=dataset,
)
trainer.train()
```

--------------------------------

### Install vLLM

Source: https://docs.ai21.com/docs/self-deployment

Installs the vLLM library with a specified version range.

```bash
pip install vllm>=0.6.5,<=0.8.5.post1
```

--------------------------------

### AI21 Development Guides

Source: https://docs.ai21.com/docs/use-cases-validated-output

Provides guides on essential AI development topics, including fine-tuning, quantization, and tokenization, to help users optimize their AI models and applications.

```APIDOC
Guides:
  Fine-tuning: https://docs.ai21.com/docs/fine-tuning
  Quantization: https://docs.ai21.com/docs/quantization
  Tokenization: https://docs.ai21.com/docs/tokenization
```

--------------------------------

### Content Requirements Example

Source: https://docs.ai21.com/docs/walkthrough-guide

An example of a content-focused requirement, directing the AI to include a specific number of concrete examples to illustrate concepts.

```python
{
    "name": "include_examples",
    "description": "Provide at least 2 concrete examples for each concept"
}
```

--------------------------------

### Install AI21 Tokenizer

Source: https://docs.ai21.com/docs/tokenization

Installs the AI21 tokenizer library using pip. This is the first step to using the tokenizer for Jamba models.

```bash
pip install ai21-tokenizer
```

--------------------------------

### Install AI21 Python SDK

Source: https://docs.ai21.com/docs/sdk

Installs the AI21 Python SDK using pip. This is the first step to using the SDK in your Python projects.

```shell
pip install ai21
```

--------------------------------

### Technical Requirements Example

Source: https://docs.ai21.com/docs/walkthrough-guide

An example of a technical requirement, guiding the AI to adhere to specific coding standards like PEP 8 and utilize type hints in its output.

```python
{
    "name": "python_best_practices",
    "description": "Follow PEP 8 style guidelines and use type hints"
}
```

--------------------------------

### Self Deployment Guide

Source: https://docs.ai21.com/docs/cloud-platform-deployment

Information on deploying AI21 models independently. This guide is for users who prefer to manage their own infrastructure for inference.

```APIDOC
Self Deployment Guide:
  Description: Instructions for self-managed deployments of AI21 models.
  Use Case: Users requiring full control over their deployment environment.
  Link: https://docs.ai21.com/docs/self-deployment
```

--------------------------------

### Install vLLM

Source: https://docs.ai21.com/docs/quantization

Installs the vLLM library with a specific version range, required for ExpertsInt8 quantization.

```bash
pip install vllm>=0.6.5,<=0.8.5.post1
```

--------------------------------

### Launch vLLM Server

Source: https://docs.ai21.com/docs/self-deployment

Starts the vLLM inference server with the AI21 Jamba Mini model, specifying quantization, tensor parallel size, and tool call parser.

```bash
vllm serve ai21labs/AI21-Jamba-Mini-1.6 \
  --quantization="experts_int8" \
  --tensor-parallel-size=8 \
  --enable-auto-tool-choice \
  --tool-call-parser jamba
```

--------------------------------

### Initialize and Start Training (LoRA)

Source: https://docs.ai21.com/docs/fine-tuning

Initializes the SFTTrainer with the model, tokenizer, training arguments, LoRA configuration, and dataset, then starts the fine-tuning process.

```python
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    args=training_args,
    peft_config=lora_config,
    train_dataset=dataset,
)
trainer.train()
```

--------------------------------

### Install Dependencies for QLoRA

Source: https://docs.ai21.com/docs/fine-tuning

Installs the necessary Python packages for QLoRA fine-tuning, including trl, transformers, torch, datasets, peft, and bitsandbytes.

```bash
pip install trl transformers torch datasets peft bitsandbytes
```

--------------------------------

### Install Libraries for 8-bit Quantization

Source: https://docs.ai21.com/docs/quantization

Installs the necessary Python libraries for using 8-bit quantization with Hugging Face Transformers.

```bash
pip install transformers torch bitsandbytes accelerate
```

--------------------------------

### Start vLLM Inference Server

Source: https://docs.ai21.com/docs/vllm

Launches the vLLM inference server for the AI21 Jamba Mini model with specific quantization and tool-choice configurations. This enables API-based inference.

```bash
vllm serve ai21labs/AI21-Jamba-Mini-1.7 \
  --quantization="experts_int8" \
  --enable-auto-tool-choice \
  --tool-call-parser jamba
```

--------------------------------

### Requirements to Avoid Examples

Source: https://docs.ai21.com/docs/walkthrough-guide

Illustrates poorly defined requirements that are vague, contradictory, or unmeasurable. These examples highlight common pitfalls to ensure more effective AI output.

```python
# Too vague
{"name": "good_quality", "description": "Make it good"}
# Contradictory
{"name": "short_and_detailed", "description": "Be brief but very detailed"}
# Unmeasurable
{"name": "creative", "description": "Be creative and original"}
```

--------------------------------

### Format Requirements Example

Source: https://docs.ai21.com/docs/walkthrough-guide

An example of a format-specific requirement for Maestro, instructing the AI to use markdown for structuring the output, including headers and code blocks.

```python
{
    "name": "markdown_format",
    "description": "Use proper markdown with headers, bullet points, and code blocks"
}
```

--------------------------------

### Advertisement Selection Prompt Example

Source: https://docs.ai21.com/docs/prompt-engineering

Demonstrates a prompt structure for an AI model to select the best advertisement based on provided criteria and examples. It uses delimiters like '####' to separate examples and instructions.

```text
Examples:
EXAMPLE AD 1
EXAMPLE AD 2
EXAMPLE ANSWER 1
####
EXAMPLE AD 3
EXAMPLE AD 4
EXAMPLE ANSWER 2
###
I want you to decide which of the following two advertisements are best, given the following criteria.
Criteria:
- Shorter is generally better
- If there is some kind of rhyme within the ad, that can add a little value
- References to superheroes are a huge plus
...
AD 1
AD 2
```

--------------------------------

### AI21 API Reference - Introduction

Source: https://docs.ai21.com/docs/model-availability-across-platforms

This section serves as an introduction to the AI21 API Reference, guiding users on how to access and utilize the various endpoints for AI21's services.

```APIDOC
API Reference Introduction:

This documentation provides detailed information about the AI21 API, including available endpoints, request parameters, response formats, and authentication methods. Use the API to integrate AI21's powerful language models into your applications.

Key areas covered:
- Authentication
- Model Endpoints (e.g., for Jamba models)
- Input/Output formats
- Rate Limiting
- Error Handling
```

--------------------------------

### AI21 Labs Fine-tuning and Quantization Guides

Source: https://docs.ai21.com/docs/training-data-1

Guides on customizing AI21 Labs models through fine-tuning and optimizing them using quantization techniques. This section covers the processes and best practices for both.

```APIDOC
Guides - Fine-tuning & Quantization:
  Fine-tuning:
    - Provides a step-by-step guide to fine-tuning AI21 models with custom datasets.
    - Explains data preparation, training parameters, and evaluation metrics.
  Quantization:
    - Details methods for quantizing models to reduce size and improve inference speed.
    - Discusses the trade-offs between quantization and model accuracy.
```

--------------------------------

### AI21 API Reference Introduction

Source: https://docs.ai21.com/docs/overview

Provides an introduction to the AI21 API Reference, guiding users on how to access and utilize the platform's API documentation.

```APIDOC
API Reference:
  Introduction: https://docs.ai21.com/reference/introduction
  Overview: https://docs.ai21.com/docs/overview
```

--------------------------------

### Install LoRA Fine-tuning Dependencies

Source: https://docs.ai21.com/docs/fine-tuning

Installs the necessary Python libraries for performing LoRA fine-tuning with AI21's Jamba models. This includes libraries for transformer models, datasets, and parameter-efficient fine-tuning.

```python
pip install trl transformers torch datasets peft
```

--------------------------------

### Prepare Dataset

Source: https://docs.ai21.com/docs/fine-tuning

Loads a dataset for fine-tuning. The example uses the 'philschmid/dolly-15k-oai-style' dataset, but users should replace this with their own data.

```python
# Load dataset (replace with your own dataset)
dataset = load_dataset("philschmid/dolly-15k-oai-style", split="train")
```

--------------------------------

### Install vLLM for Jamba Models

Source: https://docs.ai21.com/docs/vllm

This snippet demonstrates how to set up a Python virtual environment and install the vLLM package, which is recommended for deploying AI21's Jamba models. Ensure you use a compatible version of vLLM (v0.6.5 to v0.8.5.post1) for optimal performance.

```bash
# Create and activate virtual environment
python -m venv vllm-env
source vllm-env/bin/activate

```

--------------------------------

### AI21 Chat Completion Example

Source: https://docs.ai21.com/docs/sdk

Demonstrates how to use the AI21 Python SDK to create a chat completion. It shows how to initialize the client, define messages with roles, and call the chat completions API with a specified model.

```python
from ai21 import AI21Client
from ai21.models.chat import ChatMessage
client = AI21Client(
    # defaults to os.environ.get('AI21_API_KEY')
    api_key='my_api_key',
)
system = "You're a support engineer in a SaaS company"
messages = [
    ChatMessage(content=system, role="system"),
    ChatMessage(content="Hello, I need help with a signup process.", role="user"),
]
chat_completions = client.chat.completions.create(
    messages=messages,
    model="jamba-mini",
)

```

--------------------------------

### Install vLLM with Python Virtual Environment

Source: https://docs.ai21.com/docs/self-deployment

This snippet demonstrates how to set up a Python virtual environment and install the vLLM package, which is recommended for deploying AI21's Jamba models. Ensure you use a compatible vLLM version (v0.6.5 to v0.8.5.post1).

```python
python -m venv vllm-env
source vllm-env/bin/activate
```

--------------------------------

### Jamba SDK Integration

Source: https://docs.ai21.com/docs/jamba-foundation-models

Integrate Jamba into your projects using the AI21 SDK. This section provides guidance on setup and usage with sample code.

```Python
from ai21 import AI21Studio

# Initialize the client with your API key
client = AI21Studio(api_key="YOUR_API_KEY")

# Example for text completion
response = client.jamba.complete(
    model="jamba-1.5",
    prompt="The quick brown fox",
    num_results=1,
    max_tokens=10,
    temperature=0.7
)

print(response.completions[0].data.text)

# Example for instruction following
response_instruct = client.jamba.instruct(
    model="jamba-1.5-instruct",
    prompt="Write a short poem about a cat.",
    num_results=1,
    max_tokens=50,
    temperature=0.7
)

print(response_instruct.completions[0].data.text)

```

--------------------------------

### AI21 Labs API Reference Introduction

Source: https://docs.ai21.com/docs/use-cases-validated-output

Provides an introduction to the AI21 Labs API Reference, guiding users on how to access and utilize the API documentation for various AI models and services.

```APIDOC
API Reference:
  Introduction: https://docs.ai21.com/reference/introduction
  Overview: https://docs.ai21.com/docs/overview
  SDK: https://docs.ai21.com/docs/sdk
```

--------------------------------

### AI21 Labs API Reference Introduction

Source: https://docs.ai21.com/docs/training-data-1

This section provides an introduction to the AI21 Labs API Reference, guiding users on how to access and utilize the API for various AI tasks. It outlines the structure and content of the API documentation.

```APIDOC
API Reference:
  Introduction:
    - Provides an overview of the AI21 Labs API.
    - Guides users on API key management and authentication.
    - Outlines available models and their capabilities.
    - Links to detailed documentation for specific endpoints and functionalities.
```

--------------------------------

### AI21 Labs SDK Documentation

Source: https://docs.ai21.com/docs/training-data-1

Information regarding the AI21 Labs Software Development Kit (SDK), which simplifies integration with AI21 Labs' models and services. It covers installation and basic usage patterns.

```APIDOC
SDK:
  Overview:
    - Provides guidance on installing and setting up the AI21 Labs SDK.
    - Demonstrates basic usage patterns for interacting with AI models.
    - Lists supported programming languages and environments.
```

--------------------------------

### AI21 Labs API Reference Introduction

Source: https://docs.ai21.com/docs/safety-research-1

Provides an overview of the AI21 Labs API, guiding users to different sections for specific functionalities and endpoints. It serves as a starting point for developers integrating AI21's models.

```APIDOC
API Reference:
  Introduction: Guides users to various API documentation sections.
  Guides:
    Overview: General introduction to AI21 Labs' capabilities.
    SDK: Information on Software Development Kits for integration.
  Foundation Models:
    Jamba: Details on Jamba foundation models.
    Model Availability by Platform: Information on model availability across different platforms.
    Prompt Engineering for Jamba Models: Guidance on effective prompt engineering for Jamba.
    Function Calling: Documentation on implementing function calling.
    Batch API: Information on using the Batch API for Jamba models.
  AI21 Maestro [Beta]:
    Overview: Introduction to the AI21 Maestro platform.
    RAG: Details on Retrieval-Augmented Generation within Maestro.
    Validated Output: Information on validated output features.
  Private AI:
    vLLM: Documentation related to vLLM.
    Cloud Platform Deployment: Guidance on deploying on cloud platforms.
    Troubleshooting & Performance Optimization: Tips for resolving issues and improving performance.
    Local Inference: Information on running models locally.
  Guides:
    Fine-tuning: Steps and best practices for fine-tuning models.
    Quantization: Explanation of quantization techniques.
    Tokenization: Details on tokenization processes.
  Usage:
    Account: Information on managing user accounts.
    Pricing: Details on pricing and cost management.
  AI Ethics & Data Transparency:
    Responsible Use: Guidelines for responsible AI deployment.
    Safety Research: Information on AI safety research initiatives.
    Training Data: Details on the training data used.
  Additional Resources:
    Status: Link to system status updates.
    Community: Link to the AI21 Labs community forum (Discord).
    Blog: Link to the official AI21 Labs blog.
```

--------------------------------

### AI21 Fine-tuning Guide

Source: https://docs.ai21.com/docs/conversational-rag

A guide on the process of fine-tuning AI21 models to adapt them for specific tasks or datasets.

```APIDOC
Fine-tuning Guide:
  Purpose: Instructions and best practices for fine-tuning AI21 models.
  Content:
    - Steps for preparing data.
    - Model training parameters.
    - Evaluation metrics.
```

--------------------------------

### Traditional LLM Prompting Example

Source: https://docs.ai21.com/docs/walkthrough-guide

Demonstrates a common approach to prompting Large Language Models (LLMs) using Python. It highlights the unreliability of LLMs in consistently adhering to all specified constraints within a single prompt, leading to potential issues like unmet requirements and lack of output visibility.

```Python
client.complete(
    prompt="""Write a Python function that:
    - Calculates fibonacci numbers
    - Is under 10 lines
    - Has proper docstrings
    - Uses descriptive variable names"""
)
```

--------------------------------

### Style Requirements Example

Source: https://docs.ai21.com/docs/walkthrough-guide

An example of a style requirement, specifying the desired tone for the AI's output, such as using professional business language and avoiding slang.

```python
{
    "name": "professional_tone",
    "description": "Use formal business language, avoid contractions and slang"
}
```

--------------------------------

### Initialize Tokenizer and Configure Quantization (QLoRA)

Source: https://docs.ai21.com/docs/fine-tuning

Initializes the tokenizer and sets up a BitsAndBytesConfig for 4-bit quantization, specifying the quantization type and compute data type for efficient fine-tuning.

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from datasets import load_dataset
from trl import SFTTrainer, SFTConfig
from peft import LoraConfig
tokenizer = AutoTokenizer.from_pretrained("ai21labs/AI21-Jamba-Mini-1.7")
# Configure 4-bit quantization
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,  # Enable 4-bit quantization
    bnb_4bit_quant_type="nf4",  # Use NormalFloat 4-bit quantization
    bnb_4bit_compute_dtype=torch.bfloat16,  # Compute in bfloat16 for better stability
)
```

--------------------------------

### Install vLLM for FP8 Quantization

Source: https://docs.ai21.com/docs/quantization

This snippet shows how to install the vLLM library, which is a prerequisite for using FP8 quantization with AI21's Jamba models on Hopper architecture GPUs.

```python
pip install vllm>=0.6.5,<=0.8.5.post1
```

--------------------------------

### LLM Evaluation Prompt Example

Source: https://docs.ai21.com/docs/prompt-engineering

This example demonstrates how to use another LLM to evaluate the output of a primary LLM. It includes criteria for assessment and a specified response format, ensuring objective evaluation.

```APIDOC
<<context from original prompt>>
###
Based on the context above, does the following answer provide a correct and full answer to the question based on this context.
Pay attention to the following criteria:
- The answer must be correct according to the information in the context
- The answer that uses information that is not available in the context is a bad answer.
- If the question can’t be answered by the information in the context, the answer should be “Answer not in document
- If the answer is in the context, and is provided, that should increase the score of the answer.
- The answer must fully answer the question and not be too short. 
- All relevant information should be in the answer.
- A vague or general answer is not very good.
Reply in the following format:
- Score: Good, OK, or Bad
- Reasoning: Why this score was chosen.
###
<<answer from original prompt>>


```

--------------------------------

### Prompt Design Tips: Clarity and Formatting

Source: https://docs.ai21.com/docs/prompt-engineering

Demonstrates the importance of clear, unambiguous instructions and structured output formats for AI prompts.

```text
## Bad
List all the following animals, objects and places in the story.  
  
{story}:  
  
Your lists:
## Better
List all the following animals, objects and places in the story.Your output should be in the following JSON format:   
  
{ ‘animals’: a list of all animals in following the story, ‘Objects’: a list of all objects in following the story, ‘places’: a list of all objects in following the story }  
  
Story:  
  
{story}  
  
–End of story –
```

--------------------------------

### AI21 API Reference - Introduction

Source: https://docs.ai21.com/docs/fine-tuning

Provides an overview of the AI21 API, detailing its capabilities, authentication methods, and general usage guidelines. It serves as the entry point for developers looking to integrate AI21's language models into their applications.

```APIDOC
API Reference:
  Introduction:
    - Overview: Provides a general introduction to the AI21 API.
    - Authentication: Details how to authenticate API requests using API keys.
    - Rate Limits: Information on request limits and how to manage them.
    - Error Handling: Explains common error codes and how to handle them.
    - SDKs: Links to available Software Development Kits for various programming languages.

  Models:
    - Jamba:
      - Description: Information about the Jamba family of foundation models.
      - Capabilities: Details on text generation, summarization, translation, etc.
      - Endpoints:
        - /jamba/complete: For text completion tasks.
        - /jamba/embed: For generating text embeddings.

  Fine-tuning:
    - Overview: Explains the process of fine-tuning AI21 models.
    - Methods:
      - Full Fine-tuning: Details on updating all model parameters.
      - LoRA: Information on Low-Rank Adaptation for efficient fine-tuning.
      - QLoRA: Details on combining LoRA with 4-bit quantization.
    - Prerequisites: Required libraries and hardware specifications.
    - Implementation Guides: Step-by-step instructions for fine-tuning.

  AI Maestro [Beta]:
    - Overview: Introduction to the AI Maestro platform.
    - Features:
      - RAG (Retrieval-Augmented Generation)
      - Validated Output

  Private AI:
    - vLLM: Information on deploying models using vLLM.
    - Cloud Platform Deployment: Guides for deploying on cloud infrastructure.
    - Local Inference: Instructions for running models locally.
    - Troubleshooting & Performance Optimization: Tips for improving model performance.

  Usage:
    - Account Management: How to manage your AI21 account and API keys.
    - Pricing: Information on the cost of using AI21 services.

  AI Ethics & Data Transparency:
    - Responsible Use: Guidelines for ethical AI deployment.
    - Safety Research: Information on AI21's safety research initiatives.
    - Training Data: Details about the data used for training models.
```

--------------------------------

### Classification/Ranking Prompt Example

Source: https://docs.ai21.com/docs/prompt-engineering

Demonstrates a prompt for classifying sentence consistency using predefined categories.

```text
Analyze whether the two sentences provided are consistent or inconsistent. Classify into one of the following classes: 
[Consistent, partially consistent, inconsistent]  

Sentence A:  
“When talking on the phone, the defendant confessed to felony murder.”  

Sentence B:  
“The defendant admitted nothing when talking on the phone.”  

Your classification:
```

--------------------------------

### AI21 Tokenization Guide

Source: https://docs.ai21.com/docs/conversational-rag

Explanation of tokenization processes used by AI21 models, including how text is converted into tokens for processing.

```APIDOC
Tokenization Guide:
  Purpose: Understanding how text is tokenized for AI21 models.
  Details:
    - Tokenizer algorithms used.
    - Handling of special characters and subwords.
    - Vocabulary size.
```

--------------------------------

### Python Chat Completion Example

Source: https://docs.ai21.com/reference/jamba-1-6-api-ref

Demonstrates how to use the AI21 Python SDK to create a chat completion with the Jamba model.

```Python
from ai21 import AI21Client
from ai21.models.chat import ChatMessage
messages = [
    ChatMessage(role="user", content="Hello how are you?"),
]
client = AI21Client()
client.chat.completions.create(
    messages=messages,
    model="jamba-large",
    max_tokens=1024,
)
```

--------------------------------

### Local Inference with llama.cpp

Source: https://docs.ai21.com/docs/local-inference

Instructions for running AI21's Jamba models locally using llama.cpp. It highlights the use of GGUF format and optional GPU acceleration, and advises caution regarding third-party GGUF conversions.

```markdown
# Local Inference
For fully local execution, [llama.cpp](https://github.com/ggml-org/llama.cpp) enables running compatible open models in the GGUF format, with optional GPU acceleration.   
  
AI21 publishes official Jamba model weights on the [Hugging Face Hub](https://huggingface.co/ai21labs), and community contributors may provide GGUF-format conversions (e.g., Jamba Mini 1.7) for use with `llama.cpp`.
**Note:** AI21 does not distribute or support GGUF builds and cannot verify the accuracy of third-party conversions. Be sure to review the model’s license terms and consult the `llama.cpp` documentation before use.
```

--------------------------------

### Asynchronous Tokenization

Source: https://docs.ai21.com/docs/tokenization

Provides an example of using the asynchronous tokenizer for high-performance or server applications. It shows how to encode and decode text asynchronously.

```python
import asyncio
from ai21_tokenizer import Tokenizer
async def main():
    tokenizer = await Tokenizer.get_async_tokenizer()
    text = "Async tokenization for async operations!"
    encoded = await tokenizer.encode(text)
    decoded = await tokenizer.decode(encoded)
    print(f"Original: {text}")
    print(f"Encoded: {encoded}")
    print(f"Decoded: {decoded}")
asyncio.run(main())
```

--------------------------------

### Complete Python Example for Currency Conversion

Source: https://docs.ai21.com/docs/function-calling

A comprehensive Python script demonstrating the entire function calling workflow: defining the tool schema, implementing the currency conversion function using an external API, making the initial chat request, executing the function call, and obtaining the final response.

```Python
import json
import requests
from ai21 import AI21Client
from ai21.models.chat import ChatMessage

def convert_currency(amount: float, from_currency: str, to_currency: str) -> str:
    """Convert an amount from one currency to another using live exchange rates"""
    # Using exchangerate-api.com (free tier available)
    try:
        response = requests.get(
            f"https://api.exchangerate-api.com/v4/latest/{from_currency.upper()}"
        )
        response.raise_for_status() # Raise an exception for bad status codes
        data = response.json()
        if to_currency.upper() not in data["rates"]:
            return json.dumps({"error": f"Currency {to_currency} not found"})
        
        exchange_rate = data["rates"][to_currency.upper()]
        converted_amount = amount * exchange_rate
        result = {
            "original_amount": amount,
            "from_currency": from_currency.upper(),
            "to_currency": to_currency.upper(),
            "exchange_rate": exchange_rate,
            "converted_amount": round(converted_amount, 2)
        }
        return json.dumps(result)
    except requests.exceptions.RequestException as e:
        return json.dumps({"error": f"API request failed: {e}"})
    except Exception as e:
        return json.dumps({"error": f"An unexpected error occurred: {e}"})

# Define the function schema for the model
tools = [
    {
        "type": "function",
        "function": {
            "name": "convert_currency",
            "description": "Convert an amount from one currency to another using current exchange rates",
            "parameters": {
                "type": "object",
                "properties": {
                    "amount": {
                        "type": "number",
                        "description": "The amount to convert"
                    },
                    "from_currency": {
                        "type": "string",
                        "description": "The source currency code (e.g., USD, EUR, GBP)"
                    },
                    "to_currency": {
                        "type": "string",
                        "description": "The target currency code (e.g., USD, EUR, GBP)"
                    }
                },
                "required": ["amount", "from_currency", "to_currency"]
            }
        }
    }
]

# --- Main execution flow ---
client = AI21Client()
messages = [
    ChatMessage(
        role="user", 
        content="How much is 100 USD in euros?"
    )
]

# 1. Make the initial chat request
response = client.chat.completions.create(
    model="jamba-large",
    messages=messages,
    tools=tools,
    max_tokens=150
)

message = response.choices[0].message

# 2. Execute Function Calls if needed
if message.tool_calls:
    for tool_call in message.tool_calls:
        function_name = tool_call.function.name
        function_args = json.loads(tool_call.function.arguments)
        
        function_result = None
        if function_name == "convert_currency":
            function_result = convert_currency(**function_args)
        
        if function_result is not None:
            # Add the function call and result to the conversation
            messages.append(message)  # Assistant's function call
            messages.append(ChatMessage(
                role="tool",
                tool_call_id=tool_call.id,
                content=function_result
            ))

            # 3. Get the final response with function results
            final_response = client.chat.completions.create(
                model="jamba-large",
                messages=messages,
                tools=tools,
                max_tokens=150
            )
            print(final_response.choices[0].message.content)
        else:
            print(f"Unknown function call: {function_name}")
else:
    # No function call needed, model responded directly
    print(message.content)
```

--------------------------------

### AI21 Quantization Guide

Source: https://docs.ai21.com/docs/conversational-rag

Details on quantization techniques for AI21 models, aimed at reducing model size and improving inference speed.

```APIDOC
Quantization Guide:
  Purpose: Information on applying quantization to AI21 models.
  Benefits:
    - Reduced model size.
    - Faster inference.
  Considerations:
    - Potential impact on accuracy.
```

--------------------------------

### AI21 File Upload Response

Source: https://docs.ai21.com/reference/manage-library-ref/upload-workspace-files

Example of a successful response when uploading a file to the AI21 library, containing the file's unique identifier.

```JSON
{
  "id": "da13301a-14e4-4487-aa2f-cc6048e73cdc"
}
```

--------------------------------

### AI21 API Reference - Introduction

Source: https://docs.ai21.com/docs/jamba-foundation-models

Provides an overview of the AI21 API, including authentication, endpoints, and general usage guidelines. This section is crucial for developers integrating AI21 models into their applications.

```APIDOC
AI21 API Reference:

Introduction:
  This document outlines the AI21 API, providing essential information for developers to integrate AI21's advanced language models into their applications.

Authentication:
  - API Key: Obtain your API key from the AI21 Studio dashboard.
  - Header: Include your API key in the 'Authorization' header as 'Bearer YOUR_API_KEY'.

Base URL:
  - The base URL for all API requests is https://api.ai21.com/v1/

Rate Limiting:
  - Standard rate limits apply. Refer to the AI21 Studio documentation for details on specific limits and how to request increases.

Error Handling:
  - API responses include standard HTTP status codes.
  - Error details are provided in the JSON response body.

Key Concepts:
  - Models: AI21 offers various models, including the Jamba family, each with different capabilities and performance characteristics.
  - Endpoints: Specific API endpoints are available for tasks such as text generation, completion, and embedding.

Versioning:
  - The API is versioned. Ensure you are using the latest stable version or specify the version in your requests if necessary.

Related Resources:
  - [AI21 Studio](https://studio.ai21.com/)
  - [API Key Management](https://studio.ai21.com/v2/account/api-key)
  - [Playground](https://studio.ai21.com/v2)
```