### Project Setup and Dependency Management with uv

Source: https://github.com/toon-format/toon-python/blob/main/CONTRIBUTING.md

This snippet demonstrates how to clone the toon-python repository, install dependencies using uv, and run tests. uv automatically handles virtual environment creation.

```bash
git clone https://github.com/toon-format/toon-python.git
cd toon-python
uv sync
uv run pytest
uv run pytest --cov=src/toon_format --cov-report=term-missing
```

--------------------------------

### Simple Configuration Example in Python and TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

A basic example showing a Python dictionary representing application configuration and its equivalent TOON format. Demonstrates key-value pairs and boolean/numeric types.

```python
{
    "app": "myapp",
    "version": "1.0.0",
    "debug": False,
    "port": 8080
}
```

```toon
app: myapp
version: "1.0.0"
debug: false
port: 8080
```

--------------------------------

### Install and Verify toon_format from PyPI

Source: https://github.com/toon-format/toon-python/blob/main/PUBLISHING.md

Installs the toon_format package from the production PyPI and verifies the installation by printing the package version.

```bash
pip install toon_format
python -c "import toon_format; print(toon_format.__version__)"
```

--------------------------------

### Install TOON Python Library

Source: https://github.com/toon-format/toon-python/blob/main/README.md

Instructions for installing the TOON library from source or GitHub using uv or pip.

```bash
git clone https://github.com/toon-format/toon-python.git
cd toon-python
uv sync

pip install git+https://github.com/toon-format/toon-python.git
```

--------------------------------

### Install toon_format from TestPyPI

Source: https://github.com/toon-format/toon-python/blob/main/PUBLISHING.md

Installs the toon_format package from the TestPyPI repository. This is a crucial step to verify the package before publishing to the main PyPI.

```bash
pip install --index-url https://test.pypi.org/simple/ toon_format
```

--------------------------------

### Toon CLI Usage

Source: https://context7.com/toon-format/toon-python/llms.txt

Command-line interface examples for converting files between JSON and Toon, handling stdin/stdout, and applying encoding/decoding flags.

```bash
# Convert JSON to Toon
toon input.json -o output.toon

# Convert Toon to JSON
toon data.toon -o output.json

# Custom options via CLI
toon data.json --encode --delimiter "\t" --indent 4 --length-marker -o output.toon

# Pipeline example
cat input.json | toon - | toon --decode - | jq .
```

--------------------------------

### TOON Token Efficiency Estimation in Python

Source: https://github.com/toon-format/toon-python/blob/main/docs/README.md

Provides examples of using `estimate_savings` and `compare_formats` from the `toon_format` package to quantify token savings and compare TOON with JSON.

```python
from toon_format import estimate_savings, compare_formats

data = {"employees": [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]}

# Get savings metrics
result = estimate_savings(data)
print(f"Saves {result['savings_percent']:.1f}% tokens")

# Get formatted comparison
print(compare_formats(data))
```

--------------------------------

### CLI Usage for TOON Conversion

Source: https://github.com/toon-format/toon-python/blob/main/README.md

Command line interface examples for encoding and decoding files, including support for stdin/stdout and custom formatting options.

```bash
toon input.json -o output.toon
toon data.toon -o output.json
echo '{"x": 1}' | toon -

toon data.json --encode --delimiter "\t" --length-marker
toon data.toon --decode --no-strict --indent 4
```

--------------------------------

### Configure TOON Encoder and Decoder Options

Source: https://github.com/toon-format/toon-python/blob/main/tests/README.md

Examples of configuration objects used to control the behavior of TOON encoders and decoders during testing.

```json
{
  "delimiter": ",",
  "indent": 2,
  "lengthMarker": ""
}

{
  "indent": 2,
  "strict": true
}
```

--------------------------------

### Decode TOON Strings to Python

Source: https://github.com/toon-format/toon-python/blob/main/docs/api.md

Shows how to parse TOON-formatted strings back into Python objects. Includes examples of using both dictionaries and DecodeOptions classes for configuration.

```python
from toon_format import decode
from toon_format.types import DecodeOptions

# Simple decoding
decode("name: Alice\nage: 30")

# Tabular arrays
decode("users[2,]{id,name}:\n  1,Alice\n  2,Bob")

# With options (class or dict)
decode("  item: value", DecodeOptions(indent=4, strict=False))
```

--------------------------------

### Apply String Quoting and Escape Sequences

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Provides examples of when strings must be quoted in TOON and how to use escape sequences for special characters.

```python
{
    "text": "Hello \"world\"\nNew line",
    "path": "C:\\Users\\Alice"
}
```

```toon
text: "Hello \"world\"\nNew line"
path: "C:\\Users\\Alice"
```

--------------------------------

### Build and Test toon_format Package Locally

Source: https://github.com/toon-format/toon-python/blob/main/PUBLISHING.md

Builds the toon_format Python package and performs local verification. This includes cleaning previous build artifacts, creating the package distribution files, and testing the installation and version import in a clean virtual environment.

```bash
# Clean previous builds
rm -rf dist/ build/ *.egg-info

# Build the package
python -m build

# Verify the package contents
python -m zipfile -l dist/toon_format-X.Y.Z-py3-none-any.whl

# Test installation in a clean environment
python -m venv test_env
test_env/bin/pip install dist/toon_format-X.Y.Z-py3-none-any.whl
test_env/bin/python -c "import toon_format; print(toon_format.__version__)"
rm -rf test_env
```

--------------------------------

### Nested Structure with Arrays in Python and TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Presents a more complex example with nested objects and arrays, comparing a Python dictionary structure to its TOON representation. Highlights array length indicators and nested object formatting.

```python
{
    "metadata": {
        "version": 2,
        "author": "Alice"
    },
    "items": [
        {"id": 1, "name": "Item1", "qty": 10},
        {"id": 2, "name": "Item2", "qty": 5}
    ],
    "tags": ["alpha", "beta", "gamma"]
}
```

```toon
metadata:
  version: 2
  author: Alice
items[2,]{id,name,qty}:
  1,Item1,10
  2,Item2,5
tags[3]: alpha,beta,gamma
```

--------------------------------

### Decode Toon to Python Objects

Source: https://context7.com/toon-format/toon-python/llms.txt

Explains how to parse Toon strings back into Python objects. Includes examples of strict vs. lenient modes and handling nested structures.

```python
from toon_format import decode, DecodeOptions

# Strict mode (default) validates structure
strict_options = DecodeOptions(indent=2, strict=True)
result = decode("users[2,]{id,name}:\n  1,Alice\n  2,Bob", strict_options)

# Lenient mode accepts inconsistencies like length mismatches
lenient_options = DecodeOptions(indent=2, strict=False)
mismatched = decode("items[10]: a,b,c", lenient_options)

# Robust error handling pattern
def robust_decode(toon_str, fallback=None):
    try:
        return decode(toon_str, DecodeOptions(strict=True))
    except Exception:
        return decode(toon_str, DecodeOptions(strict=False)) if True else fallback
```

--------------------------------

### Order Preservation in Python and TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Illustrates that TOON always preserves the order of object keys and array elements during encoding and decoding, using Python's `OrderedDict` as an example. The decoded output maintains the original key order.

```python
from collections import OrderedDict

data = OrderedDict([("z", 1), ("a", 2), ("m", 3)])
toon = encode(data)
```

```toon
z: 1
a: 2
m: 3
```

```python
decoded = decode(toon)
list(decoded.keys())  # ['z', 'a', 'm']
```

--------------------------------

### Advanced Encoding and Decoding with Options

Source: https://github.com/toon-format/toon-python/blob/main/README.md

Demonstrates how to pass configuration options like delimiters, indentation, and strict mode to the encode and decode functions.

```python
encode({"id": 123}, {"delimiter": "\t", "indent": 4, "lengthMarker": "#"})
decode("id: 123", {"indent": 2, "strict": True})
```

--------------------------------

### Run Tests and Checks with uv

Source: https://github.com/toon-format/toon-python/blob/main/PUBLISHING.md

Execute project tests, linting, and type checking using the 'uv' command-line tool. This ensures the code quality and correctness before building and publishing.

```bash
uv run pytest
uv run ruff check .
uv run mypy src/toon_format
```

--------------------------------

### Handle Decoding Errors

Source: https://github.com/toon-format/toon-python/blob/main/docs/api.md

Provides examples of catching ToonDecodeError exceptions when encountering malformed TOON input, such as unterminated strings or length mismatches.

```python
from toon_format import decode, ToonDecodeError

# Unterminated string
try:
    decode('text: "unterminated')
except ToonDecodeError as e:
    print(e)

# Array length mismatch
try:
    decode("items[3]: a,b")
except ToonDecodeError as e:
    print(e)
```

--------------------------------

### Define Error Test Case

Source: https://github.com/toon-format/toon-python/blob/main/tests/README.md

An example of a test case configured to expect an error, used to validate strict parsing or invalid input handling.

```json
{
  "name": "throws on array length mismatch",
  "input": "tags[3]: a,b",
  "expected": null,
  "shouldError": true,
  "options": { "strict": true }
}
```

--------------------------------

### Running Tests with pytest and Coverage

Source: https://github.com/toon-format/toon-python/blob/main/CONTRIBUTING.md

This snippet covers running the full test suite using pytest and generating a coverage report. It also shows how to fail the build if line coverage drops below 85%.

```bash
uv run pytest tests/
uv run pytest --cov=toon_format --cov-report=term --cov-fail-under=85
```

--------------------------------

### Linting and Formatting with ruff

Source: https://github.com/toon-format/toon-python/blob/main/CONTRIBUTING.md

These commands illustrate how to use ruff for checking code style and formatting the source and test files in the toon-python project. This ensures code consistency.

```bash
uv run ruff check src/ tests/
uv run ruff format src/ tests/
```

--------------------------------

### Python: Encode Data to TOON with Length Markers

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Shows how to use the `toon_format` library to encode data into TOON format, specifically demonstrating the use of `lengthMarker` for explicit validation hints to LLMs. This helps ensure array lengths are correctly interpreted.

```python
from toon_format import encode

data = {"items": ["a", "b", "c"]}
toon = encode(data, {"lengthMarker": "#"})
# items[#3]: a,b,c
```

--------------------------------

### Type Checking with mypy

Source: https://github.com/toon-format/toon-python/blob/main/CONTRIBUTING.md

This command shows how to run mypy for type checking on the source code of the toon-python project. Type hints are mandatory for all code.

```bash
uv run mypy src/
```

--------------------------------

### Create and Push Git Tag for Release

Source: https://github.com/toon-format/toon-python/blob/main/PUBLISHING.md

Creates an annotated Git tag for the new release version and pushes both the tag and the main branch to the remote repository.

```bash
git tag -a vX.Y.Z -m "Release version X.Y.Z"
git push origin main
git push origin vX.Y.Z
```

--------------------------------

### Implement Primitive and Tabular Arrays

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Illustrates the use of length indicators [N] for arrays and the CSV-like tabular format for uniform object lists.

```python
[1, 2, 3, 4, 5]
```

```toon
[5]: 1,2,3,4,5
```

```python
[
    {"id": 1, "name": "Alice", "age": 30},
    {"id": 2, "name": "Bob", "age": 25},
    {"id": 3, "name": "Charlie", "age": 35}
]
```

```toon
[3,]{id,name,age}:
  1,Alice,30
  2,Bob,25
  3,Charlie,35
```

--------------------------------

### Indentation Handling in Python and TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Compares Python dictionary structures with their TOON equivalents, demonstrating default 2-space indentation and how to configure it to 4 spaces using the `encode` function. Strict mode enforces consistent indentation and disallows tabs.

```python
{
    "level1": {
        "level2": {
            "level3": "value"
        }
    }
}
```

```toon
level1:
  level2:
    level3: value
```

```python
encode(data, {"indent": 4})
```

```toon
level1:
    level2:
        level3: value
```

--------------------------------

### Encoding Data for LLM Prompts

Source: https://context7.com/toon-format/toon-python/llms.txt

Demonstrates how to encode Python dictionaries into the TOON format to minimize token consumption within LLM prompts.

```python
context_data = {
    "users": [
        {"id": 1, "name": "Alice", "role": "admin"},
        {"id": 2, "name": "Bob", "role": "user"}
    ],
    "settings": {"max_users": 100}
}
toon_context = encode(context_data)
print(toon_context)
```

--------------------------------

### Configure Encoding Options

Source: https://github.com/toon-format/toon-python/blob/main/docs/api.md

Demonstrates advanced encoding features including handling large integers, custom delimiters, length markers, and indentation settings.

```python
# Large integer handling
encode({"bigInt": 9007199254740992})

# Custom delimiters
encode([1, 2, 3], {"delimiter": "|"})

# Length markers
encode(users, {"lengthMarker": "#"})

# Zero indentation
encode({"outer": {"inner": 1}}, {"indent": 0})
```

--------------------------------

### TOON Encoding and Decoding with Options in Python

Source: https://github.com/toon-format/toon-python/blob/main/docs/README.md

Shows how to customize TOON encoding and decoding behavior using options, such as specifying a custom delimiter for encoding or enabling lenient decoding.

```python
# Custom delimiter
encode([1, 2, 3], {"delimiter": "\t"})
# [3\t]: 1\t2\t3

# Lenient decoding
decode("items[5]: a,b,c", {"strict": False})
# {'items': ['a', 'b', 'c']}  # Accepts length mismatch
```

--------------------------------

### Implement Type-Safe TOON Encoding and Decoding

Source: https://github.com/toon-format/toon-python/blob/main/docs/api.md

Demonstrates how to use type hints with the TOON library to ensure data integrity during encoding and decoding. It utilizes EncodeOptions and DecodeOptions to configure the transformation process.

```python
from typing import Any, Dict, List, Union
from toon_format import encode, decode
from toon_format.types import EncodeOptions, DecodeOptions, JsonValue

# Type-safe usage
data: Dict[str, Any] = {"key": "value"}
options: EncodeOptions = EncodeOptions(delimiter=",")
result: str = encode(data, options)

decoded: JsonValue = decode(result)
```

--------------------------------

### LLM Integration Pattern

Source: https://context7.com/toon-format/toon-python/llms.txt

Template for instructing LLMs to output data in Toon format to optimize token usage.

```python
TOON_SYSTEM_PROMPT = """Respond using TOON format:
- Use `key: value` for objects
- Use indentation for nesting
- Use `[N]` to indicate array lengths
- Use tabular format `[N,]{fields}:` for uniform arrays"""
```

--------------------------------

### Define Simple and Nested Objects in TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Demonstrates how to map Python dictionary structures to TOON object syntax. It covers basic key-value pairs and nested object hierarchies.

```python
{"name": "Alice", "age": 30, "active": True}
```

```toon
name: Alice
age: 30
active: true
```

```python
{
    "user": {
        "name": "Alice",
        "settings": {
            "theme": "dark"
        }
    }
}
```

```toon
user:
  name: Alice
  settings:
    theme: dark
```

--------------------------------

### Basic TOON Encoding in Python

Source: https://github.com/toon-format/toon-python/blob/main/docs/README.md

Demonstrates the basic usage of the `encode` function from the `toon_format` package to convert a Python dictionary into TOON format.

```python
from toon_format import encode

data = {"name": "Alice", "age": 30}
print(encode(data))
```

--------------------------------

### Python: Estimate Token Savings with toon_format

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Estimates token savings between JSON and TOON formats for given data structures. Useful for understanding potential cost reductions before integrating TOON into LLM applications. It takes a Python dictionary as input and returns token counts and savings percentage.

```python
from toon_format import estimate_savings

user_data = {
    "users": [
        {"id": 1, "name": "Alice", "email": "alice@example.com", "active": True},
        {"id": 2, "name": "Bob", "email": "bob@example.com", "active": True},
        {"id": 3, "name": "Charlie", "email": "charlie@example.com", "active": False}
    ]
}

result = estimate_savings(user_data)
print(f"JSON: {result['json_tokens']} tokens")
print(f"TOON: {result['toon_tokens']} tokens")
print(f"Savings: {result['savings_percent']:.1f}%")
```

--------------------------------

### compare_formats

Source: https://context7.com/toon-format/toon-python/llms.txt

Generates a formatted comparison table showing JSON vs TOON metrics including token counts, character sizes, and savings percentage.

```APIDOC
## compare_formats

### Description
Generates a formatted comparison table showing JSON vs TOON metrics including token counts, character sizes, and savings percentage.

### Usage

**Visual comparison for documentation:**
```python
from toon_format import compare_formats

api_response = {
    "status": "success",
    "results": [
        {"id": 1, "score": 0.95, "category": "A"},
        {"id": 2, "score": 0.87, "category": "B"},
        {"id": 3, "score": 0.92, "category": "A"}
    ],
    "total": 3
}

print(compare_formats(api_response))
```

**Compare different data structures:**
```python
from toon_format import compare_formats

simple_config = {"app": "myapp", "version": "1.0.0", "debug": False, "port": 8080}
print("\nSimple config:")
print(compare_formats(simple_config))

nested_data = {
    "metadata": {"version": 2, "author": "Alice"},
    "items": [{"id": 1, "name": "Item1"}, {"id": 2, "name": "Item2"}],
    "tags": ["alpha", "beta", "gamma"]
}
print("\nNested data:")
print(compare_formats(nested_data))
```

### Output for API Response Example
```
Format Comparison
────────────────────────────────────────────────
Format      Tokens    Size (chars)
JSON            78             189
TOON            48             112
────────────────────────────────────────────────
Savings: 30 tokens (38.5%)
```

### Output for Simple Config Example
```
Simple config:
Format Comparison
────────────────────────────────────────────────
Format      Tokens    Size (chars)
JSON            32              65
TOON            20              39
────────────────────────────────────────────────
Savings: 12 tokens (37.5%)
```

### Output for Nested Data Example
```
Nested data:
Format Comparison
────────────────────────────────────────────────
Format      Tokens    Size (chars)
JSON            71             177
TOON            45             109
────────────────────────────────────────────────
Savings: 26 tokens (36.6%)
```
```

--------------------------------

### Handle Object Keys with Special Characters

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Shows how to handle non-standard keys in TOON, such as numeric keys, keys with spaces, or dashes, which require quoting.

```python
{
    "simple_key": 1,
    "with-dash": 2,
    "123": 3,
    "with space": 4,
    "": 5
}
```

```toon
simple_key: 1
with-dash: 2
"123": 3
"with space": 4
"": 5
```

--------------------------------

### Configure Array Delimiters

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Demonstrates how to change the default comma delimiter to tabs or pipes for array data in TOON.

```python
encode([1, 2, 3], {"delimiter": "\t"})
```

```toon
[3	]: 1	2	3
```

```python
encode([1, 2, 3], {"delimiter": "|"})
```

```toon
[3|]: 1|2|3
```

--------------------------------

### Compare Formats: JSON vs. TOON Metrics

Source: https://context7.com/toon-format/toon-python/llms.txt

Generates a formatted comparison table detailing token counts, character sizes, and savings percentages between JSON and TOON formats for various data structures. This function, `compare_formats`, aids in visualizing the efficiency differences.

```python
from toon_format import compare_formats

# Visual comparison for documentation
api_response = {
    "status": "success",
    "results": [
        {"id": 1, "score": 0.95, "category": "A"},
        {"id": 2, "score": 0.87, "category": "B"},
        {"id": 3, "score": 0.92, "category": "A"}
    ],
    "total": 3
}

print(compare_formats(api_response))

# Compare different data structures
simple_config = {"app": "myapp", "version": "1.0.0", "debug": False, "port": 8080}
print("\nSimple config:")
print(compare_formats(simple_config))

nested_data = {
    "metadata": {"version": 2, "author": "Alice"},
    "items": [{"id": 1, "name": "Item1"}, {"id": 2, "name": "Item2"}],
    "tags": ["alpha", "beta", "gamma"]
}
print("\nNested data:")
print(compare_formats(nested_data))
```

--------------------------------

### TOON Decoding and Error Handling

Source: https://context7.com/toon-format/toon-python/llms.txt

Demonstrates how to handle various decoding errors and provides a safe decoding wrapper pattern.

```APIDOC
## TOON Decoding and Error Handling

### Description
This section covers handling various decoding errors that can occur with the TOON format and demonstrates a safe decoding wrapper pattern.

### Code Examples

**Handling Decode Errors:**
```python
from toon_format import decode, ToonDecodeError

# Test cases for various decode errors
test_cases = [
    ('text: "unterminated', "Unterminated string"),
    ("items[3]: a,b", "Array length mismatch"),
    ("outer:\n   inner: value", "Invalid indentation"),
]

for toon_input, description in test_cases:
    try:
        decode(toon_input)
    except ToonDecodeError as e:
        print(f"{description}: {e}")
        if hasattr(e, 'line') and e.line:
            print(f"  at line {e.line}")
```

**Safe Decoding Wrapper:**
```python
from toon_format import decode, ToonDecodeError

def safe_decode(toon_str, default=None):
    """Safely decode TOON with fallback."""
    try:
        return decode(toon_str, {"strict": True})
    except ToonDecodeError as e:
        print(f"Decode failed: {e}")
        return default

# Usage example
result = safe_decode("invalid[: data", default={})
print(f"Result: {result}")
```

### Output of Safe Decoding Example
```
Decode failed: ...
Result: {}
```
```

--------------------------------

### Representing Numbers in Python and TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Demonstrates how integers, floats, and special number representations are handled in both Python and the TOON format. Note that TOON encoders must use decimal form, and non-finite values are normalized to null.

```python
42
-17
0
```

```toon
42
-17
0
```

```python
3.14
-0.5
0.0
```

```toon
3.14
-0.5
0
```

```python
9007199254740993  # Exceeds JS safe integer
```

```toon
"9007199254740993"  # Quoted for JS compatibility
```

--------------------------------

### Analyze Token Efficiency

Source: https://github.com/toon-format/toon-python/blob/main/README.md

Utilities to estimate token savings, compare TOON with other formats, and count tokens using tiktoken.

```python
from toon_format import estimate_savings, compare_formats, count_tokens

data = {"users": [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]}
result = estimate_savings(data)
print(f"Saves {result['savings_percent']:.1f}% tokens")

print(compare_formats(data))

toon_str = encode(data)
tokens = count_tokens(toon_str)
```

--------------------------------

### Compare Formats via Table

Source: https://github.com/toon-format/toon-python/blob/main/docs/api.md

Generates a formatted string table comparing JSON and TOON metrics, including token counts and character sizes. Useful for quick visual performance analysis.

```python
from toon_format import compare_formats

data = {
    "users": [
        {"id": 1, "name": "Alice", "age": 30},
        {"id": 2, "name": "Bob", "age": 25}
    ]
}

print(compare_formats(data))
```

--------------------------------

### Basic TOON Decoding in Python

Source: https://github.com/toon-format/toon-python/blob/main/docs/README.md

Illustrates the fundamental use of the `decode` function from the `toon_format` package to parse a TOON formatted string back into a Python dictionary.

```python
from toon_format import decode

toon = "items[2]: apple,banana"
data = decode(toon)
# {'items': ['apple', 'banana']}
```

--------------------------------

### Python: Encode Data to TOON and Count Tokens with toon_format

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Demonstrates how to encode Python data into TOON format and count the resulting tokens. This function is essential for integrating TOON into LLM workflows, allowing for real-time token monitoring and comparison between TOON and JSON.

```python
from toon_format import encode, count_tokens

def send_to_llm(data, use_toon=True):
    """Send data to LLM with optional TOON encoding."""
    if use_toon:
        formatted = encode(data)
        format_type = "TOON"
    else:
        formatted = json.dumps(data, indent=2)
        format_type = "JSON"

    tokens = count_tokens(formatted)
    print(f"[{format_type}] Sending {tokens} tokens")

    # Your LLM API call here
    # response = openai.ChatCompletion.create(...)

    return formatted, tokens

# Example usage
data = {"items": [{"id": 1}, {"id": 2}]}
formatted, token_count = send_to_llm(data, use_toon=True)
```

--------------------------------

### Python: Compare JSON and TOON Formats with toon_format

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Provides a detailed comparison report between JSON and TOON formats for a given data structure, including token counts and character sizes. This is useful for documentation and in-depth analysis of format efficiency.

```python
from toon_format import compare_formats

api_response = {
    "status": "success",
    "results": [
        {"id": 1, "score": 0.95, "category": "A"},
        {"id": 2, "score": 0.87, "category": "B"},
        {"id": 3, "score": 0.92, "category": "A"}
    ],
    "total": 3
}

print(compare_formats(api_response))
```

--------------------------------

### Integrate TOON with Anthropic Claude API

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Shows how to configure Claude to respond in TOON format and clean the resulting string to ensure successful decoding.

```python
import anthropic
from toon_format import decode

def claude_toon(prompt):
    client = anthropic.Anthropic()

    message = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        messages=[{
            "role": "user",
            "content": f"{prompt}\n\nRespond in TOON format (Token-Oriented Object Notation)."
        }]
    )

    toon_str = message.content[0].text

    if "```" in toon_str:
        toon_str = toon_str.split("```")[1].strip()
        if toon_str.startswith("toon\n"):
            toon_str = toon_str[5:]

    return decode(toon_str)
```

--------------------------------

### Integrate TOON with OpenAI API

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Demonstrates how to request TOON format from OpenAI's ChatCompletion API and extract the data from markdown code blocks for decoding.

```python
import openai
from toon_format import decode

def ask_for_toon_data(prompt):
    response = openai.ChatCompletion.create(
        model="gpt-5",
        messages=[
            {"role": "system", "content": "Respond using TOON format"},
            {"role": "user", "content": prompt}
        ]
    )

    toon_str = response.choices[0].message.content

    if "```toon" in toon_str:
        toon_str = toon_str.split("```toon")[1].split("```")[0].strip()
    elif "```" in toon_str:
        toon_str = toon_str.split("```")[1].split("```")[0].strip()

    return decode(toon_str)
```

--------------------------------

### Encode Python Objects to TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/api.md

Demonstrates how to convert Python dictionaries and lists into TOON-formatted strings using the encode function. Supports custom configuration options like delimiters and length markers.

```python
from toon_format import encode
from toon_format.types import EncodeOptions

# Simple encoding
encode({"name": "Alice", "age": 30})

# With options (dict)
encode([1, 2, 3], {"delimiter": "\t"})

# With typed options (TypedDict)
options: EncodeOptions = {"delimiter": "|", "indent": 4, "lengthMarker": "#"}
encode([1, 2, 3], options)
```

--------------------------------

### Python: Estimate LLM Request Costs with toon_format

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Calculates estimated costs for LLM API requests using both JSON and TOON formats. This function helps in quantifying dollar savings based on token usage and provided pricing. It requires data and an estimated cost per 1K tokens.

```python
from toon_format import estimate_savings

prompt_data = {
    "context": [
        {"role": "system", "content": "You are a helpful assistant"},
        {"role": "user", "content": "Analyze this data"}
    ],
    "data": [
        {"id": i, "value": f"Item {i}", "score": i * 10}
        for i in range(1, 101)  # 100 items
    ]
}

result = estimate_savings(prompt_data["data"])

# GPT-5 pricing (example: $0.01 per 1K tokens)
cost_per_1k = 0.01
json_cost = (result['json_tokens'] / 1000) * cost_per_1k
toon_cost = (result['toon_tokens'] / 1000) * cost_per_1k

print(f"JSON cost per request: ${json_cost:.4f}")
print(f"TOON cost per request: ${toon_cost:.4f}")
print(f"Savings per request: ${json_cost - toon_cost:.4f}")
print(f"Savings per 10,000 requests: ${(json_cost - toon_cost) * 10000:.2f}")
```

--------------------------------

### Configure Custom Delimiters for TOON Encoding

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Shows how to specify custom delimiters during the encoding process to handle data containing commas or tabs.

```python
from toon_format import encode

# Use tab for data with commas
encode(data, {"delimiter": "\t"})

# Use pipe for data with tabs
encode(data, {"delimiter": "|"})
```

--------------------------------

### Count Tokens in Strings using TOON

Source: https://context7.com/toon-format/toon-python/llms.txt

Demonstrates how to count tokens in both JSON and TOON formatted strings using the `count_tokens` function, which leverages the `tiktoken` library. It highlights the token reduction achieved by using TOON format compared to JSON.

```python
from toon_format import count_tokens, encode
import json

# Count tokens in different formats
data = {"users": [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]}

json_str = json.dumps(data)
toon_str = encode(data)

json_tokens = count_tokens(json_str)
toon_tokens = count_tokens(toon_str)

print(f"JSON ({len(json_str)} chars): {json_tokens} tokens")
print(f"TOON ({len(toon_str)} chars): {toon_tokens} tokens")
print(f"Reduction: {(1 - toon_tokens/json_tokens) * 100:.1f}%")

# Production integration pattern
def send_to_llm(data, use_toon=True):
    """Send data to LLM with token tracking."""
    if use_toon:
        formatted = encode(data)
        format_type = "TOON"
    else:
        formatted = json.dumps(data, indent=2)
        format_type = "JSON"

    tokens = count_tokens(formatted)
    print(f"[{format_type}] Sending {tokens} tokens")

    # Your LLM API call here
    return formatted, tokens

formatted, token_count = send_to_llm(data, use_toon=True)
```

--------------------------------

### Strict Validation for TOON Decoding

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Demonstrates how to enable strict mode during decoding to ensure data integrity and handle potential parsing errors.

```python
try:
    data = decode(response, {"strict": True})
except ToonDecodeError:
    # Handle validation failure
    pass
```

--------------------------------

### Encode Data to Toon Format

Source: https://context7.com/toon-format/toon-python/llms.txt

Demonstrates how to serialize Python dictionaries and lists into Toon strings. Supports custom delimiters, indentation levels, and optional length markers for validation.

```python
from toon_format import encode, EncodeOptions

# Default encoding
data = [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]
print(encode(data))

# Custom options: Tab delimiter and 2-space indent
options = {"delimiter": "\t", "indent": 2}
print(encode(data, options))

# With length marker for explicit validation
options_marker = {"delimiter": ",", "indent": 2, "lengthMarker": "#"}
print(encode(data, options_marker))
```

--------------------------------

### Configure Decoding Validation

Source: https://github.com/toon-format/toon-python/blob/main/docs/api.md

Illustrates the difference between strict and lenient decoding modes. Strict mode enforces indentation, delimiter consistency, and array length validation.

```python
from toon_format import decode, ToonDecodeError
from toon_format.types import DecodeOptions

# Strict validation (default)
try:
    decode("items[5]: a,b,c", DecodeOptions(strict=True))
except ToonDecodeError as e:
    print(f"Error: {e}")

# Lenient parsing
result = decode("items[5]: a,b,c", DecodeOptions(strict=False))
```

--------------------------------

### Debug and Validate TOON Output

Source: https://github.com/toon-format/toon-python/blob/main/docs/llm-integration.md

Provides utility snippets for logging raw TOON output, toggling strict validation modes, and asserting data structure integrity.

```python
# Log raw output
print("Raw TOON from model:")
print(repr(toon_str))

try:
    data = decode(toon_str)
except ToonDecodeError as e:
    print(f"Decode error: {e}")

# Strict vs Lenient mode
decode(toon_str, {"strict": True})
decode(toon_str, {"strict": False})

# Validate structure
data = decode(toon_str)
assert "users" in data
assert isinstance(data["users"], list)
```

--------------------------------

### Estimate Savings: JSON vs. TOON Token Counts

Source: https://context7.com/toon-format/toon-python/llms.txt

Calculates and compares the token counts between JSON and TOON formats for given data structures using the `estimate_savings` function. This helps in understanding the efficiency gains of using TOON, especially for API usage and cost estimation.

```python
from toon_format import estimate_savings

# Measure savings for tabular data (highest efficiency)
user_data = {
    "users": [
        {"id": 1, "name": "Alice", "email": "alice@example.com", "active": True},
        {"id": 2, "name": "Bob", "email": "bob@example.com", "active": True},
        {"id": 3, "name": "Charlie", "email": "charlie@example.com", "active": False}
    ]
}

result = estimate_savings(user_data)
print(f"JSON tokens: {result['json_tokens']}")
print(f"TOON tokens: {result['toon_tokens']}")
print(f"Absolute savings: {result['savings']} tokens")
print(f"Percentage savings: {result['savings_percent']:.1f}%")

# Cost estimation for API usage
api_data = [{"id": i, "value": f"Item {i}", "score": i * 10} for i in range(1, 51)]
result = estimate_savings(api_data)

cost_per_1k = 0.01  # Example: $0.01 per 1K tokens
json_cost = (result['json_tokens'] / 1000) * cost_per_1k
toon_cost = (result['toon_tokens'] / 1000) * cost_per_1k

print(f"JSON cost per request: ${json_cost:.4f}")
print(f"TOON cost per request: ${toon_cost:.4f}")
print(f"Savings per 10,000 requests: ${(json_cost - toon_cost) * 10000:.2f}")
```

--------------------------------

### estimate_savings

Source: https://context7.com/toon-format/toon-python/llms.txt

Compares token counts between JSON and TOON formats to measure efficiency gains. Requires the tiktoken library.

```APIDOC
## estimate_savings

### Description
Compares token counts between JSON and TOON formats to measure efficiency gains. Requires the `tiktoken` library for token counting.

### Usage

**Measure savings for tabular data:**
```python
from toon_format import estimate_savings

user_data = {
    "users": [
        {"id": 1, "name": "Alice", "email": "alice@example.com", "active": True},
        {"id": 2, "name": "Bob", "email": "bob@example.com", "active": True},
        {"id": 3, "name": "Charlie", "email": "charlie@example.com", "active": False}
    ]
}

result = estimate_savings(user_data)
print(f"JSON tokens: {result['json_tokens']}")
print(f"TOON tokens: {result['toon_tokens']}")
print(f"Absolute savings: {result['savings']} tokens")
print(f"Percentage savings: {result['savings_percent']:.1f}%")
```

**Cost estimation for API usage:**
```python
from toon_format import estimate_savings

api_data = [{"id": i, "value": f"Item {i}", "score": i * 10} for i in range(1, 51)]
result = estimate_savings(api_data)

cost_per_1k = 0.01  # Example: $0.01 per 1K tokens
json_cost = (result['json_tokens'] / 1000) * cost_per_1k
toon_cost = (result['toon_tokens'] / 1000) * cost_per_1k

print(f"JSON cost per request: ${json_cost:.4f}")
print(f"TOON cost per request: ${toon_cost:.4f}")
print(f"Savings per 10,000 requests: ${(json_cost - toon_cost) * 10000:.2f}")
```

### Output for Tabular Data Example
```
JSON tokens: 112
TOON tokens: 68
Absolute savings: 44 tokens
Percentage savings: 39.3%
```
```

--------------------------------

### POST /encode

Source: https://context7.com/toon-format/toon-python/llms.txt

Encodes Python objects into the TOON string format using specified delimiter, indentation, and length marker options.

```APIDOC
## POST /encode

### Description
Converts a Python dictionary or list into a TOON-formatted string. Supports customization of delimiters, indentation levels, and optional length markers for validation.

### Method
POST

### Endpoint
/encode

### Parameters
#### Request Body
- **data** (object/array) - Required - The data structure to encode.
- **options** (object) - Optional - Configuration object containing:
  - **delimiter** (string) - The character used to separate values in tabular arrays.
  - **indent** (integer) - Number of spaces for indentation.
  - **lengthMarker** (string) - Character used to mark array lengths.

### Request Example
{
  "data": [{"id": 1, "name": "Alice"}],
  "options": {"delimiter": ",", "indent": 2}
}

### Response
#### Success Response (200)
- **result** (string) - The encoded TOON string.

#### Response Example
{
  "result": "[1,]{id,name}:\n  1,Alice"
}
```

--------------------------------

### EncodeOptions

Source: https://context7.com/toon-format/toon-python/llms.txt

Defines the type for encoding configuration, controlling delimiter, indentation, and length marker settings.

```APIDOC
## EncodeOptions

### Description
`EncodeOptions` is a `TypedDict` used for configuring the encoding process. It allows control over delimiter, indentation, and length marker settings.

### Usage

```python
from toon_format import encode
from toon_format.types import EncodeOptions

# Example usage with custom options
options: EncodeOptions = {
    "delimiter": "::",
    "indent": 4,
    "use_length_prefix": True
}

data = {"key": "value", "list": [1, 2, 3]}
encoded_data = encode(data, options=options)
print(encoded_data)
```

### Example Output with Custom Options
```
key::value
list::[
    1,
    2,
    3
]
```
```

--------------------------------

### Estimate Token Savings

Source: https://github.com/toon-format/toon-python/blob/main/docs/api.md

Compares the token usage between standard JSON and the TOON format. Returns a dictionary containing token counts and percentage savings, helping developers optimize data structures.

```python
from toon_format import estimate_savings

data = {
    "employees": [
        {"id": 1, "name": "Alice"},
        {"id": 2, "name": "Bob"}
    ]
}

result = estimate_savings(data)
print(f"JSON tokens: {result['json_tokens']}")
print(f"TOON tokens: {result['toon_tokens']}")
print(f"Savings: {result['savings_percent']:.1f}%")
```

--------------------------------

### Boolean Representation in Python and TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Shows the conversion of Python's boolean values (True, False) to their TOON equivalents (true, false). TOON uses lowercase for boolean literals.

```python
True   # true in TOON (lowercase)
False  # false in TOON (lowercase)
```

```toon
true
false
```

--------------------------------

### Token Efficiency Comparison: JSON vs TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

A direct comparison of a JSON object and its TOON equivalent, illustrating the significant reduction in character count achieved by TOON. This highlights TOON's design for token efficiency.

```json
{"users":[{"id":1,"name":"Alice","age":30,"active":true},{"id":2,"name":"Bob","age":25,"active":true},{"id":3,"name":"Charlie","age":35,"active":false}]}
```

```toon
users[3,]{id,name,age,active}:
  1,Alice,30,true
  2,Bob,25,true
  3,Charlie,35,false
```

--------------------------------

### Blank Line Handling in TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Details the rules for blank lines in TOON. Blank lines are not allowed within arrays in strict mode but are permitted and ignored between top-level keys.

```toon
# ❌ Invalid (blank line in array)
items[3]:
  - a

  - b
  - c
```

```toon
# ✅ Valid (no blank lines)
items[3]:
  - a
  - b
  - c
```

```toon
# ✅ Valid (blank lines between objects)
name: Alice

age: 30
```

--------------------------------

### Null Representation in Python and TOON

Source: https://github.com/toon-format/toon-python/blob/main/docs/format.md

Illustrates how Python's None value is represented as null in the TOON format. TOON uses lowercase for the null literal.

```python
None  # null in TOON (lowercase)
```

```toon
null
```