### Initialize and Use JinaEmbeddings

Source: https://docs.chonkie.ai/oss/embeddings/jina-embeddings

Initialize JinaEmbeddings and the SemanticChunker. This example shows the basic setup for chunking text.

```python
from chonkie import JinaEmbeddings

# Initialize the Jina embeddings
embeddings = JinaEmbeddings()

# Initialize the semantic chunker
chunker = SemanticChunker(embeddings)

# Chunk the text
text = ... # Your text string

# CHONK!
chunks = chunker(text)
```

--------------------------------

### Python Installation with All Features

Source: https://docs.chonkie.ai/oss/installation

Installs Chonkie with all available features.

```bash
pip install "chonkie[all]"
```

--------------------------------

### Pipeline Validation Examples

Source: https://docs.chonkie.ai/oss/pipelines

Demonstrates valid and invalid pipeline configurations based on chunker presence, input source, and chef count. Use these examples to ensure your pipeline setup is correct before execution.

```python
# ❌ Invalid - no chunker
Pipeline().fetch_from("file", path="doc.txt").run()

# ❌ Invalid - multiple chefs
Pipeline()
    .process_with("text")
    .process_with("markdown")  # Error!
    .chunk_with("recursive")

# ✅ Valid - has chunker and input source
Pipeline()
    .fetch_from("file", path="doc.txt")
    .chunk_with("recursive", chunk_size=512)
    .run()

# ✅ Valid - text input, no fetcher needed
Pipeline()
    .chunk_with("recursive")
    .run(texts="Hello world")
```

--------------------------------

### Install Chonkie with Qdrant Support

Source: https://docs.chonkie.ai/oss/handshakes/qdrant-handshake

Install the necessary dependencies for using Qdrant with Chonkie.

```bash
pip install chonkie[qdrant]
```

--------------------------------

### Install MistralOCR Chef

Source: https://docs.chonkie.ai/oss/chefs/mistral-ocr

Install the MistralOCR chef with the necessary dependencies.

```bash
pip install chonkie[mistral]
```

--------------------------------

### Install Chonkie with Viz Support

Source: https://docs.chonkie.ai/oss/utils/visualizer

Install the 'chonkie' library with the 'viz' optional dependency to enable visualization features.

```bash
pip install "chonkie[viz]"
```

--------------------------------

### Install Chonkie with Hub Support

Source: https://docs.chonkie.ai/oss/utils/hubbie

Install the 'hub' optional dependency for Chonkie to use the Hubbie utility.

```bash
pip install "chonkie[hub]"
```

--------------------------------

### Install GroqGenie Support

Source: https://docs.chonkie.ai/oss/changelog

Install Chonkie with Groq support to use the GroqGenie. This enables fast inference on Groq hardware.

```bash
pip install "chonkie[groq]"
```

--------------------------------

### Install Chonkie with uv

Source: https://docs.chonkie.ai/oss/quick-start

Install the core Chonkie package using uv. This is an alternative package installer for Python.

```bash
uv add chonkie
```

--------------------------------

### Install CodeChunker with Dependencies

Source: https://docs.chonkie.ai/oss/chunkers/code-chunker

Install the CodeChunker along with its necessary code parsing dependencies using pip.

```bash
pip install "chonkie[code]"
```

--------------------------------

### Install Chonkie Fetchers

Source: https://docs.chonkie.ai/oss/fetchers/overview

Install the Chonkie package, which includes fetchers, using pip.

```bash
pip install chonkie
```

--------------------------------

### Install Chonkie Cloud API (JavaScript with bun)

Source: https://docs.chonkie.ai/oss/installation

Installs the Chonkie JavaScript cloud package for API access using bun.

```bash
bun add @chonkiejs/cloud
```

--------------------------------

### Migrate to SemanticChunker

Source: https://docs.chonkie.ai/oss/chunkers/sdpm-chunker

Example showing how to migrate from the legacy SDPMChunker to the recommended SemanticChunker, highlighting parameter differences and renamings.

```python
from chonkie import SemanticChunker

chunker = SemanticChunker(
    embedding_model="minishlab/potion-base-32M",
    threshold=0.7,  # Explicit threshold instead of "auto"
    chunk_size=512,
    min_sentences_per_chunk=1,  # Renamed parameter
    skip_window=1  # Same functionality
)

chunks = chunker.chunk(text)
for chunk in chunks:
    print(f"Token count: {chunk.token_count}")
```

--------------------------------

### Initialize Model2VecEmbeddings

Source: https://docs.chonkie.ai/oss/embeddings/model2vec-embeddings

Instantiate the Model2VecEmbeddings class. Ensure the model2vec library is installed.

```python
from chonkie import Model2VecEmbeddings

embeddings = Model2VecEmbeddings()
```

--------------------------------

### Quick Start Docker Compose

Source: https://docs.chonkie.ai/oss/api/docker

Use this command to quickly start the Chonkie API service using docker-compose. The API will be accessible at http://localhost:8000.

```bash
docker compose up
```

--------------------------------

### Initialize and Use Legacy SDPMChunker

Source: https://docs.chonkie.ai/oss/chunkers/sdpm-chunker

Example of initializing the legacy SDPMChunker with all original parameters and using its chunking methods.

```python
from chonkie.legacy import SDPMChunker

# All original parameters still work
chunker = SDPMChunker(
    embedding_model="minishlab/potion-base-32M",
    mode="window",
    threshold="auto",
    chunk_size=2048,
    similarity_window=1,
    min_sentences=1,
    min_chunk_size=2,
    min_characters_per_sentence=12,
    threshold_step=0.01,
    delim=[. , ! , ? , \n],
    include_delim="prev",
    skip_window=1
)

# Original methods preserved
chunks = chunker.chunk(text)
batch_chunks = chunker.chunk_batch(texts)
```

--------------------------------

### Install Chonkie Tokenizer (JavaScript with npm)

Source: https://docs.chonkie.ai/oss/installation

Installs the Chonkie JavaScript tokenizer package for custom tokenizers using npm.

```bash
npm install @chonkiejs/token
```

--------------------------------

### Install Chonkie Tokenizer (JavaScript with bun)

Source: https://docs.chonkie.ai/oss/installation

Installs the Chonkie JavaScript tokenizer package for custom tokenizers using bun.

```bash
bun add @chonkiejs/token
```

--------------------------------

### Install Chonkie Cloud API (JavaScript with npm)

Source: https://docs.chonkie.ai/oss/installation

Installs the Chonkie JavaScript cloud package for API access using npm.

```bash
npm install @chonkiejs/cloud
```

--------------------------------

### Python Installation with Cohere Embeddings Support

Source: https://docs.chonkie.ai/oss/installation

Installs Chonkie with support for Cohere embeddings.

```bash
pip install "chonkie[cohere]"
```

--------------------------------

### Install LiteParse

Source: https://docs.chonkie.ai/oss/chefs/liteparse

Install the LiteParse package with optional dependencies for full functionality.

```bash
pip install chonkie[liteparse]
```

--------------------------------

### Install Chonkie with Milvus Support

Source: https://docs.chonkie.ai/oss/handshakes/milvus-handshake

Install the Chonkie library with the necessary dependencies for Milvus integration using pip.

```bash
pip install chonkie[milvus]
```

--------------------------------

### Install MongoDB Dependencies

Source: https://docs.chonkie.ai/oss/handshakes/mongodb-handshake

Install the Chonkie SDK with MongoDB support using pip.

```bash
pip install chonkie[mongodb]
```

--------------------------------

### Install TableChef with Pandas Support

Source: https://docs.chonkie.ai/oss/chefs/tablechef

Install the TableChef package with the necessary dependencies for handling CSV and Excel files. This command installs the 'table' extra.

```bash
pip install "chonkie[table]"
```

--------------------------------

### Legacy SDPMChunker Usage Example

Source: https://docs.chonkie.ai/oss/chunkers/sdpm-chunker

Example demonstrating the usage of the legacy SDPMChunker with its specific parameters and outputting sentence counts per chunk.

```python
from chonkie.legacy import SDPMChunker

chunker = SDPMChunker(
    embedding_model="minishlab/potion-base-32M",
    mode="window",
    threshold="auto",
    chunk_size=512,
    min_sentences=1,
    min_chunk_size=2,
    skip_window=1
)

chunks = chunker.chunk(text)
for chunk in chunks:
    print(f"Sentences: {len(chunk.sentences)}")
```

--------------------------------

### Start Chonkie API Server

Source: https://docs.chonkie.ai/oss/api/quickstart

Start the Chonkie API server using the default configuration. The server will be accessible at http://0.0.0.0:8000.

```bash
chonkie serve
# 🦛 Starting Chonkie API server on http://0.0.0.0:8000
# 📚 API docs available at http://0.0.0.0:8000/docs
# 🔍 Log level: info
#
# Press CTRL+C to stop the server
```

--------------------------------

### Install Legacy SDPMChunker

Source: https://docs.chonkie.ai/oss/chunkers/sdpm-chunker

Install the legacy version of the SDPMChunker using pip. This command installs the 'semantic' extra, which includes the legacy module.

```bash
pip install "chonkie[semantic]"
```

--------------------------------

### Python Installation with OpenAI Embeddings Support

Source: https://docs.chonkie.ai/oss/installation

Installs Chonkie with support for OpenAI embeddings.

```bash
pip install "chonkie[openai]"
```

--------------------------------

### Install Chonkie Cloud API (JavaScript with yarn)

Source: https://docs.chonkie.ai/oss/installation

Installs the Chonkie JavaScript cloud package for API access using yarn.

```bash
yarn add @chonkiejs/cloud
```

--------------------------------

### Install Chonkie with Pinecone support

Source: https://docs.chonkie.ai/oss/handshakes/pinecone-handshake

Install the Chonkie library with the necessary dependencies for Pinecone integration using pip.

```bash
pip install chonkie[pinecone]
```

--------------------------------

### Install LateChunker with Sentence Transformers

Source: https://docs.chonkie.ai/oss/chunkers/late-chunker

Install the LateChunker along with the necessary sentence-transformers library. This is required for the LateChunker to function.

```bash
pip install "chonkie[st]"
```

--------------------------------

### Python Installation with Multiple Features

Source: https://docs.chonkie.ai/oss/installation

Installs Chonkie with a combination of specified features, such as SentenceTransformer, CodeChunker, and Genie.

```bash
pip install "chonkie[st, code, genie]"
```

--------------------------------

### Start Chonkie API Server with Debug Logging

Source: https://docs.chonkie.ai/oss/api/quickstart

Start the Chonkie API server with debug logging enabled for more detailed output.

```bash
chonkie serve --log-level debug
```

--------------------------------

### Basic Async Chunking Example

Source: https://docs.chonkie.ai/oss/chunkers/overview

Shows how to initialize a chunker and use its asynchronous methods to chunk a single text and a batch of texts.

```python
import asyncio
from chonkie import RecursiveChunker

async def main():
    chunker = RecursiveChunker(chunk_size=512)

    chunks = await chunker.achunk("Your document text here...")
    all_chunks = await chunker.achunk_batch([
        "First document...",
        "Second document...",
        "Third document...",
    ])

asyncio.run(main())
```

--------------------------------

### Install Weaviate Dependencies

Source: https://docs.chonkie.ai/oss/handshakes/weaviate-handshake

Install the Weaviate extra dependencies for Chonkie using pip.

```bash
pip install chonkie[weaviate]
```

--------------------------------

### Install Chonkie Tokenizer (JavaScript with yarn)

Source: https://docs.chonkie.ai/oss/installation

Installs the Chonkie JavaScript tokenizer package for custom tokenizers using yarn.

```bash
yarn add @chonkiejs/token
```

--------------------------------

### Example Azure OpenAI Embeddings Initialization

Source: https://docs.chonkie.ai/oss/embeddings/azure-embeddings

A concrete example of initializing the AzureOpenAIEmbeddings class with specific Azure OpenAI service parameters.

```python
embeddings = AzureOpenAIEmbeddings(
	azure_endpoint="https://my-resource.openai.azure.com/",
	azure_api_key="my-key",
	model="text-embedding-3-small",
	deployment="embedding-deployment"
)
```

--------------------------------

### Install Chonkie Cloud API (JavaScript with pnpm)

Source: https://docs.chonkie.ai/oss/installation

Installs the Chonkie JavaScript cloud package for API access using pnpm.

```bash
pnpm add @chonkiejs/cloud
```

--------------------------------

### Install Chonkie Tokenizer (JavaScript with pnpm)

Source: https://docs.chonkie.ai/oss/installation

Installs the Chonkie JavaScript tokenizer package for custom tokenizers using pnpm.

```bash
pnpm add @chonkiejs/token
```

--------------------------------

### Install Turbopuffer Handshake Dependencies

Source: https://docs.chonkie.ai/oss/handshakes/turbopuffer-handshake

Install the necessary Python package for Turbopuffer integration.

```bash
pip install chonkie[turbopuffer]
```

--------------------------------

### Install DatasetsPorter

Source: https://docs.chonkie.ai/oss/porters/datasets-porter

Install the DatasetsPorter and its dependencies using pip. This command includes the 'datasets' library required for Hugging Face Dataset integration.

```bash
pip install "chonkie[datasets]"
```

--------------------------------

### Install LanceDB Handshake Dependencies

Source: https://docs.chonkie.ai/oss/handshakes/lancedb-handshake

Install the necessary dependencies for using the LanceDB handshake with Chonkie.

```bash
pip install chonkie[lancedb]
```

--------------------------------

### Chonkie CLI: Multiple Key-Value Parameters

Source: https://docs.chonkie.ai/oss/experimental/chonkie-cli

Configure multiple parameters for a component by repeating the `*_params` option. This example sets the tokenizer and minimum characters per chunk.

```bash
--chunker-params tokenizer=gpt2 --chunker-params min_characters_per_chunk=50
```

--------------------------------

### Install Chonkie API

Source: https://docs.chonkie.ai/oss/api/quickstart

Install the Chonkie API package with necessary extras for API, semantic chunking, and code chunking. The 'openai' extra is for API-based embedding providers.

```bash
pip install "chonkie[api,semantic,code,openai]"
```

--------------------------------

### Install Chonkie JS with bun

Source: https://docs.chonkie.ai/oss/quick-start

Install the core Chonkie JavaScript package using bun. This is a fast, all-in-one JavaScript runtime.

```bash
bun add @chonkiejs/core
```

--------------------------------

### Initialize MarkdownChef with Default Tokenizer

Source: https://docs.chonkie.ai/oss/chefs/markdownchef

Basic initialization of MarkdownChef using the default tokenizer. No additional setup is required.

```python
from chonkie import MarkdownChef

# Basic initialization with default tokenizer
chef = MarkdownChef()
```

--------------------------------

### Install VoyageAI Embeddings

Source: https://docs.chonkie.ai/oss/embeddings/voyageai-embeddings

Install the necessary libraries for VoyageAI embeddings. This includes chonkie with voyageai support.

```bash
pip install "chonkie[voyageai]"
```

--------------------------------

### Install CerebrasGenie Support

Source: https://docs.chonkie.ai/oss/changelog

Install Chonkie with Cerebras support to use the CerebrasGenie. This enables ultra-fast LLM inference on Cerebras hardware.

```bash
pip install "chonkie[cerebras]"
```

--------------------------------

### Start Chonkie API Server Directly with Uvicorn

Source: https://docs.chonkie.ai/oss/api/quickstart

Start the Chonkie API server directly using Uvicorn, specifying the host and port.

```bash
uvicorn chonkie.api.main:app --host 0.0.0.0 --port 8000
```

--------------------------------

### Initialize SentenceTransformerEmbeddings

Source: https://docs.chonkie.ai/oss/embeddings/sentence-transformer-embeddings

Instantiate the SentenceTransformerEmbeddings class with a model name. Ensure the 'sentence-transformers' library is installed.

```python
from chonkie import SentenceTransformerEmbeddings

embeddings = SentenceTransformerEmbeddings("all-MiniLM-L6-v2")
```

--------------------------------

### Install Pgvector Handshake

Source: https://docs.chonkie.ai/oss/handshakes/pgvector-handshake

Install the necessary Python package for Pgvector integration. Ensure you have PostgreSQL with the pgvector extension enabled.

```bash
pip install chonkie[pgvector]
```

--------------------------------

### Response Format Example

Source: https://docs.chonkie.ai/oss/api/endpoints

All chunking endpoints return a list of chunk objects, each containing text, start index, end index, and token count.

```json
[
  {
    "text": "chunk content",
    "start_index": 0,
    "end_index": 42,
    "token_count": 8
  }
]
```

--------------------------------

### Install CodeChunker Dependencies

Source: https://docs.chonkie.ai/oss/experimental/code-chunker

Install the necessary Python package to use the experimental CodeChunker. This command includes the 'code' extra, which provides the required dependencies.

```bash
pip install chonkie[code]
```

--------------------------------

### Get Embeddings Handler for Different Models

Source: https://docs.chonkie.ai/oss/embeddings/auto-embeddings

Load the embeddings handler for SentenceTransformer, OpenAI, or Model2Vec by providing the model name to AutoEmbeddings.get_embeddings(). Ensure the appropriate libraries are installed for each model.

```python
from chonkie import AutoEmbeddings

# Get the embeddings handler for SentenceTransformer
embeddings = AutoEmbeddings.get_embeddings("all-MiniLM-L6-v2")

# Get the embeddings handler for OpenAI
embeddings = AutoEmbeddings.get_embeddings("text-embedding-3-large")

# Get the embeddings handler for Model2Vec
embeddings = AutoEmbeddings.get_embeddings("minishlab/potion-base-32M")

```

--------------------------------

### JavaScript: FastChunker with Pattern-Based Splitting

Source: https://docs.chonkie.ai/oss/chunkers/fast-chunker

Initialize the FastChunker in JavaScript using a pattern for splitting, similar to the Python example. The `prefix: true` option retains the pattern at the start of subsequent chunks.

```javascript
chunker = await FastChunker.create({
  chunkSize: 4096,
  pattern: "▁",         # Metaspace character
  prefix: true,         # Keep pattern at start of next chunk
});
```

--------------------------------

### JavaScript: FastChunker for Paragraph Splitting

Source: https://docs.chonkie.ai/oss/chunkers/fast-chunker

Configure the FastChunker in JavaScript to split text at paragraph boundaries using double newlines. This asynchronous setup mirrors the Python paragraph splitting example.

```javascript
chunker = await FastChunker.create({
  chunkSize: 8192,
  delimiters: "\n\n",
});
```

--------------------------------

### Initialize OpenAIEmbeddings

Source: https://docs.chonkie.ai/oss/embeddings/openai-embeddings

Instantiate the OpenAIEmbeddings class to begin generating embeddings. Ensure the openai library is installed.

```python
from chonkie import OpenAIEmbeddings

embeddings = OpenAIEmbeddings()
```

--------------------------------

### Initialize PineconeHandshake with API Key

Source: https://docs.chonkie.ai/oss/handshakes/pinecone-handshake

Initialize the PineconeHandshake by providing your Pinecone API key. This is the most basic setup.

```python
from chonkie import PineconeHandshake
handshake = PineconeHandshake(api_key="YOUR_API_KEY")
```

--------------------------------

### Install Gemini Embeddings

Source: https://docs.chonkie.ai/oss/embeddings/gemini-embeddings

Install the necessary libraries for Gemini embeddings using pip. This command installs the chonkie package with Gemini support.

```bash
pip install "chonkie[gemini]"
```

--------------------------------

### Install TeraflopAI Chunker

Source: https://docs.chonkie.ai/oss/chunkers/teraflopai-chunker

Install the TeraflopAI Chunker along with its dependencies using pip. This command installs the 'chonkie' package with the 'teraflopai' extra.

```bash
pip install "chonkie[teraflopai]"
```

--------------------------------

### Initialize QdrantHandshake

Source: https://docs.chonkie.ai/oss/handshakes/qdrant-handshake

Initialize the QdrantHandshake with a Qdrant URL.

```python
from chonkie import QdrantHandshake
handshake = QdrantHandshake(url="http://localhost:6333")
```

--------------------------------

### Pipeline Error Handling Example

Source: https://docs.chonkie.ai/oss/pipelines

Shows how to handle potential exceptions during pipeline execution, such as FileNotFoundError, ValueError for configuration issues, and RuntimeError for general execution failures.

```python
from pathlib import Path

try:
    doc = Pipeline()
        .fetch_from("file", path="missing.txt")
        .chunk_with("recursive")
        .run()
except FileNotFoundError as e:
    print(f"File not found: {e}")
except ValueError as e:
    print(f"Configuration error: {e}")
except RuntimeError as e:
    print(f"Pipeline execution failed: {e}")
```

--------------------------------

### Load and Run Pre-configured Pipelines

Source: https://docs.chonkie.ai/oss/pipelines

Load a pre-configured pipeline recipe from the Chonkie Hub or a local file. Use this to quickly set up common processing tasks like markdown parsing.

```python
from chonkie.pipelines import Pipeline

# Load markdown processing recipe
pipeline = Pipeline.from_recipe("markdown")

# Run with your content
doc = pipeline.run(texts="# My Markdown\n\nContent here")

# Load custom local recipe
pipeline = Pipeline.from_recipe("custom", path="./my_recipe.json")
```

--------------------------------

### Install Semantic Chunker (JavaScript)

Source: https://docs.chonkie.ai/oss/chunkers/semantic-chunker

Install the core Chonkie package for JavaScript to use the Semantic Chunker.

```bash
npm install @chonkiejs/core
```

--------------------------------

### Initialize MistralOCR Chef

Source: https://docs.chonkie.ai/oss/chefs/mistral-ocr

Initialize the MistralOCR chef using default settings or with a custom model and API key.

```python
from chonkie import MistralOCR

# Default initialization (uses MISTRAL_API_KEY env var)
ocr = MistralOCR()

# Custom model and explicit API key
ocr = MistralOCR(model="mistral-ocr-2505", api_key="sk-...")
```

--------------------------------

### Initialize and Chunk Text in Python

Source: https://docs.chonkie.ai/oss/chunkers/fast-chunker

Demonstrates how to initialize the FastChunker with custom chunk size and delimiters, and then use it to chunk a given text. It also shows how to access information about each generated chunk.

```python
from chonkie import FastChunker

# Initialize the chunker
chunker = FastChunker(
    chunk_size=1024,
    delimiters=". \n",
)

# Chunk your text
text = "Your long document text here..."
chunks = chunker.chunk(text)

# Access chunk information
for chunk in chunks:
    print(f"Chunk: {chunk.text[:50]}...")
    print(f"Bytes: {len(chunk.text)}")
    print(f"Position: {chunk.start_index}-{chunk.end_index}")
```

--------------------------------

### Initialize QdrantHandshake for Qdrant Cloud

Source: https://docs.chonkie.ai/oss/handshakes/qdrant-handshake

Initialize the QdrantHandshake for Qdrant Cloud, providing the cloud URL and API key.

```python
from qdrant_client import QdrantClient
handshake = QdrantHandshake(
    url="YOUR_CLOUD_URL",
    api_key="YOUR_API_KEY",
)
```

--------------------------------

### Initialize WeaviateHandshake with Server Connection

Source: https://docs.chonkie.ai/oss/handshakes/weaviate-handshake

Initialize the WeaviateHandshake by providing the URL and API key for a Weaviate server.

```python
from chonkie import WeaviateHandshake

# Or connect to a Weaviate server
handshake = WeaviateHandshake(url="http://localhost:8080", api_key= "YOUR_API_KEY")
```

--------------------------------

### Initialize and Use GroqGenie

Source: https://docs.chonkie.ai/oss/changelog

Initialize GroqGenie with a specified model for text generation. This Genie provides fast inference on Groq hardware.

```python
from chonkie import GroqGenie

genie = GroqGenie(model="llama-3.3-70b-versatile")
response = genie.generate("Hello!")
```

--------------------------------

### Initialize the Visualizer

Source: https://docs.chonkie.ai/oss/utils/visualizer

Import and initialize the Visualizer class from the chonkie.utils module to prepare for visualization.

```python
# Import the Visualizer
from chonkie import Visualizer

# Initialize the Visualizer
viz = Visualizer()
```

--------------------------------

### Install Jina Embeddings Support

Source: https://docs.chonkie.ai/oss/changelog

Install Chonkie with support for JinaAI embeddings. This allows the use of JinaEmbeddings with SemanticChunker and SDPMChunker.

```bash
pip install "chonkie[jina]"
```

--------------------------------

### Initialize TeraflopAIChunker with External Client

Source: https://docs.chonkie.ai/oss/chunkers/teraflopai-chunker

Initialize the TeraflopAIChunker using an existing TeraflopAI client instance. This allows for pre-configured clients to be passed directly, ignoring separate url and api_key parameters.

```python
from teraflopai import TeraflopAI

client = TeraflopAI(
    url="https://api.segmentation.teraflopai.com/v1/segmentation/free",
    api_key="your_api_key_here",
)
chunker = TeraflopAIChunker(client=client)
```

--------------------------------

### Chonkie Pipeline Command Help

Source: https://docs.chonkie.ai/oss/experimental/chonkie-cli

Displays help information for the 'chonkie pipeline' command, outlining its usage, arguments, and available options for running processing pipelines.

```bash
chonkie pipeline --help
```

--------------------------------

### Install Chonkie JS with pnpm

Source: https://docs.chonkie.ai/oss/quick-start

Install the core Chonkie JavaScript package using pnpm. This is an alternative package manager for Node.js.

```bash
pnpm add @chonkiejs/core
```

--------------------------------

### Initialize WeaviateHandshake for Weaviate Cloud

Source: https://docs.chonkie.ai/oss/handshakes/weaviate-handshake

Initialize the WeaviateHandshake for Weaviate Cloud by providing the cloud URL and API key.

```python
from chonkie import WeaviateHandshake

handshake = WeaviateHandshake(
    url="YOUR_CLOUD_URL",
    api_key="YOUR_API_KEY"
)
```

--------------------------------

### Example Usage of VoyageAI Embeddings

Source: https://docs.chonkie.ai/oss/embeddings/voyageai-embeddings

A concise example demonstrating the initialization and usage of VoyageAI embeddings for embedding single texts or batches.

```python
embeddings = VoyageAIEmbeddings()
vectors = embeddings.embed("your text here")
# or you can
vectors = embeddings.embed_batch(["text1", "text2"])
```

--------------------------------

### Initialize and Chunk Text in JavaScript

Source: https://docs.chonkie.ai/oss/chunkers/fast-chunker

Shows how to initialize the FastChunker in JavaScript using `FastChunker.create` with specified chunk size and delimiters, and then chunk a text. It also illustrates accessing details of each chunk.

```javascript
import { FastChunker } from "@chonkiejs/core";

// Initialize the chunker
const chunker = await FastChunker.create({
  chunkSize: 1024,
  delimiters: ". \n",
});

// Chunk your text
const text = "Your long document text here...";
const chunks = await chunker.chunk(text);

// Access chunk information
for (const chunk of chunks) {
  console.log(`Chunk: ${chunk.text.slice(0, 50)}...`);
  console.log(`Bytes: ${chunk.text.length}`);
  console.log(`Position: ${chunk.startIndex}-${chunk.endIndex}`);
}
```

--------------------------------

### Install Chonkie JS with yarn

Source: https://docs.chonkie.ai/oss/quick-start

Install the core Chonkie JavaScript package using yarn. This is another popular package manager for Node.js.

```bash
yarn add @chonkiejs/core
```

--------------------------------

### Initialize AutoTikTokenizer

Source: https://docs.chonkie.ai/oss/chunkers/sentence-chunker

An alternative tokenizer backend. Requires the 'autotiktokenizer' library.

```python
from autotiktokenizer import AutoTikTokenizer
tokenizer = AutoTikTokenizer.from_pretrained("gpt2")
```

--------------------------------

### Install Neural Chunker

Source: https://docs.chonkie.ai/oss/chunkers/neural-chunker

Install the NeuralChunker with its specific dependencies using pip. This command includes the necessary packages for the deep learning model.

```bash
pip install "chonkie[neural]"
```

--------------------------------

### Initialize NeuralChunker with Default Parameters

Source: https://docs.chonkie.ai/oss/chunkers/neural-chunker

Instantiate the NeuralChunker using default model and device settings. The default model is 'mirth/chonky_modernbert_base_1' and it runs on the CPU.

```python
from chonkie import NeuralChunker

# Basic initialization with default parameters
chunker = NeuralChunker(
    model="mirth/chonky_modernbert_base_1",  # Default model
    device_map="cpu",                        # Device to run the model on ('cpu', 'cuda', etc.)
    min_characters_per_chunk=10,             # Minimum characters for a chunk
)
```

--------------------------------

### Install Elasticsearch Handshake Dependencies

Source: https://docs.chonkie.ai/oss/handshakes/elastic-handshake

Install the necessary dependencies for the Elasticsearch handshake using pip. This command includes the 'elastic' extra for Elasticsearch support.

```bash
pip install chonkie[elastic]
```

--------------------------------

### List All Pipelines

Source: https://docs.chonkie.ai/oss/api/pipelines

Use this endpoint to retrieve a list of all available pipelines, ordered by their creation date.

```bash
curl http://localhost:8000/v1/pipelines
```

--------------------------------

### Install Chroma Handshake Dependencies

Source: https://docs.chonkie.ai/oss/handshakes/chroma-handshake

Install the necessary packages for using the Chroma handshake functionality. This includes the core chonkie library with chroma support.

```bash
pip install chonkie[chroma]
```

--------------------------------

### Start Chonkie API Server on Custom Port

Source: https://docs.chonkie.ai/oss/api/quickstart

Start the Chonkie API server on a custom port (e.g., 3000) and enable auto-reloading for development.

```bash
chonkie serve --port 3000 --reload
```

--------------------------------

### Initialize TextChef

Source: https://docs.chonkie.ai/oss/chefs/textchef

Instantiate the TextChef for processing text files. No parameters are required for simple initialization.

```python
from chonkie import TextChef

# Simple initialization - no parameters required
chef = TextChef()
```

--------------------------------

### Initialize Hugging Face Tokenizer

Source: https://docs.chonkie.ai/oss/chunkers/sentence-chunker

Supports tokenizers from the Hugging Face ecosystem. Requires the 'tokenizers' library.

```python
from tokenizers import Tokenizer
tokenizer = Tokenizer.from_pretrained("gpt2")
```

--------------------------------

### Initialize LiteParse (Custom Configuration)

Source: https://docs.chonkie.ai/oss/chefs/liteparse

Initialize LiteParse with custom parameters for OCR, page limits, resolution, and parallel processing.

```python
from chonkie import LiteParse

# Custom configuration
chef = LiteParse(
    ocr_enabled=True,
    ocr_language="eng",
    dpi=300,
    max_pages=100,
    target_pages="1-10",
    num_workers=8,
)
```

--------------------------------

### Install Slumber Chunker with Genie

Source: https://docs.chonkie.ai/oss/chunkers/slumber-chunker

Install the Slumber Chunker along with the necessary libraries for Genie integration. This command includes optional dependencies for connecting to various generative model APIs.

```bash
pip install "chonkie[genie]"
```

--------------------------------

### Handling Consecutive Delimiters in FastChunker

Source: https://docs.chonkie.ai/oss/chunkers/fast-chunker

Splits text at the start of consecutive whitespace runs when `consecutive=True`. This prevents splitting within multiple spaces and ensures chunks start at logical breaks.

```python
from chonkie import FastChunker

# Split at START of consecutive whitespace runs
chunker = FastChunker(
    chunk_size=10,
    pattern=" ",
    consecutive=True,
)

text = """First           paragraph with multiple sentences.
    This is still the first paragraph.

    Second paragraph starts here.
    More content         in the second paragraph."""  # Multiple spaces between words
chunks = chunker.chunk(text)

# Without consecutive=True: might split in middle of "   "
# With consecutive=True: splits at START of "   "
for chunk in chunks:
    print(f"Chunk: '{chunk.text}'")
```

--------------------------------

### Get a Pipeline

Source: https://docs.chonkie.ai/oss/api/pipelines

Retrieves a specific pipeline by its unique identifier.

```APIDOC
## Get a Pipeline

### Description
Retrieves a specific pipeline using its ID.

### Method
GET

### Endpoint
/v1/pipelines/{pipeline_id}

#### Path Parameters
- **pipeline_id** (string) - Required - The unique identifier of the pipeline to retrieve.
```

--------------------------------

### Initialize QdrantHandshake with Qdrant Client

Source: https://docs.chonkie.ai/oss/handshakes/qdrant-handshake

Initialize the QdrantHandshake using an existing QdrantClient instance and specifying a collection name.

```python
from qdrant_client import QdrantClient
client = QdrantClient(":memory:")
handshake = QdrantHandshake(client=client, collection_name="my_collection")
```

--------------------------------

### Search Weaviate using a Query

Source: https://docs.chonkie.ai/oss/handshakes/weaviate-handshake

Initialize the WeaviateHandshake and perform a search using a text query to find similar chunks. Results include score and text.

```python
from chonkie import WeaviateHandshake

# Initialize the handshake
handshake = WeaviateHandshake(
    url="YOUR_CLOUD_URL",
    api_key="YOUR_API_KEY",
    collection_name="my_documents"
)
results = handshake.search(query="chonk your texts", limit=2)
for result in results:
    print(result["score"], result["text"])
```