### Error Handling Example

Source: https://github.com/moeki0/baran/blob/main/README.md

Demonstrates how to handle potential runtime errors during splitter initialization.

```APIDOC
## Error Handling Example

### Description

Demonstrates how to handle potential runtime errors during splitter initialization, specifically when `chunk_overlap` is greater than or equal to `chunk_size`.

### Code Example

```ruby
begin
  # This will raise an error
  invalid_splitter = Baran::TextSplitter.new(
    chunk_size: 100,
    chunk_overlap: 100  # overlap >= chunk_size
  )
rescue RuntimeError => e
  puts "Error: #{e.message}"
  # => "Error: Cannot have chunk_overlap >= chunk_size"
end
```
```

--------------------------------

### Best Practices: Choosing the Right Splitter

Source: https://github.com/moeki0/baran/blob/main/README.md

Provides a quick guide on selecting the most suitable text splitter for different use cases.

```APIDOC
## Best Practices: Choosing the Right Splitter

### Description

A quick guide on selecting the most suitable text splitter for different use cases.

### Guidelines

- **CharacterTextSplitter**: Simple documents with consistent structure.
- **RecursiveCharacterTextSplitter**: General-purpose text splitting.
- **SentenceTextSplitter**: When sentence integrity is important.
- **MarkdownSplitter**: For Markdown documents and documentation.
```

--------------------------------

### Comparing Text Splitting Strategies

Source: https://github.com/moeki0/baran/blob/main/README.md

This example compares the output of `CharacterTextSplitter` and `RecursiveCharacterTextSplitter` when applied to the same Markdown text. It reads a sample Markdown file and then processes it with both splitters, setting the same `chunk_size` for a direct comparison of their segmentation results.

```ruby
text = File.read('sample.md')

# Character-based splitting
char_splitter = Baran::CharacterTextSplitter.new(chunk_size: 500)
char_chunks = char_splitter.chunks(text)

# Recursive splitting
recursive_splitter = Baran::RecursiveCharacterTextSplitter.new(chunk_size: 500)
recursive_chunks = recursive_splitter.chunks(text)
```

--------------------------------

### Basic Text Splitting with Baran

Source: https://github.com/moeki0/baran/blob/main/README.md

Demonstrates the fundamental usage of Baran for splitting text. It initializes a `CharacterTextSplitter` and processes a sample string, then iterates through the resulting chunks to display their text and cursor position. This is a starting point for understanding how Baran segments text.

```ruby
require 'baran'

# Basic text splitting
splitter = Baran::CharacterTextSplitter.new(chunk_size: 500, chunk_overlap: 50)
chunks = splitter.chunks("Your long text here...")

# Access chunk data
chunks.each do |chunk|
  puts "Text: #{chunk[:text]}"
  puts "Position: #{chunk[:cursor]}"
end
```

--------------------------------

### Initialize CharacterTextSplitter with Chunk Overlap

Source: https://github.com/moeki0/baran/blob/main/README.md

Illustrates initializing the CharacterTextSplitter and RecursiveCharacterTextSplitter with specific chunk sizes and overlap values. It provides examples for general documents and technical documents, suggesting overlap percentages.

```ruby
# General documents: 5-10% of chunk size
general_splitter = Baran::CharacterTextSplitter.new(
  chunk_size: 1000,
  chunk_overlap: 100  # 10%
)

# Technical documents: Higher overlap for better context
technical_splitter = Baran::RecursiveCharacterTextSplitter.new(
  chunk_size: 800,
  chunk_overlap: 150  # ~19%
)
```

--------------------------------

### Metadata Attachment with Baran

Source: https://context7.com/moeki0/baran/llms.txt

Shows how to attach custom metadata to text chunks generated by Baran splitters. The examples demonstrate adding metadata to chunks from a single file and processing multiple documents within a directory, preparing the data for insertion into a vector database.

```ruby
require 'baran'
require 'json'
require 'digest'

splitter = Baran::RecursiveCharacterTextSplitter.new(
  chunk_size: 800,
  chunk_overlap: 100
)

# Single document with metadata
file_path = "documents/research_paper.txt"
content = File.read(file_path)

chunks = splitter.chunks(content, metadata: {
  source: file_path,
  author: "Dr. Smith",
  created_at: Time.now.iso8601,
  document_type: "research",
  category: "machine_learning"
})

# Preparing chunks for vector database insertion
chunks_for_db = chunks.map do |chunk|
  {
    id: Digest::SHA256.hexdigest(chunk[:text])[0..15],
    text: chunk[:text],
    position: chunk[:cursor],
    **chunk[:metadata]
  }
end

puts JSON.pretty_generate(chunks_for_db.first)
# => {
#      "id": "a1b2c3d4e5f6g7h8",
#      "text": "Research paper content...",
#      "position": 0,
#      "source": "documents/research_paper.txt",
#      "author": "Dr. Smith",
#      "created_at": "2025-10-07T10:30:00Z",
#      "document_type": "research",
#      "category": "machine_learning"
#    }

# Batch processing multiple documents
class DocumentProcessor
  def initialize(chunk_size: 1000, chunk_overlap: 150)
    @splitter = Baran::RecursiveCharacterTextSplitter.new(
      chunk_size: chunk_size,
      chunk_overlap: chunk_overlap
    )
  end

  def process_directory(dir_path)
    all_chunks = []

    Dir.glob("#{dir_path}/**/*.{txt,md}").each do |file|
      content = File.read(file)
      file_metadata = {
        file_path: file,
        file_name: File.basename(file),
        file_size: File.size(file),
        modified_at: File.mtime(file).iso8601,
        extension: File.extname(file)
      }

      chunks = @splitter.chunks(content, metadata: file_metadata)
      all_chunks.concat(chunks)
    end

    all_chunks
  end
end

processor = DocumentProcessor.new
chunks = processor.process_directory("./knowledge_base")
puts "Processed #{chunks.length} chunks from knowledge base"
```

--------------------------------

### Handling Metadata with Splitter

Source: https://github.com/moeki0/baran/blob/main/README.md

Demonstrates advanced usage of Baran by attaching rich metadata to text chunks. This example reads a document, splits it using `RecursiveCharacterTextSplitter`, and includes source, creation time, and author information in the metadata for each chunk. It then iterates through the chunks, printing their text, cursor, and metadata.

```ruby
splitter = Baran::RecursiveCharacterTextSplitter.new
document_text = File.read('document.txt')

chunks = splitter.chunks(
  document_text,
  metadata: {
    source: 'document.txt',
    created_at: Time.now,
    author: 'Author Name'
  }
)

chunks.each do |chunk|
  puts "Text: #{chunk[:text]}"
  puts "Position: #{chunk[:cursor]}"
  puts "Source: #{chunk[:metadata][:source]}"
end
```

--------------------------------

### Markdown Splitting with Baran

Source: https://context7.com/moeki0/baran/llms.txt

Demonstrates how to use the MarkdownSplitter from the Baran gem to split Markdown text while preserving its structure. It prioritizes headers, code blocks, and horizontal rules for more meaningful chunking. The example shows iterating through generated chunks and also processing multiple Markdown files in a directory.

```ruby
require 'baran'

splitter = Baran::MarkdownSplitter.new(
  chunk_size: 500,
  chunk_overlap: 100
)

markdown = <<~MD
  # Main Title

  Introduction paragraph with important context.

  ## First Section

  Content under first section with details.

  ### Subsection

  More detailed information here.

  ```ruby
  def example
    puts "code block"
  end
  ```

  ## Second Section

  ---

  Final notes after horizontal rule.
MD

chunks = splitter.chunks(markdown, metadata: { doc: "readme.md", format: "markdown" })

chunks.each_with_index do |chunk, i|
  puts "Chunk #{i + 1} (#{chunk[:text].length} chars):"
  puts chunk[:text]
  puts "\nMetadata: #{chunk[:metadata]}"
  puts "=" * 50
end

# Priority order ensures headers stay with their content:
# 1. H1 headers (\n# )
# 2. H2 headers (\n## )
# 3. H3-H6 headers
# 4. Code blocks (```\n\n)
# 5. Horizontal rules
# 6. Paragraph breaks (\n\n)
# 7. Line breaks (\n)
# 8. Spaces and characters

# Use case: splitting documentation
docs_dir = Dir.glob("docs/**/*.md")
all_chunks = []

docs_dir.each do |file|
  content = File.read(file)
  file_chunks = splitter.chunks(
    content,
    metadata: { file: file, updated: File.mtime(file) }
  )
  all_chunks.concat(file_chunks)
end

puts "Total chunks for vector DB: #{all_chunks.length}"
```

--------------------------------

### Markdown Text Splitting

Source: https://github.com/moeki0/baran/blob/main/README.md

Introduces the `MarkdownSplitter`, which intelligently splits Markdown text by respecting its structural elements like headers, code blocks, and horizontal rules. It prioritizes splitting at specific Markdown syntax. The example shows setting chunk size, overlap, and associating metadata.

```ruby
splitter = Baran::MarkdownSplitter.new(
    chunk_size: 1500,
    chunk_overlap: 150
)
chunks = splitter.chunks(markdown_text, metadata: { format: "markdown" })
# => [{ cursor: 0, text: "# Header\n\nContent...", metadata: { format: "markdown" } }, ...]
```

--------------------------------

### Run Baran Gem Tests

Source: https://github.com/moeki0/baran/blob/main/README.md

Shows the command to run all tests for the Baran gem. This is typically executed after setting up the development environment.

```bash
bundle exec rake
```

--------------------------------

### Initialize MarkdownSplitter and Chunk Text

Source: https://github.com/moeki0/baran/blob/main/README.md

Demonstrates how to initialize the MarkdownSplitter with a specified chunk size and then use it to split a given text into markdown-aware chunks. It also prints the number of chunks generated.

```ruby
md_splitter = Baran::MarkdownSplitter.new(chunk_size: 500)
md_chunks = md_splitter.chunks(text)

puts "Character-based: #{char_chunks.length} chunks"
puts "Recursive: #{recursive_chunks.length} chunks"
puts "Markdown-aware: #{md_chunks.length} chunks"
```

--------------------------------

### Best Practices: Choosing Chunk Size

Source: https://github.com/moeki0/baran/blob/main/README.md

Provides guidance on selecting the appropriate chunk size based on the context window of different language models.

```APIDOC
## Best Practices: Choosing Chunk Size

### Description

Guidance on selecting the appropriate chunk size based on the context window of different language models.

### Code Examples

```ruby
# For GPT-3.5 (4K context window)
small_splitter = Baran::RecursiveCharacterTextSplitter.new(chunk_size: 500)

# For GPT-4 (8K context window)
medium_splitter = Baran::RecursiveCharacterTextSplitter.new(chunk_size: 1000)

# For Claude-2 (100K context window)
large_splitter = Baran::RecursiveCharacterTextSplitter.new(chunk_size: 4000)
```
```

--------------------------------

### Performance Considerations: Streaming Processing

Source: https://github.com/moeki0/baran/blob/main/README.md

Illustrates how to process large files efficiently using a streaming approach.

```APIDOC
## Performance Considerations: Streaming Processing

### Description

For large files, consider streaming processing to manage memory usage and improve performance.

### Code Example

```ruby
def process_large_file(file_path)
  splitter = Baran::RecursiveCharacterTextSplitter.new
  
  File.foreach(file_path, "\n\n") do |paragraph|
    chunks = splitter.chunks(paragraph)
    chunks.each { |chunk| yield chunk }
  end
end

process_large_file('huge_document.txt') do |chunk|
  # Process each chunk individually
  save_to_database(chunk)
end
```
```

--------------------------------

### Custom Text Splitting with Baran

Source: https://context7.com/moeki0/baran/llms.txt

Illustrates how to extend Baran's TextSplitter base class to create custom splitting strategies, such as splitting XML documents. It also shows error handling for invalid configurations (overlap >= size) and demonstrates the basic chunking process with character-based splitting.

```ruby
require 'baran'

# Creating a custom splitter for XML documents
class XMLSplitter < Baran::TextSplitter
  def splitted(text)
    # Split on XML closing tags
    splits = text.split(/<\/\w+>/).map { |s| s + "</tag>" }
    merged(splits, "")
  end
end

xml_splitter = XMLSplitter.new(chunk_size: 1000, chunk_overlap: 100)
xml_data = "<root><item>Data 1</item><item>Data 2</item></root>"
xml_chunks = xml_splitter.chunks(xml_data)

# Error handling for invalid configuration
begin
  invalid_splitter = Baran::TextSplitter.new(
    chunk_size: 100,
    chunk_overlap: 100
  )
rescue RuntimeError => e
  puts e.message
  # => "Cannot have chunk_overlap >= chunk_size"
end

# Understanding the chunking process
splitter = Baran::CharacterTextSplitter.new(
  chunk_size: 50,
  chunk_overlap: 10
)

text = "a" * 45 + "\n\n" + "b" * 45 + "\n\n" + "c" * 45
result = splitter.chunks(text)

result.each do |chunk|
  puts "Cursor: #{chunk[:cursor]}, Length: #{chunk[:text].length}"
end
# Demonstrates how overlap maintains context between chunks
```

--------------------------------

### TextSplitter (Base Class) Initialization

Source: https://github.com/moeki0/baran/blob/main/README.md

Initializes the base TextSplitter class with chunk size and overlap parameters.

```APIDOC
## TextSplitter (Base Class)

Base class for all text splitters.

### Method

`initialize(chunk_size: 1024, chunk_overlap: 64)`

### Parameters

#### Path Parameters

None

#### Query Parameters

None

#### Request Body

None

### Parameters

- **chunk_size** (Integer) - Optional - Maximum characters per chunk
- **chunk_overlap** (Integer) - Optional - Characters to overlap between chunks

### Request Example

```ruby
Baran::TextSplitter.new(chunk_size: 500, chunk_overlap: 50)
```

### Response

#### Success Response (200)

Initializes the splitter object.

#### Response Example

```json
{
  "message": "Splitter initialized successfully"
}
```
```

--------------------------------

### Initialize RecursiveCharacterTextSplitter with Different Chunk Sizes

Source: https://github.com/moeki0/baran/blob/main/README.md

Shows how to initialize the RecursiveCharacterTextSplitter with varying chunk sizes suitable for different language models like GPT-3.5, GPT-4, and Claude-2, based on their context window limitations.

```ruby
# For GPT-3.5 (4K context window)
small_splitter = Baran::RecursiveCharacterTextSplitter.new(chunk_size: 500)

# For GPT-4 (8K context window)
medium_splitter = Baran::RecursiveCharacterTextSplitter.new(chunk_size: 1000)

# For Claude-2 (100K context window)
large_splitter = Baran::RecursiveCharacterTextSplitter.new(chunk_size: 4000)
```

--------------------------------

### SentenceTextSplitter

Source: https://github.com/moeki0/baran/blob/main/README.md

Splits text at sentence boundaries using regex pattern matching.

```APIDOC
## SentenceTextSplitter

### Description

Splits text at sentence boundaries using regex pattern matching. Detects sentences ending with `.`, `!`, or `?` followed by whitespace or end of string.

### Method

`initialize(chunk_size: 1024, chunk_overlap: 64)`

### Parameters

- **chunk_size** (Integer) - Optional - Maximum characters per chunk (default: 1024).
- **chunk_overlap** (Integer) - Optional - Characters to overlap between chunks (default: 64).

### Request Example

```ruby
splitter = Baran::SentenceTextSplitter.new(chunk_size: 250)
```
```

--------------------------------

### Character Text Splitting with Custom Separator

Source: https://github.com/moeki0/baran/blob/main/README.md

Illustrates how to use the `CharacterTextSplitter` with specific parameters for `chunk_size`, `chunk_overlap`, and a custom `separator`. It also shows how to attach metadata to the generated chunks, such as the source file name. The output is an array of hashes, each containing the chunk's cursor, text, and metadata.

```ruby
splitter = Baran::CharacterTextSplitter.new(
    chunk_size: 1024,
    chunk_overlap: 64,
    separator: "\n\n"
)
chunks = splitter.chunks(text, metadata: { source: "document.txt" })
# => [{ cursor: 0, text: "...", metadata: { source: "document.txt" } }, ...]
```

--------------------------------

### Best Practices: Setting Overlap

Source: https://github.com/moeki0/baran/blob/main/README.md

Offers recommendations for setting the chunk overlap based on document type.

```APIDOC
## Best Practices: Setting Overlap

### Description

Recommendations for setting the chunk overlap based on document type.

### Code Examples

```ruby
# General documents: 5-10% of chunk size
general_splitter = Baran::CharacterTextSplitter.new(
  chunk_size: 1000,
  chunk_overlap: 100  # 10%
)

# Technical documents: Higher overlap for better context
technical_splitter = Baran::RecursiveCharacterTextSplitter.new(
  chunk_size: 800,
  chunk_overlap: 150  # ~19%
)
```
```

--------------------------------

### MarkdownSplitter API

Source: https://context7.com/moeki0/baran/llms.txt

Splits Markdown documents while preserving their structural elements like headings, lists, and code blocks.

```APIDOC
## MarkdownSplitter - Structure-preserving Markdown document splitting

### Description
Splits Markdown documents by respecting structural elements such as headings, lists, and code blocks, ensuring semantic coherence within chunks.

### Method
Instance methods on `Baran::MarkdownSplitter` class.

### Endpoint
N/A (Ruby Gem)

### Parameters
#### Initialization Parameters
- **chunk_size** (Integer) - Required - The maximum size of each chunk.
- **chunk_overlap** (Integer) - Optional - The number of characters to overlap between consecutive chunks.

#### Method Parameters
- **text** (String) - Required - The Markdown text to split.
- **metadata** (Hash) - Optional - Metadata to associate with each chunk.

### Request Example
```ruby
require 'baran'

splitter = Baran::MarkdownSplitter.new(
  chunk_size: 500,
  chunk_overlap: 50
)

markdown_content = "# Document Title\n\nThis is the introduction..."
chunks = splitter.chunks(markdown_content, metadata: { file: "report.md" })
```

### Response
#### Success Response (chunks)
- **cursor** (Integer) - The starting position of the chunk in the original text.
- **text** (String) - The content of the chunk.
- **metadata** (Hash) - Any associated metadata.

#### Response Example
```json
{
  "cursor": 0,
  "text": "# Document Title\n\nThis is the introduction...",
  "metadata": {
    "file": "report.md"
  }
}
```
```

--------------------------------

### TextSplitter.chunks Method

Source: https://github.com/moeki0/baran/blob/main/README.md

Splits the input text into chunks with optional metadata.

```APIDOC
## TextSplitter `chunks` Method

### Description

Returns an array of chunk hashes, each containing the `:text`, `:cursor`, and optional `:metadata`.

### Method

`chunks(text, metadata: nil)`

### Parameters

#### Path Parameters

None

#### Query Parameters

None

#### Request Body

None

### Parameters

- **text** (String) - Required - The text to be split into chunks.
- **metadata** (Object) - Optional - Additional metadata to include with each chunk.

### Request Example

```ruby
text_to_split = "This is a long text that needs to be split into smaller chunks."
metadata = { source: "document.txt" }
splitter = Baran::TextSplitter.new
chunks = splitter.chunks(text_to_split, metadata: metadata)
```

### Response

#### Success Response (200)

- **chunks** (Array) - An array of chunk objects.
  - **text** (String) - The content of the chunk.
  - **cursor** (Integer) - The starting position of the chunk in the original text.
  - **metadata** (Object) - The metadata associated with the chunk (if provided).

#### Response Example

```json
[
  {
    "text": "This is the first chunk.",
    "cursor": 0,
    "metadata": {"source": "document.txt"}
  },
  {
    "text": "This is the second chunk.",
    "cursor": 30,
    "metadata": {"source": "document.txt"}
  }
]
```
```

--------------------------------

### RecursiveCharacterTextSplitter

Source: https://github.com/moeki0/baran/blob/main/README.md

Recursively splits text using multiple separators in priority order.

```APIDOC
## RecursiveCharacterTextSplitter

### Description

Recursively splits text using multiple separators in priority order.

### Method

`initialize(chunk_size: 1024, chunk_overlap: 64, separators: ["\n\n", "\n", " "])
`

### Parameters

- **chunk_size** (Integer) - Optional - Maximum characters per chunk (default: 1024).
- **chunk_overlap** (Integer) - Optional - Characters to overlap between chunks (default: 64).
- **separators** (Array) - Optional - Array of separators in priority order (default: ["\n\n", "\n", " "]).

### Request Example

```ruby
splitter = Baran::RecursiveCharacterTextSplitter.new(chunk_size: 500, separators: ["---", "\n"])
```
```

--------------------------------

### MarkdownSplitter

Source: https://github.com/moeki0/baran/blob/main/README.md

Splits Markdown text while preserving document structure.

```APIDOC
## MarkdownSplitter

### Description

Splits Markdown text while preserving document structure. Inherits from `RecursiveCharacterTextSplitter` with Markdown-specific separators.

### Method

`initialize(chunk_size: 1024, chunk_overlap: 64)`

### Parameters

- **chunk_size** (Integer) - Optional - Maximum characters per chunk (default: 1024).
- **chunk_overlap** (Integer) - Optional - Characters to overlap between chunks (default: 64).

### Request Example

```ruby
splitter = Baran::MarkdownSplitter.new(chunk_size: 700)
```
```

--------------------------------

### Processing Large Documents with Baran

Source: https://github.com/moeki0/baran/blob/main/README.md

Provides a class-based approach for processing large files using Baran. The `DocumentProcessor` class initializes a `RecursiveCharacterTextSplitter` and has a method to read a file, split its content, and then simulate saving each chunk to a vector store. Metadata includes file path, size, and processing timestamp.

```ruby
class DocumentProcessor
  def initialize
    @splitter = Baran::RecursiveCharacterTextSplitter.new(
      chunk_size: 1000,
      chunk_overlap: 100
    )
  end

  def process_file(file_path)
    content = File.read(file_path)
    
    chunks = @splitter.chunks(
      content,
      metadata: {
        file_path: file_path,
        file_size: File.size(file_path),
        processed_at: Time.now
      }
    )

    chunks.each_with_index do |chunk, index|
      save_to_vector_store(chunk, index)
    end
  end

  private

  def save_to_vector_store(chunk, index)
    # Your vector storage logic here
    puts "Saved chunk #{index}: #{chunk[:text].length} chars"
  end
end
```

--------------------------------

### RecursiveCharacterTextSplitter API

Source: https://context7.com/moeki0/baran/llms.txt

Attempts splitting with multiple separators in priority order, recursively dividing text that exceeds chunk size until all chunks fit within constraints.

```APIDOC
## RecursiveCharacterTextSplitter - Multi-level hierarchical text splitting

### Description
Recursively splits text using a list of separators in priority order, ensuring chunks adhere to size constraints while preserving structure.

### Method
Instance methods on `Baran::RecursiveCharacterTextSplitter` class.

### Endpoint
N/A (Ruby Gem)

### Parameters
#### Initialization Parameters
- **chunk_size** (Integer) - Required - The maximum size of each chunk.
- **chunk_overlap** (Integer) - Optional - The number of characters to overlap between consecutive chunks.
- **separators** (Array of Strings) - Optional - A list of separators to try in order. Defaults to `["\n\n", "\n", " "]`.

#### Method Parameters
- **text** (String) - Required - The text to split.
- **metadata** (Hash) - Optional - Metadata to associate with each chunk.

### Request Example
```ruby
require 'baran'

splitter = Baran::RecursiveCharacterTextSplitter.new(
  chunk_size: 100,
  chunk_overlap: 20,
  separators: ["\n\nclass ", "\n\ndef ", "\n\n", "\n", " ", ""]
)

document = "# Introduction\n\nThis is a long document..."
chunks = splitter.chunks(document, metadata: { doc_id: "doc_001" })
```

### Response
#### Success Response (chunks)
- **cursor** (Integer) - The starting position of the chunk in the original text.
- **text** (String) - The content of the chunk.
- **metadata** (Hash) - Any associated metadata.

#### Response Example
```json
{
  "cursor": 0,
  "text": "# Introduction\n\nThis is a long document...",
  "metadata": {
    "doc_id": "doc_001"
  }
}
```
```

--------------------------------

### Process Large Files with Streaming

Source: https://github.com/moeki0/baran/blob/main/README.md

Provides a Ruby function `process_large_file` that demonstrates how to process large files efficiently using streaming. It utilizes RecursiveCharacterTextSplitter to chunk the file paragraph by paragraph and yields each chunk for further processing.

```ruby
def process_large_file(file_path)
  splitter = Baran::RecursiveCharacterTextSplitter.new
  
  File.foreach(file_path, "\n\n") do |paragraph|
    chunks = splitter.chunks(paragraph)
    chunks.each { |chunk| yield chunk }
  end
end

process_large_file('huge_document.txt') do |chunk|
  # Process each chunk individually
  save_to_database(chunk)
end
```

--------------------------------

### Sentence Text Splitting

Source: https://github.com/moeki0/baran/blob/main/README.md

Details the `SentenceTextSplitter` for segmenting text based on natural sentence boundaries, identified by periods, exclamation marks, and question marks. This splitter is suitable when preserving the integrity of individual sentences is important. It includes parameters for chunk size and overlap.

```ruby
splitter = Baran::SentenceTextSplitter.new(
    chunk_size: 2000,
    chunk_overlap: 200
)
chunks = splitter.chunks(text)
# => [{ cursor: 0, text: "Complete sentence.", metadata: nil }, ...]
```

--------------------------------

### CharacterTextSplitter - Simple Delimiter Text Splitting in Ruby

Source: https://context7.com/moeki0/baran/llms.txt

Splits text using specified separators, maintaining chunk size constraints and overlap. Useful for basic text division where a clear delimiter exists. It processes text based on a given separator and returns chunks with position and optional metadata.

```ruby
require 'baran'

# Basic usage with default separator (\n\n)
splitter = Baran::CharacterTextSplitter.new(
  chunk_size: 100,
  chunk_overlap: 20
)

text = "First paragraph here.\n\nSecond paragraph here.\n\nThird paragraph here."
chunks = splitter.chunks(text)

chunks.each do |chunk|
  puts "Position: #{chunk[:cursor]}"
  puts "Text: #{chunk[:text]}"
  puts "---"
end
# => Position: 0
#    Text: First paragraph here.
#    ---
#    Position: 23
#    Text: Second paragraph here.
#    ---
#    Position: 47
#    Text: Third paragraph here.

# Custom separator usage
csv_splitter = Baran::CharacterTextSplitter.new(
  chunk_size: 500,
  chunk_overlap: 50,
  separator: ","
)

csv_data = "item1,item2,item3,item4,item5"
csv_chunks = csv_splitter.chunks(csv_data, metadata: { source: "data.csv" })
csv_chunks.first
# => { cursor: 0, text: "item1,item2,item3", metadata: { source: "data.csv" } }
```

--------------------------------

### Handle Invalid Chunk Size and Overlap Error

Source: https://github.com/moeki0/baran/blob/main/README.md

Demonstrates error handling when initializing a TextSplitter with invalid parameters where chunk overlap is greater than or equal to the chunk size. It uses a begin-rescue block to catch the RuntimeError and print the error message.

```ruby
begin
  # This will raise an error
  invalid_splitter = Baran::TextSplitter.new(
    chunk_size: 100,
    chunk_overlap: 100  # overlap >= chunk_size
  )
rescue RuntimeError => e
  puts "Error: #{e.message}"
  # => "Cannot have chunk_overlap >= chunk_size"
end
```

--------------------------------

### CharacterTextSplitter

Source: https://github.com/moeki0/baran/blob/main/README.md

Splits text using a specified separator character.

```APIDOC
## CharacterTextSplitter

### Description

Splits text using a specified separator character.

### Method

`initialize(chunk_size: 1024, chunk_overlap: 64, separator: "\n\n")`

### Parameters

- **chunk_size** (Integer) - Optional - Maximum characters per chunk (default: 1024).
- **chunk_overlap** (Integer) - Optional - Characters to overlap between chunks (default: 64).
- **separator** (String) - Optional - Character(s) to split on (default: "\n\n").

### Request Example

```ruby
splitter = Baran::CharacterTextSplitter.new(separator: ", ", chunk_size: 200)
```
```

--------------------------------

### RecursiveCharacterTextSplitter - Hierarchical Text Splitting in Ruby

Source: https://context7.com/moeki0/baran/llms.txt

Splits text hierarchically using multiple separators in priority order. It recursively divides text that exceeds chunk size, ensuring chunks fit within constraints while preserving natural text flow. Useful for complex documents with varying structures.

```ruby
require 'baran'

# Default separators: ["\n\n", "\n", " "]
splitter = Baran::RecursiveCharacterTextSplitter.new(
  chunk_size: 100,
  chunk_overlap: 20
)

document = <<~TEXT
  # Introduction

  This is a long document with multiple paragraphs.
  It needs to be split intelligently.

  ## Section One

  Content for section one goes here.
  More content follows.
TEXT

chunks = splitter.chunks(document, metadata: { doc_id: "doc_001" })

chunks.each_with_index do |chunk, i|
  puts "Chunk #{i + 1}:"
  puts "  Length: #{chunk[:text].length}"
  puts "  Starts at: #{chunk[:cursor]}"
  puts "  Metadata: #{chunk[:metadata]}"
  puts "  Preview: #{chunk[:text][0..50]}..."
  puts
end
# Output shows document split at paragraph breaks first, then line breaks,
# then spaces, maintaining natural text flow

# Custom separators for code files
code_splitter = Baran::RecursiveCharacterTextSplitter.new(
  chunk_size: 500,
  chunk_overlap: 50,
  separators: ["\n\nclass ", "\n\ndef ", "\n\n", "\n", " ", ""]
)

ruby_code = File.read('app.rb')
code_chunks = code_splitter.chunks(ruby_code, metadata: { file: 'app.rb', type: 'ruby' })
```

--------------------------------

### SentenceTextSplitter - Sentence Boundary Splitting in Ruby

Source: https://context7.com/moeki0/baran/llms.txt

Splits text based on sentence endings (periods, exclamation marks, question marks). This method ensures grammatical integrity within each chunk, making it ideal for processing natural language text where sentence structure is important. It handles incomplete sentences gracefully.

```ruby
require 'baran'

splitter = Baran::SentenceTextSplitter.new(
  chunk_size: 200,
  chunk_overlap: 40
)

article = "Ruby is a dynamic language. It was created by Yukihiro Matsumoto! " \
          "Why is Ruby popular? It has an elegant syntax. Developers love it. " \
          "Many frameworks use Ruby."

chunks = splitter.chunks(article, metadata: { source: "article.txt" })

chunks.each do |chunk|
  puts chunk[:text]
  puts "Position: #{chunk[:cursor]}, Length: #{chunk[:text].length}"
  puts "---"
end
# => Ruby is a dynamic language. It was created by Yukihiro Matsumoto!
#    Position: 0, Length: 66
#    ---
#    Why is Ruby popular? It has an elegant syntax. Developers love it.
#    Position: 67, Length: 66
#    ---
#    Many frameworks use Ruby.
#    Position: 134, Length: 24

# Handling text without proper endings
incomplete_text = "This sentence ends properly. This one doesn't"
incomplete_chunks = splitter.chunks(incomplete_text)
# Gracefully handles incomplete sentences
```

--------------------------------

### CharacterTextSplitter API

Source: https://context7.com/moeki0/baran/llms.txt

Splits text at specified separators (like double newlines or custom delimiters) while maintaining chunk size constraints and overlap between chunks for context continuity.

```APIDOC
## CharacterTextSplitter - Simple delimiter-based text splitting

### Description
Splits text at specified separators while maintaining chunk size constraints and overlap between chunks.

### Method
Instance methods on `Baran::CharacterTextSplitter` class.

### Endpoint
N/A (Ruby Gem)

### Parameters
#### Initialization Parameters
- **chunk_size** (Integer) - Required - The maximum size of each chunk.
- **chunk_overlap** (Integer) - Optional - The number of characters to overlap between consecutive chunks.
- **separator** (String) - Optional - The delimiter to split the text by. Defaults to `"\n\n"`.

#### Method Parameters
- **text** (String) - Required - The text to split.
- **metadata** (Hash) - Optional - Metadata to associate with each chunk.

### Request Example
```ruby
require 'baran'

splitter = Baran::CharacterTextSplitter.new(
  chunk_size: 100,
  chunk_overlap: 20,
  separator: ","
)

text = "item1,item2,item3,item4,item5"
chunks = splitter.chunks(text, metadata: { source: "data.csv" })
```

### Response
#### Success Response (chunks)
- **cursor** (Integer) - The starting position of the chunk in the original text.
- **text** (String) - The content of the chunk.
- **metadata** (Hash) - Any associated metadata.

#### Response Example
```json
{
  "cursor": 0,
  "text": "item1,item2,item3",
  "metadata": {
    "source": "data.csv"
  }
}
```
```

--------------------------------

### Recursive Character Text Splitting

Source: https://github.com/moeki0/baran/blob/main/README.md

Explains the `RecursiveCharacterTextSplitter`, which splits text by recursively applying a list of separators. This method is useful for preserving semantic structure by attempting to split at meaningful boundaries first. It also demonstrates adding custom metadata to the chunks.

```ruby
splitter = Baran::RecursiveCharacterTextSplitter.new(
    chunk_size: 1024,
    chunk_overlap: 64,
    separators: ["\n\n", "\n", " ", ""]
)
chunks = splitter.chunks(text, metadata: { type: "article" })
# => [{ cursor: 0, text: "...", metadata: { type: "article" } }, ...]
```

--------------------------------

### SentenceTextSplitter API

Source: https://context7.com/moeki0/baran/llms.txt

Splits text at sentence endings (periods, exclamation marks, question marks) to maintain grammatical integrity within chunks.

```APIDOC
## SentenceTextSplitter - Sentence-boundary-aware splitting

### Description
Splits text at sentence endings, ensuring that each chunk represents a grammatically complete sentence or a sequence of sentences.

### Method
Instance methods on `Baran::SentenceTextSplitter` class.

### Endpoint
N/A (Ruby Gem)

### Parameters
#### Initialization Parameters
- **chunk_size** (Integer) - Required - The maximum size of each chunk.
- **chunk_overlap** (Integer) - Optional - The number of characters to overlap between consecutive chunks.

#### Method Parameters
- **text** (String) - Required - The text to split.
- **metadata** (Hash) - Optional - Metadata to associate with each chunk.

### Request Example
```ruby
require 'baran'

splitter = Baran::SentenceTextSplitter.new(
  chunk_size: 200,
  chunk_overlap: 40
)

article = "Ruby is a dynamic language. It was created by Yukihiro Matsumoto! Why is Ruby popular?"
chunks = splitter.chunks(article, metadata: { source: "article.txt" })
```

### Response
#### Success Response (chunks)
- **cursor** (Integer) - The starting position of the chunk in the original text.
- **text** (String) - The content of the chunk.
- **metadata** (Hash) - Any associated metadata.

#### Response Example
```json
{
  "text": "Ruby is a dynamic language. It was created by Yukihiro Matsumoto!",
  "cursor": 0,
  "metadata": {
    "source": "article.txt"
  }
}
```
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.