### Install chunking_evaluation Package Source: https://github.com/brandonstarxel/chunking_evaluation/blob/main/README.md Installs the chunking_evaluation package directly from its GitHub repository using pip. This command fetches and installs the latest version of the package and its dependencies. ```bash pip install git+https://github.com/brandonstarxel/chunking_evaluation.git ``` -------------------------------- ### Configure Retrieval Strategies in Evaluation Source: https://context7.com/brandonstarxel/chunking_evaluation/llms.txt Shows how to toggle between adaptive retrieval (retrieving chunks based on content) and fixed-k retrieval. Also demonstrates optional caching of embeddings to disk. ```python results_adaptive = evaluation.run( chunker=chunker, embedding_function=embedding_func, retrieve=-1 ) results_fixed = evaluation.run( chunker=chunker, embedding_function=embedding_func, retrieve=5, db_to_save_chunks="./chunk_cache" ) ``` -------------------------------- ### Implement Custom Text Chunker in Python Source: https://context7.com/brandonstarxel/chunking_evaluation/llms.txt Defines an abstract base class `BaseChunker` for creating custom text chunking strategies. Users inherit from `BaseChunker` and implement the `split_text` method to define their chunking logic. This example demonstrates simple fixed-character chunking. ```python from chunking_evaluation import BaseChunker from typing import List class CustomChunker(BaseChunker): def __init__(self, chunk_size: int = 1200): self.chunk_size = chunk_size def split_text(self, text: str) -> List[str]: # Simple fixed-character chunking chunks = [] for i in range(0, len(text), self.chunk_size): chunks.append(text[i:i + self.chunk_size]) return chunks # Usage chunker = CustomChunker(chunk_size=1000) text = "Your long document text here..." chunks = chunker.split_text(text) print(f"Created {len(chunks)} chunks") # Output: Created 5 chunks ```