### Install embetter

Source: https://github.com/koaning/embetter/blob/main/docs/index.md

Installs the embetter library. Optional dependencies for specific embedding types can be installed using extra package names like `[text]`, `[sbert]`, etc., or `[all]` for everything.

```bash
python -m pip install embetter
python -m pip install "embetter[text]"
python -m pip install "embetter[sbert]"
python -m pip install "embetter[spacy]"
python -m pip install "embetter[sense2vec]"
python -m pip install "embetter[bpemb]"
python -m pip install "embetter[gensim]"
python -m pip install "embetter[vision]"
python -m pip install "embetter[all]"
```

--------------------------------

### Install embetter Package

Source: https://github.com/koaning/embetter/blob/main/README.md

Installs the embetter library. Optional dependencies can be installed using bracket notation, such as `embetter[text]` for text-specific features or `embetter[all]` for all available features.

```shell
python -m pip install embetter
python -m pip install "embetter[text]"
python -m pip install "embetter[spacy]"
python -m pip install "embetter[sense2vec]"
python -m pip install "embetter[gensim]"
python -m pip install "embetter[bpemb]"
python -m pip install "embetter[vision]"
python -m pip install "embetter[all]"
```

--------------------------------

### Load SentenceEncoder Models

Source: https://github.com/koaning/embetter/blob/main/docs/applications.md

Shows how to load various SentenceEncoder models from Hugging Face, compatible with the sentence-transformers library. It provides examples for 'thenlper/gte-small', 'thenlper/gte-base', and 'thenlper/gte-large'. Note that some models may require specific text prefixes.

```python
# https://huggingface.co/thenlper/gte-small
model = SentenceEncoder('thenlper/gte-small')
model = SentenceEncoder('thenlper/gte-base')
model = SentenceEncoder('thenlper/gte-large')
```

--------------------------------

### Image Embedding Pipeline Example

Source: https://github.com/koaning/embetter/blob/main/README.md

Illustrates building a scikit-learn pipeline for image embedding. It selects image paths from a DataFrame column, loads them as PIL Image objects, and then encodes them using a CLIP encoder.

```python
import pandas as pd
from sklearn.pipeline import make_pipeline 
from sklearn.linear_model import LogisticRegression

from embetter.grab import ColumnGrabber
from embetter.vision import ImageLoader
from embetter.multi import ClipEncoder

# This pipeline grabs the `img_path` column from a dataframe
# then it grabs the image paths and turns them into `PIL.Image` objects
# which then get fed into CLIP which can also handle images.
image_emb_pipeline = make_pipeline(
  ColumnGrabber("img_path"),
  ImageLoader(convert="RGB"),
  ClipEncoder()
)

dataf = pd.DataFrame({
  "img_path": ["tests/data/thiscatdoesnotexist.jpeg"]
})
image_emb_pipeline.fit_transform(dataf)
```

--------------------------------

### CohereEncoder API Documentation

Source: https://github.com/koaning/embetter/blob/main/docs/API/external.md

Documentation for the CohereEncoder class, used for encoding text via Cohere's embedding models. This entry outlines its setup and operational methods.

```APIDOC
embetter.external.CohereEncoder:
  Description: Encodes text using Cohere's embedding models.
  Initialization:
    __init__(model_name: str, api_key: Optional[str] = None, **kwargs):
      Initializes the CohereEncoder with a specified Cohere model and API key.
      Parameters:
        model_name (str): The name of the Cohere model to use (e.g., 'embed-english-light-v2.0').
        api_key (Optional[str]): Your Cohere API key. If not provided, it attempts to read from the COHERE_API_KEY environment variable.
        **kwargs: Additional keyword arguments passed to the underlying Cohere client.
  Methods:
    embed(texts: List[str], **kwargs) -> List[List[float]]:
      Encodes a list of texts into numerical embeddings using Cohere.
      Parameters:
        texts (List[str]): A list of strings to encode.
        **kwargs: Additional arguments passed to the embedding API call.
      Returns:
        List[List[float]]: A list of embedding vectors, where each vector corresponds to an input text.
    __call__(texts: Union[str, List[str]], **kwargs) -> Union[List[float], List[List[float]]]:
      Allows the encoder to be called directly like a function.
      Parameters:
        texts (Union[str, List[str]]): A single string or a list of strings to encode.
      Returns:
        Union[List[float], List[List[float]]]: The embedding(s) for the input text(s).
  Notes:
    Requires a Cohere API key, which can be provided directly or via the COHERE_API_KEY environment variable.
```

--------------------------------

### Embeddings with Modal GPU Acceleration

Source: https://github.com/koaning/embetter/blob/main/docs/applications.md

Demonstrates accelerating embedding calculations using Modal. It sets up a Modal stub and a GPU-enabled function to run SentenceEncoder transformations on data. The example includes saving embeddings to an HDF5 file and measuring execution time.

```python
import time
import h5py
import modal


stub = modal.Stub("example-get-started")
image = (modal.Image.debian_slim()
         .pip_install("simsity", "embetter[text]", "h5py")
         .run_commands("python -c 'from embetter.text import SentenceEncoder; SentenceEncoder()'" ))


# This is the function that actually runs the embedding, 
# notice that there's a GPU attached.
@stub.function(image=image, gpu="any")
def create(data):
    from embetter.text import SentenceEncoder
    return SentenceEncoder().transform(data)


@stub.local_entrypoint()
def main():
    tic = time.time()

    # You'd need to write your own function to read in the texts
    data = read_text() # Assuming read_text() is defined elsewhere
    
    # This runs our decorated function on external hardware
    X = create.call(data)

    # Next we save it to disk for re-use
    with h5py.File('embeddings.h5', 'w') as hf:
        hf.create_dataset("embeddings",  data=X)
    toc = time.time()
    print(f"took {toc - tic}s to embed shape {X.shape}")
```

--------------------------------

### Text Embedding Pipeline Example

Source: https://github.com/koaning/embetter/blob/main/README.md

Demonstrates creating a scikit-learn pipeline for text embedding and classification. It first extracts text from a DataFrame column using `ColumnGrabber` and then embeds it using `SentenceEncoder`. The embedded features are then used with `LogisticRegression` for classification.

```python
import pandas as pd
from sklearn.pipeline import make_pipeline 
from sklearn.linear_model import LogisticRegression

from embetter.grab import ColumnGrabber
from embetter.text import SentenceEncoder

# This pipeline grabs the `text` column from a dataframe
# which then get fed into Sentence-Transformers' all-MiniLM-L6-v2.
text_emb_pipeline = make_pipeline(
  ColumnGrabber("text"),
  SentenceEncoder('all-MiniLM-L6-v2')
)

# This pipeline can also be trained to make predictions, using
# the embedded features. 
text_clf_pipeline = make_pipeline(
  text_emb_pipeline,
  LogisticRegression()
)

dataf = pd.DataFrame({
  "text": ["positive sentiment", "super negative"],
  "label_col": ["pos", "neg"]
})
X = text_emb_pipeline.fit_transform(dataf, dataf['label_col'])
text_clf_pipeline.fit(dataf, dataf['label_col']).predict(dataf)
```

--------------------------------

### Store Lite Embeddings on Disk

Source: https://github.com/koaning/embetter/blob/main/docs/applications.md

Demonstrates how to save the trained lightweight embeddings directly to disk during the training process using the `path` argument in `learn_lite_doc_embeddings`. This allows for persistent storage and reuse of the trained embedding model.

```python
from embetter.text import learn_lite_doc_embeddings

# Assuming 'texts' is already loaded
# enc = learn_lite_doc_embeddings(texts, dim=300, path="stored/on/disk.emb")
```

--------------------------------

### Manual Pipeline for Lite Embeddings

Source: https://github.com/koaning/embetter/blob/main/docs/applications.md

Provides the underlying implementation for creating lightweight text embeddings using a pipeline of `TfidfVectorizer` and `TruncatedSVD` from scikit-learn. This demonstrates the core technique without relying on the `embetter` utility, allowing for custom configurations.

```python
from sklearn.decomposition import TruncatedSVD
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.pipeline import make_pipeline

# Assuming 'texts' is already loaded
# enc = make_pipeline(
#     TfidfVectorizer(),
#     TruncatedSVD()
# )
```

--------------------------------

### AzureOpenAIEncoder API Documentation

Source: https://github.com/koaning/embetter/blob/main/docs/API/external.md

Documentation for the AzureOpenAIEncoder class, designed for encoding text using Azure OpenAI Service. This entry covers its specific initialization parameters and usage.

```APIDOC
embetter.external.AzureOpenAIEncoder:
  Description: Encodes text using Azure OpenAI's embedding models.
  Initialization:
    __init__(model_name: str, api_key: str, api_base: str, api_type: str = 'azure', embedding_ctx_length: int = 8191, batch_size: int = 16, **kwargs):
      Initializes the AzureOpenAIEncoder with Azure-specific API configurations.
      Parameters:
        model_name (str): The name of the Azure OpenAI model to use (e.g., 'text-embedding-ada-002').
        api_key (str): Your Azure OpenAI API key.
        api_base (str): The Azure OpenAI API base URL (e.g., 'https://your-resource-name.openai.azure.com/').
        api_type (str): The API type, must be 'azure'. Defaults to 'azure'.
        embedding_ctx_length (int): The maximum context length for embeddings. Defaults to 8191.
        batch_size (int): The number of texts to embed in a single batch. Defaults to 16.
        **kwargs: Additional keyword arguments passed to the underlying Azure OpenAI client.
  Methods:
    embed(texts: List[str], **kwargs) -> List[List[float]]:
      Encodes a list of texts into numerical embeddings using Azure OpenAI.
      Parameters:
        texts (List[str]): A list of strings to encode.
        **kwargs: Additional arguments passed to the embedding API call.
      Returns:
        List[List[float]]: A list of embedding vectors, where each vector corresponds to an input text.
    __call__(texts: Union[str, List[str]], **kwargs) -> Union[List[float], List[List[float]]]:
      Allows the encoder to be called directly like a function.
      Parameters:
        texts (Union[str, List[str]]): A single string or a list of strings to encode.
      Returns:
        Union[List[float], List[List[float]]]: The embedding(s) for the input text(s).
  Notes:
    Requires explicit provision of `api_key` and `api_base` for Azure OpenAI Service.
```

--------------------------------

### OpenAIEncoder API Documentation

Source: https://github.com/koaning/embetter/blob/main/docs/API/external.md

Documentation for the OpenAIEncoder class, used for encoding text via OpenAI models. This entry details its initialization and core functionalities.

```APIDOC
embetter.external.OpenAIEncoder:
  Description: Encodes text using OpenAI's embedding models.
  Initialization:
    __init__(model_name: str, provider: str = 'openai', api_key: Optional[str] = None, organization: Optional[str] = None, api_base: Optional[str] = None, api_type: Optional[str] = None, embedding_ctx_length: int = 8191, batch_size: int = 16, **kwargs):
      Initializes the OpenAIEncoder with specified model and API configurations.
      Parameters:
        model_name (str): The name of the OpenAI model to use (e.g., 'text-embedding-ada-002').
        provider (str): The API provider, defaults to 'openai'.
        api_key (Optional[str]): Your OpenAI API key. If not provided, it attempts to read from the OPENAI_API_KEY environment variable.
        organization (Optional[str]): Your OpenAI organization ID. If not provided, it attempts to read from the OPENAI_ORG_ID environment variable.
        api_base (Optional[str]): Custom API base URL if not using the default OpenAI endpoint.
        api_type (Optional[str]): Type of API, e.g., 'openai', 'azure'.
        embedding_ctx_length (int): The maximum context length for embeddings. Defaults to 8191.
        batch_size (int): The number of texts to embed in a single batch. Defaults to 16.
        **kwargs: Additional keyword arguments passed to the underlying OpenAI client.
  Methods:
    embed(texts: List[str], **kwargs) -> List[List[float]]:
      Encodes a list of texts into numerical embeddings.
      Parameters:
        texts (List[str]): A list of strings to encode.
        **kwargs: Additional arguments passed to the embedding API call.
      Returns:
        List[List[float]]: A list of embedding vectors, where each vector corresponds to an input text.
    _get_embedding(text: str, **kwargs) -> List[float]:
      Internal method to get embedding for a single text.
    _get_batch_embedding(texts: List[str], **kwargs) -> List[List[float]]:
      Internal method to get embeddings for a batch of texts.
    __call__(texts: Union[str, List[str]], **kwargs) -> Union[List[float], List[List[float]]]:
      Allows the encoder to be called directly like a function.
      Parameters:
        texts (Union[str, List[str]]): A single string or a list of strings to encode.
      Returns:
        Union[List[float], List[List[float]]]: The embedding(s) for the input text(s).
  Notes:
    Ensure your OpenAI API key and organization ID are correctly set either as environment variables or passed during initialization.
```

--------------------------------

### Train Lite Document Embeddings with TfidfVectorizer and TruncatedSVD

Source: https://github.com/koaning/embetter/blob/main/docs/applications.md

Illustrates training lightweight document embeddings using scikit-learn's `TfidfVectorizer` and `TruncatedSVD` via the `learn_lite_doc_embeddings` utility. This method is fast and can produce reasonable representations, with options for character-based analysis and n-gram ranges for robustness. It requires `srsly`, `umap`, `cluestar`, and `embetter`.

```python
import srsly
from umap import UMAP
from cluestar import plot_text
from embetter.text import learn_lite_doc_embeddings

# Train embeddings 
texts = [ex['text'] for ex in srsly.read_jsonl("datasets/new-dataset.jsonl")]
enc = learn_lite_doc_embeddings(texts, dim=300)

# Create a 2D UMAP representation
X_orig = enc.transform(texts)  # this takes ~56ms
X = UMAP().fit_transform(X_orig)

# Plot the UMAP representation with the text
plot_text(X, texts)
```

--------------------------------

### Image Embedding Pipeline

Source: https://github.com/koaning/embetter/blob/main/docs/index.md

Illustrates building a scikit-learn pipeline for image embedding using CLIP. It involves grabbing image paths from a DataFrame, loading them as PIL Images, and then encoding them with a CLIP model.

```python
import pandas as pd
from sklearn.pipeline import make_pipeline 
from sklearn.linear_model import LogisticRegression

from embetter.grab import ColumnGrabber
from embetter.vision import ImageLoader
from embetter.multi import ClipEncoder

# This pipeline grabs the `img_path` column from a dataframe
# then it grabs the image paths and turns them into `PIL.Image` objects
# which then get fed into CLIP which can also handle images.
image_emb_pipeline = make_pipeline(
  ColumnGrabber("img_path"),
  ImageLoader(convert="RGB"),
  ClipEncoder()
)

dataf = pd.DataFrame({
  "img_path": ["tests/data/thiscatdoesnotexist.jpeg"]
})
image_emb_pipeline.fit_transform(dataf)
```

--------------------------------

### embetter API Components

Source: https://github.com/koaning/embetter/blob/main/README.md

Lists the core components available in the embetter library for various embedding tasks. All components are scikit-learn compatible and stateless, utilizing pre-trained models.

```APIDOC
embetter API Components:

# Helpers to grab data from pandas columns
from embetter.grab import ColumnGrabber
  - ColumnGrabber(column_name: str)
    - Purpose: Selects a specific column from a pandas DataFrame.
    - Parameters:
      - column_name: The name of the column to extract.

# Representations/Helpers for computer vision
from embetter.vision import ImageLoader, TimmEncoder, ColorHistogramEncoder
  - ImageLoader(convert: str = None)
    - Purpose: Loads images from file paths and optionally converts them to a specified format (e.g., 'RGB').
    - Parameters:
      - convert: The target color format (e.g., 'RGB', 'L').
  - TimmEncoder(model_name: str)
    - Purpose: Encodes images using models from the `timm` library.
    - Parameters:
      - model_name: The name of the pre-trained model from `timm`.
  - ColorHistogramEncoder()
    - Purpose: Generates a color histogram representation of an image.

# Representations for text
from embetter.text import SentenceEncoder, MatryoshkaEncoder, Sense2VecEncoder, BytePairEncoder, spaCyEncoder, GensimEncoder, TextEncoder
  - SentenceEncoder(model_name: str)
    - Purpose: Encodes text into sentence embeddings using Sentence-Transformers.
    - Parameters:
      - model_name: The name of the Sentence-Transformer model (e.g., 'all-MiniLM-L6-v2').
  - MatryoshkaEncoder(model_name: str)
    - Purpose: Encodes text with Matryoshka embeddings.
    - Parameters:
      - model_name: The name of the underlying embedding model.
  - Sense2VecEncoder(model_name: str)
    - Purpose: Encodes text using Sense2Vec models.
    - Parameters:
      - model_name: The name of the Sense2Vec model.
  - BytePairEncoder(model_name: str)
    - Purpose: Encodes text using Byte-Pair Encoding.
    - Parameters:
      - model_name: The name of the BPE model.
  - spaCyEncoder(model_name: str)
    - Purpose: Encodes text using spaCy's word vectors.
    - Parameters:
      - model_name: The name of the spaCy language model.
  - GensimEncoder(model_name: str)
    - Purpose: Encodes text using Gensim models.
    - Parameters:
      - model_name: The name of the Gensim model.
  - TextEncoder()
    - Purpose: A general text encoder, likely a wrapper for other text embedding methods.

# Representations from multi-modal models
from embetter.multi import ClipEncoder
  - ClipEncoder()
    - Purpose: Encodes text and images using CLIP models.

# Finetuning components
from embetter.finetune import FeedForwardTuner, ContrastiveTuner, ContrastiveLearner, SbertLearner
  - FeedForwardTuner()
    - Purpose: Component for finetuning embeddings with a feed-forward network.
  - ContrastiveTuner()
    - Purpose: Component for finetuning embeddings using contrastive learning.
  - ContrastiveLearner()
    - Purpose: Learner for contrastive embedding tasks.
  - SbertLearner()
    - Purpose: Learner specifically for Sentence-BERT finetuning.

# External embedding providers
from embetter.external import CohereEncoder, OpenAIEncoder
  - CohereEncoder(model_name: str, api_key: str)
    - Purpose: Encodes text using Cohere's embedding models.
    - Parameters:
      - model_name: The name of the Cohere model.
      - api_key: Your Cohere API key.
  - OpenAIEncoder(model_name: str, api_key: str)
    - Purpose: Encodes text using OpenAI's embedding models.
    - Parameters:
      - model_name: The name of the OpenAI model (e.g., 'text-embedding-ada-002').
      - api_key: Your OpenAI API key.
```

--------------------------------

### Cache Embeddings with DiskCache

Source: https://github.com/koaning/embetter/blob/main/docs/applications.md

Demonstrates how to use the `cached` utility from `embetter.utils` to cache sentence embeddings using `diskcache`. This significantly speeds up subsequent calls by storing computed embeddings on disk. It requires `embetter` and `diskcache` libraries.

```python
from embetter.text import SentenceEncoder
from embetter.utils import cached

encoder = cached("sentence-enc", SentenceEncoder('all-MiniLM-L6-v2'))

examples = [f"this is a pretty long text, which is more expensive {i}" for i in range(10_000)]

# This might be a bit slow ~17.2s on our machine
encoder.transform(examples)

# This should be quicker ~4.71s on our machine
encoder.transform(examples)
```

--------------------------------

### ClipEncoder API Documentation

Source: https://github.com/koaning/embetter/blob/main/docs/API/multimodal.md

Documentation for the ClipEncoder class. This class is part of the embetter.multi module and is designed for encoding clips. The specific methods, parameters, and return values are not detailed in the provided source, but it represents a key component for embedding functionalities.

```APIDOC
embetter.multi.ClipEncoder
  (No specific methods or parameters detailed in the provided source.)
  This class is expected to handle the encoding of various types of clips, likely for use in embedding models. Further details on its implementation, input requirements, and output formats would be available in the full project documentation.
```

--------------------------------

### Visualize Lite Embeddings with Sentence Transformers and UMAP

Source: https://github.com/koaning/embetter/blob/main/docs/applications.md

Shows how to generate embeddings using `SentenceEncoder` from Sentence Transformers, reduce dimensionality with UMAP, and visualize the results using `plot_text`. This provides an alternative to the TfidfVectorizer approach and allows comparison of embedding quality and clustering.

```python
from embetter.text import SentenceEncoder
from umap import UMAP
from cluestar import plot_text

# Assuming 'texts' is already loaded
sent_enc = SentenceEncoder()
X_orig = sent_enc.transform(texts) # this takes ~13.5s 
X = UMAP().fit_transform(X_orig)
plot_text(X, texts)
```

--------------------------------

### DifferenceClassifier Usage

Source: https://github.com/koaning/embetter/blob/main/docs/applications.md

Demonstrates training and prediction using the DifferenceClassifier for deduplication tasks. It shows how to fit the model with text pairs and similarity labels, then use predict and predict_proba methods. The underlying classifier head is a scikit-learn model.

```python
from embetter.model import DifferenceClassifier
from embetter.text import SentenceEncoder

mod = DifferenceClassifier(enc=SentenceEncoder())

# Suppose this is input data
texts1 = ["hello", "firehydrant", "greetings"]
texts2 = ["no",    "yes",         "greeting"]

# You will need to have some definition of "similar"
similar = [0, 0, 1]

# Train a model to detect similarity
mod.fit(X1=texts1, X2=texts2, y=similar)
mod.predict(X1=texts1, X2=texts2)
mod.predict_proba(X1=texts1, X2=texts2)

# The classifier head is a scikit-learn model, which you could save
# seperately if you like. The model can be accessed via:
mod.clf_head
```

--------------------------------

### Embetter Component Imports

Source: https://github.com/koaning/embetter/blob/main/docs/index.md

Shows the import statements for various embedding and finetuning components provided by the embetter library. These components are designed to be scikit-learn compatible and are generally pretrained.

```python
# Helpers to grab text or image from pandas column.
from embetter.grab import ColumnGrabber

# Representations/Helpers for computer vision
from embetter.vision import ImageLoader, TimmEncoder, ColorHistogramEncoder

# Representations for text
from embetter.text import SentenceEncoder, Sense2VecEncoder, BytePairEncoder, spaCyEncoder, GensimEncoder

# Representations from multi-modal models
from embetter.multi import ClipEncoder

# Finetuning components 
from embetter.finetune import FeedForwardTuner, ContrastiveTuner, ContrastiveLearner, SbertLearner

# External embedding providers, typically needs an API key
from embetter.external import CohereEncoder, OpenAIEncoder
```

--------------------------------

### TextEncoder API

Source: https://github.com/koaning/embetter/blob/main/docs/API/text.md

API documentation for the base TextEncoder class. This serves as a foundational interface for text encoding operations within the embetter library.

```APIDOC
embetter.text.TextEncoder:
  Abstract base class for text encoders.
  Provides a common interface for converting text into numerical representations (embeddings).
```

--------------------------------

### Text Embedding Pipeline

Source: https://github.com/koaning/embetter/blob/main/docs/index.md

Demonstrates creating a scikit-learn pipeline for text embedding using Sentence-Transformers. It includes grabbing text from a pandas DataFrame column and encoding it. The pipeline can be used for feature extraction or integrated into a classifier.

```python
import pandas as pd
from sklearn.pipeline import make_pipeline 
from sklearn.linear_model import LogisticRegression

from embetter.grab import ColumnGrabber
from embetter.text import SentenceEncoder

# This pipeline grabs the `text` column from a dataframe
# which then get fed into Sentence-Transformers' all-MiniLM-L6-v2.
text_emb_pipeline = make_pipeline(
  ColumnGrabber("text"),
  SentenceEncoder('all-MiniLM-L6-v2')
)

# This pipeline can also be trained to make predictions, using
# the embedded features. 
text_clf_pipeline = make_pipeline(
  text_emb_pipeline,
  LogisticRegression()
)

dataf = pd.DataFrame({
  "text": ["positive sentiment", "super negative"],
  "label_col": ["pos", "neg"]
})
X = text_emb_pipeline.fit_transform(dataf, dataf['label_col'])
text_clf_pipeline.fit(dataf, dataf['label_col']).predict(dataf)
```

--------------------------------

### LiteDocEncoder API

Source: https://github.com/koaning/embetter/blob/main/docs/API/text.md

API documentation for the LiteDocEncoder class. This encoder is optimized for efficient document encoding, potentially with reduced computational overhead.

```APIDOC
embetter.text.LiteTextEncoder:
  An optimized encoder for lightweight document representation.
  Suitable for scenarios requiring fast encoding and lower memory footprint.
```

--------------------------------

### BytePairEncoder API

Source: https://github.com/koaning/embetter/blob/main/docs/API/text.md

API documentation for the BytePairEncoder class. This encoder likely uses Byte Pair Encoding (BPE) or a similar subword tokenization strategy for text representation.

```APIDOC
embetter.text.BytePairEncoder:
  Encodes text using subword tokenization, such as Byte Pair Encoding.
  Handles out-of-vocabulary words effectively by breaking them into known subword units.
```

--------------------------------

### Retrieve Cached Embeddings from DiskCache

Source: https://github.com/koaning/embetter/blob/main/docs/applications.md

Shows how to directly access pre-calculated embeddings stored in a `diskcache` instance. It's crucial to use the same cache name as specified in the `cached` function. The key is typically a string representing the input text.

```python
from diskcache import Cache

# Make sure that you use the same name as in `cached`
cache = Cache("sentence-enc")
# Use a string as a key, if it's precalculated you'll get an array back.
cache["this is a pretty long text, which is more expensive 0"]
```

--------------------------------

### GensimEncoder API

Source: https://github.com/koaning/embetter/blob/main/docs/API/text.md

API documentation for the GensimEncoder class. This encoder integrates with the Gensim library, allowing the use of its various topic modeling and embedding algorithms (e.g., Word2Vec, Doc2Vec).

```APIDOC
embetter.text.GensimEncoder:
  Leverages Gensim library for text embedding generation.
  Supports models like Word2Vec, Doc2Vec, and FastText for creating vector representations.
```

--------------------------------

### spaCyEncoder API

Source: https://github.com/koaning/embetter/blob/main/docs/API/text.md

API documentation for the spaCyEncoder class. This encoder uses the spaCy library to generate embeddings, leveraging its efficient NLP pipelines.

```APIDOC
embetter.text.spaCyEncoder:
  Encodes text using spaCy's built-in word or document vectors.
  Leverages spaCy's optimized NLP processing for embedding generation.
```

--------------------------------

### KerasNLPEncoder API

Source: https://github.com/koaning/embetter/blob/main/docs/API/text.md

API documentation for the KerasNLPEncoder class. This encoder integrates with the KerasNLP library to leverage its text processing and embedding capabilities.

```APIDOC
embetter.text.KerasNLPEncoder:
  Utilizes KerasNLP models for text encoding.
  Provides access to a wide range of pre-trained text embedding models within the Keras ecosystem.
```

--------------------------------

### Sense2VecEncoder API

Source: https://github.com/koaning/embetter/blob/main/docs/API/text.md

API documentation for the Sense2VecEncoder class. This encoder is based on the Sense2Vec model, which captures word senses and their contextual meanings.

```APIDOC
embetter.text.Sense2VecEncoder:
  Generates embeddings using the Sense2Vec model.
  Captures semantic meaning and word senses for richer representations.
```

--------------------------------

### MatryoshkaEncoder API

Source: https://github.com/koaning/embetter/blob/main/docs/API/text.md

API documentation for the MatryoshkaEncoder class. This encoder implements Matryoshka embeddings, which are hierarchical and can be truncated to different dimensions.

```APIDOC
embetter.text.MatryoshkaEncoder:
  Generates Matryoshka embeddings for text.
  Allows for flexible embedding dimensions by truncating the full embedding vector.
```

--------------------------------

### SentenceEncoder API

Source: https://github.com/koaning/embetter/blob/main/docs/API/text.md

API documentation for the SentenceEncoder class. This encoder is designed to generate embeddings for sentences, typically using pre-trained models.

```APIDOC
embetter.text.SentenceEncoder:
  Encodes sentences into fixed-size embedding vectors.
  Often utilizes transformer-based models for semantic representation.
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.