### Install stream2sentence

Source: https://github.com/koljab/stream2sentence/blob/master/README.md

Install the library via pip.

```bash
pip install stream2sentence
```

--------------------------------

### Process German Text with NLTK

Source: https://context7.com/koljab/stream2sentence/llms.txt

Demonstrates processing German text to extract sentences using NLTK. Ensure NLTK is installed and configured for German.

```python
german_text = "Hallo, wie geht es dir? Mir geht es gut."
german_sentences = list(generate_sentences(
    (chunk for chunk in german_text),  # Convert to generator
    minimum_sentence_length=3,
    context_size=5,
    minimum_first_fragment_length=3,
    quick_yield_single_sentence_fragment=True
))
print(german_sentences)
```

--------------------------------

### Handle Tricky English Sentences

Source: https://context7.com/koljab/stream2sentence/llms.txt

Processes English text containing abbreviations and decimals to correctly identify sentence boundaries. No special setup is required beyond importing the function.

```python
tricky_text = "Good muffins cost $3.88 in New York. I called Dr. Jones."
sentences = list(generate_sentences(tricky_text))
print(sentences)
```

--------------------------------

### Quick Yield Every Fragment

Source: https://context7.com/koljab/stream2sentence/llms.txt

Provides the most granular output by yielding every possible fragment at each delimiter. Use `quick_yield_every_fragment=True` for maximum responsiveness.

```python
result = list(generate_sentences(
    text,
    quick_yield_every_fragment=True,
    minimum_first_fragment_length=5,
    minimum_sentence_length=5
))
print("Every fragment:", result)
```

--------------------------------

### Configuration Parameters

Source: https://github.com/koljab/stream2sentence/blob/master/README.md

Configuration options for fine-tuning the sentence processing behavior.

```APIDOC
## Configuration Parameters

### Description
Settings to control how text is processed into fragments and sentences.

### Parameters
- **log_characters** (bool) - Optional - When True, logs each processed character to the console. Default: False.
- **sentence_fragment_delimiters** (str) - Optional - Characters considered as potential sentence fragment delimiters. Default: ".?!;:,
…)]}。-".
- **full_sentence_delimiters** (str) - Optional - Characters considered as full sentence delimiters. Default: ".?!
…。".
- **force_first_fragment_after_words** (int) - Optional - Forces the yield of the first sentence fragment after this many words. Default: 15.
```

--------------------------------

### Basic generate_sentences Usage

Source: https://context7.com/koljab/stream2sentence/llms.txt

Demonstrates basic usage of generate_sentences with a text string and a generator yielding text chunks. Ensure the input is iterable.

```python
from stream2sentence import generate_sentences

# Basic usage with a text string (strings are iterable)
text = "This is a test. This is another test sentence. Just testing out the module."
for sentence in generate_sentences(text):
    print(sentence)
# Output:
# This is a test.
# This is another test sentence.
# Just testing out the module.

# Using a generator for streaming text chunks
def text_chunk_generator():
    yield "Hello, world! "
    yield "How are you doing today? "
    yield "I hope everything is going well."

for sentence in generate_sentences(text_chunk_generator()):
    print(sentence)
# Output:
# Hello, world!
# How are you doing today?
# I hope everything is going well.
```

--------------------------------

### Quick Yield First Fragment of All Sentences

Source: https://context7.com/koljab/stream2sentence/llms.txt

Yields the first fragment of every sentence as soon as it's available, improving responsiveness. Set `quick_yield_for_all_sentences=True`.

```python
result = list(generate_sentences(
    text,
    quick_yield_for_all_sentences=True,
    minimum_first_fragment_length=5,
    minimum_sentence_length=10
))
print("All sentences:", result)
```

--------------------------------

### Quick Yield First Sentence Fragment

Source: https://context7.com/koljab/stream2sentence/llms.txt

Enables yielding the first fragment of a sentence early, useful for real-time applications. Requires `quick_yield_single_sentence_fragment=True` and `minimum_first_fragment_length`.

```python
result = list(generate_sentences(
    text,
    quick_yield_single_sentence_fragment=True,
    minimum_first_fragment_length=5,
    minimum_sentence_length=10
))
print("Single fragment:", result)
```

--------------------------------

### Pre-initializing Tokenizers with init_tokenizer

Source: https://context7.com/koljab/stream2sentence/llms.txt

Initialize NLTK or Stanza tokenizers in advance to avoid delays during the first call to generate_sentences. Supports various configurations including language, debug mode, and offline usage.

```python
from stream2sentence import init_tokenizer, generate_sentences

# Pre-initialize NLTK tokenizer (downloads punkt_tab data if needed)
init_tokenizer("nltk", debug=True)

# Pre-initialize Stanza tokenizer for a specific language
init_tokenizer("stanza", language="en")

# For multilingual support with Stanza
init_tokenizer("stanza", language="multilingual")

# For Chinese text processing
init_tokenizer("stanza", language="zh")

# Offline mode (skip downloads, use cached models)
init_tokenizer("stanza", language="en", offline=True)

# Now use generate_sentences without initialization delay
text = "Hello world. This is a test."
sentences = list(generate_sentences(text, tokenizer="nltk"))
print(sentences)
# Output: ['Hello world.', 'This is a test.']
```

--------------------------------

### generate_sentences Quick Yield Mode

Source: https://context7.com/koljab/stream2sentence/llms.txt

Enables quick-yield mode for real-time applications, yielding the first sentence fragment as soon as possible. Configure minimum lengths for sentences and first fragments.

```python
from stream2sentence import generate_sentences

# Quick yield mode for real-time applications (e.g., TTS)
text = "First, this important point. Second, another key insight."
sentences = list(generate_sentences(
    text,
    quick_yield_single_sentence_fragment=True,  # Yield first fragment ASAP
    minimum_sentence_length=3,
    minimum_first_fragment_length=3
))
print(sentences)
# Output: ['First,', 'this important point.', 'Second, another key insight.']
```

--------------------------------

### Import Time-Based Sentence Generation

Source: https://github.com/koljab/stream2sentence/blob/master/README.md

Import the `generate_sentences_time_based` function for time-based sentence generation. This function takes a generator yielding text chunks and uses a target tokens per second to manage output timing.

```python
from stream2sentence.stream2sentence_time_based import generate_sentences_time_based
```

--------------------------------

### Multi-language Sentence Generation (Chinese)

Source: https://context7.com/koljab/stream2sentence/llms.txt

Process and split Chinese text into sentences using the Stanza tokenizer. Configure minimum sentence length and context size for accurate boundary detection.

```python
from stream2sentence import generate_sentences

# Chinese text processing with Stanza
chinese_text = "我喜欢读书。天气很好。我们去公园吧。今天是星期五。"
chinese_sentences = list(generate_sentences(
    chinese_text,
    minimum_sentence_length=2,
    context_size=2,
    tokenizer="stanza",
    language="zh"
))
print(chinese_sentences)
# Output: ['我喜欢读书。', '天气很好。', '我们去公园吧。', '今天是星期五。']
```

--------------------------------

### Default Sentence Generation

Source: https://context7.com/koljab/stream2sentence/llms.txt

Illustrates the default behavior of `generate_sentences` which yields only complete sentences meeting the minimum length criteria. Requires `minimum_sentence_length` to be set.

```python
from stream2sentence import generate_sentences

text = "First, this is important. Second, we need to consider this carefully. Third, let's conclude."

# Default behavior - yield complete sentences only
result = list(generate_sentences(text, minimum_sentence_length=10))
print("Default:", result)
```

--------------------------------

### generate_sentences with OpenAI Streaming

Source: https://context7.com/koljab/stream2sentence/llms.txt

Processes real-time streaming responses from OpenAI's chat completions API into sentences. This enables immediate sentence-by-sentence output for applications like TTS.

```python
from stream2sentence import generate_sentences
from openai import OpenAI

client = OpenAI()

def stream_from_openai(prompt: str):
    """Generator that yields text chunks from OpenAI streaming response"""
    stream = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}],
        stream=True,
    )
    for chunk in stream:
        if (text_chunk := chunk.choices[0].delta.content):
            yield text_chunk

# Process the LLM output stream into sentences
text_stream = stream_from_openai("Write a three-sentence story about a robot.")

for idx, sentence in enumerate(generate_sentences(
    text_stream,
    minimum_sentence_length=10,
    quick_yield_single_sentence_fragment=True,
    minimum_first_fragment_length=10
), start=1):
    print(f"Sentence {idx}: {sentence}")
    # Each sentence can be immediately sent to TTS engine

# Example output:
# Sentence 1: In a small workshop,
# Sentence 2: a curious robot named Bolt discovered an old music box.
# Sentence 3: As the melody played, Bolt learned that even machines could feel wonder.
```

--------------------------------

### Time-Based Sentence Generation

Source: https://context7.com/koljab/stream2sentence/llms.txt

Generates sentences from a text stream with output deadlines based on target tokens-per-second. Useful for real-time speech synthesis to prevent output lag.

```python
from stream2sentence.stream2sentence_time_based import generate_sentences_time_based
import time

def simulated_llm_stream(text, tokens_per_second=8):
    """Simulates LLM output at a specific token rate"""
    words = text.split()
    words_per_token = 0.75
    delay = 1 / (tokens_per_second * words_per_token)
    for word in words:
        time.sleep(delay)
        yield word + " "

text = """In 1996 Mr. Stewart hosted a short-lived talk show.
It aired on Sunday nights in the United Kingdom on BBC Two.
The show featured panelists from the UK and the United States."""

start_time = time.time()
for sentence in generate_sentences_time_based(
    simulated_llm_stream(text, tokens_per_second=8),
    target_tps=4,           # Target 4 tokens/sec (human speech rate)
    lead_time=1.0,          # Wait 1 second before first output
    min_output_lengths=[2, 3, 4],  # Min words: 2 for 1st, 3 for 2nd, 4+ for rest
    max_wait_for_fragments=[3, 2], # Max wait for fragments beyond deadline
    deadline_offsets_dynamic=[0.1], # Account for TTS generation time
):
    elapsed = time.time() - start_time
    print(f"[{elapsed:.1f}s] {sentence}")

# Output (timing varies based on system):
# [1.2s] In 1996 Mr. Stewart hosted a short-lived talk show.
# [2.8s] It aired on Sunday nights in the United Kingdom on BBC Two.
# [4.5s] The show featured panelists from the UK and the United States.
```

--------------------------------

### Force First Fragment After N Words

Source: https://context7.com/koljab/stream2sentence/llms.txt

Prevents long delays by forcing the first fragment output after a specified number of words, even without punctuation. Configure using `force_first_fragment_after_words`.

```python
long_text = "This is a very long opening sentence without any punctuation that would normally cause a significant delay before any output is produced"
result = list(generate_sentences(
    long_text,
    quick_yield_single_sentence_fragment=True,
    force_first_fragment_after_words=10,
    minimum_first_fragment_length=5
))
print("Forced fragment:", result)
```

--------------------------------

### Incremental Sentence Splitting with SentenceSplitter

Source: https://context7.com/koljab/stream2sentence/llms.txt

Use the SentenceSplitter class for stateful, incremental sentence splitting. It's ideal for event-driven architectures and provides fine-grained control over buffering. Supports custom configurations for context size, minimum lengths, and text cleanup.

```python
from stream2sentence import SentenceSplitter

# Create a sentence splitter with custom configuration
splitter = SentenceSplitter(
    context_size=12,
    minimum_sentence_length=10,
    minimum_first_fragment_length=10,
    quick_yield_single_sentence_fragment=True,
    cleanup_text_emojis=True,
    tokenizer="nltk",
    language="en"
)

# Simulate receiving text chunks from a stream
chunks = [
    "The weather ",
    "is beautiful today. ",
    "Let's go ",
    "for a walk in the park. ",
    "Don't forget your sunscreen!"
]

# Process chunks incrementally
all_sentences = []
for chunk in chunks:
    splitter.add(chunk)
    for sentence in splitter.stream():
        all_sentences.append(sentence)
        print(f"Streamed: {sentence}")

# Flush any remaining text in the buffer
for sentence in splitter.flush():
    all_sentences.append(sentence)
    print(f"Flushed: {sentence}")

print(f"\nAll sentences: {all_sentences}")
# Output:
# Streamed: The weather is beautiful today.
# Streamed: Let's go for a walk in the park.
# Flushed: Don't forget your sunscreen!
# All sentences: ['The weather is beautiful today.', "Let's go for a walk in the park.", "Don't forget your sunscreen!"]
```

--------------------------------

### Async generate_sentences_async Usage

Source: https://context7.com/koljab/stream2sentence/llms.txt

Provides an asynchronous version of generate_sentences for use with async generators, preventing blocking of the event loop. Ideal for async API responses.

```python
import asyncio
from stream2sentence import generate_sentences_async

async def async_text_stream():
    """Simulates an async text stream (e.g., from an LLM API)"""
    chunks = ["The quick ", "brown fox ", "jumps over ", "the lazy dog. ",
              "Pack my box ", "with five dozen ", "liquor jugs."]
    for chunk in chunks:
        await asyncio.sleep(0.1)  # Simulate network delay
        yield chunk

async def process_stream():
    sentences = []
    async for sentence in generate_sentences_async(
        async_text_stream(),
        minimum_sentence_length=10,
        context_size=12
    ):
        sentences.append(sentence)
        print(f"Received: {sentence}")
    return sentences

# Run the async processing
result = asyncio.run(process_stream())
# Output:
# Received: The quick brown fox jumps over the lazy dog.
# Received: Pack my box with five dozen liquor jugs.
```

--------------------------------

### Generate Sentences from Stream

Source: https://github.com/koljab/stream2sentence/blob/master/README.md

Pass a generator of text chunks to generate_sentences to receive a generator of processed sentences.

```python
from stream2sentence import generate_sentences

# Dummy generator for demonstration
def dummy_generator():
    yield "This is a sentence. And here's another! Yet, "
    yield "there's more. This ends now."

for sentence in generate_sentences(dummy_generator()):
    print(sentence)
```

--------------------------------

### Time-based Sentence Generation

Source: https://github.com/koljab/stream2sentence/blob/master/README.md

API for the time-based strategy which yields output based on a target tokens per second (tps) rate.

```APIDOC
## generate_sentences_time_based

### Description
Yields the best available output (full sentence, longest fragment, or buffer) based on a target tokens per second (tps) deadline.

### Parameters
- **generator** (Iterator[str]) - Required - A generator that yields chunks of text as a stream of characters.
- **target_tps** (float) - Optional - The rate in tokens per second to calculate deadlines. Default: 4.
- **lead_time** (float) - Optional - Time in seconds to wait for the buffer to build before returning values. Default: 1.
- **max_wait_for_fragments** (list[int]) - Optional - Max time in seconds the Nth sentence will wait beyond the deadline for a fragment.
- **min_output_lengths** (list[int]) - Optional - Minimum output size in words for the corresponding output sentence.
- **preferred_sentence_fragment_delimiters** (list[str]) - Optional - Delimiters checked first for fragments.
- **sentence_fragment_delimiters** (list[str]) - Optional - Delimiters checked after preferred delimiters.
- **delimiter_ignore_prefixes** (list[str]) - Optional - Prefixes to ignore when identifying delimiters.
- **wait_for_if_non_fragment** (list[str]) - Optional - Strings to avoid as the last value if the whole buffer is output.
- **deadline_offsets_static** (list[float]) - Optional - Constant time in seconds to subtract from the deadline.
- **deadline_offsets_dynamic** (list[float]) - Optional - Time added to account for TTS generation latency.
```

--------------------------------

### stream2sentence - Real-Time Sentence Detection

Source: https://github.com/koljab/stream2sentence/blob/master/README.md

The stream2sentence library processes a generator of text chunks to detect and yield sentences in real-time. It offers various configuration options to balance speed and accuracy, clean output, and customize tokenization.

```APIDOC
## stream2sentence - Real-Time Sentence Detection

This library processes a generator of characters or text chunks to detect and yield sentences in real-time. It's particularly useful for applications like text-to-speech synthesis where immediate sentence fragments are needed.

### Installation

```bash
pip install stream2sentence
```

### Usage

Pass a generator of characters or text chunks to `generate_sentences()` to get a generator of sentences in return.

```python
from stream2sentence import generate_sentences

# Dummy generator for demonstration
def dummy_generator():
    yield "This is a sentence. And here's another! Yet, "
    yield "there's more. This ends now."

for sentence in generate_sentences(dummy_generator()):
    print(sentence)
```

### Configuration Parameters for `generate_sentences()`

#### Core Parameters

- **generator** (Iterator[str]) - Required - The primary input source, yielding chunks of text to be processed.
- **context_size** (int) - Optional - Number of characters considered for sentence boundary detection. Default: 12.
- **context_size_look_overhead** (int) - Optional - Additional characters to examine beyond `context_size` for sentence splitting. Default: 12.
- **minimum_sentence_length** (int) - Optional - Minimum character count for a text chunk to be considered a sentence. Shorter fragments are buffered. Default: 10.
- **minimum_first_fragment_length** (int) - Optional - Minimum character count required for the first sentence fragment. Default: 10.

#### Yield Control

- **quick_yield_single_sentence_fragment** (bool) - Optional - When True, yields the first fragment of the first sentence as quickly as possible. Default: False.
- **quick_yield_for_all_sentences** (bool) - Optional - When True, yields the first fragment of every sentence as quickly as possible. Automatically sets `quick_yield_single_sentence_fragment` to True. Default: False.
- **quick_yield_every_fragment** (bool) - Optional - When True, yields every fragment of every sentence as quickly as possible. Automatically sets `quick_yield_for_all_sentences` and `quick_yield_single_sentence_fragment` to True. Default: False.

#### Text Cleanup

- **cleanup_text_links** (bool) - Optional - When True, removes hyperlinks from the output sentences. Default: False.
- **cleanup_text_emojis** (bool) - Optional - When True, removes emoji characters from the output sentences. Default: False.

#### Tokenization

- **tokenize_sentences** (Callable) - Optional - Custom function for sentence tokenization. If None, uses the default tokenizer specified by `tokenizer`. Default: None.
- **tokenizer** (str) - Optional - Specifies the tokenizer to use. Options: "nltk" or "stanza". Default: "nltk".
- **language** (str) - Optional - Language setting for the tokenizer. Use "en" for English or "multilingual" for Stanza tokenizer. Default: "en".
```

--------------------------------

### generate_sentences Text Cleanup

Source: https://context7.com/koljab/stream2sentence/llms.txt

Utilizes text cleanup options to remove links and emojis from the input stream before sentence detection. Set `cleanup_text_links` and `cleanup_text_emojis` to True.

```python
from stream2sentence import generate_sentences

# Text cleanup - remove links and emojis
messy_text = "Check out https://example.com for more info! 😀 Have a great day."
clean_sentences = list(generate_sentences(
    messy_text,
    cleanup_text_links=True,
    cleanup_text_emojis=True
))
print(clean_sentences)
# Output: ['Check out  for more info!', 'Have a great day.']
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.