Turftopic

https://x-tabdeveloping.github.io/turftopic/
Admin
Turftopic is a topic modeling library that provides stable, minimal, and scalable implementations of...
Tokens:152,196
Snippets:602
Trust Score:8.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### Install Turftopic using pip

Source: https://x-tabdeveloping.github.io/turftopic/index

This command installs the Turftopic library from the Python Package Index (PyPI). Ensure you have pip installed and accessible in your environment.

```bash
pip install turftopic

```

--------------------------------

### FASTopic Model Initialization and Usage

Source: https://x-tabdeveloping.github.io/turftopic/FASTopic

This section details how to install the turftopic library with the necessary dependencies for FASTopic, and provides a basic Python code example for initializing and fitting the FASTopic model to a corpus.

```APIDOC
## FASTopic Model

### Description
Implementation of the FASTopic model with a Turftopic API. The implementation is based on the original FASTopic package, but is adapted for optimal use in Turftopic. You will need to install torch to use this model.

### Installation
```bash
pip install turftopic[torch]
## OR:
pip install turftopic[pyro-ppl]
```

### Usage Example
```python
from turftopic import FASTopic

corpus: list[str] = ["some text", "more text", ...]

model = FASTopic(n_components=10).fit(corpus)
model.print_topics()
```

### Parameters

**`n_components`** (int) - Required - Number of topics. If you're using priors on the weight, feel free to overshoot with this value.

**`encoder`** (Union[Encoder, str], default: `'sentence-transformers/all-MiniLM-L6-v2'`) - Model to encode documents/terms, all-MiniLM-L6-v2 is the default.

**`vectorizer`** (Optional[CountVectorizer], default: `None`) - Vectorizer used for term extraction. Can be used to prune or filter the vocabulary.

**`random_state`** (Optional[int], default: `None`) - Random state to use so that results are exactly reproducible.

**`batch_size`** (Optional[int], default: `None`)

**`DT_alpha`** (float, default: `3.0`) - Sinkhorn alpha between document embeddings and topic embeddings.

**`TW_alpha`** (float, default: `2.0`) - Sinkhorn alpha between topic embeddings and word embeddings.

**`theta_temp`** (float, default: `1.0`) - Temperature parameter of used in softmax to compute topic probabilities in documents.

**`n_epochs`** (int, default: `200`) - Number of epochs to train the model for.

**`learning_rate`** (float, default: `0.002`) - Learning rate for the ADAM optimizer.

**`device`** (str, default: `'cpu'`) - Device to run the model on. Defaults to CPU.

### Source Code
Source code in `turftopic/models/fastopic.py`

```

--------------------------------

### Basic KeyNMF Topic Modeling with Turftopic and Scikit-learn

Source: https://x-tabdeveloping.github.io/turftopic/index

This Python code demonstrates the basic usage of Turftopic's KeyNMF model. It fetches the 20Newsgroups dataset using scikit-learn, trains the KeyNMF model on the corpus, and then prints the identified topics. This example assumes familiarity with scikit-learn conventions.

```python
from turftopic import KeyNMF
from sklearn.datasets import fetch_20newsgroups

newsgroups = fetch_20newsgroups(
    subset="all",
    remove=("headers", "footers", "quotes"),
)
corspus = newsgroups.data
model = KeyNMF(20).fit(corpus)
model.print_topics()

```

--------------------------------

### Install FASTopic with PyTorch Support

Source: https://x-tabdeveloping.github.io/turftopic/FASTopic

This command installs the turftopic library with PyTorch support, which is a dependency for using the FASTopic model. Ensure you have pip installed.

```bash
pip install turftopic[torch]
```

--------------------------------

### Install Turftopic and Dependencies

Source: https://x-tabdeveloping.github.io/turftopic/tutorials/arxiv_ml

Installs the `turftopic` library along with `datasets` and `plotly`. It also installs optional dependencies for UMAP-based clustering and datamapplot visualization. This command is executed in the terminal.

```bash
pip install datasets plotly turftopic[umap-learn, datamapplot]
```

--------------------------------

### Install turftopic with FASTopic support

Source: https://x-tabdeveloping.github.io/turftopic/FASTopic

To use the FASTopic model, you need to install the turftopic library with the appropriate optional dependencies. You can choose between installing with PyTorch or Pyro-PP-L support. This command installs the library and its required packages for FASTopic.

```bash
pip install turftopic[torch]
## OR:
pip install turftopic[pyro-ppl]
```

--------------------------------

### Install topic-wizard (Shell)

Source: https://x-tabdeveloping.github.io/turftopic/model_interpretation

Command to install the topic-wizard library, a tool for interactive topic model visualization.

```bash
pip install topic-wizard
```

--------------------------------

### Install Libraries with Pip

Source: https://x-tabdeveloping.github.io/turftopic/tutorials/ideologies

Installs necessary Python libraries including Turftopic, Plotly, datasets, and pandas for data manipulation and visualization. These are essential for running the tutorial's code.

```shell
pip install datasets plotly pandas turftopic

```

--------------------------------

### Install datamapplot with Turftopic (Shell)

Source: https://x-tabdeveloping.github.io/turftopic/model_interpretation

Command to install the datamapplot library along with Turftopic, enabling interactive cluster exploration.

```bash
pip install turftopic[datamapplot]
```

--------------------------------

### Install Turftopic and Dependencies

Source: https://x-tabdeveloping.github.io/turftopic/tutorials/reviews

Installs the turftopic library with spaCy support, along with plotly and pandas for data handling and visualization. It also downloads a small English spaCy pipeline for phrase extraction.

```shell
pip install turftopic[spacy] plotly pandas
python -m spacy download en_core_web_sm
```

--------------------------------

### Installing openTSNE for Turftopic

Source: https://x-tabdeveloping.github.io/turftopic/clustering

Instructions to install the openTSNE package, which can significantly speed up clustering topic models when used with Turftopic.

```bash
pip install turftopic[opentsne]
```

--------------------------------

### Install spaCy and Download Model

Source: https://x-tabdeveloping.github.io/turftopic/model_definition_and_training

Installs the Turftopic package with spaCy support and downloads a small English language model for spaCy. This is a prerequisite for using spaCy-based vectorizers.

```bash
pip install turftopic[spacy]
python -m spacy download "en_core_web_sm"
```

--------------------------------

### Installing UMAP for Turftopic

Source: https://x-tabdeveloping.github.io/turftopic/clustering

Instructions to install the umap-learn package, required for using UMAP as a dimensionality reduction method in Turftopic.

```bash
pip install umap-learn
```

--------------------------------

### Install Turftopic with Jieba for Chinese Tokenization

Source: https://x-tabdeveloping.github.io/turftopic/vectorizers

This command installs the Turftopic package with the necessary Jieba dependency for handling Chinese text tokenization. Ensure you have pip installed.

```bash
pip install turftopic[jieba]
```

--------------------------------

### Install MTEB Library

Source: https://x-tabdeveloping.github.io/turftopic/multimodal

This command installs the MTEB (Massive Text Embedding Benchmark) library, which is often used to access and manage embedding models. It's recommended to install version 1.x.x as specified by 'mteb<2.0.0' for compatibility with Turftopic's multimodal features.

```shell
pip install "mteb<2.0.0"
```

--------------------------------

### Install Turftopic with Topic-Wizard

Source: https://x-tabdeveloping.github.io/turftopic/tutorials/religious

Installs the turftopic library along with the topic-wizard extra, which likely provides additional functionalities for topic modeling visualizations or utilities. This command is executed in a shell environment.

```bash
pip install turftopic[topic-wizard]
```

--------------------------------

### Install OpenAI Package and Set API Key

Source: https://x-tabdeveloping.github.io/turftopic/model_definition_and_training

Installs the OpenAI Python package and sets the OpenAI API key as an environment variable. This is necessary for using the OpenAIAnalyzer.

```bash
pip install openai
export OPENAI_API_KEY="sk-<your key goes here>"
```

--------------------------------

### Install OpenAI and Set API Key for Turftopic

Source: https://x-tabdeveloping.github.io/turftopic/analyzers

This section details the necessary steps to integrate OpenAI's API with Turftopic. It involves installing the turftopic[openai] package and setting the OPENAI_API_KEY environment variable. This allows Turftopic to utilize OpenAI's language models for analysis tasks.

```bash
pip install turftopic[openai]
export OPENAI_API_KEY="sk-<your key goes here>"
```

--------------------------------

### Install Plotly for Plotting

Source: https://x-tabdeveloping.github.io/turftopic/dynamic

Installs the Plotly library, which is required for visualizations in Turftopic's dynamic topic modeling features.

```shell
pip install plotly

```

--------------------------------

### Install Turftopic with Pyro Support

Source: https://x-tabdeveloping.github.io/turftopic/ctm

This command installs the Turftopic library with the necessary dependencies for using Pyro-based probabilistic models, specifically for Variational Autoencoding Topic Models.

```shell
pip install turftopic[pyro-ppl]
```

--------------------------------

### Install ONNX Backend for Sentence Transformers (Shell)

Source: https://x-tabdeveloping.github.io/turftopic/encoders

Command to install the Sentence Transformers library with the necessary dependencies for using the ONNX backend, which can significantly speed up model inference for compatible models.

```shell
pip install sentence-transformers[onnx, onnx-gpu]
```

--------------------------------

### Install Turftopic with UMAP-learn Support

Source: https://x-tabdeveloping.github.io/turftopic/clustering

Installs the turftopic library with the necessary dependencies for UMAP-learn, which is often used for dimensionality reduction in topic modeling.

```bash
pip install turftopic[umap-learn]
```

--------------------------------

### Install and Use LemmaCountVectorizer

Source: https://x-tabdeveloping.github.io/turftopic/vectorizers

Installs Turftopic with SpaCy support and a SpaCy English pipeline, then demonstrates using LemmaCountVectorizer with KeyNMF for topic modeling based on lemmatized words.

```bash
pip install turftopic[spacy]
python -m spacy download en_core_web_sm

```

```python
from turftopic import KeyNMF
from turftopic.vectorizers.spacy import LemmaCountVectorizer

vectorizer = LemmaCountVectorizer("en_core_web_sm")
model = KeyNMF(10, vectorizer=vectorizer)
model.fit(corpus)
model.print_topics()

```

--------------------------------

### Visualize Topic Model with topicwizard (Python)

Source: https://x-tabdeveloping.github.io/turftopic/model_interpretation

Launches the topicwizard interactive visualization using a model object and a corpus. Requires the topic-wizard library to be installed.

```python
import topicwizard

topicwizard.visualize(corpus=documents, model=model)
```

--------------------------------

### Initialize KeyNMF Topic Model

Source: https://x-tabdeveloping.github.io/turftopic/model_definition_and_training

Initializes a KeyNMF topic model with a specified number of components and top N words per topic. This is a common starting point for topic modeling.

```python
from turftopic import KeyNMF

model = KeyNMF(n_components=10, top_n=15)
```

--------------------------------

### Use LemmaCountVectorizer with KeyNMF

Source: https://x-tabdeveloping.github.io/turftopic/model_definition_and_training

Initializes a KeyNMF model using LemmaCountVectorizer for topic extraction based on word lemmas. Requires spaCy installation and model download.

```python
from turftopic import KeyNMF
from turftopic.vectorizers.spacy import LemmaCountVectorizer

model = KeyNMF(10, vectorizer=LemmaCountVectorizer("en_core_web_sm"))
model.fit(corpus)
model.print_topics()
```

--------------------------------

### FASTopic Model API

Source: https://x-tabdeveloping.github.io/turftopic/FASTopic

This section details the FASTopic model implementation within Turftopic, which is based on the original FASTopic package and adapted for API usage. It requires PyTorch to be installed.

```APIDOC
## FASTopic Model Implementation

### Description
Implementation of the FASTopic model with a Turftopic API. The implementation is based on the original FASTopic package, but is adapted for optimal use in Turftopic (you can pre-compute embeddings for instance).

### Requirements
- You will need to install `torch` to use this model.
```
pip install turftopic[torch]
```
```

--------------------------------

### Get Time Slices

Source: https://x-tabdeveloping.github.io/turftopic/model_interpretation

Retrieves the start and end datetimes for each time slice available in the topic model. This is crucial for understanding the temporal granularity of the data.

```APIDOC
## GET /get_time_slices

### Description
Returns the starting and ending datetime of each timeslice in the model.

### Method
GET

### Endpoint
/get_time_slices

### Parameters
None

### Request Example
```json
{} 
```

### Response
#### Success Response (200)
- **list[tuple[datetime, datetime]]** - A list of tuples, where each tuple contains the start and end datetime of a time slice.

#### Response Example
```json
[
  ["2023-01-01T00:00:00", "2023-02-01T00:00:00"],
  ["2023-02-01T00:00:00", "2023-03-01T00:00:00"]
]
```
```

--------------------------------

### Semi-supervised Modeling with UMAP

Source: https://x-tabdeveloping.github.io/turftopic/clustering

Illustrates the setup for semi-supervised topic modeling using UMAP as the dimensionality reduction method. It requires the 'turftopic[umap-learn]' installation and a corpus with potential class labels.

```python
from umap import UMAP
from turftopic import ClusteringTopicModel

corpus: list[str] = [...] # Assume corpus is defined

# UMAP can also understand missing class labels if you only have them on some examples
```

--------------------------------

### Get Time Slices from turftopic

Source: https://x-tabdeveloping.github.io/turftopic/model_interpretation

Returns the starting and ending datetime of each time slice in the model. This function is applicable to dynamic topic models. It raises an AttributeError if the model is not dynamic and lacks the 'time_bin_edges' attribute.

```python
def get_time_slices(self) -> list[tuple[datetime, datetime]]:
    """Returns starting and ending datetime of
    each timeslice in the model."""
    bins = getattr(self, "time_bin_edges", None)
    if bins is None:
        raise AttributeError(
            "Topic model is not dynamic, time_bin_edges attribute is missing."
        )
    res = []
    for i_bin, slice_end in enumerate(bins[1:]):
        res.append((bins[i_bin], slice_end))
    return res

```

--------------------------------

### Get Time Slice Datetimes (Python)

Source: https://x-tabdeveloping.github.io/turftopic/model_interpretation

This function returns a list of tuples, where each tuple represents the start and end datetime of a time slice in a dynamic topic model. It requires the 'time_bin_edges' attribute to be present. If not found, it raises an AttributeError.

```python
bins = getattr(self, "time_bin_edges", None)
        if bins is None:
            raise AttributeError(
                "Topic model is not dynamic, time_bin_edges attribute is missing."
            )
        res = []
        for i_bin, slice_end in enumerate(bins[1:]):
            res.append((bins[i_bin], slice_end))
        return res
```

--------------------------------

### Multilingual Topic Modeling with GMM

Source: https://x-tabdeveloping.github.io/turftopic/cross_lingual

Provides an example of initializing the GMM (Gaussian Mixture Model) for cross-lingual topic modeling. It requires a multilingual encoder, such as 'paraphrase-multilingual-MiniLM-L12-v2', to ensure proper embedding generation across different languages. This setup is suitable for clustering documents in a multilingual context.

```python
from turftopic import GMM

model = GMM(encoder="paraphrase-multilingual-MiniLM-L12-v2")
```

--------------------------------

### Initialize and Prepare Topic Model Data (Python)

Source: https://x-tabdeveloping.github.io/turftopic/model_interpretation

Initializes a KeyNMF topic model and prepares the topic data from a corpus. This is a foundational step for subsequent interpretation and visualization.

```python
from turftopic import KeyNMF

model = KeyNMF(10)
topic_data = model.prepare_topic_data(corpus)
```

--------------------------------

### Instantiate and Fit KeyNMF Model with Corpus

Source: https://x-tabdeveloping.github.io/turftopic/KeyNMF

Demonstrates how to initialize the KeyNMF model with a specified number of components and top keywords, then fit it to a given corpus of text documents. Requires the 'turftopic' library. Input is a list of strings, output is a fitted model object.

```python
from turftopic import KeyNMF

corpus: list[str] = ["some text", "more text", ...]

model = KeyNMF(10, top_n=10).fit(corpus)
model.print_topics()
```

--------------------------------

### Load and Subsample ML Papers Dataset

Source: https://x-tabdeveloping.github.io/turftopic/tutorials/arxiv_ml

Loads the ML ArXiv Papers dataset from HuggingFace Hub using the `datasets` library and then subsamples it to 10,000 examples for faster processing. The abstracts are extracted into a list.

```python
from datasets import load_dataset

ds = load_dataset("CShorten/ML-ArXiv-Papers", split="train")
# Subsampling dataset
ds = ds.train_test_split(seed=42, test_size=10_000)["test"]
abstracts = ds["abstract"]

```

--------------------------------

### Use FASTopic model in Python

Source: https://x-tabdeveloping.github.io/turftopic/FASTopic

This Python snippet demonstrates how to use the FASTopic model from the turftopic library. It shows initializing the FASTopic model with a specified number of components, fitting it to a corpus of text, and then printing the discovered topics. Ensure you have installed turftopic with the necessary dependencies.

```python
from turftopic import FASTopic

corpus: list[str] = ["some text", "more text", ...]

model = FASTopic(10).fit(corpus)
model.print_topics()
```

--------------------------------

### Noun Phrase Features with NounPhraseCountVectorizer

Source: https://x-tabdeveloping.github.io/turftopic/vectorizers

This example shows how to use Turftopic's `NounPhraseCountVectorizer` for extracting noun phrases as features, leveraging the SpaCy package. It requires installing `turftopic[spacy]` and a SpaCy language pipeline (e.g., `en_core_web_sm`). The snippet initializes `KeyNMF` with the `NounPhraseCountVectorizer`, specifying the SpaCy model. The model is then fitted to the corpus and its topics are printed. While this method can be slower due to SpaCy's processing, it may produce more accurate and higher-quality results.

```python
from turftopic import KeyNMF
from turftopic.vectorizers.spacy import NounPhraseCountVectorizer

model = KeyNMF(
    n_components=10,
    vectorizer=NounPhraseCountVectorizer("en_core_web_sm"),
)
model.fit(corpus)
model.print_topics()
```

--------------------------------

### Finetuning a Pretrained KeyNMF Model

Source: https://x-tabdeveloping.github.io/turftopic/online

Illustrates how to load a pre-trained KeyNMF model and finetune it on a new, unseen corpus using `partial_fit`. This allows adapting existing topic models to novel data and saves the finetuned model to disk.

```python
from turftopic import load_model

model = load_model("pretrained_keynmf_model")

new_corpus: list[str] = [...]
# Finetune the model to the new corpus
model.partial_fit(new_corpus)

model.to_disk("finetuned_model/")

```

--------------------------------

### Initialize OpenAIAnalyzer for Topic Analysis

Source: https://x-tabdeveloping.github.io/turftopic/model_definition_and_training

Initializes an OpenAIAnalyzer for generating topic names and descriptions using OpenAI's models. Requires the OpenAI package and API key to be set up.

```python
from turftopic.analyzers import OpenAIAnalyzer


```

--------------------------------

### OpenAIAnalyzer Class Initialization and Text Generation

Source: https://x-tabdeveloping.github.io/turftopic/analyzers

Initializes the OpenAIAnalyzer with various prompts and model configurations and provides a method to generate text using the OpenAI API. It requires the 'openai' library and takes parameters like model name, context, and specific prompts for summarization, naming, and descriptions.

```python
import openai
from typing import Optional

class OpenAIAnalyzer(Analyzer):
    """Analyze topic model with an OpenAI LLM.

    Parameters
    ----------
    model_name: str, default 'gpt-5-nano'
        OpenAI model to use.
    use_summaries: bool, default False
        Indicates whether the language model should summarize documents before
        analyzing the topics.
    context: str, default None
        Additional context provided to the analyzer for analysis.
        e.g. "Analyze topics from blog posts related to morality and religion"
    system_prompt: str, default None
        System prompt to use for the language model.
    summary_prompt: str, default None
        Prompt to use for abstractive summarization.
    namer_prompt: str, default None
        Prompt template for naming topics.
    description_prompt: str, default None
        Prompt template for generating topic descriptions.
    """

    def __init__(
        self,
        model_name: str = "gpt-5-nano",
        context: Optional[str] = None,
        use_summaries: bool = False,
        system_prompt: Optional[str] = None,
        summary_prompt: Optional[str] = None,
        namer_prompt: Optional[str] = None,
        description_prompt: Optional[str] = None,
    ):
        self.client = openai.OpenAI()
        self.model_name = model_name
        self.system_prompt = system_prompt or self.system_prompt
        self.summary_prompt = summary_prompt or self.summary_prompt
        self.namer_prompt = namer_prompt or self.namer_prompt
        self.description_prompt = description_prompt or self.description_prompt
        self.use_summaries = use_summaries

    def generate_text(self, prompt: str) -> str:
        messages = [
            {"role": "system", "content": self.system_prompt},
            {"role": "user", "content": prompt},
        ]
        response = self.client.chat.completions.create(
            messages=messages,
            model=self.model_name,
        )
        return response.choices[0].message.content

```

--------------------------------

### Dynamic Online Topic Modeling with KeyNMF

Source: https://x-tabdeveloping.github.io/turftopic/KeyNMF

Demonstrates how to perform dynamic online topic modeling using KeyNMF. This approach is suitable for large, time-evolving corpora. It requires predefining time bins and uses the `partial_fit_dynamic` method to update the model with text and corresponding timestamps. The example shows initialization of bins and then fitting the model in batches.

```python
from datetime import datetime
from turftopic import KeyNMF

# Assuming 'batched' function is defined elsewhere

# Define time bins (e.g., yearly from 2020 to 2030)
model = KeyNMF(5, top_n=10)
bins = [datetime(year=y, month=1, day=1) for y in range(2020, 2030 + 2, 1)]

corpus: list[str] = [...]  # Your list of documents
timestamps: list[datetime] = [...] # Corresponding timestamps

for batch in batched(zip(corpus, timestamps)):
    text_batch, ts_batch = zip(*batch)
    model.partial_fit_dynamic(text_batch, timestamps=ts_batch, bins=bins)
```

--------------------------------

### Install and Use KeyphraseCountVectorizer

Source: https://x-tabdeveloping.github.io/turftopic/vectorizers

Installs the keyphrase-vectorizers library and demonstrates its integration with Turftopic's KeyNMF model for extracting candidate keyphrases from text.

```bash
pip install keyphrase-vectorizers

```

```python
from keyphrase_vectorizers import KeyphraseCountVectorizer
from turftopic import KeyNMF

vectorizer = KeyphraseCountVectorizer()
model = KeyNMF(10, vectorizer=vectorizer).fit(corpus)

```

--------------------------------

### Fit and Interpret KeyNMF Model

Source: https://x-tabdeveloping.github.io/turftopic/KeyNMF

Demonstrates the basic usage of KeyNMF for topic modeling. This involves initializing the model with a specified number of topics and an encoder, fitting it to a corpus, and then printing the extracted topics. It assumes the 'turftopic' library is installed and the 'corpus' variable is available.

```python
from turftopic import KeyNMF

model = KeyNMF(10, encoder="paraphrase-MiniLM-L3-v2")
model.fit(corpus)

model.print_topics()

```

--------------------------------

### Install Plotly for Concept Compass

Source: https://x-tabdeveloping.github.io/turftopic/s3

Provides the command to install the plotly library, which is a dependency for generating concept compass visualizations in turftopic.

```bash
pip install plotly
```

--------------------------------

### Install and Use StemmingCountVectorizer

Source: https://x-tabdeveloping.github.io/turftopic/vectorizers

Installs Turftopic with Snowball Stemmer support and demonstrates using StemmingCountVectorizer with KeyNMF for topic modeling. This is useful when lemmatization is not aggressive enough.

```bash
pip install turftopic[snowball]

```

```python
from turftopic import KeyNMF
from turftopic.vectorizers.snowball import StemmingCountVectorizer

vectorizer = StemmingCountVectorizer(language="english")
model = KeyNMF(10, vectorizer=vectorizer)
model.fit(corpus)
model.print_topics()

```

--------------------------------

### Initialize LLMAnalyzer in Python

Source: https://x-tabdeveloping.github.io/turftopic/analyzers

Initializes the LLMAnalyzer with specified parameters. It loads the tokenizer and causal language model from Hugging Face Hub, configuring them for analysis. Key parameters include the model name, whether to use document summaries, additional context, custom prompts for summarization, naming, and description generation, the device for model execution, maximum new tokens for generation, and an option to enable thinking mode.

```python
class LLMAnalyzer(Analyzer):
    """Analyze topic model with an open LLM from HF Hub.

    Parameters
    ----------
    model_name: str, default 'HuggingFaceTB/SmolLM3-3B'
        Open LLM to use from HF Hub.
    use_summaries: bool, default False
        Indicates whether the language model should summarize documents before
        analyzing the topics.
    context: str, default None
        Additional context provided to the analyzer for analysis.
        e.g. "Analyze topics from blog posts related to morality and religion"
    system_prompt: str, default None
        Ignored, exists for compatibility
    summary_prompt: str, default None
        Prompt to use for abstractive summarization.
    namer_prompt: str, default None
        Prompt template for naming topics.
    description_prompt: str, default None
        Prompt template for generating topic descriptions.
    device: str, default "cpu"
        ID of the device to run the language model on.
    max_new_tokens: int, default 32768
        Max new tokens to generate when analyzing.
    enable_thinking: bool, default False
        Indicates whether thinking mode should be enabled.
    """

    def __init__(
        self,
        model_name: str = "HuggingFaceTB/SmolLM3-3B",
        context: Optional[str] = None,
        use_summaries: bool = False,
        system_prompt: Optional[str] = None,
        summary_prompt: Optional[str] = None,
        namer_prompt: Optional[str] = None,
        description_prompt: Optional[str] = None,
        max_new_tokens: int = 32768,
        device: str = "cpu",
        enable_thinking: bool = False,
    ):
        self.device = device
        self.model_name = model_name
        # load the tokenizer and the model
        self.tokenizer = AutoTokenizer.from_pretrained(self.model_name)
        self.model = AutoModelForCausalLM.from_pretrained(
            self.model_name,
        ).to(self.device)
        self.summary_prompt = summary_prompt or self.summary_prompt
        self.namer_prompt = namer_prompt or self.namer_prompt
        self.description_prompt = description_prompt or self.description_prompt
        self.use_summaries = use_summaries
        self.max_new_tokens = max_new_tokens
        self.enable_thinking = enable_thinking
```

--------------------------------

### T5Analyzer Initialization in Python

Source: https://x-tabdeveloping.github.io/turftopic/analyzers

Initializes the T5Analyzer with various parameters for text-to-text model analysis. It configures the model name, summarization options, custom prompts, and the device for execution. Dependencies include the 'pipeline' function from the transformers library.

```python
class T5Analyzer(Analyzer):
    """Analyze topic model with a text-to-text model.

    Parameters
    ----------
    model_name: str, default 'google/flan-t5-small'
        Text-to-text model to use for analyses.
    use_summaries: bool, default False
        Indicates whether the language model should summarize documents before
        analyzing the topics.
    context: str, default None
        Additional context provided to the analyzer for analysis.
        e.g. "Analyze topics from blog posts related to morality and religion"
    system_prompt: str, default None
        Ignored, exists for compatibility
    summary_prompt: str, default None
        Prompt to use for abstractive summarization.
    namer_prompt: str, default None
        Prompt template for naming topics.
    description_prompt: str, default None
        Prompt template for generating topic descriptions.
    device: str, default "cpu"
        ID of the device to run the language model on.
    """

    def __init__(
        self,
        model_name: str = "google/flan-t5-small",
        context: Optional[str] = None,
        use_summaries: bool = False,
        system_prompt: Optional[str] = None,
        summary_prompt: Optional[str] = None,
        namer_prompt: Optional[str] = T5_NAME_PROMPT,
        description_prompt: Optional[str] = T5_DESC_PROMPT,
        device: str = "cpu",
    ):
        self.device = device
        self.model_name = model_name
        self.pipeline = pipeline(
            task="text2text-generation",
            model=self.model_name,
            device=self.device,
        )
        self.summary_prompt = summary_prompt or self.summary_prompt
        self.namer_prompt = namer_prompt or self.namer_prompt
        self.description_prompt = description_prompt or self.description_prompt
        self.use_summaries = use_summaries

```

--------------------------------

### Initialize and Fit BERTopic Model - Python

Source: https://x-tabdeveloping.github.io/turftopic/clustering

This Python code demonstrates how to initialize and use the BERTopic model from the turftopic library. It includes fitting the model to a corpus of text and then printing the discovered topics. Ensure the turftopic library and its dependencies are installed.

```python
from turftopic import BERTopic

corpus: list[str] = ["some text", "more text", ...]

model = BERTopic().fit(corpus)
model.print_topics()
```

--------------------------------

### Initialize OpenAI Analyzer in Turftopic

Source: https://x-tabdeveloping.github.io/turftopic/analyzers

This code demonstrates how to initialize the OpenAIAnalyzer for use with Turftopic. It specifies the model to be used, defaulting to 'gpt-5-nano', which is highlighted as a cost-effective option that provides satisfactory results. This setup enables Turftopic to leverage OpenAI's powerful language models.

```python
from turftopic.analyzers import OpenAIAnalyzer

analyzer = OpenAIAnalyzer('gpt-5-nano')
```

--------------------------------

### Extract Topics to Pandas DataFrame

Source: https://x-tabdeveloping.github.io/turftopic/model_interpretation

This function extracts topics and their associated words into a pandas DataFrame. It requires the 'pandas' library to be installed. Users can control the number of top words per topic, whether to show scores, and whether to include negative terms. Raises ModuleNotFoundError if pandas is not installed.

```python
def topics_df(
        self,
        top_k: int = 10,
        show_scores: bool = False,
        show_negative: Optional[bool] = None,
    ):
    """Extracts topics into a pandas dataframe.

    Parameters
    ----------
    top_k: int, default 10
        Number of top words to return for each topic.
    show_scores: bool, default False
        Indicates whether to show importance scores for each word.
    show_negative: bool, default False
        Indicates whether the most negative terms should also be displayed.
    """
    try:
        import pandas as pd
    except ModuleNotFoundError:
        raise ModuleNotFoundError(
            "You need to pip install pandas to be able to use dataframes."
        )
    columns, *rows = self._topics_table(top_k, show_scores, show_negative)
    return pd.DataFrame(rows, columns=columns)
```

--------------------------------

### Get Vocabulary

Source: https://x-tabdeveloping.github.io/turftopic/KeyNMF

Retrieves the vocabulary (unique terms) learned by the topic model.

```APIDOC
## GET /topic-modeling/vocab

### Description
Retrieves the vocabulary (unique terms) learned by the topic model.

### Method
GET

### Endpoint
/topic-modeling/vocab

### Parameters
None

### Response
#### Success Response (200)
- **vocabulary** (array of strings) - An array containing all the unique terms in the model's vocabulary.

#### Response Example
```json
{
  "vocabulary": ["term1", "term2", "term3", "term4", "term5"]
}
```
```

--------------------------------

### ChineseCountVectorizer Usage

Source: https://x-tabdeveloping.github.io/turftopic/vectorizers

This section details how to install the necessary package and use the ChineseCountVectorizer for topic modeling with Chinese text.

```APIDOC
## Installing Turftopic with Jieba Support

To use the Chinese vectorizer, you need to install the `turftopic` package with the `jieba` extra.

### Command
```bash
pip install turftopic[jieba]
```

## ChineseCountVectorizer

The `ChineseCountVectorizer` is a specialized vectorizer for Chinese text that comes preloaded with the Jieba tokenizer and a Chinese stop word list. It is recommended for topic modeling with Chinese.

### Initialization

Initialize the `ChineseCountVectorizer` with desired parameters. `min_df` controls the minimum document frequency for terms, and `stop_words='chinese'` utilizes the built-in Chinese stop word list.

```python
from turftopic import KeyNMF
from turftopic.vectorizers.chinese import ChineseCountVectorizer

vectorizer = ChineseCountVectorizer(min_df=10, stop_words="chinese")
```

### Model Fitting and Topic Printing

After initializing the vectorizer, you can use it to fit a topic model like `KeyNMF` and then print the identified topics.

```python
# Assuming 'corpus' is your list of Chinese documents
model = KeyNMF(10, vectorizer=vectorizer, encoder="BAAI/bge-small-zh-v1.5")
model.fit(corpus)

model.print_topics()
```

### Example Topics

The following table shows an example of topics extracted using `ChineseCountVectorizer`:

| Topic ID | Highest Ranking  
---|---
| 0 | 消息, 时间, 科技, 媒体报道, 美国, 据, 国外, 讯, 宣布, 称  
| 1 | 体育讯, 新浪, 球员, 球队, 赛季, 火箭, nba, 已经, 主场, 时间  
| 2 | 记者, 本报讯, 昨日, 获悉, 新华网, 基金, 通讯员, 采访, 男子, 昨天  
| 3 | 股, 下跌, 上涨, 震荡, 板块, 大盘, 股指, 涨幅, 沪, 反弹  
| ...  

```

--------------------------------

### Initialize KeyNMF with Instruction-Tuned Embeddings (Python)

Source: https://x-tabdeveloping.github.io/turftopic/encoders

Shows how to initialize a KeyNMF model using an instruction-tuned embedding model from Sentence Transformers. This is useful for tasks where documents and keywords are encoded differently, leveraging specific prompts for queries and passages to improve retrieval accuracy.

```python
from turftopic import KeyNMF
from sentence_transformers import SentenceTransformer

encoder = SentenceTransformer(
    "intfloat/multilingual-e5-large-instruct",
    prompts={
        "query": "Instruct: Retrieve relevant keywords from the given document. Query: "
        "passage": "Passage: "
    },
    # Make sure to set default prompt to query!
    default_prompt_name="query",
)
model = KeyNMF(10, encoder=encoder)
```
Tokens:
Raw