### Install tiktoken and openai packages

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Install the required packages for the example. Run this command in your terminal.

```bash
pip install tiktoken openai
```

--------------------------------

### LiteLLM Usage Example

Source: https://maartengr.github.io/BERTopic/api/representations.html

Shows how to set up and use the LiteLLM representation model with BERTopic, including environment setup and custom prompt usage.

```APIDOC
## LiteLLM Usage Example

### Description
Demonstrates how to configure and use the LiteLLM representation model within BERTopic.

### Setup
```python
import os
from bertopic.representation import LiteLLM
from bertopic import BERTopic

# set ENV variables
os.environ["OPENAI_API_KEY"] = "your-openai-key"

# Create your representation model
representation_model = LiteLLM(model="gpt-3.5-turbo")

# Use the representation model in BERTopic on top of the default pipeline
topic_model = BERTopic(representation_model=representation_model)
```

### Custom Prompt
```python
prompt = "I have the following documents: [DOCUMENTS] \nThese documents are about the following topic: '"
representation_model = LiteLLM(model="gpt", prompt=prompt)
```
```

--------------------------------

### Install llama-cpp-python with CUDA acceleration

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Installation with hardware acceleration using CMAKE_ARGS and FORCE_CMAKE. Note that other installation options exist depending on your hardware and OS.

```bash
CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python
```

--------------------------------

### Install LangChain and OpenAI

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Install the required packages for using LangChain with BERTopic.

```bash
pip install langchain, openai
```

--------------------------------

### Install llama-cpp-python

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Basic installation of the llama-cpp-python package via pip.

```bash
pip install llama-cpp-python
```

--------------------------------

### Install langchain and openai packages

Source: https://maartengr.github.io/BERTopic/api/representations.html

Install the required packages before using the LangChain representation model.

```bash
pip install langchain
```

```bash
pip install openai
```

--------------------------------

### Rendered prompt example

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Shows how the prompt template is rendered after replacing [DOCUMENTS] and [KEYWORDS] with actual topic data. This is an example output, not a code snippet to run.

```text
"""
I have a topic that contains the following documents:
- Our videos are also made possible by your support on patreon.co.
- If you want to help us make more videos, you can do so on patreon.com or get one of our posters from our shop.
- If you want to help us make more videos, you can do so there.
- And if you want to support us in our endeavor to survive in the world of online video, and make more videos, you can do so on patreon.com.

The topic is described by the following keywords: videos video you our support want this us channel patreon make on we if facebook to patreoncom can for and more watch

Based on the above information, can you give a short label of the topic?
"""
```

--------------------------------

### Install BERTopic with pip

Source: https://maartengr.github.io/BERTopic/

Basic installation command for BERTopic using pip.

```bash
pip install bertopic
```

--------------------------------

### Install BERTopic with optional backends

Source: https://maartengr.github.io/BERTopic/

Installation commands for optional embedding backends (flair, gensim, spacy, use) and vision support.

```bash
# Choose an embedding backend
pip install bertopic[flair, gensim, spacy, use]

# Topic modeling with images
pip install bertopic[vision]
```

--------------------------------

### Lightweight Installation

Source: https://maartengr.github.io/BERTopic/changelog.html

Install BERTopic without sentence-transformers and use a CPU-based embedding pipeline.

```APIDOC
## Lightweight Installation

### Description
Install BERTopic without sentence-transformers to reduce environment size and enable CPU-only usage.

### Installation Commands
```
pip install --no-deps bertopic
pip install --upgrade numpy hdbscan umap-learn pandas scikit-learn tqdm plotly pyyaml
```

### Usage Example
```python
from sklearn.pipeline import make_pipeline
from sklearn.decomposition import TruncatedSVD
from sklearn.feature_extraction.text import TfidfVectorizer

pipe = make_pipeline(
    TfidfVectorizer(),
    TruncatedSVD(100)
)

topic_model = BERTopic(embedding_model=pipe)
```

### Notes
- The default embedding model is `"all-MiniLM-L6-v2"`.
- This lightweight installation avoids `sentence-transformers` and `pytorch`.
- The resulting model can run on CPU without a GPU.
```

--------------------------------

### Install OpenAI package

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Install the openai package to use OpenAI's API with BERTopic.

```bash
pip install openai
```

--------------------------------

### Use a Runnable with LangChain Expression Language

Source: https://maartengr.github.io/BERTopic/api/representations.html

Create a privacy-preserving chain using LangChain Expression Language and Microsoft Presidio. The example shows imports and setup, but the prompt and llm are not fully defined.

```python
from bertopic.representation import LangChain
from langchain.chains.question_answering import load_qa_chain
from langchain.chat_models import ChatAnthropic
from langchain.schema.document import Document
from langchain.schema.runnable import RunnablePassthrough
from langchain_experimental.data_anonymizer.presidio import PresidioReversibleAnonymizer

prompt = ...
llm = ...

# We will construct a special privacy-preserving chain using Microsoft Presidio

pii_handler = PresidioReversibleAnonymizer(analyzed_fields=["PERSON"])
```

--------------------------------

### Example: Using topics_per_class

Source: https://maartengr.github.io/BERTopic/api/bertopic.html

Example from the BERTopic API docs showing how to fit a model and generate topics per class.

```python
from bertopic import BERTopic
topic_model = BERTopic()
topics, probs = topic_model.fit_transform(docs)
topics_per_class = topic_model.topics_per_class(docs, classes)
```

--------------------------------

### Install Cohere

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Install the Cohere Python package using pip.

```bash
pip install cohere
```

--------------------------------

### OpenAIBackend usage example

Source: https://maartengr.github.io/BERTopic/api/backends.html

Shows how to instantiate an OpenAIBackend with an OpenAI client and the default embedding model 'text-embedding-ada-002'.

```python
import openai
from bertopic.backend import OpenAIBackend

client = openai.OpenAI(api_key="sk-...")
openai_embedder = OpenAIBackend(client, "text-embedding-ada-002")
```

--------------------------------

### Install model2vec with distillation dependencies

Source: https://maartengr.github.io/BERTopic/getting_started/embeddings/embeddings.html

Install model2vec with the additional dependencies required for distillation.

```bash
pip install model2vec[distill]
```

--------------------------------

### Install litellm package

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Install the litellm package, which is required to use LiteLLM with BERTopic.

```bash
pip install litellm
```

--------------------------------

### Install cuML and BERTopic for CUDA 13

Source: https://maartengr.github.io/BERTopic/getting_started/dim_reduction/dim_reduction.html

Installation commands for cuML with CUDA 13. Install cuML first, then BERTopic, to avoid pip resolver issues.

```bash
!pip install cuml-cu13
!pip install bertopic
```

--------------------------------

### Install cuML and BERTopic for CUDA 12

Source: https://maartengr.github.io/BERTopic/getting_started/dim_reduction/dim_reduction.html

Installation commands for cuML with CUDA 12. Install cuML first, then BERTopic, to avoid pip resolver issues.

```bash
!pip install cuml-cu12
!pip install bertopic
```

--------------------------------

### LiteLLM custom prompt example

Source: https://maartengr.github.io/BERTopic/api/representations.html

Shows how to use a custom prompt with LiteLLM by passing a prompt string to the model.

```python
prompt = "I have the following documents: [DOCUMENTS] \nThese documents are about the following topic: '"
representation_model = LiteLLM(model="gpt", prompt=prompt)
```

--------------------------------

### LangChainBackend usage example

Source: https://maartengr.github.io/BERTopic/api/backends.html

Shows how to instantiate LangChainBackend with a HuggingFaceInstructEmbeddings model from langchain_community.embeddings.

```python
from langchain_community.embeddings import HuggingFaceInstructEmbeddings
from bertopic.backend import LangChainBackend

hf_embedding = HuggingFaceInstructEmbeddings()
langchain_embedder = LangChainBackend(hf_embedding)
```

--------------------------------

### Basic visualization with visualize_documents

Source: https://maartengr.github.io/BERTopic/api/plotting/documents.html

Minimal example to visualize documents and topics. Note that this re-calculates embeddings and reduces them to 2D.

```python
topic_model.visualize_documents(docs)
```

--------------------------------

### Install model2vec

Source: https://maartengr.github.io/BERTopic/getting_started/embeddings/embeddings.html

Install the model2vec package via pip.

```bash
pip install model2vec
```

--------------------------------

### Speed up BERTopic with cuML GPU acceleration

Source: https://maartengr.github.io/BERTopic/faq.html

Use cuML's GPU-accelerated UMAP and HDBSCAN to speed up BERTopic. Requires a GPU and cuML installed. The example creates UMAP and HDBSCAN instances and passes them to BERTopic.

```python
from bertopic import BERTopic
from cuml.cluster import HDBSCAN
from cuml.manifold import UMAP

# Create instances of GPU-accelerated UMAP and HDBSCAN
umap_model = UMAP(n_components=5, n_neighbors=15, min_dist=0.0)
hdbscan_model = HDBSCAN(min_samples=10, gen_min_span_tree=True, prediction_data=True)

# Pass the above models to be used in BERTopic
topic_model = BERTopic(umap_model=umap_model, hdbscan_model=hdbscan_model)
```

--------------------------------

### Guided BERTopic with seed_topic_list

Source: https://maartengr.github.io/BERTopic/changelog.html

Shows how to initialize and fit a BERTopic model with a seed_topic_list to guide topic creation. The seed topics are used to nudge the model towards desired topics, and the fit_transform call returns topics and probabilities.

```python
seed_topic_list = [["company", "billion", "quarter", "shrs", "earnings"],
                   ["acquisition", "procurement", "merge"],
                   ["exchange", "currency", "trading", "rate", "euro"],
                   ["grain", "wheat", "corn"],
                   ["coffee", "cocoa"],
                   ["natural", "gas", "oil", "fuel", "products", "petrol"]]

topic_model = BERTopic(seed_topic_list=seed_topic_list)
topics, probs = topic_model.fit_transform(docs)
```

--------------------------------

### Install optional dependencies for Model2Vec and UMAP

Source: https://maartengr.github.io/BERTopic/getting_started/tips_and_tricks/tips_and_tricks.html

Installs model2vec and umap-learn, which are needed if you want to use Model2Vec for embeddings and UMAP for dimensionality reduction.

```bash
pip install model2vec umap-learn
```

--------------------------------

### Example structure of updated_topics

Source: https://maartengr.github.io/BERTopic/getting_started/representation/representation.html

Shows the required structure for updated_topics returned by a custom model. Note the example uses a string key "1" with a list of tuples, but the source shows a typo with a comma instead of a colon after "1".

```python
updated_topics = {
    "1", [("space", 0.9), ("nasa", 0.7)],
    "2": [("science", 0.66), ("article", 0.6)]
}
```

--------------------------------

### Guided BERTopic with seed_topic_list on 20 Newsgroups

Source: https://maartengr.github.io/BERTopic/getting_started/guided/guided.html

Initializes a BERTopic model with a seed_topic_list to guide topic modeling. The seed topics are lists of words representing desired topics. The model is fit on the 20 Newsgroups dataset. Note that BERTopic is only nudged towards these topics; if they don't exist or are too specific, they may not be modeled.

```python
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups

docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))["data"]

seed_topic_list = [["drug", "cancer", "drugs", "doctor"],
                   ["windows", "drive", "dos", "file"],
                   ["space", "launch", "orbit", "lunar"]]

topic_model = BERTopic(seed_topic_list=seed_topic_list)
topics, probs = topic_model.fit_transform(docs)
```

--------------------------------

### Install BERTopic without dependencies

Source: https://maartengr.github.io/BERTopic/getting_started/tips_and_tricks/tips_and_tricks.html

Installs a bare-bones version of BERTopic without sentence-transformers, UMAP, HDBSCAN, or plotly. Use this to reduce environment size or when using alternative embedding models.

```bash
pip install --no-deps bertopic
pip install --upgrade numpy pandas scikit-learn tqdm pyyaml
```

--------------------------------

### Lightweight installation without SentenceTransformers

Source: https://maartengr.github.io/BERTopic/changelog.html

Install BERTopic without SentenceTransformers for a lightweight package. Use the two pip commands in sequence: first install without dependencies, then upgrade the required packages.

```bash
pip install --no-deps bertopic
```

```bash
pip install --upgrade numpy hdbscan umap-learn pandas scikit-learn tqdm plotly pyyaml
```

--------------------------------

### Full hierarchy tree output

Source: https://maartengr.github.io/BERTopic/getting_started/hierarchicaltopics/hierarchicaltopics.html

The complete text-based hierarchy tree for the example dataset, showing all topics and their hierarchical relationships.

```text
  .
  ├─people_armenian_said_god_armenians
  │    ├─god_jesus_jehovah_lord_christ
  │    │    ├─god_jesus_jehovah_lord_christ
  │    │    │    ├─jehovah_lord_mormon_mcconkie_god
  │    │    │    │    ├─■──ra_satan_thou_god_lucifer ── Topic: 94
  │    │    │    │    └─■──jehovah_lord_mormon_mcconkie_unto ── Topic: 78
  │    │    │    └─jesus_mary_god_hell_sin
  │    │    │         ├─jesus_hell_god_eternal_heaven
  │    │    │         │    ├─hell_jesus_eternal_god_heaven
  │    │    │         │    │    ├─■──jesus_tomb_disciples_resurrection_john ── Topic: 69
  │    │    │         │    │    └─■──hell_eternal_god_jesus_heaven ── Topic: 53
  │    │    │         │    └─■──aaron_baptism_sin_law_god ── Topic: 89
  │    │    │         └─■──mary_sin_maria_priest_conception ── Topic: 56
  │    │    └─■──marriage_married_marry_ceremony_marriages ── Topic: 110
  │    └─people_armenian_armenians_said_mr
  │         ├─people_armenian_armenians_said_israel
  │         │    ├─god_homosexual_homosexuality_atheists_sex
  │         │    │    ├─homosexual_homosexuality_sex_gay_homosexuals
  │         │    │    │    ├─■──kinsey_sex_gay_men_sexual ── Topic: 44
  │         │    │    │    └─homosexuality_homosexual_sin_homosexuals_gay
  │         │    │    │         ├─■──gay_homosexual_homosexuals_sexual_cramer ── Topic: 50
  │         │    │    │         └─■──homosexuality_homosexual_sin_paul_sex ── Topic: 27
  │         │    │    └─god_atheists_atheism_moral_atheist
  │         │    │         ├─islam_quran_judas_islamic_book
  │         │    │         │    ├─■──jim_context_challenges_articles_quote ── Topic: 36
  │         │    │         │    └─islam_quran_judas_islamic_book
  │         │    │         │         ├─■──islam_quran_islamic_rushdie_muslims ── Topic: 31
  │         │    │         │         └─■──judas_scripture_bible_books_greek ── Topic: 33
  │         │    │         └─atheists_atheism_god_moral_atheist
  │         │    │              ├─atheists_atheism_god_atheist_argument
  │         │    │              │    ├─■──atheists_atheism_god_atheist_argument ── Topic: 21
  │         │    │              │    └─■──br_god_exist_genetic_existence ── Topic: 124
  │         │    │              └─■──moral_morality_objective_immoral_morals ── Topic: 29
  │         │    └─armenian_armenians_people_israel_said
  │         │         ├─armenian_armenians_israel_people_jews
  │         │         │    ├─tax_rights_government_income_taxes
  │         │         │    │    ├─■──rights_right_slavery_slaves_residence ── Topic: 106
  │         │         │    │    └─tax_government_taxes_income_libertarians
  │         │         │    │         ├─■──government_libertarians_libertarian_regulation_party ── Topic: 58
  │         │         │    │         └─■──tax_taxes_income_billion_deficit ── Topic: 41
  │         │         │    └─armenian_armenians_israel_people_jews
  │         │         │         ├─gun_guns_militia_firearms_amendment
  │         │         │         │    ├─■──blacks_penalty_death_cruel_punishment ── Topic: 55
  │         │         │         │    └─■──gun_guns_militia_firearms_amendment ── Topic: 7
  │         │         │         └─armenian_armenians_israel_jews_turkish
  │         │         │              ├─■──israel_israeli_jews_arab_jewish ── Topic: 4
  │         │         │              └─■──armenian_armenians_turkish_armenia_azerbaijan ── Topic: 15
  │         │         └─stephanopoulos_president_mr_myers_ms
  │         │              ├─■──serbs_muslims_stephanopoulos_mr_bosnia ── Topic: 35
```

--------------------------------

### Configure BERTopic with custom models for each algorithm step

Source: https://maartengr.github.io/BERTopic/algorithm/algorithm.html

This snippet demonstrates the full BERTopic pipeline setup. It imports necessary libraries, defines each model for steps 1-6, and creates a BERTopic instance with all components. The optional representation model (KeyBERTInspired) is included but can be omitted if not needed.

```python
from umap import UMAP
from hdbscan import HDBSCAN
from sentence_transformers import SentenceTransformer
from sklearn.feature_extraction.text import CountVectorizer

from bertopic import BERTopic
from bertopic.representation import KeyBERTInspired
from bertopic.vectorizers import ClassTfidfTransformer


# Step 1 - Extract embeddings
embedding_model = SentenceTransformer("all-MiniLM-L6-v2")

# Step 2 - Reduce dimensionality
umap_model = UMAP(n_neighbors=15, n_components=5, min_dist=0.0, metric='cosine')

# Step 3 - Cluster reduced embeddings
hdbscan_model = HDBSCAN(min_cluster_size=15, metric='euclidean', cluster_selection_method='eom', prediction_data=True)

# Step 4 - Tokenize topics
vectorizer_model = CountVectorizer(stop_words="english")

# Step 5 - Create topic representation
ctfidf_model = ClassTfidfTransformer()

# Step 6 - (Optional) Fine-tune topic representations with
# a `bertopic.representation` model
representation_model = KeyBERTInspired()

# All steps together
topic_model = BERTopic(
  embedding_model=embedding_model,          # Step 1 - Extract embeddings
  umap_model=umap_model,                    # Step 2 - Reduce dimensionality
  hdbscan_model=hdbscan_model,              # Step 3 - Cluster reduced embeddings
  vectorizer_model=vectorizer_model,        # Step 4 - Tokenize topics
  ctfidf_model=ctfidf_model,                # Step 5 - Extract topic words
  representation_model=representation_model # Step 6 - (Optional) Fine-tune topic representations
)
```

--------------------------------

### Download a local LLM (zephyr-7b-alpha GGUF)

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Downloads a 4-bit quantized GGUF model from Hugging Face using wget. This model is used in the subsequent BERTopic examples.

```bash
wget https://huggingface.co/TheBloke/zephyr-7B-alpha-GGUF/resolve/main/zephyr-7b-alpha.Q4_K_M.gguf
```

--------------------------------

### Rendered prompt example with documents and keywords

Source: https://maartengr.github.io/BERTopic/changelog.html

Shows how the custom prompt is rendered after BERTopic replaces the [DOCUMENTS] and [KEYWORDS] tags with actual content. This is the final prompt passed to OpenAI's API.

```text
"""
I have a topic that contains the following documents:
- Our videos are also made possible by your support on patreon.co.
- If you want to help us make more videos, you can do so on patreon.com or get one of our posters from our shop.
- If you want to help us make more videos, you can do so there.
- And if you want to support us in our endeavor to survive in the world of online video, and make more videos, you can do so on patreon.com.

The topic is described by the following keywords: videos video you our support want this us channel patreon make on we if facebook to patreoncom can for and more watch

Based on the information above, extract a short topic label in the following format:
topic: <topic label>
"""
```

--------------------------------

### Create prompt template for Llama topic labeling

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Defines a system prompt, an example prompt, and a main prompt with [DOCUMENTS] and [KEYWORDS] placeholders. The prompts are concatenated to form the final prompt used for topic labeling.

```python
# System prompt describes information given to all conversations
system_prompt = """
<s>[INST] <<SYS>>
You are a helpful, respectful and honest assistant for labeling topics.
<</SYS>>
"""

# Example prompt demonstrating the output we are looking for
example_prompt = """
I have a topic that contains the following documents:
- Traditional diets in most cultures were primarily plant-based with a little meat on top, but with the rise of industrial style meat production and factory farming, meat has become a staple food.
- Meat, but especially beef, is the word food in terms of emissions.
- Eating meat doesn't make you a bad person, not eating meat doesn't make you a good one.

The topic is described by the following keywords: 'meat, beef, eat, eating, emissions, steak, food, health, processed, chicken'.

Based on the information about the topic above, please create a short label of this topic. Make sure you to only return the label and nothing more.

[/INST] Environmental impacts of eating meat
"""

# Our main prompt with documents ([DOCUMENTS]) and keywords ([KEYWORDS]) tags
main_prompt = """
[INST]
I have a topic that contains the following documents:
[DOCUMENTS]

The topic is described by the following keywords: '[KEYWORDS]'.

Based on the information about the topic above, please create a short label of this topic. Make sure you to only return the label and nothing more.
[/INST]
"""

prompt = system_prompt + example_prompt + main_prompt
```

--------------------------------

### Quick start: fit BERTopic on 20 newsgroups

Source: https://maartengr.github.io/BERTopic/

Loads the 20 newsgroups dataset, creates a BERTopic model, and fits it to extract topics and probabilities.

```python
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups

docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))['data']

topic_model = BERTopic()
topics, probs = topic_model.fit_transform(docs)
```

--------------------------------

### Use LiteLLM with BERTopic

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Set the OPENAI_API_KEY environment variable and create a LiteLLM representation model for BERTopic. The example uses the gpt-4o-mini model.

```python
import os
from bertopic import BERTopic
from bertopic.representation import LiteLLM

# set ENV variables
os.environ["OPENAI_API_KEY"] = "MY_KEY"

# Create your representation model
representation_model = LiteLLM(model="gpt-4o-mini")

# Create our BERTopic model
topic_model = BERTopic(representation_model=representation_model,  verbose=True)
```

--------------------------------

### Use Cohere API with BERTopic

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Set up a Cohere client with your API key and use it as the representation model in BERTopic. Requires installing cohere and having an API key.

```python
import cohere
from bertopic.representation import Cohere
from bertopic import BERTopic

# Create your representation model
co = cohere.Client(my_api_key)
representation_model = Cohere(co)

# Use the representation model in BERTopic on top of the default pipeline
topic_model = BERTopic(representation_model=representation_model)
```

--------------------------------

### Get text-based hierarchy tree with topic_model.get_topic_tree

Source: https://maartengr.github.io/BERTopic/getting_started/hierarchicaltopics/hierarchicaltopics.html

Use get_topic_tree to create a text-based representation of the hierarchy. The example shows the output when printing the tree, which helps identify topics that could be logically merged.

```python
>>> tree = topic_model.get_topic_tree(hierarchical_topics)
>>> print(tree)
.
└─atheists_atheism_god_moral_atheist
     ├─atheists_atheism_god_atheist_argument
     │    ├─■──atheists_atheism_god_atheist_argument ── Topic: 21
     │    └─■──br_god_exist_genetic_existence ── Topic: 124
     └─■──moral_morality_objective_immoral_morals ── Topic: 29
```

--------------------------------

### Using topics_over_time with nr_bins

Source: https://maartengr.github.io/BERTopic/api/bertopic.html

Example usage of the topics_over_time method. It is advised to bin timestamps when there are over 100 unique timestamps, as shown with nr_bins=20.

```python
from bertopic import BERTopic
topic_model = BERTopic()
topics, probs = topic_model.fit_transform(docs)
topics_over_time = topic_model.topics_over_time(docs, timestamps, nr_bins=20)
```

--------------------------------

### Visualize topic distribution with visualize_distribution

Source: https://maartengr.github.io/BERTopic/api/bertopic.html

Example usage of visualize_distribution to create a figure and save it to an HTML file. Requires a fitted topic model.

```python
fig = topic_model.visualize_distribution(topic_model.probabilities_[0])
fig.write_html("path/to/file.html")
```

--------------------------------

### CohereBackend basic usage

Source: https://maartengr.github.io/BERTopic/api/backends.html

Shows how to instantiate CohereBackend with a Cohere client. Requires an API key.

```python
import cohere
from bertopic.backend import CohereBackend

client = cohere.Client("APIKEY")
cohere_model = CohereBackend(client)
```

--------------------------------

### Summarization prompt for OpenAI

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Use a summarization prompt to get a short description of the topic instead of a label. Set nr_docs=5 and delay_in_seconds=3.

```python
summarization_prompt = """
I have a topic that is described by the following keywords: [KEYWORDS]
In this topic, the following documents are a small but representative subset of all documents in the topic:
[DOCUMENTS]

Based on the information above, please give a description of this topic in the following format:
topic: <description>
"""

representation_model = OpenAI(client, model="gpt-4o-mini", prompt=summarization_prompt, nr_docs=5, delay_in_seconds=3)
```

--------------------------------

### HDBSCAN with Custom Parameters

Source: https://maartengr.github.io/BERTopic/getting_started/clustering/clustering.html

Example of using HDBSCAN with custom parameters (min_cluster_size, metric, cluster_selection_method, prediction_data) and passing it to BERTopic via the hdbscan_model parameter.

```python
from bertopic import BERTopic
from hdbscan import HDBSCAN

hdbscan_model = HDBSCAN(min_cluster_size=15, metric='euclidean', cluster_selection_method='eom', prediction_data=True)
topic_model = BERTopic(hdbscan_model=hdbscan_model)
```

--------------------------------

### Use FastEmbed backend with BERTopic

Source: https://maartengr.github.io/BERTopic/getting_started/embeddings/embeddings.html

Create a FastEmbedBackend with a supported embedding model, such as 'BAAI/bge-small-en-v1.5'. Pass the backend to BERTopic as the embedding model. Check the supported FastEmbed text embedding models before starting.

```python
from bertopic.backend import FastEmbedBackend

embedding_model = FastEmbedBackend("BAAI/bge-small-en-v1.5")
topic_model = BERTopic(embedding_model=embedding_model)
```

--------------------------------

### Basic hierarchical_topics usage

Source: https://maartengr.github.io/BERTopic/api/bertopic.html

Shows the standard workflow: fit a BERTopic model on documents, then call hierarchical_topics to get a hierarchy of topics. Requires the model to be fitted first.

```python
from bertopic import BERTopic
topic_model = BERTopic()
topics, probs = topic_model.fit_transform(docs)
hierarchical_topics = topic_model.hierarchical_topics(docs)
```

--------------------------------

### Use a different sentence-transformers model (BAAI/bge-base-en-v1.5)

Source: https://maartengr.github.io/BERTopic/getting_started/embeddings/embeddings.html

Select any model from the sentence-transformers library by passing its name to SentenceTransformer and then to BERTopic. This example uses the BAAI/bge-base-en-v1.5 model.

```python
from sentence_transformers import SentenceTransformer

embedding_model = SentenceTransformer("BAAI/bge-base-en-v1.5")
topic_model = BERTopic(embedding_model=embedding_model)
```

--------------------------------

### Create custom topic labels with .generate_topic_labels and .set_topic_labels

Source: https://maartengr.github.io/BERTopic/changelog.html

Generates custom labels for topics and sets them internally in BERTopic. The example shows generating labels with specific parameters and then applying them.

```python
# Generate topic labels
topic_labels = topic_model.generate_topic_labels(nr_words=3, topic_prefix=False, word_length=10, separator=", ")

# Set them internally in BERTopic
topic_model.set_topic_labels(topics_labels)
```

--------------------------------

### Online topic modeling with BERTopic partial_fit

Source: https://maartengr.github.io/BERTopic/api/bertopic.html

Example showing how to perform online topic modeling with BERTopic using partial_fit. It uses IncrementalPCA, MiniBatchKMeans, and OnlineCountVectorizer to incrementally train on batches of 1000 documents from the 20newsgroups dataset.

```python
from sklearn.datasets import fetch_20newsgroups
from sklearn.cluster import MiniBatchKMeans
from sklearn.decomposition import IncrementalPCA
from bertopic.vectorizers import OnlineCountVectorizer
from bertopic import BERTopic

# Prepare documents
docs = fetch_20newsgroups(subset=subset,  remove=('headers', 'footers', 'quotes'))["data"]

# Prepare sub-models that support online learning
umap_model = IncrementalPCA(n_components=5)
cluster_model = MiniBatchKMeans(n_clusters=50, random_state=0)
vectorizer_model = OnlineCountVectorizer(stop_words="english", decay=.01)

topic_model = BERTopic(umap_model=umap_model,
                       hdbscan_model=cluster_model,
                       vectorizer_model=vectorizer_model)

# Incrementally fit the topic model by training on 1000 documents at a time
for index in range(0, len(docs), 1000):
    topic_model.partial_fit(docs[index: index+1000])
```

--------------------------------

### BERTopic usage example with sentence embeddings

Source: https://maartengr.github.io/BERTopic/api/bertopic.html

Shows how to create a BERTopic model, generate embeddings with SentenceTransformer, and fit the model to documents. Requires the bertopic, scikit-learn, and sentence-transformers packages.

```python
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups
from sentence_transformers import SentenceTransformer

# Create embeddings
docs = fetch_20newsgroups(subset='all')['data']
sentence_model = SentenceTransformer("all-MiniLM-L6-v2")
embeddings = sentence_model.encode(docs, show_progress_bar=True)

# Create topic model
topic_model = BERTopic()
topics, probs = topic_model.fit_transform(docs, embeddings)
```

--------------------------------

### get_representative_docs note: create dataframe from docs and topics

Source: https://maartengr.github.io/BERTopic/api/bertopic.html

Shows how to create a dataframe of documents and topics when using fit_transform or fit, as referenced in the note for get_representative_docs.

```python
# When you used `.fit_transform`:
df = pd.DataFrame({"Document": docs, "Topic": topic})

# When you used `.fit`:
df = pd.DataFrame({"Document": docs, "Topic": topic_model.topics_})
```

--------------------------------

### Prepare data with 20 Newsgroups

Source: https://maartengr.github.io/BERTopic/getting_started/topicsperclass/topicsperclass.html

Loads the 20 Newsgroups dataset, removes headers/footers/quotes, and creates lists of documents, targets, target names, and class labels for each document.

```python
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups

data = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))
docs = data["data"]
targets = data["target"]
target_names = data["target_names"]
classes = [data["target_names"][i] for i in data["target"]]
```

--------------------------------

### Generate WordCloud for a BERTopic Topic

Source: https://maartengr.github.io/BERTopic/getting_started/tips_and_tricks/tips_and_tricks.html

Use this script to create a wordcloud for a specific topic in BERTopic. Requires the wordcloud package (pip install wordcloud). The function takes the topic model and topic index, extracts word frequencies via get_topic, and displays the wordcloud. To show more words, increase top_n_words when instantiating BERTopic or use .update_topics() after training.

```python
from wordcloud import WordCloud
import matplotlib.pyplot as plt

def create_wordcloud(model, topic):
    text = {word: value for word, value in model.get_topic(topic)}
    wc = WordCloud(background_color="white", max_words=1000)
    wc.generate_from_frequencies(text)
    plt.imshow(wc, interpolation="bilinear")
    plt.axis("off")
    plt.show()

# Show wordcloud
create_wordcloud(topic_model, topic=1)
```

--------------------------------

### Get frequency of all topics

Source: https://maartengr.github.io/BERTopic/api/bertopic.html

Call get_topic_freq() without arguments to get a DataFrame with the frequencies of all topics.

```python
frequency = topic_model.get_topic_freq()
```

--------------------------------

### Set up BERTopic with Llama TextGeneration representation model

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Creates a TextGeneration representation model using the Llama generator and prompt, then initializes a BERTopic model with verbose=True.

```python
from bertopic.representation import TextGeneration
from bertopic import BERTopic

# Text generation with Llama
llama2 = TextGeneration(generator, prompt=prompt)
representation_model = {
    "Llama2": llama2,
}

# Create our BERTopic model
topic_model = BERTopic(representation_model=representation_model,  verbose=True)
```

--------------------------------

### Get frequency of a single topic

Source: https://maartengr.github.io/BERTopic/api/bertopic.html

Pass a topic ID to get_topic_freq() to get the frequency of that specific topic.

```python
frequency = topic_model.get_topic_freq(12)
```

--------------------------------

### Prompt template with [DOCUMENTS] and [KEYWORDS] tags

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Defines a prompt template for an LLM to generate a short topic label. The [DOCUMENTS] tag is replaced with the top 4 most representative documents and [KEYWORDS] with the topic's keywords. This is the base prompt used in the examples below.

```python
prompt = """
I have topic that contains the following documents: \n[DOCUMENTS]
The topic is described by the following keywords: [KEYWORDS]

Based on the above information, can you give a short label of the topic?
"""
```

--------------------------------

### KeyBERTInspired.__init__

Source: https://maartengr.github.io/BERTopic/api/representations.html

Initializes the KeyBERTInspired representation model with configurable parameters for topic representation fine-tuning.

```APIDOC
## KeyBERTInspired.__init__

### Description
Initializes the KeyBERTInspired representation model. This model uses a KeyBERT-like approach to fine-tune topic representations by extracting representative documents and candidate words, then computing embeddings and cosine similarities.

### Method
Constructor

### Signature
```python
KeyBERTInspired(
    top_n_words: int = 10,
    nr_repr_docs: int = 5,
    nr_samples: int = 500,
    nr_candidate_words: int = 100,
    random_state: int = 42,
)
```

### Parameters
- **top_n_words** (int) - Optional - The top n words to extract per topic. Default: 10.
- **nr_repr_docs** (int) - Optional - The number of representative documents to extract per cluster. Default: 5.
- **nr_samples** (int) - Optional - The number of candidate documents to extract per cluster. Default: 500.
- **nr_candidate_words** (int) - Optional - The number of candidate words per cluster. Default: 100.
- **random_state** (int) - Optional - The random state for randomly sampling candidate documents. Default: 42.

### Usage Example
```python
from bertopic.representation import KeyBERTInspired
from bertopic import BERTopic

# Create your representation model
representation_model = KeyBERTInspired()

# Use the representation model in BERTopic on top of the default pipeline
topic_model = BERTopic(representation_model=representation_model)
```
```

--------------------------------

### Create a LangChain QA Chain

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Create a question-answering chain using OpenAI with temperature 0 and your API key.

```python
from langchain.chains.question_answering import load_qa_chain
from langchain.llms import OpenAI
chain = load_qa_chain(OpenAI(temperature=0, openai_api_key=my_openai_api_key), chain_type="stuff")
```

--------------------------------

### Custom prompt with LiteLLM

Source: https://maartengr.github.io/BERTopic/api/representations.html

Define a custom prompt using the [DOCUMENTS] placeholder and pass it to LiteLLM.

```python
prompt = "I have the following documents: [DOCUMENTS]"
representation_model = LiteLLM(model="gpt", prompt=prompt)
```

--------------------------------

### LiteLLM class implementation

Source: https://maartengr.github.io/BERTopic/api/representations.html

The full implementation of the LiteLLM class, including initialization parameters, topic extraction logic, and prompt creation.

```python
def __init__(
    self,
    model: str = "gpt-3.5-turbo",
    prompt: str | None = None,
    generator_kwargs: Mapping[str, Any] = {},
    delay_in_seconds: float | None = None,
    exponential_backoff: bool = False,
    nr_docs: int = 4,
    diversity: float | None = None,
):
    self.model = model
    self.prompt = prompt if prompt else DEFAULT_PROMPT
    self.default_prompt_ = DEFAULT_PROMPT
    self.delay_in_seconds = delay_in_seconds
    self.exponential_backoff = exponential_backoff
    self.nr_docs = nr_docs
    self.diversity = diversity

    self.generator_kwargs = generator_kwargs
    if self.generator_kwargs.get("model"):
        self.model = generator_kwargs.get("model")
    if self.generator_kwargs.get("prompt"):
        del self.generator_kwargs["prompt"]

def extract_topics(
    self, topic_model, documents: pd.DataFrame, c_tf_idf: csr_matrix, topics: Mapping[str, List[Tuple[str, float]]]
) -> Mapping[str, List[Tuple[str, float]]]:
    """Extract topics.

    Arguments:
        topic_model: A BERTopic model
        documents: All input documents
        c_tf_idf: The topic c-TF-IDF representation
        topics: The candidate topics as calculated with c-TF-IDF

    Returns:
        updated_topics: Updated topic representations
    """
    # Extract the top n representative documents per topic
    repr_docs_mappings, _, _, _ = topic_model._extract_representative_docs(
        c_tf_idf, documents, topics, 500, self.nr_docs, self.diversity
    )

    # Generate using a (Large) Language Model
    updated_topics = {}
    for topic, docs in tqdm(repr_docs_mappings.items(), disable=not topic_model.verbose):
        prompt = self._create_prompt(docs, topic, topics)

        # Delay
        if self.delay_in_seconds:
            time.sleep(self.delay_in_seconds)

        messages = [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": prompt},
        ]
        kwargs = {"model": self.model, "messages": messages, **self.generator_kwargs}
        if self.exponential_backoff:
            response = chat_completions_with_backoff(**kwargs)
        else:
            response = completion(**kwargs)
        label = response["choices"][0]["message"]["content"].strip().replace("topic: ", "")

        updated_topics[topic] = [(label, 1)]

    return updated_topics

def _create_prompt(self, docs, topic, topics):
    keywords = next(zip(*topics[topic]))

    # Use the Default Chat Prompt
    if self.prompt == DEFAULT_PROMPT:
        prompt = self.prompt.replace("[KEYWORDS]", " ".join(keywords))
        prompt = self._replace_documents(prompt, docs)

    # Use a custom prompt that leverages keywords, documents or both using
    # custom tags, namely [KEYWORDS] and [DOCUMENTS] respectively
    else:
        prompt = self.prompt
        if "[KEYWORDS]" in prompt:
            prompt = prompt.replace("[KEYWORDS]", " ".join(keywords))
        if "[DOCUMENTS]" in prompt:
            prompt = self._replace_documents(prompt, docs)

    return prompt

@staticmethod
def _replace_documents(prompt, docs):
    to_replace = ""
    for doc in docs:
        to_replace += f"- {doc[:255]}\n"
    prompt = prompt.replace("[DOCUMENTS]", to_replace)
    return prompt
```

--------------------------------

### Install ctransformers and update transformers for Zephyr

Source: https://maartengr.github.io/BERTopic/getting_started/representation/llm

Installs ctransformers with CUDA support and updates the transformers library from the Hugging Face GitHub repository to handle quantized models.

```bash
pip install ctransformers[cuda]
pip install --upgrade git+https://github.com/huggingface/transformers
```

--------------------------------

### KeyBERTInspired.__init__

Source: https://maartengr.github.io/BERTopic/api/representations.html

Initializes a KeyBERTInspired representation model with configurable parameters for topic representation fine-tuning.

```APIDOC
## KeyBERTInspired.__init__

### Description
Initializes a KeyBERTInspired representation model. This model uses a KeyBERT-like algorithm to fine-tune topic representations by extracting representative documents and words based on c-TF-IDF and cosine similarity.

### Method
Constructor

### Signature
`__init__(top_n_words=10, nr_repr_docs=5, nr_samples=500, nr_candidate_words=100, random_state=42)`

### Parameters
- **top_n_words** (int) - Optional - The top n words to extract per topic. Default: 10.
- **nr_repr_docs** (int) - Optional - The number of representative documents to extract per cluster. Default: 5.
- **nr_samples** (int) - Optional - The number of candidate documents to extract per cluster. Default: 500.
- **nr_candidate_words** (int) - Optional - The number of candidate words per cluster. Default: 100.
- **random_state** (int) - Optional - The random state for randomly sampling candidate documents. Default: 42.

### Usage Example
```python
from bertopic.representation import KeyBERTInspired
from bertopic import BERTopic

# Create your representation model
representation_model = KeyBERTInspired()

# Use the representation model in BERTopic on top of the default pipeline
topic_model = BERTopic(representation_model=representation_model)
```

### Source Code
```python
def __init__(
    self,
    top_n_words: int = 10,
    nr_repr_docs: int = 5,
    nr_samples: int = 500,
    nr_candidate_words: int = 100,
    random_state: int = 42,
):
    self.top_n_words = top_n_words
    self.nr_repr_docs = nr_repr_docs
    self.nr_samples = nr_samples
    self.nr_candidate_words = nr_candidate_words
    self.random_state = random_state
```
```

--------------------------------

### Define and fit multi-aspect representation models

Source: https://maartengr.github.io/BERTopic/getting_started/multiaspect/multiaspect.html

Creates a dictionary of representation models to generate multiple topic representations in a single fit. The main representation is defined with the 'Main' key, and additional aspects are defined with custom keys. The example uses KeyBERTInspired, PartOfSpeech, and a list combining KeyBERTInspired and MaximalMarginalRelevance.

```python
from bertopic.representation import KeyBERTInspired
from bertopic.representation import PartOfSpeech
from bertopic.representation import MaximalMarginalRelevance
from sklearn.datasets import fetch_20newsgroups

# Documents to train on
docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))['data']

# The main representation of a topic
main_representation = KeyBERTInspired()

# Additional ways of representing a topic
aspect_model1 = PartOfSpeech("en_core_web_sm")
aspect_model2 = [KeyBERTInspired(top_n_words=30), MaximalMarginalRelevance(diversity=.5)]

# Add all models together to be run in a single `fit`
representation_model = {
   "Main": main_representation,
   "Aspect1":  aspect_model1,
   "Aspect2":  aspect_model2
}
topic_model = BERTopic(representation_model=representation_model).fit(docs)
```