topicwizard

https://x-tabdeveloping.github.io/topicwizard/
Admin
topicwizard is an opinionated Python library for topic model visualization built on Dash and Plotly,...
Tokens:3,796
Snippets:34
Trust Score:8.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### Install topicwizard using pip

Source: https://x-tabdeveloping.github.io/topicwizard/index

Installs the topic-wizard package from PyPI. This is the primary method for getting topicwizard onto your system.

```bash
pip install topic-wizard

```

--------------------------------

### Create a topic model with turftopic

Source: https://x-tabdeveloping.github.io/topicwizard/index

Instantiates a KeyNMF model from the turftopic library, suitable for building contextually sensitive topic models. This requires turftopic to be installed.

```python
from turftopic import KeyNMF

model = KeyNMF(n_components=10)

```

--------------------------------

### Initialize Topic Models: LDA, NMF, and DMM

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Demonstrates initializing various topic models. Includes Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF) from scikit-learn, and Discrete Matrix Model (DMM) from tweetopic for short texts. Requires 'tweetopic' to be installed for DMM.

```python
# LDA for long texts
from sklearn.decomposition import LatentDirichletAllocation

model = LatentDirichletAllocation(n_components=10)
```

```python
# You can use NMF too
from sklearn.decomposition import NMF

model = NMF(n_components=10)
```

```python
# Or tweetopic's DMM for short texts
# pip install tweetopic

from tweetopic import DMM

model = DMM(n_components=10)
```

--------------------------------

### Visualize with Group Labels using 20Newsgroups

Source: https://x-tabdeveloping.github.io/topicwizard/application

This example demonstrates how to include group labels in the TopicWizard visualization. It fetches the 20Newsgroups dataset, trains an NMF model, and passes the corpus, model, and group labels to `topicwizard.visualize()`. This adds an extra page for investigating label relations.

```python
import topicwizard
from topicwizard.pipeline import make_topic_pipeline
from sklearn.datasets import fetch_20newsgroups
import numpy as np
from sklearn.decomposition import NMF
from sklearn.feature_extraction.text import CountVectorizer

newsgroups = fetch_20newsgroups(subset="all")
corpus = newsgroups.data
# Sklearn gives the labels back as integers, we have to map them back to
# the actual textual label.
group_labels = np.array(newsgroups.target_names)[newsgroups.target]

# Here we fit a topic model to the corpus
pipeline = make_topic_pipeline(
    CountVectorizer(stop_words="english"),
    NMF(n_components=30),
).fit(corpus)

# Notice that I'm passing the labels as the group_labels argument
topicwizard.visualize(corpus, model=pipeline, group_labels=group_labels)

```

--------------------------------

### Visualize Embeddings with Excluded Pages

Source: https://x-tabdeveloping.github.io/topicwizard/application

This example demonstrates optimizing TopicWizard for visualizing word embeddings. By excluding 'documents' and 'topics' pages, it allows for focused visualization when using matrix decomposition methods like LSI for creating word embeddings.

```python
topicwizard.visualize(texts, model=pipeline, exclude_pages=["documents", "topics"])

```

--------------------------------

### Visualize Topic Data with Topicwizard Figures and Web App

Source: https://x-tabdeveloping.github.io/topicwizard/topic_data

This snippet demonstrates how to use the TopicData object with topicwizard's visualization utilities. It shows how to generate a topic map figure and how to pass the TopicData object to the web app visualization function. Ensure topicwizard is installed.

```python
import topicwizard
from topicwizard.figures import topic_map

# Assuming topic_data is already defined

# Usage with figures
topic_map(topic_data)

# Usage with web app
# Beware that topic_data is a keyword argument
topicwizard.visualize(topic_data=topic_data)

```

--------------------------------

### Get Topic Names from Fitted Pipeline

Source: https://x-tabdeveloping.github.io/topicwizard/usage

After fitting a `TopicPipeline` with text data, this code retrieves the inferred topic names using `get_feature_names_out()`.

```python
topic_pipeline.fit(texts)
print(topic_pipeline.get_feature_names_out())
```

--------------------------------

### Visualize topic models using topicwizard

Source: https://x-tabdeveloping.github.io/topicwizard/index

Launches the topicwizard web application for visual exploration of a topic model. It takes the text corpus and the trained model as input.

```python
import topicwizard

topicwizard.visualize(texts, model=model)

```

--------------------------------

### Visualize Topic Model with TopicWizard

Source: https://x-tabdeveloping.github.io/topicwizard/application

This code shows the basic usage of the `topicwizard.visualize()` function to display a trained topic model. It takes the corpus (texts) and the fitted model pipeline as input, launching an interactive web application.

```python
import topicwizard

topicwizard.visualize(texts, model=pipeline)

```

--------------------------------

### Visualize Topics and Classify Documents with TopicPipeline

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Demonstrates transforming texts into topics using a pipeline and visualizing the results. It also shows how to create a rule-based classifier for specific topics, like 'corona', using FunctionClassifier and scikit-learn's make_pipeline.

```python
import plotly.express as px

texts = [
   "Coronavirus killed 50000 people today.",
   "Donald Trump's presidential campaing is going very well",
   "Protests against police brutality have been going on all around the US.",
]
topic_df = pipeline.transform(texts)
topic_df.index = texts
px.imshow(topic_df).show()
```

```python
# Install human-learn from PyPI
# pip install human-learn

from hulearn.classification import FunctionClassifier
from sklearn.pipeline import make_pipeline

topic_pipeline = make_topic_pipeline(vectorizer, model).fit(texts)

# Investigate topics
topicwizard.visualize(topic_pipeline)

# Creating rule for classifying something as a corona document
def corona_rule(df, threshold=0.5):
    is_about_corona = df["11_vaccine_pandemic_virus_coronavirus"] > threshold
    return is_about_corona.astype(int)

# Freezing topic pipeline
topic_pipeline.freeze = True
classifier = FunctionClassifier(corona_rule)
cls_pipeline = make_pipeline(topic_pipeline, classifier)
```

--------------------------------

### Create topicwizard TopicPipeline

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Creates a `TopicPipeline` using topicwizard's `make_topic_pipeline`. This is recommended over scikit-learn's `Pipeline` for better integration with topicwizard's features.

```python
from topicwizard.pipeline import make_topic_pipeline

topic_pipeline = make_topic_pipeline(vectorizer, model)
```

--------------------------------

### Create a topic pipeline with scikit-learn models

Source: https://x-tabdeveloping.github.io/topicwizard/index

Builds a scikit-learn compatible topic model pipeline using CountVectorizer and NMF. This is suitable for classical topic modeling approaches.

```python
from sklearn.decomposition import NMF
from sklearn.feature_extraction.text import CountVectorizer
from topicwizard.pipeline import make_topic_pipeline

bow_vectorizer = CountVectorizer()
nmf = NMF(n_components=10)
model = make_topic_pipeline(bow_vectorizer, nmf)

```

--------------------------------

### Initialize CountVectorizer for Text Vectorization

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Initializes a CountVectorizer from scikit-learn to convert texts into bag-of-words vectors. This is a common first step in many text processing pipelines.

```python
from sklearn.feature_extraction.text import CountVectorizer

vectorizer = CountVectorizer()
```

--------------------------------

### Create Scikit-learn Pipeline

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Combines a vectorizer and a topic model into a scikit-learn pipeline using `make_pipeline`. This allows sequential processing of text data.

```python
from sklearn.pipeline import make_pipeline

topic_pipeline = make_pipeline(vectorizer, model)
```

--------------------------------

### Train NMF Topic Model with TopicWizard

Source: https://x-tabdeveloping.github.io/topicwizard/application

This snippet demonstrates how to train a Non-negative Matrix Factorization (NMF) topic model using scikit-learn and prepare it for visualization with TopicWizard. It utilizes `CountVectorizer` for text processing and `make_topic_pipeline` for creating a unified pipeline.

```python
# Training a compatible topic model
from sklearn.decomposition import NMF
from sklearn.feature_extraction.text import CountVectorizer
from topicwizard.pipeline import make_topic_pipeline

bow_vectorizer = CountVectorizer()
nmf = NMF(n_components=10)
pipeline = make_topic_pipeline(bow_vectorizer, nmf)
pipeline.fit(texts)

```

--------------------------------

### Visualize with SemanticSignalSeparation Model

Source: https://x-tabdeveloping.github.io/topicwizard/application

This snippet illustrates how to use TopicWizard with a different topic modeling approach, `SemanticSignalSeparation` from the `turftopic` library. It shows two ways to pass the model: directly or after preparing topic data.

```python
from turftopic import SemanticSignalSeparation

model = SemanticSignalSeparation(n_components=10)

topicwizard.visualize(texts, model=model)
## OR

topic_data = model.prepare_topic_data(texts)
topicwizard.visualize(topic_data=topic_data)

```

--------------------------------

### Convert Existing Pipeline to TopicPipeline

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Converts an existing scikit-learn `Pipeline` object into a `TopicPipeline` using the `TopicPipeline.from_pipeline()` class method.

```python
from topicwizard.pipeline import TopicPipeline

topic_pipeline = TopicPipeline.from_pipeline(pipeline)
```

--------------------------------

### Generate a word map visualization

Source: https://x-tabdeveloping.github.io/topicwizard/index

Creates an interactive word map figure from prepared topic data. This helps in understanding the relationships between words within topics.

```python
from topicwizard.figures import word_map

topic_data = topic_pipeline.prepare_topic_data(corpus)

word_map(topic_data)

```

--------------------------------

### Configure TopicPipeline for DataFrame Output

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Shows how to configure a `TopicPipeline` to output pandas DataFrames instead of matrices. This can be achieved by setting `pandas_out=True` during pipeline creation or by using scikit-learn's `set_output` API.

```python
# Set a parameter
pipeline = make_topic_pipeline(vectorizer, model, pandas_out=True)
```

--------------------------------

### Exclude Pages for Performance Optimization

Source: https://x-tabdeveloping.github.io/topicwizard/application

This snippet shows how to improve TopicWizard's startup performance by excluding specific pages. By using the `exclude_pages` argument, you can disable computationally intensive visualizations like 'documents' and 'words', focusing only on the 'topics' page.

```python
topicwizard.visualize(texts, model=pipeline, exclude_pages=["documents", "words"])

```

--------------------------------

### Set TopicPipeline Output to Pandas

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Configures the TopicPipeline to output results in a pandas DataFrame format. This is useful for detailed analysis of topic content within individual documents.

```python
pipeline = make_topic_pipeline(vectorizer, model).set_output(transform="pandas")
```

--------------------------------

### Create Group Topic Barcharts with Topic Wizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Generates a joint plot of topic content for all groups, displayed as bar charts. Requires a corpus, group labels, and an optional pipeline. The `top_n` parameter controls the number of top topics to display.

```python
from topicwizard.figures import group_topic_barcharts

group_topic_barcharts(corpus, group_labels, pipeline=pipeline, top_n=5)
```

--------------------------------

### Create Group Word Clouds with Topic Wizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Generates word clouds for each group label based solely on word counts, ignoring topic relevance. Requires a corpus, group labels, and an optional pipeline.

```python
from topicwizard.figures import group_wordclouds

group_wordclouds(corpus, group_labels, pipeline=pipeline)
```

--------------------------------

### Generate Word Clouds using topicwizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Produces a joint word cloud plot of all topics. You can specify the relevance metric with the alpha keyword parameter.

```python
from topicwizard.figures import topic_wordclouds

topic_wordclouds(topic_data)
```

--------------------------------

### Generate Document Topic Distribution using topicwizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Displays topic distributions for a given document or list of documents on a bar chart. It can also display topic distribution over time in a single document on a line chart by using windows of tokens.

```python
from topicwizard.figures import document_topic_distribution

document_topic_distribution(
    topic_data,
    "New cure against type 2 diabetes in development."
)
```

```python
from topicwizard.figures import document_topic_timeline

document_topic_timeline(
    topic_data,
    "New cure against type 2 diabetes in development."
)
```

--------------------------------

### Freeze Pipeline Components for Downstream Tasks

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Demonstrates freezing the components of a fitted `TopicPipeline` to prevent them from being retrained in a subsequent pipeline, such as a classification pipeline. This is useful for using topic representations as features.

```python
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression

topic_pipeline = make_topic_pipeline(vectorizer, model).fit(texts)

# Investigate topics
topicwizard.visualize(topic_pipeline)

# Freezing topic pipeline
topic_pipeline.freeze = True
# Constructing classification pipeline
cls_pipeline = make_pipeline(topic_pipeline, LogisticRegression())
cls_pipeline.fit(X, y)
```

--------------------------------

### Generate Word Map using topicwizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Displays a word map where words are labeled based on a Z-value cutoff and colored by their most relevant topic. You can either let UMAP discover the axes or specify topic axes for projection.

```python
from topicwizard.figures import word_map

word_map(topic_data)
```

```python
word_map(
  topic_data,
  topic_axes=(
     "9_api_apis_register_automatedsarcasmgenerator",
     "4_study_studying_assessments_exams"
  )
)
```

--------------------------------

### Generate Document Map using topicwizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Displays a map of documents as a self-contained plot, showing how documents relate to each other and the underlying topics. Documents are colored by discrete topics and cannot be selected or searched.

```python
from topicwizard.figures import document_map

document_map(topic_data)
```

--------------------------------

### Configure TopicPipeline Row Normalization

Source: https://x-tabdeveloping.github.io/topicwizard/usage

Shows how to enable or disable row normalization in the TopicPipeline. Normalizing row importances to act as probabilities is useful for further calculations or thresholding.

```python
pipeline = make_topic_pipeline(vectorizer, model, norm_row=True)
```

```python
pipeline = make_topic_pipeline(vectorizer, model, norm_row=False)
```

--------------------------------

### Generate Word Association Barchart using topicwizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Visualizes the most relevant topics for a given set of words using barcharts. This function helps identify which topics contain specific words.

```python
from topicwizard.figures import word_association_barchart

word_association_barchart(topic_data, ["supreme", "court"])
```

--------------------------------

### Generate Word Barplots using topicwizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Displays a joint plot of all topics with word importances on a bar chart. You can specify the relevance metric with the alpha keyword and control the number of words displayed using top_n.

```python
from topicwizard.figures import topic_barcharts

topic_barcharts(topic_data)
```

```python
topic_barcharts(topic_data, top_n=5)
```

--------------------------------

### Generate Topic Map using topicwizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Displays a semantic map of topics in your model. This function takes a TopicData object and generates an interactive Plotly figure.

```python
from topicwizard.figures import topic_map

topic_map(topic_data)
```

--------------------------------

### Generate Group Map using topicwizard

Source: https://x-tabdeveloping.github.io/topicwizard/figures

Displays the group map as a standalone plot, with groups colored according to their dominant topic. This function takes TopicData and group labels as input.

```python
from topicwizard.figures import group_map

group_map(topic_data, group_labels)
```

--------------------------------

### TopicData Structure

Source: https://x-tabdeveloping.github.io/topicwizard/topic_data

The TopicData type is a TypedDict representing inference data for topic models. It contains information about the corpus, vocabulary, document-term matrices, topic matrices, document representations, transformation functions, and topic names, enabling the reproduction of visualizations.

```APIDOC
## topicwizard.data.TopicData

### Description
Inference data used to produce visualizations in the application and figures.

### Type
TypedDict (Python Dictionary)

### Fields
#### corpus
- **Type**: `list` of `str`
- **Description**: The corpus on which inference was run.

#### vocab
- **Type**: `ndarray` of `shape (n_vocab,)`
- **Description**: Array of all words in the vocabulary of the topic model.

#### document_term_matrix
- **Type**: `ndarray` of `shape (n_documents, n_vocab)`
- **Description**: Bag-of-words document representations. Elements of the matrix are word importances/frequencies for given documents.

#### document_topic_matrix
- **Type**: `ndarray` of `shape (n_documents, n_topics)`
- **Description**: Topic importances for each document.

#### topic_term_matrix
- **Type**: `ndarray` of `shape (n_topics, n_vocab)`
- **Description**: Importances of each term for each topic in a matrix.

#### document_representation
- **Type**: `ndarray` of `shape (n_documents, n_dimensions)`
- **Description**: Embedded representations for documents. Can also be a sparse BoW matrix for classical models.

#### transform
- **Type**: `(list[str]) -> ndarray`, optional
- **Description**: Function that transforms documents to document-topic matrices. Can be None in the case of transductive models.

#### topic_names
- **Type**: `list` of `str`
- **Description**: Names or topic descriptions inferred for topics by the model.

### Usage Examples
```python
import topicwizard
from topicwizard.figures import topic_map

# Assuming topic_data is a TopicData object

# Usage with figures
topic_map(topic_data)

# Usage with web app
topicwizard.visualize(topic_data=topic_data)
```
```
Tokens:
Raw