### Install topicwizard using pip Source: https://x-tabdeveloping.github.io/topicwizard/index Installs the topic-wizard package from PyPI. This is the primary method for getting topicwizard onto your system. ```bash pip install topic-wizard ``` -------------------------------- ### Create a topic model with turftopic Source: https://x-tabdeveloping.github.io/topicwizard/index Instantiates a KeyNMF model from the turftopic library, suitable for building contextually sensitive topic models. This requires turftopic to be installed. ```python from turftopic import KeyNMF model = KeyNMF(n_components=10) ``` -------------------------------- ### Initialize Topic Models: LDA, NMF, and DMM Source: https://x-tabdeveloping.github.io/topicwizard/usage Demonstrates initializing various topic models. Includes Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF) from scikit-learn, and Discrete Matrix Model (DMM) from tweetopic for short texts. Requires 'tweetopic' to be installed for DMM. ```python # LDA for long texts from sklearn.decomposition import LatentDirichletAllocation model = LatentDirichletAllocation(n_components=10) ``` ```python # You can use NMF too from sklearn.decomposition import NMF model = NMF(n_components=10) ``` ```python # Or tweetopic's DMM for short texts # pip install tweetopic from tweetopic import DMM model = DMM(n_components=10) ``` -------------------------------- ### Visualize with Group Labels using 20Newsgroups Source: https://x-tabdeveloping.github.io/topicwizard/application This example demonstrates how to include group labels in the TopicWizard visualization. It fetches the 20Newsgroups dataset, trains an NMF model, and passes the corpus, model, and group labels to `topicwizard.visualize()`. This adds an extra page for investigating label relations. ```python import topicwizard from topicwizard.pipeline import make_topic_pipeline from sklearn.datasets import fetch_20newsgroups import numpy as np from sklearn.decomposition import NMF from sklearn.feature_extraction.text import CountVectorizer newsgroups = fetch_20newsgroups(subset="all") corpus = newsgroups.data # Sklearn gives the labels back as integers, we have to map them back to # the actual textual label. group_labels = np.array(newsgroups.target_names)[newsgroups.target] # Here we fit a topic model to the corpus pipeline = make_topic_pipeline( CountVectorizer(stop_words="english"), NMF(n_components=30), ).fit(corpus) # Notice that I'm passing the labels as the group_labels argument topicwizard.visualize(corpus, model=pipeline, group_labels=group_labels) ``` -------------------------------- ### Visualize Embeddings with Excluded Pages Source: https://x-tabdeveloping.github.io/topicwizard/application This example demonstrates optimizing TopicWizard for visualizing word embeddings. By excluding 'documents' and 'topics' pages, it allows for focused visualization when using matrix decomposition methods like LSI for creating word embeddings. ```python topicwizard.visualize(texts, model=pipeline, exclude_pages=["documents", "topics"]) ``` -------------------------------- ### Visualize Topic Data with Topicwizard Figures and Web App Source: https://x-tabdeveloping.github.io/topicwizard/topic_data This snippet demonstrates how to use the TopicData object with topicwizard's visualization utilities. It shows how to generate a topic map figure and how to pass the TopicData object to the web app visualization function. Ensure topicwizard is installed. ```python import topicwizard from topicwizard.figures import topic_map # Assuming topic_data is already defined # Usage with figures topic_map(topic_data) # Usage with web app # Beware that topic_data is a keyword argument topicwizard.visualize(topic_data=topic_data) ``` -------------------------------- ### Get Topic Names from Fitted Pipeline Source: https://x-tabdeveloping.github.io/topicwizard/usage After fitting a `TopicPipeline` with text data, this code retrieves the inferred topic names using `get_feature_names_out()`. ```python topic_pipeline.fit(texts) print(topic_pipeline.get_feature_names_out()) ``` -------------------------------- ### Visualize topic models using topicwizard Source: https://x-tabdeveloping.github.io/topicwizard/index Launches the topicwizard web application for visual exploration of a topic model. It takes the text corpus and the trained model as input. ```python import topicwizard topicwizard.visualize(texts, model=model) ``` -------------------------------- ### Visualize Topic Model with TopicWizard Source: https://x-tabdeveloping.github.io/topicwizard/application This code shows the basic usage of the `topicwizard.visualize()` function to display a trained topic model. It takes the corpus (texts) and the fitted model pipeline as input, launching an interactive web application. ```python import topicwizard topicwizard.visualize(texts, model=pipeline) ``` -------------------------------- ### Visualize Topics and Classify Documents with TopicPipeline Source: https://x-tabdeveloping.github.io/topicwizard/usage Demonstrates transforming texts into topics using a pipeline and visualizing the results. It also shows how to create a rule-based classifier for specific topics, like 'corona', using FunctionClassifier and scikit-learn's make_pipeline. ```python import plotly.express as px texts = [ "Coronavirus killed 50000 people today.", "Donald Trump's presidential campaing is going very well", "Protests against police brutality have been going on all around the US.", ] topic_df = pipeline.transform(texts) topic_df.index = texts px.imshow(topic_df).show() ``` ```python # Install human-learn from PyPI # pip install human-learn from hulearn.classification import FunctionClassifier from sklearn.pipeline import make_pipeline topic_pipeline = make_topic_pipeline(vectorizer, model).fit(texts) # Investigate topics topicwizard.visualize(topic_pipeline) # Creating rule for classifying something as a corona document def corona_rule(df, threshold=0.5): is_about_corona = df["11_vaccine_pandemic_virus_coronavirus"] > threshold return is_about_corona.astype(int) # Freezing topic pipeline topic_pipeline.freeze = True classifier = FunctionClassifier(corona_rule) cls_pipeline = make_pipeline(topic_pipeline, classifier) ``` -------------------------------- ### Create topicwizard TopicPipeline Source: https://x-tabdeveloping.github.io/topicwizard/usage Creates a `TopicPipeline` using topicwizard's `make_topic_pipeline`. This is recommended over scikit-learn's `Pipeline` for better integration with topicwizard's features. ```python from topicwizard.pipeline import make_topic_pipeline topic_pipeline = make_topic_pipeline(vectorizer, model) ``` -------------------------------- ### Create a topic pipeline with scikit-learn models Source: https://x-tabdeveloping.github.io/topicwizard/index Builds a scikit-learn compatible topic model pipeline using CountVectorizer and NMF. This is suitable for classical topic modeling approaches. ```python from sklearn.decomposition import NMF from sklearn.feature_extraction.text import CountVectorizer from topicwizard.pipeline import make_topic_pipeline bow_vectorizer = CountVectorizer() nmf = NMF(n_components=10) model = make_topic_pipeline(bow_vectorizer, nmf) ``` -------------------------------- ### Initialize CountVectorizer for Text Vectorization Source: https://x-tabdeveloping.github.io/topicwizard/usage Initializes a CountVectorizer from scikit-learn to convert texts into bag-of-words vectors. This is a common first step in many text processing pipelines. ```python from sklearn.feature_extraction.text import CountVectorizer vectorizer = CountVectorizer() ``` -------------------------------- ### Create Scikit-learn Pipeline Source: https://x-tabdeveloping.github.io/topicwizard/usage Combines a vectorizer and a topic model into a scikit-learn pipeline using `make_pipeline`. This allows sequential processing of text data. ```python from sklearn.pipeline import make_pipeline topic_pipeline = make_pipeline(vectorizer, model) ``` -------------------------------- ### Train NMF Topic Model with TopicWizard Source: https://x-tabdeveloping.github.io/topicwizard/application This snippet demonstrates how to train a Non-negative Matrix Factorization (NMF) topic model using scikit-learn and prepare it for visualization with TopicWizard. It utilizes `CountVectorizer` for text processing and `make_topic_pipeline` for creating a unified pipeline. ```python # Training a compatible topic model from sklearn.decomposition import NMF from sklearn.feature_extraction.text import CountVectorizer from topicwizard.pipeline import make_topic_pipeline bow_vectorizer = CountVectorizer() nmf = NMF(n_components=10) pipeline = make_topic_pipeline(bow_vectorizer, nmf) pipeline.fit(texts) ``` -------------------------------- ### Visualize with SemanticSignalSeparation Model Source: https://x-tabdeveloping.github.io/topicwizard/application This snippet illustrates how to use TopicWizard with a different topic modeling approach, `SemanticSignalSeparation` from the `turftopic` library. It shows two ways to pass the model: directly or after preparing topic data. ```python from turftopic import SemanticSignalSeparation model = SemanticSignalSeparation(n_components=10) topicwizard.visualize(texts, model=model) ## OR topic_data = model.prepare_topic_data(texts) topicwizard.visualize(topic_data=topic_data) ``` -------------------------------- ### Convert Existing Pipeline to TopicPipeline Source: https://x-tabdeveloping.github.io/topicwizard/usage Converts an existing scikit-learn `Pipeline` object into a `TopicPipeline` using the `TopicPipeline.from_pipeline()` class method. ```python from topicwizard.pipeline import TopicPipeline topic_pipeline = TopicPipeline.from_pipeline(pipeline) ``` -------------------------------- ### Generate a word map visualization Source: https://x-tabdeveloping.github.io/topicwizard/index Creates an interactive word map figure from prepared topic data. This helps in understanding the relationships between words within topics. ```python from topicwizard.figures import word_map topic_data = topic_pipeline.prepare_topic_data(corpus) word_map(topic_data) ``` -------------------------------- ### Configure TopicPipeline for DataFrame Output Source: https://x-tabdeveloping.github.io/topicwizard/usage Shows how to configure a `TopicPipeline` to output pandas DataFrames instead of matrices. This can be achieved by setting `pandas_out=True` during pipeline creation or by using scikit-learn's `set_output` API. ```python # Set a parameter pipeline = make_topic_pipeline(vectorizer, model, pandas_out=True) ``` -------------------------------- ### Exclude Pages for Performance Optimization Source: https://x-tabdeveloping.github.io/topicwizard/application This snippet shows how to improve TopicWizard's startup performance by excluding specific pages. By using the `exclude_pages` argument, you can disable computationally intensive visualizations like 'documents' and 'words', focusing only on the 'topics' page. ```python topicwizard.visualize(texts, model=pipeline, exclude_pages=["documents", "words"]) ``` -------------------------------- ### Set TopicPipeline Output to Pandas Source: https://x-tabdeveloping.github.io/topicwizard/usage Configures the TopicPipeline to output results in a pandas DataFrame format. This is useful for detailed analysis of topic content within individual documents. ```python pipeline = make_topic_pipeline(vectorizer, model).set_output(transform="pandas") ``` -------------------------------- ### Create Group Topic Barcharts with Topic Wizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Generates a joint plot of topic content for all groups, displayed as bar charts. Requires a corpus, group labels, and an optional pipeline. The `top_n` parameter controls the number of top topics to display. ```python from topicwizard.figures import group_topic_barcharts group_topic_barcharts(corpus, group_labels, pipeline=pipeline, top_n=5) ``` -------------------------------- ### Create Group Word Clouds with Topic Wizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Generates word clouds for each group label based solely on word counts, ignoring topic relevance. Requires a corpus, group labels, and an optional pipeline. ```python from topicwizard.figures import group_wordclouds group_wordclouds(corpus, group_labels, pipeline=pipeline) ``` -------------------------------- ### Generate Word Clouds using topicwizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Produces a joint word cloud plot of all topics. You can specify the relevance metric with the alpha keyword parameter. ```python from topicwizard.figures import topic_wordclouds topic_wordclouds(topic_data) ``` -------------------------------- ### Generate Document Topic Distribution using topicwizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Displays topic distributions for a given document or list of documents on a bar chart. It can also display topic distribution over time in a single document on a line chart by using windows of tokens. ```python from topicwizard.figures import document_topic_distribution document_topic_distribution( topic_data, "New cure against type 2 diabetes in development." ) ``` ```python from topicwizard.figures import document_topic_timeline document_topic_timeline( topic_data, "New cure against type 2 diabetes in development." ) ``` -------------------------------- ### Freeze Pipeline Components for Downstream Tasks Source: https://x-tabdeveloping.github.io/topicwizard/usage Demonstrates freezing the components of a fitted `TopicPipeline` to prevent them from being retrained in a subsequent pipeline, such as a classification pipeline. This is useful for using topic representations as features. ```python from sklearn.pipeline import make_pipeline from sklearn.linear_model import LogisticRegression topic_pipeline = make_topic_pipeline(vectorizer, model).fit(texts) # Investigate topics topicwizard.visualize(topic_pipeline) # Freezing topic pipeline topic_pipeline.freeze = True # Constructing classification pipeline cls_pipeline = make_pipeline(topic_pipeline, LogisticRegression()) cls_pipeline.fit(X, y) ``` -------------------------------- ### Generate Word Map using topicwizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Displays a word map where words are labeled based on a Z-value cutoff and colored by their most relevant topic. You can either let UMAP discover the axes or specify topic axes for projection. ```python from topicwizard.figures import word_map word_map(topic_data) ``` ```python word_map( topic_data, topic_axes=( "9_api_apis_register_automatedsarcasmgenerator", "4_study_studying_assessments_exams" ) ) ``` -------------------------------- ### Generate Document Map using topicwizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Displays a map of documents as a self-contained plot, showing how documents relate to each other and the underlying topics. Documents are colored by discrete topics and cannot be selected or searched. ```python from topicwizard.figures import document_map document_map(topic_data) ``` -------------------------------- ### Configure TopicPipeline Row Normalization Source: https://x-tabdeveloping.github.io/topicwizard/usage Shows how to enable or disable row normalization in the TopicPipeline. Normalizing row importances to act as probabilities is useful for further calculations or thresholding. ```python pipeline = make_topic_pipeline(vectorizer, model, norm_row=True) ``` ```python pipeline = make_topic_pipeline(vectorizer, model, norm_row=False) ``` -------------------------------- ### Generate Word Association Barchart using topicwizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Visualizes the most relevant topics for a given set of words using barcharts. This function helps identify which topics contain specific words. ```python from topicwizard.figures import word_association_barchart word_association_barchart(topic_data, ["supreme", "court"]) ``` -------------------------------- ### Generate Word Barplots using topicwizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Displays a joint plot of all topics with word importances on a bar chart. You can specify the relevance metric with the alpha keyword and control the number of words displayed using top_n. ```python from topicwizard.figures import topic_barcharts topic_barcharts(topic_data) ``` ```python topic_barcharts(topic_data, top_n=5) ``` -------------------------------- ### Generate Topic Map using topicwizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Displays a semantic map of topics in your model. This function takes a TopicData object and generates an interactive Plotly figure. ```python from topicwizard.figures import topic_map topic_map(topic_data) ``` -------------------------------- ### Generate Group Map using topicwizard Source: https://x-tabdeveloping.github.io/topicwizard/figures Displays the group map as a standalone plot, with groups colored according to their dominant topic. This function takes TopicData and group labels as input. ```python from topicwizard.figures import group_map group_map(topic_data, group_labels) ``` -------------------------------- ### TopicData Structure Source: https://x-tabdeveloping.github.io/topicwizard/topic_data The TopicData type is a TypedDict representing inference data for topic models. It contains information about the corpus, vocabulary, document-term matrices, topic matrices, document representations, transformation functions, and topic names, enabling the reproduction of visualizations. ```APIDOC ## topicwizard.data.TopicData ### Description Inference data used to produce visualizations in the application and figures. ### Type TypedDict (Python Dictionary) ### Fields #### corpus - **Type**: `list` of `str` - **Description**: The corpus on which inference was run. #### vocab - **Type**: `ndarray` of `shape (n_vocab,)` - **Description**: Array of all words in the vocabulary of the topic model. #### document_term_matrix - **Type**: `ndarray` of `shape (n_documents, n_vocab)` - **Description**: Bag-of-words document representations. Elements of the matrix are word importances/frequencies for given documents. #### document_topic_matrix - **Type**: `ndarray` of `shape (n_documents, n_topics)` - **Description**: Topic importances for each document. #### topic_term_matrix - **Type**: `ndarray` of `shape (n_topics, n_vocab)` - **Description**: Importances of each term for each topic in a matrix. #### document_representation - **Type**: `ndarray` of `shape (n_documents, n_dimensions)` - **Description**: Embedded representations for documents. Can also be a sparse BoW matrix for classical models. #### transform - **Type**: `(list[str]) -> ndarray`, optional - **Description**: Function that transforms documents to document-topic matrices. Can be None in the case of transductive models. #### topic_names - **Type**: `list` of `str` - **Description**: Names or topic descriptions inferred for topics by the model. ### Usage Examples ```python import topicwizard from topicwizard.figures import topic_map # Assuming topic_data is a TopicData object # Usage with figures topic_map(topic_data) # Usage with web app topicwizard.visualize(topic_data=topic_data) ``` ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.