### Install Docusaurus Globally

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started-installation.md

Alternative installation method if you prefer to install Docusaurus globally. This is useful if you encounter issues with local installations or prefer global access.

```bash
yarn global add docusaurus-init
```

```bash
npm install --global docusaurus-init
```

--------------------------------

### Install Texthero

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started.md

Install Texthero using pip. Use the -U flag to upgrade to the latest version.

```bash
pip install texthero
```

```bash
pip install texthero -U
```

--------------------------------

### Docstring Example with Long Pipe

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Illustrates how to format long sequences of pipe operations within a docstring example by enclosing the code in parentheses.

```python
>>> s = (
...     s.pipe(hero.clean)
...      .pipe(hero.tokenize)
...      .pipe(hero.term_frequency)
...      .pipe(hero.flatten)
... )
```

--------------------------------

### Start Docusaurus Development Server

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started-installation.md

Navigate to the 'website' directory and run this command to start the local development server. The site will be available at http://localhost:3000.

```bash
cd website
yarn start
```

```bash
cd website
npm start
```

--------------------------------

### Text Preprocessing, TF-IDF, K-means, PCA, and Scatter Plot

Source: https://github.com/jbesomi/texthero/blob/master/examples/README.md.ipynb

This example preprocesses text, computes TF-IDF, performs K-means clustering, applies PCA, and visualizes the results with a scatter plot. Ensure pandas and texthero are installed.

```python
import texthero as hero
import pandas as pd

df = pd.read_csv(
    "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)

df['tfidf'] = (
    df['text']
    .pipe(hero.clean)
    .pipe(hero.tfidf)
)

df['kmeans_labels'] = (
    df['tfidf']
    .pipe(hero.kmeans, n_clusters=5)
    .astype(str)
)

df['pca'] = (
    df['tfidf']
    .pipe(hero.pca)
)

fig = hero.scatterplot(df, 'pca', color='kmeans_labels', title="K-means BBC Sport news", return_figure=True)
fig.write_image("../github/scatterplot_bbcsport_kmeans.svg")
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Install texthero along with its development dependencies. This is required for all contributors to run tests and other development tasks.

```bash
pip install -e '.[dev]'
```

--------------------------------

### Initialize Docusaurus Project

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started-installation.md

Run this command to set up a new Docusaurus project. Ensure Node.js and Yarn are installed.

```bash
npx docusaurus-init
```

--------------------------------

### Install texthero in Development Mode

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Install texthero locally from the source code in development mode. The `-e` flag ensures changes take effect immediately without reinstallation.

```bash
pip install -e .
```

--------------------------------

### Full Example: PCA Scatter Plot

Source: https://github.com/jbesomi/texthero/blob/master/examples/getting-started.ipynb

A complete example demonstrating loading data, performing text cleaning, TF-IDF, PCA, and generating a scatter plot visualization.

```python
import texthero as hero
import pandas as pd

df = pd.read_csv(
    "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)
df['pca'] = (
    df['text']
    .pipe(hero.clean)
    .pipe(hero.tfidf)
    .pipe(hero.pca)
)

hero.scatterplot(df, col='pca', color='topic', title="PCA BBC Sport news")
```

--------------------------------

### Install Texthero

Source: https://github.com/jbesomi/texthero/blob/master/README.md

Install the Texthero library using pip. This command will also install necessary dependencies like Gensim, NLTK, SpaCy, and scikit-learn.

```bash
pip install texthero
```

--------------------------------

### Text Exploration Pipeline Example

Source: https://context7.com/jbesomi/texthero/llms.txt

An example demonstrating a text exploration pipeline using Texthero for cleaning, entity extraction, finding top words, and generating a word cloud.

```APIDOC
## Text Exploration Pipeline

### Description
This example demonstrates a text exploration pipeline using Texthero. It covers cleaning text, extracting named entities, identifying the most frequent words, and generating a word cloud visualization.

### Method
This is a pipeline example and does not represent a single API endpoint. It utilizes multiple Texthero functions in sequence.

### Endpoint
N/A

### Parameters
N/A

### Request Example
```python
import texthero as hero
import pandas as pd

# Sample data
df = pd.DataFrame({
    "text": [
        "Apple announces new iPhone with better camera",
        "Google releases new Android update",
        "Microsoft acquires gaming company",
        "Amazon reports record quarterly earnings"
    ]
})

# Clean and analyze
df['clean'] = hero.clean(df['text'])

# Extract entities
df['entities'] = hero.named_entities(df['text'])

# Get top words
top = hero.top_words(df['clean'])
print("Most common words:", top.head(10))

# Visualize word cloud
hero.wordcloud(df['clean'])
```

### Response
#### Success Response (200)
Prints the most common words to the console and displays a word cloud visualization.
```

--------------------------------

### Example Docstring for remove_digits Function

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Demonstrates a correctly formatted docstring for a Python function, including parameters, optional arguments with defaults, and doctest examples.

```python
def remove_digits(input: pd.Series, only_blocks=True) -> pd.Series:
   """
   Remove all digits from a series and replace it with a single space.

   Parameters
   ----------

   input : pd.Series
   only_blocks : bool, optional, default=True
               Remove only blocks of digits. For instance, `hel1234lo 1234` becomes `hel1234lo`.

   Examples
   --------
   >>> s = pd.Series("7ex7hero is fun 1111")
   >>> remove_digits(s)
   0    7ex7hero is fun 
   dtype: object
   >>> remove_digits(s, only_blocks=False)
   0    exhero is fun 
   dtype: object
   """
```

--------------------------------

### Text Exploration Pipeline Example

Source: https://context7.com/jbesomi/texthero/llms.txt

This snippet demonstrates a text exploration pipeline. It includes cleaning text, extracting named entities, finding the most common words, and generating a word cloud.

```python
import texthero as hero
import pandas as pd

# Sample data
df = pd.DataFrame({
    "text": [
        "Apple announces new iPhone with better camera",
        "Google releases new Android update",
        "Microsoft acquires gaming company",
        "Amazon reports record quarterly earnings"
    ]
})

# Clean and analyze
df['clean'] = hero.clean(df['text'])

# Extract entities
df['entities'] = hero.named_entities(df['text'])

# Get top words
top = hero.top_words(df['clean'])
print("Most common words:", top.head(10))

# Visualize word cloud
hero.wordcloud(df['clean'])
```

--------------------------------

### Text Classification Pipeline Example

Source: https://context7.com/jbesomi/texthero/llms.txt

An example demonstrating a complete text classification pipeline using Texthero for preprocessing, vectorization, and clustering.

```APIDOC
## Text Classification Pipeline

### Description
This example demonstrates a complete text classification pipeline using Texthero. It includes cleaning text, creating TF-IDF vectors, performing K-means clustering, reducing dimensions with PCA, and visualizing the results.

### Method
This is a pipeline example and does not represent a single API endpoint. It utilizes multiple Texthero functions in sequence.

### Endpoint
N/A

### Parameters
N/A

### Request Example
```python
import texthero as hero
import pandas as pd

# Load data
df = pd.read_csv("https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv")

# Complete pipeline: clean, vectorize, reduce dimensions, cluster
df['clean_text'] = hero.clean(df['text'])
df['tfidf'] = df['clean_text'].pipe(hero.tfidf)
df['kmeans_labels'] = df['tfidf'].pipe(hero.kmeans, n_clusters=5).astype(str)
df['pca'] = df['tfidf'].pipe(hero.pca)

# Visualize results
hero.scatterplot(df, 'pca', color='kmeans_labels', title="K-means BBC Sport news")
```

### Response
#### Success Response (200)
Displays a scatter plot visualizing the document clusters based on PCA and K-means labels.
```

--------------------------------

### Enable Git Pre-commit Hook

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Install the git pre-commit hook to automatically format code before staging changes.

```bash
pre-commit install
```

--------------------------------

### Text Preprocessing, TF-IDF, K-means, and Visualization

Source: https://github.com/jbesomi/texthero/blob/master/README.md

This example shows a pipeline involving text cleaning, TF-IDF representation, K-means clustering, PCA for dimensionality reduction, and visualization of the clustered data.

```python
import texthero as hero
import pandas as pd

df = pd.read_csv(
    "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)

df['tfidf'] = (
    df['text']
    .pipe(hero.clean)
    .pipe(hero.tfidf)
)

df['kmeans_labels'] = (
    df['tfidf']
    .pipe(hero.kmeans, n_clusters=5)
    .astype(str)
)

df['pca'] = df['tfidf'].pipe(hero.pca)

hero.scatterplot(df, 'pca', color='kmeans_labels', title="K-means BBC Sport news")
```

--------------------------------

### Text Classification Pipeline Example

Source: https://context7.com/jbesomi/texthero/llms.txt

This example illustrates a complete text classification pipeline using Texthero. It loads data, cleans text, applies TF-IDF, performs K-means clustering, and visualizes the results.

```python
import texthero as hero
import pandas as pd

# Load data
df = pd.read_csv("https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv")

# Complete pipeline: clean, vectorize, reduce dimensions, cluster
df['clean_text'] = hero.clean(df['text'])
df['tfidf'] = df['clean_text'].pipe(hero.tfidf)
df['kmeans_labels'] = df['tfidf'].pipe(hero.kmeans, n_clusters=5).astype(str)
df['pca'] = df['tfidf'].pipe(hero.pca)

# Visualize results
hero.scatterplot(df, 'pca', color='kmeans_labels', title="K-means BBC Sport news")
```

--------------------------------

### Import Superheroes NLP Dataset with Pandas

Source: https://github.com/jbesomi/texthero/blob/master/dataset/Superheroes NLP Dataset/README.md

Use this code to load the complete dataset directly from GitHub into a pandas DataFrame. Ensure you have pandas installed.

```python
import pandas as pd

df = pd.read_csv("https://raw.githubusercontent.com/jbesomi/texthero/master/dataset/superheroes_nlp_dataset.csv")

df.head()
```

--------------------------------

### Text Cleaning, TF-IDF, and PCA Visualization

Source: https://github.com/jbesomi/texthero/blob/master/README.md

This example demonstrates a common Texthero pipeline: cleaning text, applying TF-IDF for representation, performing PCA for dimensionality reduction, and visualizing the results with a scatterplot.

```python
import texthero as hero
import pandas as pd

df = pd.read_csv(
   "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)

df['pca'] = (
   df['text']
   .pipe(hero.clean)
   .pipe(hero.tfidf)
   .pipe(hero.pca)
)
hero.scatterplot(df, 'pca', color='topic', title="PCA BBC Sport news")
```

--------------------------------

### Proxy Configuration for Docusaurus Server

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started-installation.md

If you are behind a corporate proxy, set the NO_PROXY environment variable to 'localhost' before starting the Docusaurus server. This ensures the development server can make requests without proxy interference.

```sh
SET NO_PROXY=localhost
yarn start (or npm run start)
```

--------------------------------

### Remove stopwords using default NLTK list

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api/texthero.preprocessing.remove_stopwords.md

Use this function to remove common English stopwords from your text data. Ensure you have NLTK installed and its stopwords corpus downloaded. This example demonstrates removing stopwords from a single string.

```python
import texthero as hero
import pandas as pd
s = pd.Series("Texthero is not only for the heroes")
hero.remove_stopwords(s)
```

--------------------------------

### Extract Named Entities with SpaCy

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api/texthero.nlp.named_entities.md

Use this function to extract named entities from a Pandas Series. Ensure SpaCy is installed and configured. The output is a list of tuples, each containing the entity name, label, and its starting and ending character positions.

```python
import texthero as hero
import pandas as pd
s = pd.Series("Yesterday I was in NY with Bill de Blasio")
hero.named_entities(s)[0]
[('Yesterday', 'DATE', 0, 9), ('NY', 'GPE', 19, 21), ('Bill de Blasio', 'PERSON', 27, 41)]
```

--------------------------------

### Connect to Upstream Repository

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Connect your cloned repository to the original upstream repository. This command should be run once.

```bash
cd texthero
git remote add upstream git@github.com:jbesomi/texthero.git
```

--------------------------------

### Get Most Frequent Words

Source: https://context7.com/jbesomi/texthero/llms.txt

This snippet demonstrates how to retrieve the most frequent words from a Pandas Series. It also shows how to get normalized frequencies.

```python
import texthero as hero
import pandas as pd

s = pd.Series(["one two two three three three"])
top = hero.top_words(s)
print(top)
# three    3
# two      2
# one      1
# dtype: int64
```

```python
# Normalized frequencies
top_normalized = hero.top_words(s, normalize=True)
print(top_normalized)
# three    0.5
# two      0.333
# one      0.167
# dtype: float64
```

--------------------------------

### Execute Tests and Formatting Scripts

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Scripts available for testing, formatting, and checking code quality. `./tests.sh` runs unit tests and doctests, `./format.sh` formats code with black, and `./check.sh` formats code and updates documentation.

```bash
./tests.sh
```

```bash
./format.sh
```

```bash
./check.sh
```

--------------------------------

### hero.top_words - Get Most Frequent Words

Source: https://context7.com/jbesomi/texthero/llms.txt

Returns the most frequent words in the text corpus, with an option for normalized frequencies.

```APIDOC
## hero.top_words - Get Most Frequent Words

### Description
Returns the most frequent words in the text corpus.

### Method
`hero.top_words(s, normalize=False)`

### Parameters
#### Path Parameters
None

#### Query Parameters
- **s** (pd.Series) - Required - The pandas Series containing the text data.
- **normalize** (bool) - Optional - If True, returns normalized frequencies (0 to 1).

### Request Example
```python
import texthero as hero
import pandas as pd

s = pd.Series(["one two two three three three"])
top = hero.top_words(s)
print(top)

top_normalized = hero.top_words(s, normalize=True)
print(top_normalized)
```

### Response
#### Success Response (200)
A pandas Series with words as index and their frequencies (or normalized frequencies) as values.

#### Response Example
```
three    3
two      2
one      1
dtype: int64

three    0.5
two      0.333
one      0.167
dtype: float64
```
```

--------------------------------

### Running Texthero Tests (Alternative)

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

An alternative command to execute all unit tests using Python's unittest module, equivalent to running ./tests.sh.

```bash
python3 -m unittest discover -s tests -t .
```

--------------------------------

### Create a New Working Branch

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Create a new branch for your work. Naming the branch descriptively, including an issue tracker ID if applicable, is a good practice.

```bash
git checkout -b new-branch
```

--------------------------------

### Load Dataset

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started.md

Load the BBC Sport dataset into a Pandas DataFrame for analysis.

```python
df = pd.read_csv(
   "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)
```

--------------------------------

### Stage and Commit Changes

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Stage specific files and commit them with a descriptive message. It's recommended to commit regularly and group related changes.

```bash
git add README.md
git commit -m "added README.md"
```

--------------------------------

### texthero.nlp.named_entities

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api/texthero.nlp.named_entities.md

Extracts named entities from a Pandas Series using Spacy. Returns a Series where each element is a list of tuples, with each tuple containing the entity text, its label, and its start and end character positions.

```APIDOC
## texthero.nlp.named_entities

### Description
Return named-entities. Return a Pandas Series where each rows contains a list of tuples containing information regarding the given named entities.

Tuple: (entity’name, entity’label, starting character, ending character)

Under the hood, named_entities make use of Spacy name entity recognition.

### Method
POST

### Endpoint
/texthero/nlp/named_entities

### Parameters
#### Request Body
- **s** (Pandas Series) - Required - The input Series containing text data.
- **package** (string) - Optional - The NLP package to use. Defaults to 'spacy'.

### Request Example
```json
{
  "s": "Yesterday I was in NY with Bill de Blasio",
  "package": "spacy"
}
```

### Response
#### Success Response (200)
- **entities** (list of tuples) - A list where each tuple represents a named entity with the format (entity_text, entity_label, start_char, end_char).

List of labels:
- PERSON: People, including fictional.
- NORP: Nationalities or religious or political groups.
- FAC: Buildings, airports, highways, bridges, etc.
- ORG: Companies, agencies, institutions, etc.
- GPE: Countries, cities, states.
- LOC: Non-GPE locations, mountain ranges, bodies of water.
- PRODUCT: Objects, vehicles, foods, etc. (Not services.)
- EVENT: Named hurricanes, battles, wars, sports events, etc.
- WORK_OF_ART: Titles of books, songs, etc.
- LAW: Named documents made into laws.
- LANGUAGE: Any named language.
- DATE: Absolute or relative dates or periods.
- TIME: Times smaller than a day.
- PERCENT: Percentage, including "%".
- MONEY: Monetary values, including unit.
- QUANTITY: Measurements, as of weight or distance.
- ORDINAL: "first", "second", etc.
- CARDINAL: Numerals that do not fall under another type.

#### Response Example
```json
{
  "entities": [
    ["Yesterday", "DATE", 0, 9],
    ["NY", "GPE", 19, 21],
    ["Bill de Blasio", "PERSON", 27, 41]
  ]
}
```
```

--------------------------------

### Import Libraries

Source: https://github.com/jbesomi/texthero/blob/master/examples/getting-started.ipynb

Import necessary libraries for TextHero and Pandas operations.

```python
import texthero as hero
import pandas as pd
```

--------------------------------

### Replace stopwords with a symbol

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api/texthero.preprocessing.replace_stopwords.md

Use this function to replace stopwords in a Pandas Series with a specified symbol. By default, it uses NLTK's English stopwords. Ensure NLTK is installed and stopwords are downloaded if using the default.

```python
import pandas as pd
from texthero.preprocessing import replace_stopwords

s = pd.Series("the book of the jungle")
replace_stopwords(s, "X")
```

--------------------------------

### Formatting Texthero Code

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Format all code within the Texthero project using the black formatter. This script should be run before submitting code.

```bash
cd scripts
./format.sh
```

--------------------------------

### Remove stopwords with custom list

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api/texthero.preprocessing.remove_stopwords.md

Extend the default NLTK stopwords with your own custom words to remove. This is useful for domain-specific text where certain terms should also be treated as stopwords. The example shows adding 'heroes' to the default list.

```python
import texthero as hero
from texthero import stopwords
import pandas as pd
default_stopwords = stopwords.DEFAULT
custom_stopwords = default_stopwords.union(set(["heroes"]))
s = pd.Series("Texthero is not only for the heroes")
hero.remove_stopwords(s, custom_stopwords)
```

--------------------------------

### Superheroes NLP Dataset Preview

Source: https://github.com/jbesomi/texthero/blob/master/dataset/Superheroes NLP Dataset/README.md

This is a preview of the first 5 rows and 81 columns of the Superheroes NLP Dataset after being loaded into a pandas DataFrame.

```bash
            name               real_name               full_name overall_score  ... has_durability has_stamina  has_agility  has_super_strength
0        3-D Man     Delroy Garrett, Jr.     Delroy Garrett, Jr.             6  ...            0.0         0.0          0.0                 1.0
1  514A (Gotham)             Bruce Wayne                     NaN            10  ...            1.0         0.0          0.0                 1.0
2         A-Bomb  Richard Milhouse Jones  Richard Milhouse Jones            20  ...            1.0         1.0          1.0                 1.0
3             Aa                      Aa                     NaN            12  ...            0.0         0.0          0.0                 0.0
4     Aaron Cash              Aaron Cash              Aaron Cash             5  ...            0.0         0.0          0.0                 0.0

[5 rows x 81 columns]
```

--------------------------------

### Calculate Term Frequency with Feature Names

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api/texthero.representation.term_frequency.md

To also get the list of unique words (features) used in the term frequency calculation, set return_feature_names to True. This returns a tuple containing the term frequency Series and the feature names.

```python
import texthero as hero
import pandas as pd
s = pd.Series(["Sentence one", "Sentence two"])
hero.term_frequency(s, return_feature_names=True)
```

--------------------------------

### Groupby, Max, Stack, and Rename in Pandas

Source: https://github.com/jbesomi/texthero/blob/master/website/blog/2020-04-27-rename-columns-pandas.md

This snippet shows how to group a DataFrame by 'artist', find the maximum value for each group, stack the resulting DataFrame, rename the index levels to 'artist' and 'sentiment', and reset the index to create a new DataFrame with a 'r' column.

```python
df_empath = (
    df_empath.groupby(['artist'])
             .max()
             .stack()
             .rename_axis(['artist', 'sentiment'])
             .reset_index(name='r')
)
```

--------------------------------

### Running Texthero Tests

Source: https://github.com/jbesomi/texthero/blob/master/CONTRIBUTING.md

Execute all unit tests for the Texthero project. This command also runs doctests embedded in docstrings.

```bash
$ cd scripts
$ ./tests.sh
```

--------------------------------

### Import Libraries and Create Pandas Series

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api/texthero.representation.pca.md

Imports necessary libraries (texthero and pandas) and creates a sample Pandas Series for text data. This is a prerequisite for using the pca function.

```python
import texthero as hero
import pandas as pd
s = pd.Series(["Sentence one", "Sentence two"])
```

--------------------------------

### Initialize Texthero Series

Source: https://github.com/jbesomi/texthero/blob/master/README.md

Initializes a pandas Series with text data for subsequent preprocessing steps.

```python
import texthero as hero
import pandas as pd
text = "This sèntencé    (123 /) needs to [OK!] be cleaned!   "
s = pd.Series(text)
s
```

--------------------------------

### Generate Word Cloud Visualization

Source: https://context7.com/jbesomi/texthero/llms.txt

This snippet shows how to clean a Pandas Series of text and generate a word cloud visualization. It also demonstrates how to return the figure object for further customization or saving.

```python
import texthero as hero
import pandas as pd

s = pd.Series([
    "Python programming data science",
    "Machine learning deep learning AI",
    "Data analysis visualization Python"
])

# Clean and generate wordcloud
s_clean = hero.clean(s)
hero.wordcloud(s_clean, width=800, height=400, max_words=100)
```

```python
# Return figure for saving
fig = hero.wordcloud(s_clean, return_figure=True)
fig.savefig("wordcloud.png")
```

--------------------------------

### Display DataFrame Head

Source: https://github.com/jbesomi/texthero/blob/master/examples/getting-started.ipynb

Displays the first two rows of the DataFrame to quickly inspect the data.

```python
df.head(2)
```

--------------------------------

### Create Scatter Plot for Document Vectors

Source: https://context7.com/jbesomi/texthero/llms.txt

Generate interactive 2D or 3D scatter plots to visualize document vectors. Requires a DataFrame with text data.

```python
import texthero as hero
import pandas as pd

df = pd.DataFrame({
    "texts": [
        "Football Sports Soccer",
        "music violin orchestra",
        "football fun sports",
        "music fun guitar"
    ]
})
```

--------------------------------

### Cluster Documents with K-Means

Source: https://context7.com/jbesomi/texthero/llms.txt

Apply K-Means clustering to group documents into a specified number of clusters. Requires preprocessed, tokenized, and term-frequency-weighted text data.

```python
import texthero as hero
import pandas as pd

s = pd.Series([
    "Football Sports Soccer",
    "music violin orchestra",
    "football fun sports",
    "music fun guitar"
])
s = s.pipe(hero.clean).pipe(hero.tokenize).pipe(hero.term_frequency)

clusters = hero.kmeans(s, n_clusters=2, random_state=42)
print(clusters)
```

--------------------------------

### Part-of-Speech (POS) Tagging

Source: https://context7.com/jbesomi/texthero/llms.txt

Returns part-of-speech tags for each token in the text. Returns a list of POS tags for each document.

```APIDOC
## hero.pos_tag - Part-of-Speech Tagging

### Description
Returns part-of-speech tags for each token in the text.

### Method
`hero.pos_tag`

### Parameters
#### Path Parameters
None

#### Query Parameters
- **s** (pd.Series) - Required - Input Series of text documents.

### Request Example
```python
import texthero as hero
import pandas as pd

s = pd.Series(["Today is such a beautiful day"])
pos_tags = hero.pos_tag(s)
print(pos_tags[0])
```

### Response
#### Success Response (200)
- **pos_tags** (pd.Series) - Series where each element is a list of POS tags for the corresponding document. Each tag is a tuple: (token, token_type, tag, start_char, end_char).

#### Response Example
```
# [('Today', 'NOUN', 'NN', 0, 5), ('is', 'AUX', 'VBZ', 6, 8), ('such', 'DET', 'PDT', 9, 13), ('a', 'DET', 'DT', 14, 15), ('beautiful', 'ADJ', 'JJ', 16, 25), ('day', 'NOUN', 'NN', 26, 29)]
```
```

--------------------------------

### Tokenize Text with texthero.tokenize

Source: https://context7.com/jbesomi/texthero/llms.txt

Splits text into a list of words while handling punctuation intelligently. Requires pandas and texthero imports.

```python
import texthero as hero
import pandas as pd

s = pd.Series(["Today you're looking great!"])
result = hero.tokenize(s)
print(result)
```

--------------------------------

### Tokenization

Source: https://context7.com/jbesomi/texthero/llms.txt

Function for splitting text into tokens.

```APIDOC
## hero.tokenize - Split Text into Tokens

### Description
Tokenizes text into a list of words while handling punctuation intelligently.

### Method
Not specified (assumed to be a Series transformation)

### Endpoint
Not applicable (library function)

### Parameters
- **s** (pd.Series) - The input Series containing text.

### Request Example
```python
import texthero as hero
import pandas as pd

s = pd.Series(["Today you're looking great!"])
result = hero.tokenize(s)
print(result)
```

### Response
#### Success Response (200)
- **result** (pd.Series) - Series where each element is a list of tokens.

#### Response Example
```
0    [Today, you're, looking, great, !]
dtype: object
```
```

--------------------------------

### Groupby, Mean, Stack, and Rename in Pandas

Source: https://github.com/jbesomi/texthero/blob/master/website/blog/2020-04-27-rename-columns-pandas.md

Use this pattern to group a DataFrame by a column, calculate the mean of the remaining columns, stack the result, rename the index levels, and reset the index to a DataFrame.

```python
df.groupby(['artist']).mean().stack().rename_axis(['one', 'bar']).reset_index(name='ooo')
```

--------------------------------

### Perform NMF for Topic Discovery

Source: https://context7.com/jbesomi/texthero/llms.txt

Use Non-Negative Matrix Factorization for topic discovery and dimensionality reduction on text data. Requires preprocessed and tokenized text data.

```python
import texthero as hero
import pandas as pd

s = pd.Series(["Football Sports Soccer", "Music Violin Orchestra", "Football Music"])
s = s.pipe(hero.clean).pipe(hero.tokenize).pipe(hero.term_frequency)

nmf_result = hero.nmf(s, n_components=2, random_state=42)
print(nmf_result)
```

--------------------------------

### Chained Text Processing Pipeline

Source: https://github.com/jbesomi/texthero/blob/master/examples/getting-started.ipynb

Demonstrates chaining multiple TextHero operations (clean, tfidf, pca) using the pipe method for efficient text processing and dimensionality reduction.

```python
df['pca'] = (
            df['text']
            .pipe(hero.clean)
            .pipe(hero.tfidf)
            .pipe(hero.pca)
   )
```

--------------------------------

### Stemming without Punctuation

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started-preprocessing.md

Illustrates the `stem` function from TextHero when the input text lacks punctuation. This shows how stemming affects the words differently compared to text with punctuation.

```python
>>> text = "I love climbing and running"
>>> hero .stem(pd.Series(text), stem="snowball")
   0    i love climb and run
   dtype: object
```

--------------------------------

### Custom Text Cleaning Pipeline

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started.md

Apply a custom cleaning pipeline to the text data by specifying a list of preprocessing functions.

```python
from texthero import preprocessing

custom_pipeline = [preprocessing.fillna,
                   preprocessing.lowercase,
                   preprocessing.remove_whitespace]
df['clean_text'] = hero.clean(df['text'], custom_pipeline)
```

--------------------------------

### Update Docusaurus to Latest Version

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started-installation.md

Execute these commands to upgrade Docusaurus to the latest version. This is recommended for accessing new features and bug fixes.

```bash
yarn upgrade docusaurus --latest
```

```bash
npm update docusaurus
```

--------------------------------

### Custom Text Cleaning Pipeline

Source: https://github.com/jbesomi/texthero/blob/master/examples/getting-started.ipynb

Applies a custom preprocessing pipeline to clean text data. This allows for fine-grained control over cleaning steps like filling missing values, lowercasing, and removing whitespace.

```python
from texthero import preprocessing

custom_pipeline = [preprocessing.fillna,
                   preprocessing.lowercase,
                   preprocessing.remove_whitespace]
df['clean_text'] = hero.clean(df['text'])
```

```python
df['clean_text'] = df['clean_text'].pipe(hero.clean, custom_pipeline)
```

--------------------------------

### PCA Scatter Plot Visualization

Source: https://github.com/jbesomi/texthero/blob/master/examples/getting-started.ipynb

Generates a scatter plot of the PCA results, coloring points by topic. This visualization helps in understanding the separation and clustering of different topics in the reduced dimensional space.

```python
hero.scatterplot(df, col='pca', color='topic', title="PCA BBC Sport news")
```

--------------------------------

### Stemming with Punctuation

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/getting-started-preprocessing.md

Demonstrates the `stem` function from TextHero when the input text contains punctuation. It's recommended to use `remove_punctuation` before `stem` for better results.

```python
>>> text = "I love climbing and running."
>>> hero .stem(pd.Series(text), stem="snowball")
   0    i love climb and running.
   dtype: object
```

--------------------------------

### Ignore Missing Imports in Mypy

Source: https://github.com/jbesomi/texthero/blob/master/tests/README.md

Use this command to silence mypy warnings about missing type hints in third-party packages.

```bash
mypy --ignore-missing-imports
```

--------------------------------

### texthero.preprocessing.get_default_pipeline

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api/texthero.preprocessing.get_default_pipeline.md

Retrieves the default text cleaning pipeline, which is a list of callable functions for preprocessing pandas Series.

```APIDOC
## texthero.preprocessing.get_default_pipeline

### Description
Return a list containing all the methods used in the default cleaning pipeline.

### Method
GET

### Endpoint
/texthero/preprocessing/get_default_pipeline

### Parameters
None

### Request Body
None

### Response
#### Success Response (200)
- **pipeline** (List[Callable[[pandas.core.series.Series], pandas.core.series.Series]]) - A list of functions that constitute the default text cleaning pipeline.

#### Response Example
```json
{
  "pipeline": [
    "texthero.preprocessing.fillna",
    "texthero.preprocessing.lowercase",
    "texthero.preprocessing.remove_digits",
    "texthero.preprocessing.remove_punctuation",
    "texthero.preprocessing.remove_diacritics",
    "texthero.preprocessing.remove_stopwords",
    "texthero.preprocessing.remove_whitespace"
  ]
}
```
```

--------------------------------

### Non-Negative Matrix Factorization (NMF)

Source: https://context7.com/jbesomi/texthero/llms.txt

Performs NMF for topic discovery and dimensionality reduction. It takes a Series of term frequency vectors and returns topic distributions for each document.

```APIDOC
## hero.nmf - Non-Negative Matrix Factorization

### Description
Performs NMF for topic discovery and dimensionality reduction.

### Method
`hero.nmf`

### Parameters
#### Path Parameters
None

#### Query Parameters
- **s** (pd.Series) - Required - Input Series of term frequency vectors.
- **n_components** (int) - Optional - Number of topics to extract. Defaults to 2.
- **random_state** (int) - Optional - Seed for random number generator for reproducibility.

### Request Example
```python
import texthero as hero
import pandas as pd

s = pd.Series(["Football Sports Soccer", "Music Violin Orchestra", "Football Music"])
s = s.pipe(hero.clean).pipe(hero.tokenize).pipe(hero.term_frequency)
nmf_result = hero.nmf(s, n_components=2, random_state=42)
print(nmf_result)
```

### Response
#### Success Response (200)
- **nmf_result** (pd.Series) - Series where each element is a list of topic distributions for a document.

#### Response Example
```
# 0    [0.908, 0.0]      # Sports topic
# 1    [0.0, 0.772]      # Music topic
# 2    [0.372, 0.316]    # Mixed
# dtype: object
```
```

--------------------------------

### preprocessing.replace_urls

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api/texthero.preprocessing.replace_urls.md

Replaces all URLs in a Pandas Series with a specified symbol.

```APIDOC
## POST /texthero/preprocessing/replace_urls

### Description
Replaces all URLs from the given Pandas Series with the given symbol.

### Method
POST

### Endpoint
/texthero/preprocessing/replace_urls

### Parameters
#### Request Body
- **s** (pandas.core.series.Series) - Required - The input Pandas Series containing text.
- **symbol** (str) - Required - The symbol to replace URLs with.

### Request Example
```json
{
  "s": "Go to: https://example.com",
  "symbol": "<URL>"
}
```

### Response
#### Success Response (200)
- **result** (pandas.core.series.Series) - The Series with URLs replaced.

#### Response Example
```json
{
  "result": "Go to: <URL>"
}
```
```

--------------------------------

### Word Cloud Generation

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api-visualization.md

Generates and displays a word cloud image from a Pandas Series of text data using the WordCloud library.

```APIDOC
## wordcloud

### Description
Plot wordcloud image using WordCloud from word_cloud package.

### Method
POST

### Endpoint
/api/visualization/wordcloud

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **s** (Series) - Required - The input Pandas Series containing text data.
- **font_path** (string) - Required - Path to the font file to be used for the word cloud.
- **width** (integer) - Required - The width of the generated word cloud image.
- **height** (integer) - Required - The height of the generated word cloud image.
- **background_color** (string) - Optional - The background color of the word cloud. Defaults to 'black'.
- **max_words** (integer) - Optional - The maximum number of words to display in the word cloud.
- **stopwords** (set of strings) - Optional - A set of words to exclude from the word cloud.

### Request Example
```json
{
  "s": "your_text_series",
  "font_path": "/path/to/font.ttf",
  "width": 800,
  "height": 600,
  "background_color": "white",
  "max_words": 200,
  "stopwords": ["the", "a", "is"]
}
```

### Response
#### Success Response (200)
- **image** (string) - A base64 encoded string of the generated word cloud image.

#### Response Example
```json
{
  "image": "iVBORw0KGgoAAAANSUhEUgAA..."
}
```
```

--------------------------------

### Preprocessing Functions

Source: https://github.com/jbesomi/texthero/blob/master/website/docs/api-preprocessing.md

This section covers various text preprocessing functions.

```APIDOC
## remove_round_brackets

### Description
Remove content within parentheses () and parentheses.

### Method
Not specified (assumed to be a function call)

### Endpoint
Not applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **s** (Pandas Series) - The input Series to process.

### Request Example
```python
texthero.preprocessing.remove_round_brackets(s)
```

### Response
#### Success Response (200)
- **Pandas Series** - Series with content within parentheses removed.

#### Response Example
```json
{
  "example": "response body"
}
```
```

```APIDOC
## remove_square_brackets

### Description
Remove content within square brackets [] and the square brackets.

### Method
Not specified (assumed to be a function call)

### Endpoint
Not applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **s** (Pandas Series) - The input Series to process.

### Request Example
```python
texthero.preprocessing.remove_square_brackets(s)
```

### Response
#### Success Response (200)
- **Pandas Series** - Series with content within square brackets removed.

#### Response Example
```json
{
  "example": "response body"
}
```
```

```APIDOC
## remove_stopwords

### Description
Remove all instances of specified words.

### Method
Not specified (assumed to be a function call)

### Endpoint
Not applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **input** (Pandas Series) - The input Series to process.
- **stopwords** (list of strings) - A list of words to remove.

### Request Example
```python
texthero.preprocessing.remove_stopwords(input, stopwords)
```

### Response
#### Success Response (200)
- **Pandas Series** - Series with specified stopwords removed.

#### Response Example
```json
{
  "example": "response body"
}
```
```

```APIDOC
## remove_urls

### Description
Remove all URLs from a given Pandas Series.

### Method
Not specified (assumed to be a function call)

### Endpoint
Not applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **s** (Pandas Series) - The input Series to process.

### Request Example
```python
texthero.preprocessing.remove_urls(s)
```

### Response
#### Success Response (200)
- **Pandas Series** - Series with URLs removed.

#### Response Example
```json
{
  "example": "response body"
}
```
```

```APIDOC
## replace_urls

### Description
Replace all URLs with the given symbol.

### Method
Not specified (assumed to be a function call)

### Endpoint
Not applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **s** (Pandas Series) - The input Series to process.
- **symbol** (string) - The symbol to replace URLs with.

### Request Example
```python
texthero.preprocessing.replace_urls(s, symbol)
```

### Response
#### Success Response (200)
- **Pandas Series** - Series with URLs replaced by the specified symbol.

#### Response Example
```json
{
  "example": "response body"
}
```
```

```APIDOC
## remove_whitespace

### Description
Remove any extra white spaces.

### Method
Not specified (assumed to be a function call)

### Endpoint
Not applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **input** (Pandas Series) - The input Series to process.

### Request Example
```python
texthero.preprocessing.remove_whitespace(input)
```

### Response
#### Success Response (200)
- **Pandas Series** - Series with extra whitespace removed.

#### Response Example
```json
{
  "example": "response body"
}
```
```

```APIDOC
## replace_punctuation

### Description
Replace all punctuation with a given symbol.

### Method
Not specified (assumed to be a function call)

### Endpoint
Not applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **input** (Pandas Series) - The input Series to process.
- **symbol** (string) - The symbol to replace punctuation with.

### Request Example
```python
texthero.preprocessing.replace_punctuation(input, symbol)
```

### Response
#### Success Response (200)
- **Pandas Series** - Series with punctuation replaced by the specified symbol.

#### Response Example
```json
{
  "example": "response body"
}
```
```

```APIDOC
## replace_stopwords

### Description
Replace all instances of specified words with a symbol.

### Method
Not specified (assumed to be a function call)

### Endpoint
Not applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **input** (Pandas Series) - The input Series to process.
- **symbol** (string) - The symbol to replace stopwords with.
- **stopwords** (list of strings) - A list of words to replace.

### Request Example
```python
texthero.preprocessing.replace_stopwords(input, symbol, stopwords)
```

### Response
#### Success Response (200)
- **Pandas Series** - Series with specified stopwords replaced by the symbol.

#### Response Example
```json
{
  "example": "response body"
}
```
```

```APIDOC
## tokenize

### Description
Tokenize each row of the given Series.

### Method
Not specified (assumed to be a function call)

### Endpoint
Not applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **s** (Pandas Series) - The input Series to process.

### Request Example
```python
texthero.preprocessing.tokenize(s)
```

### Response
#### Success Response (200)
- **Pandas Series** - Series with each row tokenized.

#### Response Example
```json
{
  "example": "response body"
}
```
```

--------------------------------

### Text Preprocessing, TF-IDF, PCA, and Scatter Plot

Source: https://github.com/jbesomi/texthero/blob/master/examples/README.md.ipynb

This snippet preprocesses text, calculates TF-IDF, applies PCA for dimensionality reduction, and generates a scatter plot visualization. It requires pandas and texthero libraries.

```python
import texthero as hero
import pandas as pd

df = pd.read_csv(
   "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)

df['pca'] = (
   df['text']
   .pipe(hero.clean)
   .pipe(hero.tfidf)
   .pipe(hero.pca)
)
fig = hero.scatterplot(df, 'pca', color='topic', title="PCA BBC Sport news", return_figure=True)
fig.write_image("../github/scatterplot_bbcsport.svg")
```

--------------------------------

### Default Text Cleaning Pipeline

Source: https://context7.com/jbesomi/texthero/llms.txt

Applies a comprehensive default cleaning pipeline including lowercasing, removing digits, HTML tags, punctuation, diacritics, stopwords, and extra whitespace. A custom pipeline can also be provided.

```python
import texthero as hero
import pandas as pd

# Basic cleaning
s = pd.Series(["<html>Hello WORLD! 123</html>", "  This is   a tëst...  "])
cleaned = hero.clean(s)
print(cleaned)
```

```python
import texthero as hero
import pandas as pd

# Custom pipeline
custom_pipeline = [hero.lowercase, hero.remove_punctuation, hero.remove_whitespace]
s = pd.Series(["Hello, World!   How are you?"])
cleaned = hero.clean(s, pipeline=custom_pipeline)
print(cleaned)
```