Texthero

https://texthero.org/docs
Admin
Texthero is a Python NLP package that provides efficient text preprocessing, vector representation,...
Tokens:20,962
Snippets:176
Trust Score:6.8
Update:5 months ago
Show doc for...
Tokens:
Raw
### Full Example: Load, Represent, and Visualize Text Data

Source: https://texthero.org/docs/getting-started

A comprehensive example demonstrating the complete workflow: loading a CSV dataset, performing text cleaning, TF-IDF representation, PCA, and visualizing the results with a scatter plot. This snippet includes necessary imports and data loading.

```python
import texthero as hero
import pandas as pd

df = pd.read_csv(
    "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)
df['pca'] = (
    df['text']
    .pipe(hero.clean)
    .pipe(hero.tfidf)
    .pipe(hero.pca)
)

hero.scatterplot(df, col='pca', color='topic', title="PCA BBC Sport news")
```

--------------------------------

### Install Texthero

Source: https://texthero.org/docs/getting-started

Install Texthero using pip. To upgrade to the latest version, use the -U flag.

```bash
pip install texthero
```

```bash
pip install texthero -U
```

--------------------------------

### Perform PCA on Pandas Series with Texthero

Source: https://texthero.org/docs/api/texthero.representation.pca.html

Demonstrates the initial setup for using the pca function. Ensure pandas and texthero are imported and a Pandas Series is created.

```python
import texthero as hero
import pandas as pd
s = pd.Series(["Sentence one", "Sentence two"])
```

--------------------------------

### named_entities

Source: https://texthero.org/docs/api/texthero.nlp.named_entities

Returns a Pandas Series where each row contains a list of tuples detailing the named entities found in the corresponding text. Each tuple includes the entity's name, label, starting character index, and ending character index. This function utilizes SpaCy for Named Entity Recognition.

```APIDOC
## named_entities

### Description
Return named-entities. Return a Pandas Series where each rows contains a list of tuples containing information regarding the given named entities. Tuple: (entity’name, entity’label, starting character, ending character)
Under the hood, named_entities make use of Spacy name entity recognition.

### Method Signature
`named_entities(_s_ , _package ='spacy'_)`

### Parameters
#### Path Parameters
- **_s_** (Pandas Series) - Required - The input Pandas Series containing text data.
- **_package_** (string) - Optional - The NLP package to use. Defaults to 'spacy'.

### List of Labels
- PERSON: People, including fictional.
- NORP: Nationalities or religious or political groups.
- FAC: Buildings, airports, highways, bridges, etc.
- ORG : Companies, agencies, institutions, etc.
- GPE: Countries, cities, states.
- LOC: Non-GPE locations, mountain ranges, bodies of water.
- PRODUCT: Objects, vehicles, foods, etc. (Not services.)
- EVENT: Named hurricanes, battles, wars, sports events, etc.
- WORK_OF_ART: Titles of books, songs, etc.
- LAW: Named documents made into laws.
- LANGUAGE: Any named language.
- DATE: Absolute or relative dates or periods.
- TIME: Times smaller than a day.
- PERCENT: Percentage, including "%“.
- MONEY: Monetary values, including unit.
- QUANTITY: Measurements, as of weight or distance.
- ORDINAL: “first”, “second”, etc.
- CARDINAL: Numerals that do not fall under another type.

### Request Example
```python
import texthero as hero
import pandas as pd
s = pd.Series("Yesterday I was in NY with Bill de Blasio")
hero.named_entities(s)[0]
```

### Response Example
```
[('Yesterday', 'DATE', 0, 9), ('NY', 'GPE', 19, 21), ('Bill de Blasio', 'PERSON', 27, 41)]
```
```

--------------------------------

### Extract Named Entities with Texthero

Source: https://texthero.org/docs/api/texthero.nlp.named_entities

Use this function to extract named entities from a Pandas Series. Ensure Spacy is installed and configured for the desired language model.

```python
import texthero as hero
import pandas as pd
s = pd.Series("Yesterday I was in NY with Bill de Blasio")
hero.named_entities(s)[0]
[('Yesterday', 'DATE', 0, 9), ('NY', 'GPE', 19, 21), ('Bill de Blasio', 'PERSON', 27, 41)]
```

--------------------------------

### Stemming a Pandas Series with Texthero

Source: https://texthero.org/docs/api/texthero.preprocessing.stem

Use this function to stem text in a Pandas Series. It defaults to the 'snowball' stemmer and 'english' language. Ensure you have pandas and texthero installed.

```python
import texthero as hero
import pandas as pd
s = pd.Series("I used to go 	
 running.")
hero.preprocessing.stem(s)
```

--------------------------------

### named_entities

Source: https://texthero.org/docs/api/texthero.nlp.named_entities.html

Extracts named entities from a Pandas Series. It returns a Series where each row contains a list of tuples, with each tuple representing a named entity and its details (name, label, start character, end character). This function utilizes Spacy for its Named Entity Recognition.

```APIDOC
## named_entities

### Description
Return named-entities. Return a Pandas Series where each rows contains a list of tuples containing information regarding the given named entities. Tuple: (entity’name, entity’label, starting character, ending character)
Under the hood, named_entities make use of Spacy name entity recognition.

### Method Signature
`named_entities(_s_ , _package ='spacy'_)`

### Parameters
* **_s_** (Pandas Series) - The input text data.
* **_package_** (string) - The NLP package to use, defaults to 'spacy'.

### List of Labels
* PERSON: People, including fictional.
* NORP: Nationalities or religious or political groups.
* FAC: Buildings, airports, highways, bridges, etc.
* ORG : Companies, agencies, institutions, etc.
* GPE: Countries, cities, states.
* LOC: Non-GPE locations, mountain ranges, bodies of water.
* PRODUCT: Objects, vehicles, foods, etc. (Not services.)
* EVENT: Named hurricanes, battles, wars, sports events, etc.
* WORK_OF_ART: Titles of books, songs, etc.
* LAW: Named documents made into laws.
* LANGUAGE: Any named language.
* DATE: Absolute or relative dates or periods.
* TIME: Times smaller than a day.
* PERCENT: Percentage, including "%."
* MONEY: Monetary values, including unit.
* QUANTITY: Measurements, as of weight or distance.
* ORDINAL: "first", "second", etc.
* CARDINAL: Numerals that do not fall under another type.

### Request Example
```python
import texthero as hero
import pandas as pd
s = pd.Series("Yesterday I was in NY with Bill de Blasio")
hero.named_entities(s)[0]
```

### Response Example
```
[('Yesterday', 'DATE', 0, 9), ('NY', 'GPE', 19, 21), ('Bill de Blasio', 'PERSON', 27, 41)]
```
```

--------------------------------

### Load BBC Sport Dataset

Source: https://texthero.org/docs/getting-started

Load the BBC Sport dataset into a Pandas DataFrame from a CSV file hosted on GitHub.

```python
df = pd.read_csv(
   "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)
```

--------------------------------

### Import Texthero and Pandas

Source: https://texthero.org

Import the necessary libraries for using Texthero and Pandas.

```python
import texthero as hero
import pandas as pd
```

--------------------------------

### Display DataFrame Head

Source: https://texthero.org/docs/getting-started

Display the first two rows of the DataFrame to inspect the loaded data.

```python
df.head(2)
```

--------------------------------

### get_default_pipeline

Source: https://texthero.org/docs/api/texthero.preprocessing.get_default_pipeline.html

Returns a list containing all the methods used in the default cleaning pipeline.

```APIDOC
## get_default_pipeline()

### Description
Return a list containing all the methods used in the default cleaning pipeline.

### Returns
- List[Callable[[pandas.core.series.Series], pandas.core.series.Series]] - A list with the following functions:
  1. `texthero.preprocessing.fillna()`
  2. `texthero.preprocessing.lowercase()`
  3. `texthero.preprocessing.remove_digits()`
  4. `texthero.preprocessing.remove_punctuation()`
  5. `texthero.preprocessing.remove_diacritics()`
  6. `texthero.preprocessing.remove_stopwords()`
  7. `texthero.preprocessing.remove_whitespace()`
```

--------------------------------

### kmeans

Source: https://texthero.org/docs/api-representation

Perform K-means clustering algorithm.

```APIDOC
## kmeans

### Description
Perform K-means clustering algorithm.

### Parameters
- **s** (Pandas Series) - Input data.
- **n_clusters** (int, optional) - The number of clusters to form as well as the number of centroids to generate.
- **init** (str, optional) - Method for initialization.
- **n_init** (int, optional) - Number of time the k-means algorithm will be run with different centroid seeds.
- **…** (any) - Additional parameters for K-means.
```

--------------------------------

### Load Text Dataset with Pandas

Source: https://texthero.org

Load a CSV dataset into a Pandas DataFrame. Ensure the dataset contains a 'text' column.

```python
df = pd.read_csv(
    "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)
df.head(2)
```

--------------------------------

### Visualize PCA Results

Source: https://texthero.org/docs/getting-started

Create a scatter plot to visualize the results of PCA, coloring points by topic. This helps in understanding the distribution and separation of topics in the reduced dimensional space.

```python
hero.scatterplot(df, col='pca', color='topic', title="PCA BBC Sport news")
```

--------------------------------

### Find Top Words per Topic

Source: https://texthero.org/docs/getting-started

Identify the most common words for each topic in the dataset. This is useful for interpreting the meaning of each identified topic.

```python
NUM_TOP_WORDS = 5
df.groupby('topic')['text'].apply(lambda x: hero.top_words(x)[:NUM_TOP_WORDS])
```

--------------------------------

### Reduce Dimension with PCA and Visualize

Source: https://texthero.org

Apply Principal Component Analysis (PCA) to reduce the dimensionality of the TF-IDF vectors and then create a scatter plot to visualize the vector space, colored by topic.

```python
df['pca'] = hero.pca(df['tfidf'])
hero.scatterplot(
    df, 
    col='pca', 
    color='topic', 
    title="PCA BBC Sport news"
)
```

--------------------------------

### clean

Source: https://texthero.org/docs/api/texthero.preprocessing.clean.html

Pre-processes a text-based Pandas Series by applying a series of cleaning steps. A custom pipeline can be provided, otherwise, a default pipeline is used.

```APIDOC
## clean

### Description
Pre-processes a text-based Pandas Series by applying a series of cleaning steps. A custom pipeline can be provided, otherwise, a default pipeline is used.

### Signature
`clean(_s : pandas.core.series.Series, _pipeline =None_) -> pandas.core.series.Series`

### Parameters
#### Path Parameters
- **_s** (pandas.core.series.Series) - The input Pandas Series containing text data.
- **_pipeline** (None) - Optional. A custom pipeline of preprocessing steps. If None, the default pipeline is used.

### Default Pipeline
1. `texthero.preprocessing.fillna()`
2. `texthero.preprocessing.lowercase()`
3. `texthero.preprocessing.remove_digits()`
4. `texthero.preprocessing.remove_punctuation()`
5. `texthero.preprocessing.remove_diacritics()`
6. `texthero.preprocessing.remove_stopwords()`
7. `texthero.preprocessing.remove_whitespace()`

### Returns
- pandas.core.series.Series - The pre-processed Pandas Series.
```

--------------------------------

### Calculate Term Frequency with Feature Names

Source: https://texthero.org/docs/api/texthero.representation.term_frequency.html

To obtain the feature names along with the term frequency representation, set `return_feature_names=True`. This is useful for understanding which words correspond to each feature in the output.

```python
import texthero as hero
import pandas as pd
s = pd.Series(["Sentence one", "Sentence two"])
hero.term_frequency(s, return_feature_names=True)
```

--------------------------------

### Custom Text Cleaning Pipeline

Source: https://texthero.org/docs/getting-started

Apply a custom preprocessing pipeline to clean text data by specifying a list of preprocessing functions.

```python
from texthero import preprocessing

custom_pipeline = [preprocessing.fillna,
                   preprocessing.lowercase,
                   preprocessing.remove_whitespace]
df['clean_text'] = hero.clean(df['text'], custom_pipeline)
```

--------------------------------

### wordcloud

Source: https://texthero.org/docs/api/texthero.visualization.wordcloud

Plots a word cloud image using the WordCloud library. It takes a pandas Series as input and offers various parameters to customize the appearance of the word cloud, such as font, size, colors, and masking.

```APIDOC
## wordcloud

### Description
Plot wordcloud image using WordCloud from word_cloud package. Most of the arguments are very similar if not equal to the mother function. In contrast, all words are taken into account when computing the wordcloud, inclusive stopwords. They can be easily removed with preprocessing.remove_stopwords. Word are compute using generate_from_frequencies.

### Method
python

### Parameters
#### Path Parameters
- **s** (pandas.core.series.Series) - Required - The input pandas Series containing the text data.
- **font_path** (str) - Optional - Font path to the font that will be used (OTF or TTF). Defaults to DroidSansMono path on a Linux machine. If you are on another OS or don’t have this font, you need to adjust this path.
- **width** (int) - Optional - Width of the canvas. Defaults to 400.
- **height** (int) - Optional - Height of the canvas. Defaults to 200.
- **max_words** (number) - Optional - The maximum number of words. Defaults to 200.
- **mask** (nd-array or None) - Optional - When set, gives a binary mask on where to draw words. When set, width and height will be ignored and the shape of mask will be used instead. All white (#FF or #FFFFFF) entries will be considerd “masked out” while other entries will be free to draw on. Defaults to None.
- **contour_width** (float) - Optional - If mask is not None and contour_width > 0, draw the mask contour. Defaults to 0.
- **contour_color** (color value) - Optional - Mask contour color. Defaults to 'PAPAYAWHIP'.
- **background_color** (color value) - Optional - Background color for the word cloud image. Defaults to 'PAPAYAWHIP'.
- **relative_scaling** (float) - Optional - Importance of relative word frequencies for font-size. With relative_scaling=0, only word-ranks are considered. With relative_scaling=1, a word that is twice as frequent will have twice the size. If you want to consider the word frequencies and not only their rank, relative_scaling around .5 often looks good. If ‘auto’ it will be set to 0.5 unless repeat is true, in which case it will be set to 0. Defaults to 'auto'.
- **colormap** (string or matplotlib colormap) - Optional - Matplotlib colormap to randomly draw colors from for each word. Defaults to 'viridis'.
- **return_figure** (bool) - Optional - Whether to return the matplotlib figure object. Defaults to False.

### Request Example
```python
from texthero import preprocessing
from texthero import visualization

df = pd.read_csv("your_dataset.csv")
s = df.text

# Preprocess the text to remove stopwords
s = preprocessing.remove_stopwords(s)

# Plot wordcloud
try:
    visualization.wordcloud(s, background_color="white", colormap="viridis")
except ImportError:
    print(
```

--------------------------------

### Clean, TF-IDF, and PCA in One Step

Source: https://texthero.org/docs/getting-started

Perform text cleaning, TF-IDF representation, and PCA dimensionality reduction in a single chained operation. This is useful for quickly preparing text data for analysis.

```python
df['pca'] = (
            df['text']
            .pipe(hero.clean)
            .pipe(hero.tfidf)
            .pipe(hero.pca)
   )
```

--------------------------------

### dbscan Function

Source: https://texthero.org/docs/api/texthero.representation.dbscan.html

Performs DBSCAN clustering on the input data. Allows customization of clustering parameters like epsilon, minimum samples, and distance metric.

```APIDOC
## dbscan

Performs DBSCAN clustering.

### Parameters
- **s** (type) - Description
- **eps** (float, optional) - Defaults to 0.5. The maximum distance between two samples for one to be considered as in the neighborhood of the other.
- **min_samples** (int, optional) - Defaults to 5. The number of samples in a neighborhood for a point to be considered as a core point.
- **metric** (str or callable, optional) - Defaults to 'euclidean'. The distance metric to use when calculating the distance between instances in a feature array.
- **metric_params** (dict, optional) - Defaults to None. Additional keyword arguments for the metric function.
- **algorithm** (str, optional) - Defaults to 'auto'. The algorithm to use to compute the nearest neighbors.
- **leaf_size** (int, optional) - Defaults to 30. Leaf size passed to BallTree or KDTree.
- **p** (float, optional) - Defaults to None. The power parameter when using the Minkowski metric.
- **n_jobs** (int, optional) - Defaults to None. The number of parallel jobs to run for computation.
```

--------------------------------

### Clean Text Data with Pipe

Source: https://texthero.org/docs/getting-started

Achieve the same text cleaning results as hero.clean by using the Pandas pipe function on the 'text' Series.

```python
df['clean_text'] = df['text'].pipe(hero.clean)
```

--------------------------------

### dbscan

Source: https://texthero.org/docs/api-representation

Perform DBSCAN clustering on the input data.

```APIDOC
## dbscan

### Description
Perform DBSCAN clustering.

### Parameters
- **s** (Pandas Series) - Input data.
- **eps** (float, optional) - The maximum distance between two samples for one to be considered as in the neighborhood of the other.
- **min_samples** (int, optional) - The number of samples in a neighborhood for a point to be considered as a core point.
- **metric** (str or callable, optional) - The metric to use when calculating distance between instances in a feature array.
- **…** (any) - Additional parameters for DBSCAN.
```

--------------------------------

### tokenize

Source: https://texthero.org/docs/api/texthero.preprocessing.tokenize.html

Tokenize each row of the given Pandas Series and return a Pandas Series where each row contains a list of tokens. The algorithm adds a space between punctuation symbols, with exceptions for symbols between alphanumeric characters, and then splits the string.

```APIDOC
## tokenize

### Description
Tokenize each row of the given Pandas Series and return a Pandas Series where each row contains a list of tokens.
Algorithm: add a space between any punctuation symbol at exception if the symbol is between two alphanumeric character and split.

### Signature
tokenize(_s : pandas.core.series.Series_) → pandas.core.series.Series

### Parameters
#### Path Parameters
- **_s** (pandas.core.series.Series) - The input Pandas Series to tokenize.

### Request Example
```python
import texthero as hero
import pandas as pd
s = pd.Series(["Today you're looking great!"])
hero.tokenize(s)
```

### Response
#### Success Response (pandas.core.series.Series)
- A Pandas Series where each element is a list of tokens from the corresponding input string.

### Response Example
```
0    [Today, you're, looking, great, !]
dtype: object
```
```

--------------------------------

### Represent Text with TF-IDF and Return Feature Names

Source: https://texthero.org/docs/api/texthero.representation.tfidf.html

To also retrieve the feature names alongside the TF-IDF representation, set `return_feature_names=True`. This returns a tuple containing the TF-IDF series and a list of feature names.

```python
>>> import texthero as hero
>>> import pandas as pd
>>> s = pd.Series(["Sentence one", "Sentence two"])
>>> hero.tfidf(s, return_feature_names=True)
(0    [0.5797386715376657, 0.8148024746671689, 0.0]
1    [0.5797386715376657, 0.0, 0.8148024746671689]
dtype: object, ['Sentence', 'one', 'two'])
```

--------------------------------

### texthero.preprocess.clean

Source: https://texthero.org/docs/api-preprocessing

Pre-processes a text-based Pandas Series using a specified cleaning pipeline.

```APIDOC
## texthero.preprocess.clean

### Description
Pre-process a text-based Pandas Series.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Method
None (This is a function call)

### Endpoint
None (This is a function call)

### Request Example
```python
import texthero as hero
import pandas as pd

s = pd.Series(['Sample text with <angle brackets> and (parentheses).', 'Another example with {curly braces}.'])
cleaned_s = hero.clean(s)
```

### Response
#### Success Response
- **s** (Pandas Series) - The pre-processed Pandas Series.
```

--------------------------------

### texthero.preprocess.replace_urls

Source: https://texthero.org/docs/api-preprocessing

Replaces all URLs in a given Pandas Series with a specified symbol.

```APIDOC
## texthero.preprocess.replace_urls

### Description
Replace all urls with the given symbol.

### Parameters
#### Query Parameters
- **symbol** (string) - The symbol to replace URLs with.

### Method
None (This is a function call)

### Endpoint
None (This is a function call)

### Request Example
```python
import texthero as hero
import pandas as pd

s = pd.Series(['Check this link: http://example.com', 'Visit us at https://another.org/page'])
replaced_s = hero.replace_urls(s, '[URL]')
```

### Response
#### Success Response
- **s** (Pandas Series) - The Pandas Series with URLs replaced by the specified symbol.
```

--------------------------------

### Custom Text Cleaning Pipeline with Pipe

Source: https://texthero.org/docs/getting-started

Apply a custom preprocessing pipeline using the Pandas pipe function with Texthero's clean method.

```python
df['clean_text'] = df['clean_text'].pipe(hero.clean, custom_pipeline)
```

--------------------------------

### meanshift

Source: https://texthero.org/docs/api-representation

Perform mean shift clustering.

```APIDOC
## meanshift

### Description
Perform mean shift clustering.

### Parameters
- **s** (Pandas Series) - Input data.
- **bandwidth** (float, optional) - Bandwidth of the kernels.
- **seeds** (ndarray, optional) - Seed points.
- **…** (any) - Additional parameters for Mean Shift.
```

--------------------------------

### Represent Text Series with TF-IDF and Feature Names

Source: https://texthero.org/docs/api/texthero.representation.tfidf

To also retrieve the feature names alongside the TF-IDF representation, set `return_feature_names=True`. Ensure pandas and texthero are imported.

```python
import texthero as hero
import pandas as pd
s = pd.Series(["Sentence one", "Sentence two"])
hero.tfidf(s, return_feature_names=True)
```

--------------------------------

### wordcloud

Source: https://texthero.org/docs/api/texthero.visualization.wordcloud.html

Plots a word cloud image from a pandas Series. This function utilizes the word_cloud package and offers parameters similar to its underlying functions, with the addition of preprocessing stopwords.

```APIDOC
## wordcloud

### Description
Plot wordcloud image using WordCloud from word_cloud package. Most of the arguments are very similar if not equal to the mother function. In contrast, all words are taken into account when computing the wordcloud, inclusive stopwords. They can be easily removed with preprocessing.remove_stopwords. Word are compute using generate_from_frequencies.

### Method Signature
wordcloud(_s : pandas.core.series.Series, _font_path : str = None, _width : int = 400, _height : int = 200, _max_words =200, _mask =None, _contour_width =0, _contour_color ='PAPAYAWHIP', _background_color ='PAPAYAWHIP', _relative_scaling ='auto', _colormap =None, _return_figure =False_)

### Parameters
#### Positional Parameters
- **s** (pandas.core.series.Series) - The input pandas Series containing the text data.

#### Keyword Parameters
- **font_path** (str, optional) - Path to the font file (OTF or TTF). Defaults to a system-specific path.
- **width** (int, optional) - Width of the word cloud canvas. Defaults to 400.
- **height** (int, optional) - Height of the word cloud canvas. Defaults to 200.
- **max_words** (int, optional) - The maximum number of words to display. Defaults to 200.
- **mask** (numpy.ndarray or None, optional) - A binary mask to shape the word cloud. If provided, width and height are ignored. Defaults to None.
- **contour_width** (float, optional) - Width of the contour line if mask is used. Defaults to 0.
- **contour_color** (color value, optional) - Color of the mask contour. Defaults to 'PAPAYAWHIP'.
- **background_color** (color value, optional) - Background color of the word cloud. Defaults to 'PAPAYAWHIP'.
- **relative_scaling** (float or str, optional) - Importance of relative word frequencies for font size. Defaults to 'auto'.
- **colormap** (str or matplotlib colormap, optional) - Matplotlib colormap for word colors. Defaults to 'viridis'.
- **return_figure** (bool, optional) - Whether to return the matplotlib figure object. Defaults to False.

### Notes
- Words are computed using `generate_from_frequencies`.
- Stopwords can be removed using `preprocessing.remove_stopwords`.
```

--------------------------------

### replace_urls

Source: https://texthero.org/docs/api/texthero.preprocessing.replace_urls.html

Replaces all URLs in a Pandas Series with a specified symbol.

```APIDOC
## replace_urls

### Description
Replace all urls with the given symbol.

### Method Signature
`replace_urls(_s : pandas.core.series.Series_, _symbol : str_) -> pandas.core.series.Series`

### Parameters
#### Path Parameters
- **_s** (pandas.core.series.Series) - Required - The input Pandas Series.
- **_symbol** (str) - Required - The symbol to replace URLs with.

### Request Example
```python
import texthero as hero
import pandas as pd
s = pd.Series("Go to: https://example.com")
hero.replace_urls(s, "<URL>")
```

### Response
#### Success Response (pandas.core.series.Series)
- Returns a Pandas Series with URLs replaced by the specified symbol.

### Response Example
```
0    Go to: <URL>
dtype: object
```
```

--------------------------------

### scatterplot

Source: https://texthero.org/docs/api/texthero.visualization.scatterplot.html

Show scatterplot using python plotly scatter.

```APIDOC
## scatterplot

### Description
Show scatterplot using python plotly scatter.

### Parameters
- **df** (pandas.core.frame.DataFrame) - Required - The DataFrame to use for plotting.
- **col** (str) - Required - The name of the column of the DataFrame used for x and y axis.
- **color** (str) - Optional - The column to use for coloring the points.
- **hover_data** (list) - Optional - Data to display on hover.
- **title** (str) - Optional - The title of the plot.
- **return_figure** (bool) - Optional - Whether to return the plotly figure object.
```

--------------------------------

### texthero.preprocess.replace_stopwords

Source: https://texthero.org/docs/api-preprocessing

Replaces all instances of specified words with a given symbol. Supports custom stopwords.

```APIDOC
## texthero.preprocess.replace_stopwords

### Description
Replace all instances of words with symbol.

### Parameters
#### Query Parameters
- **symbol** (string) - The symbol to replace stopwords with.
- **stopwords** (list) - A list of words to replace.
- **...** - Additional parameters may be supported.

### Method
None (This is a function call)

### Endpoint
None (This is a function call)

### Request Example
```python
import texthero as hero

text = 'This is a sample text with some common words.'
custom_stopwords = ['is', 'a', 'with', 'some']
replaced_text = hero.replace_stopwords(text, '[STOP]', custom_stopwords)
print(replaced_text)
```

### Response
#### Success Response
- **input** (string) - The input string with specified stopwords replaced by the symbol.
```

--------------------------------

### named_entities

Source: https://texthero.org/docs/api-nlp

Returns named entities found in the input text.

```APIDOC
## named_entities(s[, package])

### Description
Return named-entities.

### Parameters
#### Path Parameters
- **s** (string) - Required - The input text.
- **package** (string) - Optional - The NLP package to use.
```

--------------------------------

### Show Top Words

Source: https://texthero.org

Display the most frequent words in the text corpus. `NUM_TOP_WORDS` can be adjusted to show a different number of top words.

```python
NUM_TOP_WORDS = 5
hero.top_words(df['text'])[:NUM_TOP_WORDS]
```

--------------------------------

### pca

Source: https://texthero.org/docs/api-representation

Perform principal component analysis on the given Pandas Series.

```APIDOC
## pca

### Description
Perform principal component analysis on the given Pandas Series.

### Parameters
- **s** (Pandas Series) - Input data.
- **n_components** (int, optional) - Number of components to keep.
```

--------------------------------

### texthero.preprocess.replace_punctuation

Source: https://texthero.org/docs/api-preprocessing

Replaces all punctuation in the input with a specified symbol.

```APIDOC
## texthero.preprocess.replace_punctuation

### Description
Replace all punctuation with a given symbol.

### Parameters
#### Query Parameters
- **symbol** (string) - The symbol to replace punctuation with.

### Method
None (This is a function call)

### Endpoint
None (This is a function call)

### Request Example
```python
import texthero as hero

text = 'Hello, world! How are you?'
replaced_text = hero.replace_punctuation(text, '_')
print(replaced_text)
```

### Response
#### Success Response
- **input** (string) - The input string with punctuation replaced by the specified symbol.
```

--------------------------------

### kmeans Function

Source: https://texthero.org/docs/api/texthero.representation.kmeans.html

Performs K-means clustering algorithm on a pandas Series.

```APIDOC
## kmeans

### Description
Perform K-means clustering algorithm.

### Signature
kmeans(_s : pandas.core.series.Series, _n_clusters =5, _init ='k-means++'_, _n_init =10, _max_iter =300, _tol =0.0001, _precompute_distances ='auto'_, _verbose =0, _random_state =None, _copy_x =True, _n_jobs =- 1, _algorithm ='auto'_)

### Parameters
- **_s** (pandas.core.series.Series) - The input pandas Series to cluster.
- **_n_clusters** (int) - The number of clusters to form as well as the number of centroids to generate. Defaults to 5.
- **_init** (str or callable) - Method for initialization, defaults to 'k-means++'.
- **_n_init** (int or 'auto') - Number of times the k-means algorithm will be run with different centroid seeds. Defaults to 10.
- **_max_iter** (int) - Maximum number of iterations of the k-means algorithm for a single run. Defaults to 300.
- **_tol** (float) - Relative tolerance with regards to Frobenius norm of the difference of the cluster centers in consecutive iterations. Defaults to 0.0001.
- **_precompute_distances** (bool or {'auto'}) - Precompute distances (faster but memory intensive). Defaults to 'auto'.
- **_verbose** (int) - Verbosity mode. Defaults to 0.
- **_random_state** (int, RandomState instance or None) - Determines random number generation for centroid initialization. Defaults to None.
- **_copy_x** (bool) - When pre-computing distances it is more efficient to not make a copy of the original data. Defaults to True.
- **_n_jobs** (int or None) - Compute in parallel using n jobs. Defaults to -1 (all processors).
- **_algorithm** (str or callable) - K-means algorithm to use. Defaults to 'auto'.
```

--------------------------------

### TFIDF Representation

Source: https://texthero.org/docs/getting-started

Convert the cleaned text data into a TF-IDF (Term Frequency-Inverse Document Frequency) vector representation.

```python
df['tfidf_clean_text'] = hero.tfidf(df['clean_text'])
```

--------------------------------

### stem

Source: https://texthero.org/docs/api/texthero.preprocessing.stem

Stem series using either porter or snowball NLTK stemmers. The act of stemming means removing the end of a words with an heuristic process. It’s useful in context where the meaning of the word is important rather than his derivation. Stemming is very efficient and adapt in case the given dataset is large. texthero.preprocessing.stem make use of two NLTK stemming algorithms known as `nltk.stem.SnowballStemmer` and `nltk.stem.PorterStemmer`. SnowballStemmer should be used when the Pandas Series contains non-English text has it has multilanguage support.

```APIDOC
## stem

### Description
Stem series using either porter or snowball NLTK stemmers. This process removes the end of words heuristically, preserving the core meaning.

### Method Signature
`stem`(_input : pandas.core.series.Series_, _stem ='snowball'_, _language ='english'_) → pandas.core.series.Series

### Parameters
#### input
- **input** (pandas.core.series.Series) - The input Pandas Series to be stemmed.

#### stem
- **stem** (str) - Optional. The stemming algorithm to use. Can be either ‘snowball’ or ‘porter’. Defaults to ‘snowball’.

#### language
- **language** (str) - Optional. The language for the stemming algorithm. Supported languages include: danish, dutch, english, finnish, french, german, hungarian, italian, norwegian, portuguese, romanian, russian, spanish and swedish. Defaults to ‘english’.

### Notes
By default NLTK stemming algorithms lowercase all text.

### Example
```python
>>> import texthero as hero
>>> import pandas as pd
>>> s = pd.Series("I used to go \t\n running.")
>>> hero.preprocessing.stem(s)
0    i use to go running.
dtype: object
```
```

--------------------------------

### Represent Text Series with TF-IDF

Source: https://texthero.org/docs/api/texthero.representation.tfidf

Use this to represent a text-based Pandas Series using TF-IDF. Ensure pandas and texthero are imported.

```python
import texthero as hero
import pandas as pd
s = pd.Series(["Sentence one", "Sentence two"])
hero.tfidf(s)
```

--------------------------------

### meanshift Function

Source: https://texthero.org/docs/api/texthero.representation.meanshift.html

Performs mean shift clustering on the input data.

```APIDOC
## meanshift

### Description
Perform mean shift clustering.

### Parameters
- **s** (type) - Description
- **bandwidth** (type, optional) - Description
- **seeds** (type, optional) - Description
- **bin_seeding** (type, optional) - Description
- **min_bin_freq** (type, optional) - Description
- **cluster_all** (type, optional) - Description
- **n_jobs** (type, optional) - Description
- **max_iter** (type, optional) - Description
```

--------------------------------

### Check for Content in Pandas Series

Source: https://texthero.org/docs/api/texthero.preprocessing.has_content.html

Use `has_content` to identify rows with meaningful text. It returns `False` for NaN, whitespace-only strings, and empty strings.

```python
>>> s = pd.Series(["content", np.nan, "\t\n", " "])
>>> has_content(s)
0     True
1    False
2    False
3    False
dtype: bool
```

--------------------------------

### Represent Text with TF-IDF

Source: https://texthero.org

Generate TF-IDF (Term Frequency-Inverse Document Frequency) features for the text data. `max_features` limits the vocabulary size.

```python
df['tfidf'] = (
    hero.tfidf(df['text'], max_features=100)
)
```

--------------------------------

### texthero.preprocess.tokenize

Source: https://texthero.org/docs/api-preprocessing

Tokenizes each row of the given Pandas Series.

```APIDOC
## texthero.preprocess.tokenize

### Description
Tokenize each row of the given Series.

### Parameters
None

### Method
None (This is a function call)

### Endpoint
None (This is a function call)

### Request Example
```python
import texthero as hero
import pandas as pd

s = pd.Series(['This is the first sentence.', 'And this is the second.'])
tokenized_s = hero.tokenize(s)
```

### Response
#### Success Response
- **s** (Pandas Series) - The Pandas Series with each row tokenized (split into words/tokens).
```

--------------------------------

### texthero.visualization.top_words

Source: https://texthero.org/docs/api/texthero.visualization.top_words.html

Returns a pandas Series with the top words as the index and their counts as the value. Tokenization is performed by splitting by space and removing punctuation that is not between characters.

```APIDOC
## texthero.visualization.top_words

### Description
Returns a pandas Series with the top words as the index and their counts as the value. Tokenization is performed by splitting by space and removing punctuation that is not between characters.

### Signature
`top_words(_s : pandas.core.series.Series_, _normalize =False_) → pandas.core.series.Series`

### Parameters
#### Query Parameters
- **normalize** (bool) - Optional - When set to true, return normalized values.

### Returns
- pandas.core.series.Series - A Series containing top words and their counts.
```

--------------------------------

### Dimensionality Reduction with PCA

Source: https://texthero.org/docs/getting-started

Apply Principal Component Analysis (PCA) to reduce the dimensionality of the TF-IDF vectors to two dimensions for visualization.

```python
df['pca_tfidf_clean_text'] = hero.pca(df['tfidf_clean_text'])
```

--------------------------------

### texthero.preprocess.remove_punctuation

Source: https://texthero.org/docs/api-preprocessing

Replaces all punctuation in the input with a single space.

```APIDOC
## texthero.preprocess.remove_punctuation

### Description
Replace all punctuation with a single space (” “).

### Parameters
None

### Method
None (This is a function call)

### Endpoint
None (This is a function call)

### Request Example
```python
import texthero as hero

text = 'Hello, world! How are you?'
cleaned_text = hero.remove_punctuation(text)
print(cleaned_text)
```

### Response
#### Success Response
- **input** (string) - The input string with punctuation replaced by spaces.
```

--------------------------------

### tsne

Source: https://texthero.org/docs/api-representation

Perform TSNE on the given pandas series.

```APIDOC
## tsne

### Description
Perform TSNE on the given pandas series.

### Parameters
- **s** (Pandas Series) - Input data.
- **n_components** (int, optional) - Dimension of the embedded space after t-SNE is applied.
- **perplexity** (float, optional) - Related to the effective number of neighbors of each point.
- **…** (any) - Additional parameters for TSNE.
```

--------------------------------

### noun_chunks

Source: https://texthero.org/docs/api/texthero.nlp.noun_chunks.html

Extracts noun chunks from the input text. Noun chunks are groups of consecutive words that form a noun phrase.

```APIDOC
## noun_chunks(_s_)

### Description
Return noun_chunks, group of consecutive words that belong together.

### Parameters
#### Path Parameters
- **s** (string) - Required - The input text to process.
```

--------------------------------

### Tokenize Pandas Series with texthero

Source: https://texthero.org/docs/api/texthero.preprocessing.tokenize.html

Use `hero.tokenize` to split each string in a Pandas Series into a list of tokens. This function handles punctuation by adding spaces around symbols, except when they are between alphanumeric characters, before performing the split.

```python
import texthero as hero
import pandas as pd
s = pd.Series(["Today you're looking great!"])
hero.tokenize(s)
```

--------------------------------

### replace_stopwords

Source: https://texthero.org/docs/api/texthero.preprocessing.replace_stopwords.html

Replaces all instances of words in a Pandas Series with a specified symbol. By default, it uses NLTK's English stopwords.

```APIDOC
## replace_stopwords

### Description
Replace all instances of words with symbol.
By default uses NLTK’s english stopwords of 179 words.

### Parameters

- **input** (pandas.core.series.Series) - The input Pandas Series.
- **symbol** (str) - Character(s) to replace words with.
- **stopwords** (Set[str], Optional) - Set of stopwords string to remove. If not passed, by default it used NLTK English stopwords.

### Returns

pandas.core.series.Series - The Series with stopwords replaced.

### Example

```python
>>> import pandas as pd
>>> from texthero.preprocessing import replace_stopwords
>>> s = pd.Series("the book of the jungle")
>>> replace_stopwords(s, "X")
0    X book X X jungle
dtype: object
```
```

--------------------------------

### tfidf

Source: https://texthero.org/docs/api-representation

Represent a text-based Pandas Series using TF-IDF.

```APIDOC
## tfidf

### Description
Represent a text-based Pandas Series using TF-IDF.

### Parameters
- **s** (Pandas Series) - Input text data.
- **max_features** (int, optional) - If not None, below which all terms will be removed.
- **min_df** (float or int, optional) - When building the vocabulary ignore terms that have a document frequency strictly lower than the given threshold.
- **…** (any) - Additional parameters for TF-IDF.
```

--------------------------------

### tfidf

Source: https://texthero.org/docs/api/texthero.representation.tfidf.html

Represents a text-based Pandas Series using TF-IDF. It can optionally return feature names.

```APIDOC
## tfidf

### Description
Represent a text-based Pandas Series using TF-IDF.

### Parameters
- **s** (pandas.core.series.Series) - The input Pandas Series containing text data.
- **max_features** (int, optional) - Maximum number of features to keep.
- **min_df** (int, optional) - When building the vocabulary, ignore terms that have a document frequency strictly lower than the given threshold. Defaults to 1.
- **return_feature_names** (Boolean, optional) - If True, return a tuple of (tfidf_series, features_names). Defaults to False.

### Request Example
```python
import texthero as hero
import pandas as pd

s = pd.Series(["Sentence one", "Sentence two"])
result = hero.tfidf(s)
print(result)
```

### Response Example
```json
{
  "0": "[0.5797386715376657, 0.8148024746671689, 0.0]",
  "1": "[0.5797386715376657, 0.0, 0.8148024746671689]"
}
```

### Request Example with Feature Names
```python
import texthero as hero
import pandas as pd

s = pd.Series(["Sentence one", "Sentence two"])
tfidf_series, feature_names = hero.tfidf(s, return_feature_names=True)
print(tfidf_series)
print(feature_names)
```

### Response Example with Feature Names
```json
{
  "tfidf_series": {
    "0": "[0.5797386715376657, 0.8148024746671689, 0.0]",
    "1": "[0.5797386715376657, 0.0, 0.8148024746671689]"
  },
  "feature_names": ["Sentence", "one", "two"]
}
```
```

--------------------------------

### Remove Default Stopwords with Texthero

Source: https://texthero.org/docs/api/texthero.preprocessing.remove_stopwords.html

Use the default NLTK English stopwords to remove common words from a Pandas Series. Ensure Texthero and Pandas are imported.

```python
import texthero as hero
import pandas as pd
s = pd.Series("Texthero is not only for the heroes")
hero.remove_stopwords(s)
```

--------------------------------

### texthero.preprocess.remove_stopwords

Source: https://texthero.org/docs/api-preprocessing

Removes all instances of specified words from the input. Supports custom stopwords.

```APIDOC
## texthero.preprocess.remove_stopwords

### Description
Remove all instances of words.

### Parameters
#### Query Parameters
- **stopwords** (list) - A list of words to remove.
- **...** - Additional parameters may be supported.

### Method
None (This is a function call)

### Endpoint
None (This is a function call)

### Request Example
```python
import texthero as hero

text = 'This is a sample text with some common words.'
custom_stopwords = ['is', 'a', 'with', 'some']
cleaned_text = hero.remove_stopwords(text, custom_stopwords)
print(cleaned_text)
```

### Response
#### Success Response
- **input** (string) - The input string with specified stopwords removed.
```

--------------------------------

### Calculate Term Frequency

Source: https://texthero.org/docs/api/texthero.representation.term_frequency.html

Use this snippet to calculate the term frequency representation of a Pandas Series. Ensure you have imported texthero and pandas.

```python
import texthero as hero
import pandas as pd
s = pd.Series(["Sentence one", "Sentence two"])
hero.term_frequency(s)
```

--------------------------------

### texthero.preprocess.remove_urls

Source: https://texthero.org/docs/api-preprocessing

Removes all URLs from a given Pandas Series.

```APIDOC
## texthero.preprocess.remove_urls

### Description
Remove all urls from a given Pandas Series.

### Parameters
None

### Method
None (This is a function call)

### Endpoint
None (This is a function call)

### Request Example
```python
import texthero as hero
import pandas as pd

s = pd.Series(['Check this link: http://example.com', 'Visit us at https://another.org/page'])
cleaned_s = hero.remove_urls(s)
```

### Response
#### Success Response
- **s** (Pandas Series) - The Pandas Series with URLs removed.
```

--------------------------------

### pca

Source: https://texthero.org/docs/api/texthero.representation.pca.html

Performs Principal Component Analysis on a Pandas Series of text. This is useful for reducing the dimensionality of text representations.

```APIDOC
## pca

### Description
Perform principal component analysis on the given Pandas Series. In general, `pca` should be called after the text has already been represented.

### Parameters

- **s** (Pandas Series) - The input Pandas Series containing text data.
- **n_components** (Int) - Optional. The number of components to keep. Defaults to 2. If not set or None, all components are kept.

### Example
```python
>>> import texthero as hero
>>> import pandas as pd
>>> s = pd.Series(["Sentence one", "Sentence two"])
>>> # Assuming 's' has been represented previously
>>> # hero.pca(s)
```
```

--------------------------------

### texthero.preprocess.remove_whitespace

Source: https://texthero.org/docs/api-preprocessing

Removes any extra whitespace from the input.

```APIDOC
## texthero.preprocess.remove_whitespace

### Description
Remove any extra white spaces.

### Parameters
None

### Method
None (This is a function call)

### Endpoint
None (This is a function call)

### Request Example
```python
import texthero as hero

text = '  This   text  has   extra   spaces.  '
cleaned_text = hero.remove_whitespace(text)
print(cleaned_text)
```

### Response
#### Success Response
- **input** (string) - The input string with extra whitespace removed.
```
Tokens:
Raw