### Install Documentation Requirements

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/CONTRIBUTING.md

Install the Python packages required for generating the documentation. This is a prerequisite for building the docs locally.

```bash
pip install -r docs/requirements.txt
```

--------------------------------

### Install and Configure Pre-commit Hooks

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/CONTRIBUTING.md

Commands to install pre-commit and set up the git hooks for automated code checks before committing. Ensure you have Python and pip installed.

```bash
pip install pre-commit
```

```bash
pre-commit install
```

--------------------------------

### Example Output

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/README.md

This is the expected output when running the provided Python code snippet with the specified example text and models.

```text
Concreteness: 1.95
Frequency Index: -0.7684649336888104
Clause count: 10
Entity grid:
{'ESPECTÁCULO': ['S', '-', '-'], 'CIELO': ['X', '-', '-'], 'MIRADA': ['O', '-', '-'], 'UNIVERSO': ['O', '-', 'S'], 'ORIGEN': ['X', '-', '-'], 'FUNCIONAMIENTO': ['X', '-', '-'], 'CIVILIZACIONES': ['-', 'S', '-'], 'CULTURAS': ['-', 'X', '-'], 'COSMOLOGÍAS': ['-', 'O', '-'], 'EJEMPLO': ['-', '-', 'X'], 'TAL': ['-', '-', 'X'], 'CICLOS': ['-', '-', 'X'], 'QUE': ['-', '-', 'S'], 'SE': ['-', '-', 'O'], 'OTRAS': ['-', '-', 'S'], 'PRINCIPIO': ['-', '-', 'O'], 'OBRA': ['-', '-', 'X'], 'DIVINIDAD': ['-', '-', 'X']}
```

--------------------------------

### Build Documentation Locally

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/CONTRIBUTING.md

Run this command to build the documentation locally. Ensure you have installed the necessary requirements first.

```bash
sphinx-build -a -b html -W docs/ docs/_build/
```

--------------------------------

### Install TRUNAJOD and spaCy Model

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Install the TRUNAJOD library and download the necessary spaCy Spanish model. Load the spaCy model and the TRUNAJOD bundled models for subsequent text analysis.

```python
# Install TRUNAJOD
# pip install trunajod

# Download spaCy Spanish model
# python -m spacy download es_core_news_sm

import pickle
import tarfile
import spacy

# Load spaCy model
lp = spacy.load("es_core_news_sm", disable=["ner", "textcat"])

# Model loader for TRUNAJOD bundled models
class ModelLoader:
    def __init__(self, model_file):
        tar = tarfile.open(model_file, "r:gz")
        self.crea_frequency = {}
        self.infinitive_map = {}
        self.lemmatizer = {}
        self.spanish_lexicosemantic_norms = {}
        self.stopwords = {}
        self.wordnet_noun_synsets = {}
        self.wordnet_verb_synsets = {}

        for member in tar.getmembers():
            f = tar.extractfile(member)
            if "crea_frequency" in member.name:
                self.crea_frequency = pickle.loads(f.read())
            if "infinitive_map" in member.name:
                self.infinitive_map = pickle.loads(f.read())
            if "lemmatizer" in member.name:
                self.lemmatizer = pickle.loads(f.read())
            if "spanish_lexicosemantic_norms" in member.name:
                self.spanish_lexicosemantic_norms = pickle.loads(f.read())
            if "stopwords" in member.name:
                self.stopwords = pickle.loads(f.read())
            if "wordnet_noun_synsets" in member.name:
                self.wordnet_noun_synsets = pickle.loads(f.read())
            if "wordnet_verb_synsets" in member.name:
                self.wordnet_verb_synsets = pickle.loads(f.read())

# Load TRUNAJOD models
model = ModelLoader("trunajod_models_v0.1.tar.gz")

# Process example text
example_text = (
    "El espectáculo del cielo nocturno cautiva la mirada y suscita preguntas "
    "sobre el universo, su origen y su funcionamiento."
)
doc = nlp(example_text)
```

--------------------------------

### Initialize EntityGrid with Stanza Model

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Demonstrates how to initialize an EntityGrid object using a Stanza model for text processing. Ensure the 'stanza' library is installed and a model is downloaded.

```python
from TRUNAJOD.entity_grid import EntityGrid

# Using Stanza (requires stanza library)
# import stanza
# stanza_nlp = stanza.Pipeline('es')
# stanza_doc = stanza_nlp("El científico estudia. La ciencia avanza.")

# Create entity grid with Stanza model
# egrid = EntityGrid(stanza_doc, model_name="stanza")
```

--------------------------------

### Get Entity Grid and Transition Probabilities

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Retrieves the raw entity grid representation and calculates transition probabilities between entity roles (Subject, Object, Other, None).

```python
grid = egrid.get_egrid()
print("Entity Grid:")
for entity, roles in grid.items():
    print(f"  {entity}: {roles}")
# Entity roles: S=Subject, O=Object, X=Other, -=Not mentioned

# Get transition probabilities
print(f"\nSS transitions: {egrid.get_ss_transitions():.3f}")  # Subject to Subject
print(f"SO transitions: {egrid.get_so_transitions():.3f}")  # Subject to Object
print(f"S- transitions: {egrid.get_sn_transitions():.3f}")  # Subject to None
print(f"OS transitions: {egrid.get_os_transitions():.3f}")  # Object to Subject
print(f"OO transitions: {egrid.get_oo_transitions():.3f}")  # Object to Object
print(f"-- transitions: {egrid.get_nn_transitions():.3f}")  # None to None
```

--------------------------------

### Python Docstring Example with Citation

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/CONTRIBUTING.md

Example of a Python docstring that includes a citation using the :cite: role, referencing a paper for a specific measurement calculation.

```python
def d_estimate(
    doc: Doc, min_range: int = 35, max_range: int = 50, trials: int = 5
) -> float:
    r"""Compute D measurement for lexical diversity.

    The measurement is based in :cite:`richards2000measuring`. We pick ``n``
    numbers of tokens, varying ``N`` from ``min_range`` up to ``max_range``.
    For each ``n`` we do the following:

    # Continues...
    """
```

--------------------------------

### Load TRUNAJOD and spaCy Models

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/README.md

This snippet shows how to load pre-built TRUNAJOD models from a tar.gz file and a spaCy language model. Ensure the specified model file exists and the spaCy model is installed.

```python
from TRUNAJOD import surface_proxies
from TRUNAJOD.entity_grid import EntityGrid
from TRUNAJOD.lexico_semantic_norms import LexicoSemanticNorm
import pickle
import spacy
import tarfile


class ModelLoader(object):
    """Class to load model."""
    def __init__(self, model_file):
        tar = tarfile.open(model_file, "r:gz")
        self.crea_frequency = {}
        self.infinitive_map = {}
        self.lemmatizer = {}
        self.spanish_lexicosemantic_norms = {}
        self.stopwords = {}
        self.wordnet_noun_synsets = {}
        self.wordnet_verb_synsets = {}

        for member in tar.getmembers():
            f = tar.extractfile(member)
            if "crea_frequency" in member.name:
                self.crea_frequency = pickle.loads(f.read())
            if "infinitive_map" in member.name:
                self.infinitive_map = pickle.loads(f.read())
            if "lemmatizer" in member.name:
                self.lemmatizer = pickle.loads(f.read())
            if "spanish_lexicosemantic_norms" in member.name:
                self.spanish_lexicosemantic_norms = pickle.loads(f.read())
            if "stopwords" in member.name:
                self.stopwords = pickle.loads(f.read())
            if "wordnet_noun_synsets" in member.name:
                self.wordnet_noun_synsets = pickle.loads(f.read())
            if "wordnet_verb_synsets" in member.name:
                self.wordnet_verb_synsets = pickle.loads(f.read())


# Load TRUNAJOD models
model = ModelLoader("trunajod_models_v0.1.tar.gz")

# Load spaCy model
lp = spacy.load("es_core_news_sm", disable=["ner", "textcat"])

example_text = (
    "El espectáculo del cielo nocturno cautiva la mirada y suscita preguntas"
    "sobre el universo, su origen y su funcionamiento. No es sorprendente que "
    "todas las civilizaciones y culturas hayan formado sus propias "
    "cosmologías. Unas relatan, por ejemplo, que el universo ha"
    "sido siempre tal como es, con ciclos que inmutablemente se repiten; "
    "otras explican que este universo ha tenido un principio, "
    "que ha aparecido por obra creadora de una divinidad."
)

doc = nlp(example_text)

# Lexico-semantic norms
lexico_semantic_norms = LexicoSemanticNorm(
    doc,
    model.spanish_lexicosemantic_norms,
    model.lemmatizer
)

# Frequency index
freq_index = surface_proxies.frequency_index(doc, model.crea_frequency)

# Clause count (heurístically)
clause_count = surface_proxies.clause_count(doc, model.infinitive_map)

# Compute Entity Grid
egid = EntityGrid(doc)

print("Concreteness: {}".format(lexico_semantic_norms.get_concreteness()))
print("Frequency Index: {}".format(freq_index))
print("Clause count: {}".format(clause_count))
print("Entity grid:")
print(egrid.get_egrid())

```

--------------------------------

### Read Text File and Get Stopwords

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Illustrates how to read text from a file and load custom stopwords using TRUNAJOD utility functions. These functions are commented out and require file paths.

```python
from TRUNAJOD.utils import read_text, get_stopwords

# Read text file
# text = read_text("my_document.txt")

# Load custom stopwords
# stopwords = get_stopwords("stopwords.txt")
```

--------------------------------

### Get Concreteness, Imageability, Familiarity with EsPal

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Retrieves concreteness, imageability, and familiarity scores using the built-in EsPal dictionary. This function is suitable for Spanish texts.

```python
from TRUNAJOD.lexico_semantic_norms import get_conc_imag_familiarity
from TRUNAJOD.lexicosemantic_norms_espal import LSNorm

doc = nlp("El perro corre por el parque verde bajo el sol brillante.")

# Uses built-in EsPal dictionary
lsnorm = get_conc_imag_familiarity(doc)

print(f"Concreteness: {lsnorm[LSNorm.CONCRETENESS]:.3f}")
print(f"Imageability: {lsnorm[LSNorm.IMAGEABILITY]:.3f}")
print(f"Familiarity: {lsnorm[LSNorm.FAMILIARITY]:.3f}")
```

--------------------------------

### Extract Text Complexity Features

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

An example of extracting various text complexity features from a Spanish text using multiple TRUNAJOD modules. It includes surface proxies, lexical diversity, givenness, and discourse markers.

```python
import spacy
from TRUNAJOD import surface_proxies
from TRUNAJOD.entity_grid import EntityGrid, get_local_coherence
from TRUNAJOD.lexico_semantic_norms import LexicoSemanticNorm
from TRUNAJOD.emotions import Emotions
from TRUNAJOD.discourse_markers import get_overall_markers
from TRUNAJOD.givenness import pronoun_density, pronoun_noun_ratio
from TRUNAJOD.ttr import lexical_diversity_mtld, yule_k

nlp = spacy.load("es_core_news_sm", disable=["ner", "textcat"])

text = """
El universo es un lugar vasto y misterioso que ha cautivado la imaginación
humana durante milenios. Los científicos estudian las estrellas y las galaxias
para comprender mejor nuestro lugar en el cosmos. Sin embargo, muchos secretos
permanecen ocultos. Por ejemplo, la materia oscura constituye la mayor parte
del universo, pero aún no podemos observarla directamente. Además, la energía
oscura acelera la expansión del universo de maneras que desafían nuestra
comprensión actual de la física.
"""

doc = nlp(text)

# Extract features
features = {
    # Surface proxies
    "word_count": surface_proxies.word_count(doc),
    "sentence_count": surface_proxies.sentence_count(doc),
    "avg_sentence_length": surface_proxies.average_sentence_length(doc),
    "avg_word_length": surface_proxies.average_word_length(doc),
    "lexical_density": surface_proxies.lexical_density(doc),
    "pos_dissimilarity": surface_proxies.pos_dissimilarity(doc),
    "syntactic_similarity": surface_proxies.syntactic_similarity(doc),

    # Lexical diversity
    "mtld": lexical_diversity_mtld(doc),
    "yule_k": yule_k(doc),

    # Givenness
    "pronoun_density": pronoun_density(doc),
    "pronoun_noun_ratio": pronoun_noun_ratio(doc),

    # Discourse markers
    "discourse_markers": get_overall_markers(doc),
}
```

--------------------------------

### Create Entity Grid for Coherence Analysis

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Creates an entity grid from processed text to analyze entity role transitions across sentences. Requires at least two sentences in the document.

```python
from TRUNAJOD.entity_grid import EntityGrid, get_local_coherence

doc = nlp(
    "El científico descubrió una nueva estrella. "
    "La estrella brilla con luz azul. "
    "El descubrimiento sorprendió a la comunidad científica."
)

# Create entity grid (requires at least 2 sentences)
egid = EntityGrid(doc)

```

--------------------------------

### Compute POS Ratios and Distributions

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates ratios for specific Part-of-Speech types (supporting regex) and the overall verb-to-noun ratio. Also provides a full POS distribution count.

```python
from TRUNAJOD import surface_proxies

doc = nlp("Los científicos estudian el universo con telescopios modernos.")

# Get ratio of specific POS types (supports regex)
verb_ratio = surface_proxies.pos_ratio(doc, "VERB|AUX")
noun_ratio = surface_proxies.pos_ratio(doc, "NOUN|PROPN")
verb_noun_ratio = surface_proxies.verb_noun_ratio(doc)

print(f"Verb ratio: {verb_ratio:.3f}")
print(f"Noun ratio: {noun_ratio:.3f}")
print(f"Verb/Noun ratio: {verb_noun_ratio:.3f}")

# Get full POS distribution
distribution = surface_proxies.pos_distribution(doc)
print(f"POS distribution: {distribution}")
# {'DET': 2, 'NOUN': 3, 'VERB': 1, 'ADP': 1, 'ADJ': 1}
```

--------------------------------

### Count Clauses and Compute Clause Metrics

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Heuristically counts clauses and computes metrics like average clause length and subordination index. Requires the 'infinitive_map' model from TRUNAJOD.

```python
from TRUNAJOD import surface_proxies

doc = nlp(
    "El científico que estudia el universo descubrió que las galaxias "
    "se expanden mientras el tiempo pasa."
)

# Requires infinitive_map from TRUNAJOD models
clause_count = surface_proxies.clause_count(doc, model.infinitive_map)
avg_clause_len = surface_proxies.average_clause_length(doc, model.infinitive_map)
subordination = surface_proxies.subordination(doc, model.infinitive_map)

print(f"Clause count: {clause_count}")
print(f"Average clause length: {avg_clause_len:.2f}")
print(f"Subordination index: {subordination:.2f}")  # Clauses per sentence
```

--------------------------------

### Lemmatize and Check Part-of-Speech for Spanish Tokens

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Demonstrates lemmatization and part-of-speech checking for tokens in a Spanish sentence using TRUNAJOD utility functions. It identifies nouns, verbs, adjectives, adverbs, and pronouns.

```python
from TRUNAJOD.utils import (
    lemmatize, is_word, is_noun, is_verb,
    is_adjective, is_adverb, is_pronoun
)

doc = nlp("El científico estudia las estrellas brillantes.")

for token in doc:
    if is_word(token):
        lemma = lemmatize(model.lemmatizer, token.text.lower())
        pos_info = []
        if is_noun(token): pos_info.append("NOUN")
        if is_verb(token): pos_info.append("VERB")
        if is_adjective(token): pos_info.append("ADJ")
        if is_adverb(token): pos_info.append("ADV")
        if is_pronoun(token): pos_info.append("PRON")

        print(f"{token.text} -> {lemma} ({', '.join(pos_info) or token.pos_})")
```

--------------------------------

### Compute Pronoun Density and Ratio

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates pronoun density and the pronoun-to-noun ratio in a document. Higher values suggest increased reliance on pronouns, potentially indicating greater text complexity or inference requirements.

```python
from TRUNAJOD.givenness import pronoun_density, pronoun_noun_ratio

doc = nlp(
    "El científico estudió la estrella. Él descubrió que ella emite luz azul. "
    "Su descubrimiento cambió nuestra comprensión del universo."
)

p_density = pronoun_density(doc)
pn_ratio = pronoun_noun_ratio(doc)

print(f"Pronoun density: {p_density:.3f}")
print(f"Pronoun-noun ratio: {pn_ratio:.3f}")
# Higher values indicate more pronouns, suggesting more inference required
```

--------------------------------

### Compute Local Coherence Metrics

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates various local coherence measures (unweighted, weighted, and distance-weighted) using entity grid approaches. Requires an initialized EntityGrid object.

```python
from TRUNAJOD.entity_grid import EntityGrid, get_local_coherence

doc = nlp(
    "El universo contiene millones de galaxias. "
    "Las galaxias están formadas por estrellas. "
    "Las estrellas producen luz y calor."
)

egird = EntityGrid(doc)

# Get all local coherence measures
(
    local_coherence_PU,
    local_coherence_PW,
    local_coherence_PACC,
    local_coherence_PU_dist,
    local_coherence_PW_dist,
    local_coherence_PACC_dist
) = get_local_coherence(egrid)

print(f"Local coherence PU: {local_coherence_PU:.3f}")
print(f"Local coherence PW: {local_coherence_PW:.3f}")
print(f"Local coherence PACC: {local_coherence_PACC:.3f}")
print(f"Local coherence PU (dist): {local_coherence_PU_dist:.3f}")
print(f"Local coherence PW (dist): {local_coherence_PW_dist:.3f}")
print(f"Local coherence PACC (dist): {local_coherence_PACC_dist:.3f}")
```

--------------------------------

### Analyze Text Complexity with TRUNAJOD

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/README.md

This Python script analyzes text complexity using various TRUNAJOD indices for Chilean school texts. It processes .docx files, extracts features like lexical diversity and density, and visualizes the results using box plots grouped by school grade. Ensure 'es_core_news_sm' spaCy model is downloaded.

```python
"""Example of TRUNAJOD usage."""
import glob

import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
import spacy
import textract  # To read .docx files
import TRUNAJOD.givenness
import TRUNAJOD.ttr
from TRUNAJOD import surface_proxies
from TRUNAJOD.syllabizer import Syllabizer

plt.rcParams["figure.figsize"] = (11, 4)
plt.rcParams["figure.dpi"] = 200


nlp = spacy.load("es_core_news_sm", disable=["ner", "textcat"])

features = {
    "lexical_diversity_mltd": [],
    "lexical_density": [],
    "pos_dissimilarity": [],
    "connection_words_ratio": [],
    "grade": [],
}
for filename in glob.glob("corpus/*/*.docx"):
    text = textract.process(filename).decode("utf8")
    doc = nlp(text)
    features["lexical_diversity_mltd"].append(
        TRUNAJOD.ttr.lexical_diversity_mtld(doc)
    )
    features["lexical_density"].append(surface_proxies.lexical_density(doc))
    features["pos_dissimilarity"].append(
        surface_proxies.pos_dissimilarity(doc)
    )
    features["connection_words_ratio"].append(
        surface_proxies.connection_words_ratio(doc)
    )

    # In our case corpus was organized as:
    # corpus/5B/5_2_55.docx where the folder that
    # contained the doc, contained the school level, in
    # this example 5th grade
    features["grade"].append(filename.split("/")[1][0])

df = pd.DataFrame(features)


fig, axes = plt.subplots(2, 2)

sns.boxplot(x="grade", y="lexical_diversity_mltd", data=df, ax=axes[0, 0])
sns.boxplot(x="grade", y="lexical_density", data=df, ax=axes[0, 1])
sns.boxplot(x="grade", y="pos_dissimilarity", data=df, ax=axes[1, 0])
sns.boxplot(x="grade", y="connection_words_ratio", data=df, ax=axes[1, 1])

```

--------------------------------

### Count Basic Text Elements

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculate the total number of words, sentences, and characters in a given text using the `surface_proxies` module. This provides fundamental statistics about the text's composition.

```python
from TRUNAJOD import surface_proxies

doc = nlp("El universo es vasto. Las estrellas brillan en la noche.")

word_count = surface_proxies.word_count(doc)
sentence_count = surface_proxies.sentence_count(doc)
char_count = surface_proxies.char_count(doc)

print(f"Words: {word_count}")        # Words: 10
print(f"Sentences: {sentence_count}")  # Sentences: 2
print(f"Characters: {char_count}")   # Characters: 43
```

--------------------------------

### Measure Connection Words Ratio and Negation Density

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Measures the ratio of connection words and the density of negation within a text.

```python
from TRUNAJOD import surface_proxies

doc = nlp(
    "El universo es vasto y misterioso, pero no es infinito. "
    "Ni la ciencia ni la filosofía pueden explicarlo completamente."
)

connection_ratio = surface_proxies.connection_words_ratio(doc)
negation = surface_proxies.negation_density(doc)

print(f"Connection words ratio: {connection_ratio:.3f}")
print(f"Negation density: {negation:.3f}")
```

--------------------------------

### Lexico-Semantic Norms Module

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/docs/api_reference/lexico_semantic_norms.rst

This snippet outlines the available members within the TRUNAJOD.lexico_semantic_norms module.

```APIDOC
## TRUNAJOD.lexico_semantic_norms Module

### Description
This module provides functionalities related to lexico-semantic norms.

### Members
This module exposes the following members:

- `lexico_semantic_norms` (module): Provides access to lexico-semantic norms functionalities.

### Bibliography
References related to lexico-semantic norms can be found in `lexico_ref.bib`.
```

--------------------------------

### Discourse Markers Module

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/docs/api_reference/discourse_markers.rst

Documentation for the TRUNAJOD.discourse_markers module, detailing its members and functionalities.

```APIDOC
## Discourse Markers Module API

### Description
This section details the members and functionalities available within the TRUNAJOD.discourse_markers Python module.

### Module
TRUNAJOD.discourse_markers

### Members
This module exposes various functions and classes related to discourse markers. Please refer to the source code or specific documentation for each member for detailed usage, parameters, and return values.

### Bibliography
References for discourse markers can be found in the `discourse_markers_ref.bib` file.
```

--------------------------------

### Semantic Measures Module

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/docs/api_reference/semantic_measures.rst

This section details the functions available within the TRUNAJOD.semantic_measures module.

```APIDOC
## Semantic Measures API

### Description
This API provides access to various semantic measurement functions within the TRUNAJOD library.

### Module
TRUNAJOD.semantic_measures

### Functions
This module exposes the following functions:

- **`measure_semantic_similarity`**: Calculates the semantic similarity between two pieces of text.
- **`get_semantic_embedding`**: Generates a semantic embedding vector for a given text.
- **`compare_text_meaning`**: Compares the meaning of two texts using semantic analysis.

### Bibliography
References for the semantic measures are available in `semantic_ref.bib`.
```

--------------------------------

### Compute First and Second Person Density

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Computes the density of first and second person references within a given text.

```python
from TRUNAJOD import surface_proxies

doc = nlp("Yo creo que tú puedes entender el universo si estudias mucho.")

density = surface_proxies.first_second_person_density(doc)
print(f"1st/2nd person density: {density:.3f}")
```

--------------------------------

### Syllabizer Module

Source: https://github.com/dpalmasan/trunajod2.0/blob/master/docs/api_reference/syllabizer.rst

This section details the members available within the TRUNAJOD.syllabizer module.

```APIDOC
## Syllabizer Module API

### Description
Provides access to the functions and classes within the TRUNAJOD.syllabizer module.

### Members
This module contains the following members:

- `syllabizer` (function): Function to syllabize words.
- `syllabizer_list` (function): Function to syllabize a list of words.
- `syllabizer_sentence` (function): Function to syllabize a sentence.

### Usage Examples

#### Syllabizing a single word
```python
from TRUNAJOD.syllabizer import syllabizer

word = "ejemplo"
result = syllabizer(word)
print(result)
# Expected output: ['e', 'j', 'e', 'm', 'p', 'l', 'o']
```

#### Syllabizing a list of words
```python
from TRUNAJOD.syllabizer import syllabizer_list

words = ["hola", "mundo"]
result = syllabizer_list(words)
print(result)
# Expected output: [['h', 'o', 'l', 'a'], ['m', 'u', 'n', 'd', 'o']]
```

#### Syllabizing a sentence
```python
from TRUNAJOD.syllabizer import syllabizer_sentence

sentence = "hola mundo"
result = syllabizer_sentence(sentence)
print(result)
# Expected output: [['h', 'o', 'l', 'a'], ['m', 'u', 'n', 'd', 'o']]
```
```

--------------------------------

### Extract Emotion Scores

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Computes emotion scores (joy, anger, fear, disgust, surprise, sadness) from text using a Spanish emotion lexicon. An optional lemmatizer can be provided for improved accuracy.

```python
from TRUNAJOD.emotions import Emotions

doc = nlp(
    "La noticia triste causó mucha tristeza y dolor en la comunidad. "
    "Sin embargo, la esperanza y la alegría regresaron pronto."
)

# Can optionally pass a lemmatizer for better matching
emotions = Emotions(doc, lemmatizer=model.lemmatizer)

print(f"Alegría (Joy): {emotions.get_alegria():.3f}")
print(f"Enojo (Anger): {emotions.get_enojo():.3f}")
print(f"Miedo (Fear): {emotions.get_miedo():.3f}")
print(f"Repulsión (Disgust): {emotions.get_repulsion():.3f}")
print(f"Sorpresa (Surprise): {emotions.get_sorpresa():.3f}")
print(f"Tristeza (Sadness): {emotions.get_tristeza():.3f}")
```

--------------------------------

### Compute Lexico-Semantic Norms

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates average lexico-semantic norms (arousal, concreteness, familiarity, etc.) for a given document using a specified dictionary and lemmatizer.

```python
from TRUNAJOD.lexico_semantic_norms import LexicoSemanticNorm

doc = nlp(
    "El espectáculo del cielo nocturno cautiva la mirada y suscita preguntas "
    "sobre el universo, su origen y su funcionamiento."
)

# Requires lexico-semantic norm dictionary and lemmatizer from TRUNAJOD models
lsn = LexicoSemanticNorm(
    doc,
    model.spanish_lexicosemantic_norms,
    model.lemmatizer
)

print(f"Arousal: {lsn.get_arousal():.3f}")
print(f"Concreteness: {lsn.get_concreteness():.3f}")
print(f"Context availability: {lsn.get_context_availability():.3f}")
print(f"Familiarity: {lsn.get_familiarity():.3f}")
print(f"Imageability: {lsn.get_imageability():.3f}")
print(f"Valence: {lsn.get_valence():.3f}")
```

--------------------------------

### Compute Basic Type-Token Ratio (TTR)

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates the basic Type-Token Ratio from a list of words. This measures lexical diversity by comparing the number of unique words to the total number of words.

```python
from TRUNAJOD.ttr import type_token_ratio

word_list = ["el", "universo", "es", "vasto", "el", "universo", "es", "misterioso"]
ttr = type_token_ratio(word_list)
print(f"TTR: {ttr:.3f}")  # 0.625 (5 unique / 8 total)
```

--------------------------------

### Print Text Complexity Features

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Prints calculated text complexity features. It formats float values to four decimal places.

```python
print("Text Complexity Features:")
print("-" * 40)
for feature, value in features.items():
    if isinstance(value, float):
        print(f"{feature}: {value:.4f}")
    else:
        print(f"{feature}: {value}")
```

--------------------------------

### Compute Syntactic Complexity Measures

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Computes syntactic complexity metrics, specifically the average number of words before the root of a sentence and the density of noun phrases.

```python
from TRUNAJOD import surface_proxies

doc = nlp(
    "El famoso científico alemán estudió el fenómeno. "
    "Los investigadores principales analizaron los resultados."
)

words_before = surface_proxies.words_before_root(doc)
np_density = surface_proxies.noun_phrase_density(doc)

print(f"Words before root: {words_before:.2f}")
print(f"Noun phrase density: {np_density:.2f}")
```

--------------------------------

### Calculate Average Sentence and Word Length

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Compute the average sentence length in words and the average word length in characters. These metrics can serve as indicators of text readability.

```python
from TRUNAJOD import surface_proxies

doc = nlp("El universo es vasto. Las estrellas brillan intensamente en la noche oscura.")

avg_sentence_len = surface_proxies.average_sentence_length(doc)
avg_word_len = surface_proxies.average_word_length(doc)

print(f"Average sentence length: {avg_sentence_len:.2f}")  # ~6.0 words per sentence
print(f"Average word length: {avg_word_len:.2f}")          # ~5.5 chars per word
```

--------------------------------

### Calculate Syllable Count and Ratio

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Determine the total number of syllables in a text and the ratio of syllables per word. This uses a Spanish syllabizer and can indicate text complexity.

```python
from TRUNAJOD import surface_proxies

doc = nlp("El espectáculo del cielo nocturno cautiva la mirada.")

syllable_count = surface_proxies.syllable_count(doc)
syllable_ratio = surface_proxies.syllable_word_ratio(doc)

print(f"Total syllables: {syllable_count}")      # Total syllables: ~18
print(f"Syllables per word: {syllable_ratio:.2f}")  # ~2.25 syllables per word
```

--------------------------------

### Split Spanish Word into Syllables

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Splits a Spanish word into its constituent syllables using the Syllabizer.split method. This requires creating a CharLine object from the word.

```python
from TRUNAJOD.syllabizer import Syllabizer, CharLine

word = "extraordinario"
charline = CharLine(word)
syllables = Syllabizer.split(charline)

print(f"Word: {word}")
print(f"Syllables: {[s.word for s in syllables]}")
# ['ex', 'tra', 'or', 'di', 'na', 'rio']
```

--------------------------------

### Count Spanish Discourse Markers

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates the counts of various types of Spanish discourse markers, including cause, context, equality, polysemic, revision, and vague meaning words. An overall marker count is also provided.

```python
from TRUNAJOD.discourse_markers import (
    get_cause_dm_count,
    get_context_dm_count,
    get_equality_dm_count,
    get_polysemic_dm_count,
    get_revision_dm_count,
    get_closed_class_vague_meaning_count,
    get_overall_markers
)

doc = nlp(
    "Debido a la gravedad, los planetas orbitan el sol. "
    "Sin embargo, algunos asteroides tienen órbitas irregulares. "
    "Por ejemplo, el asteroide Apophis tiene una trayectoria peculiar. "
    "Además, los cometas también muestran comportamientos únicos."
)

print(f"Cause markers: {get_cause_dm_count(doc):.3f}")
print(f"Context markers: {get_context_dm_count(doc):.3f}")
print(f"Equality markers: {get_equality_dm_count(doc):.3f}")
print(f"Polysemic markers: {get_polysemic_dm_count(doc):.3f}")
print(f"Revision markers: {get_revision_dm_count(doc):.3f}")
print(f"Vague meaning words: {get_closed_class_vague_meaning_count(doc):.3f}")
print(f"Overall markers: {get_overall_markers(doc):.3f}")
```

--------------------------------

### Calculate Entity Grid Coherence

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates local coherence using entity grids. This snippet should be used within a try-except block to handle potential RuntimeErrors.

```python
try:
    egrid = EntityGrid(doc)
    coherence = get_local_coherence(egrid)
    features["local_coherence_PW"] = coherence[1]
    features["local_coherence_PACC"] = coherence[2]
except RuntimeError:
    pass
```

--------------------------------

### Compute Lexical Density

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculate the lexical density of a text, which is the ratio of content words (nouns, verbs, adjectives, adverbs) to the total number of words. Higher values suggest a more content-rich text.

```python
from TRUNAJOD import surface_proxies

doc = nlp("El científico investigó cuidadosamente los fenómenos astronómicos complejos.")

density = surface_proxies.lexical_density(doc)
print(f"Lexical density: {density:.3f}")  # Higher values indicate more content-rich text
```

--------------------------------

### Compute D Estimate for Lexical Diversity

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Computes the D measurement for lexical diversity using curve fitting. This function requires a minimum text length and allows specifying the range of tokens to sample and the number of trials.

```python
from TRUNAJOD.ttr import d_estimate

doc = nlp(
    "El universo contiene millones de galaxias diferentes. "
    "Cada galaxia tiene billones de estrellas únicas. "
    "Las estrellas forman sistemas planetarios complejos. "
    "Los planetas orbitan alrededor de sus estrellas madre. "
    "La vida puede existir en algunos de estos mundos distantes."
)

# Requires sufficient text length (default: samples 35-50 tokens)
d = d_estimate(doc, min_range=35, max_range=50, trials=5)
print(f"D estimate: {d:.2f}")
```

--------------------------------

### Compute Synonym Overlap Between Sentences

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates the semantic overlap between adjacent sentences by comparing their synonym sets. Requires lemmatized sentences and a synset dictionary.

```python
from TRUNAJOD.semantic_measures import overlap, get_synsets

# Prepare lemmatized sentences
doc = nlp(
    "El científico estudia el cosmos. "
    "El investigador analiza el universo."
)

sentences = list(doc.sents)
lemma_list_group = [
    [token.lemma_ for token in sent if token.pos_ not in ("PUNCT", "SPACE")]
    for sent in sentences
]

# Requires synset dictionary from TRUNAJOD models
synonym_overlap = overlap(lemma_list_group, model.wordnet_noun_synsets)
print(f"Synonym overlap: {synonym_overlap:.3f}")
```

--------------------------------

### Compute Yule's K Lexical Diversity

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates Yule's K characteristic, a measure of lexical diversity where a higher K value indicates more word repetition and thus lower diversity.

```python
from TRUNAJOD.ttr import yule_k

doc = nlp(
    "El científico estudia el universo. El universo es vasto. "
    "El científico descubre secretos del universo misterioso."
)

k = yule_k(doc)
print(f"Yule's K: {k:.2f}")
# Higher K indicates more repetition (less diversity)
```

--------------------------------

### Compute Syntactic Similarity Between Sentences

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Computes the average syntactic similarity between adjacent sentences based on their parse tree structure. Higher values indicate more similar syntactic structures.

```python
from TRUNAJOD import surface_proxies

doc = nlp(
    "El científico estudia el universo. "
    "El astrónomo observa las galaxias. "
    "El investigador analiza los datos."
)

similarity = surface_proxies.syntactic_similarity(doc)
print(f"Syntactic similarity: {similarity:.3f}")
# Higher values indicate more similar syntactic structures
```

--------------------------------

### Compute Word Variation Index

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates the Word Variation Index, which serves as a measure of idea density within a text. It requires an NLP-processed document.

```python
from TRUNAJOD.ttr import word_variation_index

doc = nlp(
    "El científico investigador estudia los fenómenos astronómicos complejos. "
    "Los telescopios modernos capturan imágenes detalladas del cosmos lejano."
)

wvi = word_variation_index(doc)
print(f"Word Variation Index: {wvi:.2f}")
```

--------------------------------

### Compute MTLD Lexical Diversity

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Calculates the Measure of Textual Lexical Diversity (MTLD) using a default or custom Type-Token Ratio (TTR) segment threshold. This method is applied bi-directionally for robustness.

```python
from TRUNAJOD.ttr import lexical_diversity_mtld

doc = nlp(
    "El universo contiene galaxias enormes. Las galaxias tienen estrellas brillantes. "
    "Las estrellas producen luz intensa. La luz viaja por el espacio infinito. "
    "El espacio contiene misterios profundos."
)

# Default TTR segment threshold is 0.72
mtld = lexical_diversity_mtld(doc)
print(f"MTLD lexical diversity: {mtld:.2f}")

# Custom TTR threshold
mtld_custom = lexical_diversity_mtld(doc, ttr_segment=0.70)
print(f"MTLD (threshold 0.70): {mtld_custom:.2f}")
```

--------------------------------

### Compute Frequency Index of Rarest Word

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Computes the average frequency of the rarest word per sentence using a provided frequency dictionary. Requires the 'crea_frequency' dictionary from TRUNAJOD models. Negative values indicate rare words, while values closer to 0 suggest common words.

```python
from TRUNAJOD import surface_proxies

doc = nlp(
    "El espectáculo del cielo nocturno cautiva la mirada. "
    "Las estrellas brillan en la oscuridad."
)

# Requires CREA frequency dictionary from TRUNAJOD models
freq_index = surface_proxies.frequency_index(doc, model.crea_frequency)
print(f"Frequency index: {freq_index:.4f}")
# Negative values indicate rare words; closer to 0 means common words
```

--------------------------------

### Find Discourse Marker Matches with Custom List

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Identifies discourse markers within a given text using a custom list of markers. This is useful for analyzing non-Spanish texts or specific marker sets.

```python
from TRUNAJOD.discourse_markers import find_matches

text = "Therefore, the experiment succeeded. However, more tests are needed."
custom_markers = ["therefore", "however", "moreover", "consequently"]

matches = find_matches(text, custom_markers)
print(f"Custom marker matches: {matches}")  # 2
```

--------------------------------

### Measure POS Dissimilarity Between Sentences

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Measures the Part-of-Speech dissimilarity between adjacent sentences in a document. Lower values indicate more similar sentence structures.

```python
from TRUNAJOD import surface_proxies

doc = nlp(
    "El científico estudia las estrellas. "
    "Los telescopios capturan imágenes asombrosas. "
    "La investigación revela nuevos descubrimientos."
)

dissimilarity = surface_proxies.pos_dissimilarity(doc)
print(f"POS dissimilarity: {dissimilarity:.3f}")
# Lower values indicate more similar sentence structures
```

--------------------------------

### Calculate Average Semantic Similarity

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Computes the average semantic similarity between sentences in a given text using word embeddings. Requires the text to be processed by an NLP model.

```python
doc = nlp(text)
sentences = list(doc.sents)
n_sentences = len(sentences)

# Create generator of sentence docs
docs = (nlp(sent.text) for sent in sentences)

similarity = avg_w2v_semantic_similarity(docs, n_sentences)
print(f"Average semantic similarity: {similarity:.3f}")
```

--------------------------------

### Count Syllables in Spanish Words

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Uses the Syllabizer class to count the number of syllables in a list of Spanish words. This function is useful for phonetic analysis.

```python
from TRUNAJOD.syllabizer import Syllabizer

words = ["espectáculo", "universo", "cielo", "extraordinario", "a"]
for word in words:
    syllables = Syllabizer.number_of_syllables(word)
    print(f"'{word}': {syllables} syllables")

# 'espectáculo': 5 syllables
# 'universo': 4 syllables
# 'cielo': 2 syllables
# 'extraordinario': 6 syllables
# 'a': 1 syllable
```

--------------------------------

### Calculate Average Word2Vec Semantic Similarity

Source: https://context7.com/dpalmasan/trunajod2.0/llms.txt

Computes the average semantic similarity between adjacent sentences in a text using word vector embeddings. This measure can indicate the topical coherence of the text.

```python
from TRUNAJOD.semantic_measures import avg_w2v_semantic_similarity

text = (
    "El universo es infinito y misterioso. "
    "Las galaxias contienen millones de estrellas. "
    "Las estrellas producen luz y energía."
)


```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.