### Install g2p_en Source: https://github.com/kyubyong/g2p/blob/master/README.rst Install the g2p_en module using pip or by building from source. ```bash pip install g2p_en ``` ```bash python setup.py install ``` -------------------------------- ### G2P Conversion Example Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Demonstrates basic usage of the G2P converter to transform text into phonemes. Ensure the G2P model is initialized. ```python from g2p import G2p g2p = G2p() text = "This is a test." phonemes = g2p(text) print(f'Text: {text}\nPhonemes: {phonemes}') ``` -------------------------------- ### G2P Error Handling Example Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Illustrates how to handle potential errors during G2P conversion, such as invalid input. This example shows a try-except block for catching exceptions. ```python from g2p import G2p from g2p.exceptions import G2pError g2p = G2p() try: invalid_text = "" phonemes = g2p(invalid_text) print(phonemes) except G2pError as e: print(f"An error occurred: {e}") print("Please provide valid text input.") ``` -------------------------------- ### Manual NLTK Data Setup for G2P Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Manually download required NLTK data packages before initializing the G2p class. This is necessary if the library fails to auto-download them. ```python import nltk nltk.download('averaged_perceptron_tagger') nltk.download('cmudict') from g2p_en import G2p g2p = G2p() # Should succeed now ``` -------------------------------- ### Full Reinstall g2p_en Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Use this command to uninstall and then upgrade the g2p_en library to the latest version. This can resolve issues caused by corrupted installations. ```bash pip uninstall g2p_en -y pip install --upgrade g2p_en ``` -------------------------------- ### Homograph Resolution Logic Example Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/types.md Demonstrates the logic within `G2p.__call__()` for selecting the correct pronunciation of a homograph based on the provided part of speech tag. ```python word, pos = "refuse", "VB" # verb, past tense pron1, pron2, pos1 = homograph2features["refuse"] if pos.startswith(pos1): # pos.startswith("V") → True pronunciation = pron1 else: pronunciation = pron2 ``` -------------------------------- ### Number Expansion Example Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Shows how to use the expand module to convert numbers within text into their word representations. This is useful for processing text containing numerical data. ```python from g2p.expand import normalize_numbers text = "There are 12 apples and 3 oranges." expanded_text = normalize_numbers(text) print(f'Original: {text}\nExpanded: {expanded_text}') ``` -------------------------------- ### Error Handling Documentation Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Comprehensive guide to error scenarios, exception types, and handling strategies within the G2P project. ```APIDOC ## Errors ### Error Scenarios - 10 distinct error scenarios are documented. - Each scenario includes: - Trigger conditions - Exception type - When the error occurs - Source location of the error - Code examples for handling the error - Strategies for preventing the error ### Exception Hierarchy - The hierarchy of exceptions is detailed. ### Recovery Strategies - A section dedicated to recovery strategies is included. ``` -------------------------------- ### g2p-en Python Dependencies Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/configuration.md Lists the required Python packages and their minimum version constraints for installing g2p-en. These packages are essential for the library's functionality. ```python install_requires = [ 'numpy>=1.13.1', 'nltk>=3.2.4', 'inflect>=0.3.1', 'distance>=0.1.3', ] ``` -------------------------------- ### Example Homograph Dictionary Entry Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/types.md Illustrates a sample entry in the homograph dictionary for the word 'refuse', showing pronunciations for verb and noun forms, and the corresponding POS tag. ```python { "refuse": ( ["R", "IH0", "F", "Y", "UW1", "Z"], # Pronunciation as verb ["R", "EH1", "F", "Y", "UW2", "Z"], # Pronunciation as noun "V" # POS tag when pron1 applies ), "abstract": ( ["AE0", "B", "S", "T", "R", "AE1", "K", "T"], # Verb form ["AE1", "B", "S", "T", "R", "AE2", "K", "T"], # Noun/adjective form "V" # Verb tag ) } ``` -------------------------------- ### Automatic Number Expansion in Text Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/README.md Demonstrates how the G2p library automatically expands numerical figures within a sentence into their word equivalents before phoneme conversion. No explicit setup is required for this feature. ```python g2p = G2p() result = g2p("I have $250 and 3 items.") # Numbers are automatically expanded: "two hundred fifty dollars", "three" ``` -------------------------------- ### Set NLTK Data Path Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/configuration.md Example of setting the NLTK_DATA environment variable to override the default path for NLTK data. This is useful for managing corpus and model data locations. ```bash export NLTK_DATA=/data/nltk_data python -c "from g2p_en import G2p; g2p = G2p()" ``` -------------------------------- ### Configuration Documentation Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Details on how to configure and set up the G2P project, including dependencies and model files. ```APIDOC ## Configuration ### Setup - Constructor parameters: None explicitly mentioned as configurable. - NLTK auto-download behavior. - Environment variable support. ### Dependencies - All dependency versions are listed. - Python 3.x requirement. ### Models - Model files are documented. ### Parameters - Hard-coded parameters are listed. - Processing steps are enumerated. ### Non-Customizable Aspects - Explicit note on the non-customizable nature of certain aspects. ``` -------------------------------- ### G2P Library Usage Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/configuration.md Demonstrates the correct and incorrect ways to instantiate the G2P library. Only the default instantiation is supported; custom parameters are not allowed. ```python # ❌ Not supported: g2p = G2p(weights_path="/custom/model.npz") g2p = G2p(use_dictionary=False) g2p = G2p(predict_oov=False) # ✓ Only way to use: g2p = G2p() result = g2p(text) ``` -------------------------------- ### Initialize G2p Class Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/configuration.md Instantiate the G2p class. The constructor does not accept any parameters. ```python from g2p_en import G2p g2p = G2p() # No arguments ``` -------------------------------- ### Basic Usage of g2p-en for Text Conversion Source: https://github.com/kyubyong/g2p/blob/master/README.md Demonstrates how to initialize the G2p class and convert various English texts, including numbers, abbreviations, homographs, and new words, into their phoneme representations. ```python from g2p_en import G2p texts = ["I have $250 in my pocket.", # number -> spell-out "popular pets, e.g. cats and dogs", # e.g. -> for example "I refuse to collect the refuse around here.", # homograph "I'm an activationist."] # newly coined word g2p = G2p() for text in texts: out = g2p(text) print(out) >>> ['AY1', ' ', 'HH', 'AE1', 'V', ' ', 'T', 'UW1', ' ', 'HH', 'AH1', 'N', 'D', 'R', 'AH0', 'D', ' ', 'F', 'IH1', 'F', 'T', 'IY0', ' ', 'D', 'AA1', 'L', 'ER0', 'Z', ' ', 'IH0', 'N', ' ', 'M', 'AY1', ' ', 'P', 'AA1', 'K', 'AH0', 'T', ' ', '.'] >>> ['P', 'AA1', 'P', 'Y', 'AH0', 'L', 'ER0', ' ', 'P', 'EH1', 'T', 'S', ' ', ',', ' ', 'F', 'AO1', 'R', ' ', 'IH0', 'G', 'Z', 'AE1', 'M', 'P', 'AH0', 'L', ' ', 'K', 'AE1', 'T', 'S', ' ', 'AH0', 'N', 'D', ' ', 'D', 'AA1', 'G', 'Z'] >>> ['AY1', ' ', 'R', 'IH0', 'F', 'Y', 'UW1', 'Z', ' ', 'T', 'UW1', ' ', 'K', 'AH0', 'L', 'EH1', 'K', 'T', ' ', 'DH', 'AH0', ' ', 'R', 'EH1', 'F', 'Y', 'UW2', 'Z', ' ', 'ER0', 'AW1', 'N', 'D', ' ', 'HH', 'IY1', 'R', ' ', '.'] >>> ['AY1', ' ', 'AH0', 'M', ' ', 'AE1', 'N', ' ', 'AE2', 'K', 'T', 'IH0', 'V', 'EY1', 'SH', 'AH0', 'N', 'IH0', 'S', 'T', ' ', '.'] ``` -------------------------------- ### Import G2p Class Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/README.md Import the main G2p class for grapheme-to-phoneme conversion. ```python from g2p_en import G2p ``` -------------------------------- ### G2p Class Initialization Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Instantiate the G2p class to begin grapheme-to-phoneme conversion. Ensure NLTK data and model checkpoints are available. ```python from g2p_en import G2p g2p = G2p() ``` -------------------------------- ### Handle Missing Model Checkpoint File Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Catches FileNotFoundError if the 'checkpoint20.npz' model file is missing during G2p initialization. Suggests reinstalling the package. ```python from g2p_en import G2p try: g2p = G2p() except FileNotFoundError as e: print(f"Model checkpoint missing: {e}") print("Please reinstall the g2p_en package:") print(" pip install --force-reinstall g2p_en") ``` -------------------------------- ### G2P Pipeline Initialization and Usage Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/api-reference/expand.md Demonstrates how to initialize the G2p class and use it to process text containing numbers, currency, and ordinals. The `normalize_numbers` function is called internally as the first preprocessing step. ```python from g2p_en import G2p g2p = G2p() result = g2p("I have $250 and 2 items on the 3rd shelf.") # Internally calls: normalize_numbers("I have $250 and 2 items on the 3rd shelf.") # Result includes: "two hundred fifty dollars", "two", "third" ``` -------------------------------- ### G2p Constructor Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/api-reference/g2p.md Initializes the G2p converter. This process loads necessary data like phoneme mappings, grapheme embeddings, neural network weights, and dictionaries. It can automatically download required NLTK data if missing. ```APIDOC ## G2p() ### Description Initializes the G2p converter with phoneme mappings, grapheme embeddings, neural network weights, CMU dictionary, and homograph features. ### Returns `G2p` instance ready for conversion. ### Throws - `LookupError`: Raised if required NLTK data (`averaged_perceptron_tagger`, `cmudict`) cannot be found. The constructor automatically downloads missing data via `nltk.download()`. ### Notes Initialization loads: - 29 graphemes (pad, unk, end-of-sequence, a-z) - 74 phonemes (ARPAbet phoneme set) - Pre-trained GRU encoder-decoder weights from `checkpoint20.npz` - CMU Pronouncing Dictionary corpus - Homograph pronunciation mappings from `homographs.en` ### Example ```python from g2p_en import G2p # Create converter instance g2p = G2p() # Basic usage result = g2p("Hello world") print(result) # ['HH', 'AH0', 'L', 'OW1', ' ', 'W', 'ER1', 'L', 'D'] ``` ``` -------------------------------- ### Initialize G2P and Convert Text to Phonemes Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/00_START_HERE.md Instantiate the G2p class and use it to convert a given text string into a list of ARPAbet phonemes. This is the primary method for text-to-phoneme conversion. ```python from g2p_en import G2p g2p = G2p() phonemes = g2p("I have $250 in my pocket.") # → ['AY1', ' ', 'HH', 'AE1', 'V', ...] ``` -------------------------------- ### Basic Usage of g2p_en Source: https://github.com/kyubyong/g2p/blob/master/README.rst Demonstrates converting various English texts to phonemes using the G2p class. Handles numbers, abbreviations, homographs, and new words. ```python from g2p_en import G2p texts = ["I have $250 in my pocket.", # number -> spell-out "popular pets, e.g. cats and dogs", # e.g. -> for example "I refuse to collect the refuse around here.", # homograph "I'm an activationist."] # newly coined word g2p = G2p() for text in texts: out = g2p(text) print(out) ``` -------------------------------- ### Verify Model Checkpoint File Path Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md A bash command to print the expected path of the 'checkpoint20.npz' file, useful for debugging missing file errors. ```bash python -c "import g2p_en; import os; print(os.path.join(os.path.dirname(g2p_en.__file__), 'checkpoint20.npz'))" ``` -------------------------------- ### Handle Missing Homographs File Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Catches FileNotFoundError if the 'homographs.en' file is missing during G2p initialization. Recommends package reinstallation. ```python from g2p_en import G2p try: g2p = G2p() except FileNotFoundError as e: if "homographs.en" in str(e): print("Homographs file missing. Reinstall package:") print(" pip install --force-reinstall g2p_en") raise ``` -------------------------------- ### Graceful Fallback for G2P Initialization Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Implement a try-except block to catch initialization errors for the G2p class. If an error occurs, a fallback function can be used to provide empty results, preventing application crashes. ```python from g2p_en import G2p import logging logging.basicConfig() logger = logging.getLogger(__name__) try: g2p = G2p() except Exception as e: logger.error(f"G2p initialization failed: {e}") # Fallback: use pre-computed pronunciations pronunciation_cache = {} def fallback_g2p(text): return pronunciation_cache.get(text, []) g2p = fallback_g2p ``` -------------------------------- ### G2P API Reference Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Details on the G2P class, its constructor, and core methods for grapheme-to-phoneme conversion. ```APIDOC ## G2p() ### Description Constructor for the G2p class. Initializes the grapheme-to-phoneme conversion model. ### Parameters (No specific parameters documented in the source for the constructor itself, but notes on initialization are mentioned.) ### Methods - **__call__(text)**: Performs grapheme-to-phoneme conversion on the input text. - **encode(text)**: Encodes text into a sequence of phonemes. - **predict(text)**: Predicts phoneme sequences for the given text. - **grucell(input, state)**: Internal GRU cell computation. - **gru(input, state)**: GRU layer computation. - **sigmoid(x)**: Sigmoid activation function. - **load_variables(path)**: Loads pre-trained model variables. ### Instance Attributes (Documented, but specific attributes not listed in source.) ### Source Location (Provided for all methods.) ### Examples (Multiple code examples provided.) ``` -------------------------------- ### Expand API Reference Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Documentation for number expansion and related helper functions. ```APIDOC ## expand Module ### Description Provides functionality for expanding numbers and related text processing tasks. ### Functions - **normalize_numbers(number_string)**: Normalizes number strings according to specified conversion rules. - **6 helper functions**: (Specific names and descriptions not provided in source.) ### Conversion Rules (A matrix detailing conversion rules is available.) ### Examples (Conversion rule examples are provided.) ### Module Variables (Listed in the documentation.) ### Integration Explains integration with the G2p module. ``` -------------------------------- ### G2p Class Methods Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/DOCUMENTATION_INDEX.md Overview of the methods available in the G2p class for Grapheme-to-Phoneme conversion. ```python g2p_en.G2p └─ __init__() → None └─ __call__(text: str) → list[str] └─ load_variables() → None └─ encode(word: str) → np.ndarray └─ predict(word: str) → list[str] └─ grucell(...) → np.ndarray └─ gru(...) → np.ndarray └─ sigmoid(x: np.ndarray) → np.ndarray ``` -------------------------------- ### Types Documentation Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Defines and explains the various types used within the G2P project, including phonemes, graphemes, and structures. ```APIDOC ## Types ### Phonemes - 74 phonemes documented with IPA and examples. - 14 vowels with stress markers explained. - 23 consonants with IPA. ### Graphemes - Grapheme set includes 29 characters. ### Structures - Homograph structure explained with examples. ### Other Type Information - CMU dictionary format. - Neural network weight shapes. - Type usage summary. ``` -------------------------------- ### Automatic NLTK Data Downloads Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/configuration.md The G2p constructor automatically checks for and downloads required NLTK data if not found. This includes the averaged perceptron tagger and the CMU Pronouncing Dictionary. ```python def __init__(self): # ... other initialization ... try: nltk.data.find('taggers/averaged_perceptron_tagger.zip') except LookupError: nltk.download('averaged_perceptron_tagger') # Auto-download try: nltk.data.find('corpora/cmudict.zip') except LookupError: nltk.download('cmudict') # Auto-download # ... rest of initialization ... ``` -------------------------------- ### Load Neural Network Weights Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/api-reference/g2p.md Loads pre-trained GRU encoder-decoder model weights from a checkpoint file. This method is automatically called during initialization. ```python def load_variables(self) -> None: # Loads NumPy-serialized GRU encoder-decoder model weights from checkpoint20.npz. # Populates instance attributes like enc_emb, enc_w_ih, etc. pass ``` -------------------------------- ### Check and Download NLTK Data Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Ensures required NLTK data like 'averaged_perceptron_tagger' and 'cmudict' are available. Attempts to download them if missing during G2p initialization. ```python try: nltk.data.find('taggers/averaged_perceptron_tagger.zip') except LookupError: nltk.download('averaged_perceptron_tagger') try: nltk.data.find('corpora/cmudict.zip') except LookupError: nltk.download('cmudict') ``` -------------------------------- ### Batch Processing Multiple Texts Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/README.md Shows how to process a list of sentences efficiently for phoneme conversion. This is useful for converting multiple text inputs in a single run. Ensure the G2p class is imported. ```python from g2p_en import G2p g2p = G2p() texts = [ "The quick brown fox", "jumps over the lazy dog", "I'm ready to go!" ] for text in texts: phonemes = g2p(text) print(f"{text:<30} → {' '.join(phonemes)}") ``` -------------------------------- ### G2p Class __call__ Method Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Use the __call__ method to convert a string of graphemes to phonemes. This is the primary method for inference. ```python phonemes = g2p('This is a test.') ``` -------------------------------- ### G2p Class Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Comprehensive reference for the G2p class, which is the main interface for the g2p-en library. It includes details on its constructor, core methods like __call__, load_variables, encode, and predict, as well as internal methods and instance attributes. ```APIDOC ## G2p Class ### Description Provides the primary interface for the g2p-en library, handling the conversion of graphemes to phonemes. ### Methods - **__init__()**: Constructor for the G2p class. Initializes the model and loads necessary variables. - **__call__(text)**: Processes the input text through the full algorithm flow, from preprocessing to phoneme prediction. - **load_variables()**: Loads the pre-trained model weights and other necessary variables. - **encode(text)**: Encodes the input text into a sequence of phonemes. - **predict(sequence)**: Predicts phonemes for a given sequence of graphemes or intermediate representations. - **grucell(input, state)**: Implements a Gated Recurrent Unit (GRU) cell operation. - **gru(input, state)**: Applies the GRU layer operation. - **sigmoid(x)**: Computes the sigmoid activation function. ### Parameters Detailed parameter tables for all methods are available in the source documentation. ### Return Types Return type documentation for all methods is available in the source documentation. ### Exceptions Exception documentation per method is available in the source documentation. ### Usage Examples 10+ usage examples are provided in the source documentation. ``` -------------------------------- ### Homograph Disambiguation with POS Tagging Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/README.md Illustrates how the G2p library uses Part-of-Speech (POS) tagging to differentiate pronunciations of homographs like 'refuse'. The library correctly assigns different phonemes based on whether the word is used as a verb or a noun. ```python g2p = G2p() # "refuse" as verb (to decline) verb = g2p("I refuse to go.") # First "refuse" → verb pronunciation # "refuse" as noun (trash) noun = g2p("This refuse must be disposed.") # "refuse" → noun pronunciation (different phonemes due to POS) ``` -------------------------------- ### g2p_en.G2p Class Methods Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/DOCUMENTATION_INDEX.md The G2p class provides the core functionality for converting text to phonemes. It includes methods for initialization, direct text conversion, loading variables, encoding words, prediction, and internal neural network operations. ```APIDOC ## Class: g2p_en.G2p ### Description Provides the main interface for text-to-phoneme conversion. ### Methods #### `__init__()` - **Description**: Initializes the G2p object. - **Returns**: None #### `__call__(text: str)` - **Description**: Converts a given text string into a list of phoneme strings. - **Parameters**: - `text` (str) - The input text to convert. - **Returns**: list[str] - A list of phoneme strings. #### `load_variables()` - **Description**: Loads necessary variables for the G2p model. - **Returns**: None #### `encode(word: str)` - **Description**: Encodes a word into a numerical representation (numpy array). - **Parameters**: - `word` (str) - The input word to encode. - **Returns**: np.ndarray - The encoded numerical representation of the word. #### `predict(word: str)` - **Description**: Predicts the phoneme sequence for a given word. - **Parameters**: - `word` (str) - The input word for prediction. - **Returns**: list[str] - A list of predicted phoneme strings. #### `grucell(...)` - **Description**: Internal method representing a Gated Recurrent Unit cell operation. - **Returns**: np.ndarray #### `gru(...)` - **Description**: Internal method representing a Gated Recurrent Unit layer operation. - **Returns**: np.ndarray #### `sigmoid(x: np.ndarray)` - **Description**: Applies the sigmoid activation function to a numpy array. - **Parameters**: - `x` (np.ndarray) - The input numpy array. - **Returns**: np.ndarray - The result of the sigmoid function. ``` -------------------------------- ### Handle NLTK Data Download Failure Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Catches LookupError during G2p initialization if NLTK data download fails. Provides a fallback to manually download the data and retry initialization. ```python import nltk from g2p_en import G2p try: g2p = G2p() except LookupError as e: print(f"NLTK data download failed: {e}") # Manually download if auto-download fails: nltk.download('averaged_perceptron_tagger', quiet=True) nltk.download('cmudict', quiet=True) g2p = G2p() ``` -------------------------------- ### load_variables() Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/api-reference/g2p.md Loads the pre-trained neural network weights for the GRU encoder-decoder model. This method is automatically called during the `__init__()` process and populates various instance attributes related to the model's weights and biases. ```APIDOC ## load_variables() ### Description Loads pre-trained neural network weights from checkpoint file. ### Method Signature ```python def load_variables(self) -> None: ``` ### Description Loads NumPy-serialized GRU encoder-decoder model weights from `checkpoint20.npz`. Called during `__init__()`. Populates the following instance attributes: - `enc_emb` (29, 64): Grapheme embedding matrix - `enc_w_ih` (384, 64): Encoder input-to-hidden weights - `enc_w_hh` (384, 128): Encoder hidden-to-hidden weights - `enc_b_ih` (384,): Encoder input bias - `enc_b_hh` (384,): Encoder hidden bias - `dec_emb` (74, 64): Phoneme embedding matrix - `dec_w_ih` (384, 64): Decoder input-to-hidden weights - `dec_w_hh` (384, 128): Decoder hidden-to-hidden weights - `dec_b_ih` (384,): Decoder input bias - `dec_b_hh` (384,): Decoder hidden bias - `fc_w` (74, 128): Fully-connected layer weights (128→74 phonemes) - `fc_b` (74,): Fully-connected layer bias ### Raises - `FileNotFoundError`: If `checkpoint20.npz` is not found in the package directory ``` -------------------------------- ### Pre-download NLTK Data Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Manually downloads NLTK data corpora before initializing G2p to prevent runtime LookupErrors. ```python import nltk nltk.download('averaged_perceptron_tagger') nltk.download('cmudict') ``` -------------------------------- ### CMU Dictionary Entry Format Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/types.md Illustrates the structure of CMU dictionary entries as used by G2P. It shows words mapped to lists of phonetic pronunciations, with G2P utilizing only the first variant. ```python dict[str, list[list[str]]] # Example: { "hello": [ ["HH", "AH0", "L", "OW1"], # Primary pronunciation ["HH", "EH1", "L", "OW0"] # Alternative pronunciation ], "world": [ ["W", "ER1", "L", "D"] # Only variant ] } ``` -------------------------------- ### gru(x, steps, w_ih, w_hh, b_ih, b_hh, h0=None) Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/api-reference/g2p.md Executes a multi-step GRU recurrence over an input sequence. It can optionally take an initial hidden state and returns the full sequence of hidden states. ```APIDOC ## gru(x, steps, w_ih, w_hh, b_ih, b_hh, h0=None) ### Description Multi-step GRU recurrence. Unrolls the GRU cell over a specified number of time steps for an input sequence. ### Parameters #### Path Parameters - **x** (np.ndarray) - Required - Input sequence, shape (batch, steps, emb_dim) - **steps** (int) - Required - Number of time steps to unroll - **w_ih** (np.ndarray) - Required - Input-to-hidden weights - **w_hh** (np.ndarray) - Required - Hidden-to-hidden weights - **b_ih** (np.ndarray) - Required - Input-to-hidden bias - **b_hh** (np.ndarray) - Required - Hidden-to-hidden bias - **h0** (np.ndarray) - Optional - Initial hidden state. If None, initialized to zeros with shape (batch, hidden_dim) ### Returns `np.ndarray` of shape (batch, steps, hidden_dim) — Full hidden state sequence ``` -------------------------------- ### Initialize TweetTokenizer for Text Tokenization Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/configuration.md Initializes NLTK's TweetTokenizer for text tokenization. This tokenizer preserves contractions and handles emoticons and URLs naturally. ```python from nltk.tokenize import TweetTokenizer word_tokenize = TweetTokenizer().tokenize ``` -------------------------------- ### G2p Class sigmoid Method Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Applies the sigmoid activation function. ```python activated_output = g2p.sigmoid(input_data) ``` -------------------------------- ### G2P Public Functions Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/DOCUMENTATION_INDEX.md List of public utility functions for text expansion and normalization within the g2p_en.expand module. ```python g2p_en.expand.normalize_numbers(text: str) → str g2p_en.expand._remove_commas(m: re.Match) → str g2p_en.expand._expand_decimal_point(m: re.Match) → str g2p_en.expand._expand_dollars(m: re.Match) → str g2p_en.expand._expand_ordinal(m: re.Match) → str g2p_en.expand._expand_number(m: re.Match) → str ``` -------------------------------- ### Basic Text to Phoneme Conversion Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/README.md Converts a simple English sentence into its phoneme representation using the G2p class. Ensure the G2p class is imported. ```python from g2p_en import G2p g2p = G2p() result = g2p("Hello, world!") print(result) # ['HH', 'AH0', 'L', 'OW1', ',', ' ', 'W', 'ER1', 'L', 'D', '!'] ``` -------------------------------- ### G2p Class grucell Method Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Internal GRU cell computation. ```python output, state = g2p.grucell(input_data, state) ``` -------------------------------- ### __call__(text) Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/api-reference/g2p.md Converts English text to a list of phonemes. This method handles various text normalization steps, tokenization, and uses a multi-stage lookup process (dictionary, homograph, deep learning) to determine the correct phoneme sequence. ```APIDOC ## __call__(text) ### Description Converts English text to a list of phonemes and preserves non-alphabetic tokens. ### Method Signature ```python def __call__(self, text: str) -> list[str]: ``` ### Parameters #### Path Parameters - **text** (`str`) - Required - Input English text to convert ### Returns `list[str]` — List of phonemes (ARPAbet format) and preserved punctuation/spaces. Each element is either: - A phoneme code (e.g., 'AH0', 'HH', 'L') - A space character `' '` - Punctuation (`.`, `,`, `?`, `!`) ### Conversion Algorithm The method applies the following steps in order: 1. **Preprocessing**: - Converts unicode to ASCII (strips accents) - Normalizes numbers to spelled-out words (e.g., `$250` → `two hundred fifty dollars`) - Handles abbreviations (e.g., `i.e.` → `that is`, `e.g.` → `for example`) - Converts to lowercase - Removes characters outside `[a-z '.,?!\]` 2. **Tokenization**: Uses NLTK's TweetTokenizer to split text and pos_tag to assign POS tags 3. **Phoneme Lookup** (per word): - If no alphabetic characters: preserve as-is - If homograph (in homograph dictionary): apply POS-specific pronunciation - If in CMU dictionary: use first pronunciation variant - Otherwise: predict pronunciation using trained GRU model 4. **Concatenation**: Joins phoneme sequences with spaces between words ### Example ```python g2p = G2p() # Number expansion result = g2p("I have $250 in my pocket.") print(result) # ['AY1', ' ', 'HH', 'AE1', 'V', ' ', 'T', 'UW1', ' ', 'HH', 'AH1', 'N', 'D', 'R', 'AH0', 'D', ' ', 'F', 'IH1', 'F', 'T', 'IY0', ' ', 'D', 'AA1', 'L', 'ER0', 'Z', ' ', ...] # Abbreviation expansion result = g2p("popular pets, e.g. cats and dogs") print(result) # ['P', 'AA1', 'P', 'Y', 'AH0', 'L', 'ER0', ' ', 'P', 'EH1', 'T', 'S', ' ', ',', ' ', 'F', 'AO1', 'R', ' ', ...] # Homograph disambiguation result = g2p("I refuse to collect the refuse around here.") print(result) # First 'refuse' (verb): ['R', 'IH0', 'F', 'Y', 'UW1', 'Z', ...] # Second 'refuse' (noun): ['R', 'EH1', 'F', 'Y', 'UW2', 'Z', ...] # Out-of-vocabulary word prediction result = g2p("I'm an activationist.") print(result) # ['AY1', ' ', 'AH0', 'M', ' ', 'AE2', 'K', 'T', 'IH0', 'V', 'EY1', 'SH', 'AH0', 'N', 'IH0', 'S', 'T', ' ', '.'] ``` ``` -------------------------------- ### Handle empty or whitespace-only input Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Process various forms of empty or whitespace-only strings and check for empty results. ```python from g2p_en import G2p g2p = G2p() texts = ["", " ", "\n\t", "hello"] for text in texts: result = g2p(text) if not result: print(f"Empty result for input: {repr(text)}") else: print(f"Input: {repr(text)}, Result: {result}") ``` -------------------------------- ### Basic Grapheme-to-Phoneme Conversion Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/api-reference/g2p.md Performs basic Grapheme-to-Phoneme conversion on a given text. The result is a list of phonemes in ARPAbet format. ```python from g2p_en import G2p # Create converter instance g2p = G2p() # Basic usage result = g2p("Hello world") print(result) # ['HH', 'AH0', 'L', 'OW1', ' ', 'W', 'ER1', 'L', 'D'] ``` -------------------------------- ### G2p Output Type Definition Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/types.md The __call__() method of the G2p class returns a list of strings. Each string can be a phoneme, a space, or punctuation. ```python list[str] # Each element is either: # - A phoneme string (e.g., "AH0", "HH", "L") # - A space character " " # - Punctuation (".", ",", "?", "!") ``` -------------------------------- ### Helper Function: _expand_dollars Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Internal helper to expand dollar amounts in a number string. ```python from g2p_en.expand import _expand_dollars expanded_dollars = _expand_dollars('1234') ``` -------------------------------- ### G2P Text Conversion Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/configuration.md Call the G2P model with the text to convert. The only accepted parameter is the input text string. ```python result = g2p(text) # Only parameter: text (str) ``` -------------------------------- ### Helper Function: _expand_decimal_point Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Internal helper to expand decimal points in a number string. ```python from g2p_en.expand import _expand_decimal_point expanded_decimal = _expand_decimal_point('12.34') ``` -------------------------------- ### Public Types in G2P Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/DOCUMENTATION_INDEX.md Summary of the public type definitions used within the G2P library. ```python ARPAbet Phoneme Set (74 phonemes) Grapheme Set (29 characters) Homograph Dictionary Structure CMU Dictionary Format Neural Network Weight Tensors ``` -------------------------------- ### Monitor G2P memory footprint Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/errors.md Check the memory usage of the G2P library instance to prevent MemoryErrors. ```python import psutil import os process = psutil.Process(os.getpid()) mem_before = process.memory_info().rss / 1024 / 1024 g2p = G2p() mem_after = process.memory_info().rss / 1024 / 1024 print(f"G2p memory footprint: {mem_after - mem_before:.1f} MB") ``` -------------------------------- ### Error Handling for NLTK Downloads Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/README.md Provides a robust error handling mechanism for cases where NLTK data (like taggers and dictionaries) might not be automatically downloaded. It includes a fallback to manually download required NLTK resources if the initial G2p instantiation fails due to missing data. ```python from g2p_en import G2p import nltk try: g2p = G2p() except LookupError: # Auto-download failed; manual fallback nltk.download('averaged_perceptron_tagger') nltk.download('cmudict') g2p = G2p() try: result = g2p("Some text") except (TypeError, ValueError) as e: print(f"Conversion error: {e}") ``` -------------------------------- ### G2p Class gru Method Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Internal GRU layer computation. ```python output = g2p.gru(input_data) ``` -------------------------------- ### normalize_numbers Function Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/MANIFEST.txt Reference for the normalize_numbers function, which handles the conversion of numerical strings into their expanded, spoken-word representations. Includes details on helper functions and conversion rules. ```APIDOC ## normalize_numbers Function ### Description Expands numerical strings into their spoken-word representations, handling various formats like decimals, currency, and ordinals. ### Method `normalize_numbers(number_string)` ### Parameters - **number_string** (string) - The numerical string to be expanded. ### Return Type - (string) - The expanded, spoken-word representation of the input number. ### Helper Functions - `_remove_commas(number_string)` - `_expand_decimal_point(number_string)` - `_expand_dollars(number_string)` - `_expand_ordinal(number_string)` - `_expand_number(number_string)` (includes special handling for years) ### Conversion Rules A matrix detailing the conversion rules is available in the source documentation. ### Example Conversions Example conversions are provided in the source documentation. ### Dependencies Dependencies for this function are listed in the source documentation. ``` -------------------------------- ### g2p_en.expand Public Functions Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/DOCUMENTATION_INDEX.md Utility functions for expanding text, particularly for numbers and special characters, used in text normalization. ```APIDOC ## Public Functions in `g2p_en.expand` ### `normalize_numbers(text: str)` - **Description**: Normalizes numbers within a text string. - **Parameters**: - `text` (str) - The input text containing numbers. - **Returns**: str - The text with numbers normalized. ### `_remove_commas(m: re.Match)` - **Description**: Internal helper function to remove commas from a regex match object. - **Parameters**: - `m` (re.Match) - The regex match object. - **Returns**: str - The matched string with commas removed. ### `_expand_decimal_point(m: re.Match)` - **Description**: Internal helper function to expand decimal points in a regex match. - **Parameters**: - `m` (re.Match) - The regex match object. - **Returns**: str - The expanded string. ### `_expand_dollars(m: re.Match)` - **Description**: Internal helper function to expand dollar amounts in a regex match. - **Parameters**: - `m` (re.Match) - The regex match object. - **Returns**: str - The expanded string representing the dollar amount. ### `_expand_ordinal(m: re.Match)` - **Description**: Internal helper function to expand ordinal numbers in a regex match. - **Parameters**: - `m` (re.Match) - The regex match object. - **Returns**: str - The expanded ordinal string. ### `_expand_number(m: re.Match)` - **Description**: Internal helper function to expand a general number in a regex match. - **Parameters**: - `m` (re.Match) - The regex match object. - **Returns**: str - The expanded number string. ``` -------------------------------- ### sigmoid(x) Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/api-reference/g2p.md Computes the element-wise sigmoid activation function for a given input array. This is a standard activation function used in neural networks. ```APIDOC ## sigmoid(x) ### Description Element-wise sigmoid activation function. Computes `1 / (1 + exp(-x))` for each element in the input array. ### Parameters #### Path Parameters - **x** (np.ndarray) - Required - Input array of any shape ### Returns `np.ndarray` of same shape as input — Element-wise sigmoid: `1 / (1 + exp(-x))` ``` -------------------------------- ### Expand Decimal Point Source: https://github.com/kyubyong/g2p/blob/master/_autodocs/api-reference/expand.md Internal helper function to expand decimal points in numbers to the word 'point'. Used via regex substitution. ```python def _expand_decimal_point(m: re.Match) -> str: # ... implementation details ... pass # Example Usage (conceptual): # Input: 3.14 -> Output: 3 point 14 ```