Vietnamese Text Normalizer

https://pypi.org/project/vietnormalizer/
Admin
A Python library for normalizing Vietnamese text, designed for Text-to-Speech (TTS) and Natural...
Tokens:4,764
Snippets:73
Trust Score:10
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install vietnormalizer with pip

Source: https://pypi.org/project/vietnormalizer

Install the vietnormalizer library using pip.

```bash
pip install vietnormalizer

```

--------------------------------

### Install vietnormalizer from source

Source: https://pypi.org/project/vietnormalizer

Install the vietnormalizer library from its source code repository.

```bash
git clone https://github.com/nghimestudio/vietnormalizer.git
cd vietnormalizer
pip install -e .

```

--------------------------------

### Normalize currency amounts

Source: https://pypi.org/project/vietnormalizer/0.2.3

Converts currency amounts, including those with thousand separators, into Vietnamese words. This example shows the conversion of '50.000đ' to 'năm mươi nghìn đồng'.

```python
# Currency with thousand separators
normalizer.normalize("Giá 50.000đ cho mỗi người")
# → "giá năm mươi nghìn đồng cho mỗi người"

```

--------------------------------

### Normalize date ranges

Source: https://pypi.org/project/vietnormalizer/0.2.3

Converts date ranges into their Vietnamese word representation. This example shows the normalization of '25-26/12/2023' to 'ngày hai mươi lăm đến hai mươi sáu tháng mười hai năm hai nghìn không trăm hai mươi ba'.

```python
# Date ranges
normalizer.normalize("ngày 25-26/12/2023")
# → "ngày hai mươi lăm đến hai mươi sáu tháng mười hai năm hai nghìn không trăm hai mươi ba"

```

--------------------------------

### Configure Custom Dictionaries

Source: https://pypi.org/project/vietnormalizer

Specify paths to custom CSV files for acronyms and non-Vietnamese words, or provide a directory containing both. Dictionaries can be reloaded at runtime.

```python
normalizer = VietnameseNormalizer(
    acronyms_path="path/to/acronyms.csv",
    non_vietnamese_words_path="path/to/words.csv"
)
```

```python
# Or specify a directory containing both files
normalizer = VietnameseNormalizer(data_dir="path/to/data/")
```

```python
# Reload dictionaries at runtime
normalizer.reload_dictionaries(acronyms_path="path/to/updated.csv")
```

--------------------------------

### Directly Use VietnameseTextProcessor

Source: https://pypi.org/project/vietnormalizer

Instantiate VietnameseTextProcessor to directly convert numbers to words or process Vietnamese text, handling numbers, dates, times, and units without dictionary replacements.

```python
from vietnormalizer import VietnameseTextProcessor

processor = VietnameseTextProcessor()

# Convert numbers
processor.number_to_words("123")  # "một trăm hai mươi ba"

# Process text (numbers, dates, times, units - no dictionary replacements)
processor.process_vietnamese_text("Giá 50.000đ lúc 15h30")
```

--------------------------------

### Normalize year ranges, ordinals, and percentages

Source: https://pypi.org/project/vietnormalizer

Convert year ranges, ordinal numbers (like 'thứ 2'), and percentages into Vietnamese words. Requires importing VietnameseNormalizer.

```python
from vietnormalizer import VietnameseNormalizer

normalizer = VietnameseNormalizer()

# Year ranges, ordinals, percentages
normalizer.normalize("1873-1907, thứ 2, tăng 6,5%")
# → "một nghìn tám trăm bảy mươi ba đến một nghìn chín trăm lẻ bảy, thứ hai, tăng sáu phẩy năm phần trăm"

```

--------------------------------

### Normalize currency amounts

Source: https://pypi.org/project/vietnormalizer

Convert currency amounts, including VND with thousand separators, into Vietnamese words. Requires importing VietnameseNormalizer.

```python
from vietnormalizer import VietnameseNormalizer

normalizer = VietnameseNormalizer()

# Currency with thousand separators
normalizer.normalize("Giá 50.000đ cho mỗi người")
# → "giá năm mươi nghìn đồng cho mỗi người"

```

--------------------------------

### Rule-based transliteration for unknown words

Source: https://pypi.org/project/vietnormalizer

Transliterate words not found in the dictionary into Vietnamese phonetics using rule-based logic. Requires importing VietnameseNormalizer.

```python
from vietnormalizer import VietnameseNormalizer

normalizer = VietnameseNormalizer()

# Rule-based transliteration for words NOT in dictionary
normalizer.normalize("database server configuration")
# → "đa-ta-bê xơ-vơ con-phi-gu-raân"

```

--------------------------------

### Normalize year ranges, ordinals, and percentages

Source: https://pypi.org/project/vietnormalizer/0.2.3

Demonstrates the normalization of year ranges, ordinal numbers, and percentages. This snippet covers formats like '1873-1907', 'thứ 2', and '6,5%'.

```python
# Year ranges, ordinals, percentages
normalizer.normalize("1873-1907, thứ 2, tăng 6,5%")
# → "một nghìn tám trăm bảy mươi ba đến một nghìn chín trăm lẻ bảy, thứ hai, tăng sáu phẩy năm phần trăm"

```

--------------------------------

### CSV Format for Acronyms

Source: https://pypi.org/project/vietnormalizer

Define custom acronyms and their transliterations in a CSV file with 'acronym' and 'transliteration' columns.

```csv
acronym,transliteration
NASA,na-sa
GDP,tổng sản phẩm quốc nội
AI,trí tuệ nhân tạo
```

--------------------------------

### Normalize dates, numbers, and times

Source: https://pypi.org/project/vietnormalizer

Convert dates, numbers, and times into Vietnamese words. Requires importing VietnameseNormalizer.

```python
from vietnormalizer import VietnameseNormalizer

normalizer = VietnameseNormalizer()

# Numbers, dates, and times
normalizer.normalize("Hôm nay là 25/12/2023, lúc 14:30")
# → "hôm nay là ngày hai mươi lăm tháng mười hai năm hai nghìn không trăm hai mươi ba, lúc mười bốn giờ ba mươi phút"

```

--------------------------------

### Expand acronyms

Source: https://pypi.org/project/vietnormalizer/0.2.3

Expands common acronyms into their full Vietnamese phrases using a built-in dictionary. This improves readability by replacing abbreviations with their meanings.

```python
# Acronym expansion (from built-in dictionary)
normalizer.normalize("Tôi xem TV và dùng AI hàng ngày")
# → "tôi xem ti vi và dùng trí tuệ nhân tạo hàng ngày"

```

--------------------------------

### Normalize acronyms

Source: https://pypi.org/project/vietnormalizer

Expand acronyms into their full Vietnamese forms using the built-in dictionary. Requires importing VietnameseNormalizer.

```python
from vietnormalizer import VietnameseNormalizer

normalizer = VietnameseNormalizer()

# Acronym expansion (from built-in dictionary)
normalizer.normalize("Tôi xem TV và dùng AI hàng ngày")
# → "tôi xem ti vi và dùng trí tuệ nhân tạo hàng ngày"

```

--------------------------------

### Normalize percentage ranges

Source: https://pypi.org/project/vietnormalizer

Convert percentage ranges (e.g., '3-5%') into Vietnamese words. Requires importing VietnameseNormalizer.

```python
from vietnormalizer import VietnameseNormalizer

normalizer = VietnameseNormalizer()

# Percentage ranges
normalizer.normalize("3-5% dân số")
# → "ba đến năm phần trăm dân số"

```

--------------------------------

### Normalize measurement units

Source: https://pypi.org/project/vietnormalizer

Convert measurement units like km/h and m2 into Vietnamese words. Requires importing VietnameseNormalizer.

```python
from vietnormalizer import VietnameseNormalizer

normalizer = VietnameseNormalizer()

# Measurement units
normalizer.normalize("Tốc độ 120km/h, diện tích 500m2")
# → "tốc độ một trăm hai mươi ki-lô-mét trên giờ, diện tích năm trăm mét vuông"

```

--------------------------------

### Normalize non-Vietnamese words

Source: https://pypi.org/project/vietnormalizer

Replace non-Vietnamese words with their Vietnamese equivalents using the built-in dictionary. Requires importing VietnameseNormalizer.

```python
from vietnormalizer import VietnameseNormalizer

normalizer = VietnameseNormalizer()

# Non-Vietnamese word replacement (from built-in dictionary)
normalizer.normalize("Hello container from Singapore")
# → "hê-lô công-tê-nơ phờ-rôm xin-ga-po"

```

--------------------------------

### Transliterate unknown words

Source: https://pypi.org/project/vietnormalizer/0.2.3

Applies rule-based transliteration to words not found in the dictionary, converting them into Vietnamese phonetics. This ensures that even unfamiliar words are rendered in a Vietnamese-like manner.

```python
# Rule-based transliteration for words NOT in dictionary
normalizer.normalize("database server configuration")
# → "đa-ta-bê xơ-vơ con-phi-gu-raân"

```

--------------------------------

### Replace non-Vietnamese words

Source: https://pypi.org/project/vietnormalizer/0.2.3

Replaces common non-Vietnamese words with their Vietnamese equivalents using a built-in dictionary. This is useful for standardizing text that includes foreign terms.

```python
# Non-Vietnamese word replacement (from built-in dictionary)
normalizer.normalize("Hello container from Singapore")
# → "hê-lô công-tê-nơ phờ-rôm xin-ga-po"

```

--------------------------------

### Normalize date ranges

Source: https://pypi.org/project/vietnormalizer

Convert date ranges (e.g., '25-26/12') into Vietnamese words. Requires importing VietnameseNormalizer.

```python
from vietnormalizer import VietnameseNormalizer

normalizer = VietnameseNormalizer()

# Date ranges
normalizer.normalize("ngày 25-26/12/2023")
# → "ngày hai mươi lăm đến hai mươi sáu tháng mười hai năm hai nghìn không trăm hai mươi ba"

```

--------------------------------

### Disable Preprocessing in Normalization

Source: https://pypi.org/project/vietnormalizer

When calling the normalize function, set `enable_preprocessing=False` to skip initial preprocessing steps like number and date conversion, focusing only on dictionary replacements and transliteration.

```python
# Only apply dictionary replacements and transliteration, skip number/date conversion
normalizer.normalize(text, enable_preprocessing=False)
```

--------------------------------

### CSV Format for Non-Vietnamese Words

Source: https://pypi.org/project/vietnormalizer

Define custom transliterations for non-Vietnamese words in a CSV file with 'original' and 'transliteration' columns.

```csv
original,transliteration
container,công-tê-nơ
singapore,xin-ga-po
server,xơ-vơ
```

--------------------------------

### Direct Word Transliteration

Source: https://pypi.org/project/vietnormalizer

Perform direct transliteration of words. `transliterate_word` checks if the word is Vietnamese first, while `english_to_vietnamese` always transliterates.

```python
from vietnormalizer import transliterate_word, english_to_vietnamese

transliterate_word("database")   # "đa-ta-bâi" (checks if Vietnamese first)
transliterate_word("xin")        # "xin" (detected as Vietnamese, kept as-is)
english_to_vietnamese("computer") # "com-pu-tơ" (always transliterates)
```

--------------------------------

### Detect Vietnamese Words

Source: https://pypi.org/project/vietnormalizer

Use the `is_vietnamese_word` function to check if a string follows Vietnamese word structure and diacritic rules.

```python
from vietnormalizer import is_vietnamese_word

is_vietnamese_word("xin")      # True (valid Vietnamese structure)
is_vietnamese_word("chào")     # True (has Vietnamese diacritics)
is_vietnamese_word("database") # False (contains 'b' ending, invalid structure)
is_vietnamese_word("flow")     # False (contains 'f' and 'w')
```

--------------------------------

### Disable Transliteration

Source: https://pypi.org/project/vietnormalizer

Disable transliteration to only use CSV dictionary replacements. Words not found in the CSV will remain unchanged.

```python
normalizer = VietnameseNormalizer(enable_transliteration=False)
normalizer.normalize("machine learning algorithm")
# → "ma-sin li-nin algorithm"  (words in CSV replaced, others kept as-is)
```

```python
# Or override per-call
normalizer = VietnameseNormalizer(enable_transliteration=True)
normalizer.normalize("machine learning", enable_transliteration=False)
```
Tokens:
Raw