### Install Project Dependencies

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/apps/languages/DEVELOPER_GUIDE.md

Installs all necessary Python packages listed in the requirements.txt file. This is a prerequisite for running the project.

```shell
pip install -r requirements.txt
```

--------------------------------

### Install GCC and G++ Compilers

Source: https://github.com/undertheseanlp/underthesea/wiki/Development-Guide

Installs the GNU Compiler Collection (GCC) and its C++ counterpart (G++) using apt-get. These are essential for compiling C/C++ code, which may be a dependency for the project.

```bash
apt-get update
apt-get install gcc g++
```

--------------------------------

### Run Development Server

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/apps/languages/DEVELOPER_GUIDE.md

Starts the Django development server, making the application accessible via a local URL. This command is used for local development and testing.

```python
python manage.py runserver
```

--------------------------------

### Setup Nginx Server Configuration

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/apps/service/SERVER_SETUP_TUTORIAL.md

This snippet demonstrates how to set up Nginx by copying the default configuration file to the appropriate directories and then reloading the Nginx service to apply the changes. Ensure you have the necessary permissions to modify Nginx configuration files.

```bash
cp conf/nginx/default /etc/nginx/sites-available/default
cp conf/nginx/default /etc/nginx/sites-enabled/default
nginx -s reload
```

--------------------------------

### Setup Environment for Rasa Chatbot

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/labs/chatbot/README.md

This code snippet outlines the necessary commands to set up a Python environment for running Rasa chatbots. It includes creating a conda environment, activating it, and installing specific versions of Rasa, Rasa-X, and other required Python packages.

```bash
conda create -n chatbot python=3.8
conda activate chatbot
pip install --upgrade pip==20.2    # more detail https://forum.rasa.com/t/pip-takes-long-time/39274/3
pip install rasa==2.8.2
pip install rasa-x==0.42.0 --extra-index-url https://pypi.rasa.com/simple
pip install sanic-jwt==1.6.0       # more detail https://forum.rasa.com/t/pip-install-rasa-x-not-working-local-mode-still-takes-to-long/48247/5
pip install questionary==1.5.1     # more detail https://forum.rasa.com/t/error-this-event-loop-is-already-running/24017/11
```

--------------------------------

### Setup Resources Pre-commit Hook (Bash)

Source: https://github.com/undertheseanlp/underthesea/wiki/Developer-Guides

Sets up a pre-commit hook for the Resources project to automatically build resources before each commit. This ensures that the resources are up-to-date. It involves editing the pre-commit file and changing its permissions.

```bash
echo 'Build resources'
python build.py
```

```bash
$ chmod u+x .git/hooks/pre-commit
```

--------------------------------

### Run Chatbot Actions

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/labs/chatbot/baggage_claim/README.md

Commands to run the chatbot's actions and train the model. Requires navigating to the example directory.

```bash
cd underthesea/examples/chatbot/baggage_claim
rasa run actions 
rasa train
rasa x 
```

--------------------------------

### Clone and Install Underthesea with uv

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/developer/contributing.md

This snippet shows how to clone the Underthesea repository and set up a development environment using uv. It installs the project in editable mode with development dependencies.

```bash
# Clone the repository
git clone https://github.com/undertheseanlp/underthesea.git
cd underthesea

# Create virtual environment with uv
uv venv
source .venv/bin/activate

# Install in development mode
uv pip install -e ".[dev]"
```

--------------------------------

### Install underthesea Agent Module

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/technical-reports/agents/index.md

Installs the underthesea library with agent capabilities and sets environment variables for API keys. Supports both OpenAI and Azure OpenAI.

```bash
pip install "underthesea[agent]"

# Set API key
export OPENAI_API_KEY="sk-..."
# Or for Azure OpenAI:
export AZURE_OPENAI_API_KEY="..."
export AZURE_OPENAI_ENDPOINT="https://xxx.openai.azure.com"
```

--------------------------------

### Setup Environment for Underthesea Service

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/apps/service/README.md

This snippet details the commands to set up the Python environment for the Underthesea service. It involves creating a Conda environment, activating it, installing dependencies from requirements.txt, and running the server.

```bash
conda create -n python=3.8 service
source activate service
pip install -r requirements.txt
python manage.py runserver 0.0.0.0:80
```

--------------------------------

### Install Anaconda 2 - 4.4.0

Source: https://github.com/undertheseanlp/underthesea/wiki/Development-Guide

Installs Anaconda Python 2 distribution version 4.4.0. This involves updating package lists, installing prerequisite packages, downloading the Anaconda installer, and setting up the PATH environment variable. It ensures a consistent Python environment for the project.

```bash
apt-get update --fix-missing && apt-get install -y wget bzip2 ca-certificates \
  libglib2.0-0 libxext6 libsm6 libxrender1 \
  git mercurial subversion
echo 'export PATH=/opt/conda/bin:$PATH' > /etc/profile.d/conda.sh && \
  wget --quiet https://repo.continuum.io/archive/Anaconda2-4.4.0-Linux-x86_64.sh -O ~/anaconda.sh && \
  /bin/bash ~/anaconda.sh -b -p /opt/conda && \
  rm ~/anaconda.sh
```

--------------------------------

### Test Chatbot Core

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/labs/chatbot/baggage_claim/README.md

Commands to validate and test the core conversational flows of the chatbot. Requires navigating to the example directory.

```bash
cd underthesea/examples/chatbot/baggage_claim
rasa data validate
rasa test core --stories tests
```

--------------------------------

### Install Underthesea NLP Toolkit

Source: https://context7.com/undertheseanlp/underthesea/llms.txt

Install the Underthesea library using pip. Basic installation includes core functionality, while optional dependencies like '[deep]', '[voice]', '[prompt]', '[langdetect]', and '[agent]' enable advanced features such as deep learning, text-to-speech, prompt-based classification, language detection, and AI agents.

```bash
# Basic installation
pip install underthesea

# Install with optional dependencies
pip install "underthesea[deep]"       # Deep learning support
pip install "underthesea[voice]"      # Text-to-Speech support
pip install "underthesea[prompt]"     # OpenAI-based classification
pip install "underthesea[langdetect]" # Language detection
pip install "underthesea[agent]"      # Conversational AI agent
```

--------------------------------

### Install Underthesea Base Package

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/intro.md

Installs the core underthesea Python package. This is the fundamental step to begin using the library's NLP capabilities.

```bash
pip install underthesea
```

--------------------------------

### Basic Language Detection Examples (Python)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/lang-detect.md

Demonstrates basic usage of the `lang_detect` function for various languages including Vietnamese, English, Chinese, and Japanese. Ensure the `underthesea[langdetect]` package is installed.

```python
from underthesea import lang_detect

# Vietnamese
lang_detect("Cựu binh Mỹ trả nhật ký nhẹ lòng")
# 'vi'

# English
lang_detect("Hello, how are you today?")
# 'en'

# Chinese
lang_detect("你好，今天怎么样？")
# 'zh'

# Japanese
lang_detect("こんにちは、元気ですか？")
# 'ja'
```

--------------------------------

### Install underthesea with deep learning dependencies

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/translate.md

Installs the underthesea library with the necessary deep learning dependencies required for translation functionality. This command uses pip to manage the installation.

```bash
pip install "underthesea[deep]"
```

--------------------------------

### Publish to PyPI Manually

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/developer/releasing.md

Steps to manually publish the package to PyPI using build and twine, after installing them.

```bash
uv pip install build twine
python -m build
twine upload dist/*
```

--------------------------------

### Install Conversational AI Agent Dependencies

Source: https://github.com/undertheseanlp/underthesea/blob/main/README.md

Installs the necessary Python packages for the conversational AI agent and sets environment variables for API keys. Supports both OpenAI and Azure OpenAI.

```bash
pip install "underthesea[agent]"
export OPENAI_API_KEY=your_api_key
# Or for Azure OpenAI:
# export AZURE_OPENAI_API_KEY=your_key
# export AZURE_OPENAI_ENDPOINT=https://xxx.openai.azure.com
```

--------------------------------

### Install Underthesea with Prompt-Based Classification Support

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/intro.md

Installs underthesea with the 'prompt' extra, enabling OpenAI-based classification features. This allows for text classification using prompt engineering with OpenAI models.

```bash
pip install "underthesea[prompt]"
```

--------------------------------

### Run Rasa X for Chatbot Development

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/labs/chatbot/README.md

This command demonstrates how to activate the previously set up conda environment and navigate to a specific chatbot example directory before running the Rasa X interactive development tool. Rasa X is used for building, training, and testing Rasa chatbots.

```bash
conda activate chatbot 
cd underthesea/examples/chatbot/topup
rasa x
```

--------------------------------

### Integration with TextClassifier

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/core/text-preprocessor.md

Example demonstrating how TextPreprocessor is used in conjunction with TextClassifier and how the preprocessor configuration is saved and restored with the model.

```APIDOC
## Integration with TextClassifier

### Usage Example

```python
from underthesea_core import TextClassifier, TextPreprocessor

pp = TextPreprocessor()
clf = TextClassifier(preprocessor=pp)
clf.fit(texts, labels)

# Preprocessor is saved together with the model
clf.save("model.bin")
clf = TextClassifier.load("model.bin")  # preprocessor is restored
```

### Description

When a `TextPreprocessor` instance is provided to the `TextClassifier` during initialization, its configuration is automatically serialized and saved with the `TextClassifier` model. Upon loading the model, the associated `TextPreprocessor` instance is restored, ensuring that the preprocessing steps remain consistent with the model's training data.
```

--------------------------------

### Run Underthesea Service

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/apps/service/README.md

This snippet provides the command to execute the run.sh script, which is used to start the Underthesea service.

```bash
bash run.sh
```

--------------------------------

### Feature Comparison

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/technical-reports/agents/index.md

Compares underthesea with other frameworks based on features like setup complexity, tool definition, and multi-agent support.

```APIDOC
## Comparison with Other Frameworks

### Feature Comparison

| Feature             | underthesea | LangChain | OpenAI SDK | CrewAI | Phidata |
|---------------------|-------------|-----------|------------|--------|---------|
| Setup Complexity    | Low         | Medium    | Low        | Medium | Low     |
| Tool Definition     | Function + decorator | Class-based | Function   | Class-based | Toolkit |
| Multi-agent         | Manual      | LangGraph | Built-in   | Built-in | Built-in |
| Memory              | In-memory   | Multiple  | Session    | Built-in | Built-in |
| MCP Support         | No          | Yes       | Yes        | No     | Yes     |
| Providers           | OpenAI, Azure | 100+      | OpenAI     | OpenAI+ | 50+     |
```

--------------------------------

### Agent Initialization with Predefined Tool Collections (Python)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/technical-reports/agents/index.md

Shows how to initialize an Agent using predefined tool collections like `default_tools`, `core_tools`, and `web_tools`. This allows for quick setup of agents with specific capabilities.

```python
from underthesea.agent import Agent, default_tools, core_tools, web_tools

# All tools
full_agent = Agent(name="full", tools=default_tools)

# Safe agent (no system access)
safe_agent = Agent(name="safe", tools=core_tools)

# Web-enabled agent
web_agent = Agent(name="web", tools=core_tools + web_tools)

# Use agent
response = full_agent("What time is it and what's the weather in Hanoi?")
```

--------------------------------

### Performance Test Example (Python)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/blog/2026-02-02-rewrite-rust-crf-model.md

Example of using underthesea functions like word_tokenize after installation to observe performance improvements. This snippet highlights the speedup achieved for word tokenization.

```python
from underthesea import word_tokenize, pos_tag, ner, chunk

word_tokenize("Việt Nam")  # 20% faster!
```

--------------------------------

### Using Default Tools

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/technical-reports/agents/index.md

Shows how to initialize agents with different sets of predefined tools.

```APIDOC
## Using Default Tools

### Description
This section demonstrates initializing agents with various combinations of default tool collections.

### Usage Examples

```python
from underthesea.agent import Agent, default_tools, core_tools, web_tools

# All tools
full_agent = Agent(name="full", tools=default_tools)

# Safe agent (no system access)
safe_agent = Agent(name="safe", tools=core_tools)

# Web-enabled agent
web_agent = Agent(name="web", tools=core_tools + web_tools)

# Use agent
response = full_agent("What time is it and what's the weather in Hanoi?")
```
```

--------------------------------

### Setup Project Environment and Load Data

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/labs/notebooks/COL/Challenge.ipynb

Initializes the Python environment by loading necessary extensions and libraries. It then configures the project path and adds it to the system's path for module imports. Finally, it defines and constructs paths to dataset files.

```python
%load_ext autoreload
%autoreload 2

# add project folder
import os
from os.path import dirname, join
PROJECT_FOLDER = dirname(dirname(os.getcwd()))
os.sys.path.append(PROJECT_FOLDER)

# add dependencies
from underthesea.utils.col_analyzer import UDAnalyzer, computeIDF
from underthesea.utils.col_script import UDDataset
from IPython.display import display, display_png
from wordcloud import WordCloud
from PIL import Image
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# init folder
DATASETS_FOLDER = join(PROJECT_FOLDER, "datasets")
COL_FOLDER = join(DATASETS_FOLDER, "UD_Vietnamese-COL")
raw_file = join(COL_FOLDER, "corpus", "raw", "202108.txt")
```

--------------------------------

### Initialize and Migrate Database

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/apps/languages/DEVELOPER_GUIDE.md

Commands to manage database schema and apply migrations for the Django project. It also includes creating an initial superuser for administrative access.

```python
python manage.py makemigrations
python manage.py migrate
python manage.py createsuperuser
```

--------------------------------

### Optional Translation Dependency Check (Python)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/developer/architecture.md

Checks for the optional 'transformers' library required for translation functionality. Raises an ImportError if the library is not found, guiding the user to install it with the 'deep' extra.

```python
def translate(text):
    try:
        from transformers import AutoModel
    except ImportError:
        raise ImportError(
            "Translation requires deep learning dependencies. "
            "Install with: pip install 'underthesea[deep]'"
        )
    # ... translation logic
```

--------------------------------

### Agent with Custom System Prompt

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/agent.md

Shows how to provide a custom system prompt to the agent to guide its behavior. In this example, the agent is instructed to act as a Vietnamese cuisine assistant.

```python
from underthesea import agent

response = agent(
    "Chào bạn",
    system_prompt="Bạn là trợ lý chuyên về ẩm thực Việt Nam."
)
```

--------------------------------

### Run Chatbot Actions and Training

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/labs/chatbot/topup/README.md

This snippet shows the command-line interface commands to run the chatbot actions, train the Rasa model, and launch the Rasa X interface. It assumes you are in the correct directory within the underthesea project.

```bash
cd underthesea/examples/chatbot/topup 
rasa run actions 
rasa train
rasa x
```

--------------------------------

### Simple Agent (Singleton)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/technical-reports/agents/index.md

Demonstrates basic usage of the underthesea agent with and without a system prompt, including history management.

```APIDOC
## Simple Agent (Singleton)

### Description
This section shows how to use the basic agent functionality for simple conversations and history management.

### Usage Examples

```python
from underthesea import agent

# Basic conversation
response = agent("Xin chào!")
print(response)

# With custom system prompt
response = agent("NLP là gì?", system_prompt="Bạn là chuyên gia NLP")

# Check history
print(agent.history)

# Reset conversation
agent.reset()
```
```

--------------------------------

### Word Tokenization Format Example

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/index.md

Demonstrates the usage of the 'format' parameter in the `word_tokenize` function. It shows how to get tokenized output as a list of strings (default) or a single string with tokens joined by underscores.

```python
word_tokenize("Việt Nam", format=None)   # ['Việt Nam']
word_tokenize("Việt Nam", format="text") # 'Việt_Nam'
```

--------------------------------

### Tool Class Usage and Constructor (Python)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/agent.md

Shows how to wrap Python functions as agent tools using the Tool class. It includes an example of defining a function, creating a Tool instance with optional name and description, converting it to OpenAI format, and executing it directly.

```python
from underthesea.agent import Tool

def search(query: str, limit: int = 10) -> list:
    """Search for items matching the query."""
    return [{"title": f"Result for {query}"}]

tool = Tool(
    func=search,
    name="web_search",           # Optional, defaults to function name
    description="Search the web" # Optional, defaults to docstring
)

# Convert to OpenAI format
openai_format = tool.to_openai_tool()

# Execute directly
result = tool(query="python", limit=5)
```

--------------------------------

### Create and Activate Conda Environment for Underthesea

Source: https://github.com/undertheseanlp/underthesea/wiki/Development-Guide

Creates a new conda environment named 'underthesea' with Python 3.4, activates it, and installs 'tox' and 'virtualenv'. This isolates project dependencies and provides a clean environment for development and testing.

```bash
conda create -n underthesea python=3.4
source activate underthesea
pip install tox
conda install virtualenv
```

--------------------------------

### Start Frontend Development Server (Yarn/React)

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/apps/languages/README.md

Command to navigate to the frontend directory and initiate the React development server using Yarn. This command is necessary for running the client-side interface of the Languages application.

```bash
cd frontend && yarn start
```

--------------------------------

### Setup Underthesea Pre-commit Hooks (Bash)

Source: https://github.com/undertheseanlp/underthesea/wiki/Developer-Guides

Configures pre-commit hooks for the Underthesea project to run flake8, black, and pylint before each commit. This ensures code quality and adherence to standards. It involves editing the pre-commit file and changing its permissions.

```bash
echo 'Run Flake8'
flake8 --max-complexity 10 --ignore E501,W504,W605 .
echo 'Run Black'
black underthesea/utils/vietnamese_ipa.py
black underthesea/utils/vietnamese_ipa_rules.py
black --check underthesea/utils/vietnamese_ipa.py
black --check underthesea/utils/vietnamese_ipa_rules.py
echo 'Run pylint'
pylint ./underthesea/utils/vietnamese_ipa.py --fail-under 9
```

```bash
$ chmod u+x .git/hooks/pre-commit
```

--------------------------------

### Example Using Multiple Tools with an Agent (Python)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/agent.md

Demonstrates how to create an agent that can utilize multiple tools. It defines several functions (get_weather, search_news, translate_text), wraps them in Tool instances, and initializes an Agent with these tools and a specific instruction set for a Vietnamese assistant.

```python
from underthesea.agent import Agent, Tool

def get_weather(location: str) -> dict:
    """Get current weather for a location."""
    return {"location": location, "temp": 25}

def search_news(query: str) -> str:
    """Search Vietnamese news articles."""
    return f"Found articles about: {query}"

def translate_text(text: str, target_lang: str = "en") -> str:
    """Translate Vietnamese text."""
    return f"Translated: {text}"

agent = Agent(
    name="multi_tool_agent",
    tools=[
        Tool(get_weather),
        Tool(search_news),
        Tool(translate_text, description="Translate Vietnamese to other languages"),
    ],
    instruction="You are a helpful Vietnamese assistant with access to weather, news, and translation tools."
)

# The agent decides which tool to use based on the query
response = agent("Thời tiết ở Đà Nẵng thế nào?")  # Uses get_weather
response = agent("Tin tức về AI hôm nay")          # Uses search_news
response = agent("Dịch 'Xin chào' sang tiếng Anh") # Uses translate_text
```

--------------------------------

### Detect Language of Text

Source: https://context7.com/undertheseanlp/underthesea/llms.txt

Employs the underthesea lang_detect function to identify the language of a given text, supporting 176 languages. Requires 'underthesea[langdetect]' installation. Includes examples for detecting Vietnamese, English, Chinese, and French, and filtering documents by language.

```python
# Requires: pip install "underthesea[langdetect]"
from underthesea import lang_detect

# Detect Vietnamese
lang_detect("Cựu binh Mỹ trả nhật ký nhẹ lòng khi thấy cuộc sống hòa bình tại Việt Nam")
# 'vi'

# Detect other languages
lang_detect("Hello, how are you today?")  # 'en'
lang_detect("你好，今天怎么样？")           # 'zh'
lang_detect("Bonjour le monde")            # 'fr'

# Filter documents by language
documents = [
    "Việt Nam là một đất nước xinh đẹp",
    "This is an English sentence",
    "Hôm nay trời đẹp quá"
]
vietnamese_docs = [doc for doc in documents if lang_detect(doc) == 'vi']
print(vietnamese_docs)
# ['Việt Nam là một đất nước xinh đẹp', 'Hôm nay trời đẹp quá']
```

--------------------------------

### Custom Agent with User-Defined Tools (Python)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/technical-reports/agents/index.md

Illustrates how to create a custom agent with specific tools. This example defines two tools, `search_database` and `send_email`, and integrates them into an agent with a custom instruction.

```python
from underthesea.agent import Agent, Tool

def search_database(query: str) -> list:
    """Search internal database."""
    return [{"id": 1, "title": f"Result for {query}"}]

def send_email(to: str, subject: str, body: str) -> dict:
    """Send an email."""
    return {"status": "sent", "to": to}

agent = Agent(
    name="assistant",
    tools=[
        Tool(search_database),
        Tool(send_email, description="Send email to user"),
    ],
    instruction="You are a helpful office assistant."
)

response = agent("Find documents about AI and email the results to john@example.com")
```

--------------------------------

### Multi-language Detection Examples (Python)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/technical-reports/language-identification.md

Provides examples of using the `lang_detect` function to identify various languages, including Vietnamese, English, Japanese, Chinese, Korean, and Thai. Each example shows the input text and the expected language code output.

```python
from underthesea import lang_detect

# Vietnamese
lang_detect("Xin chào, tôi là người Việt Nam")  # vi

# English
lang_detect("Hello, how are you?")  # en

# Japanese
lang_detect("こんにちは世界")  # ja

# Chinese
lang_detect("你好世界")  # zh

# Korean
lang_detect("안녕하세요")  # ko

# Thai
lang_detect("สวัสดีครับ")  # th
```

--------------------------------

### LLM Class Initialization and Chat

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/agent.md

Demonstrates using the LLM class directly for more control over AI interactions. It shows initialization with a specific model and performing a chat with a system prompt and user message.

```python
from underthesea.agent import LLM

# Initialize
llm = LLM(model="gpt-4")

# Chat with custom messages
messages = [
    {"role": "system", "content": "You are a Vietnamese language expert."},
    {"role": "user", "content": "Explain word segmentation in Vietnamese."}
]
response = llm.chat(messages)
print(response)
```

--------------------------------

### Install underthesea-core

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/core/index.md

Installs the underthesea-core Python package using pip. This package provides high-performance Rust extensions for underthesea.

```bash
pip install underthesea-core
```

--------------------------------

### Clone Repository and Run Tests with Tox

Source: https://github.com/undertheseanlp/underthesea/wiki/Development-Guide

Clones the underthesea project from GitHub and executes the tests using 'tox'. Tox is a tool for automating testing in Python, ensuring the project functions correctly across different environments.

```bash
git clone https://github.com/magizbox/underthesea
tox
```

--------------------------------

### Install Underthesea with Text-to-Speech Support

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/intro.md

Installs underthesea with the 'voice' extra, providing Text-to-Speech capabilities. This allows for converting Vietnamese text into spoken audio.

```bash
pip install "underthesea[voice]"
```

--------------------------------

### Direct Tool Usage without LLM (Python)

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/technical-reports/agents/index.md

Demonstrates how to use individual tools directly without involving a language model. This is useful for specific, deterministic tasks like calculations or data retrieval.

```python
from underthesea.agent import calculator_tool, wikipedia_tool

# Use tools without LLM
result = calculator_tool(expression="sqrt(144) + 2**10")
print(result)  # {'expression': '...', 'result': 1036.0}

wiki = wikipedia_tool(query="Hà Nội", lang="vi")
print(wiki["summary"])
```

--------------------------------

### Run Tests with Tox

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/developer/releasing.md

Commands to execute linting and core tests using the Tox testing framework.

```bash
tox -e lint
tox -e core
```

--------------------------------

### Install underthesea Library

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/blog/2026-02-03-rust-text-classifier.md

Installs the underthesea library, version 9.2.9, which includes the banking intent classification model. This is a prerequisite for using the classification functionality.

```bash
pip install underthesea==9.2.9
```

--------------------------------

### TextPreprocessor Constructor and Usage

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/core/text-preprocessor.md

This section details how to initialize and use the TextPreprocessor class, including default settings and custom configurations.

```APIDOC
## TextPreprocessor

Configurable Vietnamese text preprocessing pipeline.

### Constructor

```python
TextPreprocessor(
    lowercase=True,
    unicode_normalize=True,
    remove_urls=True,
    normalize_repeated_chars=True,
    normalize_punctuation=True,
    teencode=None,
    negation_words=None,
    negation_window=2,
    use_defaults=True,
)
```

### Parameters

#### Constructor Parameters

- **lowercase** (bool) - Optional - Convert text to lowercase. Default: `True`
- **unicode_normalize** (bool) - Optional - Apply Unicode NFC normalization. Default: `True`
- **remove_urls** (bool) - Optional - Remove URLs (http/https/www). Default: `True`
- **normalize_repeated_chars** (bool) - Optional - Reduce 3+ repeated chars to 2. Default: `True`
- **normalize_punctuation** (bool) - Optional - Reduce repeated punctuation. Default: `True`
- **teencode** (dict | None) - Optional - Custom teencode dictionary. With `use_defaults=True`, defaults to built-in Vietnamese teencode. Default: `None`
- **negation_words** (list[str] | None) - Optional - Custom negation words. With `use_defaults=True`, defaults to built-in Vietnamese negation words. Default: `None`
- **negation_window** (int) - Optional - Number of words after negation word to mark with `NEG_` prefix. Default: `2`
- **use_defaults** (bool) - Optional - When `True`, use Vietnamese defaults for teencode/negation if not provided. When `False`, `None` means disabled. Default: `True`

### Usage Examples

#### Default Vietnamese preprocessing

```python
from underthesea_core import TextPreprocessor

pp = TextPreprocessor()
result = pp.transform("Sản phẩm ko đẹp lắm!!!")
# Expected output: "sản phẩm không NEG_đẹp NEG_lắm!"
```

#### Batch processing

```python
results = pp.transform_batch(["Ko đẹp", "SP tốt lắm!!!"])
# Expected output: ["không NEG_đẹp", "sản phẩm tốt lắm!"]
```

#### Custom teencode dictionary

```python
pp = TextPreprocessor(teencode={"ko": "không", "dc": "được"})
```

#### Custom negation words and window

```python
pp = TextPreprocessor(
    negation_words=["không", "chưa", "chẳng"],
    negation_window=3,
)
```

#### Disable specific steps

```python
pp = TextPreprocessor(lowercase=False, remove_urls=False)
```

#### Disable teencode and negation entirely

```python
pp = TextPreprocessor(teencode=None, negation_words=None, use_defaults=False)
```
```

--------------------------------

### Install underthesea with agent support

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/api/agent.md

Installs the underthesea library with the necessary dependencies for the agent functionality. This command uses pip to manage Python packages.

```bash
pip install "underthesea[agent]"
```

--------------------------------

### Run Backend Server (Python/Django)

Source: https://github.com/undertheseanlp/underthesea/blob/main/extensions/apps/languages/README.md

Command to navigate to the backend directory and start the Django development server. This is essential for running the server-side logic of the Languages application.

```bash
cd backend && python manage.py runserver
```

--------------------------------

### Run Specific Unit Tests with unittest

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/versioned_docs/version-9.2.11/developer/contributing.md

Examples of how to run specific test suites for Underthesea modules using the `unittest` discovery mechanism. These commands are executed within the project's virtual environment.

```bash
# Word tokenization tests
uv run python -m unittest discover tests.pipeline.word_tokenize

# POS tagging tests
uv run python -m unittest discover tests.pipeline.pos_tag

# NER tests
uv run python -m unittest tests.pipeline.ner.test_ner

# Classification tests
uv run python -m unittest tests.pipeline.classification.test_bank

# Translation tests
uv run python -m unittest discover tests.pipeline.translate
```

--------------------------------

### Install underthesea Python Package

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/blog/2026-02-08-rewrite-fasttext-in-rust.md

Installs the underthesea Python package, which includes Rust FastText inference capabilities. This is the primary way to integrate the library into Python projects.

```bash
pip install underthesea>=9.2.9
```

--------------------------------

### Agent with Default Tools (Python)

Source: https://context7.com/undertheseanlp/underthesea/llms.txt

Shows how to initialize an Underthesea agent using a comprehensive set of default tools. These tools cover common functionalities like calculators, date/time, web search, and file operations.

```python
from underthesea.agent import Agent, default_tools, core_tools, web_tools

agent_with_tools = Agent(
    name="assistant",
    tools=default_tools,  # 12 built-in tools
)
agent_with_tools("What time is it?")      # Uses current_datetime_tool
agent_with_tools("Calculate sqrt(144)")    # Uses calculator_tool
agent_with_tools("Search for Python")      # Uses web_search_tool
```

--------------------------------

### Install underthesea Package

Source: https://github.com/undertheseanlp/underthesea/blob/main/docs/blog/2026-02-02-rewrite-rust-crf-model.md

Command to install a specific version of the underthesea Python package using pip. This command is used to try out the latest version with performance improvements.

```bash
pip install underthesea==9.2.5
```