### Setup Environment

Source: https://github.com/ai4bharat/indictrans2/blob/main/huggingface_interface/README.md

Clones the repository, navigates to the directory, and installs dependencies for running HF compatible models.

```bash
git clone https://github.com/AI4Bharat/IndicTrans2
cd IndicTrans2/huggingface_interface
source install.sh
```

--------------------------------

### Installation

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Installation commands for HuggingFace models, standalone inference, and preprocessing/postprocessing.

```bash
# HuggingFace models
pip install transformers torch

# Standalone inference
pip install fairseq sentencepiece indicnlp sacremoses

# Preprocessing/postprocessing
pip install git+https://github.com/VarunGumma/IndicTransToolkit.git
```

--------------------------------

### IndicProcessor Workflow Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicProcessor.md

A step-by-step example demonstrating tokenization, generation, decoding, and postprocessing using IndicProcessor.

```python
inputs = tokenizer(
    batch,
    truncation=True,
    padding="longest",
    return_tensors="pt",
    return_attention_mask=True,
).to(device)

with torch.no_grad():
    generated_ids = model.generate(
        **inputs,
        max_length=256,
        num_beams=5,
        num_return_sequences=1,
    )

translated = tokenizer.batch_decode(
    generated_ids,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=True,
)

final_output = processor.postprocess_batch(
    translated,
    lang="hin_Deva",
)

for src, tgt in zip(sentences, final_output):
    print(f"{src} → {tgt}")
```

--------------------------------

### Generation Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransForConditionalGeneration.md

Example demonstrating how to use the `generate` method for translation, including tokenization, model loading, generation, and decoding.

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)

# Encode input
inputs = tokenizer("eng_Latn hin_Deva Hello, how are you?", return_tensors="pt")

# Generate translation
generated_ids = model.generate(
    inputs['input_ids'],
    max_length=256,
    num_beams=5,
    early_stopping=True,
)

# Decode output
translation = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
```

--------------------------------

### Installation

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicProcessor.md

Install the IndicTransToolkit library using pip.

```bash
pip install git+https://github.com/VarunGumma/IndicTransToolkit.git
```

--------------------------------

### Integration Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/inference_utilities.md

Typical preprocessing pipeline using utility functions.

```python
from inference.engine import (
    split_sentences,
    apply_lang_tags,
    truncate_long_sentences,
    Model
)
from inference.normalize_regex_inference import normalize

# 1. Split input paragraph into sentences
text = "Large paragraph here..."
sentences = split_sentences(text, "eng_Latn")

# 2. Normalize entities (Indic numerals, URLs, emails)
normalized_sents = []
entity_maps = []
for sent in sentences:
    norm_sent, entity_map = normalize(sent)
    normalized_sents.append(norm_sent)
    entity_maps.append(entity_map)

# 3. Truncate very long sentences
truncated_sents, truncated_maps = truncate_long_sentences(
    normalized_sents,
    entity_maps
)

# 4. Add language tags
tagged_sents = apply_lang_tags(truncated_sents, "eng_Latn", "hin_Deva")

# 5. Translate using model
model = Model("path/to/model")
translations = model.batch_translate(sentences, "eng_Latn", "hin_Deva")
```

--------------------------------

### Production Workflow Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/README.md

An example demonstrating the production workflow for IndicTrans2, including preprocessing, generation, and postprocessing of sentences.

```python
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from IndicTransToolkit.processor import IndicProcessor

processor = IndicProcessor(inference=True)
tokenizer = AutoTokenizer.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)

sentences = ["Contact support@example.com", "Visit https://example.com"]

# Preprocess
batch = processor.preprocess_batch(sentences, "eng_Latn", "hin_Deva")
inputs = tokenizer(batch, return_tensors="pt", padding=True)

# Generate
with torch.no_grad():
    generated = model.generate(**inputs, max_length=256, num_beams=5)

# Postprocess
output = tokenizer.batch_decode(generated, skip_special_tokens=True)
final = processor.postprocess_batch(output, "hin_Deva")

for src, tgt in zip(sentences, final):
    print(f"{src} → {tgt}")
```

--------------------------------

### preprocess_batch Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicProcessor.md

Example of preprocessing a batch of sentences with entity placeholders and language tags.

```python
from IndicTransToolkit.processor import IndicProcessor

processor = IndicProcessor(inference=True)

sentences = [
    "Contact us at support@example.com",
    "Visit https://example.com for more info",
]

preprocessed = processor.preprocess_batch(
    sentences,
    src_lang="eng_Latn",
    tgt_lang="hin_Deva",
)

# Returns sentences with entity placeholders and language tags
# ["eng_Latn hin_Deva Contact us at <ID1>", "eng_Latn hin_Deva Visit <ID2> for more info"]
```

--------------------------------

### Start model training

Source: https://github.com/ai4bharat/indictrans2/blob/main/README.md

Command to start model training after data binarization.

```bash
bash train.sh <exp_dir> <model_arch>
```

--------------------------------

### Training with Labels Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransForConditionalGeneration.md

A snippet demonstrating the preparation of input_ids and labels for training a model.

```python
# Prepare training data
input_ids = tokenizer("eng_Latn hin_Deva Hello world", return_tensors="pt")["input_ids"]
labels = tokenizer("नमस्ते दुनिया", return_tensors="pt")["input_ids"]

```

--------------------------------

### Basic Translation Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransForConditionalGeneration.md

A complete example demonstrating how to load the model and tokenizer, perform a basic English to Hindi translation, and print the result.

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    trust_remote_code=True
)
model = AutoModelForSeq2SeqLM.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    trust_remote_code=True
)

# English to Hindi translation
text = "eng_Latn hin_Deva Hello, how are you today?"
inputs = tokenizer(text, return_tensors="pt")

generated_ids = model.generate(**inputs, max_length=256, num_beams=5)
translation = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
print(translation)  # Output: ["नमस्ते, आप आज कैसे हैं?"]
```

--------------------------------

### Generation Parameters

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of setting generation parameters for the model.

```python
generated = model.generate(
    input_ids,
    max_length=256,              # Max output length
    num_beams=5,                 # Beam width (1=greedy)
    temperature=1.0,             # Sampling temperature
    top_k=50,                    # Top-k sampling
    top_p=1.0,                   # Nucleus sampling
    num_return_sequences=1,      # Results per input
    early_stopping=True,         # Stop at EOS
    do_sample=False,             # Use sampling?
    repetition_penalty=1.0,      # Penalize repeats
    length_penalty=1.0,          # Length reward
)
```

--------------------------------

### Batch Translation Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransForConditionalGeneration.md

An example showing how to perform batch translation using IndicProcessor for preprocessing and postprocessing, along with tokenization and generation.

```python
from IndicTransToolkit.processor import IndicProcessor

# Initialize processor for formatting
ip = IndicProcessor(inference=True)

# Input sentences
sentences = [
    "When I was young, I went to the park.",
    "She is very intelligent and hardworking."
]

# Preprocess
batch = ip.preprocess_batch(sentences, src_lang="eng_Latn", tgt_lang="hin_Deva")

# Tokenize
inputs = tokenizer(
    batch,
    truncation=True,
    padding="longest",
    return_tensors="pt",
)

# Generate
with torch.no_grad():
    generated_ids = model.generate(
        **inputs,
        max_length=256,
        num_beams=5,
        num_return_sequences=1,
    )

# Decode and postprocess
translations = tokenizer.batch_decode(
    generated_ids,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=True,
)
translations = ip.postprocess_batch(translations, lang="hin_Deva")
print(translations)
```

--------------------------------

### Install Dependencies

Source: https://github.com/ai4bharat/indictrans2/blob/main/README.md

Shell script to install all project dependencies and requirements.

```shell
#!/bin/bash
# Install all the dependencies and requirements associated with the project.
source install.sh

```

--------------------------------

### IndicTransConfig Initialization

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransConfig.md

Example of initializing IndicTransConfig directly or loading from a pretrained model.

```python
from huggingface_interface.configuration_indictrans import IndicTransConfig

config = IndicTransConfig(
    encoder_vocab_size=50265,
    decoder_vocab_size=50265,
    encoder_layers=6,
    decoder_layers=6,
)

# Or load from pretrained model
from transformers import AutoConfig
config = AutoConfig.from_pretrained("ai4bharat/indictrans2-en-indic-1B")
```

--------------------------------

### Install Triton Client Dependencies

Source: https://github.com/ai4bharat/indictrans2/blob/main/inference/triton_server/README.md

Installs the necessary Python packages for the Triton client.

```bash
pip install tritonclient[all] gevent
```

--------------------------------

### Create Custom Configuration

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransConfig.md

Examples of creating minimal and large custom `IndicTransConfig` objects, and saving/loading them.

```python
from huggingface_interface.configuration_indictrans import IndicTransConfig

# Create minimal config
config = IndicTransConfig(
    encoder_vocab_size=50000,
    decoder_vocab_size=50000,
)

# Create large config
config = IndicTransConfig(
    encoder_vocab_size=50000,
    decoder_vocab_size=50000,
    encoder_layers=24,
    decoder_layers=24,
    encoder_ffn_dim=4096,
    decoder_ffn_dim=4096,
    encoder_attention_heads=16,
    decoder_attention_heads=16,
    encoder_embed_dim=1024,
    decoder_embed_dim=1024,
)

# Save configuration
config.save_pretrained("path/to/config")

# Load configuration
loaded_config = IndicTransConfig.from_pretrained("path/to/config")
```

--------------------------------

### Environment Variables

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/configuration.md

Example environment variables for controlling device, cache path, and logging level.

```bash
# Force specific device
export CUDA_VISIBLE_DEVICES=0

# Control quantization
export TRANSFORMERS_CACHE="path/to/cache"

# Enable debug logging
export LOGLEVEL=DEBUG
```

--------------------------------

### Installation Dependencies

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/README.md

Core dependencies and optional packages for IndicTrans2 installation.

```bash
# Core dependencies
pip install transformers torch

# For standalone inference
pip install fairseq sentencepiece indicnlp sacremoses

# For preprocessing with toolkit
pip install git+https://github.com/VarunGumma/IndicTransToolkit.git

# For CTranslate2 inference
pip install ctranslate2

# Optional: For quantization
pip install bitsandbytes

# Optional: For Flash Attention
pip install flash-attn
```

--------------------------------

### Configuration: IndicTransConfig

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of creating and saving an IndicTransConfig object for model configuration.

```python
from huggingface_interface.configuration_indictrans import IndicTransConfig

config = IndicTransConfig(
    encoder_layers=12,
    decoder_layers=12,
    encoder_embed_dim=512,
    decoder_embed_dim=512,
    encoder_attention_heads=16,
    decoder_attention_heads=16,
    dropout=0.1,
    attention_dropout=0.0,
)

config.save_pretrained("path/to/config")
loaded = IndicTransConfig.from_pretrained("path/to/config")
```

--------------------------------

### postprocess_batch Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicProcessor.md

Example of postprocessing translated sentences to restore original entities.

```python
# After model.generate() and tokenizer.decode()
model_output = [
    "हमें support@example.com पर संपर्क करें",
    "अधिक जानकारी के लिए https://example.com पर जाएं",
]

postprocessed = processor.postprocess_batch(
    model_output,
    lang="hin_Deva",
)

# Returns cleaned sentences with original entities restored
# ["हमें support@example.com पर संपर्क करें", "अधिक जानकारी के लिए https://example.com पर जाएं"]
```

--------------------------------

### Preprocessing Patterns

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of using the normalize function with predefined patterns.

```python
from inference.normalize_regex_inference import (
    EMAIL_PATTERN,
    URL_PATTERN,
    NUMERAL_PATTERN,
    OTHER_PATTERN,
)

# Or use all
from inference.normalize_regex_inference import normalize

text, entity_map = normalize(text)  # All patterns
```

--------------------------------

### Typical Configuration Dictionary

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/types.md

An example of a configuration dictionary used for initializing IndicTrans2 models, showing common keys and their typical values.

```python
config_dict = {
    "model_name_or_path": "ai4bharat/indictrans2-en-indic-1B",
    "encoder_layers": 12,
    "decoder_layers": 12,
    "encoder_attention_heads": 16,
    "decoder_attention_heads": 16,
    "encoder_embed_dim": 512,
    "decoder_embed_dim": 512,
    "encoder_ffn_dim": 2048,
    "decoder_ffn_dim": 2048,
    "dropout": 0.1,
    "attention_dropout": 0.0,
    "activation_function": "relu",
    "max_position_embeddings": 210,
}
```

--------------------------------

### Load Pretrained Configuration

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransConfig.md

Example of loading pre-trained configurations for both base and distilled models using `AutoConfig`.

```python
from transformers import AutoConfig

# Load base model config
config = AutoConfig.from_pretrained("ai4bharat/indictrans2-en-indic-1B")

# Load distilled model config
config = AutoConfig.from_pretrained("ai4bharat/indictrans2-en-indic-dist-200M")
```

--------------------------------

### Attention Implementation Configuration

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/types.md

Example of setting the attention implementation in IndicTransConfig.

```python
from huggingface_interface.configuration_indictrans import IndicTransConfig

config = IndicTransConfig()
config.attn_implementation = "flash_attention_2"
```

--------------------------------

### Temperature Sampling (Diverse)

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of using temperature sampling for diverse translation outputs.

```python
inputs = tokenizer(text, return_tensors="pt")
output = model.generate(
    **inputs,
    max_length=256,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
)
```

--------------------------------

### Install dependencies

Source: https://github.com/ai4bharat/indictrans2/blob/main/huggingface_interface/colab_inference.ipynb

Installs necessary Python packages including nltk, sacremoses, pandas, regex, mock, transformers, mosestokenizer, bitsandbytes, scipy, accelerate, datasets, and sentencepiece. It also downloads the 'punkt' tokenizer from nltk and clones and installs the IndicTransToolkit.

```python
%%capture
!python3 -m pip install nltk sacremoses pandas regex mock transformers==4.53.2 mosestokenizer
!python3 -c "import nltk; nltk.download('punkt')"
!python3 -m pip install bitsandbytes scipy accelerate datasets
!python3 -m pip install sentencepiece

!git clone https://github.com/VarunGumma/IndicTransToolkit.git
%cd IndicTransToolkit
!python3 -m pip install --editable ./
%cd ..
```

--------------------------------

### Batch Translation

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of translating a batch of sentences.

```python
batch = ["eng_Latn hin_Deva " + s for s in sentences]
inputs = tokenizer(batch, return_tensors="pt", padding=True)
output = model.generate(**inputs, max_length=256)
results = tokenizer.batch_decode(output, skip_special_tokens=True)
```

--------------------------------

### Preprocessing: IndicProcessor

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of using IndicProcessor for preprocessing and postprocessing text for translation.

```python
from IndicTransToolkit.processor import IndicProcessor

processor = IndicProcessor(inference=True)

# Preprocess
batch = processor.preprocess_batch(
    sentences=["Contact us at john@example.com"],
    src_lang="eng_Latn",
    tgt_lang="hin_Deva",
)

# Postprocess (after model.generate() and tokenizer.decode())
final = processor.postprocess_batch(
    sentences=["हमसे john@example.com पर संपर्क करें"],
    lang="hin_Deva",
)
```

--------------------------------

### Model: HuggingFace Integration

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of using IndicTransForConditionalGeneration with HuggingFace Transformers for translation.

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    trust_remote_code=True
)
model = AutoModelForSeq2SeqLM.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    trust_remote_code=True
)

inputs = tokenizer("eng_Latn hin_Deva Hello", return_tensors="pt")

generated = model.generate(
    **inputs,
    max_length=256,
    num_beams=5,
    early_stopping=True,
    num_return_sequences=1,
)

output = tokenizer.decode(generated[0], skip_special_tokens=True)
```

--------------------------------

### IndicProcessor Custom Placeholder Prefix Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicProcessor.md

Illustrates how to potentially customize the placeholder prefix, though specific configurations are detailed elsewhere.

```python
processor = IndicProcessor(inference=True)
# Most configurations use <ID{n}> format
```

--------------------------------

### Typical Translation Workflow

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicProcessor.md

Complete example using IndicProcessor with HuggingFace models for translation.

```python
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from IndicTransToolkit.processor import IndicProcessor

# Initialize
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = IndicProcessor(inference=True)

tokenizer = AutoTokenizer.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    trust_remote_code=True
)
model = AutoModelForSeq2SeqLM.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    trust_remote_code=True,
).to(device)
model.eval()

# Input sentences
sentences = [
    "Hello, how are you?",
    "Contact me at john@example.com",
]

# Step 1: Preprocess
batch = processor.preprocess_batch(
    sentences,
    src_lang="eng_Latn",
    tgt_lang="hin_Deva",
)


```

--------------------------------

### Attention Implementation Selection

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransConfig.md

Example demonstrating how to select and use Flash Attention v2 for faster inference by modifying the configuration.

```python
from transformers import AutoConfig, AutoModelForSeq2SeqLM
from transformers.utils import is_flash_attn_2_available

# Use Flash Attention v2 for faster inference (requires flash-attn library)
config = AutoConfig.from_pretrained("ai4bharat/indictrans2-en-indic-1B")
if is_flash_attn_2_available():
    config.attn_implementation = "flash_attention_2"

model = AutoModelForSeq2SeqLM.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    config=config,
    trust_remote_code=True,
)
```

--------------------------------

### Loading Pretrained Models

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/configuration.md

Example code for loading configuration, model, and tokenizer for a given pretrained model ID.

```python
config = AutoConfig.from_pretrained("model_id")
model = AutoModelForSeq2SeqLM.from_pretrained("model_id", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("model_id", trust_remote_code=True)
```

--------------------------------

### IndicTransToolkit Preprocessing/Postprocessing

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

Example of using IndicTransToolkit for preprocessing and postprocessing batches of sentences.

```python
from IndicTransToolkit.processor import IndicProcessor

processor = IndicProcessor(inference=True)
preprocessed = processor.preprocess_batch(sentences, src_lang, tgt_lang)
postprocessed = processor.postprocess_batch(translated, lang)
```

--------------------------------

### Tokenizer Input Preparation

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/types.md

Example of tokenizing input text using AutoTokenizer and preparing it for the model.

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)

inputs = tokenizer(
    "eng_Latn hin_Deva Hello world",
    return_tensors="pt",
    padding=True,
    truncation=True,
)
# inputs is TokenizerInputs dict with input_ids and attention_mask
```

--------------------------------

### ISO Language Code Usage

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/types.md

Example showing how to use ISO language codes when the input format is set to 'iso'.

```python
model = Model(
    "path/to/model",
    input_lang_code_format="iso"
)

# Can now use ISO codes
translation = model.translate_paragraph(
    "Hello world",
    src_lang="en",  # ISO code for English
    tgt_lang="hi",  # ISO code for Hindi
)
```

--------------------------------

### Quantize Model

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

Example of loading the model with 8-bit quantization for memory efficiency.

```python
from transformers import BitsAndBytesConfig

qconfig = BitsAndBytesConfig(
    load_in_8bit=True,
    bnb_8bit_use_double_quant=True,
    bnb_8bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForSeq2SeqLM.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    quantization_config=qconfig,
    trust_remote_code=True
)
```

--------------------------------

### Use Flash Attention

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

Example of loading the model with Flash Attention implementation for faster inference.

```python
from transformers.utils import is_flash_attn_2_available

if is_flash_attn_2_available():
    model = AutoModelForSeq2SeqLM.from_pretrained(
        "ai4bharat/indictrans2-en-indic-1B",
        attn_implementation="flash_attention_2",
        trust_remote_code=True
    )
```

--------------------------------

### Tokenization Parameters

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/configuration.md

Example of tokenizing text with common parameters like max_length, truncation, and padding.

```python
inputs = tokenizer(
    text,
    max_length=256,
    truncation=True,
    padding="longest",
    return_tensors="pt",
    return_attention_mask=True,
    return_token_type_ids=False,
)
```

--------------------------------

### Greedy Decoding (Fast)

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of using greedy decoding for fast translation.

```python
inputs = tokenizer(text, return_tensors="pt")
output = model.generate(
    **inputs,
    max_length=256,
    num_beams=1,
)
```

--------------------------------

### Production Translation with Preprocessing

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

An example demonstrating a production-ready translation workflow including preprocessing, generation, and postprocessing.

```python
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from IndicTransToolkit.processor import IndicProcessor

processor = IndicProcessor(inference=True)
tokenizer = AutoTokenizer.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)

sentences = ["Contact us at support@example.com"]

# Preprocess
batch = processor.preprocess_batch(sentences, "eng_Latn", "hin_Deva")
inputs = tokenizer(batch, return_tensors="pt")

# Generate
with torch.no_grad():
    generated = model.generate(**inputs, max_length=256, num_beams=5)

# Postprocess
output = tokenizer.batch_decode(generated, skip_special_tokens=True)
final = processor.postprocess_batch(output, "hin_Deva")
print(final[0])
```

--------------------------------

### IndicProcessor Entity Placeholder Handling Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicProcessor.md

Demonstrates how IndicProcessor handles entity placeholders for emails and URLs during preprocessing and postprocessing.

```python
processor = IndicProcessor(inference=True)

input_text = "Email: john@example.com\nWebsite: https://example.com"
preprocessed = processor.preprocess_batch(
    [input_text],
    src_lang="eng_Latn",
    tgt_lang="hin_Deva",
)

# preprocessed contains:
# "eng_Latn hin_Deva Email: <ID1>\nWebsite: <ID2>"

# After translation and postprocessing:
# "ईमेल: john@example.com\nवेबसाइट: https://example.com"
```

--------------------------------

### Beam Search (Best Quality)

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of using beam search for translation, prioritizing quality.

```python
inputs = tokenizer(text, return_tensors="pt")
output = model.generate(
    **inputs,
    max_length=256,
    num_beams=5,
    early_stopping=True,
    length_penalty=0.6,
)
```

--------------------------------

### Base Model Configuration (1B)

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/configuration.md

Example of initializing `IndicTransConfig` for a base 1B parameter model.

```python
from huggingface_interface.configuration_indictrans import IndicTransConfig

config = IndicTransConfig(
    encoder_vocab_size=50265,
    decoder_vocab_size=50265,
    encoder_embed_dim=512,
    decoder_embed_dim=512,
    encoder_layers=12,
    decoder_layers=12,
    encoder_attention_heads=16,
    decoder_attention_heads=16,
    encoder_ffn_dim=2048,
    decoder_ffn_dim=2048,
    dropout=0.1,
    attention_dropout=0.0,
    max_source_positions=210,
    max_target_positions=210,
)
```

--------------------------------

### Loading a Model with AutoModelForSeq2SeqLM

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/configuration.md

Example of loading a pre-trained IndicTrans2 model using `AutoModelForSeq2SeqLM.from_pretrained` with common parameters.

```python
from transformers import AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    trust_remote_code=True,
    device_map="auto",
    torch_dtype=torch.float16,
    quantization_config=None,
    attn_implementation="eager",
    low_cpu_mem_usage=True,
)
```

--------------------------------

### ISO Code Conversion Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

Demonstrates how to convert ISO language codes to FLORES codes using the provided mapping.

```python
from inference.flores_codes_map_indic import iso_to_flores

flores_code = iso_to_flores["hi"]  # "hin_Deva"
```

--------------------------------

### Extended Context Configuration (RoPE variants)

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/configuration.md

Example of initializing `IndicTransConfig` for models with extended context using RoPE.

```python
config = IndicTransConfig(
    encoder_vocab_size=50265,
    decoder_vocab_size=50265,
    encoder_embed_dim=512,
    decoder_embed_dim=512,
    encoder_layers=12,
    decoder_layers=12,
    max_source_positions=2048,    # Extended from 210
    max_target_positions=2048,    # Extended from 210
)
```

--------------------------------

### Typical Generation Parameters Dictionary

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/types.md

An example of a parameters dictionary used for controlling text generation with IndicTrans2 models, including common keys like max_length, num_beams, and temperature.

```python
generation_config = {
    "max_length": 256,
    "min_length": 1,
    "num_beams": 5,
    "early_stopping": True,
    "temperature": 1.0,
    "top_k": 50,
    "top_p": 1.0,
    "num_return_sequences": 1,
}
```

--------------------------------

### Language Code Mapping Example Usage

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/inference_utilities.md

Demonstrates how to use the language code mapping dictionaries to convert between FLORES codes and ISO language codes.

```python
from inference.flores_codes_map_indic import flores_codes, iso_to_flores

iso_lang = flores_codes["hin_Deva"]  # Returns "hi"
flores_lang = iso_to_flores["hi"]     # Returns "hin_Deva"
```

--------------------------------

### Regex Pattern Constants and Usage

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/types.md

Importing and using predefined regex patterns for entity preservation during normalization, along with an example of custom pattern creation and normalization.

```python
from inference.normalize_regex_inference import (
    EMAIL_PATTERN,
    URL_PATTERN,
    NUMERAL_PATTERN,
    OTHER_PATTERN,
)

# Or create custom pattern
custom_pattern = r"custom_regex_here"

from inference.normalize_regex_inference import normalize

patterns = [EMAIL_PATTERN, URL_PATTERN]
text = "Email: test@example.com at https://example.com"
normalized, mapping = normalize(text, patterns=patterns)
```

--------------------------------

### Navigate to Triton Server Directory

Source: https://github.com/ai4bharat/indictrans2/blob/main/inference/triton_server/azure_ml/README.md

Change the current directory to the Triton server inference directory.

```bash
cd inference/triton_server
```

--------------------------------

### FLORES Language Code Example

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/types.md

Example demonstrating the use of FLORES language codes for translation.

```python
from inference.engine import Model

model = Model("path/to/model")
translation = model.translate_paragraph(
    "Hello world",
    src_lang="eng_Latn",    # FLORES code
    tgt_lang="hin_Deva",    # FLORES code
)
```

--------------------------------

### Push Docker Image to Container Registry

Source: https://github.com/ai4bharat/indictrans2/blob/main/inference/triton_server/azure_ml/README.md

Log in to Azure Container Registry, tag the Docker image, and push it to the registry.

```bash
az acr login --name $DOCKER_REGISTRY
docker tag indictrans2_triton $DOCKER_REGISTRY.azurecr.io/nmt/triton-indictrans-v2:latest
docker push $DOCKER_REGISTRY.azurecr.io/nmt/triton-indictrans-v2:latest
```

--------------------------------

### Create Execution Environment in Azure ML

Source: https://github.com/ai4bharat/indictrans2/blob/main/inference/triton_server/azure_ml/README.md

Create an environment in Azure Machine Learning from a YAML configuration file.

```bash
az ml environment create -f azure_ml/environment.yml -g $RESOURCE_GROUP -w $WORKSPACE_NAME
```

--------------------------------

### Run Sample Triton Client

Source: https://github.com/ai4bharat/indictrans2/blob/main/inference/triton_server/README.md

Executes the sample client script to interact with the running Triton server.

```bash
python3 triton_server/client.py
```

--------------------------------

### Key Method: Model.batch_translate()

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Signature and description for the batch_translate method.

```python
results: List[str] = model.batch_translate(
    batch: List[str],
    src_lang: str,
    tgt_lang: str
) -> List[str]
```

--------------------------------

### Multi-GPU Training with DataParallel

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransModel.md

Example of wrapping a model with `nn.DataParallel` for multi-GPU training.

```python
import torch.nn as nn

model = nn.DataParallel(model)
```

--------------------------------

### Configuration Imports

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Import for IndicTrans specific configuration.

```python
# Configuration
from huggingface_interface.configuration_indictrans import IndicTransConfig
```

--------------------------------

### Model.preprocess_batch()

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Preprocess with entity preservation and language tags.

```python
preprocessed, maps = model.preprocess_batch(
    batch: List[str],
    src_lang: str,
    tgt_lang: str
) -> Tuple[List[str], List[Dict]]
```

--------------------------------

### Multi-GPU Training with DistributedDataParallel

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransModel.md

Example of wrapping a model with `nn.parallel.DistributedDataParallel` for distributed multi-GPU training.

```python
# or
model = nn.parallel.DistributedDataParallel(model)
```

--------------------------------

### Extract Encoder Output

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

Example of extracting encoder hidden states from the model.

```python
encoder = model.get_encoder()
outputs = encoder(input_ids=inputs['input_ids'])
hidden_states = outputs.last_hidden_state
```

--------------------------------

### Translate Batch

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

Example of translating a batch of sentences using the IndicTrans2 model.

```python
sentences = ["Hello", "How are you?"]
batch = ["eng_Latn hin_Deva " + s for s in sentences]
inputs = tokenizer(batch, return_tensors="pt", padding=True)
generated = model.generate(**inputs, max_length=256, num_beams=5)
output = tokenizer.batch_decode(generated, skip_special_tokens=True)
```

--------------------------------

### Distilled Model Name

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of a distilled model name for faster inference.

```python
model_name = "ai4bharat/indictrans2-en-indic-dist-200M"
```

--------------------------------

### Model Initialization

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/configuration.md

Demonstrates how to create, load, and save IndicTransConfig objects.

```python
from huggingface_interface.configuration_indictrans import IndicTransConfig
from transformers import AutoConfig

# Create new config
config = IndicTransConfig(
    encoder_vocab_size=50000,
    decoder_vocab_size=50000,
)

# Load pretrained config
config = AutoConfig.from_pretrained("ai4bharat/indictrans2-en-indic-1B")

# Save config
config.save_pretrained("path/to/config")

# Load saved config
loaded = IndicTransConfig.from_pretrained("path/to/config")
```

--------------------------------

### Test Inference with Client Script

Source: https://github.com/ai4bharat/indictrans2/blob/main/inference/triton_server/azure_ml/README.md

Instructions to configure and run a client script for testing the deployed inference endpoint.

```python
ENABLE_SSL = True
ENDPOINT_URL = "YOUR_ENDPOINT_URL"
HTTP_HEADERS = {
    "Authorization": "Bearer YOUR_AUTH_KEY"
}

# ... rest of client.py script
```

--------------------------------

### With Entity Preservation

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of translating sentences while preserving entities using IndicProcessor.

```python
processor = IndicProcessor(inference=True)
batch = processor.preprocess_batch(sentences, "eng_Latn", "hin_Deva")
inputs = tokenizer(batch, return_tensors="pt", padding=True)
output = model.generate(**inputs, max_length=256)
decoded = tokenizer.batch_decode(output, skip_special_tokens=True)
final = processor.postprocess_batch(decoded, "hin_Deva")
```

--------------------------------

### Clone Repository and Navigate

Source: https://github.com/ai4bharat/indictrans2/blob/main/README.md

Instructions to clone the IndicTrans2 GitHub repository and navigate into the project directory.

```bash
# Clone the github repository and navigate to the project directory.
git clone https://github.com/AI4Bharat/IndicTrans2
cd IndicTrans2

```

--------------------------------

### Troubleshooting Missing Language Support

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicProcessor.md

Shows how to check for supported languages and ensure correct FLORES language codes are used, highlighting the difference between FLORES and ISO codes.

```python
from IndicTransToolkit.processor import IndicProcessor

processor = IndicProcessor(inference=True)

# Check supported languages
supported_langs = {
    "asm_Beng", "ben_Beng", "brx_Deva", "eng_Latn",
    # ... all 26 codes
}

# Ensure your codes match exactly (case-sensitive)
lang_code = "hin_Deva"  # Correct
lang_code = "hi_Deva"   # Wrong (use FLORES, not ISO)
```

--------------------------------

### Directory Structure

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Overview of the IndicTrans2 project directory structure.

```bash
/indictrans2/
├── inference/
│   ├── engine.py              # Model class
│   ├── normalize_*.py         # Normalization functions
│   ├── custom_interactive.py  # Fairseq wrapper
│   └── flores_codes_map_indic.py
├── huggingface_interface/
│   ├── modeling_indictrans.py # HF models
│   ├── configuration_indictrans.py
│   └── example.py
└── [training/evaluation/data scripts...]
```

--------------------------------

### Translate Single Sentence

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

Example of translating a single sentence using the IndicTrans2 model.

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained("ai4bharat/indictrans2-en-indic-1B", trust_remote_code=True)

text = "eng_Latn hin_Deva Hello, how are you?"
inputs = tokenizer(text, return_tensors="pt")
generated = model.generate(**inputs, max_length=256, num_beams=5)
print(tokenizer.decode(generated[0], skip_special_tokens=True))
```

--------------------------------

### Register Model with Azure ML

Source: https://github.com/ai4bharat/indictrans2/blob/main/inference/triton_server/azure_ml/README.md

Create a model in Azure Machine Learning using a YAML configuration file.

```bash
az ml model create --file azure_ml/model.yml --resource-group $RESOURCE_GROUP --workspace-name $WORKSPACE_NAME
```

--------------------------------

### Minimal Standalone Inference

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

A minimal example for standalone inference using the `inference.engine.Model` class.

```python
from inference.engine import Model

model = Model(ckpt_dir="path/to/indictrans2-en-indic-1B", device="cuda")
translation = model.translate_paragraph(
    "Hello world",
    src_lang="eng_Latn",
    tgt_lang="hin_Deva"
)
print(translation)  # नमस्ते दुनिया
```

--------------------------------

### Single Sentence Translation

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Example of translating a single sentence using the tokenizer and model.

```python
text = "eng_Latn hin_Deva Hello world"
inputs = tokenizer(text, return_tensors="pt")
output = model.generate(**inputs, max_length=256)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

--------------------------------

### Attach Deployment to Endpoint

Source: https://github.com/ai4bharat/indictrans2/blob/main/inference/triton_server/azure_ml/README.md

Create an online deployment for an endpoint in Azure Machine Learning using a YAML configuration file and route all traffic to it.

```bash
az ml online-deployment create -f azure_ml/deployment.yml --all-traffic -g $RESOURCE_GROUP -w $WORKSPACE_NAME
```

--------------------------------

### Distilled Configuration (200M-320M)

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/configuration.md

Example of initializing `IndicTransConfig` for a distilled model (200M-320M parameters).

```python
config = IndicTransConfig(
    encoder_vocab_size=50265,
    decoder_vocab_size=50265,
    encoder_embed_dim=256,
    decoder_embed_dim=256,
    encoder_layers=6,
    decoder_layers=6,
    encoder_attention_heads=8,
    decoder_attention_heads=8,
    encoder_ffn_dim=1024,
    decoder_ffn_dim=1024,
    dropout=0.1,
    attention_dropout=0.0,
    max_source_positions=210,
    max_target_positions=210,
)
```

--------------------------------

### IndicTransModel Initialization

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransModel.md

Demonstrates how to initialize IndicTransModel with a configuration or load a pretrained model and access its base.

```python
from huggingface_interface.configuration_indictrans import IndicTransConfig
from huggingface_interface.modeling_indictrans import IndicTransModel

config = IndicTransConfig()
model = IndicTransModel(config)

# Or load a pretrained full model and access the base model
from transformers import AutoModel

base_model = AutoModel.from_pretrained(
    "ai4bharat/indictrans2-en-indic-1B",
    trust_remote_code=True
)
```

--------------------------------

### Access/Replace Output Embeddings

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/IndicTransForConditionalGeneration.md

Shows how to get the current output embedding layer and replace it with a custom one.

```python
# Get current output head
output_head = model.get_output_embeddings()

# Replace with custom head
import torch.nn as nn
new_head = nn.Linear(512, 50000)
model.set_output_embeddings(new_head)
```

--------------------------------

### Fine-tune Model

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/API_Overview.md

Example of fine-tuning the IndicTrans2 model using Hugging Face's Seq2SeqTrainer.

```python
from transformers import Seq2SeqTrainer, Seq2SeqTrainingArguments

training_args = Seq2SeqTrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=8,
    save_steps=500,
)

trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    tokenizer=tokenizer,
)

trainer.train()
```

--------------------------------

### Flash Attention Configuration

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Configuration to enable Flash Attention 2 for improved performance if available.

```python
from transformers.utils import is_flash_attn_2_available

if is_flash_attn_2_available():
    model = AutoModelForSeq2SeqLM.from_pretrained(
        model_name,
        attn_implementation="flash_attention_2",
        trust_remote_code=True
    )
```

--------------------------------

### Quantization Configuration

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Configuration for loading models in 8-bit precision to reduce memory usage.

```python
from transformers import BitsAndBytesConfig
qconfig = BitsAndBytesConfig(
    load_in_8bit=True,
    bnb_8bit_use_double_quant=True,
)
```

--------------------------------

### Model.postprocess()

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/QUICKREF.md

Postprocess to target language script with entity restoration.

```python
postprocessed = model.postprocess(
    sents: List[str],
    placeholder_entity_map: List[Dict],
    lang: str,
    common_lang: str = "hin_Deva"
) -> List[str]
```

--------------------------------

### Accessing Seq2SeqLMOutput Fields and Loss

Source: https://github.com/ai4bharat/indictrans2/blob/main/_autodocs/types.md

Example of accessing loss, logits, and hidden states from Seq2SeqLMOutput.

```python
outputs = model(
    input_ids=input_ids,
    labels=labels,
    return_dict=True,
    output_hidden_states=True,
    output_attentions=True,
)

loss = outputs.loss
logits = outputs.logits
all_decoder_states = outputs.decoder_hidden_states  # Tuple of 6 tensors
```

--------------------------------

### Initialize model and tokenizer parameters

Source: https://github.com/ai4bharat/indictrans2/blob/main/huggingface_interface/colab_inference.ipynb

Sets up PyTorch, defines constants for batch size and device (GPU if available, otherwise CPU), and initializes quantization configuration.

```python
import torch
from transformers import AutoModelForSeq2SeqLM, BitsAndBytesConfig, AutoTokenizer
from IndicTransToolkit.processor import IndicProcessor

BATCH_SIZE = 4
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
quantization = None
```