### Install IndicNLP Library

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Command to install the missing library dependency.

```bash
pip install indic-nlp-library
```

--------------------------------

### Install project dependencies

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/README.md

Install the required Python packages including torch, fasttext, and the indic-nlp-library.

```bash
pip install torch requests tqdm scipy fasttext faiss-cpu numpy gensim scikit-learn indic-nlp-library morfessor
```

--------------------------------

### Install Required Packages

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Install common dependencies for text classification and word analogy tasks.

```bash
pip install fasttext gensim torch
```

--------------------------------

### IndicNLP Library Setup

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/configuration.md

Environment configuration for IndicNLP normalization resources.

```bash
# Download IndicNLP normalization resources (if not auto-downloaded)
# or set INDIC_NLP_DATA environment variable pointing to resource directory
```

--------------------------------

### Execute the evaluation data setup script

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/example-workflows.md

Command to run the data preparation script from the terminal.

```bash
python setup_evaluation_data.py
```

--------------------------------

### Evaluate word analogies

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

Example usage for loading embeddings and running the evaluation function.

```python
import numpy as np
from src.utils import load_embeddings

# Load embeddings
params = Params()
params.src_emb = 'indicnlp.v1.hi.bin'
params.max_vocab = 200000
params.emb_dim = 300
src_dico, _src_emb = load_embeddings(params, source=True)

word2id = src_dico.word2id
embeddings = _src_emb.weight.data.cpu().numpy()

# Evaluate
accuracies = get_wordanalogy_scores_customfname(
    'questions-words-hi.txt',
    'hi',
    word2id,
    embeddings,
    lower=True
)

# Access results
print(f"Capital-common-countries accuracy: {accuracies['capital-common-countries']:.4f}")
```

--------------------------------

### Word Analogy Output

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

Example output format returned by the evaluation script.

```text
Params(...)
<output from get_wordanalogy_scores_customfname()>
Average score: 0.6841
```

--------------------------------

### Example Usage of read_word_similarity

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-similarity.md

Demonstrates loading a similarity database file into a list of tuples.

```python
sim_database = read_word_similarity('iiith_wordsim_hi.txt', delim='\t')

# sim_database example:
# [('कपड़ा', 'वस्त्र', 8.92),
#  ('स्कूल', 'शिक्षा', 8.08),
#  ('अदालत', 'कानून', 8.46)]
```

--------------------------------

### Example Output for Word Similarity Evaluation

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-similarity.md

Displays the expected console output format after running the evaluation.

```text
Correlation: 0.5234
P-value: 0.000001
Coverage: 0.95
```

--------------------------------

### Invalid Word Analogy File Examples

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Examples of malformed analogy files that trigger assertion or value errors.

```text
: capital-common-countries
Athens Greece Baghdad  # Only 3 words instead of 4!
```

```text
Athens Greece Baghdad Iraq  # No category marker before questions
```

--------------------------------

### Usage of length_normalize_dimensionwise

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/embeddings-module.md

Example showing how to normalize a matrix of vectors.

```python
normalized = length_normalize_dimensionwise(vectors)
# Each dimension now has unit norm across vocabulary
```

--------------------------------

### Usage of mean_center

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/embeddings-module.md

Example demonstrating centering of an embedding matrix.

```python
centered = mean_center(vectors)
# Mean of centered matrix across axis 0 is now ~0
```

--------------------------------

### Example Usage of compute_word_similarity

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-similarity.md

Shows the full workflow of loading embeddings, normalizing them, reading the database, and calculating the correlation metrics.

```python
import embeddings

# Load embeddings
with open('indicnlp.v1.hi.vec', 'r') as f:
    emb_info = embeddings.read(f, max_voc=200000)

# Normalize for cosine similarity
emb_words, emb_vectors = emb_info
emb_vectors = embeddings.length_normalize(emb_vectors)
emb_info = (emb_words, emb_vectors)

# Read similarity database
sim_db = read_word_similarity('iiith_wordsim_hi.txt')

# Evaluate
correlation, pvalue, coverage = compute_word_similarity(emb_info, sim_db)

print(f'Spearman Correlation: {correlation:.4f}')
print(f'P-value: {pvalue:.6f}')
print(f'Coverage: {coverage:.2%}')
```

--------------------------------

### Load embeddings from file

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/embeddings-module.md

Example of loading word embeddings from a file handle with specified vocabulary limits and data type.

```python
with open('embeddings.vec', 'r', encoding='utf-8') as f:
    words, vectors = read(f, max_voc=200000, dtype='float32')
    
# words is a list of 200,000 words
# vectors is a (200000, 300) numpy array of embeddings
```

--------------------------------

### Define Analogy Questions File Format

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/types.md

Specifies the structure for word analogy questions, using category markers starting with a colon followed by four space-separated words per line.

```text
: category_name
word1 word2 word3 word4
word1 word2 word3 word4

: another_category
...
```

--------------------------------

### Usage of mean_center_embeddingwise

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/embeddings-module.md

Example showing how to center word vectors independently.

```python
centered = mean_center_embeddingwise(vectors)
# Each word vector now has mean 0 across its dimensions
```

--------------------------------

### Convert document to vector using txtcls

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/text-classification.md

Example usage showing how to load embeddings and convert a document string into a vector using the txtcls module.

```python
from gensim.models import KeyedVectors
import txtcls

# Load embeddings
emb = KeyedVectors.load_word2vec_format(
    'indicnlp.v1.hi.bin', 
    binary=False, 
    encoding='utf8'
)

# Convert document to vector
doc_text = "यह एक नमूना दस्तावेज है जो वर्गीकरण के लिए परीक्षण किया जाएगा。"
doc_vector = txtcls.doc2vec(doc_text, lang='hi', emb=emb)

# doc_vector is a numpy array of shape (300,) for 300-dim embeddings
```

--------------------------------

### Example evaluation output

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

The formatted table output generated by the evaluation process.

```text
======================================================
                 Category      Found  Not found  Accuracy
======================================================
   capital-common-countries       506         29    0.8851
        capital-world              4         75    0.0000
             currency              0          0       nan
family                          245          0    0.7918
gram1: adjective-to-adverb       124         13    0.6048
gram2: opposite                  147         15    0.5714
gram3: comparative                62          0    0.5968
gram4: superlative                34          0    0.7059
gram5: present-participle         30          0    0.7667
gram6: nationality-adjective     149          0    0.8053
gram7: past-tense                 72          0    0.6111
gram8: plural                    140          0    0.7857
gram9: plural-verbs               96          0    0.6979
======================================================

Coverage: 0.9211
```

--------------------------------

### Configure MUSE Python Path

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Set the PYTHONPATH environment variable to include the MUSE installation directory before execution.

```bash
export PYTHONPATH=$PYTHONPATH:$MUSE_PATH
python scripts/word_analogy/word_analogy.py ...
```

--------------------------------

### Usage of length_normalize

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/embeddings-module.md

Example demonstrating L2 normalization of a numpy array of vectors.

```python
import numpy as np

vectors = np.array([[3.0, 4.0], [1.0, 0.0]], dtype=np.float32)
normalized = length_normalize(vectors)
# normalized rows will have unit length

# Verify: np.linalg.norm(normalized[0]) == 1.0
```

--------------------------------

### Define FastText Embedding File Format

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/types.md

Specifies the structure of .vec files, starting with vocabulary size and embedding dimension followed by word vectors.

```text
<vocabulary_size> <embedding_dimension>
<word1> <val1> <val2> ... <valD>
<word2> <val1> <val2> ... <valD>
...
```

--------------------------------

### Configure Analogy Programmatically

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/configuration.md

Define configuration parameters in a dictionary and pass them to the evaluation function.

```python
# Configure parameters
analogy_config = {
    'analogy_fname': 'questions-words-hi.txt',
    'embeddings_path': 'indicnlp.v1.hi.bin',
    'lang': 'hi',
    'emb_dim': 300,
    'max_vocab': 200000,
    'lower': True,
    'cuda': True
}

# Run evaluation
accuracies = score_analogy(**analogy_config)
```

--------------------------------

### Run Word Analogy CLI

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

Executes the word analogy evaluation script via the command line.

```bash
python scripts/word_analogy/word_analogy.py \
    --analogy_fname <questions_file> \
    --embeddings_path <embedding_file> \
    --lang <language_code> \
    --emb_dim <dimension> \
    [--max_vocab <vocab_size>] \
    [--cuda] \
    [--lower]
```

```bash
# CPU evaluation without lowercasing
python scripts/word_analogy/word_analogy.py \
    --analogy_fname questions-words-hi.txt \
    --embeddings_path indicnlp.v1.hi.bin \
    --lang hi \
    --emb_dim 300

# GPU evaluation with max 150K vocabulary
python scripts/word_analogy/word_analogy.py \
    --analogy_fname questions-words-hi.txt \
    --embeddings_path indicnlp.v1.hi.bin \
    --lang hi \
    --emb_dim 300 \
    --max_vocab 150000 \
    --cuda
```

--------------------------------

### Execute Word Analogy Evaluation

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/configuration.md

Run the evaluation script using either CPU or GPU acceleration.

```bash
# CPU evaluation
python word_analogy.py --analogy_fname q.txt --embeddings_path emb.bin \
    --lang hi --emb_dim 300

# GPU evaluation (if available)
python word_analogy.py --analogy_fname q.txt --embeddings_path emb.bin \
    --lang hi --emb_dim 300 --cuda
```

--------------------------------

### Display Project File Structure

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/INDEX.md

Visual representation of the project directory layout and file organization.

```text
/workspace/home/output/
├── README.md                    # Start here
├── INDEX.md                     # This file
├── project-overview.md          # 120 lines
├── configuration.md             # 550 lines
├── types.md                     # 250 lines
├── errors.md                    # 450 lines
├── example-workflows.md         # 650 lines
└── api-reference/               # 1450 lines total
    ├── embeddings-module.md     # 350 lines
    ├── word-similarity.md       # 350 lines
    ├── text-classification.md   # 450 lines
    └── word-analogy.md          # 500 lines
```

--------------------------------

### Configure environment for word analogy evaluation

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/README.md

Adds the MUSE directory to the Python path before running evaluation scripts.

```bash
export PYTHONPATH=$PYTHONPATH:$MUSE_PATH
```

--------------------------------

### Initialize TxtCls Classifier

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/text-classification.md

Instantiate the classifier by specifying the dataset directory and target language.

```python
classifier = TxtCls(data_dir='/path/to/classification/datasets', lang='hi')
```

--------------------------------

### Execute Word Analogy with GPU Acceleration

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/configuration.md

Toggle between CPU and GPU execution modes using the --cuda flag to significantly improve performance.

```bash
# CPU (default, no CUDA flag)
python word_analogy.py ... # ~30-60 sec for 200K vocab

# GPU (with --cuda flag)
python word_analogy.py ... --cuda  # ~2-5 sec for 200K vocab
```

--------------------------------

### Handling TxtCls Initialization Error

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Demonstrates catching the exception raised when an invalid data directory is provided.

```python
from txtcls import TxtCls

# This will raise an exception
try:
    classifier = TxtCls(data_dir='/nonexistent/path', lang='hi')
except Exception as e:
    print(f"Error: {e}")  # Error: Please download the dataset first
    # Download dataset and retry with correct path
```

--------------------------------

### Handle CUDA availability in word analogy

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Check for GPU availability before moving embeddings to the device to prevent runtime errors.

```python
if params.cuda:
    src_emb.cuda()  # Will fail if no GPU
```

--------------------------------

### python scripts/word_analogy/word_analogy.py

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

Executes the word analogy evaluation script using the provided embedding file and analogy questions.

```APIDOC
## CLI Command: python scripts/word_analogy/word_analogy.py

### Description
Evaluates word embeddings by performing analogy tasks based on a provided questions file.

### Arguments
- **--analogy_fname** (str) - Required - Path to analogy questions file
- **--embeddings_path** (str) - Required - Path to embedding binary file
- **--lang** (str) - Required - Language code
- **--emb_dim** (int) - Required - Embedding dimension
- **--max_vocab** (int) - Optional - Maximum vocabulary to load (Default: 200000)
- **--cuda** (flag) - Optional - Enable CUDA acceleration
- **--lower** (flag) - Optional - Convert to lowercase

### Example Command
```bash
python scripts/word_analogy/word_analogy.py \
    --analogy_fname questions-words-hi.txt \
    --embeddings_path indicnlp.v1.hi.bin \
    --lang hi \
    --emb_dim 300 \
    --max_vocab 150000 \
    --cuda
```
```

--------------------------------

### Run Evaluation via Command Line

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/example-workflows.md

Set the environment variable for the MUSE path and execute the evaluation script.

```bash
export PYTHONPATH=$PYTHONPATH:$MUSE_PATH
python evaluate_analogy.py
```

--------------------------------

### Get Word ID Helper Function

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

Internal helper function for looking up word indices with optional lowercasing. Not intended for direct external use.

```python
def get_word_id(word, word2id, lower)
```

--------------------------------

### Evaluate embeddings on word analogy

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/README.md

Run the word analogy evaluation script, requiring the MUSE path to be set in PYTHONPATH.

```bash
export PYTHONPATH=$PYTHONPATH:$MUSE_PATH
python scripts/word_analogy/word_analogy.py \
    --analogy_fname questions-words-hi.txt \
    --embeddings_path indicnlp.v1.hi.bin \
    --lang hi \
    --emb_dim 300 \
    --cuda
```

--------------------------------

### Define analogy file format

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

The expected structure for the input analogy file.

```text
: category_name
word1 word2 word3 word4
word1 word2 word3 word4
...

: another_category
...
```

--------------------------------

### Import MUSE Module Dependencies

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Required imports for word analogy tasks.

```python
from src.utils import load_embeddings
from torch import nn
```

--------------------------------

### Define Params Class

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

Initializes the configuration container for embedding parameters.

```python
class Params(object)
```

```python
def __init__(self)
```

--------------------------------

### Programmatic Classifier Configuration

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/configuration.md

Initialization of the classifier and loading of word embeddings for evaluation.

```python
# Initialize classifier with dataset
classifier = TxtCls(
    data_dir='/path/to/datasets',  # Must exist
    lang='hi'                        # Must match subdirectory name
)

# Load embeddings with specific format
emb = KeyedVectors.load_word2vec_format(
    embedding_path,
    binary=False,        # Text format (.vec)
    encoding='utf8'      # UTF-8 encoding required
)

# or for binary FastText format:
emb = KeyedVectors.load_word2vec_format(
    embedding_path,
    binary=True          # Binary FastText format
)

# Evaluate
accuracy = classifier.evaluate(emb)
```

--------------------------------

### Valid Word Analogy File Format

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

The expected structure for analogy files, requiring category markers and four-word lines.

```text
: capital-common-countries
Athens Greece Baghdad Iraq
Abuja Nigeria Algiers Algeria

: capital-world
Abuja Nigeria Accra Ghana
```

--------------------------------

### Download and extract evaluation datasets in Python

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/example-workflows.md

Automates the retrieval and extraction of tar.gz evaluation datasets from specified URLs. Requires the requests library.

```python
#!/usr/bin/env python3
"""
Prepare evaluation data for embeddings.
"""

import os
import tarfile
import requests
from pathlib import Path

def download_and_extract(url, target_dir, filename=None):
    """
    Download and extract tar.gz file.
    """
    os.makedirs(target_dir, exist_ok=True)
    
    if filename is None:
        filename = url.split('/')[-1]
    
    filepath = os.path.join(target_dir, filename)
    
    # Download
    print(f"Downloading {url}...")
    response = requests.get(url, stream=True)
    response.raise_for_status()
    
    with open(filepath, 'wb') as f:
        for chunk in response.iter_content(chunk_size=8192):
            f.write(chunk)
    
    # Extract
    print(f"Extracting to {target_dir}...")
    with tarfile.open(filepath, 'r:gz') as tar:
        tar.extractall(path=target_dir)
    
    print("Done!")

def setup_evaluation_data(output_dir='data'):
    """
    Download and setup all evaluation datasets.
    """
    
    os.makedirs(output_dir, exist_ok=True)
    
    # 1. Download classification datasets
    print("\n" + "="*60)
    print("Downloading IndicNLP News Articles Dataset")
    print("="*60)
    download_and_extract(
        'https://storage.googleapis.com/ai4bharat-public-indic-nlp-corpora/evaluations/classification/indicnlp-news-articles.tgz',
        output_dir
    )
    
    # 2. Download word similarity database
    print("\n" + "="*60)
    print("Downloading IIIT-H Word Similarity Database")
    print("="*60)
    download_and_extract(
        'https://storage.googleapis.com/ai4bharat-public-indic-nlp-corpora/evaluations/word_similarity/iiith_wordsim.tgz',
        output_dir
    )
    
    # 3. Download public classification datasets
    print("\n" + "="*60)
    print("Downloading Public Classification Datasets")
    print("="*60)
    download_and_extract(
        'https://storage.googleapis.com/ai4bharat-public-indic-nlp-corpora/evaluations/classification/classification_public_datasets.tgz',
        output_dir
    )
    
    print("\n" + "="*60)
    print("All data downloaded successfully!")
    print("="*60)

if __name__ == '__main__':
    setup_evaluation_data()
```

--------------------------------

### Programmatic Word Similarity Configuration

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/configuration.md

Demonstrates loading embeddings with a custom vocabulary limit, normalizing vectors, and reading a similarity database with a specific delimiter.

```python
# Load embeddings with specific max vocabulary
max_vocab = 150000  # Load top 150K words
with open('embeddings.vec', 'r') as f:
    emb_info = embeddings.read(f, max_voc=max_vocab)

# Optionally normalize embeddings
emb_words, emb_vectors = emb_info
emb_vectors = embeddings.length_normalize(emb_vectors)

# Read similarity database with custom delimiter
sim_db = read_word_similarity('db.txt', delim='	')
```

--------------------------------

### Run evaluation script via CLI

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/example-workflows.md

Executes the word similarity evaluation script from the command line.

```bash
python evaluate_wordsim.py
```

--------------------------------

### Evaluate word analogy

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/README.md

Runs the word analogy evaluation script with specified language and embedding dimensions.

```bash
python  scripts/word_analogy/word_analogy.py \
    --analogy_fname <analogy_fname> \
    --embeddings_path <embedding_file_path> \
    --lang 'hi' \
    --emb_dim 300 \
    --cuda
```

--------------------------------

### Handle OOV words in word similarity

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Demonstrates how word similarity functions handle OOV words by returning a coverage metric less than 1.0.

```python
# Word similarity with some OOV
sim_db = [('कपड़ा', 'वस्त्र', 8.92), ('unknown', 'word', 5.0)]
emb_info = load_embeddings_somehow()
corr, pval, cov = compute_word_similarity(emb_info, sim_db)
# cov will be < 1.0 if ('unknown', 'word') not in vocabulary
```

--------------------------------

### TxtCls Initialization Exception

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Defines the class structure where a DirectoryNotFound error is raised if the data directory does not exist.

```python
class TxtCls:
    def __init__(self, *args, **kwargs):
        if not os.path.exists(self.data_dir):
            raise 'Please download the dataset first'
```

--------------------------------

### Perform k-NN search with FAISS

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/types.md

Initializes a flat L2 index, adds vectors to it, and performs a k-nearest neighbor search.

```python
index = faiss.IndexFlatL2(dim)
index.add(database)
distances, indices = index.search(queries, k=4)
```

--------------------------------

### Initialize PyTorch embedding layer

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/types.md

Creates a PyTorch embedding layer and initializes its weights with pre-computed numpy embeddings.

```python
embedding = nn.Embedding(vocab_size, emb_dim)
embedding.weight.data.copy_(numpy_embeddings)
```

--------------------------------

### Dataset Directory Structure

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/configuration.md

The expected directory layout for training and testing data, organized by language subdirectories.

```text
data_dir/
  {lang}/
    {lang}-train.csv
    {lang}-test.csv
  hi/
    hi-train.csv
    hi-test.csv
  bn/
    bn-train.csv
    bn-test.csv
  ...
```

--------------------------------

### Evaluate Word Embeddings with Python

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/example-workflows.md

A script to perform word analogy evaluation on single or multiple embedding files. Requires the MUSE library path to be configured.

```python
#!/usr/bin/env python3
"""
Evaluate embeddings on word analogy task.
"""

import os
import sys
import numpy as np

# Add MUSE to path
MUSE_PATH = os.environ.get('MUSE_PATH', '/path/to/MUSE')
sys.path.insert(0, MUSE_PATH)

from word_analogy import score_analogy

def evaluate_word_analogy(analogy_file, embedding_file, language, embedding_dim=300,
                         max_vocab=200000, use_cuda=True):
    """
    Evaluate embeddings on word analogy task.
    
    Args:
        analogy_file: Path to analogy questions file
        embedding_file: Path to embedding binary file
        language: Language code
        embedding_dim: Embedding dimension (typically 300)
        max_vocab: Maximum vocabulary size
        use_cuda: Whether to use GPU acceleration
    
    Returns:
        dict mapping category names to accuracy scores
    """
    
    # Validate files
    if not os.path.exists(analogy_file):
        raise FileNotFoundError(f"Analogy file not found: {analogy_file}")
    
    if not os.path.exists(embedding_file):
        raise FileNotFoundError(f"Embedding file not found: {embedding_file}")
    
    # Run evaluation
    print(f"Evaluating {embedding_file} on {analogy_file}")
    print(f"Language: {language}, Dimension: {embedding_dim}")
    print(f"Max vocabulary: {max_vocab}, CUDA: {use_cuda}")
    print()
    
    accuracies = score_analogy(
        analogy_fname=analogy_file,
        embeddings_path=embedding_file,
        lang=language,
        emb_dim=embedding_dim,
        max_vocab=max_vocab,
        lower=True,
        cuda=use_cuda
    )
    
    # Compute statistics
    categories = sorted(accuracies.keys())
    scores = [accuracies[cat] for cat in categories]
    
    print("\n" + "="*60)
    print("SUMMARY")
    print("="*60)
    
    # Filter NaN values (categories with no questions)
    valid_scores = [s for s in scores if not np.isnan(s)]
    
    if valid_scores:
        print(f"Average accuracy: {np.mean(valid_scores):.4f}")
        print(f"Median accuracy: {np.median(valid_scores):.4f}")
        print(f"Min accuracy: {np.min(valid_scores):.4f}")
        print(f"Max accuracy: {np.max(valid_scores):.4f}")
    
    print("="*60)
    
    return accuracies

def compare_embeddings(embedding_files, analogy_file, language, embedding_dim=300):
    """
    Compare multiple embedding versions on the same analogy task.
    
    Args:
        embedding_files: Dict of {name: path} for different embedding versions
        analogy_file: Path to analogy questions file
        language: Language code
        embedding_dim: Embedding dimension
    """
    
    results = {}
    
    for name, path in embedding_files.items():
        print(f"\n{'='*60}")
        print(f"Evaluating: {name}")
        print(f"{'='*60}\n")
        
        accuracies = evaluate_word_analogy(
            analogy_file,
            path,
            language,
            embedding_dim=embedding_dim
        )
        results[name] = accuracies
    
    # Comparison table
    print("\n" + "="*60)
    print("COMPARISON")
    print("="*60)
    
    categories = sorted(results[list(results.keys())[0]].keys())
    
    print(f"{'Category':<40} ", end='')
    for name in results.keys():
        print(f"{name:<15} ", end='')
    print()
    print("-" * (40 + 15 * len(results)))
    
    for cat in categories:
        print(f"{cat:<40} ", end='')
        for name in results.keys():
            acc = results[name][cat]
            if np.isnan(acc):
                print(f"{'N/A':<15} ", end='')
            else:
                print(f"{acc:.4f}           ", end='')
        print()
    
    # Average scores
    print("-" * (40 + 15 * len(results)))
    print(f"{'Average':<40} ", end='')
    for name in results.keys():
        accs = [a for a in results[name].values() if not np.isnan(a)]
        if accs:
            avg = np.mean(accs)
            print(f"{avg:.4f}           ", end='')
        else:
            print(f"{'N/A':<15} ", end='')
    print()
    print("="*60)

if __name__ == '__main__':
    # Single embedding evaluation
    accuracies = evaluate_word_analogy(
        analogy_file='questions-words-hi.txt',
        embedding_file='indicnlp.v1.hi.bin',
        language='hi',
        embedding_dim=300,
        max_vocab=200000,
        use_cuda=True
    )
    
    # Or compare multiple embeddings
    # compare_embeddings(
    #     embedding_files={
    #         'IndicNLP v1': 'indicnlp.v1.hi.bin',
    #         'Custom': 'custom_embeddings.bin',
    #     },
    #     analogy_file='questions-words-hi.txt',
    #     language='hi'
    # )
```

--------------------------------

### Run Complete Evaluation Pipeline in Python

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/example-workflows.md

Executes the full evaluation suite for a given embedding file and language. Requires specific directory structures for datasets and scripts.

```python
#!/usr/bin/env python3
"""
Complete evaluation pipeline for embeddings across all tasks.
"""

import os
import json
from datetime import datetime

def run_complete_evaluation(embedding_path, language, output_dir='results'):
    """
    Run complete evaluation on word similarity, text classification, and analogy.
    
    Args:
        embedding_path: Path to embedding file
        language: Language code
        output_dir: Directory to save results
    """
    
    os.makedirs(output_dir, exist_ok=True)
    
    results = {
        'timestamp': datetime.now().isoformat(),
        'embedding': embedding_path,
        'language': language,
        'tasks': {}
    }
    
    # Task 1: Word Similarity
    print("\n" + "="*60)
    print("TASK 1: Word Similarity Evaluation")
    print("="*60)
    try:
        # Configure paths
        sim_db = f'iiith_wordsim_{language}.txt'
        
        # Run evaluation
        sys.path.insert(0, 'scripts/word_similarity')
        import embeddings
        from wordsim import read_word_similarity, compute_word_similarity
        
        with open(embedding_path, 'r') as f:
            emb_words, emb_vectors = embeddings.read(f, max_voc=200000)
        
        emb_vectors = embeddings.length_normalize(emb_vectors)
        sim_db = read_word_similarity(sim_db)
        
        corr, pval, cov = compute_word_similarity(
            (emb_words, emb_vectors), sim_db
        )
        
        results['tasks']['word_similarity'] = {
            'correlation': float(corr),
            'p_value': float(pval),
            'coverage': float(cov),
            'status': 'success'
        }
        
        print(f"✓ Correlation: {corr:.4f}")
        print(f"✓ Coverage: {cov:.1%}")
        
    except Exception as e:
        print(f"✗ Error: {e}")
        results['tasks']['word_similarity'] = {'status': 'failed', 'error': str(e)}
    
    # Task 2: Text Classification
    print("\n" + "="*60)
    print("TASK 2: Text Classification Evaluation")
    print("="*60)
    try:
        from gensim.models import KeyedVectors
        from txtcls import TxtCls
        
        emb = KeyedVectors.load_word2vec_format(
            embedding_path, binary=False, encoding='utf8'
        )
        
        classifier = TxtCls(data_dir='indicnlp-news-articles', lang=language)
        accuracy = classifier.evaluate(emb)
        
        results['tasks']['text_classification'] = {
            'accuracy': float(accuracy),
            'status': 'success'
        }
        
        print(f"✓ Accuracy: {accuracy:.4f}")
        
    except Exception as e:
        print(f"✗ Error: {e}")
        results['tasks']['text_classification'] = {'status': 'failed', 'error': str(e)}
    
    # Task 3: Word Analogy
    print("\n" + "="*60)
    print("TASK 3: Word Analogy Evaluation")
    print("="*60)
    try:
        os.environ['PYTHONPATH'] = os.environ.get('PYTHONPATH', '') + f':{os.environ.get("MUSE_PATH", "")}'
        from word_analogy import score_analogy
        
        analogy_file = f'questions-words-{language}.txt'
        accuracies = score_analogy(
            analogy_fname=analogy_file,
            embeddings_path=embedding_path,
            lang=language,
            emb_dim=300,
            cuda=True
        )
        
        avg_acc = sum(accuracies.values()) / len(accuracies)
        results['tasks']['word_analogy'] = {
            'accuracies': {k: float(v) for k, v in accuracies.items()},
            'average': float(avg_acc),
            'status': 'success'
        }
        
        print(f"✓ Average accuracy: {avg_acc:.4f}")
        
    except Exception as e:
        print(f"✗ Error: {e}")
        results['tasks']['word_analogy'] = {'status': 'failed', 'error': str(e)}
    
    # Save results
    output_file = os.path.join(output_dir, f'{language}_results.json')
    with open(output_file, 'w') as f:
        json.dump(results, f, indent=2)
    
    print("\n" + "="*60)
    print("SUMMARY")
    print("="*60)
    print(f"Language: {language}")
    for task, result in results['tasks'].items():
        status = result.get('status', 'unknown')
        print(f"{task}: {status}")
    print(f"Results saved to: {output_file}")
    print("="*60)
    
    return results

if __name__ == '__main__':
    # Evaluate Hindi embeddings
    results = run_complete_evaluation(
        embedding_path='indicnlp.v1.hi.bin',
        language='hi',
        output_dir='evaluation_results'
    )
```

--------------------------------

### Evaluate embeddings using the classification script

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/README.md

Run the evaluation script by providing the path to the embeddings, the data directory, and the target language code.

```bash
python3 scripts/txtcls.py --emb_path <path> --data_dir <path> --lang <lang code>
```

```bash
python3 scripts/txtcls.py --emb_path <path> --data_dir <path> --lang <lang code>
```

--------------------------------

### Execute Evaluation Script via Command Line

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/example-workflows.md

Runs the evaluation script using the Python interpreter.

```bash
python complete_evaluation.py
```

--------------------------------

### Verify and Convert Text Encoding

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Check file encoding using the file command and convert non-UTF-8 files to UTF-8 using iconv.

```bash
file -i data.csv
# Should output: data.csv: text/plain; charset=utf-8
```

```bash
iconv -f ISO-8859-1 -t UTF-8 input.csv > output.csv
```

--------------------------------

### TxtCls Constructor

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/text-classification.md

Initializes the TxtCls classifier with a data directory and language code.

```APIDOC
## TxtCls(data_dir, lang)

### Description
Initializes the classifier instance for a specific language and dataset directory.

### Parameters
- **data_dir** (str) - Required - Path to root directory containing language subdirectories.
- **lang** (str) - Required - Language code (e.g., 'hi', 'bn', 'ta').

### Example
```python
classifier = TxtCls(data_dir='/path/to/classification/datasets', lang='hi')
```
```

--------------------------------

### Load FastText .bin embeddings

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/configuration.md

Load binary embeddings using Gensim for classification or custom utilities for word analogy tasks.

```python
# Gensim (for text classification)
emb = KeyedVectors.load_word2vec_format(
    'embeddings.bin',
    binary=True,
    encoding='utf8'
)

# MUSE (for word analogy)
from src.utils import load_embeddings
params.src_emb = 'embeddings.bin'
dico, emb = load_embeddings(params, source=True)
```

--------------------------------

### Import IndicNLP Library

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Required imports for tokenization and normalization modules.

```python
from indicnlp.tokenize import indic_tokenize
from indicnlp.normalize import indic_normalize
from indicnlp.tokenize import sentence_tokenize
```

--------------------------------

### CLI Output Format

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-similarity.md

The expected output format printed to stdout after running the evaluation script.

```text
Max Vocabulary: {vocabulary_size}
Correlation: {spearman_coefficient}
p-value: {statistical_pvalue}
Coverage: {coverage_ratio}
```

--------------------------------

### Evaluate Text Classification Workflow

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/text-classification.md

Executes a full evaluation pipeline including loading word embeddings and initializing the TxtCls classifier.

```python
#!/usr/bin/env python3
import sys
from gensim.models import KeyedVectors
from txtcls import TxtCls

# Configuration
embedding_path = 'indicnlp.v1.hi.bin'
dataset_dir = 'indicnlp-news-articles'
lang = 'hi'

# Load embeddings
print('Loading embeddings...')
emb = KeyedVectors.load_word2vec_format(
    embedding_path,
    binary=False,
    encoding='utf8'
)

# Initialize classifier
print(f'Initializing classifier for {lang}...')
classifier = TxtCls(data_dir=dataset_dir, lang=lang)

# Evaluate
print('Evaluating...')
accuracy = classifier.evaluate(emb)
print(f'Classification Accuracy: {accuracy:.4f}')
```

--------------------------------

### Run Word Similarity Evaluation via CLI

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-similarity.md

Executes the evaluation script using a specified embedding file and similarity database.

```bash
python scripts/word_similarity/wordsim.py <embedding_file> <similarity_database> [max_vocabulary]
```

```bash
python scripts/word_similarity/wordsim.py \
    indicnlp.v1.hi.vec \
    iiith_wordsim_hi.txt \
    200000
```

--------------------------------

### Run Evaluation Workflow

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/example-workflows.md

Command to execute the classification evaluation script.

```bash
python evaluate_classification.py
```

--------------------------------

### Evaluating Embeddings with score_analogy

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

Demonstrates how to invoke the evaluation function and calculate the average accuracy from the returned dictionary.

```python
accuracies = score_analogy(
    analogy_fname='questions-words-hi.txt',
    embeddings_path='indicnlp.v1.hi.bin',
    lang='hi',
    emb_dim=300,
    max_vocab=200000,
    lower=True,
    cuda=True
)

avg_score = np.mean(list(accuracies.values()))
print(f'Average accuracy across categories: {avg_score:.4f}')
```

--------------------------------

### FAISS Index Initialization

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

The line in txtcls.py where the FAISS index is initialized with a hardcoded dimension.

```python
index = faiss.IndexFlatL2(dim)  # dim = 300 (hardcoded)
```

--------------------------------

### Build Word-to-Index Mapping

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-similarity.md

Internal function for creating a word-to-index dictionary mapping for embedding matrices.

```python
def build_w2i(word_list)
```

--------------------------------

### Train FastText word embeddings

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/README.md

Executes the FastText skipgram model with specified hyperparameters.

```bash
$FASTTEXT_HOME/build/fasttext skipgram \
	-epoch 10 -thread 30 -ws 5 -neg 10    -minCount 5 -dim 300 \
	-input $mono_path \
	-output $output_emb_prefix
```

--------------------------------

### Directory Structure Requirement

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Expected directory layout for the IndicNLP News Articles dataset.

```text
data_dir/
  {lang}/
    {lang}-train.csv
    {lang}-test.csv
```

--------------------------------

### Run Text Classification via CLI

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/text-classification.md

Executes the classification script using specified embedding paths, data directories, and language codes.

```bash
python scripts/txtcls.py --emb_path <embedding_file> --data_dir <dataset_dir> --lang <language_code>
```

```bash
python scripts/txtcls.py \
    --emb_path indicnlp.v1.hi.bin \
    --data_dir /path/to/indicnlp-news-articles \
    --lang hi
```

--------------------------------

### score_analogy()

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

Evaluates word embeddings on a word analogy task by loading embeddings and calculating category-wise accuracy scores.

```APIDOC
## score_analogy(analogy_fname, embeddings_path, lang, emb_dim, max_vocab=200000, lower=True, cuda=True)

### Description
High-level interface to evaluate embeddings on word analogy task. Handles embedding loading and setup.

### Parameters
- **analogy_fname** (str) - Required - Path to analogy file.
- **embeddings_path** (str) - Required - Path to embedding file (.bin or .vec format).
- **lang** (str) - Required - Language code for source embeddings.
- **emb_dim** (int) - Required - Embedding dimension.
- **max_vocab** (int) - Optional - Maximum vocabulary size to load (Default: 200000).
- **lower** (bool) - Optional - Convert analogy words to lowercase for lookup (Default: True).
- **cuda** (bool) - Optional - Use GPU for computation if available (Default: True).

### Return Type
- **dict[str, float]** - Dictionary mapping category names to accuracy scores.

### Example Usage
```python
accuracies = score_analogy(
    analogy_fname='questions-words-hi.txt',
    embeddings_path='indicnlp.v1.hi.bin',
    lang='hi',
    emb_dim=300,
    max_vocab=200000,
    lower=True,
    cuda=True
)
```
```

--------------------------------

### Evaluate embeddings on word similarity

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/README.md

Run the word similarity evaluation script using a .vec embedding file and a word similarity dataset.

```bash
python scripts/word_similarity/wordsim.py \
    indicnlp.v1.hi.vec \
    iiith_wordsim_hi.txt \
    200000
```

--------------------------------

### Reduce vocabulary size for embedding loading

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/errors.md

Mitigate slow loading times by limiting the maximum vocabulary size.

```python
# Reduce vocabulary size
max_vocab = 100000  # vs 200000
```

--------------------------------

### Facebook Word Analogy Dataset Format

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/word-analogy.md

The dataset uses a colon-prefixed category header followed by space-separated word quadruplets.

```text
: capital-common-countries
Athens Greece Baghdad Iraq
Abuja Nigeria Algiers Algeria
...

: capital-world
Abuja Nigeria Accra Ghana
...

: currency
Algeria dinar Angola kwanza
...
```

--------------------------------

### Configure Text Classification Memory

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/configuration.md

Note that embedding dimensions and k-values are hardcoded and impact FAISS index memory consumption.

```python
# k=4 hardcoded; cannot be changed without code modification
# embedding_dim=300 hardcoded
# FAISS index: O(n_train * 300) memory
```

--------------------------------

### load_data()

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/text-classification.md

Loads a CSV file containing labeled documents.

```APIDOC
## load_data(fname)

### Description
Loads a CSV file with [label, text] columns.

### Parameters
- **fname** (str) - Required - Path to CSV file.

### Return Type
- **list[list[str]]** - List of rows where each row is [label_string, document_text].
```

--------------------------------

### process_dataset()

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/text-classification.md

Converts raw dataset into a matrix of document embeddings with integer-encoded labels.

```APIDOC
## process_dataset(data, emb)

### Description
Encodes labels and converts text to vectors, returning a combined numpy array.

### Parameters
- **data** (numpy.ndarray) - Required - Array of shape (n_samples, 2) with [label_string, text].
- **emb** (gensim.models.KeyedVectors) - Required - Loaded embedding model.

### Return Type
- **numpy.ndarray** - Array of shape (n_samples, 301) containing label IDs and 300-dim embeddings.
```

--------------------------------

### write(words, matrix, file)

Source: https://github.com/ai4bharat/indicnlp_corpus/blob/master/_autodocs/api-reference/embeddings-module.md

Writes word embeddings to a file in the FastText text format.

```APIDOC
## write(words, matrix, file)

### Description
Writes word embeddings to a FastText text format file. The output includes a header line with vocabulary size and dimension, followed by one word and its vector per line.

### Parameters
- **words** (list[str]) - Required - List of word strings; must have length equal to matrix rows
- **matrix** (numpy.ndarray or array-like) - Required - 2D array of shape (n_words, embedding_dim) containing word vectors
- **file** (file object) - Required - Open file handle in write mode; must support text output

### Return Type
None

### Example Usage
```python
import numpy as np

words = ['word1', 'word2', 'word3']
vectors = np.random.randn(3, 300)

with open('output.vec', 'w') as f:
    write(words, vectors, f)
```
```