Vakyansh

https://open-speech-ekstep.github.io/
Admin
Vakyansh is an open-source platform providing speech recognition and text-to-speech models,...
Tokens:44,917
Snippets:396
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install and Start Development Server

Source: https://open-speech-ekstep.github.io/crowdsource_platform

Commands to install project dependencies and launch the local development server on http://localhost:8080.

```bash
npm install

npm run dev

# Local dev server will automatically starts on http://localhost:8080
```

--------------------------------

### Installation and Setup for vakyansh-wav2vec2-experimentation

Source: https://open-speech-ekstep.github.io/model_training

Installation and setup commands for the vakyansh-wav2vec2-experimentation repository. Creates a conda environment with Python 3.7, installs required packages, sets up fairseq from the ekstep-wav2vec2 branch, and builds KenLM and wav2letter dependencies. Note that fairseq should be installed outside the vakyansh repo and USE_CUDA=0 is set for CPU.

```bash
git clone https://github.com/Open-Speech-EkStep/vakyansh-wav2vec2-experimentation.git

conda create --name <env_name> python=3.7
conda activate <env_name>
cd vakyansh-wav2vec2-experimentation

### Packages
pip install packaging soundfile swifter
pip install -r requirements.txt
pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

### For fairseq setup(fairseq should be installed outside vakyansh-wav2vec2-experimentation repo)
cd ..
git clone -b ekstep-wav2vec2 https://github.com/Open-Speech-EkStep/fairseq.git
cd fairseq
pip install -e .

### install other libraries

### For Kenlm, openblas

cd ..
sudo apt-get install liblzma-dev libbz2-dev libzstd-dev libsndfile1-dev libopenblas-dev libfftw3-dev libgflags-dev libgoogle-glog-dev
sudo apt install build-essential cmake libboost-system-dev libboost-thread-dev libboost-program-options-dev libboost-test-dev libeigen3-dev zlib1g-dev libbz2-dev liblzma-dev


git clone https://github.com/kpu/kenlm.git
cd kenlm
mkdir -p build && cd build
cmake .. 
make -j 16
cd ..
export KENLM_ROOT_DIR=$PWD
export USE_CUDA=0 ## for cpu
cd ..


### wav2letter
git clone -b v0.2 https://github.com/facebookresearch/wav2letter.git
cd wav2letter
git checkout b1d1f89f586120a978a4666cffd45c55f0a2e564
cd bindings/python
pip install -e .
```

--------------------------------

### Set up conda environment and install requirements

Source: https://open-speech-ekstep.github.io/tts_model_training

Create and activate a conda virtual environment with Python 3.7, then install the project dependencies from requirements.txt.

```bash
cd ./vakyansh-tts
conda create --name <env_name> python=3.7
conda activate <env_name>
pip install -r requirements.txt

```

--------------------------------

### Start Finetuning

Source: https://open-speech-ekstep.github.io/model_training

Command to initiate the finetuning process after configuring parameters.

```bash
$ bash start_finetuning.sh
```

--------------------------------

### Install indic-punct repository

Source: https://open-speech-ekstep.github.io/punctuation_model_training

Commands to clone the repository, install dependencies, and set up the package for local use.

```bash
git clone https://github.com/Open-Speech-EkStep/indic-punct.git
cd indic-punct
bash install.sh
python setup.py bdist_wheel
pip install -e .
```

--------------------------------

### Initialize gcloud

Source: https://open-speech-ekstep.github.io/data_collection

Initializes the Google Cloud SDK after installation.

```bash
gcloud init
```

--------------------------------

### Run HiFi-GAN Training

Source: https://open-speech-ekstep.github.io/tts_model_training

Start the training process.

```bash
cd scripts/hifi/
bash train_hifi.sh
```

--------------------------------

### Install Node.js dependencies

Source: https://open-speech-ekstep.github.io/crowdsource_platform

Command to install the required Node.js packages.

```bash
npm install

```

--------------------------------

### Clone repository and install requirements

Source: https://open-speech-ekstep.github.io/intelligent_data_pipelines

Clone the repository and install Python dependencies to set up the Intelligent Data Pipeline framework.

```bash
git clone git@github.com:Open-Speech-EkStep/audio-to-speech-pipeline.git

```

```bash
pip install -r requirements.txt

```

--------------------------------

### Install Python requirements

Source: https://open-speech-ekstep.github.io/data_collection

Installs the Python dependencies listed in requirements.txt.

```bash
pip install -r requirements.txt
```

--------------------------------

### Run application services

Source: https://open-speech-ekstep.github.io/crowdsource_platform

Commands to start the application using default, Google Cloud, or AWS configurations.

```bash
npm run
```

```bash
npm run gcp
```

```bash
npm run aws
```

--------------------------------

### Install Node.js on Linux (Ubuntu)

Source: https://open-speech-ekstep.github.io/crowdsource_platform

Command to install Node.js on Linux-based operating systems, preferred Ubuntu.

```bash
sudo apt-get install nodejs

```

--------------------------------

### Install tts_infer Package

Source: https://open-speech-ekstep.github.io/tts_model_training

Commands to clone the repository, install dependencies, and download transliteration models.

```bash
git clone https://github.com/Open-Speech-EkStep/vakyansh-tts
cd vakyansh-tts
bash install.sh
python setup.py bdist_wheel
pip install -e .
cd tts_infer
wget https://storage.googleapis.com/vakyansh-open-models/translit_models.zip && unzip -q translit_models.zip
```

--------------------------------

### Start Pretraining

Source: https://open-speech-ekstep.github.io/model_training

Command to initiate the base pretraining process after configuring parameters.

```bash
$ bash start_pretraining_base.sh
```

--------------------------------

### Start model training

Source: https://open-speech-ekstep.github.io/language_identification

Initiates the training process using parameters defined in train_config.yml.

```bash
python train.py
```

--------------------------------

### Run Gradio Inference

Source: https://open-speech-ekstep.github.io/tts_model_training

Start the Gradio interface.

```bash
cd scripts/inference/
bash gradio.sh
```

--------------------------------

### Install ffmpeg on Linux

Source: https://open-speech-ekstep.github.io/data_collection

Command to install the ffmpeg library on Linux-based operating systems, with Ubuntu being the preferred environment.

```bash
sudo apt-get install ffmpeg
```

--------------------------------

### Install NVIDIA Apex (commit 37cdaf4) for mixed-precision training

Source: https://open-speech-ekstep.github.io/tts_model_training

Install NVIDIA Apex at commit 37cdaf4 for mixed-precision training. This is required for glow-tts training.

```bash
cd ..
git clone https://github.com/NVIDIA/apex
cd apex
git checkout 37cdaf4
pip install -v --disable-pip-version-check --no-cache-dir ./
cd ../vakyansh-tts

```

--------------------------------

### Install Node.js on macOS

Source: https://open-speech-ekstep.github.io/crowdsource_platform

Command to install Node.js on macOS using Homebrew.

```bash
brew install node

```

--------------------------------

### Run Glow-TTS Training

Source: https://open-speech-ekstep.github.io/tts_model_training

Commands to start the spectrogram training process.

```bash
cd scripts/glow/
bash train_glow.sh
```

--------------------------------

### Install ffmpeg on macOS

Source: https://open-speech-ekstep.github.io/data_collection

Command to install the ffmpeg library on macOS using Homebrew.

```bash
brew install ffmpeg
```

--------------------------------

### Modify URL and language in App.js

Source: https://open-speech-ekstep.github.io/asr_streaming_service

Example lines from src/App.js in the react-example, showing how to set the proxy service URL and language code in the handleStart() method.

```javascript
  const url = 'http://localhost:9009'; // url of the proxy service
  const language = 'hi'; // this can be en, gu depends on what models you have hosted.

```

--------------------------------

### Start Inference

Source: https://open-speech-ekstep.github.io/model_training

Command to initiate the inference process after configuring paths in infer.sh.

```bash
$ bash infer.sh
```

--------------------------------

### Docker Run Command

Source: https://open-speech-ekstep.github.io/tts_model_api

Command to start the TTS API server container with GPU support and mounted model volumes.

```bash
docker run -itd -p 5000:5000 --gpus all --env languages='["hi","ml"]' -v <local path>/tts_models/:/opt/text_to_speech_open_api/deployed_models/ -v <local path>/translit_models/:/opt/text_to_speech_open_api/vakyansh-tts/src/glow_tts/tts_infer/translit_models/ gcr.io/ekstepspeechrecognition/text_to_speech_open_api:2.1.4
```

--------------------------------

### Build Monotonic Alignment Search code with install.sh

Source: https://open-speech-ekstep.github.io/tts_model_training

Build the Monotonic Alignment Search code (Cython) by running the install script.

```bash
bash install.sh

```

--------------------------------

### Prepare Data for Finetuning

Source: https://open-speech-ekstep.github.io/model_training

Commands to navigate to the finetuning directory and execute the data preparation script.

```bash
$ cd scripts/finetuning
$ bash prepare_data.sh
```

--------------------------------

### Clone the infrastructure repository

Source: https://open-speech-ekstep.github.io/intelligent_data_pipelines

Initial step to download the infrastructure configuration files.

```bash
git clone https://github.com/Open-Speech-EkStep/ekstep-deep-speech-infra.git
```

--------------------------------

### Data Filter Configuration Example

Source: https://open-speech-ekstep.github.io/intelligent_data_pipelines

Example configuration for the data_filter_config Airflow variable, demonstrating filter modes and criteria for different data sources.

```json
"data_filter_config": {
    "test_source1": {
      "language": "hindi",
      "file_mode": "n",
      "data_set": "train",
      "file_path": "data/audiotospeech/raw/download/duplicate/test_source1.csv",
      "filter": {
        "by_snr": {
          "lte": 75,
          "gte": 15
        },
        "by_duration": 2,
        "with_randomness": "true"
      }
    },
    "test_source2": {
      "language": "hindi",
      "file_mode": "n",
      "data_set": "train",
      "file_path": "data/audiotospeech/raw/download/duplicate/test_source2.csv",
      "filter": {
        "by_speaker": {
          "lte_per_speaker_duration": 60,
          "gte_per_speaker_duration": 0,
          "with_threshold": 0
        },
        "by_duration": 2
    }
  }
}
```

--------------------------------

### Prepare Data for Pretraining

Source: https://open-speech-ekstep.github.io/model_training

Commands to navigate to the pretraining directory and execute the data preparation script.

```bash
$ cd scripts/pretraining
$ bash prepare_data.sh
```

--------------------------------

### Deploy API Components via CLI

Source: https://open-speech-ekstep.github.io/api_deployment

Command to manually trigger the deployment of the proxy, envoy, and API server language pods from a bastion or window node.

```bash
python3 deploy.py --namespace <kubernetes namespace> --api-updated True --image-name gcr.io/ekstepspeechrecognition/speech_recognition_model_api --image-version <latest image version>
```

--------------------------------

### Run FastAPI Inference

Source: https://open-speech-ekstep.github.io/tts_model_training

Start the FastAPI service after configuring vakyansh-tts/scripts/inference/api.sh.

```bash
cd scripts/inference/
bash api.sh
```

--------------------------------

### Initialize Terraform modules

Source: https://open-speech-ekstep.github.io/intelligent_data_pipelines

Prepares the working directory for Terraform operations.

```hcl
terraform init
```

--------------------------------

### Change to project directory

Source: https://open-speech-ekstep.github.io/data_collection

Navigates into the cloned data-acquisition-pipeline directory.

```bash
cd data-acquisition-pipeline

```

--------------------------------

### Use tts_infer Library

Source: https://open-speech-ekstep.github.io/tts_model_training

Example Python code for text-to-speech conversion using the tts_infer package.

```python
from tts_infer.tts import TextToMel, MelToWav
from tts_infer.transliterate import XlitEngine
from tts_infer.num_to_word_on_sent import normalize_nums

import re
from scipy.io.wavfile import write

text_to_mel = TextToMel(glow_model_dir='/path/to/glow-tts/checkpoint/dir', device='cuda')
mel_to_wav = MelToWav(hifi_model_dir='/path/to/hifi/checkpoint/dir', device='cuda')

def translit(text, lang):
    reg = re.compile(r'[a-zA-Z]')
    engine = XlitEngine(lang)
    words = [engine.translit_word(word, topk=1)[lang][0] if reg.match(word) else word for word in text.split()]
    updated_sent = ' '.join(words)
    return updated_sent

def run_tts(text, lang):
    text = text.replace('।', '.') # only for hindi models
    text_num_to_word = normalize_nums(text, lang) # converting numbers to words in lang
    text_num_to_word_and_transliterated = translit(text_num_to_word, lang) # transliterating english words to lang

    mel = text_to_mel.generate_mel(text_num_to_word_and_transliterated)
    audio, sr = mel_to_wav.generate_wav(mel)
    write(filename='temp.wav', rate=sr, data=audio) # for saving wav file, if needed
    return (sr, audio)
```

--------------------------------

### Run database migrations

Source: https://open-speech-ekstep.github.io/crowdsource_platform

Commands to apply or rollback database migrations.

```bash
db-migrate up
```

```json
migations/config/migration_config.json
```

```bash
DATABASE_URL=postgresql://${DB_USER}:${DB_PASS}@${DB_HOST}/${DB_NAME}
```

```bash
db-migrate create add-new-table
```

```bash
db-migrate down
```

--------------------------------

### CSV structure for YouTube file mode

Source: https://open-speech-ekstep.github.io/data_collection

Example CSV format with columns video_url, speaker_name, and speaker_gender.

```csv
 video_url,speaker_name,speaker_gender
https://www.youtube.com/watch?v=K1vW_ZikA5o,Ram_Singh,male
https://www.youtube.com/watch?v=o82HIOgozi8,John_Doe,male
...
```

--------------------------------

### Run datacollector_bing spider

Source: https://open-speech-ekstep.github.io/data_collection

Command to start the Bing data collection spider. Configure web_crawl_config.json before running.

```bash
scrapy crawl datacollector_bing
```

--------------------------------

### Run HiFi-GAN Data Preparation

Source: https://open-speech-ekstep.github.io/tts_model_training

Execute the data preparation script.

```bash
cd scripts/hifi/
bash prepare_data.sh
```

--------------------------------

### TTS API Response Example

Source: https://open-speech-ekstep.github.io/tts_model_api

A sample JSON response containing the synthesized audio content in base64 encoding.

```json
{
  "audio": [
    {
      "audioContent": "UklGRiS4AgBXQVZFZm10"
    }
  ],
  "config": {
    "language": {
      "sourceLanguage": "hi"
    },
    "audioFormat": "wav",
    "encoding": "base64",
    "samplingRate": 22050
  }
}
```

--------------------------------

### Connect to database via Cloud SQL Proxy

Source: https://open-speech-ekstep.github.io/intelligent_data_pipelines

Sets up a local proxy to connect to the remote database instance.

```bash
./cloud_sql_proxy -dir=./cloudsql -instances=<project-id>:<zone>:<db-instance-name>=tcp:5432
```

--------------------------------

### Punctuate text using Punctuation class

Source: https://open-speech-ekstep.github.io/punctuation_model_training

Example of loading a language-specific model and applying punctuation to a list of sentences.

```python
from punctuate.punctuate_text import Punctuation
hindi = Punctuation('hi') #loads model in memory
hindi.punctuate_text(["इस श्रेणी में केवल निम्नलिखित उपश्रेणी है", "मेहुल को भारत को सौंप दिया जाए"])
```

--------------------------------

### Run data manifest creation

Source: https://open-speech-ekstep.github.io/language_identification

Executes the script to generate train and valid CSV files from audio folders.

```bash
python create_manifest.py
```
Tokens:
Raw