### Configuring Gradio Examples for Media Conversion Function Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet sets up `gr.Examples` to provide predefined inputs for the `SoniTr.batch_multilingual_media_conversion` function. It includes a list of example parameters, such as video paths, model sizes, and language selections, facilitating easy testing and demonstration of the application's capabilities. ```Python gr.Examples( examples=[ [ ["./assets/Video_main.mp4"], "", "", "", False, whisper_model_default, 4, com_t_default, "Spanish (es)", "English (en)", 1, 2, "en-CA-ClaraNeural-Female", "en-AU-WilliamNeural-Male", ], ], # no update fn=SoniTr.batch_multilingual_media_conversion, inputs=[ video_input, blink_input, directory_input, HFKEY, PREVIEW, WHISPER_MODEL_SIZE, batch_size, ``` -------------------------------- ### Installing SoniTranslate and Core Dependencies (Shell) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet clones the SoniTranslate repository, navigates into it, uninstalls conflicting packages, installs necessary system tools like `git-lfs`, and updates `pip`. It then modifies `requirements.txt` files to point to specific CUDA versions and `fairseq` builds, and installs all base and extra Python dependencies. It also includes conditional installations for `piper-tts` and `coqui-xtts` based on boolean flags, and installs `libcudnn8` for GPU support. ```Shell !git clone https://github.com/r3gm/SoniTranslate.git %cd SoniTranslate !pip uninstall chex pandas-stubs ibis-framework albumentations albucore -y -q !python -m pip install -q pip==23.1.2 !apt install git-lfs !git lfs install !sed -i 's|git+https://github.com/R3gm/whisperX.git@cuda_11_8|git+https://github.com/R3gm/whisperX.git@cuda_12_x|' requirements_base.txt !sed -i 's|fairseq==0.12.2 |https://github.com/liyaodev/fairseq/releases/download/v0.12.3.1/fairseq-0.12.3.1-cp311-cp311-linux_x86_64.whl|' requirements_extra.txt !pip install -q -r requirements_base.txt !pip install -q -r requirements_extra.txt !pip install -q ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/simple/ Install_PIPER_TTS = True # @param {type:"boolean"} if Install_PIPER_TTS: !pip install -q piper-tts==1.2.0 Install_Coqui_XTTS = True # @param {type:"boolean"} if Install_Coqui_XTTS: !pip install -q -r requirements_xtts.txt !pip install -q TTS==0.21.1 --no-deps !sudo apt install -y libcudnn8 ``` -------------------------------- ### Running SoniTranslate Application (app_rvc.py) Source: https://github.com/r3gm/sonitranslate/blob/main/docs/windows_install.md This command executes the `app_rvc.py` script, which launches the SoniTranslate web interface. Once started, the application will be accessible via a local URL (e.g., `http://127.0.0.1:7860`) in a web browser. ```Shell python app_rvc.py ``` -------------------------------- ### Initiating Audio Translation in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet prepares for the translation of the processed audio segments. It checks for a cached translation task. If not found, it updates the progress display and determines the source language for translation, prioritizing the alignment language if available, otherwise falling back to the initial source language. The actual translation logic is implied to follow this setup. ```Python if not self.task_in_cache("translate", [ TRANSLATE_AUDIO_TO, translate_process ], { "result_diarize": self.result_diarize }): prog_disp("Translating...", 0.70, is_gui, progress=progress) lang_source = ( self.align_language if self.align_language else SOURCE_LANGUAGE ) ``` -------------------------------- ### Installing Coqui XTTS Optional Packages Source: https://github.com/r3gm/sonitranslate/blob/main/docs/windows_install.md These commands install the necessary packages for Coqui XTTS, a text-to-speech model. It first installs dependencies from `requirements_xtts.txt` and then installs a specific version of TTS without its dependencies to avoid conflicts. ```Shell pip install -q -r requirements_xtts.txt pip install -q TTS==0.21.1 --no-deps ``` -------------------------------- ### Creating Argument Parser (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This function initializes an `argparse.ArgumentParser` to handle command-line arguments for the application. It defines arguments for theme, public URL sharing, displaying logs in the GUI, verbosity level for logging, interface language, and enabling CPU mode, each with default values and help text. ```python def create_parser(): parser = argparse.ArgumentParser( formatter_class=argparse.ArgumentDefaultsHelpFormatter ) parser.add_argument( "--theme", type=str, default="Taithrah/Minimal", help=( "Specify the theme; find themes in " "https://huggingface.co/spaces/gradio/theme-gallery;" " Example: --theme aliabid94/new-theme" ), ) parser.add_argument( "--public_url", action="store_true", default=False, help="Enable public link", ) parser.add_argument( "--logs_in_gui", action="store_true", default=False, help="Displays the operations performed in Logs", ) parser.add_argument( "--verbosity_level", type=str, default="info", help=( "Set logger verbosity level: " "debug, info, warning, error, or critical" ), ) parser.add_argument( "--language", type=str, default="english", help=" Select the language of the interface: english, spanish", ) parser.add_argument( "--cpu_mode", action="store_true", default=False, help="Enable CPU mode to run the program without utilizing GPU acceleration.", ) return parser ``` -------------------------------- ### Configuring Gradio Button and File Output for Video Translation Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This section defines a primary button for initiating video translation and a file output component to display the results. The `video_output` component is configured to handle multiple files and is set to be non-interactive, serving as a display for translated media. ```Python with gr.Row(): video_button = gr.Button( lg_conf["button_translate"], variant="primary", ) with gr.Row(): video_output = gr.File( label=lg_conf["output_result_label"], file_count="multiple", interactive=False, ) # gr.Video() ``` -------------------------------- ### Creating Gradio GUI for SoniTranslate (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This function initializes and structures the Gradio-based graphical user interface for the SoniTranslate application. It sets up tabs, rows, columns, and various input components like dropdowns, file inputs, and textboxes, along with their associated labels and information. ```Python def create_gui(theme, logs_in_gui=False): with gr.Blocks(theme=theme) as app: gr.Markdown(title) gr.Markdown(lg_conf["description"]) with gr.Tab(lg_conf["tab_translate"]): with gr.Row(): with gr.Column(): input_data_type = gr.Dropdown( ["SUBMIT VIDEO", "URL", "Find Video Path"], value="SUBMIT VIDEO", label=lg_conf["video_source"], ) def swap_visibility(data_type): if data_type == "URL": return ( gr.update(visible=False ``` -------------------------------- ### Initializing Configuration Storage List (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This line initializes an empty list named `configs_storage`. This list is intended to store references to Gradio UI components (specifically 'tag', 'model', and 'index' textboxes) for each TTS speaker configuration, allowing programmatic access to their values. ```Python configs_storage = [] ``` -------------------------------- ### Initializing SoniTranslate Application in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb The main class for the SoniTranslate application, inheriting from `SoniTrCache`. It initializes the processing device ('cpu' or 'cuda') based on the `cpu_mode` flag and CUDA availability, and sets up initial attributes for diarization results and alignment language. ```Python class SoniTranslate(SoniTrCache): def __init__(self, cpu_mode=False): super().__init__() if cpu_mode: os.environ["SONITR_DEVICE"] = "cpu" else: os.environ["SONITR_DEVICE"] = ( "cuda" if torch.cuda.is_available() else "cpu" ) self.device = os.environ.get("SONITR_DEVICE") self.result_diarize = None self.align_language = None ``` -------------------------------- ### Configuring Demo Mode Settings (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This code block checks for a 'SET_LIMIT' value in the 'DEMO' environment variable. If found, it activates a preview mode, sets the audio mixing method, and selects a 'medium' transcriber model, primarily to limit generation time for CPU-based demos. ```python if "SET_LIMIT" == os.getenv("DEMO"): preview = True mix_method_audio = "Adjusting volumes and mixing audio" transcriber_model = "medium" logger.info( "DEMO; set preview=True; Generation is limited to " "10 seconds to prevent CPU errors. No limitations with GPU.\n" "DEMO; set Adjusting volumes and mixing audio\n" "DEMO; set whisper model to medium" ) ``` -------------------------------- ### Chaining Gradio Event Handlers (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet demonstrates chaining Gradio event handlers using the `.then()` method. It specifies that after a preceding event (not shown), the `play_sound_alert` function should be called with `play_sound_gui` as input, updating the `sound_alert_notification` component. ```python ).then( play_sound_alert, [play_sound_gui], [sound_alert_notification] ) ``` -------------------------------- ### Defining Pitch Algorithm Options (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This Python list, `PITCH_ALGO_OPT`, defines the available options for pitch detection algorithms. These options are typically used in audio processing or voice synthesis applications to specify the method for analyzing and manipulating pitch. ```Python PITCH_ALGO_OPT = [ "pm", "harvest", "crepe", "rmvpe", "rmvpe+", ] ``` -------------------------------- ### Displaying Progress in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb A utility function for displaying progress messages. It logs the message using `logger.info` and, if a GUI is active (`is_gui` is True), updates a progress bar with the given percentage and description using the provided `progress` callback. ```Python def prog_disp(msg, percent, is_gui, progress=None): logger.info(msg) if is_gui: progress(percent, desc=msg) ``` -------------------------------- ### Defining GUI Title (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This line defines an HTML string that serves as the main title for the SoniTranslate graphical user interface. It uses HTML tags for centering, bolding, font size, and includes emojis for visual appeal. ```Python title = "
📽️ SoniTranslate 🈷️
" ``` -------------------------------- ### Setting Up Main Voiceless Track Checkbox (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet creates a Gradio checkbox to enable or disable the main voiceless track feature. It provides a label and information text from `lg_conf`, allowing users to control whether a voiceless track is used in the audio processing. ```Python main_voiceless_track = gr.Checkbox( label=lg_conf["voiceless_tk_label"], info=lg_conf["voiceless_tk_info"], ) ``` -------------------------------- ### Handling Download and Model Updates in Gradio (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet defines a two-step action for a Gradio download button. First, it calls `download_list` with URL links, then it updates various model and index elements in the UI based on `configs_storage`. The `queue=False` parameter is used for the initial download action. ```Python download_button.click( download_list, [url_links], [download_finish], queue=False ).then( update_models, [], [ elem["model"] for elem in configs_storage ] + [model_test] + [ elem["index"] for elem in configs_storage ] + [index_test], ) ``` -------------------------------- ### Setting Up RVC Test UI (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This code block defines a Gradio UI section for testing Real-time Voice Conversion (RVC). It includes a multi-line textbox for user input, a dropdown for selecting a TTS voice (populated from `SoniTr.tts_info.list_edge`), and additional configuration components for model and index settings. ```Python with gr.Column(): with gr.Accordion("Test R.V.C.", open=False): with gr.Row(variant="compact"): text_test = gr.Textbox( label="Text", value="This is an example", info="write a text", placeholder="...", lines=5, ) with gr.Column(): tts_test = gr.Dropdown( sorted(SoniTr.tts_info.list_edge), value="en-GB-ThomasNeural-Male", label="TTS", visible=True, interactive=True, ) model_test = model_conf() index_test = index_conf() ``` -------------------------------- ### Configuring Gradio Index Dropdown (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb The `index_conf` function generates a Gradio `gr.Dropdown` component. This dropdown allows users to select an 'Index' from the `index_path` list, with no default selection. It is configured to be visible and interactive, enabling the selection of an index for operations like voice conversion or synthesis. ```Python def index_conf(): return gr.Dropdown( index_path, value=None, label="Index", visible=True, interactive=True, ) ``` -------------------------------- ### Setting Output Type and Name in Gradio (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet creates Gradio UI components for defining the output settings. It includes a dropdown for selecting the main output file type and a textbox for specifying the desired name for the video output, providing control over the final output format and filename. ```Python main_output_type = gr.Dropdown( OUTPUT_TYPE_OPTIONS, value=OUTPUT_TYPE_OPTIONS[0], label=lg_conf["out_type_label"], ) VIDEO_OUTPUT_NAME = gr.Textbox( label=lg_conf["out_name_label"], ) ``` -------------------------------- ### Installing Git via Anaconda on Windows Source: https://github.com/r3gm/sonitranslate/blob/main/docs/windows_install.md This command installs Git using the Conda package manager, which is part of Anaconda or Miniconda. It specifies the 'anaconda' channel and automatically confirms the installation. This is an alternative method to installing Git manually for SoniTranslate prerequisites. ```Shell conda install -c anaconda git -y ``` -------------------------------- ### Preparing Audio Mix File in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet prepares for the final audio mixing process. If the mixing task is not already cached, it removes any existing `mix_audio_file` to ensure a fresh output. This step is a prerequisite for combining the original/voiceless audio with the translated audio based on specified mixing parameters. ```python if not self.task_in_cache("mix_aud", [ mix_method_audio, volume_original_audio, volume_translated_audio, voiceless_track ], {}): # TYPE MIX AUDIO remove_files(mix_audio_file) ``` -------------------------------- ### Configuring Diarization and Translation Processes in Gradio (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet defines Gradio dropdowns for selecting diarization models and translation processes. It populates the diarization dropdown with available `pyannote` models and the translation dropdown with predefined options, enabling users to choose the desired processing pipelines. ```Python pyannote_models_list = list( diarization_models.keys() ) diarization_process_dropdown = gr.Dropdown( pyannote_models_list, value=pyannote_models_list[1], label=lg_conf["diarization_label"], ) translate_process_dropdown = gr.Dropdown( TRANSLATION_PROCESS_OPTIONS, value=TRANSLATION_PROCESS_OPTIONS[0], label=lg_conf["tr_process_label"], ) ``` -------------------------------- ### Configuring Audio Acceleration Slider (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet defines a Gradio slider for controlling audio acceleration. It allows users to adjust the acceleration rate within a specified range (1.0 to 2.5) with a step of 0.1, providing interactive control over audio processing speed. ```Python audio_accelerate = gr.Slider( label=lg_conf["acc_max_label"], value=1.9, step=0.1, minimum=1.0, maximum=2.5, visible=True, interactive=True, info=lg_conf["acc_max_info"], ) ``` -------------------------------- ### Defining Gradio Button Configuration Function (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This Python function, `button_conf`, creates and returns a Gradio Button component. It dynamically sets the button's label by combining a localized string from `lg_conf` with the provided `tts_name`, and applies a 'primary' variant for styling. ```Python def button_conf(tts_name): return gr.Button( lg_conf["cv_button_apply"]+" "+tts_name, variant="primary", ) ``` -------------------------------- ### Separating Sounds and Generating Output Files (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb If the `output_type` includes 'sound', this snippet initiates sound separation on `base_audio_wav`. It then iterates through the separated outputs, generates unique filenames using `media_out`, and collects them into a list, finally returning the list of generated sound files. ```python if "sound" in output_type: prog_disp( "Separating sounds in the file...", 0.50, is_gui, progress=progress ) separate_out = sound_separate(base_audio_wav, output_type) final_outputs = [] for out in separate_out: final_name = media_out( media_file, f"{get_no_ext_filename(out)}", video_output_name, "wav", file_obj=out, ) final_outputs.append(final_name) logger.info(f"Done: {str(final_outputs)}") return final_outputs ``` -------------------------------- ### Checking OpenAI API Key for Translation/Transcription in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This code verifies the availability of an OpenAI API key if GPT-based translation, OpenAI Whisper transcription, or OpenAI TTS voices are selected. It calls a `check_openai_api_key()` function, implying a dependency on external API access. ```Python if ( "gpt" in translate_process or transcriber_model == "OpenAI_API_Whisper" or "OpenAI-TTS" in tts_voice00 ): check_openai_api_key() ``` -------------------------------- ### Installing SoniTranslate Requirements and Dependencies - Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab.ipynb This snippet installs all necessary dependencies for SoniTranslate. It clones the repository, navigates into the project directory, uninstalls potentially conflicting packages, installs specific versions of `pip` and `git-lfs`, modifies `requirements.txt` files to ensure compatibility with `whisperX` and `fairseq`, and then installs all base and extra requirements. It also conditionally installs `piper-tts` and `Coqui XTTS` based on user-defined boolean flags and ensures `libcudnn8` is installed. ```Python # @title Install requirements for SoniTranslate !git clone https://github.com/r3gm/SoniTranslate.git %cd SoniTranslate !pip uninstall chex pandas-stubs ibis-framework albumentations albucore -y -q !python -m pip install -q pip==23.1.2 !apt install git-lfs !git lfs install !sed -i 's|git+https://github.com/R3gm/whisperX.git@cuda_11_8|git+https://github.com/R3gm/whisperX.git@cuda_12_x|' requirements_base.txt !sed -i 's|fairseq==0.12.2 |https://github.com/liyaodev/fairseq/releases/download/v0.12.3.1/fairseq-0.12.3.1-cp311-cp311-linux_x86_64.whl|' requirements_extra.txt !pip install -q -r requirements_base.txt !pip install -q -r requirements_extra.txt !pip install -q ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/simple/ Install_PIPER_TTS = True # @param {type:"boolean"} if Install_PIPER_TTS: !pip install -q piper-tts==1.2.0 Install_Coqui_XTTS = True # @param {type:"boolean"} if Install_Coqui_XTTS: !pip install -q -r requirements_xtts.txt !pip install -q TTS==0.21.1 --no-deps !sudo apt install -y libcudnn8 ``` -------------------------------- ### Returning Raw Media File as Output (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb When `output_type` is 'raw media', this code block generates an output file path for the raw media. It dynamically sets the output format to 'wav' for audio or 'mp4' for video, and returns the path to the original base audio or video file. ```python if output_type == "raw media": output = media_out( media_file, "raw_media", video_output_name, "wav" if is_audio_file(media_file) else "mp4", file_obj=base_audio_wav if is_audio_file(media_file) else base_video_file, ) logger.info(f"Done: {output}") return output ``` -------------------------------- ### Setting Up Acceleration Rate Regulation Checkbox (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet creates a Gradio checkbox to enable or disable acceleration rate regulation. It's initialized to `False` and provides a label and information text from the `lg_conf` dictionary, allowing users to control this specific audio processing feature. ```Python acceleration_rate_regulation_gui = gr.Checkbox( False, label=lg_conf["acc_rate_label"], info=lg_conf["acc_rate_info"], ) ``` -------------------------------- ### Application Initialization and Launch (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This main execution block orchestrates the application's startup. It parses command-line arguments (or simulated ones), sets the logging level, downloads necessary models, initializes the `SoniTranslate` core, configures the interface language, creates and queues the Gradio GUI, and finally launches the application with specified parameters. ```python if __name__ == "__main__": parser = create_parser() # args = parser.parse_args() # Simulating command-line arguments args_list = f"--theme {theme_var} --verbosity_level {verbosity_level_var} --language {interface_language_var}".split() args = parser.parse_args(args_list) set_logging_level(args.verbosity_level) for id_model in UVR_MODELS: download_manager( os.path.join(MDX_DOWNLOAD_LINK, id_model), mdxnet_models_dir ) models_path, index_path = upload_model_list() SoniTr = SoniTranslate(cpu_mode=args.cpu_mode) lg_conf = get_language_config(language_data, language=args.language) app = create_gui(args.theme, logs_in_gui=args.logs_in_gui) app.queue() from IPython.display import clear_output clear_output() app.launch( max_threads=6, # share=args.public_url, show_error=True, quiet=False, debug=True, ) ``` -------------------------------- ### Triggering Document Conversion in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet illustrates a GUI event handler for a 'docs' button click, which calls `SoniTr.multilingual_docs_conversion`. It takes inputs related to text documents, input directories, source and target languages, TTS settings, and output preferences, facilitating the conversion and translation of documents. ```Python docs_button.click( SoniTr.multilingual_docs_conversion, inputs=[ text_docs, input_docs, directory_input_docs, docs_SOURCE_LANGUAGE, docs_TRANSLATE_TO, tts_documents, docs_OUTPUT_NAME, docs_translate_process_dropdown, docs_output_type, ] ``` -------------------------------- ### Document Translation Beta Processor Hook Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This `hook_beta_processor` function orchestrates a multi-step document translation workflow. It handles page processing, text extraction, translation, text-to-speech conversion, and the application of custom voices, updating progress at each stage. It takes numerous parameters to control the translation and output process. ```Python def hook_beta_processor( self, document, tgt_lang, translate_process, ori_lang, tts, name_final_file, custom_voices, custom_voices_workers, output_type, chunk_size, width, height, start_page, end_page, bcolor, is_gui, progress ): prog_disp("Processing pages...", 0.10, is_gui, progress=progress) doc_data = doc_to_txtximg_pages(document, width, height, start_page, end_page, bcolor) result_diarize = page_data_to_segments(doc_data, 1700) prog_disp("Translating...", 0.20, is_gui, progress=progress) result_diarize["segments"] = translate_text( result_diarize["segments"], tgt_lang, translate_process, chunk_size=0, source=ori_lang, ) chunk_size = ( chunk_size if chunk_size else determine_chunk_size(tts) ) doc_data = update_page_data(result_diarize, doc_data) prog_disp("Text to speech...", 0.30, is_gui, progress=progress) result_diarize = page_data_to_segments(doc_data, chunk_size) valid_speakers = audio_segmentation_to_voice( result_diarize, tgt_lang, is_gui, tts, ) # fix format and set folder output audio_files, speakers_list = accelerate_segments( result_diarize, 1.0, valid_speakers, ) # custom voice if custom_voices: prog_disp( "Applying customized voices...", 0.60, is_gui, ``` -------------------------------- ### Initializing and Running SoniTranslate Gradio Web App (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet sets up environment variables and parameters for the SoniTranslate web application, including a Hugging Face token, UI theme, interface language, and logging verbosity. It then changes the directory and imports numerous modules from the `soni_translate` library, indicating the extensive functionality available. Finally, it defines a list of directories and creates them, preparing the file system for the application's operation. ```Python YOUR_HF_TOKEN = "" #@param {type:'string'} %env YOUR_HF_TOKEN={YOUR_HF_TOKEN} theme_var = "Taithrah/Minimal" # @param ["Taithrah/Minimal", "aliabid94/new-theme", "gstaff/xkcd", "ParityError/LimeFace", "abidlabs/pakistan", "rottenlittlecreature/Moon_Goblin", "ysharma/llamas", "gradio/dracula_revamped"] interface_language_var = "english" # @param ['afrikaans', 'arabic', 'azerbaijani', 'chinese_zh_cn', 'english', 'french', 'german', 'hindi', 'indonesian', 'italian', 'japanese', 'korean', 'marathi', 'persian', 'polish', 'portuguese', 'russian', 'spanish', 'swedish', 'turkish', 'ukrainian', 'vietnamese'] verbosity_level_var = "error" # @param ["debug", "info", "warning", "error", "critical"] #@markdown ### `The interface will appear down here 👇‍‍` %cd /content/SoniTranslate import gradio as gr from soni_translate.logging_setup import ( logger, set_logging_level, configure_logging_libs, ); configure_logging_libs() # noqa import whisperx import torch import os from soni_translate.audio_segments import create_translated_audio from soni_translate.text_to_speech import ( audio_segmentation_to_voice, edge_tts_voices_list, coqui_xtts_voices_list, piper_tts_voices_list, create_wav_file_vc, accelerate_segments, ) from soni_translate.translate_segments import ( translate_text, TRANSLATION_PROCESS_OPTIONS, DOCS_TRANSLATION_PROCESS_OPTIONS ) from soni_translate.preprocessor import ( audio_video_preprocessor, audio_preprocessor, ) from soni_translate.postprocessor import ( OUTPUT_TYPE_OPTIONS, DOCS_OUTPUT_TYPE_OPTIONS, sound_separate, get_no_ext_filename, media_out, get_subtitle_speaker, ) from soni_translate.language_configuration import ( LANGUAGES, UNIDIRECTIONAL_L_LIST, LANGUAGES_LIST, BARK_VOICES_LIST, VITS_VOICES_LIST, OPENAI_TTS_MODELS, ) from soni_translate.utils import ( remove_files, download_list, upload_model_list, download_manager, run_command, is_audio_file, is_subtitle_file, copy_files, get_valid_files, get_link_list, remove_directory_contents, ) from soni_translate.mdx_net import ( UVR_MODELS, MDX_DOWNLOAD_LINK, mdxnet_models_dir, ) from soni_translate.speech_segmentation import ( ASR_MODEL_OPTIONS, COMPUTE_TYPE_GPU, COMPUTE_TYPE_CPU, find_whisper_models, transcribe_speech, align_speech, diarize_speech, diarization_models, ) from soni_translate.text_multiformat_processor import ( BORDER_COLORS, srt_file_to_segments, document_preprocessor, determine_chunk_size, plain_text_to_segments, segments_to_plain_text, process_subtitles, linguistic_level_segments, break_aling_segments, doc_to_txtximg_pages, page_data_to_segments, update_page_data, fix_timestamps_docs, create_video_from_images, merge_video_and_audio, ) from soni_translate.languages_gui import language_data, news import copy import logging import json from pydub import AudioSegment from voice_main import ClassVoices import argparse import time import hashlib import sys directories = [ "downloads", "logs", "weights", "clean_song_output", "_XTTS_", f"audio2{os.sep}audio", "audio", "outputs", ] [ os.makedirs(directory) ``` -------------------------------- ### Removing SoniTranslate Environment and Folder Source: https://github.com/r3gm/sonitranslate/blob/main/docs/windows_install.md These commands provide a way to completely remove the SoniTranslate setup. It first deactivates the environment and then removes the 'sonitr' Conda environment, allowing for a fresh installation if needed. ```Shell conda deactivate conda env remove -n sonitr ``` -------------------------------- ### Configuring Gradio Pitch Algorithm Dropdown (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb The `pitch_algo_conf` function creates and returns a Gradio `gr.Dropdown` component. This dropdown allows users to select a pitch algorithm from the `PITCH_ALGO_OPT` list, with 'rmvpe' (`PITCH_ALGO_OPT[3]`) set as the default value. It's labeled 'Pitch algorithm' and is visible and interactive. ```Python def pitch_algo_conf(): return gr.Dropdown( PITCH_ALGO_OPT, value=PITCH_ALGO_OPT[3], label="Pitch algorithm", visible=True, interactive=True, ) ``` -------------------------------- ### Applying Voice Imitation (Tone Conversion) in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb If voice imitation is enabled, this snippet applies tone conversion to the diarized audio segments. It imports `toneconverter` from `soni_translate.text_to_speech` and attempts to modify the voice characteristics based on specified parameters like `voice_imitation_max_segments` and `voice_imitation_method`. Errors during conversion are logged. ```python # Voice Imitation (Tone color converter) if voice_imitation: prog_disp( "Voice Imitation...", 0.85, is_gui, progress=progress ) from soni_translate.text_to_speech import toneconverter try: toneconverter( copy.deepcopy(self.result_diarize), voice_imitation_max_segments, voice_imitation_remove_previous, voice_imitation_vocals_dereverb, voice_imitation_method, ) except Exception as error: logger.error(str(error)) ``` -------------------------------- ### Generating JSON Output from Diarized Segments (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This code iterates through `self.result_diarize["segments"]`, extracting `start` time, `text`, and `speaker` information. It formats the speaker ID by converting it to an integer and adding 1. The extracted data is then appended to `json_data` as a list of dictionaries. Finally, `json_data` is converted into a pretty-printed JSON string using `json.dumps` and returned after encoding and decoding to handle unicode escapes. ```Python json_data = [] for segment in self.result_diarize["segments"]: start = segment["start"] text = segment["text"] speaker = int(segment.get("speaker", "SPEAKER_00")[-2:]) + 1 json_data.append( {"start": start, "text": text, "speaker": speaker} ) # Convert list of dictionaries to a JSON string with indentation json_string = json.dumps(json_data, indent=2) logger.info("Done") self.edit_subs_complete = True return json_string.encode().decode("unicode_escape") ``` -------------------------------- ### Removing SoniTranslate Conda Environment Source: https://github.com/r3gm/sonitranslate/blob/main/README.md This snippet first deactivates the 'sonitr' Conda environment and then completely removes it. This action is useful for starting a fresh installation or cleaning up the environment. ```bash conda deactivate conda env remove -n sonitr ``` -------------------------------- ### Installing Piper TTS on Windows (Experimental) Source: https://github.com/r3gm/sonitranslate/blob/main/docs/windows_install.md This experimental method for Windows installs `piper-phonemize` from a specific wheel file, followed by `sherpa-onnx` and `piper-tts` without its dependencies. This is a workaround for limited Windows support for `piper-tts`. ```Shell pip install https://github.com/R3gm/piper-phonemize/releases/download/1.2.0/piper_phonemize-1.2.0-cp310-cp310-win_amd64.whl pip install sherpa-onnx==1.9.12 pip install piper-tts==1.2.0 --no-deps ``` -------------------------------- ### Validating OpenAI API Key in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb Checks if the `OPENAI_API_KEY` environment variable is set. If not, it raises a `ValueError` with instructions on how to set the key, indicating its necessity for GPT-based translation processes. ```Python def check_openai_api_key(): if not os.environ.get("OPENAI_API_KEY"): raise ValueError( "To use GPT for translation, please set up your OpenAI API key " "as an environment variable in Linux as follows: " "export OPENAI_API_KEY='your-api-key-here'. Or change the " "translation process in Advanced settings." ) ``` -------------------------------- ### Cloning SoniTranslate GitHub Repository Source: https://github.com/r3gm/sonitranslate/blob/main/docs/windows_install.md These commands clone the SoniTranslate GitHub repository into the current directory and then navigate into the newly created 'SoniTranslate' folder. This is a prerequisite for installing project-specific dependencies. ```Shell git clone https://github.com/r3gm/SoniTranslate.git cd SoniTranslate ``` -------------------------------- ### Installing Base and Extra Python Packages Source: https://github.com/r3gm/sonitranslate/blob/main/docs/windows_install.md These commands install the core and additional Python dependencies listed in `requirements_base.txt` and `requirements_extra.txt` using pip. It also installs `onnxruntime-gpu` for optimized ONNX model execution on GPUs. ```Shell pip install -r requirements_base.txt -v pip install -r requirements_extra.txt -v pip install onnxruntime-gpu ``` -------------------------------- ### Displaying Warnings in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb A utility function for displaying warning messages. It logs the warning using `logger.warning` and, if a GUI is active (`is_gui` is True), displays a warning message using `gr.Warning`, typically for Gradio-based interfaces. ```Python def warn_disp(wrn_lang, is_gui): logger.warning(wrn_lang) if is_gui: gr.Warning(wrn_lang) ``` -------------------------------- ### Displaying Document Translation Output File in Gradio Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet creates a Gradio `gr.File` component designed to display the resulting translated document. The component is set to be non-interactive, serving purely as an output display for the generated file. ```Python with gr.Row(): docs_output = gr.File( label="Result", interactive=False, ) ``` -------------------------------- ### Adjusting Compute Type for CPU Devices (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet ensures that if the current device is a CPU and the `compute_type` is not already optimized for CPU, it is explicitly set to 'float32'. This helps prevent potential errors or performance issues when running on CPU. ```python if self.device == "cpu" and compute_type not in COMPUTE_TYPE_CPU: logger.info("Compute type changed to float32") compute_type = "float32" ``` -------------------------------- ### Validating Origin Language and ASR Support in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This code sets the `origin_language` to 'Automatic detection' if not specified. It then validates if the selected `origin_language` is supported for transcription (ASR) when no subtitle file is provided, raising an error for unsupported unidirectional languages. ```Python if not origin_language: origin_language = "Automatic detection" if origin_language in UNIDIRECTIONAL_L_LIST and not subtitle_file: raise ValueError( f"The language '{origin_language}' " "is not supported for transcription (ASR)." ) ``` -------------------------------- ### Running SoniTranslate Web Application Source: https://github.com/r3gm/sonitranslate/blob/main/README.md This command executes the `app_rvc.py` script, which launches the SoniTranslate web interface. Once started, the application will be accessible via a local URL, typically `http://127.0.0.1:7860`. ```bash python app_rvc.py ``` -------------------------------- ### Generating File Hash in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb Computes a BLAKE2b hash of a given file. It reads the file in chunks of 8192 bytes to efficiently handle large files and returns the first 18 characters of the hexadecimal digest of the computed hash. ```Python def get_hash(filepath): with open(filepath, 'rb') as f: file_hash = hashlib.blake2b() while chunk := f.read(8192): file_hash.update(chunk) return file_hash.hexdigest()[:18] ``` -------------------------------- ### Installing Coqui XTTS Optional Package Source: https://github.com/r3gm/sonitranslate/blob/main/README.md This snippet installs Coqui XTTS, an advanced text-to-speech model capable of voice cloning and multilingual speech generation. It first installs dependencies from `requirements_xtts.txt` and then the specific TTS version without its own dependencies to avoid conflicts. ```bash pip install -q -r requirements_xtts.txt pip install -q TTS==0.21.1 --no-deps ``` -------------------------------- ### Cloning SoniTranslate GitHub Repository Source: https://github.com/r3gm/sonitranslate/blob/main/README.md This command sequence clones the SoniTranslate GitHub repository into the current directory and then navigates into the newly created 'SoniTranslate' folder. This is a prerequisite for installing the project's dependencies. ```bash git clone https://github.com/r3gm/SoniTranslate.git cd SoniTranslate ``` -------------------------------- ### Adding HTML Separator to Gradio Interface Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet inserts a simple HTML horizontal rule (`
`) into the Gradio interface, followed by a closing `` tag. It serves as a visual separator to organize the UI elements, improving readability and layout. ```Python gr.HTML("
") ``` -------------------------------- ### Initializing TTS Engines and Voice List (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This method initializes the ClassVoices object and attempts to enable Piper and Coqui XTTS engines. It logs the status of each engine and handles exceptions during import. It also sets an environment variable for Coqui TTS license agreement. Finally, it populates and returns a list of available TTS voices. ```Python self.result_source_lang = None self.edit_subs_complete = False self.voiceless_id = None self.burn_subs_id = None self.vci = ClassVoices(only_cpu=cpu_mode) self.tts_voices = self.get_tts_voice_list() logger.info(f"Working in: {self.device}") def get_tts_voice_list(self): try: from piper import PiperVoice # noqa piper_enabled = True logger.info("PIPER TTS enabled") except Exception as error: logger.debug(str(error)) piper_enabled = False logger.info("PIPER TTS disabled") try: from TTS.api import TTS # noqa xtts_enabled = True logger.info("Coqui XTTS enabled") logger.info( "In this app, by using Coqui TTS (text-to-speech), you " "acknowledge and agree to the license.\n" "You confirm that you have read, understood, and agreed " "to the Terms and Conditions specified at the following " "link:\nhttps://coqui.ai/cpml.txt." ) os.environ["COQUI_TOS_AGREED"] = "1" except Exception as error: logger.debug(str(error)) xtts_enabled = False logger.info("Coqui XTTS disabled") self.tts_info = TTS_Info(piper_enabled, xtts_enabled) return self.tts_info.tts_list() ``` -------------------------------- ### Initializing Invisible XTTS Dereverb Checkbox (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This code snippet initializes a Gradio checkbox for XTTS dereverberation. It is set to `False` by default and is explicitly made `visible=False`, indicating it's conditionally hidden, likely when dereverberation is not applicable or desired. ```Python wav_speaker_dereverb = gr.Checkbox( False, label=lg_conf["xtts_dereverb_label"], info=lg_conf["xtts_dereverb_info"], visible=False ) ``` -------------------------------- ### Resolving Subtitle File Path in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This code block ensures that if a `subtitle_file` object is provided, its path is correctly extracted. It converts the `subtitle_file` from an object (e.g., a Gradio File object) to its string representation (`.name`) if it's not already a string. ```Python if subtitle_file: subtitle_file = ( subtitle_file if isinstance(subtitle_file, str) else subtitle_file.name ) ``` -------------------------------- ### Configuring XTTS Output UI Elements (Python) Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet defines a set of Gradio UI components related to XTTS (text-to-speech) output. It includes an HTML component for displaying WAV speaker output, a button to trigger WAV creation, and a Markdown component for a footer. These elements are part of the main interface for generating and managing XTTS audio. ```Python wav_speaker_output = gr.HTML() create_xtts_wav = gr.Button( lg_conf["xtts_button"] ) gr.Markdown(lg_conf["xtts_footer"]) ``` -------------------------------- ### Validating Hugging Face Token in Python Source: https://github.com/r3gm/sonitranslate/blob/main/SoniTranslate_Colab_embedded.ipynb This snippet checks for the presence of a Hugging Face token, attempting to retrieve it from environment variables if not provided. It raises a ValueError if the token is required but missing, unless diarization is disabled or only one speaker is expected. ```Python if not YOUR_HF_TOKEN: YOUR_HF_TOKEN = os.getenv("YOUR_HF_TOKEN") if diarization_model == "disable" or max_speakers == 1: if YOUR_HF_TOKEN is None: YOUR_HF_TOKEN = "" elif not YOUR_HF_TOKEN: raise ValueError("No valid Hugging Face token") else: os.environ["YOUR_HF_TOKEN"] = YOUR_HF_TOKEN ```