### Quick Start: Text-to-Speech Example Source: https://gradium.ai/api_docs.html/index A basic example demonstrating how to use the Gradium API for Text-to-Speech (TTS). It initializes the client, converts text to speech, and saves the audio to a WAV file. Requires an API key and the `asyncio` library. ```python import asyncio import gradium async def main(): client = gradium.client.GradiumClient(api_key="your-api-key") result = await client.tts( setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "wav"}, text="Welcome to Gradium! Transform your text into natural-sounding speech in seconds." ) with open("welcome.wav", "wb") as f: f.write(result.raw_data) if __name__ == "__main__": asyncio.run(main()) ``` -------------------------------- ### TTS WebSocket Client-to-Server Setup Message Source: https://gradium.ai/api_docs.html/index Example of the initial 'setup' message sent from the client to the server to configure the Text-to-Speech stream. ```json {"type": "setup", "voice_id": "YTpq7expH9539ERJ", "model_name": "default", "output_format": "wav"} ``` -------------------------------- ### Install Gradium Python Package Source: https://gradium.ai/api_docs.html/index Installs the Gradium Python client library using pip. This is the first step to using the Gradium API in your Python projects. ```bash pip install gradium ``` -------------------------------- ### Setup Message (First Message) Source: https://gradium.ai/api_docs.html/index The initial message to establish the text-to-speech session. This must be the very first message sent after connecting. ```APIDOC ## POST /api/tts/stream ### Description Initiates a text-to-speech session. This message must be sent immediately after establishing a WebSocket connection. ### Method POST (via WebSocket) ### Endpoint /api/tts/stream ### Parameters #### Header Parameters - **x-api-key** (string) - Required - Your Gradium API key #### Request Body - **type** (string) - Required - Must be "setup" - **model_name** (string) - Optional - The TTS model to use (default: "default") - **voice_id** (string) - Required - Voice ID from the library or custom voice ID - **output_format** (string) - Required - Audio format (e.g., "wav", "pcm", "opus") ### Request Example ```json { "type": "setup", "model_name": "default", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav" } ``` ### Response #### Success Response (200) - **type** (string) - Indicates the message type, expected to be "ready". - **request_id** (string) - A unique identifier for the session. #### Response Example ```json { "type": "ready", "request_id": "550e8400-e29b-41d4-a716-446655440000" } ``` ``` -------------------------------- ### Date Rule Example for Text Rewriting Source: https://gradium.ai/api_docs.html/index Illustrates the 'Date' text rewriting rule, which converts numeric dates into a more speech-friendly format. It shows examples of how different date formats are transformed. The rule preserves trailing punctuation. ```text 12/31/2020 -> 12-31 2020 16/01/1980 -> 16-01 1980 1/5. -> 1-5. ``` -------------------------------- ### TTS POST Endpoint Example (cURL) Source: https://gradium.ai/api_docs.html/index Example of using the TTS POST endpoint with cURL to convert text to speech. It includes authentication, content type, and a JSON payload for the request. The output is redirected to a WAV file. ```curl curl -L -X POST https://eu.api.gradium.ai/api/post/speech/tts \ -H "x-api-key: your_api_key" \ -H "Content-Type: application/json" \ -d '{"text": "Hello, this is a test of the text to speech system.", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav", "only_audio": true}' \ > output.wav ``` -------------------------------- ### Setup Message (First Message) Source: https://gradium.ai/api_docs.html/index The initial message sent by the client to configure the speech-to-text model and input format. This message must be the very first message sent after establishing a connection. ```APIDOC ## POST /ws/speech-to-text ### Description Initiates a WebSocket connection for speech-to-text transcription. The first message must be a 'setup' message to configure the model and input format. ### Method POST (via WebSocket upgrade) ### Endpoint /ws/speech-to-text ### Parameters #### Request Body - **type** (string) - Required - Must be "setup" - **model_name** (string) - Required - The Speech-To-Text model to use (default: "default") - **input_format** (string) - Required - Audio format - "pcm", "wav", or "opus" ### Request Example ```json { "type": "setup", "model_name": "default", "input_format": "pcm" } ``` ### Response #### Success Response (101 Switching Protocols) Upon successful setup, the server responds with a 'ready' message. #### Response Example (Ready Message) ```json { "type": "ready", "request_id": "550e8400-e29b-41d4-a716-446655440000", "model_name": "default", "sample_rate": 24000, "frame_size": 1920, "delay_in_frames": 0, "text_stream_names": [] } ``` ### Important - This must be the very first message sent after connection. - The server will close the connection if any other message is sent first. ``` -------------------------------- ### French Number Rule Examples for Text Rewriting Source: https://gradium.ai/api_docs.html/index Shows examples of the 'NumberFr' text rewriting rule for French, which converts numbers into word-based representations. It illustrates the use of 'mille', 'million(s)', and 'milliard(s)', and handling of negative numbers. ```text 1234 -> mille 234 2234 -> 2 mille 234 2000000 -> 2 millions -4500 -> moins 4 mille 500 123456000789 -> 123 milliards 456 millions 789 ``` -------------------------------- ### English Number Rule Examples for Text Rewriting Source: https://gradium.ai/api_docs.html/index Provides examples of the 'NumberEn' text rewriting rule, which expands large numbers into word-based representations for better pronunciation. Small numbers (<1000) and years (1900-2100) are kept as-is. It shows conversions for thousands, millions, billions, and negative numbers. ```text 123 -> 123 1234 -> 1 thousand 234 1000000 -> 1 million 2500000 -> 2 million 500 thousand 1002003004 -> 1 billion 2 million 3 thousand 4 -4500 -> minus 4 thousand 500 ``` -------------------------------- ### English Time Rule Example for Text Rewriting Source: https://gradium.ai/api_docs.html/index Shows examples of the 'TimeEn' text rewriting rule for English, which standardizes various time formats including those with colons, periods, and AM/PM markers. The rule converts colons to periods. ```text 3:45PM! -> 3.45PM! 12.30. -> 12.30. 12:30 -> 12.30 ``` -------------------------------- ### Speech-to-Text (STT) Best Practices Source: https://gradium.ai/api_docs.html/index Recommendations for optimizing the use of the Speech-to-Text API, including setup, audio format, chunk size, shutdown, and VAD usage. ```APIDOC ## Best Practices for STT 1. **Always send setup first**: The server expects a setup message immediately after connection. 2. **Use correct audio format**: When using PCM, ensure audio is 24kHz PCM 16-bit mono. 3. **Send appropriately sized chunks**: 1920 samples (80ms) per message is recommended. 4. **Graceful shutdown**: Send `end_of_stream` when done to properly close the session. 5. **VAD Threshold**: Our VAD provides estimated probabilities that the speaker would be silent for a fixed number of seconds in the future. The thresholds to trigger the end-of-the-turn decisions might be application-dependent; as a starting point we recommend looking at the horizon of 2s and trigger when the inactivity_prob is above 0.5: `turn_ended = msg["vad"][2]["inactivity_prob"] > 0.5`. 6. **Acting on VAD**: Whenever you decide that the VAD probabilities warrant a decision to consider the turn ended, there is still up to `delay_in_frames` audio frames processed by the model. Instead of feeding silence from the speaker, the system can be made more reactive by flushing the remainder of the turn's transcript. For that, you can feed in `delay_in_frames` chunks of silence (vectors of zeros). If those are fed in faster than realtime, the API also has a possibility to process them faster, allowing a considerably more reactive turn-around. ``` -------------------------------- ### Email Rule Examples (Multi-language) Source: https://gradium.ai/api_docs.html/index Demonstrates how email addresses are spelled out with language-specific words for special characters like '@' and '.'. Rules include EmailEn, EmailFr, EmailDe, EmailEs, EmailPt. ```text English example: foo.bar@gmail.com → foo dot bar at gmail dot com French example: foo@gmail.com → foo arobaze gmail point com ``` -------------------------------- ### Setup Connection Message Source: https://gradium.ai/api_docs.html/index The initial message sent by the client to the server to establish a connection and configure the speech-to-text model. It specifies the model name and audio input format. This message must be the very first message sent. ```json { "type": "setup", "model_name": "default", "input_format": "pcm" } ``` -------------------------------- ### URL Rule Examples (Multi-language) Source: https://gradium.ai/api_docs.html/index Illustrates how URLs are spelled out, including protocols, domains, paths, and special characters. Supports language-specific translations for characters and two-letter top-level domains. Rules include UrlEn, UrlFr, UrlDe, UrlEs, UrlPt. ```text English examples: www.example.com → www dot example dot com https://www.example.com/path → H-T-T-P-S colon slash slash www dot example dot com slash path French examples: https://www.kyutai.fr → H-T-T-P-S deux-points slash slash www point kyutai point F-R www.it-management.com/promo → www point I-T tiret management point com slash promo ``` -------------------------------- ### GET /voices/ Source: https://gradium.ai/api_docs.html/index Lists voices for the authenticated organization. Supports pagination and catalog inclusion. ```APIDOC ## GET /voices/ ### Description List voices for the authenticated organization. ### Method GET ### Endpoint EU API: https://eu.api.gradium.ai/api/voices/ US API: https://us.api.gradium.ai/api/voices/ ### Query Parameters - **skip** (integer) - Optional - Skip a number of records. Default: 0. Minimum: 0. - **limit** (integer) - Optional - Limit the number of records returned. Default: 100. Minimum: 1. - **include_catalog** (boolean) - Optional - Whether to include catalog voices. Default: false. ### Responses #### Success Response (200) Returns a list of voice objects. ```json [ { "uid": "string", "name": "string", "description": "string", "filename": "string", "start_s": 0, "is_catalog": true, "is_pro_clone": true, "language": "string" } ] ``` #### Error Response (422) Validation Error. ```json { "uid": "string", "error": "string", "was_updated": false } ``` ``` -------------------------------- ### Phone Number Rule Examples (Multi-language) Source: https://gradium.ai/api_docs.html/index Shows how phone numbers are formatted according to country-specific conventions, supporting both international and local formats. Rules include PhoneEn, PhoneFr, PhoneDe, PhoneEs, PhonePt. ```text French examples: 0123456789 → 01 23 45 67 89 +330556791936 → +33 05 56 79 19 36 (French TTS) English examples: 07596854413 → 0-7-5-9 6-8-5 4-4-1-3 +16502349653 → +1 6-5-0 2-3-4 9-6-5-3 German examples: 01511234567 → 0-1-5 1 1 2-3 4-5 6-7 +491511234567 → +49 1-5 1 1 2-3 4-5 6-7 ``` -------------------------------- ### TTS WebSocket Server-to-Client Ready Message Source: https://gradium.ai/api_docs.html/index Example of the 'ready' message sent from the server to the client, indicating it is prepared to receive text. ```json {"type": "ready", "request_id": "uuid"} ``` -------------------------------- ### German Time Rule Example for Text Rewriting Source: https://gradium.ai/api_docs.html/index Demonstrates the 'TimeDe' text rewriting rule for German, which standardizes time formats using either colons or periods as separators. ```text 8:20. -> 8.20. 22.45! -> 22.45! ``` -------------------------------- ### Get Word-Level Timestamps for TTS Output Source: https://gradium.ai/api_docs.html/index Retrieve word-level timestamps along with the generated audio. The `tts` method returns a result object containing `text_with_timestamps`, which provides the start and end times for each word. This is useful for synchronization tasks. ```python result = await client.tts( setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "wav"}, text="Hello, world!" ) for item in result.text_with_timestamps: print(f"{item.text}: {item.start_s:.2f}s - {item.stop_s:.2f}s") ``` -------------------------------- ### Server Ready Message Source: https://gradium.ai/api_docs.html/index Sent by the server to the client after receiving the setup message, indicating that the connection is ready to accept audio data. It includes session details like request ID, model name, sample rate, and frame size. ```json { "type": "ready", "request_id": "550e8400-e29b-41d4-a716-446655440000", "model_name": "default", "sample_rate": 24000, "frame_size": 1920, "delay_in_frames": 0, "text_stream_names": [] } ``` -------------------------------- ### Setup Message for Gradium AI API Source: https://gradium.ai/api_docs.html/index The initial JSON message sent to the server to configure the text-to-speech session. It specifies the model, voice, and desired audio output format. This message is mandatory and must be the first one sent after establishing a connection. ```json { "type": "setup", "model_name": "default", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav" } ``` -------------------------------- ### Text-to-Speech (TTS) with Custom Voices Source: https://gradium.ai/api_docs.html/index Example of performing Text-to-Speech using a custom voice ID. This allows you to use voices that you have trained or selected from the voice library. ```python result = await client.tts( setup={ "model_name": "default", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav" }, text="Hello with my custom voice!" ) ``` -------------------------------- ### PUT /voices/{voice_uid} Source: https://gradium.ai/api_docs.html/index Updates an existing voice identified by its UID. Allows modification of name, description, language, start time, tags, and rank. ```APIDOC ## PUT /voices/{voice_uid} ### Description Update a voice by its UID. ### Method PUT ### Endpoint EU API: https://eu.api.gradium.ai/api/voices/{voice_uid} US API: https://us.api.gradium.ai/api/voices/{voice_uid} ### Path Parameters - **voice_uid** (string) - Required - The unique identifier of the voice to update. ### Request Body - **name** (string | null) - Optional - The new name for the voice. - **description** (string | null) - Optional - The new description for the voice. - **language** (string | null) - Optional - The new language for the voice. - **start_s** (number | null) - Optional - The new start time in seconds. - **tags** (array of objects | null) - Optional - An array of tags for the voice. - **rank** (number | null) - Optional - The new rank for the voice. ### Request Example ```json { "name": "string", "description": "string", "language": "string", "start_s": 0, "tags": [ { "property1": "string", "property2": "string" } ], "rank": 0 } ``` ### Responses #### Success Response (200) Returns the updated voice object. ```json { "uid": "string", "name": "string", "description": "string", "language": "string", "start_s": 0, "stop_s": 0, "filename": "string", "org_uid": "ac85332b-4f31-4f30-a863-7c62d7956727", "is_pending": false, "has_audio": true, "is_pro_clone": false } ``` #### Error Response (422) Validation Error. ```json { "uid": "string", "error": "string", "was_updated": false } ``` ``` -------------------------------- ### French Time Rule Example for Text Rewriting Source: https://gradium.ai/api_docs.html/index Illustrates the 'TimeFr' text rewriting rule for French, which handles time formats using 'h' as a separator or colons. It shows how these formats are standardized. ```text 9h15, -> 9h15, 14:00? -> 14h00? ``` -------------------------------- ### GET /voices/{voice_uid} Source: https://gradium.ai/api_docs.html/index Retrieves a specific voice by its UID. Optional organization and key UIDs can be provided for access control. ```APIDOC ## GET /voices/{voice_uid} ### Description Get a voice by its UID. Optional org_uid and key_uid for access control. ### Method GET ### Endpoint EU API: https://eu.api.gradium.ai/api/voices/{voice_uid} US API: https://us.api.gradium.ai/api/voices/{voice_uid} ### Path Parameters - **voice_uid** (string) - Required - The unique identifier of the voice. ### Responses #### Success Response (200) Returns the details of the specified voice. ```json { "uid": "string", "name": "string", "description": "string", "filename": "string", "start_s": 0, "is_catalog": true, "is_pro_clone": true, "language": "string" } ``` #### Error Response (422) Validation Error. ```json { "uid": "string", "name": "string", "description": "string", "language": "string", "start_s": 0, "stop_s": 0, "filename": "string", "org_uid": "ac85332b-4f31-4f30-a863-7c62d7956727", "is_pending": false, "has_audio": true, "is_pro_clone": false } ``` ``` -------------------------------- ### TTS WebSocket Server-to-Client Text Stream Message (with timing) Source: https://gradium.ai/api_docs.html/index Example of a 'text' message sent from the server to the client, including timing information for synthesized speech segments. ```json {"type": "text", "text": "Hello", "start_s": 0.2, "stop_s": 0.6} ``` -------------------------------- ### TTS WebSocket Client-to-Server Text Stream Message Source: https://gradium.ai/api_docs.html/index Example of a 'text' message sent from the client to the server for Text-to-Speech conversion. ```json {"type": "text", "text": "Hello, world!"} ``` -------------------------------- ### Alphanumeric Rule Example Source: https://gradium.ai/api_docs.html/index Demonstrates the AlNum rule for handling mixed uppercase letters and digits, commonly found in license plates or product codes. Characters are grouped by type and joined with hyphens. ```text AB12CD34! → A-B 1-2 C-D 3-4! ``` -------------------------------- ### Force Audio Output with Tag Source: https://gradium.ai/api_docs.html/index Use the `` tag to immediately output all buffered audio for the input text. This is useful when the natural buffering of the TTS model causes noticeable lag. The example demonstrates sending text with a flush tag and processing the resulting audio. ```python sample_text = "Hello, this is a test from the Gradium Text to Speech system. We are testing the flush." test_audio = await client.tts( setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav'}, text=sample_text, ) ``` -------------------------------- ### Ready Message from Gradium AI API Server Source: https://gradium.ai/api_docs.html/index A JSON message sent by the server to confirm that it has received the setup message and is ready to process subsequent text requests. It includes a unique request ID for the session. ```json { "type": "ready", "request_id": "550e8400-e29b-41d4-a716-446655440000" } ``` -------------------------------- ### TTS WebSocket Client-to-Server EndOfStream Message Source: https://gradium.ai/api_docs.html/index Example of the 'end_of_stream' message sent from the client to signal the end of input text. ```json {"type": "end_of_stream"} ``` -------------------------------- ### TTS WebSocket Server-to-Client Audio Stream Message Source: https://gradium.ai/api_docs.html/index Example of an 'audio' message sent from the server to the client, containing the generated speech audio data. ```json {"type": "audio", "audio": "base64..."} ``` -------------------------------- ### TTS WebSocket Server-to-Client Error Message Source: https://gradium.ai/api_docs.html/index Example of an 'error' message sent from the server to the client, detailing any issues encountered during the TTS process. ```json {"type": "error", "message": "Error description", "code": 1008} ``` -------------------------------- ### Insert Pauses with Tag Source: https://gradium.ai/api_docs.html/index Insert pauses in the generated speech by using the `` tag, where 'X' is the duration in seconds (0.1 to 2.0). The tag must be surrounded by spaces. The example shows how to include a pause in the text input and generate audio with the specified delay. ```python sample_text = 'Hello, this is a test from the Gradium Text to Speech system. We are testing the pause.' test_audio = await client.tts( setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav'}, text=sample_text, ) ``` -------------------------------- ### Text Response with Timestamps Source: https://gradium.ai/api_docs.html/index A JSON message from the server indicating which segment of the input text has been synthesized into speech, along with the start and stop timestamps in the audio stream. This helps in synchronizing text with audio playback. ```json { "type": "text", "text": "Hello", "start_s": 0.2, "stop_s": 0.6 } ``` -------------------------------- ### Audio Response from Gradium AI API Source: https://gradium.ai/api_docs.html/index A JSON message containing base64-encoded audio data, streamed by the server in response to a text message. The format (wav, pcm, opus) is determined by the initial setup message. Multiple audio messages may be sent for a single text input. ```json { "type": "audio", "audio": "base64_encoded_audio_data..." } ``` -------------------------------- ### Text Transcription Response Source: https://gradium.ai/api_docs.html/index Sent by the server to the client, containing the transcribed text from the audio. Multiple text messages may be streamed as audio is processed. Includes the transcribed text, start time, and an optional stream ID. ```json { "type": "text", "text": "Hello world", "start_s": 0.5, "stream_id": null } ``` -------------------------------- ### Initialize Gradium Client using Environment Variable Source: https://gradium.ai/api_docs.html/index Shows how to set the Gradium API key using an environment variable and then initialize the client without explicitly passing the key. This is a more secure method for managing API keys. ```bash export GRADIUM_API_KEY=gd_your_api_key_here ``` -------------------------------- ### Initialize Gradium Client with API Key Source: https://gradium.ai/api_docs.html/index Demonstrates how to create an instance of the Gradium client using a direct API key. This is a common way to authenticate your requests to the Gradium API. ```python import gradium client = gradium.client.GradiumClient(api_key="gd_your_api_key_here") ``` -------------------------------- ### Initialize Gradium Client using Environment Variable (Python) Source: https://gradium.ai/api_docs.html/index Initializes the Gradium client after setting the `GRADIUM_API_KEY` environment variable. The client automatically picks up the key from the environment. ```python import gradium client = gradium.client.GradiumClient() ``` -------------------------------- ### WebSocket Connection Establishment Source: https://gradium.ai/api_docs.html/index Information on establishing a WebSocket connection for the ASR API, including header parameters and API endpoints. ```APIDOC ## WebSocket Connection Establishment ### Header Parameters * `x-api-key` (string, required): Your Gradium API key ### Responses * `101`: WebSocket connection established ### Endpoints * **EU API**: `https://eu.api.gradium.ai/api/speech/asr` * **US API**: `https://us.api.gradium.ai/api/api/speech/asr` ``` -------------------------------- ### Text-to-Speech with Flushing and Pauses Source: https://gradium.ai/api_docs.html/index Demonstrates how to use the `` tag to force immediate audio output and the `` tag to introduce pauses in the generated speech. ```APIDOC ## Flushing and Pauses The model only generates audio when it has enough context to do so, so generally the audio lags a few words behind the text input. The `` tag can be used to force the model to output the audio for all the text that has been input so far. ### Example with `` tag ```python sample_text = "Hello, this is a test from the Gradium Text to Speech system. We are testing the flush." test_audio = await client.tts( setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav'}, text=sample_text, ) ``` Pauses can be generated by inserting a "break time" tag as show below. The brak time is specified in seconds and should be between 0.1 and 2.0s. The tag must be preceeded and followed by a space. ### Example with `` tag ```python sample_text = 'Hello, this is a test from the Gradium Text to Speech system. We are testing the pause.' test_audio = await client.tts( setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav'}, text=sample_text, ) ``` ``` -------------------------------- ### Streaming Text-to-Speech with Async Generator Input Source: https://gradium.ai/api_docs.html/index Illustrates how to use an asynchronous generator to stream text input for real-time audio synthesis. ```APIDOC ## Async Generator Input ```python async def text_generator(): yield "Hello, " yield "this is " yield "a streaming " yield "example." stream = await client.tts_stream( setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "pcm"}, text=text_generator() ) async for chunk in stream.iter_bytes(): pass ``` ``` -------------------------------- ### Advanced Text-to-Speech Options Source: https://gradium.ai/api_docs.html/index Explains how to use the `json_config` parameter to control advanced speech synthesis options such as speed, stability, and voice similarity. ```APIDOC ## Advanced Options Some models support advanced options that can be passed using the `json_config` parameter. In the Python api, this parameter is passed as a dictionary mapping string to values (either float or string). This parameter can be used to control: * Speed of the generated speech via the `padding_bonus` parameter. * Stability of the generated speech via the `temp` temperature parameter. * Voice similarity using the `cfg_coef` parameter. * Rewrite rules using `rewrite_rules`. ### Speed Control (`padding_bonus`) You can guide the speed of the model using the padding bonus parameter. Default value is 0.0. Negative values mean that the speaker will speak faster (values between -4.0 and -0.1) Positive values mean that the speaker will speak slower (values between 0.1 and 4.0) ```python sample_text = "Hello, this is a test from the Gradium Text to Speech system. We are testing the speed." slower_audio = await client.tts( setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'padding_bonus':2.0}}, text=sample_text, ) faster_audio = await client.tts( setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'padding_bonus':-2.0}}, text=sample_text, ) ``` ### Temperature Control (`temp`) The temperature for the generation can be set with values ranging from 0 to 1.4. A value of 0 corresponds to a deterministic generation, while higher values lead to more diverse outputs. Default value is 0.7. ```python setup = {'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'temp':0.3}} audio = await client.tts(text=sample_text, setup=setup) ``` ### Voice Similarity Control (`cfg_coef`) The `cfg_coef` parameter can be used to control the similarity of the generated speech to the target voice. Values range from 1.0 to 4.0. The default value is 2.0. The higher the value, the more the model replicates the cloned voice but larger values can lead to audio artifacts. ### Rewrite Rules (`rewrite_rules`) The `rewrite_rules` parameter can be used to pass text rewriting rules that are applied before the text is synthesized. The rules should be passed as a string. More details on the rules themselves can be found below in this document. Values such as `"en"`, `"fr"`, `"de"`, `"es"`, `"pt"` enable all the rewriting rules for a given language. ``` -------------------------------- ### Create Voice Source: https://gradium.ai/api_docs.html/index Create a new voice for an organization by uploading an audio file. ```APIDOC ## Create Voice ### Description Create a new voice for an organization with audio file upload. ### Method POST ### Endpoint `/voices/` ### Request Body Schema `multipart/form-data` #### Form Fields * `audio_file` (binary, required): Audio File * `name` (string, required): Name * `input_format` (string, optional): Input Format * `description` (string, optional): Description * `language` (string, optional): Language * `start_s` (number, optional, default: 0): Start time in seconds * `timeout_s` (number, optional, default: 10): Timeout in seconds ### Responses * **201**: Successful Response * **422**: Validation Error ### Endpoints * **EU API**: `https://eu.api.gradium.ai/api/voices/` * **US API**: `https://us.api.gradium.ai/api/voices/` ``` -------------------------------- ### Enable Text Rewriting Rules via JSON Configuration Source: https://gradium.ai/api_docs.html/index Demonstrates how to enable text rewriting rules for the text-to-speech API using the `json_config` parameter. Rewrite rules normalize and expand text for better pronunciation of dates, times, numbers, etc. Rules can be enabled using language aliases or specific rule names. ```json { "json_config": { "rewrite_rules": "en" } } ``` ```json { "json_config": { "rewrite_rules": "TimeEn,Date,NumberEn,EmailEn" } } ``` -------------------------------- ### Credits API Source: https://gradium.ai/api_docs.html/index Monitor your API credit balance. ```APIDOC ## Credits ### Description Monitor API credit balance. ### Method GET ### Endpoint (Specific endpoint not provided in the input text, but typically would be something like `/credits`) ### Responses (Response details not provided in the input text. Typically would include current credit balance and usage information.) ``` -------------------------------- ### Text-to-Speech with Word-Level Timestamps Source: https://gradium.ai/api_docs.html/index Shows how to retrieve word-level timestamps for the generated audio, allowing for precise synchronization between text and audio. ```APIDOC ## Text with Timestamps The model also returns word-level timestamps for the generated audio. ### Request ```python result = await client.tts( setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "wav"}, text="Hello, world!" ) for item in result.text_with_timestamps: print(f"{item.text}: {item.start_s:.2f}s - {item.stop_s:.2f}s") ``` ``` -------------------------------- ### Text-to-Speech (TTS) Output Formats Source: https://gradium.ai/api_docs.html/index Illustrates how to specify different output formats for Text-to-Speech, including WAV and PCM. It also shows how to access the audio data as NumPy arrays for PCM output. ```python # WAV format result = await client.tts(setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "wav"}, text="Hello") # PCM format: the data is sampled at 48kHz, 16-bit signed integer, mono result = await client.tts(setup={"voice_id": "YTpq7expH9539ERJ", "output_format": "pcm"}, text="Hello") # Get numpy array from PCM pcm_array = result.pcm() pcm16_array = result.pcm16() ``` -------------------------------- ### Configure Advanced Text-to-Speech Options with JSON in Python Source: https://gradium.ai/api_docs.html/index This section explains how to use the `json_config` parameter in the Gradium AI Python API to control advanced text-to-speech synthesis options. It details parameters for controlling speech stability (temperature), expected language, and generation delay in audio frames. The `json_config` is passed as a dictionary. ```python # Example of passing json_config as a dictionary in Python # For temperature control: # json_config = {"text": {"temperature": 0.8}} # For language control: # json_config = {"language": "en"} # For delay control: # json_config = {"delay_in_frames": 16} ``` -------------------------------- ### Apply Rewrite Rules for Text Synthesis Source: https://gradium.ai/api_docs.html/index Use the `rewrite_rules` parameter in `json_config` to apply text transformations before synthesis. Rules can be specified for different languages (e.g., 'en', 'fr', 'de'). This allows for custom pronunciation or formatting adjustments. ```python # Example usage for rewrite_rules (assuming sample_text and client are defined) # setup = {'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'rewrite_rules':'en'}} # audio = await client.tts(text=sample_text, setup=setup) ``` -------------------------------- ### Text-to-Speech (TTS) - Basic Usage Source: https://gradium.ai/api_docs.html/index This endpoint allows you to convert text into natural-sounding speech using specified voice and output format. ```APIDOC ## POST /tts ### Description Converts the input text into speech using the specified voice and audio format. ### Method POST ### Endpoint /tts ### Parameters #### Request Body - **setup** (object) - Required - Configuration for the TTS request. - **model_name** (string) - Optional - The TTS model to use (default: "default"). - **voice_id** (string) - Required - The ID of the voice to be used. Found in the voice library or studio. - **output_format** (string) - Required - Audio format of the output data (supported: "pcm", "wav", "opus", "ulaw_8000", "alaw_8000", "pcm_16000", "pcm_24000"). - **text** (string) - Required - The text to be converted into speech. ### Request Example ```json { "setup": { "model_name": "default", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav" }, "text": "Hello, world!" } ``` ### Response #### Success Response (200) - **raw_data** (bytes) - The raw audio data in the specified format. - **sample_rate** (integer) - The sample rate of the audio. - **request_id** (string) - A unique identifier for the request. #### Response Example ```json { "raw_data": "(binary audio data)", "sample_rate": 48000, "request_id": "req_12345abcde" } ``` ``` -------------------------------- ### POST /api/post/speech/tts Source: https://gradium.ai/api_docs.html/index This endpoint performs text-to-speech conversion. You send text, and it returns audio data. The audio can be streamed back, and you can choose the output format. ```APIDOC ## POST /api/post/speech/tts ### Description This endpoint performs text-to-speech conversion. You send text, and it returns audio data. The audio can be streamed back, and you can choose the output format. ### Method POST ### Endpoint - EU API: `https://eu.api.gradium.ai/api/post/speech/tts` - US API: `https://us.api.gradium.ai/api/post/speech/tts` ### Parameters #### Header Parameters - **x-api-key** (string) - Required - Your Gradium API key #### Request Body - **text** (string) - Required - The text to convert to speech - **voice_id** (string) - Required - Voice ID from the library or custom voice ID - **output_format** (string) - Required - Enum: "wav", "pcm", "opus", "ulaw_8000", "alaw_8000", "pcm_16000", "pcm_24000". Audio output format. - **json_config** (string) - Optional - Additional configuration in JSON string format (e.g., `{"padding_bonus": -1.2}`) - **model_name** (string) - Optional - The TTS model to use (default: "default") - **only_audio** (boolean) - Optional - When `true`, returns raw audio bytes instead of JSON. When `false` or omitted, returns a stream of JSON messages. ### Request Example ```json { "text": "Hello, this is a test of the text to speech system.", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav", "only_audio": true } ``` ### Response #### Success Response (200) - **Audio Data**: The response body contains the raw audio bytes in the requested format when `only_audio` is `true`. The `Content-Type` header will indicate the format (e.g., `audio/wav`, `audio/ogg`, `audio/pcm`). - **JSON Stream**: When `only_audio` is `false` or omitted, the response is a stream of JSON messages containing audio and metadata. #### Error Response (400, 401, 500) ```json { "error": "Error description explaining what went wrong", "code": 400 } ``` **Common HTTP Status Codes:** - `400`: Bad Request (e.g., missing required fields, invalid voice ID) - `401`: Unauthorized (invalid or missing API key) - `500`: Internal Server Error ``` -------------------------------- ### Streaming Text-to-Speech (TTS) Source: https://gradium.ai/api_docs.html/index Demonstrates how to use the Gradium API for streaming Text-to-Speech. This allows you to receive audio chunks as they are generated, reducing latency for longer texts. It uses the PCM output format. ```python stream = await client.tts_stream( setup={ "model_name": "default", "voice_id": "LFZvm12tW_z0xfGo", "output_format": "pcm" }, text="This is a longer text that will be streamed." ) async for audio_chunk in stream.iter_bytes(): print(f"Received {len(audio_chunk)} bytes") ``` -------------------------------- ### Control Voice Similarity with cfg_coef Source: https://gradium.ai/api_docs.html/index Fine-tune the similarity of the generated speech to the target voice using the `cfg_coef` parameter in `json_config`. Values range from 1.0 to 4.0, with higher values increasing replication but potentially causing artifacts. The default is 2.0. ```python # Example usage for cfg_coef (assuming sample_text and client are defined) # setup = {'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'cfg_coef':3.5}} # audio = await client.tts(text=sample_text, setup=setup) ``` -------------------------------- ### Error Handling Source: https://gradium.ai/api_docs.html/index Details on how errors are communicated through the API. ```APIDOC ## Error Handling ### Description When errors occur, the server sends an error message as a JSON object before closing the WebSocket connection. ### Error Message Format ```json { "type": "error", "message": "Error description explaining what went wrong", "code": 1008 } ``` ### Common Error Codes - **1008**: Policy Violation (e.g., invalid API key, missing setup message) - **1011**: Internal Server Error (unexpected server-side error) ``` -------------------------------- ### Voices Library Source: https://gradium.ai/api_docs.html/index Information about the available voices in the Gradium Voices Library, offering high-quality speech synthesis across multiple languages. ```APIDOC ## Voices Library Gradium provides a selection of high-quality voices across multiple languages. Here are our voices. ``` -------------------------------- ### Control Speech Speed with padding_bonus Source: https://gradium.ai/api_docs.html/index Adjust the speaking speed of the generated audio using the `padding_bonus` parameter within `json_config`. Negative values make the speaker faster (e.g., -2.0), while positive values make them slower (e.g., 2.0). The range is typically -4.0 to 4.0. ```python sample_text = "Hello, this is a test from the Gradium Text to Speech system. We are testing the speed." slower_audio = await client.tts( setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'padding_bonus':2.0}}, text=sample_text, ) faster_audio = await client.tts( setup={'voice_id': 'YTpq7expH9539ERJ', 'output_format': 'wav', 'json_config':{'padding_bonus':-2.0}}, text=sample_text, ) ``` -------------------------------- ### Basic Text-to-Speech (TTS) Usage Source: https://gradium.ai/api_docs.html/index Performs a basic Text-to-Speech conversion using the Gradium API. It specifies the voice ID and output format (WAV), then saves the resulting audio data to a file. It also prints the sample rate and request ID from the result. ```python import gradium client = gradium.client.GradiumClient() result = await client.tts( setup={ "model_name": "default", "voice_id": "YTpq7expH9539ERJ", "output_format": "wav" }, text="Hello, world!" ) with open("output.wav", "wb") as f: f.write(result.raw_data) print(f"Sample rate: {result.sample_rate}") print(f"Request ID: {result.request_id}") ``` -------------------------------- ### Audio Message Source: https://gradium.ai/api_docs.html/index Client sends audio data to the server for transcription. The audio should be base64 encoded and adhere to specific format requirements. ```APIDOC ## POST /ws/speech-to-text (Audio) ### Description Sends audio data to the server for real-time speech-to-text transcription. Multiple audio messages can be sent sequentially. ### Method POST (via WebSocket) ### Endpoint /ws/speech-to-text ### Parameters #### Request Body - **type** (string) - Required - Must be "audio" - **audio** (string) - Required - Base64-encoded audio data ### Request Example ```json { "type": "audio", "audio": "base64_encoded_audio_data..." } ``` ### Audio Format Requirements (for PCM input) - **Sample Rate**: 24000 Hz (24kHz) - **Format**: PCM (Pulse Code Modulation) - **Bit Depth**: 16-bit signed integer (little-endian) - **Channels**: Single channel (mono) - **Chunk Size**: Recommended 1920 samples per frame (80ms at 24kHz) ### Notes - When using `"wav"` input format, the audio must be a valid WAV file using PCM data. - When using `"opus"` input format, the audio must be some ogg wrapped opus data stream. - The server will stream text and VAD responses as it processes the audio. ``` -------------------------------- ### WebSocket /api/speech/asr Source: https://gradium.ai/api_docs.html/index Connect to this WebSocket endpoint for real-time speech-to-text conversion. You can stream audio input to this endpoint and receive text transcription in real-time. ```APIDOC ## WebSocket /api/speech/asr ### Description Connect to this WebSocket endpoint for real-time speech-to-text conversion. You can stream audio input to this endpoint and receive text transcription in real-time. ### Method WebSocket ### Endpoint - EU API: `wss://eu.api.gradium.ai/api/speech/asr` - US API: `wss://us.api.gradium.ai/api/speech/asr` ### Parameters #### Header Parameters - **x-api-key** (string) - Required - Your Gradium API key ### Request Example (WebSocket connection establishment with `x-api-key` header) ### Response (Real-time transcription data streamed back over the WebSocket connection) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.