Rime

https://docs.rime.ai/
Admin
Rime provides speech synthesis technologies that balance quality, customizability, and speed for...
Tokens:87,643
Snippets:1,035
Trust Score:8.2
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install Rime CLI with Homebrew

Source: https://docs.rime.ai/cli-reference/overview

Install the Rime CLI using Homebrew by tapping the repository and then installing the package.

```bash
brew tap rimelabs/rime-cli && brew install rime-cli
```

--------------------------------

### Install Project Dependencies

Source: https://docs.rime.ai/docs/pipecat

Install all defined project dependencies using the `uv` package manager.

```bash
uv sync
```

--------------------------------

### Install uv Package Manager

Source: https://docs.rime.ai/docs/quickstart-livekit

Install the 'uv' package manager, which helps with LiveKit's complex dependencies.

```bash
pip install uv
```

--------------------------------

### Start LiveKit Agent in Development Mode

Source: https://docs.rime.ai/docs/quickstart-livekit

Run this command to start your agent in development mode. This mode provides more verbose output for debugging.

```bash
uv run agent.py dev
```

--------------------------------

### Start the Pipecat Agent

Source: https://docs.rime.ai/docs/pipecat

Run this command in the terminal to start your Pipecat agent. Ensure you have uv installed.

```bash
uv run agent.py
```

--------------------------------

### Check for Out-of-Vocabulary Words (Request Examples)

Source: https://docs.rime.ai/api-reference/other/oov

Examples of how to call the OOV API using various programming languages. Ensure you replace '<string>' with your actual text and '<authorization>' with your valid API key.

```cURL
curl --request POST \
  --url https://users.rime.ai/oov \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --fail \
  --data '{ "text": "<string>" }'
```

```Python
import requests

url = "https://users.rime.ai/oov"

payload = {
    "text": "<string>"
}
headers = {
    "Accept": "application/json",
    "Authorization": "Bearer <authorization>",
    "Content-Type": "application/json"
}

response = requests.request("POST", url, json=payload, headers=headers)

print(response.text)
```

```JavaScript
const options = {
    method: 'POST',
    headers: {
      Accept: 'application/json',
      Authorization: 'Bearer <authorization>',
      'Content-Type': 'application/json'
    },
    body: '{"text":"<string>"}'
  };

  fetch('https://users.rime.ai/oov', options)
    .then(response => response.json())
    .then(response => console.log(response))
    .catch(err => console.error(err));
```

```PHP
<?php

$curl = curl_init();

curl_setopt_array($curl, [
  CURLOPT_URL => "https://users.rime.ai/oov",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "POST",
  CURLOPT_POSTFIELDS => "{
  \"text\": \"<string>\"\n}",
  CURLOPT_HTTPHEADER => [
    "Accept: application/json",
    "Authorization: Bearer <authorization>",
    "Content-Type: application/json"
  ],
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}
```

```Go
package main

import (
  "fmt"
  "strings"
  "net/http"
  "io/ioutil"
)

func main() {

  url := "https://users.rime.ai/oov"

  payload := strings.NewReader("{\n  \"text\": \"<string>\"\n}")

  req, _ := http.NewRequest("POST", url, payload)

  req.Header.Add("Accept", "application/json")
  req.Header.Add("Authorization", "Bearer <authorization>")
  req.Header.Add("Content-Type", "application/json")

  res, _ := http.DefaultClient.Do(req)

  defer res.Body.Close()
  body, _ := ioutil.ReadAll(res.Body)

  fmt.Println(res)
  fmt.Println(string(body))

}
```

```Java
HttpResponse<String> response = Unirest.post("https://users.rime.ai/oov")
    .header("Accept", "application/json")
    .header("Authorization", "Bearer <authorization>")
    .header("Content-Type", "application/json")
    .body("{\n  \"text\": \"<string>\"\n}")
    .asString();
```

--------------------------------

### Install Rime CLI with curl

Source: https://docs.rime.ai/cli-reference/overview

Use this command to download and install the Rime CLI script.

```bash
curl -fsSL https://rime.ai/install-cli.sh | sh
```

--------------------------------

### Install uv Package Manager

Source: https://docs.rime.ai/docs/pipecat

Install the `uv` package manager for managing Python dependencies. Choose the installation method appropriate for your operating system.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

```bash
pip install uv
```

```bash
brew install uv
```

--------------------------------

### Start Docker Compose

Source: https://docs.rime.ai/docs/on-prem/quickstart

Command to start all defined services in the Docker Compose file in detached mode.

```bash
docker compose up -d
```

--------------------------------

### Coda Websockets API Usage

Source: https://docs.rime.ai/api-reference/coda/websockets

This example demonstrates how to connect to the Coda websockets API, send text for synthesis, and receive audio data. It includes setup for authentication and saving the generated audio.

```APIDOC
## Coda Websockets API

This section details the parameters available for the Coda websockets API and provides a Python example for its usage.

### Parameters

#### Request Body Parameters

*   **speaker** (string, required): Must be one of the voices listed in our documentation for `coda`.
*   **text** (string, required): The text you'd like spoken. Character limit per request is 500 via the API and 1,000 in the dashboard UI.
*   **modelId** (string): This value **must** be set to `coda` else the websockets server will default to `mistv2` for speech synthesis.
*   **audioFormat** (string): One of `mp3`, `ogg`, `mulaw`, or `pcm`.
*   **language** (string, default: "eng"): If provided, the language must match the language spoken by the provided speaker. Both the 2-letter ISO 639-1 and the 3-letter ISO 639-2/3 form are accepted.
    *   `en` | `eng` - English
    *   `es` | `spa` - Spanish
    *   `fr` | `fra` - French
    *   `pt` | `por` - Portuguese
    *   `de` | `deu` - German
    *   `ja` | `jpn` - Japanese
    See the voices documentation for which speakers support each language.
*   **samplingRate** (int, default: 24000): The sampling rate (Hz).
    *   On-cloud: Accepted values: 8000, 16000, 22050, 24000, 44100, 48000, 96000. Anything above 24000 is up sampling.
    *   On-prem: Any value is accepted.
*   **segment** (string, default: "bySentence"): Controls how text is segmented for synthesis. Available options:
    *   "immediate" - Synthesizes text immediately without waiting for complete sentences
    *   "never" - Never segments the text, waits for explicit flush or EOS
    *   "bySentence" (default) - Waits for complete sentences before synthesis
    Note: For backward compatibility, setting `immediate=true` in query params is equivalent to `segment=immediate`. If a null value is provided, it will default to "bySentence".

### Request Example (Python)

```python
import asyncio
import websockets
import os
from dotenv import load_dotenv

load_dotenv()
api_key = os.environ.get('RIME_API_KEY')
if not api_key:
    raise ValueError("RIME_API_KEY environment variable is not set")

FILE_PATH = "coda_ws.wav"

class RimeClient:
    def __init__(self, speaker, api_key):
        self.url = f"wss://users-ws.rime.ai/ws?speaker={speaker}&modelId=coda&audioFormat=wav"
        self.auth_headers = {
            "Authorization": f"Bearer {api_key}"
        }
        self.audio_data = b''

    async def send_tokens(self, websocket, message):
        for token in message:
            await websocket.send(token)

    async def handle_audio(self, websocket):
        while True:
            try:
                audio = await websocket.recv()
            except websockets.exceptions.ConnectionClosedOK:
                break
            self.audio_data += audio

    async def run(self, message):
        async with websockets.connect(self.url, additional_headers=self.auth_headers) as websocket:
            await asyncio.gather(
                self.send_tokens(websocket, message),
                self.handle_audio(websocket),
            )

    def save_audio(self, file_path):
        with open(file_path, 'wb') as f:
            f.write(self.audio_data)

message = [
    "This ",
    "is ",
    "a ",
    "test ",
    "of ",
    "the ",
    "coda ",
    "model ",
    "using ",
    "websockets ",
    "and ",
    "python.",
    "<EOS>",
]

client = RimeClient("astra", api_key=api_key)
asyncio.run(client.run(message))

print(f"Saving audio to {FILE_PATH}")
client.save_audio(FILE_PATH)
```
```

--------------------------------

### Generate Example Curl Command

Source: https://docs.rime.ai/cli-reference/rime-curl

Run without arguments to display a default example curl command for the Rime TTS API.

```bash
rime curl
```

--------------------------------

### Verify Python Installation

Source: https://docs.rime.ai/docs/pipecat

Check if Python 3.10 or later is installed on your system.

```bash
python --version
```

--------------------------------

### Verify Rime CLI Installation

Source: https://docs.rime.ai/docs/quickstart-cli

Checks the installed version of the Rime CLI.

```bash
rime --version
```

--------------------------------

### Example Usage of IntegrationHeader

Source: https://docs.rime.ai/docs/lovable

This is an example of how to use the IntegrationHeader component, specifying the partner logo and adjusting the heights of both logos for the display.

```javascript
<IntegrationHeader partnerLogo="/images/lovable.png" partnerAlt="Lovable logo" rimeHeight="50px" partnerHeight="32px" />
```

--------------------------------

### Arcana Container Liveness Response Example

Source: https://docs.rime.ai/docs/on-prem/metrics

A typical response from the /readyz endpoint, indicating the model's readiness.

```text
ok
```

--------------------------------

### Agent Greeting Example

Source: https://docs.rime.ai/docs/quickstart-livekit

This Python code snippet demonstrates how to make your agent speak a predefined greeting. Ensure your session is properly initialized before calling this.

```python
await session.say("Hey there! How can I help you today?")
```

--------------------------------

### System Prompt Example

Source: https://docs.rime.ai/docs/vapi

Define the assistant's personality and behavior using a system prompt. Keep responses concise and conversational.

```text
You are a helpful voice assistant.
Keep your responses short and conversational. No more than 2-3 sentences.
Be friendly and natural.
```

--------------------------------

### Example Rime TOML Configuration

Source: https://docs.rime.ai/cli-reference/rime-config

Illustrates the structure of a generated Rime configuration file, including default environment settings and named environments with API URLs and keys.

```toml
default_env = "users"

[env.users]
api_url = "https://users.rime.ai/v1/rime-tts"
api_key = "your_api_key_here"

[env.users-east]
api_url = "https://users-east.rime.ai/v1/rime-tts"
api_key = "your_api_key_here"
```

--------------------------------

### Install NVIDIA Drivers on Debian

Source: https://docs.rime.ai/docs/on-prem/quickstart

Installs the NVIDIA drivers on Debian-based systems. Ensure you have the correct driver version for your hardware.

```bash
# Update packages
sudo apt-get update

# Install basic toolchain and kernel headers
sudo apt-get install -y gcc make wget linux-headers-$(uname -r)

# Download and install the NVIDIA driver.
NVIDIA_DRIVER_VERSION=580.95.05
NVIDIA_DRIVER_PATH=/opt/NVIDIA-Linux-x86_64-${NVIDIA_DRIVER_VERSION}.run
sudo rm -f "${NVIDIA_DRIVER_PATH}"
sudo wget "https://us.download.nvidia.com/tesla/${NVIDIA_DRIVER_VERSION}/NVIDIA-Linux-x86_64-${NVIDIA_DRIVER_VERSION}.run" -O "${NVIDIA_DRIVER_PATH}"
sudo chmod +x "${NVIDIA_DRIVER_PATH}"
sudo "${NVIDIA_DRIVER_PATH}" --silent --no-questions
```

--------------------------------

### OOV API Response Examples

Source: https://docs.rime.ai/api-reference/other/oov

These examples show the expected JSON response from the OOV API. An empty array indicates all words were found in the dictionary.

```json
["testt", "testtt"]
```

```json
[]
```

--------------------------------

### Render LiveKit Integration Header

Source: https://docs.rime.ai/docs/livekit

Example of rendering the IntegrationHeader component with LiveKit's logo. Ensure the partnerLogo and partnerAlt props are correctly set for LiveKit.

```javascript
<IntegrationHeader partnerLogo="/images/livekit.jpg" partnerAlt="LiveKit logo" />
```

--------------------------------

### Install Rime-Reader OpenClaw Skill

Source: https://docs.rime.ai/docs/openclaw

Clone the rime-reader skill repository into your OpenClaw skills directory to enable verbatim, summary, and podcast TTS modes.

```bash
git clone https://github.com/rimelabs/rime-reader-openclaw ~/.openclaw/skills/rime-reader
```

--------------------------------

### Render Integration Header with SignalWire Logo

Source: https://docs.rime.ai/docs/signalwire

Example of rendering the IntegrationHeader component with the SignalWire logo. Ensure the partnerLogo prop points to the correct image path.

```javascript
<IntegrationHeader partnerLogo="/images/signalwire.png" partnerAlt="SignalWire logo" />
```

--------------------------------

### Python WebSocket Client Example

Source: https://docs.rime.ai/api-reference/mistv3/websockets-json

A Python client demonstrating how to connect to the Mist V3 WebSocket API, send text messages, and receive audio data.

```Python
import asyncio
import json
import websockets
import base64

class RimeClient:
    def __init__(self, speaker, api_key):
        self.url = f"wss://users-ws.rime.ai/ws3?speaker={speaker}&modelId=mistv3&audioFormat=mp3"
        self.auth_headers = {
            "Authorization": f"Bearer {api_key}"
        }
        self.audio_data = b''

    async def send_messages(self, websocket, messages):
        for message in messages:
            await websocket.send(json.dumps(message))

    async def handle_audio(self, websocket):
        while True:
            try:
                audio = await websocket.recv()
            except websockets.exceptions.ConnectionClosedOK:
                break
            message = json.loads(audio)

            if message['type'] == 'chunk':
              self.audio_data += base64.b64decode(message['data'])

            if message['type'] == 'timestamps':
                print("Rime model pronounced the words...\n")
                for w, t in zip(message['word_timestamps']['words'], message['word_timestamps']['start']):
                    print(f"'{w}' at time {t}")

    async def run(self, messages):
        async with websockets.connect(self.url, additional_headers=self.auth_headers) as websocket:
            await asyncio.gather(
                self.send_messages(websocket, messages),
                self.handle_audio(websocket),
            )

    def save_audio(self, file_path):
        with open(file_path, 'wb') as f:
            f.write(self.audio_data)
        print(f"\n Audio saved at {file_path}")


message = [
    {"text": "This "},
    {"text": "is "},
    {"text": "an "},
    {"text": "incomplete "},
    {"text": "sentence "},
    {"operation": "eos"},
]

client = RimeClient("cove", api_key="YOUR_API_KEY")
asyncio.run(client.run(message))

client.save_audio("output.mp3")

```

--------------------------------

### Tokenization Example for bySentence Segmentation

Source: https://docs.rime.ai/docs/websockets-segment

Illustrates correct and incorrect tokenization when using `segment=bySentence`. Ensure tokens are not split at sentence-ending punctuation and that sentences are separated by spaces to avoid words running together.

```text
❌  tokens = ["take ", "2.", "5ml ", "of ", "the ", "solution."]
✅  tokens = ["take ", "2.5ml ", "of ", "the ", "solution."]
```

--------------------------------

### Main Entrypoint for Agent

Source: https://docs.rime.ai/docs/pipecat

The main entrypoint for the agent script. When executed, it calls the Pipecat 'main' helper to discover the bot function, start a FastAPI server, and serve a browser client.

```python
if __name__ == "__main__":
    main()

```

--------------------------------

### Create Project Folder

Source: https://docs.rime.ai/docs/quickstart-livekit

Create a new directory for your project and navigate into it.

```bash
mkdir rime-voice-agent
cd rime-voice-agent
```

--------------------------------

### Set Up Environment Variables

Source: https://docs.rime.ai/docs/quickstart-livekit

Create a .env file to store your API keys and LiveKit credentials. Replace placeholder values with your actual credentials.

```dotenv
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret
OPENAI_API_KEY=your_openai_api_key
RIME_API_KEY=your_rime_api_key
```

--------------------------------

### Try Different Voices and Models

Source: https://docs.rime.ai/docs/quickstart-cli

Demonstrates synthesizing speech with different speaker and model combinations. Note that some models default to WAV output, while others default to MP3. Use the --format flag to override.

```bash
rime tts "The quick brown fox." -s orion -m coda
```

```bash
rime tts "The quick brown fox." -s luna -m coda
```

```bash
rime tts "The quick brown fox." -s peak -m mistv3
```

```bash
rime tts "The quick brown fox." -s peak -m mistv2
```

--------------------------------

### Create and Activate Virtual Environment (macOS/Linux)

Source: https://docs.rime.ai/docs/quickstart-livekit

Create a Python virtual environment and activate it for managing project dependencies.

```bash
python -m venv venv
source venv/bin/activate
```

--------------------------------

### Main Agent Entrypoint

Source: https://docs.rime.ai/docs/quickstart-livekit

Configures and runs the agent server using `cli.run_app`. It specifies the functions for prewarming models and handling new connections, allowing the agent to listen for incoming LiveKit connections.

```python
if __name__ == "__main__":
    cli.run_app(
        WorkerOptions(
            entrypoint_fnc=entrypoint,
            prewarm_fnc=prewarm,
        ),
    )
```

--------------------------------

### Configure Environment Variables

Source: https://docs.rime.ai/docs/pipecat

Set up your Rime and OpenAI API keys in the `.env` file.

```dotenv
RIME_API_KEY=your_rime_api_key
OPENAI_API_KEY=your_openai_api_key
```

--------------------------------

### Timestamps Event Payload Example

Source: https://docs.rime.ai/api-reference/coda/websockets-json

An example payload for the TimestampsEvent, demonstrating word-level timing information for a synthesized phrase.

```json
{
  "type": "timestamps",
  "word_timestamps": {
    "words": ["Hello", "from", "Coda", "over", "JSON", "websockets."],
    "start": [0, 0.36106, 0.54159, 0.72212, 0.90265, 1.08318],
    "end":   [0.36106, 0.54159, 0.72212, 0.90265, 1.08318, 2.88848]
  },
  "contextId": null
}
```

--------------------------------

### Initialize Rime CLI Configuration

Source: https://docs.rime.ai/cli-reference/rime-config

Creates a new Rime configuration file. It interactively prompts for an API key. Use the `--force` flag to overwrite an existing file.

```bash
rime config init
```

--------------------------------

### Create and Activate Virtual Environment (Windows)

Source: https://docs.rime.ai/docs/quickstart-livekit

Create a Python virtual environment and activate it for managing project dependencies on Windows.

```bash
python -m venv venv
virtualenv\Scripts\activate
```

--------------------------------

### Generated Curl Command Example

Source: https://docs.rime.ai/docs/quickstart-cli

An example of a generated curl command for the Rime TTS API, including authentication and payload details.

```bash
curl --request POST \
  --url 'https://users.rime.ai/v1/rime-tts' \
  --header 'Accept: audio/wav' \
  --header 'Authorization: Bearer $RIME_CLI_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
  "text": "Hello from Rime",
  "speaker": "astra",
  "modelId": "coda",
  "lang": "eng"
}'
```

--------------------------------

### Test Rime CLI with 'hello' command

Source: https://docs.rime.ai/docs/quickstart-cli

Runs a simple 'hello' command to confirm the CLI is working correctly.

```bash
rime hello
```

--------------------------------

### Full Agent Code Example

Source: https://docs.rime.ai/docs/quickstart-livekit

This snippet shows the complete Python code for a LiveKit agent, including all necessary imports, the Assistant class definition, prewarming logic, and the main entrypoint function for running the agent with worker options.

```python
from dotenv import load_dotenv
from livekit.agents import (
    Agent,
    AgentSession,
    AutoSubscribe,
    JobContext,
    JobProcess,
    RoomInputOptions,
    WorkerOptions,
    cli,
)
from livekit.plugins import openai, noise_cancellation, rime, silero
from livekit.plugins.turn_detector.multilingual import MultilingualModel

load_dotenv()

class Assistant(Agent):
    def __init__(self):
        super().__init__(
            instructions="""You are a helpful voice assistant.
            Keep your responses short and conversational - no more than 2-3 sentences.
            Be friendly and natural."""
        )

def prewarm(proc: JobProcess):
    proc.userdata["vad"] = silero.VAD.load()

async def entrypoint(ctx: JobContext):
    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
    await ctx.wait_for_participant()

    session = AgentSession(
        stt=openai.STT(model="gpt-4o-transcribe"),
        llm=openai.LLM(model="gpt-4o-mini"),
        tts=rime.TTS(model="arcana", speaker="luna"),
        vad=ctx.proc.userdata["vad"],
        turn_detection=MultilingualModel(),
    )

    await session.start(
        room=ctx.room,
        agent=Assistant(),
        room_input_options=RoomInputOptions(
            noise_cancellation=noise_cancellation.BVC()
        ),
    )

    await session.say("Hey there! How can I help you today?")

if __name__ == "__main__":
    cli.run_app(
        WorkerOptions(
            entrypoint_fnc=entrypoint,
            prewarm_fnc=prewarm,
        ),
    )

```

--------------------------------

### Phonemize API Response Example

Source: https://docs.rime.ai/api-reference/other/phonemize

Example JSON response from the Phonemize API. It includes the audioId, the phonetic transcription (phonemeString), and an authentication status.

```json
{
  "audioId": "9b2d8ad2-0618-4e96-b255-98b2f7488061",
  "phonemeString": "h0El1o !",
  "authed": 1
}
```

--------------------------------

### Synthesize Phrases Immediately with segment=immediate

Source: https://docs.rime.ai/docs/websockets-segment

This example demonstrates sending pre-segmented phrases to Rime for immediate synthesis. Ensure each phrase is a complete, speakable utterance. A small delay is included between phrases to allow synthesis to begin.

```python
import asyncio
import json
import websockets
import base64

API_KEY = "your-api-key"
URL = "wss://users-ws.rime.ai/ws3?speaker=astra&modelId=coda&segment=immediate"

async def run():
    async with websockets.connect(URL, additional_headers={"Authorization": f"Bearer {API_KEY}"}) as ws:
        # Each message is already a complete phrase
        phrases = [
            "Your order has been confirmed.",
            "It will arrive within three to five business days.",
            "Is there anything else I can help you with?",
        ]
        for phrase in phrases:
            await ws.send(json.dumps({"text": phrase}))
            await asyncio.sleep(0.05)  # small delay to allow synthesis to start

        await ws.send(json.dumps({"operation": "eos"}))

        while True:
            try:
                msg = json.loads(await ws.recv())
            except websockets.exceptions.ConnectionClosedOK:
                break
            if msg["type"] == "chunk":
                audio = base64.b64decode(msg["data"])
                # stream to player...

asyncio.run(run())
```

--------------------------------

### HTTP ORCA Header Example

Source: https://docs.rime.ai/docs/on-prem/load-balancing

This is an example of the ORCA header returned in HTTP responses by Arcana model containers. It provides metrics for load balancing.

```text
endpoint-load-metrics: TEXT application_utilization=0.5, cpu_utilization=0.3128, mem_utilization=0.2453, rps_fractional=0.0000, eps=0.0000
```

--------------------------------

### Full Agent Setup with Pipecat

Source: https://docs.rime.ai/docs/pipecat

This code sets up a full agent pipeline using various Pipecat services for speech-to-text, language processing, text-to-speech, and real-time communication. It requires environment variables for API keys.

```python
import os
from dotenv import load_dotenv

from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.frames.frames import LLMMessagesAppendFrame
from pipecat.services.openai.stt import OpenAISTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.services.rime.tts import RimeNonJsonTTSService
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.transports.base_transport import TransportParams
from pipecat.transports.smallwebrtc.transport import SmallWebRTCTransport
from pipecat.runner.run import main
from pipecat.runner.types import SmallWebRTCRunnerArguments

load_dotenv()

SYSTEM_PROMPT = """You are a helpful voice assistant.
Keep your responses short and conversational - no more than 2-3 sentences.
Be friendly and natural."""


async def bot(runner_args: SmallWebRTCRunnerArguments):
    transport = SmallWebRTCTransport(
        runner_args.webrtc_connection,
        TransportParams(
            audio_in_enabled=True,
            audio_out_enabled=True,
            vad_analyzer=SileroVADAnalyzer(),
        ),
    )

    stt = OpenAISTTService(
        api_key=os.getenv("OPENAI_API_KEY"),
        model="gpt-4o-transcribe",
    )

    llm = OpenAILLMService(
        api_key=os.getenv("OPENAI_API_KEY"),
        model="gpt-4o-mini",
    )

    tts = RimeNonJsonTTSService(
        api_key=os.getenv("RIME_API_KEY"),
        voice_id="atrium",
        model="arcana",
    )

    context = OpenAILLMContext(
        messages=[{"role": "system", "content": SYSTEM_PROMPT}]
    )
    context_aggregator = llm.create_context_aggregator(context)

    pipeline = Pipeline([
        transport.input(),
        stt,
        context_aggregator.user(),
        llm,
        tts,
        transport.output(),
        context_aggregator.assistant(),
    ])

    task = PipelineTask(
        pipeline,
        params=PipelineParams(enable_metrics=True),
    )

    @transport.event_handler("on_client_connected")
    async def on_client_connected(transport, client):
        await task.queue_frames([LLMMessagesAppendFrame(
            messages=[{"role": "system", "content": "Say hello and introduce yourself." }],
            run_llm=True
        )])

    runner = PipelineRunner(handle_sigint=runner_args.handle_sigint)
    await runner.run(task)


if __name__ == "__main__":
    main()

```

--------------------------------

### Timestamps Event Example

Source: https://docs.rime.ai/api-reference/mistv3/websockets-json

An example payload for the timestamps event, providing word-level timing information. This helps in mapping audio playback to spoken words.

```json
{
  "type": "timestamps",
  "word_timestamps": {
    "words": ["Testing", "mistv3", "timestamps."],
    "start": [0, 0.35396, 1.41584],
    "end":   [0.35396, 1.41584, 3.18564]
  },
  "contextId": null
}
```

--------------------------------

### Verify NVIDIA Driver and CUDA Installation

Source: https://docs.rime.ai/docs/on-prem/quickstart

Verifies that the NVIDIA drivers and CUDA toolkit are correctly installed and accessible by Docker. This command should list your GPU details.

```bash
docker run --rm --gpus all nvidia/cuda:12.8.1-base-ubi9 nvidia-smi
```

--------------------------------

### Generate a Greeting with `rime hello`

Source: https://docs.rime.ai/cli-reference/rime-hello-play

Use this command to generate a time-appropriate greeting. By default, it plays the audio. You can use the `--output` flag to save the greeting to a file or `--json` to output as JSON.

```bash
rime hello
```

```bash
rime hello -o greeting.wav
```

```bash
rime hello --json
```

--------------------------------

### Text Normalization API Request Examples

Source: https://docs.rime.ai/api-reference/other/textnorm

Examples of how to call the text normalization API using various programming languages. Ensure you replace placeholders like YOUR_API_KEY or <authorization> with your actual credentials.

```bash
curl --request POST \
  --url https://optimize.rime.ai/textnorm \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --fail \
  --data '{ 
    "text": "<string>"
  }'
```

```python
import requests

url = "https://optimize.rime.ai/textnorm"

payload = {
    "text": "<string>"
}
headers = {
    "Accept": "application/json",
    "Authorization": "Bearer <authorization>",
    "Content-Type": "application/json"
}

response = requests.request("POST", url, json=payload, headers=headers)

print(response.text)
```

```javascript
const options = {
  method: 'POST',
  headers: {
    Accept: 'application/json',
    Authorization: 'Bearer <authorization>',
    'Content-Type': 'application/json'
  },
  body: '{"text":"<string>"}'
};

fetch('https://optimize.rime.ai/textnorm', options)
  .then(response => response.json())
  .then(response => console.log(response))
  .catch(err => console.error(err));
```

```php
<?php

$curl = curl_init();

curl_setopt_array($curl, [
  CURLOPT_URL => "https://optimize.rime.ai/textnorm",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "POST",
  CURLOPT_POSTFIELDS => "{\n  \"text\": \"<string>\"\n}",
  CURLOPT_HTTPHEADER => [
    "Accept: application/json",
    "Authorization: Bearer <authorization>",
    "Content-Type: application/json"
  ],
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}
```

```go
package main

import (
  "fmt"
  "strings"
  "net/http"
  "io/ioutil"
)

func main() {

  url := "https://optimize.rime.ai/textnorm"

  payload := strings.NewReader("{\n  \"text\": \"<string>\"\n}")

  req, _ := http.NewRequest("POST", url, payload)

  req.Header.Add("Accept", "application/json")
  req.Header.Add("Authorization", "Bearer <authorization>")
  req.Header.Add("Content-Type", "application/json")

  res, _ := http.DefaultClient.Do(req)

  defer res.Body.Close()
  body, _ := ioutil.ReadAll(res.Body)

  fmt.Println(res)
  fmt.Println(string(body))

}
```

```java
HttpResponse<String> response = Unirest.post("https://optimize.rime.ai/textnorm")
    .header("Accept", "application/json")
    .header("Authorization", "Bearer <authorization>")
    .header("Content-Type", "application/json")
    .body("{\n  \"text\": \"<string>\"\n}")
    .asString();
```

--------------------------------

### Create Project Directory

Source: https://docs.rime.ai/docs/pipecat

Create a new directory for your Rime Pipecat agent project and navigate into it.

```bash
mkdir rime-pipecat-agent
cd rime-pipecat-agent
```

--------------------------------

### Phonemize API cURL Request

Source: https://docs.rime.ai/api-reference/other/phonemize

Example cURL request to the Phonemize API. This demonstrates how to send audio data and authentication headers.

```bash
curl --request POST \
  --url https://optimize.rime.ai/phonemize \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: audio/wav' \
  --fail \
  --data-binary @speech.wav
```

--------------------------------

### API Container Health Response Example

Source: https://docs.rime.ai/docs/on-prem/metrics

A typical JSON response from the /health endpoint, indicating the status of the API, license, and model reachability.

```json
{
  "apiStatus": "ok",
  "timestamp": "2025-04-03T13:59:58.902Z",
  "licenseStatus": "valid",
  "modelReachable": true
}
```

--------------------------------

### Streaming PCM Endpoint Request

Source: https://docs.rime.ai/docs/latency

Example cURL request to the streaming PCM endpoint. Ensure to replace $RIME_API_KEY with your actual API key.

```bash
curl -X POST http://users.rime.ai/v1/rime-tts \
  -H 'Accept: audio/L16' \
  -H 'Authorization: Bearer $RIME_API_KEY' \
  -d '{"text": "I love the book that she gave me. And I was so happy that she gave it to me.", "speaker": "cove",  "modelId": "mistv3"}'
```

--------------------------------

### Agent Conversation Pipeline

Source: https://docs.rime.ai/docs/quickstart-livekit

Sets up the agent's conversation flow, including connecting to a LiveKit room, configuring the voice pipeline with STT, LLM, TTS, and VAD, and initiating the session with noise cancellation.

```python
async def entrypoint(ctx: JobContext):
    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
    await ctx.wait_for_participant()

    session = AgentSession(
        stt=openai.STT(model="gpt-4o-transcribe"),
        llm=openai.LLM(model="gpt-4o-mini"),
        tts=rime.TTS(model="arcana", speaker="luna"),
        vad=ctx.proc.userdata["vad"],
        turn_detection=MultilingualModel(),
    )

    await session.start(
        room=ctx.room,
        agent=Assistant(),
        room_input_options=RoomInputOptions(
            noise_cancellation=noise_cancellation.BVC()
        ),
    )

    await session.say("Hey there! How can I help you today?")
```

--------------------------------

### Push Baseten Deployment Config

Source: https://docs.rime.ai/docs/baseten

Use the Truss CLI to push your Baseten deployment configuration. Ensure you have cloned the repository and navigated to the model directory.

```bash
truss push .

```

--------------------------------

### Curl with Alternate API Key Header

Source: https://docs.rime.ai/docs/on-prem/quickstart

Example of using curl to authenticate with an alternate API key header, as configured by the API_KEY_HEADER environment variable.

```bash
curl -H "x-my-platform-rime-api-key: <your-rime-api-key>" ...
```

--------------------------------

### Example Prometheus Metrics

Source: https://docs.rime.ai/docs/on-prem/metrics

Illustrative output from the /metrics endpoint, showing HTTP request and error counts. These metrics can be integrated with Prometheus for monitoring and alerting.

```json
# HELP http_requests_total Total number of HTTP requests
# TYPE http_requests_total counter
http_requests_total{endpoint="/invocations",http_status="200",method="POST"} 102018.0
http_requests_total{endpoint="/metrics",http_status="200",method="GET"} 69908.0
http_requests_total{endpoint="/invocations",http_status="500",method="POST"} 38.0

# HELP http_errors_total Total number of HTTP errors
# TYPE http_errors_total counter
http_errors_total{endpoint="/invocations",error_message="cannot access local variable '_var_var_6' where it is not associated with a value",http_status="500",method="POST"} 38.0
```

--------------------------------

### Synthesize Speech and Save to File

Source: https://docs.rime.ai/docs/quickstart-cli

Generates speech from text and saves the output to a WAV file named 'welcome.wav'.

```bash
rime tts "Hey, how's it going?" -s celeste -m coda -o welcome.wav
```

--------------------------------

### Example Voice Details Structure

Source: https://docs.rime.ai/api-reference/data/voice-details

This JSON structure illustrates the format of the data returned by the voice details endpoint. Each object represents a voice with its attributes.

```json
[
    {
        "speaker": "astra",
        "gender": "Female",
        "age": "Young Adult",
        "country": "US",
        "dialect": "American",
        "demographic": "White",
        "genre": [
            "Any"
        ],
        "modelId": "coda",
        "language": "English",
        "lang": "eng",
        "flagship": true
    },
    {
        "speaker": "luna",
        "gender": "Female",
        "age": "Young Adult",
        "country": "US",
        "dialect": "American",
        "demographic": "White",
        "genre": [
            "Any"
        ],
        "modelId": "arcana",
        "language": "English",
        "lang": "eng",
        "flagship": true
    },
    {
        "speaker": "cove",
        "gender": "Male",
        "age": "Middle",
        "country": "US",
        "dialect": "Generic",
        "demographic": "General American",
        "genre": [
            "Conversational"
        ],
        "modelId": "mistv3",
        "language": "English",
        "lang": "eng"
    },
    // and more...
]
```
Tokens:
Raw