### Python SDK Quick Start for LLM Gateway Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt A basic Python example demonstrating how to initialize the OpenAI client to connect to the LLM Gateway. It shows setting the base URL and API key for accessing the service. ```python from openai import OpenAI client = OpenAI( base_url="https://apim-ai-apis.azure-api.net/v1", api_key="YOUR_KEY" ) ``` -------------------------------- ### MCP Configuration Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Configuration example for setting up MCP servers in clients like Claude Desktop, Cursor, or Cline. ```APIDOC ## MCP Configuration (Claude Desktop / Cursor / Cline) ### Description Configuration structure for integrating with MCP servers, including URLs and necessary headers. ### Configuration Example ```json { "mcpServers": { "brainiall-speech": { "url": "https://apim-ai-apis.azure-api.net/mcp/pronunciation/mcp", "headers": {"Ocp-Apim-Subscription-Key": "YOUR_KEY"} }, "brainiall-nlp": { "url": "https://apim-ai-apis.azure-api.net/mcp/nlp/mcp", "headers": {"Ocp-Apim-Subscription-Key": "YOUR_KEY"} }, "brainiall-image": { "url": "https://apim-ai-apis.azure-api.net/mcp/image/mcp", "headers": {"Ocp-Apim-Subscription-Key": "YOUR_KEY"} } } } ``` ### Parameters - **mcpServers** (object) - Contains configurations for different MCP services. - **[service_name]** (object) - Configuration for a specific MCP service (e.g., `brainiall-speech`). - **url** (string) - The endpoint URL for the MCP service. - **headers** (object) - Headers required for authenticating with the MCP service (e.g., `Ocp-Apim-Subscription-Key`). ``` -------------------------------- ### Get Available Models (GET /v1/models) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Retrieves a list of all available models, including their pricing, context length, and capabilities. This endpoint is useful for understanding the range of models accessible through the LLM Gateway. -------------------------------- ### Get Text-to-Speech Voices Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Retrieves a list of available text-to-speech voices, including their IDs, names, genders, and accents. The voice ID format indicates the accent and gender. ```json { "voices": [ {"id": "af_heart", "name": "Heart", "gender": "female", "accent": "american"}, {"id": "af_bella", "name": "Bella", "gender": "female", "accent": "american"}, {"id": "af_nicole", "name": "Nicole", "gender": "female", "accent": "american"}, {"id": "af_sarah", "name": "Sarah", "gender": "female", "accent": "american"}, {"id": "af_sky", "name": "Sky", "gender": "female", "accent": "american"}, {"id": "am_adam", "name": "Adam", "gender": "male", "accent": "american"}, {"id": "am_michael", "name": "Michael", "gender": "male", "accent": "american"}, {"id": "bf_emma", "name": "Emma", "gender": "female", "accent": "british"}, {"id": "bf_isabella", "name": "Isabella", "gender": "female", "accent": "british"}, {"id": "bm_george", "name": "George", "gender": "male", "accent": "british"}, {"id": "bm_lewis", "name": "Lewis", "gender": "male", "accent": "british"}, {"id": "bm_daniel", "name": "Daniel", "gender": "male", "accent": "british"} ], "defaultVoice": "af_heart" } ``` -------------------------------- ### GET /v1/tts/voices Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Retrieves a list of available text-to-speech voices, including their IDs, names, genders, and accents. Also specifies the default voice. ```APIDOC ## GET /v1/tts/voices ### Description Retrieves a list of available text-to-speech voices, including their IDs, names, genders, and accents. Also specifies the default voice. ### Method GET ### Endpoint /v1/tts/voices ### Parameters #### Query Parameters None #### Request Body None ### Response #### Success Response (200) - **voices** (array) - A list of voice objects, each containing `id`, `name`, `gender`, and `accent`. - **defaultVoice** (string) - The ID of the default voice. #### Response Example ```json { "voices": [ {"id": "af_heart", "name": "Heart", "gender": "female", "accent": "american"}, {"id": "af_bella", "name": "Bella", "gender": "female", "accent": "american"}, {"id": "af_nicole", "name": "Nicole", "gender": "female", "accent": "american"}, {"id": "af_sarah", "name": "Sarah", "gender": "female", "accent": "american"}, {"id": "af_sky", "name": "Sky", "gender": "female", "accent": "american"}, {"id": "am_adam", "name": "Adam", "gender": "male", "accent": "american"}, {"id": "am_michael", "name": "Michael", "gender": "male", "accent": "american"}, {"id": "bf_emma", "name": "Emma", "gender": "female", "accent": "british"}, {"id": "bf_isabella", "name": "Isabella", "gender": "female", "accent": "british"}, {"id": "bm_george", "name": "George", "gender": "male", "accent": "british"}, {"id": "bm_lewis", "name": "Lewis", "gender": "male", "accent": "british"}, {"id": "bm_daniel", "name": "Daniel", "gender": "male", "accent": "british"} ], "defaultVoice": "af_heart" } ``` **Voice ID format**: `af_` = American female, `am_` = American male, `bf_` = British female, `bm_` = British male. ``` -------------------------------- ### LLM Gateway (OpenAI-compatible) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Provides access to 113+ models from 17 providers via AWS Bedrock, compatible with OpenAI SDK and other tools. ```APIDOC ## LLM Gateway (OpenAI-compatible) ### Description 113+ models from 17 providers via AWS Bedrock. Standard Bedrock pricing, no markup. Fully compatible with OpenAI SDK, LiteLLM, LangChain, Cline, Cursor, Aider, and any OpenAI-compatible tool. ### POST /v1/chat/completions #### Description Sends a chat message to the LLM and receives a response. #### Method POST #### Endpoint /v1/chat/completions #### Parameters ##### Request Body - **model** (string) - Required - The model to use for generation (e.g., "claude-sonnet"). - **messages** (array) - Required - A list of message objects, each with a `role` and `content`. - **role** (string) - The role of the message sender (e.g., "system", "user", "assistant"). - **content** (string) - The content of the message. - **temperature** (number) - Optional - Controls randomness. Lower values make output more deterministic. Defaults to 0.7. - **max_tokens** (integer) - Optional - The maximum number of tokens to generate. - **stream** (boolean) - Optional - Whether to stream the response. Defaults to false. - **stream_options** (object) - Optional - Options for streaming. - **tools** (array) - Optional - A list of tools the model may call. - **tool_choice** (object or string) - Optional - Controls how the tools are used. - **response_format** (object) - Optional - Specifies the desired response format (e.g., `{"type": "json_object"}`). - **type** (string) - The type of response format (e.g., "json_object", "json_schema"). - **stop** (string or array) - Optional - Sequences where the API will stop generating further tokens. - **seed** (integer) - Optional - A seed for reproducible results. - **reasoning_effort** (integer) - Optional - Controls the reasoning effort of the model. ##### Request Example ```json { "model": "claude-sonnet", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is 2 + 2?"} ], "temperature": 0.7, "max_tokens": 1024, "stream": false } ``` ##### Success Response (200) - **id** (string) - Unique identifier for the chat completion. - **choices** (array) - A list of chat completion choices. - **message** (object) - The generated message. - **role** (string) - The role of the message sender (e.g., "assistant"). - **content** (string) - The content of the message. - **finish_reason** (string) - The reason the model stopped generating tokens (e.g., "end_turn"). - **usage** (object) - Usage statistics for the request. - **prompt_tokens** (integer) - Number of tokens in the prompt. - **completion_tokens** (integer) - Number of tokens in the completion. - **total_tokens** (integer) - Total number of tokens used. ##### Response Example ```json { "id": "chatcmpl-abc123", "choices": [{"message": {"role": "assistant", "content": "2 + 2 = 4."}, "finish_reason": "end_turn"}], "usage": {"prompt_tokens": 28, "completion_tokens": 9, "total_tokens": 37} } ``` ### GET /v1/models #### Description Returns a list of all available models with their pricing, context length, and capabilities. #### Method GET #### Endpoint /v1/models ### POST /v1/embeddings #### Description Generates embeddings for the given input text. #### Method POST #### Endpoint /v1/embeddings #### Parameters ##### Request Body - **input** (string or array) - Required - The input text or list of texts to generate embeddings for. - **model** (string) - Required - The model to use for generating embeddings (e.g., "titan-embed-v2"). ##### Request Example ```json { "input": "The quick brown fox", "model": "titan-embed-v2" } ``` ``` -------------------------------- ### POST /v1/image/restore-face/base64 Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Performs GFPGAN face restoration, including optional background enhancement. ```APIDOC ## POST /v1/image/restore-face/base64 ### Description GFPGAN face restoration. Detects all faces, restores quality, optional background enhancement. ### Method POST ### Endpoint /v1/image/restore-face/base64 ### Parameters #### Request Body - **image** (string) - Required - Base64 encoded image string. - **upscale** (integer) - Optional - The upscaling factor for faces. Defaults to 2. - **enhance_background** (boolean) - Optional - Whether to enhance the background. Defaults to true. ### Request Example ```json { "image": "", "upscale": 2, "enhance_background": true } ``` ### Response #### Success Response (200) - **image** (string) - Base64 encoded restored image. - **format** (string) - The format of the output image (e.g., "png"). - **width** (integer) - The width of the output image. - **height** (integer) - The height of the output image. - **processing_time_ms** (integer) - The time taken for processing in milliseconds. #### Response Example ```json { "image": "", "format": "png", "width": 2048, "height": 2048, "processing_time_ms": 1920 } ``` ``` -------------------------------- ### Chat Completions with LLM Gateway (POST /v1/chat/completions) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Provides chat completions using various LLM models through an OpenAI-compatible gateway. Supports standard chat parameters like model, messages, temperature, and max_tokens. Can return responses in a streaming format. ```json { "model": "claude-sonnet", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is 2 + 2?"} ], "temperature": 0.7, "max_tokens": 1024, "stream": false } ``` -------------------------------- ### Named Entity Recognition Response Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt The response from the NER API contains a list of identified entities, each with its text, label (PER, ORG, LOC, MISC), start and end positions, and a confidence score. It also includes the total count of entities found. ```json { "entities": [ {"text": "Elon Musk", "label": "PER", "start": 0, "end": 9, "score": 0.999}, {"text": "Tesla", "label": "ORG", "start": 18, "end": 23, "score": 0.998}, {"text": "California", "label": "LOC", "start": 27, "end": 37, "score": 0.997} ], "count": 3 } ``` -------------------------------- ### MCP Server Configuration (JSON) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Configuration structure for connecting to various MCP (Multi-Cloud Platform) servers, including Speech, NLP, and Image services. Requires an API key for authentication. ```json { "mcpServers": { "brainiall-speech": { "url": "https://apim-ai-apis.azure-api.net/mcp/pronunciation/mcp", "headers": {"Ocp-Apim-Subscription-Key": "YOUR_KEY"} }, "brainiall-nlp": { "url": "https://apim-ai-apis.azure-api.net/mcp/nlp/mcp", "headers": {"Ocp-Apim-Subscription-Key": "YOUR_KEY"} }, "brainiall-image": { "url": "https://apim-ai-apis.azure-api.net/mcp/image/mcp", "headers": {"Ocp-Apim-Subscription-Key": "YOUR_KEY"} } } } ``` -------------------------------- ### POST /v1/image/upscale/base64 Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Performs Real-ESRGAN 4x image enhancement. Supports output up to 8192x8192 resolution. ```APIDOC ## POST /v1/image/upscale/base64 ### Description Real-ESRGAN 4x enhancement. 2-3s on GPU. Max output: 8192x8192. ### Method POST ### Endpoint /v1/image/upscale/base64 ### Parameters #### Request Body - **image** (string) - Required - Base64 encoded image string. - **scale** (integer) - Optional - The scaling factor. Defaults to 4. ### Request Example ```json { "image": "", "scale": 4 } ``` ### Response #### Success Response (200) - **image** (string) - Base64 encoded upscaled image. - **format** (string) - The format of the output image (e.g., "png"). - **width** (integer) - The width of the output image. - **height** (integer) - The height of the output image. - **scale** (integer) - The scaling factor applied. - **processing_time_ms** (integer) - The time taken for processing in milliseconds. #### Response Example ```json { "image": "", "format": "png", "width": 4096, "height": 3072, "scale": 4, "processing_time_ms": 2840 } ``` ``` -------------------------------- ### Streaming LLM Chat Completion (Python) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Shows how to perform a streaming chat completion request to an LLM. It enables streaming to receive response chunks as they are generated. Requires an API key and a specified model. ```python stream = client.chat.completions.create( model="claude-haiku", messages=[{"role": "user", "content": "Write a haiku about AI."}], stream=True ) for chunk in stream: if chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="") ``` -------------------------------- ### POST /v1/pronunciation/assess/base64 Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Assess the pronunciation of a given audio recording against a reference text. ```APIDOC ## POST /v1/pronunciation/assess/base64 ### Description Assess the pronunciation quality of an audio recording. The audio is provided in base64 encoded format. ### Method POST ### Endpoint https://apim-ai-apis.azure-api.net/v1/pronunciation/assess/base64 ### Parameters #### Headers - **Ocp-Apim-Subscription-Key** (string) - Required - Your subscription key for accessing the API. #### Request Body - **audio** (string) - Required - Base64 encoded audio data. - **text** (string) - Required - The reference text for pronunciation assessment. - **format** (string) - Required - The format of the audio (e.g., 'wav'). ### Request Example ```json { "audio": "BASE64_ENCODED_AUDIO_DATA", "text": "The quick brown fox", "format": "wav" } ``` ### Response #### Success Response (200) - **overallScore** (number) - The overall pronunciation score out of 100. - **words** (array) - An array of word-level assessment results. - **word** (string) - The word assessed. - **score** (number) - The pronunciation score for the word. - **phonemes** (array) - An array of phoneme-level assessment results. - **phoneme** (string) - The phoneme. - **score** (number) - The pronunciation score for the phoneme. #### Response Example ```json { "overallScore": 95.5, "words": [ { "word": "The", "score": 98.0, "phonemes": [ { "phoneme": "DH", "score": 99.0 }, { "phoneme": "AH", "score": 97.0 } ] } ] } ``` ``` -------------------------------- ### POST /v1/image/remove-background/base64 Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Removes the background from an image provided in base64 format. ```APIDOC ## POST /v1/image/remove-background/base64 ### Description Removes the background from a given image. The image is provided in base64 encoded format, and the result is returned as a base64 encoded image. ### Method POST ### Endpoint https://apim-ai-apis.azure-api.net/v1/image/remove-background/base64 ### Parameters #### Headers - **Ocp-Apim-Subscription-Key** (string) - Required - Your subscription key for accessing the API. #### Request Body - **image** (string) - Required - Base64 encoded image data. - **output_format** (string) - Optional - The desired output format for the image (e.g., 'png'). Defaults to the input format if not specified. ### Request Example ```json { "image": "BASE64_ENCODED_IMAGE_DATA", "output_format": "png" } ``` ### Response #### Success Response (200) - **image_base64** (string) - Base64 encoded image data with the background removed. #### Response Example ```json { "image_base64": "BASE64_ENCODED_IMAGE_WITHOUT_BACKGROUND" } ``` ``` -------------------------------- ### Text-to-Speech Synthesis (Python) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Synthesizes speech from a given text. It allows specifying the text, voice, and speech speed, and outputs the audio as a WAV file. Requires an API key. ```python import requests response = requests.post( "https://apim-ai-apis.azure-api.net/v1/tts/synthesize", headers={"Ocp-Apim-Subscription-Key": "YOUR_KEY"}, json={"text": "Welcome to Brainiall speech AI.", "voice": "bf_emma", "speed": 1.0} ) with open("output.wav", "wb") as f: f.write(response.content) ``` -------------------------------- ### Restore Face using GFPGAN (POST /v1/image/restore-face/base64) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Restores the quality of faces in an image using GFPGAN. Accepts a base64 encoded image, with optional parameters for upscaling and background enhancement. Returns the restored image, format, dimensions, and processing time. ```json {"image": "", "upscale": 2, "enhance_background": true} ``` -------------------------------- ### Pronunciation Assessment (Python) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Assesses the pronunciation of a given text against an audio recording. It takes a base64 encoded audio file and returns scores for overall pronunciation, words, and phonemes. Requires an API key and a WAV audio file. ```python import requests, base64 audio_b64 = base64.b64encode(open("recording.wav", "rb").read()).decode() response = requests.post( "https://apim-ai-apis.azure-api.net/v1/pronunciation/assess/base64", headers={"Ocp-Apim-Subscription-Key": "YOUR_KEY"}, json={"audio": audio_b64, "text": "The quick brown fox", "format": "wav"} ) result = response.json() print(f"Overall: {result['overallScore']}/100") for word in result["words"]: print(f" {word['word']}: {word['score']}/100") for ph in word["phonemes"]: print(f" /{ph['phoneme']}/: {ph['score']}/100") ``` -------------------------------- ### LLM Gateway API Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Provides access to various Large Language Models through a unified gateway. ```APIDOC ## LLM Gateway API ### Description Interact with various Large Language Models (LLMs) for tasks like text generation, summarization, and more. ### Method POST ### Endpoint Base https://apim-ai-apis.azure-api.net/v1 ### Parameters #### Headers - **Ocp-Apim-Subscription-Key** (string) - Required - Your subscription key for accessing the API. #### Request Body (for Chat Completions) - **model** (string) - Required - The LLM model to use (e.g., 'claude-sonnet', 'claude-haiku'). - **messages** (array) - Required - An array of message objects representing the conversation history. - **role** (string) - The role of the message sender ('user', 'assistant', 'system'). - **content** (string) - The content of the message. - **stream** (boolean) - Optional - If true, the response will be streamed in chunks. ### Request Example (Non-streaming) ```json { "model": "claude-sonnet", "messages": [ {"role": "user", "content": "Explain quantum computing in 3 sentences."} ] } ``` ### Request Example (Streaming) ```json { "model": "claude-haiku", "messages": [ {"role": "user", "content": "Write a haiku about AI."} ], "stream": true } ``` ### Response (Non-streaming) #### Success Response (200) - **choices** (array) - An array containing the model's response(s). - **message** (object) - **content** (string) - The generated text content. #### Response Example (Non-streaming) ```json { "choices": [ { "message": { "content": "Quantum computing harnesses quantum mechanics principles like superposition and entanglement to perform computations. This allows for solving certain complex problems exponentially faster than classical computers. Potential applications range from drug discovery to cryptography." } } ] } ``` ### Response (Streaming) #### Success Response (200) - **choices** (array) - An array containing streamed chunks of the response. - **delta** (object) - **content** (string) - The content of the current chunk. #### Response Example (Streaming Chunk) ```json { "choices": [ { "delta": { "content": "Quantum" } } ] } ``` ``` -------------------------------- ### POST /v1/nlp/toxicity Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Analyzes text to determine its toxicity level. ```APIDOC ## POST /v1/nlp/toxicity ### Description Analyzes the provided text to detect toxic content. ### Method POST ### Endpoint https://apim-ai-apis.azure-api.net/v1/nlp/toxicity ### Parameters #### Headers - **Ocp-Apim-Subscription-Key** (string) - Required - Your subscription key for accessing the API. #### Request Body - **text** (string) - Required - The text to analyze for toxicity. ### Request Example ```json { "text": "This is a sample message to check for toxicity." } ``` ### Response #### Success Response (200) - **isToxic** (boolean) - Indicates whether the text is considered toxic. - **toxicityScore** (number) - A score representing the level of toxicity (e.g., between 0 and 1). #### Response Example ```json { "isToxic": false, "toxicityScore": 0.15 } ``` ``` -------------------------------- ### POST /v1/whisper/transcribe/base64 Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Transcribes audio files into text using the Whisper model. Supports diarization and language specification. ```APIDOC ## POST /v1/whisper/transcribe/base64 ### Description Transcribes spoken audio into text using the Whisper ASR model. The audio is provided in base64 encoded format. ### Method POST ### Endpoint https://apim-ai-apis.azure-api.net/v1/whisper/transcribe/base64 ### Parameters #### Headers - **Ocp-Apim-Subscription-Key** (string) - Required - Your subscription key for accessing the API. #### Request Body - **audio** (string) - Required - Base64 encoded audio data. - **language** (string) - Optional - The language of the audio (e.g., 'en'). If not provided, the language will be auto-detected. - **diarize** (boolean) - Optional - If true, the transcription will attempt to identify different speakers. ### Request Example ```json { "audio": "BASE64_ENCODED_AUDIO_DATA", "language": "en", "diarize": true } ``` ### Response #### Success Response (200) - **text** (string) - The transcribed text. - **segments** (array) - Optional, if diarization is enabled, provides segments with speaker information. - **speaker** (string) - The identified speaker for the segment. - **start** (number) - Start time of the segment in seconds. - **end** (number) - End time of the segment in seconds. - **text** (string) - The transcribed text for the segment. #### Response Example ```json { "text": "Hello, this is a test transcription.", "segments": [ { "speaker": "SPEAKER_01", "start": 0.5, "end": 3.2, "text": "Hello, this is a test." }, { "speaker": "SPEAKER_02", "start": 3.5, "end": 5.0, "text": "Transcription." } ] } ``` ``` -------------------------------- ### Image Tools MCP Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Provides access to image manipulation tools such as background removal, upscaling, and face restoration. ```APIDOC ## Image Tools MCP — 4 tools ### Description Provides access to image manipulation tools including background removal, upscaling, and face restoration. ### Method POST (for most tools) ### Endpoint `https://apim-ai-apis.azure-api.net/mcp/image/mcp` ### Parameters #### Request Body - **tool** (string) - Required - The specific image tool to use (e.g., `remove_background`, `upscale_image`, `restore_face`). - **image** (string) - Required - The input image, likely base64 encoded. - **scale** (integer) - Optional - Scaling factor for `upscale_image`. - **upscale** (integer) - Optional - Upscaling factor for `restore_face`. - **enhance_background** (boolean) - Optional - Whether to enhance background for `restore_face`. ### Tools - `remove_background`: Removes the background from an image. - `upscale_image`: Upscales an image. - `restore_face`: Restores faces in an image. - `check_image_service`: Checks the status of the image service. ### Resources - `capabilities`: Information about the image tool capabilities. - `pricing`: Pricing details for the image tools. - `supported-formats`: List of supported image formats. ### Prompts - `image_processing_workflow`: Prompt for an image processing workflow. - `batch_image_pipeline`: Prompt for a batch image processing pipeline. ``` -------------------------------- ### LLM Chat Completion Gateway (Node.js) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Interacts with an LLM gateway using Node.js. It sends a chat completion request and logs the response content. Requires an API key and specifies the base URL for the gateway. ```javascript import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://apim-ai-apis.azure-api.net/v1", apiKey: "YOUR_KEY" }); const response = await client.chat.completions.create({ model: "claude-sonnet", messages: [{ role: "user", content: "Hello!" }] }); console.log(response.choices[0].message.content); ``` -------------------------------- ### Non-streaming LLM Chat Completion (Python) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Demonstrates how to make a non-streaming chat completion request to an LLM. It sends a user message and prints the model's response. Requires an API key and a specified model. ```python response = client.chat.completions.create( model="claude-sonnet", messages=[{"role": "user", "content": "Explain quantum computing in 3 sentences."}] ) print(response.choices[0].message.content) ``` -------------------------------- ### Image Background Removal (Python) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Removes the background from an image. It takes a base64 encoded image and returns the image with the background removed in PNG format. Requires an API key. ```python import requests, base64 image_b64 = base64.b64encode(open("photo.jpg", "rb").read()).decode() response = requests.post( "https://apim-ai-apis.azure-api.net/v1/image/remove-background/base64", headers={"Ocp-Apim-Subscription-Key": "YOUR_KEY"}, json={"image": image_b64, "output_format": "png"} ) result = response.json() with open("no_bg.png", "wb") as f: f.write(base64.b64decode(result["image_base64"])) ``` -------------------------------- ### POST /v1/whisper/transcribe/base64 Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Transcribes audio using the Whisper large-v3-turbo model, supporting 99 languages and speaker diarization. Audio is provided in base64 format. ```APIDOC ## POST /v1/whisper/transcribe/base64 ### Description Transcribes audio using the Whisper large-v3-turbo model, supporting 99 languages and speaker diarization. Audio is provided in base64 format. ### Method POST ### Endpoint /v1/whisper/transcribe/base64 ### Parameters #### Query Parameters None #### Request Body - **audio** (string) - Required - Base64 encoded audio data. - **language** (string) - Optional - BCP-47 language code (e.g., `en`, `pt`, `es`). Auto-detected if omitted. - **diarize** (boolean) - Optional - Set to `true` to enable speaker diarization. Defaults to false. - **format** (string) - Optional - The audio format (e.g., `wav`). Defaults to `wav`. ### Request Example ```json { "audio": "", "language": "en", "diarize": false, "format": "wav" } ``` ### Response #### Success Response (200) - **text** (string) - The transcribed text. - **words** (array) - An array of word objects, each with `word`, `start` time, `end` time, `confidence`, and optionally `speaker` if diarization is enabled. - **metadata** (object) - Contains `language`, `languageProbability`, and `processingTimeMs`. #### Response Example ```json { "text": "Hello, this is a test.", "words": [ {"word": "Hello", "start": 0.08, "end": 0.42, "confidence": 0.99}, {"word": "this", "start": 0.50, "end": 0.72, "confidence": 0.98} ], "metadata": {"language": "en", "languageProbability": 0.998, "processingTimeMs": 1840} } ``` **Note**: With diarization enabled, each word object will include a `speaker` field (e.g., `"speaker": "SPEAKER_00"`). ``` -------------------------------- ### Pronunciation Assessment Batch Request (JSON) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt This JSON snippet illustrates a batch request for the Pronunciation Assessment API, allowing up to 50 audio clips to be assessed in a single call. Each item includes audio, text, format, and an optional ID for tracking. ```json { "items": [ {"audio": "", "text": "Hello", "format": "wav", "id": "item-1"}, {"audio": "", "text": "World", "id": "item-2"} ] } ``` -------------------------------- ### POST /v1/image/remove-background/base64 Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Removes the background from an image provided in base64 format using a BiRefNet segmentation model. Supports PNG, JPEG, and WebP input formats. ```APIDOC ## POST /v1/image/remove-background/base64 ### Description Removes the background from an image provided in base64 format using a BiRefNet segmentation model. Supports PNG, JPEG, and WebP input formats. ### Method POST ### Endpoint /v1/image/remove-background/base64 ### Parameters #### Query Parameters None #### Request Body - **image** (string) - Required - Base64 encoded image data (PNG, JPEG, or WebP). - **output_format** (string) - Optional - The desired output format for the image (e.g., `png`). Defaults to `png`. - **return_mask** (boolean) - Optional - If true, returns the segmentation mask instead of the image with background removed. Defaults to false. ### Request Example ```json { "image": "", "output_format": "png", "return_mask": false } ``` ### Response #### Success Response (200) - **image_base64** (string) - Base64 encoded result image or mask. - **format** (string) - The format of the returned image/mask (e.g., `png`). - **original_size** (object) - An object containing the `width` and `height` of the original image. - **processing_ms** (integer) - The time taken for processing in milliseconds. #### Response Example ```json { "image_base64": "", "format": "png", "original_size": {"width": 1024, "height": 768}, "processing_ms": 380 } ``` **Note**: A multipart variant is available via `POST /v1/image/remove-background` using a `file` field for the image upload. ``` -------------------------------- ### Text-to-Speech (TTS) API Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Synthesizes speech from text using the Kokoro-based TTS engine. Supports 12 English voices and outputs 24kHz WAV audio with low latency. ```APIDOC ## POST /v1/tts/synthesize ### Description Synthesizes speech from the provided text using a selected voice and speed. ### Method POST ### Endpoint /v1/tts/synthesize ### Parameters #### Request Body - **text** (string) - Required - The text to synthesize into speech (1-5000 characters). - **voice** (string) - Optional - The ID of the voice to use (default: `af_heart`). - **speed** (number) - Optional - The speech speed multiplier (0.25-4.0, default: 1.0). - **format** (string) - Optional - The desired audio format (default: `wav`). ### Request Example ```json { "text": "Hello, welcome to Brainiall.", "voice": "af_heart", "speed": 1.0, "format": "wav" } ``` ### Response #### Success Response (200) - **Response Body**: Binary `audio/wav` data (24kHz, 16-bit PCM). - **Headers**: - **X-Audio-Duration-Ms** (number) - Duration of the generated audio in milliseconds. - **X-Voice** (string) - The voice used for synthesis. - **X-Text-Length** (number) - The length of the input text. ``` -------------------------------- ### NLP Tools MCP Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Provides access to Natural Language Processing tools for text analysis, PII detection, and language detection. ```APIDOC ## NLP Tools MCP — 6 tools ### Description Provides access to NLP tools for analyzing text, detecting PII, and identifying language. ### Method POST (for most tools) ### Endpoint `https://apim-ai-apis.azure-api.net/mcp/nlp/mcp` ### Parameters #### Request Body - **tool** (string) - Required - The specific NLP tool to use (e.g., `analyze_toxicity`, `analyze_sentiment`, `extract_entities`). - **text** (string) - Required - The input text for analysis. - **language** (string) - Optional - The language of the text. ### Tools - `analyze_toxicity`: Analyzes the toxicity of the text. - `analyze_sentiment`: Analyzes the sentiment of the text. - `extract_entities`: Extracts named entities from the text. - `detect_pii`: Detects Personally Identifiable Information (PII) in the text. - `detect_language`: Detects the language of the text. - `check_nlp_service`: Checks the status of the NLP service. ### Resources - `capabilities`: Information about the NLP tool capabilities. - `supported-languages`: List of supported languages. - `pricing`: Pricing details for the NLP tools. ### Prompts - `content_moderation_pipeline`: Prompt for content moderation. - `text_analysis_report`: Prompt for generating a text analysis report. - `pii_compliance_audit`: Prompt for a PII compliance audit. ``` -------------------------------- ### POST /v1/stt/transcribe/base64 Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Performs Speech-to-Text transcription on audio data provided in base64 format. This endpoint uses a compact English model suitable for short utterances and includes word-level timestamps. ```APIDOC ## POST /v1/stt/transcribe/base64 ### Description Performs Speech-to-Text transcription on audio data provided in base64 format. This endpoint uses a compact English model suitable for short utterances and includes word-level timestamps. ### Method POST ### Endpoint /v1/stt/transcribe/base64 ### Parameters #### Query Parameters None #### Request Body - **audio** (string) - Required - Base64 encoded audio data. - **include_timestamps** (boolean) - Optional - Whether to include word-level timestamps in the response. Defaults to false. ### Request Example ```json { "audio": "", "include_timestamps": true } ``` ### Response #### Success Response (200) - **text** (string) - The transcribed text. - **words** (array) - An array of word objects, each with `word`, `start` time, `end` time, and `confidence` (if `include_timestamps` is true). #### Response Example ```json { "text": "Hello, this is a test.", "words": [ {"word": "Hello", "start": 0.08, "end": 0.42, "confidence": 0.99}, {"word": "this", "start": 0.50, "end": 0.72, "confidence": 0.98} ] } ``` ``` -------------------------------- ### Speech AI MCP Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Provides access to various speech-related tools including pronunciation assessment, transcription, and speech synthesis. ```APIDOC ## Speech AI MCP — 10 tools ### Description Provides access to speech AI tools such as pronunciation assessment, audio transcription, and speech synthesis. ### Method POST (for most tools) ### Endpoint `https://apim-ai-apis.azure-api.net/mcp/pronunciation/mcp` ### Parameters #### Request Body - **tool** (string) - Required - The specific speech AI tool to use (e.g., `assess_pronunciation`, `transcribe_audio`, `synthesize_speech`). - **prompt** (string) - Required - The prompt or input for the selected tool. - **audio** (string) - Required for transcription/synthesis tools - Audio data, potentially base64 encoded. - **language** (string) - Optional - The language for transcription or synthesis. ### Tools - `assess_pronunciation`: Assesses the pronunciation of given audio. - `transcribe_audio`: Transcribes audio to text. - `synthesize_speech`: Synthesizes speech from text. - `list_tts_voices`: Lists available text-to-speech voices. - `transcribe_audio_pro`: Advanced audio transcription supporting 99 languages (Whisper). - `get_phoneme_inventory`: Retrieves phoneme inventory for a language. - `check_pronunciation_service`: Checks the status of the pronunciation service. - `check_stt_service`: Checks the status of the speech-to-text service. - `check_tts_service`: Checks the status of the text-to-speech service. - `check_whisper_service`: Checks the status of the Whisper service. ### Resources - `scoring-guide`: Guide for pronunciation scoring. - `audio-requirements`: Requirements for audio input. - `model-info`: Information about available models. - `response-schema`: Schema for tool responses. - `example-assessment`: Example of a pronunciation assessment. - `stt-usage-guide`: Usage guide for speech-to-text. - `tts-usage-guide`: Usage guide for text-to-speech. - `whisper-usage-guide`: Usage guide for Whisper. ### Prompts - `analyze_pronunciation`: Prompt for analyzing pronunciation. - `create_improvement_plan`: Prompt for creating an improvement plan. - `compare_attempts`: Prompt for comparing pronunciation attempts. ``` -------------------------------- ### Upscale Image using Real-ESRGAN (POST /v1/image/upscale/base64) Source: https://raw.githubusercontent.com/fasuizu-br/speech-ai-examples/main/llms-full.txt Enhances image resolution using Real-ESRGAN 4x. Requires a base64 encoded image as input and returns an enhanced image, format, dimensions, and processing time. Max output resolution is 8192x8192. ```json {"image": "", "scale": 4} ```