### Multimodal LLM Request Example (Text)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This example illustrates how to integrate image data into LLM requests for multimodal capabilities. It shows a user prompt and an assistant response that includes a special tag to trigger image capture, enabling the AI to 'see' user-provided content.

```text
You can use camera to get what you see.
When the user wants to you to see something, insert [vision:camera] into your response message.

Example
user: Look! I bought this today.
assistant: [vision:camera]Let me see.

```

--------------------------------

### Configure OpenAI Text-to-Speech Synthesizer (C#)

Source: https://context7.com/uezo/chatdollkit/llms.txt

This C# script demonstrates how to configure the built-in `OpenAISpeechSynthesizer` in ChatdollKit. It shows setting the API key, voice, model, speed, and enabling the synthesizer. It also includes an example of how to use the `PreprocessText` delegate to modify text before synthesis.

```csharp
using UnityEngine;
using ChatdollKit.SpeechSynthesizer;

public class OpenAISpeechSetup : MonoBehaviour
{
    private void Awake()
    {
        var speechSynthesizer = gameObject.GetComponent<OpenAISpeechSynthesizer>();

        // Configure OpenAI TTS
        speechSynthesizer.ApiKey = "your-openai-api-key";
        speechSynthesizer.Voice = "nova";  // Options: alloy, echo, fable, onyx, nova, shimmer
        speechSynthesizer.Model = "tts-1";  // Options: tts-1, tts-1-hd
        speechSynthesizer.Speed = 1.0f;
        speechSynthesizer.IsEnabled = true;

        // Optional: Preprocess text before synthesis
        speechSynthesizer.PreprocessText = async (text, parameters, token) =>
        {
            // Convert abbreviations, fix pronunciation, etc.
            text = text.Replace("API", "A P I");
            text = text.Replace("SDK", "S D K");
            return text;
        };
    }
}

```

--------------------------------

### Combine Voice Activity Detectors (VADs) in C#

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

Demonstrates how to combine multiple VAD functions for more robust voice detection. This example uses Silero VAD and an energy-based VAD for accurate speech capture in noisy environments. It requires the `Func` delegate and `List` collection.

```csharp
speechListener.DetectVoiceFunctions = new List<Func<float[], float, bool>>()
{
    sileroVad.IsVoiced, speechListener.IsVoiceDetectedByVolume
};

```

--------------------------------

### Setup Silero VAD for Voice Activity Detection in C#

Source: https://context7.com/uezo/chatdollkit/llms.txt

Configures the Silero VAD processor for improved voice activity detection, especially in noisy environments. This setup integrates Silero VAD with the main speech listener, allowing for more accurate turn-end detection by either using Silero's VAD function or combining it with volume-based detection.

```csharp
using UnityEngine;
using ChatdollKit.SpeechListener;
using ChatdollKit.Extension.SileroVAD;
using System;
using System.Collections.Generic;

public class SileroVADSetup : MonoBehaviour
{
    private void Awake()
    {
        var sileroVad = gameObject.GetComponent<SileroVADProcessor>();
        var speechListener = gameObject.GetComponent<SpeechListenerBase>();

        // Initialize Silero VAD
        sileroVad.Initialize();

        // Use single VAD
        speechListener.DetectVoiceFunc = sileroVad.IsVoiced;

        // Or combine multiple VADs for better accuracy in noisy environments
        speechListener.DetectVoiceFunctions = new List<Func<float[], float, bool>>()
        {
            sileroVad.IsVoiced,
            speechListener.IsVoiceDetectedByVolume
        };
    }
}
```

--------------------------------

### Configure Claude LLM Service in Unity

Source: https://context7.com/uezo/chatdollkit/llms.txt

This C# snippet shows the configuration for the ClaudeService in Unity, enabling integration with Anthropic's Claude API. It includes API key setup, model selection, response parameters, and a system prompt designed for Chain of Thought reasoning.

```csharp
using UnityEngine;
using ChatdollKit.LLM.Claude;

public class ClaudeSetup : MonoBehaviour
{
    private void Awake()
    {
        var claudeService = gameObject.GetComponent<ClaudeService>();

        // Configure Claude API
        claudeService.ApiKey = "your-anthropic-api-key";
        claudeService.Model = "claude-3-5-sonnet-20241022";
        claudeService.Temperature = 0.7f;
        claudeService.MaxTokens = 1024;
        claudeService.IsEnabled = true;

        // System prompt with Chain of Thought support
        claudeService.SystemMessageContent = @"
            You are a helpful assistant. Think step by step.
            Use <thinking>...</thinking> tags for your reasoning process.
            The content in thinking tags won't be spoken aloud.
        ";
    }
}
```

--------------------------------

### Handling User-Defined Tags in Chatdollkit (C#)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This C# code snippet shows how to implement a callback function to handle developer-defined tags extracted from AI responses. The example specifically demonstrates how to process a 'light' tag to control room lighting, with logic for 'on' and 'off' commands and a fallback for unprocessable commands.

```csharp
dialogProcessor.LLMServiceExtensions.HandleExtractedTags = (tags, session) =>
{
    if (tags.ContainsKey("light"))
    {
        var lightCommand = tags["light"];
        if (lightCommand.lower() == "on")
        {
            // Turn on the light
            Debug.Log($"Turn on the light");
        }
        else if (lightCommand.lower() == "off")
        {
            // Turn off the light
            Debug.Log($"Turn off the light");
        }
        else
        {
            Debug.LogWarning($"Unprocessable command: {lightCommand}");
        }
    }
};
```

--------------------------------

### Create Complex Animated Voice Requests in C#

Source: https://context7.com/uezo/chatdollkit/llms.txt

This example shows how to create and execute complex animated voice requests for a 3D model using ChatdollKit. It defines sequences of animations, facial expressions, and voice lines, and includes functionality to parse text with embedded tags for more dynamic responses. Dependencies include ChatdollKit.Model and Cysharp.Threading.Tasks.

```csharp
using UnityEngine;
using ChatdollKit.Model;
using Cysharp.Threading.Tasks;
using System.Threading;

public class AnimatedVoiceExample : MonoBehaviour
{
    private ModelController modelController;

    private void Start()
    {
        modelController = gameObject.GetComponent<ModelController>();
    }

    public async UniTask PlayGreeting(CancellationToken token)
    {
        var request = new AnimatedVoiceRequest();

        // First segment: Wave and greet
        request.AddAnimation("waving_arm", 3.0f);
        request.AddFace("Joy", 3.0f);
        request.AddVoice("Hello! Welcome!", preGap: 0.5f, postGap: 0.3f);

        // Second segment: Nod and ask
        request.AddAnimation("nodding_once", 2.0f);
        request.AddFace("Fun", 2.0f);
        request.AddVoice("How can I help you today?", preGap: 0.2f);

        // Optionally start idling when finished
        request.StartIdlingOnEnd = true;

        // Execute the animated voice request
        await modelController.AnimatedSay(request, token);
    }

    public async UniTask PlayFromText(string text, CancellationToken token)
    {
        // Parse text with embedded tags like [face:Joy][anim:waving_arm]Hello!
        var request = modelController.ToAnimatedVoiceRequest(text);
        await modelController.AnimatedSay(request, token);
    }
}
```

--------------------------------

### Implement Custom AI Tool for Function Calling (C#)

Source: https://context7.com/uezo/chatdollkit/llms.txt

This snippet shows how to create a custom tool, `WeatherTool`, by extending `ToolBase` for AI function calling. It defines the function name, description, and parameters, and implements the logic to execute the function, simulating an API call to get weather information. It requires `Newtonsoft.Json` and `Cysharp.Threading.Tasks`.

```csharp
using System.Collections.Generic;
using System.Threading;
using UnityEngine;
using Cysharp.Threading.Tasks;
using Newtonsoft.Json;
using ChatdollKit.LLM;

public class WeatherTool : ToolBase
{
    public string FunctionName = "get_weather";
    public string FunctionDescription = "Get current weather information for a specified location.";

    public override ILLMTool GetToolSpec()
    {
        var func = new LLMTool(FunctionName, FunctionDescription);
        func.AddProperty("location", new Dictionary<string, object>()
        {
            { "type", "string" },
            { "description", "The city name, e.g., Tokyo, New York" }
        });
        func.AddProperty("unit", new Dictionary<string, object>()
        {
            { "type", "string" },
            { "enum", new string[] { "celsius", "fahrenheit" } }
        });
        return func;
    }

    protected override async UniTask<ToolResponse> ExecuteFunction(string argumentsJsonString, CancellationToken token)
    {
        // Parse arguments from JSON
        var arguments = JsonConvert.DeserializeObject<Dictionary<string, string>>(argumentsJsonString);
        var location = arguments["location"];
        var unit = arguments.ContainsKey("unit") ? arguments["unit"] : "celsius";

        Debug.Log($"Getting weather for: {location} in {unit}");

        // Simulate API call (replace with actual weather API)
        await UniTask.Delay(500);

        var response = new Dictionary<string, object>()
        {
            { "location", location },
            { "weather", "Partly cloudy" },
            { "temperature", unit == "celsius" ? 22 : 72 },
            { "unit", unit },
            { "humidity", 65 }
        };

        return new ToolResponse(JsonConvert.SerializeObject(response));
    }
}

```

--------------------------------

### Custom Barge-in Condition Logic in C#

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

Provides an example of overriding the default barge-in trigger logic by setting a custom `BargeInCondition` delegate. This allows for fine-grained control over when a user can interrupt the AI, based on partially recognized text or recording duration. The delegate signature is `Func<string, float, bool>`.

```csharp
// Example: Require at least 3 characters for stream listeners
aiAvatar.SpeechListener.BargeInCondition = (text, recordDuration) =>
{
    if (text != null)
    {
        return text.Length >= 3;
    }
    return recordDuration >= 2.0f;
};

```

--------------------------------

### Configure External Control via Socket Server and JavaScript in C#

Source: https://context7.com/uezo/chatdollkit/llms.txt

Enables external control of the ChatdollKit application using socket communication. This setup includes conditional compilation for different platforms: a Socket Server for desktop/mobile and a JavaScript handler for WebGL applications. It defines how incoming messages are processed for dialog management and model control.

```csharp
using UnityEngine;
using ChatdollKit.Network;
using ChatdollKit.Dialog;
using ChatdollKit.Model;
using ChatdollKit.IO;

public class ExternalControlSetup : MonoBehaviour
{
    private DialogPriorityManager dialogPriorityManager;
    private ModelRequestBroker modelRequestBroker;

    private void Awake()
    {
        dialogPriorityManager = gameObject.GetComponent<DialogPriorityManager>();
        modelRequestBroker = gameObject.GetComponent<ModelRequestBroker>();

        // Configure socket server for desktop/mobile
        #if !UNITY_WEBGL || UNITY_EDITOR
        var socketServer = gameObject.GetComponent<SocketServer>();
        socketServer.Port = 8080;
        socketServer.OnDataReceived = async (message) =>
        {
            HandleExternalMessage(message, "SocketServer");
        };
        #endif

        // Configure JavaScript handler for WebGL
        #if UNITY_WEBGL && !UNITY_EDITOR
        var jsHandler = gameObject.GetComponent<JavaScriptMessageHandler>();
        jsHandler.OnDataReceived = async (message) =>
        {
            HandleExternalMessage(message, "JavaScript");
        };
        #endif
    }

    private void HandleExternalMessage(ExternalInboundMessage message, string source)
    {
        if (message.Endpoint == "dialog")
        {
            if (message.Operation == "start")
            {
                // Start a conversation with the given text
                var priority = source == "JavaScript" ? 0 : message.Priority;
                dialogPriorityManager.SetRequest(message.Text, message.Payloads, priority);
            }
            else if (message.Operation == "clear")
            {
                // Clear pending dialog requests
                dialogPriorityManager.ClearDialogRequestQueue(message.Priority);
            }
        }
        else if (message.Endpoint == "model")
        {
            // Direct model control (animations, expressions, speech)
            modelRequestBroker.SetRequest(message.Text);
        }
    }
}
```

--------------------------------

### Configure ChatGPT LLM Service in Unity

Source: https://context7.com/uezo/chatdollkit/llms.txt

This C# snippet demonstrates how to set up the ChatGPTService for integrating with OpenAI's API within Unity. It covers API key configuration, model selection, response parameters, system prompts for personality, and enabling debug mode.

```csharp
using UnityEngine;
using ChatdollKit.LLM.ChatGPT;

public class LLMSetup : MonoBehaviour
{
    private void Awake()
    {
        var chatGPTService = gameObject.GetComponent<ChatGPTService>();

        // Configure API settings
        chatGPTService.ApiKey = "your-openai-api-key";
        chatGPTService.Model = "gpt-4o-mini";
        chatGPTService.Temperature = 0.7f;
        chatGPTService.MaxTokens = 1000;

        // Set system prompt for character personality
        chatGPTService.SystemMessageContent = @"
            You are a friendly AI assistant named Sakura.
            You have four expressions: 'Joy', 'Angry', 'Sorrow', 'Fun' and 'Surprised'.
            If you want to express emotion, insert it like [face:Joy].

            You can use these animations:
            - waving_arm
            - nodding_once
            - surprise_hands_open_front

            Insert animation tags like [anim:waving_arm].

            Example:
            [face:Joy][anim:waving_arm]Hello! Nice to meet you!
        ";

        // Enable debug mode for development
        chatGPTService.DebugMode = true;
    }
}
```

--------------------------------

### Implement Custom Text-to-Speech Synthesizer (C#)

Source: https://context7.com/uezo/chatdollkit/llms.txt

This C# snippet demonstrates creating a custom text-to-speech synthesizer by extending `SpeechSynthesizerBase`. It includes logic for building a request to a hypothetical TTS API endpoint, sending it using `UnityWebRequest`, and handling the audio clip download. Dependencies include `UnityEngine.Networking` and `Cysharp.Threading.Tasks`.

```csharp
using System;
using System.Collections.Generic;
using System.Threading;
using UnityEngine;
using UnityEngine.Networking;
using Cysharp.Threading.Tasks;
using ChatdollKit.SpeechSynthesizer;

public class CustomTTSSynthesizer : SpeechSynthesizerBase
{
    public string ApiEndpoint = "https://api.example.com/tts";
    public string ApiKey;
    public string VoiceId = "default";

    protected override async UniTask<AudioClip> DownloadAudioClipAsync(
        string text,
        Dictionary<string, object> parameters,
        CancellationToken cancellationToken)
    {
        // Build request
        var requestData = new Dictionary<string, object>
        {
            { "text", text },
            { "voice_id", VoiceId },
            { "output_format", "wav" }
        };

        // Apply style from parameters if available
        if (parameters != null && parameters.ContainsKey("style"))
        {
            requestData["style"] = parameters["style"];
        }

        var jsonData = Newtonsoft.Json.JsonConvert.SerializeObject(requestData);

        using var request = new UnityWebRequest(ApiEndpoint, "POST");
        request.uploadHandler = new UploadHandlerRaw(System.Text.Encoding.UTF8.GetBytes(jsonData));
        request.downloadHandler = new DownloadHandlerAudioClip(ApiEndpoint, AudioType.WAV);
        request.SetRequestHeader("Content-Type", "application/json");
        request.SetRequestHeader("Authorization", $"Bearer {ApiKey}");
        request.timeout = Timeout;

        await request.SendWebRequest();

        if (request.result == UnityWebRequest.Result.Success)
        {
            return DownloadHandlerAudioClip.GetContent(request);
        }
        else
        {
            Debug.LogError($"TTS error: {request.error}");
            return null;
        }
    }
}

```

--------------------------------

### Capture Image for Multimodal LLM Requests (C#)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This C# code snippet demonstrates how to capture an image using a camera and provide it as binary data for multimodal requests to an LLM. It handles potential exceptions during image capture. This is typically used within a Unity environment.

```csharp
gameObject.GetComponent<ChatGPTService>().CaptureImage = async (source) =>
{
    if (simpleCamera != null)
    {
        try
        {
            return await simpleCamera.CaptureImageAsync();
        }
        catch (Exception ex)
        {
            Debug.LogError($"Error at CaptureImageAsync: {ex.Message}\n{ex.StackTrace}");
        }
    }

    return null;
};

```

--------------------------------

### Enable Multimodal Camera Integration for AI Vision in C#

Source: https://context7.com/uezo/chatdollkit/llms.txt

This code sets up camera integration to allow the AI to capture and analyze images. It configures the ChatGPT service with a system prompt that instructs the AI to use the camera when requested and defines a CaptureImage function that utilizes a SimpleCamera component. Dependencies include ChatdollKit.LLM.ChatGPT, ChatdollKit.IO, and Cysharp.Threading.Tasks.

```csharp
using UnityEngine;
using ChatdollKit.LLM.ChatGPT;
using ChatdollKit.IO;
using System;
using Cysharp.Threading.Tasks;

public class VisionSetup : MonoBehaviour
{
    public SimpleCamera simpleCamera;

    private void Start()
    {
        var chatGPTService = gameObject.GetComponent<ChatGPTService>();

        // Configure system prompt for vision
        chatGPTService.SystemMessageContent = @"
            You can use camera to see what's in front of you.
            When the user asks you to look at something, insert [vision:camera] into your response.

            Example:
            user: Look at this!
            assistant: [vision:camera]Let me see what you have there.
        ";

        // Implement image capture function
        chatGPTService.CaptureImage = async (source) =>
        {
            if (simpleCamera != null && source == "camera")
            {
                try
                {
                    return await simpleCamera.CaptureImageAsync();
                }
                catch (Exception ex)
                {
                    Debug.LogError($"Camera capture failed: {ex.Message}");
                }
            }
            return null;
        };
    }
}
```

--------------------------------

### Enable uLipSync for ChatdollKit (C#)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

Integrates uLipSync with ChatdollKit by obtaining the uLipSync component and setting up a callback to handle audio sample data for lip synchronization. This is particularly useful for WebGL builds where OVRLipSync is not supported.

```csharp
var ul = gameObject.GetComponent<uLipSync.uLipSync>();
modelController.HandlePlayingSamples = (samples) =>
{
    ul.OnDataReceived(samples, 1);
};
```

--------------------------------

### Initialize Silero VAD for Voice Activity Detection

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This C# code initializes the SileroVADProcessor and sets it as the voice detection function for a SpeechListenerBase. This integration allows for more accurate turn-end detection, especially in noisy environments, by using a machine learning-based VAD model.

```csharp
var sileroVad = gameObject.GetComponent<SileroVADProcessor>();
sileroVad.Initialize();
var speechListener = gameObject.GetComponent<SpeechListenerBase>();
speechListener.DetectVoiceFunc = sileroVad.IsVoiced;
```

--------------------------------

### Configure OpenAI Whisper Speech Listener in C#

Source: https://context7.com/uezo/chatdollkit/llms.txt

Sets up the OpenAI Whisper speech listener for speech-to-text functionality. It requires API keys and allows configuration of model, language, and recording parameters. This script assumes the `OpenAISpeechListener` component is attached to the GameObject.

```csharp
using UnityEngine;
using ChatdollKit.SpeechListener;

public class SpeechListenerSetup : MonoBehaviour
{
    private void Awake()
    {
        var speechListener = gameObject.GetComponent<OpenAISpeechListener>();

        // Configure OpenAI Whisper
        speechListener.ApiKey = "your-openai-api-key";
        speechListener.Model = "whisper-1";
        speechListener.Language = "en";  // or "ja", "zh", etc.
        speechListener.IsEnabled = true;
        speechListener.PrintResult = true;  // Log recognized text

        // Configure recording settings
        speechListener.SilenceDurationThreshold = 0.5f;
        speechListener.MinRecordingDuration = 0.3f;
        speechListener.MaxRecordingDuration = 30f;
    }
}
```

--------------------------------

### Implement Custom Speech Synthesizer in Unity

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This C# code outlines the structure for creating a custom speech synthesizer by inheriting from SpeechSynthesizerBase. It requires implementing the DownloadAudioClipAsync method to handle audio clip generation for Unity.

```csharp
UniTask<AudioClip> DownloadAudioClipAsync(string text, Dictionary<string, object> parameters, CancellationToken cancellationToken)
{
    // Implementation to download and return an AudioClip
}
```

--------------------------------

### Registering Animations in Chatdollkit (C#)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This C# code demonstrates how to register animations with the ModelController in Chatdollkit. It shows how to map animation names to specific parameters, including base and additive animations, and how to register multiple animations using a predefined registry.

```csharp
modelController.RegisterAnimation("angry_hands_on_waist", new Model.Animation("BaseParam", 0, 3.0f));
modelController.RegisterAnimation("brave_hand_on_chest", new Model.Animation("BaseParam", 1, 3.0f));
modelController.RegisterAnimation("calm_hands_on_back", new Model.Animation("BaseParam", 2, 3.0f));
modelController.RegisterAnimation("concern_right_hand_front", new Model.Animation("BaseParam", 3, 3.0f));
modelController.RegisterAnimation("energetic_right_fist_up", new Model.Animation("BaseParam", 4, 3.0f));
modelController.RegisterAnimation("energetic_right_hand_piece", new Model.Animation("BaseParam", 5, 3.0f));
modelController.RegisterAnimation("pitiable_right_hand_on_back_head", new Model.Animation("BaseParam", 7, 3.0f));
modelController.RegisterAnimation("surprise_hands_open_front", new Model.Animation("BaseParam", 8, 3.0f));
modelController.RegisterAnimation("walking", new Model.Animation("BaseParam", 9, 3.0f));
modelController.RegisterAnimation("waving_arm", new Model.Animation("BaseParam", 10, 3.0f));
// Additive
modelController.RegisterAnimation("look_away", new Model.Animation("BaseParam", 6, 3.0f, "AGIA_Layer_look_away_01", "Additive Layer"));
modelController.RegisterAnimation("nodding_once", new Model.Animation("BaseParam", 6, 3.0f, "AGIA_Layer_nodding_once_01", "Additive Layer"));
modelController.RegisterAnimation("swinging_body", new Model.Animation("BaseParam", 6, 3.0f, "AGIA_Layer_swinging_body_01", "Additive Layer"));
```

```csharp
modelController.RegisterAnimations(AGIARegistry.GetAnimations(animationCollectionKey));
```

--------------------------------

### Configure Microphone Manager with Echo Cancellation in C#

Source: https://context7.com/uezo/chatdollkit/llms.txt

This snippet shows how to configure the MicrophoneManager for controlling microphone input, including setting sample rate, noise gate threshold, and auto-start behavior. It also demonstrates platform-specific implementations for enabling echo cancellation using native plugins for iOS, Android, and macOS. Dependencies include ChatdollKit.IO.

```csharp
using UnityEngine;
using ChatdollKit.IO;

public class MicrophoneSetup : MonoBehaviour
{
    private void Awake()
    {
        var microphoneManager = gameObject.GetComponent<MicrophoneManager>();

        // Configure microphone settings
        microphoneManager.SampleRate = 44100;  // Required for WebGL
        microphoneManager.NoiseGateThresholdDB = -50f;
        microphoneManager.AutoStart = true;
        microphoneManager.IsDebug = true;

        // Enable echo cancelling with native microphone (platform-specific)
        #if UNITY_IOS && !UNITY_EDITOR
        microphoneManager.MicrophoneProvider = new IOSMicrophoneProvider();
        #elif UNITY_ANDROID && !UNITY_EDITOR
        microphoneManager.MicrophoneProvider = new AndroidMicrophoneProvider();
        #elif UNITY_STANDALONE_OSX && !UNITY_EDITOR
        microphoneManager.MicrophoneProvider = new MacMicrophoneProvider();
        #endif
    }
}
```

--------------------------------

### C# Text Preprocessing for Speech Synthesis

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

Defines the interface for preprocessing text before it is synthesized into speech. This allows for custom text manipulation to improve synthesis quality or handle specific requirements. The method takes the input string, a dictionary of parameters, and a cancellation token, returning the processed string.

```csharp
Func<string, Dictionary<string, object>, CancellationToken, UniTask<string>> PreprocessText;
```

--------------------------------

### Enable Echo Cancelling with Native Microphone Plugins in C#

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

Explains how to enable echo cancelling in Unity by using platform-specific native microphone plugins. This is necessary because Unity's built-in Microphone API lacks this feature. It requires importing the `ChatdollKit_NativeMicrophone` package and setting the appropriate `MicrophoneProvider` for iOS, Android, or macOS.

```csharp
private void Awake()
{
    var microphoneManager = gameObject.GetComponent<MicrophoneManager>();
    
    // First, import the ChatdollKit_NativeMicrophone package
    // Then, set the appropriate provider for your platform:
    
    // iOS
    microphoneManager.MicrophoneProvider = new IOSMicrophoneProvider();
    // Android
    microphoneManager.MicrophoneProvider = new AndroidMicrophoneProvider();
    // macOS
    microphoneManager.MicrophoneProvider = new MacMicrophoneProvider();
}

```

--------------------------------

### Change Base Class: LLMFunctionSkillBase to ToolBase (C#)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This code snippet demonstrates how to change the base class from LLMFunctionSkillBase to ToolBase in a C# class. This is the first step in migrating from FunctionSkill to Tool.

```csharp
public class MyFunctionSkill : LLMFunctionSkillBase

public class MyFunctionSkill : ToolBase
```

--------------------------------

### Configure Remote Control Message Handler (C#)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

Configures the message handler for remote control of ChatdollKit, differentiating between JavaScript and SocketServer communication based on the Unity environment. It attaches the appropriate handler to the GameObject and sets up a callback for received data.

```csharp
#pragma warning disable CS1998
#if UNITY_WEBGL && !UNITY_EDITOR
gameObject.GetComponent<JavaScriptMessageHandler>().OnDataReceived = async (message) =>
{
    HandleExternalMessage(message, "JavaScript");
};
#else
gameObject.GetComponent<SocketServer>().OnDataReceived = async (message) =>
{
    HandleExternalMessage(message, "SocketServer");
};
#endif
#pragma warning restore CS1998
```

--------------------------------

### Configure Long-Term Memory with ChatMemory in C#

Source: https://context7.com/uezo/chatdollkit/llms.txt

This snippet demonstrates how to integrate the ChatMemory service to store and retrieve conversation history. It configures the service URL and user ID, and sets up an event handler to add conversation turns to ChatMemory when the LLM streaming ends. Dependencies include ChatdollKit.Dialog, ChatdollKit.Extension.ChatMemory, and Cysharp.Threading.Tasks.

```csharp
using UnityEngine;
using ChatdollKit.Dialog;
using ChatdollKit.Extension.ChatMemory;
using Cysharp.Threading.Tasks;

public class LongTermMemorySetup : MonoBehaviour
{
    private void Start()
    {
        var dialogProcessor = gameObject.GetComponent<DialogProcessor>();
        var chatMemory = gameObject.GetComponent<ChatMemoryIntegrator>();

        // Configure ChatMemory service
        chatMemory.ServiceUrl = "http://localhost:8000";
        chatMemory.UserId = "user_123";

        // Store conversation history when streaming ends
        dialogProcessor.LLMServiceExtensions.OnStreamingEnd += async (text, payloads, llmSession, token) =>
        {
            // Add history to ChatMemory
            chatMemory.AddHistory(
                llmSession.ContextId,
                text,  // User's message
                llmSession.CurrentStreamBuffer,  // AI's response
                token
            ).Forget();
        };
    }
}
```

--------------------------------

### Update ExecuteFunction Signature: FunctionSkill to Tool (C#)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This code snippet illustrates the required changes to the ExecuteFunction method signature when migrating from FunctionSkill to Tool. It shows the modification of parameters and return types.

```csharp
public UniTask<FunctionResponse> ExecuteFunction(string argumentsJsonString, Request request, State state, User user, CancellationToken token)

public UniTask<ToolResponse> ExecuteFunction(string argumentsJsonString, CancellationToken token)
```

--------------------------------

### Configure AIAvatar Component in Unity

Source: https://context7.com/uezo/chatdollkit/llms.txt

This C# snippet shows how to configure the AIAvatar component in Unity. It sets up wake word, cancel word, error voice, timeouts, and microphone muting behavior for conversational AI characters.

```csharp
using UnityEngine;
using ChatdollKit;
using Cysharp.Threading.Tasks;

public class Main : MonoBehaviour
{
    private AIAvatar aiAvatar;

    private void Awake()
    {
        aiAvatar = gameObject.GetComponent<AIAvatar>();

        // Configure wake word and conversation settings
        aiAvatar.WakeWord = "hello";
        aiAvatar.CancelWord = "goodbye";
        aiAvatar.ErrorVoice = "Sorry, something went wrong.";
        aiAvatar.ConversationTimeout = 30f;
        aiAvatar.IdleTimeout = 120f;

        // Configure microphone behavior during AI speech
        aiAvatar.MicrophoneMuteBy = AIAvatar.MicrophoneMuteMethod.Threshold;
        aiAvatar.VoiceRecognitionThresholdDB = -50f;
    }

    private async void Start()
    {
        // Start a conversation programmatically
        await aiAvatar.StartConversationAsync("Tell me about the weather today");
    }
}
```

--------------------------------

### Configure AIAvatarKitStreamSpeechListener for Real-time Speech Display

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This C# code configures the AIAvatarKitStreamSpeechListener to display partial speech recognition results in real-time. It disables text animation, adjusts display timings, and manually manages the user message window for a smoother streaming experience.

```csharp
var aiavatarKitStreamSpeechListener = gameObject.GetComponent<AIAvatarKitStreamSpeechListener>();
if (aiavatarKitStreamSpeechListener != null)
{
    var userMessageWindow = (SimpleMessageWindow)aiAvatar.UserMessageWindow;

    // Disable text animation since partial results are streamed in real-time
    userMessageWindow.IsTextAnimated = false;
    // Shorter PostGap is fine for streaming display; prioritize responsiveness
    userMessageWindow.PostGap = 0.2f;

    // Manually hide user message window after PostGap on the first turn,
    // because the user message window is not managed by the normal dialog flow
    var originalOnRecognized = aiavatarKitStreamSpeechListener.OnRecognized;
    aiavatarKitStreamSpeechListener.OnRecognized = async (text) => {
        if (originalOnRecognized != null)
        {
            await originalOnRecognized(text);
        }
        if (aiAvatar.Mode != AIAvatar.AvatarMode.Conversation)
        {
            await UniTask.Delay((int)(userMessageWindow.PostGap * 1000));
            aiAvatar.UserMessageWindow?.Hide();
        }
    };

    // Display partial recognition results
    aiavatarKitStreamSpeechListener.OnPartialRecognized = (partialText) => {
        if (!string.IsNullOrEmpty(partialText))
        {
            aiAvatar.UserMessageWindow.Show(partialText);
        }
    };
}
```

--------------------------------

### Add Idle Animations to ModelController

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

Demonstrates how to add multiple idle animations to a ModelController component, allowing for random switching between animations. This involves creating Animation objects with specified parameters, weights, and optional modes, and then adding them to the ModelController.

```csharp
modelController.AddIdleAnimation(new Animation("BaseParam", 2, 5f));
modelController.AddIdleAnimation(new Animation("BaseParam", 6, 5f), weight: 2);
modelController.AddIdleAnimation(new Animation("BaseParam", 99, 5f), mode: "sleep");
```

--------------------------------

### Handle External Messages for ChatdollKit (C#)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

Handles incoming messages from external sources (JavaScript or SocketServer) for ChatdollKit. It parses the message endpoint and operation to trigger dialog requests or model requests, managing priority for dialogs.

```csharp
private void HandleExternalMessage(ExternalInboundMessage message, string source)
{
    // Assign actions based on the request's Endpoint and Operation
    if (message.Endpoint == "dialog")
    {
        if (message.Operation == "start")
        {
            if (source == "JavaScript")
            {
                dialogPriorityManager.SetRequest(message.Text, message.Payloads, 0);
            }
            else
            {
                dialogPriorityManager.SetRequest(message.Text, message.Payloads, message.Priority);
            }
        }
        else if (message.Operation == "clear")
        {
            dialogPriorityManager.ClearDialogRequestQueue(message.Priority);
        }
    }
    else if (message.Endpoint == "model")
    {
        modelRequestBroker.SetRequest(message.Text);
    }
```

--------------------------------

### Update ExecuteFunction Return Type: FunctionResponse to ToolResponse (C#)

Source: https://github.com/uezo/chatdollkit/blob/master/README.md

This code snippet highlights the change in the return type of the ExecuteFunction method from FunctionResponse to ToolResponse. This is a necessary step when migrating from FunctionSkill to Tool.

```csharp
// Before
public UniTask<FunctionResponse> ExecuteFunction(...)

// After
public UniTask<ToolResponse> ExecuteFunction(...)
```