### Install and Uninstall Tracking Modules with ModuleInstaller (C#) Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt The ModuleInstaller service provides methods to download, install, and uninstall tracking modules. It supports installation from a remote registry using module metadata or from a local zip file. It also allows marking modules for deletion on the next restart if files are locked. ```csharp using VRCFaceTracking.Core.Services; using VRCFaceTracking.Core.Models; using Microsoft.Extensions.Logging; public class ModuleManagement { private readonly ModuleInstaller _installer; private readonly ILogger _logger; public ModuleManagement(ModuleInstaller installer, ILogger logger) { _installer = installer; _logger = logger; } public async Task InstallModuleFromRegistry(TrackingModuleMetadata metadata) { // Install a module from the online registry var dllPath = await _installer.InstallRemoteModule(metadata); if (dllPath != null) { _logger.LogInformation($"Module installed at: {dllPath}"); } else { _logger.LogError("Module installation failed"); } } public async Task InstallLocalModule(string zipFilePath) { // Install a module from a local .zip file // The zip must contain a module.json file var dllPath = await _installer.InstallLocalModule(zipFilePath); if (dllPath != null) { _logger.LogInformation($"Local module installed at: {dllPath}"); } } public void UninstallModule(TrackingModuleMetadata metadata) { // Uninstall a module by deleting its directory _installer.UninstallModule(metadata); _logger.LogInformation($"Module {metadata.ModuleName} uninstalled"); } public void MarkForDeletion(InstallableTrackingModule module) { // Mark module for deletion on next restart // (used when module files are locked) _installer.MarkModuleForDeletion(module); } } // module.json structure: // { // "ModuleId": "unique-guid", // "ModuleName": "My Tracker Module", // "DllFileName": "MyTracker.dll", // "DownloadUrl": "https://example.com/module.zip", // "Version": "1.0.0", // "Author": "Developer Name" // } ``` -------------------------------- ### Manage Mutation Pipeline with UnifiedTrackingMutator Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Provides examples for controlling the mutation pipeline, including enabling the service, initializing mutations, and iterating through active components for configuration. ```csharp using VRCFaceTracking.Core.Params.Data.Mutation; public class MutationConfiguration { private readonly UnifiedTrackingMutator _mutator; public MutationConfiguration(UnifiedTrackingMutator mutator) => _mutator = mutator; public void ConfigureMutations() { _mutator.Enabled = true; _mutator.Load(); _mutator.Initialize(); foreach (var mutation in _mutator._mutations) { mutation.IsActive = true; } } } ``` -------------------------------- ### Implement Custom TrackingMutation for Data Processing Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Shows how to create a custom mutation class by inheriting from TrackingMutation. This example implements exponential smoothing to reduce jitter in facial expression data. ```csharp using VRCFaceTracking.Core.Params.Data; using VRCFaceTracking.Core.Params.Data.Mutation; public class SmoothingMutation : TrackingMutation { public override string Name => "Expression Smoothing"; public override MutationPriority Step => MutationPriority.Postprocessor; [MutationSlider(0.0f, 1.0f, "Smoothing Factor")] public float SmoothingFactor { get; set; } = 0.3f; private float[] _previousShapes; public override void Initialize(UnifiedTrackingData data) => _previousShapes = new float[data.Shapes.Length]; public override void MutateData(ref UnifiedTrackingData data) { for (int i = 0; i < data.Shapes.Length; i++) { float current = data.Shapes[i].Weight; float smoothed = _previousShapes[i] + SmoothingFactor * (current - _previousShapes[i]); data.Shapes[i].Weight = smoothed; _previousShapes[i] = smoothed; } } } ``` -------------------------------- ### Configure OSC Connection Settings in C# Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Shows how to configure OSC connection parameters such as inbound and outbound ports, and the destination address using the IOscTarget interface in C#. This is essential for sending tracking data to VRChat. ```csharp using VRCFaceTracking.Core.Models; using VRCFaceTracking.Core.Contracts.Services; public class MyService { private readonly IOscTarget _oscTarget; public MyService(IOscTarget oscTarget) { _oscTarget = oscTarget; // Configure OSC ports (VRChat defaults) _oscTarget.InPort = 9001; // Port to receive OSC from VRChat _oscTarget.OutPort = 9000; // Port to send OSC to VRChat _oscTarget.DestinationAddress = "127.0.0.1"; // Localhost // Check connection status if (_oscTarget.IsConnected) { Console.WriteLine("OSC connection established"); } // Listen for property changes _oscTarget.PropertyChanged += (sender, args) => { if (args.PropertyName == nameof(IOscTarget.IsConnected)) { Console.WriteLine($"Connection status: {_oscTarget.IsConnected}"); } }; } } ``` -------------------------------- ### Implement Custom Tracking Module in C# Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt This snippet demonstrates the implementation of a custom module by inheriting from ExtTrackingModule. It covers the lifecycle methods Initialize, Update, and Teardown, and shows how to map raw hardware data to the UnifiedTracking data structure. ```csharp using Microsoft.Extensions.Logging; using VRCFaceTracking; using VRCFaceTracking.Core.Library; using VRCFaceTracking.Core.Params.Data; using VRCFaceTracking.Core.Params.Expressions; public class MyCustomTrackingModule : ExtTrackingModule { public override (bool SupportsEye, bool SupportsExpression) Supported => (true, true); public override (bool eyeSuccess, bool expressionSuccess) Initialize(bool eyeAvailable, bool expressionAvailable) { Logger.LogInformation("Initializing custom tracking module..."); ModuleInformation = new ModuleMetadata { Name = "My Custom Tracker", Active = true, UsingEye = eyeAvailable, UsingExpression = expressionAvailable }; bool hardwareConnected = ConnectToHardware(); return (hardwareConnected && eyeAvailable, hardwareConnected && expressionAvailable); } public override void Update() { var rawData = GetRawTrackingData(); UnifiedTracking.Data.Eye.Left.Gaze = new Vector2(rawData.LeftEyeX, rawData.LeftEyeY); UnifiedTracking.Data.Eye.Right.Gaze = new Vector2(rawData.RightEyeX, rawData.RightEyeY); UnifiedTracking.Data.Eye.Left.Openness = rawData.LeftEyeOpenness; UnifiedTracking.Data.Eye.Right.Openness = rawData.RightEyeOpenness; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.JawOpen].Weight = rawData.JawOpen; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthCornerPullLeft].Weight = rawData.SmileLeft; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthCornerPullRight].Weight = rawData.SmileRight; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.EyeSquintLeft].Weight = rawData.EyeSquintL; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.EyeSquintRight].Weight = rawData.EyeSquintR; } public override void Teardown() { Logger.LogInformation("Shutting down custom tracking module..."); DisconnectFromHardware(); } private bool ConnectToHardware() => true; private void DisconnectFromHardware() { } private dynamic GetRawTrackingData() => new { LeftEyeX = 0f, LeftEyeY = 0f, RightEyeX = 0f, RightEyeY = 0f, LeftEyeOpenness = 1f, RightEyeOpenness = 1f, JawOpen = 0f, SmileLeft = 0f, SmileRight = 0f, EyeSquintL = 0f, EyeSquintR = 0f }; } ``` -------------------------------- ### OscTarget Configuration Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Configuring the OSC network target for communication with VRChat. ```APIDOC ## OscTarget Configuration ### Description Configures the network settings for the OSC client to communicate with VRChat, including port definitions and destination addresses. ### Method POST/PUT (Configuration) ### Endpoint IOscTarget ### Parameters #### Request Body - **InPort** (int) - Required - The port used to receive OSC data from VRChat. - **OutPort** (int) - Required - The port used to send OSC data to VRChat. - **DestinationAddress** (string) - Required - The IP address of the VRChat instance (default: 127.0.0.1). ### Request Example { "InPort": 9001, "OutPort": 9000, "DestinationAddress": "127.0.0.1" } ### Response #### Success Response (200) - **IsConnected** (bool) - Returns true if the connection to the specified target is established. ``` -------------------------------- ### Manage Tracking Module Lifecycle with UnifiedLibManager (C#) Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt The UnifiedLibManager service manages the complete lifecycle of tracking modules, including initialization, status tracking, and teardown. Modules are executed in sandboxed subprocesses to ensure system stability. It provides methods to initialize all modules and shut them down gracefully. ```csharp using VRCFaceTracking.Core.Library; using VRCFaceTracking.Core.Contracts.Services; using Microsoft.Extensions.Logging; public class ModuleLifecycleManager { private readonly ILibManager _libManager; public ModuleLifecycleManager(ILibManager libManager) { _libManager = libManager; } public void InitializeModules() { // Start module initialization (runs on background thread) _libManager.Initialize(); // Access loaded modules metadata foreach (var module in _libManager.LoadedModulesMetadata) { Console.WriteLine($"Module: {module.Name}"); Console.WriteLine($" Active: {module.Active}"); Console.WriteLine($" Using Eye: {module.UsingEye}"); Console.WriteLine($" Using Expression: {module.UsingExpression}"); } // Check tracking status Console.WriteLine($"Eye Tracking Status: {UnifiedLibManager.EyeStatus}"); Console.WriteLine($"Expression Status: {UnifiedLibManager.ExpressionStatus}"); } public void ShutdownModules() { // Gracefully shut down all modules _libManager.TeardownAllAndResetAsync(); } } // Module states: // - Uninitialized: Module not yet loaded or failed to load // - Idle: Module loaded but not actively providing data // - Active: Module actively providing tracking data ``` -------------------------------- ### Handle OSC Messages with OscMessage Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Demonstrates how to instantiate, encode, and parse OSC messages using the OscMessage class. It covers creating messages with different data types and converting them to byte buffers for network transmission. ```csharp using VRCFaceTracking.Core.OSC; var jawOpenMessage = new OscMessage("/avatar/parameters/JawOpen", typeof(float)); jawOpenMessage.Value = 0.75f; byte[] buffer = new byte[4096]; CancellationToken ct = CancellationToken.None; int messageLength = await jawOpenMessage.Encode(buffer, ct); byte[] receivedBytes = GetReceivedOscBytes(); int messageIndex = 0; OscMessage parsedMessage = OscMessage.TryParseOsc(receivedBytes, receivedBytes.Length, ref messageIndex); if (parsedMessage != null) { Console.WriteLine($"Received: {parsedMessage.Address} = {parsedMessage.Value}"); } byte[] GetReceivedOscBytes() => new byte[0]; ``` -------------------------------- ### ExtTrackingModule - Base Module Class Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt The `ExtTrackingModule` abstract class serves as the foundation for creating custom tracking modules for VRCFaceTracking. It defines the lifecycle methods and expected capabilities for any integrated tracking hardware. ```APIDOC ## ExtTrackingModule - Base Module Class ### Description This abstract class is the foundation for creating custom tracking modules that integrate with VRCFaceTracking. Modules must implement initialization, update, and teardown methods to manage their lifecycle and provide tracking data. ### Method Abstract Class ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response N/A ## Example Custom Module Implementation ### Description This C# code demonstrates how to create a custom tracking module by inheriting from `ExtTrackingModule` and implementing its core methods. ### Method C# ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example ```csharp using Microsoft.Extensions.Logging; using VRCFaceTracking; using VRCFaceTracking.Core.Library; using VRCFaceTracking.Core.Params.Data; using VRCFaceTracking.Core.Params.Expressions; public class MyCustomTrackingModule : ExtTrackingModule { // Declare what tracking capabilities this module provides public override (bool SupportsEye, bool SupportsExpression) Supported => (true, true); public override (bool eyeSuccess, bool expressionSuccess) Initialize(bool eyeAvailable, bool expressionAvailable) { // Initialize your tracking hardware here Logger.LogInformation("Initializing custom tracking module..."); // Set module metadata ModuleInformation = new ModuleMetadata { Name = "My Custom Tracker", Active = true, UsingEye = eyeAvailable, UsingExpression = expressionAvailable }; // Return success status for each tracking type bool hardwareConnected = ConnectToHardware(); return (hardwareConnected && eyeAvailable, hardwareConnected && expressionAvailable); } public override void Update() { // Called at ~100Hz - update tracking data here var rawData = GetRawTrackingData(); // Update eye tracking data UnifiedTracking.Data.Eye.Left.Gaze = new Vector2(rawData.LeftEyeX, rawData.LeftEyeY); UnifiedTracking.Data.Eye.Right.Gaze = new Vector2(rawData.RightEyeX, rawData.RightEyeY); UnifiedTracking.Data.Eye.Left.Openness = rawData.LeftEyeOpenness; UnifiedTracking.Data.Eye.Right.Openness = rawData.RightEyeOpenness; // Update expression shapes UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.JawOpen].Weight = rawData.JawOpen; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthCornerPullLeft].Weight = rawData.SmileLeft; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthCornerPullRight].Weight = rawData.SmileRight; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.EyeSquintLeft].Weight = rawData.EyeSquintL; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.EyeSquintRight].Weight = rawData.EyeSquintR; } public override void Teardown() { // Clean up resources when module is unloaded Logger.LogInformation("Shutting down custom tracking module..."); DisconnectFromHardware(); } private bool ConnectToHardware() => true; // Placeholder private void DisconnectFromHardware() { } // Placeholder private dynamic GetRawTrackingData() => new { LeftEyeX = 0f, LeftEyeY = 0f, RightEyeX = 0f, RightEyeY = 0f, LeftEyeOpenness = 1f, RightEyeOpenness = 1f, JawOpen = 0f, SmileLeft = 0f, SmileRight = 0f, EyeSquintL = 0f, EyeSquintR = 0f }; } ``` ### Response N/A ``` -------------------------------- ### Receive OSC Messages with OscRecvService Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Covers setting up a background listener to process incoming OSC traffic from VRChat. This is essential for reacting to avatar changes and parameter feedback updates. ```csharp using VRCFaceTracking.Core.Services; using VRCFaceTracking.Core.OSC; public class VRChatEventHandler { private readonly OscRecvService _recvService; public VRChatEventHandler(OscRecvService recvService) { _recvService = recvService; _recvService.OnMessageReceived = HandleIncomingMessage; } private void HandleIncomingMessage(OscMessage message) { if (message.Address == "/avatar/change") { Console.WriteLine($"Avatar changed to: {message.Value}"); } } } ``` -------------------------------- ### Send OSC Messages with OscSendService Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Explains how to use the OscSendService to transmit tracking data to VRChat. It shows both single message transmission and batch processing using bundles for improved efficiency. ```csharp using VRCFaceTracking.Core.Services; using VRCFaceTracking.Core.OSC; public class TrackingOutputService { private readonly OscSendService _sendService; public TrackingOutputService(OscSendService sendService) => _sendService = sendService; public async Task SendTrackingData(CancellationToken ct) { var singleMessage = new OscMessage("/avatar/parameters/JawOpen", typeof(float)); singleMessage.Value = 0.5f; await _sendService.Send(singleMessage, ct); var messages = new OscMessage[] { CreateMessage("/avatar/parameters/EyesX", 0.3f) }; await _sendService.Send(messages, ct); } private OscMessage CreateMessage(string address, float value) => new OscMessage(address, typeof(float)) { Value = value }; } ``` -------------------------------- ### Configure ParameterSenderService OSC Transmission Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Demonstrates how to interact with the ParameterSenderService to toggle global parameter relevance, enqueue custom OSC messages, and clear the pending message queue. ```csharp using VRCFaceTracking.Core.Services; using VRCFaceTracking.Core.OSC; public class ParameterConfiguration { public void ConfigureParameterSender() { ParameterSenderService.AllParametersRelevantStatic = true; var customMessage = new OscMessage("/avatar/parameters/CustomParam", typeof(float)); customMessage.Value = 0.5f; ParameterSenderService.Enqueue(customMessage); ParameterSenderService.Clear(); } } ``` -------------------------------- ### VRChat Avatar OSC Parameter Reference Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt A reference list of standard OSC parameter names, descriptions, and expected value ranges for VRChat avatar facial tracking. These parameters are used by VRCFaceTracking to map hardware input to avatar blend shapes. ```csharp // Eye Tracking Parameters (Float, 0.0 to 1.0 unless noted) // EyesX: Combined gaze X (-1.0 to 1.0) // EyesY: Combined gaze Y (-1.0 to 1.0) // LeftEyeLid: Left eyelid openness (0.0 to 1.0) // RightEyeLid: Right eyelid openness (0.0 to 1.0) // EyesDilation: Pupil dilation (0.0 to 1.0) // Lip/Expression Parameters (Float, 0.0 to 1.0) // JawOpen: Jaw open amount // MouthSmileRight: Right smile amount // CheekPuffRight: Right cheek puff // Unified Expressions v2 Parameters (prefixed with v2/) // v2/EyeWideLeft: Left eye widen // v2/JawOpen: Jaw open // v2/SmileFrown: Combined smile/frown expression // Status Parameters (Bool) // EyeTrackingActive: True when eye tracking is active // LipTrackingActive: True when lip tracking is active ``` -------------------------------- ### Set UnifiedExpressions Weights in C# Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Demonstrates how to set the weights for various facial expressions using the UnifiedExpressions enum in C#. This code directly manipulates the tracking data shapes based on biometric muscle movements. ```csharp using VRCFaceTracking.Core.Params.Expressions; using VRCFaceTracking.Core.Params.Data; // Eye expressions UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.EyeSquintRight].Weight = 0.5f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.EyeSquintLeft].Weight = 0.5f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.EyeWideRight].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.EyeWideLeft].Weight = 0.0f; // Eyebrow expressions UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.BrowPinchRight].Weight = 0.3f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.BrowPinchLeft].Weight = 0.3f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.BrowLowererRight].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.BrowLowererLeft].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.BrowInnerUpRight].Weight = 0.7f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.BrowInnerUpLeft].Weight = 0.7f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.BrowOuterUpRight].Weight = 0.4f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.BrowOuterUpLeft].Weight = 0.4f; // Jaw expressions UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.JawOpen].Weight = 0.6f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.JawRight].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.JawLeft].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.JawForward].Weight = 0.1f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthClosed].Weight = 0.0f; // Lip expressions for smile UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthCornerPullRight].Weight = 0.8f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthCornerPullLeft].Weight = 0.8f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthCornerSlantRight].Weight = 0.5f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthCornerSlantLeft].Weight = 0.5f; // Lip expressions for frown/sad UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthFrownRight].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthFrownLeft].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthStretchRight].Weight = 0.2f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.MouthStretchLeft].Weight = 0.2f; // Cheek expressions UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.CheekPuffRight].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.CheekPuffLeft].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.CheekSuckRight].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.CheekSuckLeft].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.CheekSquintRight].Weight = 0.3f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.CheekSquintLeft].Weight = 0.3f; // Tongue expressions UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.TongueOut].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.TongueUp].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.TongueDown].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.TongueRight].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.TongueLeft].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.TongueRoll].Weight = 0.0f; // Nose expressions UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.NoseSneerRight].Weight = 0.0f; UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.NoseSneerLeft].Weight = 0.0f; ``` -------------------------------- ### Combined Lip Parameters Source: https://github.com/benaclejames/vrcfacetracking/wiki/Parameters A comprehensive list of computed parameters that combine various lip tracking data points. ```APIDOC ## Combined Lip Parameters Additionally, VRCFaceTracking provides computed parameters that combine some of the parameters, to save space: ### General Combined Lip Parameters #### Query Parameters - **JawX** (Float) - Range: -1.0 to 1.0 - Jaw translation fully left to fully right - **MouthUpper** (Float) - Range: -1.0 to 1.0 - MouthUpperLeft to MouthUpperRight, with 0 being neutral - **MouthLower** (Float) - Range: -1.0 to 1.0 - MouthLowerLeft to MouthLowerRight, with 0 being neutral - **MouthX** (Float) - Range: -1.0 to 1.0 - MouthLeft (Upper/Lower) to MouthRight (Upper/Lower), with 0 being neutral - **MouthUpperInsideOverturn** (Float) - Range: -1.0 to 1.0 - MouthUpperOverturn to MouthUpperInside, with 0 being neutral - **MouthLowerInsideOverturn** (Float) - Range: -1.0 to 1.0 - MouthLowerOverturn to MouthLowerInside, with 0 being neutral - **SmileSadRight** (Float) - Range: -1.0 to 1.0 - MouthSadRight to MouthSmileRight, with 0 being neutral - **SmileSadLeft** (Float) - Range: -1.0 to 1.0 - MouthSadLeft to MouthSmileLeft, with 0 being neutral - **SmileSad** (Float) - Range: -1.0 to 1.0 - MouthSad (Left/Right) to MouthSmile (Left/Right), with 0 being neutral - **TongueY** (Float) - Range: -1.0 to 1.0 - TongueDown to TongueUp, with 0 being neutral - **TongueX** (Float) - Range: -1.0 to 1.0 - TongueLeft to TongueRight, with 0 being neutral - **TongueSteps** (Float) - Range: -1.0 to 1.0 - TongueLongStep1 to TongueLongStep2, with -1 being tongue fully in to 1 being fully out - **PuffSuckRight** (Float) - Range: -1.0 to 1.0 - CheekSuck to CheekPuffRight, with 0 being neutral - **PuffSuckLeft** (Float) - Range: -1.0 to 1.0 - CheekSuck to CheekPuffLeft, with 0 being neutral - **PuffSuck** (Float) - Range: -1.0 to 1.0 - CheekSuck to CheekPuff (Left/Right), with 0 being neutral ### Jaw Open Combined Parameters #### Query Parameters - **JawOpenApe** (Float) - Range: -1.0 to 1.0 - MouthApeShape to JawOpen, with 0 being neutral - **JawOpenPuff** (Float) - Range: -1.0 to 1.0 - CheekPuff (Left/Right) to JawOpen, with 0 being neutral - **JawOpenPuffRight** (Float) - Range: -1.0 to 1.0 - CheekPuffRight to JawOpen, with 0 being neutral - **JawOpenPuffLeft** (Float) - Range: -1.0 to 1.0 - CheekPuffLeft to JawOpen, with 0 being neutral - **JawOpenSuck** (Float) - Range: -1.0 to 1.0 - CheekSuck to JawOpen, with 0 being neutral - **JawOpenForward** (Float) - Range: -1.0 to 1.0 - JawForward to JawOpen, with 0 being neutral - **JawOpenOverlay** (Float) - Range: -1.0 to 1.0 - MouthLowerOverlay to JawOpen, with 0 being neutral ### Mouth Upper Up Right Combined Parameters #### Query Parameters - **MouthUpperUpRightUpperInside** (Float) - Range: -1.0 to 1.0 - MouthUpperInside to MouthUpperUpRight, with 0 being neutral - **MouthUpperUpRightPuffRight** (Float) - Range: -1.0 to 1.0 - CheekPuffRight to MouthUpperUpRight, with 0 being neutral - **MouthUpperUpRightApe** (Float) - Range: -1.0 to 1.0 - MouthApeShape to MouthUpperUpRight, with 0 being neutral - **MouthUpperUpRightPout** (Float) - Range: -1.0 to 1.0 - MouthPout to MouthUpperUpRight, with 0 being neutral - **MouthUpperUpRightOverlay** (Float) - Range: -1.0 to 1.0 - MouthLowerOverlay Shape to MouthUpperUpRight, with 0 being neutral - **MouthUpperUpRightSuck** (Float) - Range: -1.0 to 1.0 - CheekSuck Shape to MouthUpperUpRight, with 0 being neutral ### Mouth Upper Up Left Combined Parameters #### Query Parameters - **MouthUpperUpLeftUpperInside** (Float) - Range: -1.0 to 1.0 - MouthUpperInside to MouthUpperUpLeft, with 0 being neutral - **MouthUpperUpLeftPuffLeft** (Float) - Range: -1.0 to 1.0 - CheekPuffLeft to MouthUpperUpLeft, with 0 being neutral - **MouthUpperUpLeftApe** (Float) - Range: -1.0 to 1.0 - MouthApeShape to MouthUpperUpLeft, with 0 being neutral - **MouthUpperUpLeftPout** (Float) - Range: -1.0 to 1.0 - MouthPout to MouthUpperUpLeft, with 0 being neutral - **MouthUpperUpLeftOverlay** (Float) - Range: -1.0 to 1.0 - MouthLowerOverlay Shape to MouthUpperUpLeft, with 0 being neutral - **MouthUpperUpLeftSuck** (Float) - Range: -1.0 to 1.0 - CheekSuck Shape to MouthUpperUpLeft, with 0 being neutral ### Mouth Upper Up Combined Parameters #### Query Parameters - **MouthUpperUpUpperInside** (Float) - Range: -1.0 to 1.0 - MouthUpperInside to MouthUpperUp (Left/Right), with 0 being neutral - **MouthUpperUpInside** (Float) - Range: -1.0 to 1.0 - MouthInside (Upper/Lower) to MouthUpperUp (Left/Right), with 0 being neutral - **MouthUpperUpPuff** (Float) - Range: -1.0 to 1.0 - CheekPuff (Left/Right) to MouthUpperUp (Left/Right), with 0 being neutral - **MouthUpperUpPuffLeft** (Float) - Range: -1.0 to 1.0 - CheekPuffLeft to MouthUpperUp (Left/Right), with 0 being neutral - **MouthUpperUpPuffRight** (Float) - Range: -1.0 to 1.0 - CheekPuffRight to MouthUpperUp (Left/Right), with 0 being neutral - **MouthUpperUpApe** (Float) - Range: -1.0 to 1.0 - MouthApeShape to MouthUpperUp (Left/Right), with 0 being neutral - **MouthUpperUpPout** (Float) - Range: -1.0 to 1.0 - MouthPout to MouthUpperUp (Left/Right), with 0 being neutral - **MouthUpperUpOverlay** (Float) - Range: -1.0 to 1.0 - MouthLowerOverlay Shape to MouthUpperUp (Left/Right), with 0 being neutral - **MouthUpperUpSuck** (Float) - Range: -1.0 to 1.0 - CheekSuck Shape to MouthUpperUp (Left/Right), with 0 being neutral ``` -------------------------------- ### OscMessage - OSC Message Handling Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Handles the creation, encoding, and parsing of OSC messages used to communicate avatar parameters with VRChat. ```APIDOC ## OscMessage ### Description Handles the encoding and decoding of OSC protocol messages. Use this to construct messages for VRChat parameters. ### Method N/A (Class-based) ### Parameters - **Address** (string) - Required - The OSC address path (e.g., /avatar/parameters/JawOpen) - **Type** (Type) - Required - The data type (float, bool, int) ### Request Example var msg = new OscMessage("/avatar/parameters/JawOpen", typeof(float)); msg.Value = 0.75f; ### Response - **Value** (object) - The decoded value from the OSC message. ``` -------------------------------- ### Accessing and Modifying UnifiedTrackingData in C# Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt This snippet demonstrates how to access the global tracking data instance and update various tracking parameters such as eye gaze, openness, pupil diameter, expression shapes, and head pose. It shows how to set individual values and retrieve combined eye data. ```csharp using VRCFaceTracking.Core.Params.Data; using VRCFaceTracking.Core.Params.Expressions; using VRCFaceTracking.Core.Types; // Access the global tracking data instance var trackingData = UnifiedTracking.Data; // === Eye Tracking Data === // Set left eye gaze direction (normalized -1 to 1) trackingData.Eye.Left.Gaze = new Vector2(0.5f, -0.2f); // Looking right and slightly down trackingData.Eye.Right.Gaze = new Vector2(0.5f, -0.2f); // Set eye openness (0 = closed, 1 = fully open) trackingData.Eye.Left.Openness = 0.8f; trackingData.Eye.Right.Openness = 0.8f; // Set pupil diameter in millimeters trackingData.Eye.Left.PupilDiameter_MM = 4.5f; trackingData.Eye.Right.PupilDiameter_MM = 4.5f; // Get combined eye data (averages both eyes) UnifiedSingleEyeData combinedEye = trackingData.Eye.Combined(); // combinedEye.Gaze, combinedEye.Openness, combinedEye.PupilDiameter_MM (normalized) // === Expression Shapes === // Set individual expression weights (0.0 to 1.0) trackingData.Shapes[(int)UnifiedExpressions.JawOpen].Weight = 0.7f; trackingData.Shapes[(int)UnifiedExpressions.MouthClosed].Weight = 0.0f; trackingData.Shapes[(int)UnifiedExpressions.EyeWideLeft].Weight = 0.3f; trackingData.Shapes[(int)UnifiedExpressions.EyeWideRight].Weight = 0.3f; trackingData.Shapes[(int)UnifiedExpressions.BrowInnerUpLeft].Weight = 0.5f; trackingData.Shapes[(int)UnifiedExpressions.BrowInnerUpRight].Weight = 0.5f; trackingData.Shapes[(int)UnifiedExpressions.CheekPuffLeft].Weight = 0.8f; trackingData.Shapes[(int)UnifiedExpressions.CheekPuffRight].Weight = 0.8f; trackingData.Shapes[(int)UnifiedExpressions.TongueOut].Weight = 1.0f; // === Head Pose Data === // Set head rotation (normalized -1 to 1, representing -90 to 90 degrees) trackingData.Head.HeadYaw = 0.2f; // Looking slightly right trackingData.Head.HeadPitch = -0.1f; // Looking slightly up trackingData.Head.HeadRoll = 0.0f; // No head tilt // Set head position (normalized -1 to 1, ~0.5m range from origin) trackingData.Head.HeadPosX = 0.0f; // Centered horizontally trackingData.Head.HeadPosY = 0.1f; // Slightly above origin trackingData.Head.HeadPosZ = -0.2f; // Slightly back from origin ``` -------------------------------- ### OscRecvService - Receiving OSC Messages Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Background service that listens for and processes incoming OSC messages from VRChat. ```APIDOC ## OscRecvService ### Description Listens for incoming OSC traffic from VRChat. Useful for detecting avatar changes or parameter feedback. ### Method Event Subscription ### Parameters - **OnMessageReceived** (Action) - Required - Callback for processing incoming messages. ### Request Example _recvService.OnMessageReceived = HandleIncomingMessage; ### Response - **message** (OscMessage) - The parsed incoming OSC message containing Address and Value. ``` -------------------------------- ### OscSendService - Sending OSC Messages Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Service responsible for transmitting OSC messages or bundles over UDP to the VRChat client. ```APIDOC ## OscSendService ### Description Handles the transmission of OSC messages over UDP. Supports sending individual messages or bundles for efficiency. ### Method Async Send ### Parameters - **message** (OscMessage) - Required - The message to send. - **messages** (OscMessage[]) - Required - An array of messages to send as a bundle. ### Request Example await _sendService.Send(singleMessage, ct); ### Response - **OnMessagesDispatched** (Action) - Callback triggered after successful dispatch. ``` -------------------------------- ### Smile Combined Parameters Source: https://github.com/benaclejames/vrcfacetracking/wiki/Parameters Parameters for controlling the general smile, combining various shapes. ```APIDOC ## Smile Combined Parameters ### Description Parameters for controlling the general smile, combining various shapes. ### Method N/A (These are parameters, not an endpoint) ### Endpoint N/A ### Parameters #### Request Body Parameters - **SmileUpperOverturn** (number) - Range: -1.0 - 1.0 - MouthUpperOverturn to MouthSmile (Left/Right), with 0 being neutral - **SmileLowerOverturn** (number) - Range: -1.0 - 1.0 - MouthLowerOverturn to MouthSmile (Left/Right), with 0 being neutral - **SmileApe** (number) - Range: -1.0 - 1.0 - MouthApeShape to MouthSmile (Left/Right), with 0 being neutral - **SmileOverlay** (number) - Range: -1.0 - 1.0 - MouthLowerOverlay to MouthSmile (Left/Right), with 0 being neutral - **SmilePout** (number) - Range: -1.0 - 1.0 - MouthLowerPout to MouthSmile (Left/Right), with 0 being neutral ### Request Example ```json { "SmileUpperOverturn": 0.85, "SmileLowerOverturn": 0.65, "SmileApe": 0.25, "SmileOverlay": 0.45, "SmilePout": 0.15 } ``` ### Response N/A (Parameter descriptions, not an endpoint response) ``` -------------------------------- ### Mouth Lower Down Left Combined Parameters Source: https://github.com/benaclejames/vrcfacetracking/wiki/Parameters Parameters for controlling the lower left part of the mouth, combining various shapes. ```APIDOC ## Mouth Lower Down Left Combined Parameters ### Description Parameters for controlling the lower left part of the mouth, combining various shapes. ### Method N/A (These are parameters, not an endpoint) ### Endpoint N/A ### Parameters #### Request Body Parameters - **MouthLowerDownLeftLowerInside** (number) - Range: -1.0 - 1.0 - MouthLowerInside to MouthLowerDownLeft, with 0 being neutral - **MouthLowerDownLeftPuffLeft** (number) - Range: -1.0 - 1.0 - CheekPuffLeft to MouthLowerDownLeft, with 0 being neutral - **MouthLowerDownLeftApe** (number) - Range: -1.0 - 1.0 - MouthApeShape to MouthLowerDownLeft, with 0 being neutral - **MouthLowerDownLeftPout** (number) - Range: -1.0 - 1.0 - MouthPout to MouthLowerDownLeft, with 0 being neutral - **MouthLowerDownLeftOverlay** (number) - Range: -1.0 - 1.0 - MouthLowerOverlay Shape to MouthLowerDownLeft, with 0 being neutral - **MouthLowerDownLeftSuck** (number) - Range: -1.0 - 1.0 - CheekSuck Shape to MouthLowerDownLeft, with 0 being neutral ### Request Example ```json { "MouthLowerDownLeftLowerInside": 0.6, "MouthLowerDownLeftPuffLeft": -0.3, "MouthLowerDownLeftApe": 0.2, "MouthLowerDownLeftPout": 0.1, "MouthLowerDownLeftOverlay": 0.4, "MouthLowerDownLeftSuck": -0.5 } ``` ### Response N/A (Parameter descriptions, not an endpoint response) ``` -------------------------------- ### Mouth Lower Down Combined Parameters Source: https://github.com/benaclejames/vrcfacetracking/wiki/Parameters Parameters for controlling the general lower part of the mouth, combining various shapes. ```APIDOC ## Mouth Lower Down Combined Parameters ### Description Parameters for controlling the general lower part of the mouth, combining various shapes. ### Method N/A (These are parameters, not an endpoint) ### Endpoint N/A ### Parameters #### Request Body Parameters - **MouthLowerDownLowerInside** (number) - Range: -1.0 - 1.0 - MouthLowerInside to MouthLowerDown (Left/Right), with 0 being neutral - **MouthLowerDownInside** (number) - Range: -1.0 - 1.0 - MouthInside (Upper/Lower) to MouthLowerDown (Left/Right), with 0 being neutral - **MouthLowerDownPuff** (number) - Range: -1.0 - 1.0 - CheekPuff (Left/Right) to MouthLowerDown (Left/Right), with 0 being neutral - **MouthLowerDownPuffLeft** (number) - Range: -1.0 - 1.0 - CheekPuffLeft to MouthLowerDown (Left/Right), with 0 being neutral - **MouthLowerDownPuffRight** (number) - Range: -1.0 - 1.0 - CheekPuffRight to MouthLowerDown (Left/Right), with 0 being neutral - **MouthLowerDownApe** (number) - Range: -1.0 - 1.0 - MouthApeShape to MouthLowerDown (Left/Right), with 0 being neutral - **MouthLowerDownPout** (number) - Range: -1.0 - 1.0 - MouthPout to MouthLowerDown (Left/Right), with 0 being neutral - **MouthLowerDownOverlay** (number) - Range: -1.0 - 1.0 - MouthLowerOverlay Shape to MouthLowerDown (Left/Right), with 0 being neutral - **MouthLowerDownSuck** (number) - Range: -1.0 - 1.0 - CheekSuck Shape to MouthLowerDown (Left/Right), with 0 being neutral ### Request Example ```json { "MouthLowerDownLowerInside": 0.7, "MouthLowerDownInside": 0.2, "MouthLowerDownPuff": -0.1, "MouthLowerDownPuffLeft": 0.3, "MouthLowerDownPuffRight": -0.2, "MouthLowerDownApe": 0.1, "MouthLowerDownPout": 0.0, "MouthLowerDownOverlay": 0.5, "MouthLowerDownSuck": -0.3 } ``` ### Response N/A (Parameter descriptions, not an endpoint response) ``` -------------------------------- ### UnifiedExpressions Data Access Source: https://context7.com/benaclejames/vrcfacetracking/llms.txt Accessing and setting facial expression weights using the UnifiedExpressions enumeration. ```APIDOC ## UnifiedExpressions Data Access ### Description This interface allows developers to set the weight (0.0 to 1.0) for specific facial expressions defined in the UnifiedExpressions v2 standard. ### Method SET (Property Assignment) ### Endpoint UnifiedTracking.Data.Shapes[UnifiedExpressions] ### Parameters #### Path Parameters - **UnifiedExpressions** (enum) - Required - The specific facial expression index to modify. #### Request Body - **Weight** (float) - Required - The intensity of the expression, ranging from 0.0 (no movement) to 1.0 (full movement). ### Request Example UnifiedTracking.Data.Shapes[(int)UnifiedExpressions.JawOpen].Weight = 0.6f; ### Response #### Success Response (200) - **Status** (void) - The weight value is updated in the tracking buffer for the next frame. ``` -------------------------------- ### Tracking Status Bools Source: https://github.com/benaclejames/vrcfacetracking/wiki/Parameters These booleans indicate the active status of eye and lip tracking. ```APIDOC ## Tracking Status Bools The status of the tracking states of eye tracking and lip tracking from VRCFaceTracking. ### Parameters #### Query Parameters - **EyeTrackingActive** (Bool) - Required - Eye Tracking is Active (SRanipal Green Eyes) - **LipTrackingActive** (Bool) - Required - Lip Tracking is Active (SRanipal Green Mouth) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.