### Run example with gRPC Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/drive-from-myogestic.md Execute a synthetic example that utilizes gRPC for EMG classification and a movement palette. Ensure 'examples' and 'grpc' extras are installed. ```bash uv run --extra examples --extra grpc python examples/synthetic/emg_classification_grpc.py ``` -------------------------------- ### Setup and Installation for MyoGestic VHI Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Installs necessary NuGet packages and launches the Godot editor. Includes commands to run the LSL test sender. ```bash # Prerequisites: Godot 4.6 .NET build, .NET 8 SDK on PATH dotnet restore # restores SharpLSL, Grpc.AspNetCore, Tomlyn godot --path . # opens the editor; press F5 to run # Expected console output on successful start: # ✅ gRPC control server listening on 127.0.0.1:50051 # Run the LSL test sender to drive the predicted hand immediately: pip install pylsl numpy python tests/test_lsl_sender.py ``` -------------------------------- ### VHI Console Output Example Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/getting-started.md This is an example of the console output you should see when the gRPC control server starts successfully. ```text ✅ gRPC control server listening on 127.0.0.1:50051 ``` -------------------------------- ### Install gRPC dependencies Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/drive-from-myogestic.md Install the necessary gRPC libraries using uv sync with the extra grpc option. ```bash uv sync --extra grpc ``` -------------------------------- ### Generate C# API Reference Docs Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt This sequence of commands installs the necessary .NET tool, builds the assembly, generates the API documentation, and serves it locally for browsing. Re-run the generation command whenever XML comments in the source code change. ```bash dotnet tool restore # installs DefaultDocumentation.Console ``` ```bash ./tools/gen_api_docs.sh # builds the assembly, generates docs/reference/api/ ``` ```bash uv run --group docs properdocs serve # browse at http://127.0.0.1:8000 ``` ```bash # Re-run gen_api_docs.sh whenever /// XML comments in src/ or proto/ change ``` -------------------------------- ### Restore NuGet Packages and Run VHI Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/getting-started.md Use these commands to restore project dependencies and launch the VHI application. The first command restores packages, and the second starts the Godot editor or runs the project headlessly. ```bash # 1. Restore NuGet packages (SharpLSL, Grpc.AspNetCore, Tomlyn) dotnet restore # 2. Open the project in the Godot editor and press F5 - or run headless: godot --path . ``` -------------------------------- ### Building Standalone Executables - Godot Export Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Build standalone executables for different platforms using the Godot engine. Ensure .NET 8 is installed and the export preset names match `export_presets.cfg`. macOS requires a post-signing step. ```bash # Pin .NET 8 (global.json already present in the repo): # { "sdk": { "version": "8.0.0", "rollForward": "latestMinor" } } dotnet --version # must report 8.0.x # Export for each platform (preset names must match export_presets.cfg exactly) godot --headless --export-release "macOS" VHI.app godot --headless --export-release "Windows Desktop" VHI.exe godot --headless --export-release "Linux" VHI.x86_64 # macOS REQUIRED post-step: re-sign without the hardened runtime # (Godot's ad-hoc signing with hardened runtime silently prevents C# from starting) codesign --force --deep --sign - VHI.app ``` -------------------------------- ### Model-Robustness Validation with Stream Mode Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/stream-a-custom-pose.md This comprehensive Python script demonstrates a full use case for testing myocontrol model robustness. It systematically injects various 9-DOF poses and includes setup, continuous injection, and tear-down phases. ```python import time from itertools import product import numpy as np from myogestic.interfaces import virtual_hand vhi = virtual_hand() client = vhi.control_client() pose_outlet = vhi.control_outlet() # 1. Orchestration over gRPC. client.set_session_active(True) # gate VHI's keyboard - MyoGestic owns the hand client.set_control_mode("STREAM") # control hand reads from MyoGestic_ControlPose assert client.get_state().control_mode == "STREAM" # sanity-check # 2. Continuous pose injection over LSL. LEVELS = [0.0, 0.5, 1.0] # rest / half / full flexion per DOF DOFS = 6 # 6 finger DOFs; wrist held at 0 SETTLE_S = 0.5 for combo in product(LEVELS, repeat=DOFS): # 3^6 = 729 multi-DOF poses pose = np.array([*combo, 0, 0, 0], dtype=np.float32) pose_outlet.push(pose) time.sleep(SETTLE_S) # Your model's prediction at this moment is recorded on its own LSL # outlet; line it up with VHI_Control post-hoc via XDF timestamps. # 3. Tear down. client.set_control_mode("MOVEMENT") client.set_session_active(False) ``` -------------------------------- ### Build documentation site Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/build-and-export.md Commands to build the documentation site, including restoring .NET tools, generating C# API reference, and building the site with properdocs. Requires .NET 8 SDK, uv, and Python 3. ```bash dotnet tool restore # installs DefaultDocumentation.Console ./tools/gen_api_docs.sh # builds the assembly, regenerates the API md uv run --group docs properdocs build # the docs site itself ``` -------------------------------- ### Get VHI State via gRPC Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Call GetState first to confirm VHI is reachable and discover valid movement names. VHI hosts a gRPC server on 127.0.0.1:50051. ```python import grpc from myogestic_vhi_pb2_grpc import VhiControlStub from myogestic_vhi_pb2 import GetStateRequest channel = grpc.insecure_channel("127.0.0.1:50051") stub = VhiControlStub(channel) state = stub.GetState(GetStateRequest()) # state.current_state → "waiting" | "closing" | "holding" | "opening" | "resting" | "frozen" # state.current_movement → e.g. "Rest" # state.mode → "AI" | "Classifier" # state.control_mode → ControlMode.MOVEMENT | STREAM | IDLE # state.session_active → bool # state.available_movements → list of valid SetMovement names for the current mode print("VHI reachable — mode:", state.mode) print("Available movements:", list(state.available_movements)) # Example output: # VHI reachable — mode: AI # Available movements: ['Rest','Thumb','Index','Middle','Ring','Pinky','Fist', # 'TwoFingerPinch','ThreeFingerPinch','Pointing','ThumbExtension','IndexExtension', # 'MiddleExtension','RingExtension','PinkyExtension','WristUpDown','WristLeftRight']) ``` -------------------------------- ### Generate API Documentation Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/README.md Restore .NET tools if necessary, then execute the script to regenerate C# API reference documentation. Serve the documentation locally. ```bash dotnet tool restore # one-off: installs DefaultDocumentation.Console ./tools/gen_api_docs.sh # regenerates the C# API reference from the source uv run --group docs properdocs serve # browse at http://127.0.0.1:8000 ``` -------------------------------- ### Restore Dependencies and Run Godot Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/README.md Restore project dependencies using dotnet and then launch the Godot editor. Press F5 to run the application. ```bash dotnet restore # restore SharpLSL, Grpc.AspNetCore, Tomlyn godot --path . # open in the editor, F5 to run ``` -------------------------------- ### MyoGestic Python Client - Initialize and Control Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Initialize the MyoGestic virtual hand client and use its high-level interface for controlling hand movements and state. Requires optional gRPC dependency. Commands are fire-and-forget. ```python from myogestic.interfaces import virtual_hand from myogestic.widgets import process_launcher # Install the gRPC optional dependency first: # uv sync --extra grpc vhi = virtual_hand( # godot_bin="...", # or set GODOT_BIN env var # vhi_path="...", # or set VHI_PATH env var # grpc_host="127.0.0.1", # or set VHI_GRPC_HOST # grpc_port=50051, # or set VHI_GRPC_PORT ) # 1. Launch VHI as a managed subprocess process_launcher(vhi.launcher()) # 2. Open the async control client client = vhi.control_client() # 3. Fire-and-forget commands (return immediately; ack logged by worker thread) client.set_movement("Fist") # snap to end pose client.set_movement("Index", cycle=True) # play open/close cycle client.freeze(True) # freeze at current pose client.set_session_active(True) # recording live — disable keyboard client.set_speed(frequency_hz=1.0, hold_time_s=0.5, rest_time_s=0.5) client.set_smoothing(enabled=True, smoothing_speed=8.0) client.set_control_mode("STREAM") # "MOVEMENT" | "STREAM" | "IDLE" # 4. Synchronous state query (use on connect or explicit refresh, not every frame) state = client.get_state() if state is not None: # None == VHI not reachable print(state.mode) # "AI" | "Classifier" print(list(state.available_movements)) print(state.control_mode) # MOVEMENT | STREAM | IDLE else: print("VHI not reachable") # 5. Push arbitrary poses to the control hand when in Stream mode import numpy as np pose_outlet = vhi.control_outlet() # LSL outlet → MyoGestic_ControlPose pose = np.array([0, 0, -0.5, -0.5, -0.5, -0.5, 0, 0, 0], dtype=np.float32) pose_outlet.push(pose) # 6. Clean up client.stop() # stop worker thread, close channel ``` -------------------------------- ### Create LSL Outlet for Control Hand Poses (Stream Mode) Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Streams arbitrary runtime poses to the `MyoGestic_ControlPose` inlet. This requires the control hand to be in `Stream` mode. ```python from pylsl import StreamInfo, StreamOutlet import numpy as np info = StreamInfo( name="MyoGestic_ControlPose", type="MyoGestic_9DVector", channel_count=9, nominal_srate=32, channel_format="float32", source_id="my-control-pose", ) outlet = StreamOutlet(info) # Stream a partially closed hand (index + middle fingers only) pose = np.array([0, 0, -0.5, -0.5, 0, 0, 0, 0, 0], dtype=np.float32) outlet.push_sample(pose.tolist()) # VHI_Control still re-publishes whatever is shown → lands in the recording ``` -------------------------------- ### Push Custom 9-DOF Poses to Control Outlet Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/stream-a-custom-pose.md This Python code demonstrates how to create and push a 9-DOF pose to the LSL outlet for the control hand in Stream mode. Ensure the pose vector follows the specified channel layout. ```python import numpy as np pose_outlet = vhi.control_outlet() # Push a 9-DOF pose every frame. Channel layout (see LSL streams): # [thumb_flex, thumb_abd, index, middle, ring, pinky, wrist_flex, wrist_abd, wrist_rot] pose = np.array([0, 0, 0.5, 0.5, 0.5, 0.5, 0, 0, 0], dtype=np.float32) pose_outlet.push(pose) ``` -------------------------------- ### Export VHI for different platforms Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/build-and-export.md Command-line instructions to export the VHI project for macOS, Windows Desktop, and Linux using Godot's headless export functionality. Ensure the preset name matches the one defined in export_presets.cfg. ```bash godot --headless --export-release "macOS" VHI.app godot --headless --export-release "Windows Desktop" VHI.exe godot --headless --export-release "Linux" VHI.x86_64 ``` -------------------------------- ### Pin .NET 8 SDK with global.json Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/build-and-export.md Use a global.json file to ensure the .NET 8 SDK is the active version for exports. This prevents issues with newer .NET versions causing export failures. ```json { "sdk": { "version": "8.0.0", "rollForward": "latestMinor" } } ``` -------------------------------- ### Control Hand via gRPC with Python Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/getting-started.md This Python snippet demonstrates how to connect to the VHI's gRPC control server and command the 'control hand' to adopt a specific pose. ```python from myogestic.interfaces import virtual_hand client = virtual_hand().control_client() client.set_movement("Fist") # control hand snaps to the Fist pose ``` -------------------------------- ### Assembly Load Context Resolver Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/concepts/architecture.md A C# code snippet demonstrating the use of [ModuleInitializer] and AssemblyLoadContext.Resolving to handle .NET shared framework assembly loading within Godot. ```csharp src/SharedFrameworkAssemblyLoader.cs fixes this: a [ModuleInitializer] registers an AssemblyLoadContext.Resolving handler that probes the on-disk .NET shared-framework directories. It runs before any ASP.NET Core type is JITed, so Kestrel loads cleanly. You never call it directly - it just has to exist in the assembly. ``` -------------------------------- ### Minimal LSL Outlet Producer Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/reference/lsl-reference.md Creates a basic LSL outlet for publishing 9-DOF pose data. This can be used to send VHI's predicted hand pose. ```python from pylsl import StreamInfo, StreamOutlet info = StreamInfo("MyoGestic_Output", "MyoGestic_9DVector", 9, 32, "float32", "my-source") outlet = StreamOutlet(info) outlet.push_sample([0.0] * 9) # 9-DOF pose; VHI's predicted hand follows it ``` -------------------------------- ### GetState Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Establishes a connection handshake and discovers valid movement names. This RPC should always be called first to confirm VHI is reachable. ```APIDOC ### `GetState` — connection handshake and movement discovery Always call `GetState` first to confirm VHI is reachable and to discover valid movement names. ```python import grpc from myogestic_vhi_pb2_grpc import VhiControlStub from myogestic_vhi_pb2 import GetStateRequest channel = grpc.insecure_channel("127.0.0.1:50051") stub = VhiControlStub(channel) state = stub.GetState(GetStateRequest()) # state.current_state → "waiting" | "closing" | "holding" | "opening" | "resting" | "frozen" # state.current_movement → e.g. "Rest" # state.mode → "AI" | "Classifier" # state.control_mode → ControlMode.MOVEMENT | STREAM | IDLE # state.session_active → bool # state.available_movements → list of valid SetMovement names for the current mode print("VHI reachable — mode:", state.mode) print("Available movements:", list(state.available_movements)) # Example output: # VHI reachable — mode: AI # Available movements: ['Rest','Thumb','Index','Middle','Ring','Pinky','Fist', # 'TwoFingerPinch','ThreeFingerPinch','Pointing','ThumbExtension','IndexExtension', # 'MiddleExtension','RingExtension','PinkyExtension','WristUpDown','WristLeftRight'] ``` ``` -------------------------------- ### Create LSL Outlet for Predicted Hand Data Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Sets up an LSL outlet to stream 9-DOF hand pose data for the predicted hand. The channel names and format must match VHI's expectations. ```python from pylsl import StreamInfo, StreamOutlet import numpy as np # Create the outlet (name and channel count must match exactly) info = StreamInfo( name="MyoGestic_Output", type="MyoGestic_9DVector", channel_count=9, nominal_srate=32, # 32 Hz is typical; VHI accepts any rate channel_format="float32", source_id="my-emg-pipeline", ) # Label channels (optional but recommended for recording) channels = info.desc().append_child("channels") for label in ["ThumbFlexion","ThumbAbduction","IndexFlexion","MiddleFlexion", "RingFlexion","PinkyFlexion","WristFlexion","WristAbduction","WristRotation"]: channels.append_child("channel").append_child_value("label", label) outlet = StreamOutlet(info) # 9-DOF pose: [thumb_flex, thumb_abd, index, middle, ring, pinky, wrist_flex, wrist_abd, wrist_rot] # Closed fist: all fingers + thumb abduction pulled in, wrist neutral fist_pose = np.array([-1, -1, -1, -1, -1, -1, 0, 0, 0], dtype=np.float32) outlet.push_sample(fist_pose.tolist()) # Sinusoidal wave across all fingers at 32 Hz import time, math t0 = time.time() while True: t = time.time() - t0 phase = 2 * math.pi * t * 0.2 # 0.2 Hz pose = [ -(math.sin(phase) + 1) / 2, # ThumbFlexion -0.3, # ThumbAbduction -(math.sin(phase + math.pi/4) + 1)/2, # Index -(math.sin(phase + math.pi/2) + 1)/2, # Middle -(math.sin(phase + 3*math.pi/4)+1)/2, # Ring -(math.sin(phase + math.pi) + 1) / 2, # Pinky 0, 0, 0, # Wrist (not animated by default) ] outlet.push_sample(pose) time.sleep(1.0 / 32) ``` -------------------------------- ### Launch VHI and command control hand Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/drive-from-myogestic.md Launch VHI as a managed subprocess and interact with its gRPC control client to set movements, freeze poses, and activate sessions. Commands are fire-and-forget. ```python from myogestic.interfaces import virtual_hand from myogestic.widgets import process_launcher vhi = virtual_hand() # 1. Launch VHI as a managed subprocess (godot --path ...). # In an app, wire vhi.launcher() into a process_launcher widget. process_launcher(vhi.launcher()) # 2. Open the gRPC control client (fire-and-forget; never blocks the GUI). client = vhi.control_client() # 3. Command the control hand. client.set_movement("Fist") # snap to the Fist end pose, hold it client.set_movement("Index", cycle=True) # play the open/close cycle instead client.freeze(True) # freeze at the current pose client.set_session_active(True) # recording live - VHI ignores its keyboard ``` -------------------------------- ### Publish LSL Stream for Hand Pose Data Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/README.md Publish an LSL stream named 'MyoGestic_Output' with 9 float channels representing hand pose data. The channel layout and sign convention for flexion are specified. ```python from pylsl import StreamInfo, StreamOutlet info = StreamInfo("MyoGestic_Output", "MyoGestic_9DVector", 9, 32, "float32", "my_uid") outlet = StreamOutlet(info) outlet.push_sample([0.0] * 9) ``` -------------------------------- ### Switch Control Hand to Stream Mode Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/stream-a-custom-pose.md Use this Python snippet to set the control hand to 'STREAM' mode. This prepares it to receive custom pose data. ```python from myogestic.interfaces import virtual_hand vhi = virtual_hand() client = vhi.control_client() client.set_control_mode("STREAM") ``` -------------------------------- ### MyoGestic Python Client Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt High-level Python client for MyoGestic VHI, wrapping raw stubs with fire-and-forget threading. ```APIDOC ## MyoGestic Python Client (`virtual_hand`) ### Description MyoGestic ships a higher-level client that wraps the raw stubs with fire-and-forget threading so a 60 fps GUI never stalls on a gRPC call. ### Initialization ```python from myogestic.interfaces import virtual_hand vhi = virtual_hand( # godot_bin="...", # or set GODOT_BIN env var # vhi_path="...", # or set VHI_PATH env var # grpc_host="127.0.0.1", # or set VHI_GRPC_HOST # grpc_port=50051, # or set VHI_GRPC_PORT ) ``` ### Launching VHI ```python from myogestic.widgets import process_launcher # Install the gRPC optional dependency first: # uv sync --extra grpc # 1. Launch VHI as a managed subprocess process_launcher(vhi.launcher()) ``` ### Control Client Methods ```python # 2. Open the async control client client = vhi.control_client() # 3. Fire-and-forget commands (return immediately; ack logged by worker thread) client.set_movement("Fist") # snap to end pose client.set_movement("Index", cycle=True) # play open/close cycle client.freeze(True) # freeze at current pose client.set_session_active(True) # recording live — disable keyboard client.set_speed(frequency_hz=1.0, hold_time_s=0.5, rest_time_s=0.5) client.set_smoothing(enabled=True, smoothing_speed=8.0) client.set_control_mode("STREAM") # "MOVEMENT" | "STREAM" | "IDLE" # 6. Clean up client.stop() # stop worker thread, close channel ``` ### Synchronous State Query ```python # 4. Synchronous state query (use on connect or explicit refresh, not every frame) state = client.get_state() if state is not None: # None == VHI not reachable print(state.mode) # "AI" | "Classifier" print(list(state.available_movements)) print(state.control_mode) # MOVEMENT | STREAM | IDLE else: print("VHI not reachable") ``` ### Pushing Poses ```python # 5. Push arbitrary poses to the control hand when in Stream mode import numpy as np pose_outlet = vhi.control_outlet() # LSL outlet → MyoGestic_ControlPose pose = np.array([0, 0, -0.5, -0.5, -0.5, -0.5, 0, 0, 0], dtype=np.float32) pose_outlet.push(pose) ``` ``` -------------------------------- ### VHI Threading Model Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/concepts/architecture.md Illustrates the threading model where communication layers (LSL, gRPC) operate on background threads and marshal results to the Godot main thread for scene updates. ```mermaid flowchart TB subgraph bg["Background threads"] lslresolve["LSL resolve / pull
(Task.Run)"] grpc["gRPC handlers
(Kestrel thread pool)"] end subgraph main["Godot main thread"] process["_Process / _PhysicsProcess"] bones["bone updates, scene state"] end lslresolve -- "CallDeferred" --> process grpc -- "InvokeOnMainThread queue" --> process process --> bones ``` -------------------------------- ### Define a Custom Movement in TOML Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/add-a-custom-movement.md This TOML snippet shows how to define a new custom movement named 'MyGesture'. Each line specifies a joint and its maximum flexion pose in Euler degrees. The 'movements.toml' file supports hot-reloading, so saving this file will update the hand's behavior live. ```toml # movements.toml - one movement entry (joint_name = [x, y, z], degrees) [movements.MyGesture] wrist = [0, 0, 0] thumb_proximal = [-20, 0, 0] thumb_middle = [-30, 0, 0] thumb_distal = [-30, 0, 0] index_proximal = [-40, 0, 0] # … index_middle, index_distal, then middle_*, ring_*, pinky_* (16 joints total) ``` -------------------------------- ### gRPC Protocol Definition Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/reference/grpc-api.md The canonical source for the gRPC API contract, used to generate client stubs. ```protobuf syntax = "proto3"; package myogestic_vhi; // VHI's discrete control plane. service VhiControl { // Select a predefined movement. // cycle=false holds the end pose; cycle=true plays the loop. // Rejected outside Movement mode. rpc SetMovement(SetMovementRequest) returns (CommandAck); // Freeze / release the control hand at its current pose. // Rejected outside Movement mode. rpc Freeze(FreezeRequest) returns (CommandAck); // Movement animation timing (frequency, hold, rest). // Rejected outside Movement mode. rpc SetSpeed(SetSpeedRequest) returns (CommandAck); // Toggle predicted-hand smoothing and its speed. rpc SetSmoothing(SetSmoothingRequest) returns (CommandAck); // Not implemented - returns applied=false. rpc SetChirality(SetChiralityRequest) returns (CommandAck); // Mark a recording session active/inactive; gates VHI's local keyboard. // Orthogonal to driver mode. rpc SetSessionActive(SetSessionActiveRequest) returns (CommandAck); // Switch the control-hand driver mode. rpc SetControlMode(SetControlModeRequest) returns (CommandAck); // Query state; doubles as a connection handshake and movement-name discovery. rpc GetState(GetStateRequest) returns (StateReply); } // Returned by every command RPC. message CommandAck { // Whether the command took effect. bool applied = 1; // waiting | closing | holding | opening | resting | frozen string current_state = 2; // The currently selected movement name. string current_movement = 3; // Human-readable reason when applied == false. string message = 4; } // Returned by GetState. message StateReply { // As in CommandAck. string current_state = 1; // The currently selected movement name. string current_movement = 2; // Whether a recording session is marked active. bool session_active = 3; // Valid SetMovement names for the current mode - discover, don't hard-code. repeated string available_movements = 4; // "AI" or "Classifier" - the movement-set mode. string mode = 5; // MOVEMENT | STREAM | IDLE ControlMode control_mode = 6; } // Control mode enum. enum ControlMode { MOVEMENT = 0; // Predefined-movement state machine + keyboard. The default. STREAM = 1; // Continuous pose from the MyoGestic_ControlPose LSL inlet. IDLE = 2; // Hold the rest pose; ignore keyboard, stream, and movement commands. } // Request messages message SetMovementRequest { string movement_name = 1; bool cycle = 2; } message FreezeRequest { bool frozen = 1; } message SetSpeedRequest { float frequency_hz = 1; float hold_time_s = 2; float rest_time_s = 3; } message SetSmoothingRequest { bool enabled = 1; float smoothing_speed = 2; } message SetChiralityRequest { bool right_hand = 1; } message SetSessionActiveRequest { bool active = 1; } message SetControlModeRequest { ControlMode mode = 1; } // Empty request message. message GetStateRequest {} ``` -------------------------------- ### Subscribe to VHI_Control LSL Stream Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Subscribe to the VHI_Control stream to record ground-truth kinematics. Ensure the stream is enabled in LSLCommunicationController. ```python from pylsl import StreamInlet, resolve_stream # Subscribe to VHI_Control to record the ground-truth kinematics streams = resolve_stream("name", "VHI_Control") inlet = StreamInlet(streams[0]) while True: sample, timestamp = inlet.pull_sample(timeout=1.0) if sample: # sample is a list of 9 floats: [ThumbFlexion, ..., WristRotation] thumb_flex = sample[0] index_flex = sample[2] print(f"t={timestamp:.3f} thumb={thumb_flex:.3f} index={index_flex:.3f}") ``` -------------------------------- ### Freeze/Release Hand Pose via gRPC Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Use Freeze to freeze the control hand at its current pose or release it back to the movement state machine. The 'frozen' parameter controls the state. ```python from myogestic_vhi_pb2 import FreezeRequest # Freeze at whatever pose the hand is currently in ack = stub.Freeze(FreezeRequest(frozen=True)) print(ack.current_state) # → "frozen" # Release back to the movement state machine ack = stub.Freeze(FreezeRequest(frozen=False)) print(ack.current_state) # → "resting" ``` -------------------------------- ### Re-sign macOS app without hardened runtime Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/how-to/build-and-export.md Re-sign the macOS application bundle ad-hoc without the hardened runtime enabled. This is necessary because Godot's default ad-hoc signing with the hardened runtime prevents C# code from executing. ```bash codesign --force --deep --sign - VHI.app ``` -------------------------------- ### Control VHI Session Activity via gRPC Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Use SetSessionActive to gate VHI's local keyboard input. Setting 'active' to True means MyoGestic is the sole source of control. ```python from myogestic_vhi_pb2 import SetSessionActiveRequest # Recording session starts: VHI ignores its keyboard; MyoGestic is sole source ack = stub.SetSessionActive(SetSessionActiveRequest(active=True)) assert ack.applied ``` -------------------------------- ### Set Control Mode Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/concepts/control-modes.md Switch the control mode of the hand. Available modes are "MOVEMENT", "STREAM", and "IDLE". ```python client.set_control_mode("STREAM") # "MOVEMENT" | "STREAM" | "IDLE" ``` -------------------------------- ### SetSessionActive Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Gates VHI's local keyboard input. When active, VHI ignores its keyboard, making MyoGestic the sole source of control. ```APIDOC ### `SetSessionActive` — gate VHI's local keyboard ```python from myogestic_vhi_pb2 import SetSessionActiveRequest # Recording session starts: VHI ignores its keyboard; MyoGestic is sole source ack = stub.SetSessionActive(SetSessionActiveRequest(active=True)) assert ack.applied ``` ``` -------------------------------- ### Send Mock LSL Stream for Predicted Hand Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/getting-started.md Execute this Python script to send a mock LSL stream, which will be followed by the 'predicted hand' in VHI. ```bash python tests/test_lsl_sender.py ``` -------------------------------- ### GetState Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/reference/grpc-api.md Queries the current state of VHI. This RPC also serves as a connection handshake and allows for discovery of available movement names. ```APIDOC ## GetState ### Description Query state; doubles as a connection handshake and movement-name discovery. ### Request - *(empty)* ### Returns - `StateReply` ``` -------------------------------- ### VHI_Control Stream Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Subscribe to the VHI_Control stream to record the ground-truth kinematics of the control hand. This stream provides the current pose of the control hand at 60 Hz. ```APIDOC ## LSL Outlets — Reading Back What VHI Shows VHI publishes two 60 Hz outlets so the recording captures *exactly* what was rendered. Both are created at startup unless `EnableOutlets` is `false` on `LSLCommunicationController`. | Stream name | Carries | Rate | Source ID | |---|---|---|---| | `VHI_Control` | control hand's current pose | 60 Hz | `control_hand_001` | | `VHI_Predict` | predicted hand's current pose | 60 Hz | `predicted_hand_001` | ```python from pylsl import StreamInlet, resolve_stream # Subscribe to VHI_Control to record the ground-truth kinematics streams = resolve_stream("name", "VHI_Control") inlet = StreamInlet(streams[0]) while True: sample, timestamp = inlet.pull_sample(timeout=1.0) if sample: # sample is a list of 9 floats: [ThumbFlexion, ..., WristRotation] thumb_flex = sample[0] index_flex = sample[2] print(f"t={timestamp:.3f} thumb={thumb_flex:.3f} index={index_flex:.3f}") ``` ``` -------------------------------- ### Adjust Movement Speed via gRPC Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Use SetSpeed to adjust movement animation timing, controlling frequency, hold time, and rest time per cycle. Non-positive frequency and negative hold/rest times are ignored. ```python from myogestic_vhi_pb2 import SetSpeedRequest ack = stub.SetSpeed(SetSpeedRequest( frequency_hz=1.0, # movement cycles per second (default 0.5) hold_time_s=0.5, # seconds held at max flexion per cycle (default 1.0) rest_time_s=0.5, # seconds held at rest per cycle (default 1.0) )) assert ack.applied # Non-positive frequency is silently ignored; negative hold/rest are ignored ``` -------------------------------- ### Control Hand Movement with Keyboard Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/getting-started.md Focus the VHI window and use these keys to control the 'control hand'. Actions include cycling movements, starting/stopping, and freezing poses. ```markdown With the window focused, the **control hand** responds to: | Key | Action | |---|---| | :material-arrow-left: / :material-arrow-right: | cycle the selected movement (wraps around at both ends) | | :material-arrow-down: | start the movement | | :material-arrow-up: | stop, return to rest | | ++space++ | freeze / unfreeze at the current pose | ``` -------------------------------- ### Movement TOML Configuration - Define Hand Poses Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Configure named hand movements using TOML for hot-reloading. Defines the maximum flexion pose for each joint. Rest pose is always neutral. Edits to this file update the VHI live. ```toml # movements.toml # Location (default): user://movements.toml # macOS: ~/Library/Application Support/Godot/app_userdata/Virtual Hand Interface/movements.toml # Windows: %APPDATA%\Godot\app_userdata\Virtual Hand Interface\movements.toml # Linux: ~/.local/share/godot/app_userdata/Virtual Hand Interface/movements.toml # # Each [movements.] table defines the max-flexion pose. # Rest pose is always neutral [0, 0, 0] and is NOT stored here. # Sign convention: negative X = flexion. [movements.MyGesture] wrist = [0, 0, 0] thumb_proximal = [-20, 0, 0] thumb_middle = [-30, 0, 0] thumb_distal = [-30, 0, 0] index_proximal = [-40, 0, 0] index_middle = [-50, 0, 0] index_distal = [-40, 0, 0] middle_proximal = [0, 0, 0] middle_middle = [0, 0, 0] middle_distal = [0, 0, 0] ring_proximal = [0, 0, 0] ring_middle = [0, 0, 0] ring_distal = [0, 0, 0] pinky_proximal = [0, 0, 0] pinky_middle = [0, 0, 0] pinky_distal = [0, 0, 0] # After saving: # - "MyGesture" appears in GetState().available_movements # - SetMovement("MyGesture") plays it # - ← / → keyboard keys cycle through it ``` -------------------------------- ### Command VHI Movement via gRPC Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Use SetMovement to command the control hand to a named movement. The 'cycle' parameter determines if the movement plays as a cycle or a single pose. ```python from myogestic_vhi_pb2 import SetMovementRequest # Hold the Fist end pose (classifier output mode: cycle=False) ack = stub.SetMovement(SetMovementRequest(movement_name="Fist", cycle=False)) assert ack.applied, f"Rejected: {ack.message}" print(ack.current_state) # → "holding" # Play the open/close cycle (regression recording mode: cycle=True) ack = stub.SetMovement(SetMovementRequest(movement_name="Index", cycle=True)) # ack.current_state cycles through "closing" → "holding" → "opening" → "resting" # Return to rest ack = stub.SetMovement(SetMovementRequest(movement_name="Rest", cycle=False)) print(ack.current_state) # → "resting" # Unknown name → applied=False, message explains why ack = stub.SetMovement(SetMovementRequest(movement_name="BadName")) # ack.applied → False # ack.message → "unknown movement" ``` -------------------------------- ### SetMovement Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/reference/grpc-api.md Selects a predefined movement. `cycle=false` holds the end pose; `cycle=true` plays the loop. This RPC is rejected if VHI is not in `Movement` mode. ```APIDOC ## SetMovement ### Description Select a predefined movement. `cycle=false` holds the end pose; `cycle=true` plays the loop. Rejected outside `Movement` mode. ### Request - **movement_name** (string) - Required - The name of the movement to set. - **cycle** (bool) - Required - Whether to loop the movement. ### Returns - `CommandAck` ``` -------------------------------- ### SetSessionActive - End Session Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt This snippet shows how to deactivate the current session, which re-enables keyboard control for manual operator use. ```python ack = stub.SetSessionActive(SetSessionActiveRequest(active=False)) ``` -------------------------------- ### SetChirality Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/reference/grpc-api.md Sets the chirality (hand preference). This RPC is currently not implemented and will always return `applied=false`. ```APIDOC ## SetChirality ### Description **Not implemented** - returns `applied=false`. ### Request - **right_hand** (bool) - Required - Whether to set the right hand. ### Returns - `CommandAck` ``` -------------------------------- ### SetSmoothing Source: https://github.com/nsquaredlab/myogestic-vhi/blob/main/docs/reference/grpc-api.md Toggles predicted-hand smoothing and adjusts its speed. ```APIDOC ## SetSmoothing ### Description Toggle predicted-hand smoothing and its speed. ### Request - **enabled** (bool) - Required - Whether to enable smoothing. - **smoothing_speed** (float) - Required - The speed of the smoothing. ### Returns - `CommandAck` ``` -------------------------------- ### SetSessionActive Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Activates or deactivates the current session, controlling keyboard input for manual operator use. ```APIDOC ## SetSessionActive ### Description Activates or deactivates the current session, controlling keyboard input for manual operator use. ### Method POST ### Endpoint /SetSessionActive ### Request Body - **active** (boolean) - Required - True to activate the session, False to deactivate. ### Request Example ```json { "active": false } ``` ### Response #### Success Response (200) - **applied** (boolean) - Indicates if the session activation change was applied. ``` -------------------------------- ### SetControlMode Source: https://context7.com/nsquaredlab/myogestic-vhi/llms.txt Switches the control hand's driver mode between STREAM, IDLE, and MOVEMENT. ```APIDOC ## SetControlMode ### Description Switches the control hand's driver mode. The three modes are mutually exclusive; only one drives the bones at a time. ### Method POST ### Endpoint /SetControlMode ### Parameters #### Request Body - **mode** (ControlMode) - Required - The desired control mode. Possible values: STREAM, IDLE, MOVEMENT. ### Request Example ```json { "mode": "STREAM" } ``` ### Response #### Success Response (200) - **applied** (boolean) - Indicates if the control mode change was applied. ### Notes - Switching to `MOVEMENT` resets the hand to its resting state. - When in `MOVEMENT` mode, other modes like `SetMovement`, `Freeze`, and `SetSpeed` are rejected until switched back to `MOVEMENT`. ```