### Unity Addressables Setup Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md Steps to configure Addressables settings in Unity, including creating new settings and ignoring legacy bundles. ```Unity Window -> Asset Management -> Addressables -> Groups Create Addressables Settings Ignore ``` -------------------------------- ### Import TextMeshPro Resources Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md Imports essential resources for TextMeshPro, a text rendering solution in Unity. ```Unity Window -> TextMeshPro -> Import TMP Essential Resources ``` -------------------------------- ### Clone and Install Pyflux Source: https://github.com/pupil-labs/pupil-docs/blob/master/alpha-lab/nerfs/index.md Clones the pyflux repository from GitHub and installs it using pip. This involves navigating into the cloned directory and running the setup script. ```bash # Get gaze mapping repo git clone https://github.com/pupil-labs/pyflux.git pyflux cd pyflux pip install -e . ``` -------------------------------- ### Build Addressables Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md Builds the Addressables content using the default build script. ```Unity Build -> New Build -> Default Build Script ``` -------------------------------- ### Start/Stop Recording in Pupil Capture Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/getting-started/index.md This snippet describes the keyboard shortcut and button to start or stop a recording within the Pupil Capture software. It also mentions where the recordings are saved. ```Text Start/Stop recording: Press the `r` key on your keyboard or press the circular `R` button on the left hand side of the world window. ``` -------------------------------- ### Install APK via ADB Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md Installs the built Android application package (APK) onto the headset using the Android Debug Bridge (adb). ```Shell adb install ``` -------------------------------- ### Start Screen Marker Calibration Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/getting-started/index.md This snippet details how to initiate the screen marker calibration method in Pupil Capture by pressing the 'c' key. ```Text Click `c` on the world screen or press `c` on the keyboard to start calibrating. ``` -------------------------------- ### Checkout Neon Branch Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md This command checks out the 'neon' branch of the project repository, which is necessary for setting up the Quest 3 template. ```git git checkout neon ``` -------------------------------- ### Install Quest 3 APK using ADB Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md This command installs the 'neon-quest3.apk' file onto a connected Quest 3 headset using the Android Debug Bridge (ADB) tool. Ensure Developer Mode is enabled on the headset and ADB is set up correctly. ```Shell adb install neon-quest3.apk ``` -------------------------------- ### Checkout Pico Branch Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md This command switches the local repository to the 'neon-pico' branch. This branch is specifically configured for use with the PICO Unity Integration SDK, essential for Pico 4 development. ```Shell git checkout neon-pico ``` -------------------------------- ### Download Pupil Core Bundles Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/getting-started/index.md This component is used to display download links for Pupil Core bundles. It fetches the latest release information from a GitHub API. ```Vue ``` -------------------------------- ### Setup Pupil Cloud and GPT Annotation Tool Source: https://github.com/pupil-labs/pupil-docs/blob/master/alpha-lab/event-automation-gpt/index.md This snippet outlines the initial setup for automating event annotations using Pupil Cloud and GPT. It requires uploading recordings to Pupil Cloud, obtaining a developer token, and having an OpenAI API key. The tool provides a GUI for selecting recording details and entering custom prompts. ```Shell pip install pupil-labs-api openai # Follow setup guide in Github repository for detailed instructions. ``` -------------------------------- ### Install Dependencies Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/developer/network-api/index.md Installs the necessary Python libraries, zmq and msgpack, required for interacting with the Pupil Core Network API. ```Shell pip install zmq msgpack==0.5.6 ``` -------------------------------- ### Switch Build Platform to Android Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md Changes the target build platform to Android, which is required for Quest 3 deployment. ```Unity Build Settings -> Android ``` -------------------------------- ### Add MRTK NeonXR Variant Prefab Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md Adds the 'MRTK NeonXR Variant' prefab to a Unity scene for standalone execution of sample scenes like calibration. ```Unity Assets -> PupilLabs -> MRTK -> Prefabs -> MRTK NeonXR Variant ``` -------------------------------- ### Clone MRTK3 Repository Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md This command clones the specified MRTK3 repository from GitHub using SSH. This is the first step to obtaining the template project for local development. ```Shell git clone git@github.com:pupil-labs/MixedRealityToolkit-Unity.git ``` -------------------------------- ### Import Addressables Groups Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md Imports Addressables groups into the Unity project, specifically adding the 'NeonXR Group'. ```Unity Pupil Labs -> Addressables -> Import Groups ``` -------------------------------- ### Export Data in Pupil Player Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/getting-started/index.md This snippet explains how to export recorded data and videos using Pupil Player. It mentions the keyboard shortcut and button to initiate the export process and where the exported files are saved. ```Text You can export videos and data by pressing `e` on your keyboard or the down arrow button in the Pupil Player window. ``` -------------------------------- ### Real-Time API Usage Example Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/data-collection/ecosystem-overview/index.md This example demonstrates how to track experiment progress using events via the Pupil Labs Real-Time API. It requires the Companion device and the API-using computer to be on the same WiFi network. ```Python from pupil_realtime import Realtime # Connect to the Pupil headset realtime = Realtime(address='udp://127.0.0.1:50020') # Subscribe to data streams realtime.subscribe('gaze') realtime.subscribe('events') # Process incoming data for topic, message in realtime.recv_from_all(): if topic.split('/')[-1] == 'gaze': print(f"Gaze data: {message}") elif topic.split('/')[-1] == 'events': print(f"Event data: {message}") # Example of sending an event # realtime.send_event('experiment_start', timestamp=realtime.get_time()) # realtime.send_event('stimulus_onset', label='image_1', timestamp=realtime.get_time()) # realtime.send_event('experiment_end', timestamp=realtime.get_time()) ``` -------------------------------- ### Install Dependencies and Nerfstudio Source: https://github.com/pupil-labs/pupil-docs/blob/master/alpha-lab/nerfs/index.md Installs necessary dependencies and the Nerfstudio library using pip and conda. It includes setting up a Conda environment, installing COLMAP, PyTorch with CUDA support, and the tiny-cuda-nn bindings. ```bash conda create --name {ENV_NAME} python=3.8 conda activate {ENV_NAME} conda install-c conda-forge colmap pip install -U pip setuptools pip install torch torchvision functorch --extra-index-url https://download.pytorch.org/whl/cu117 pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch pip install glfw pip install pyrr pip install trimesh pip install PyOpenGL pip install PyOpenGL_accelerate ``` -------------------------------- ### Clone and Install Nerfstudio Source: https://github.com/pupil-labs/pupil-docs/blob/master/alpha-lab/nerfs/index.md Clones the nerfstudio repository from GitHub at a specific version and installs it using pip. This involves checking out a particular commit hash. ```bash cd .. git clone https://github.com/nerfstudio-project/nerfstudio.git nerfstudio_git #Clone it at 0.1.16 cd nerfstudio_git git checkout 7ae2fce pip install -e . ``` -------------------------------- ### Install Pupil Labs Dynamic RIM Source: https://github.com/pupil-labs/pupil-docs/blob/master/alpha-lab/map-your-gaze-to-a-2d-screen/index.md This command installs the necessary Python libraries for the Pupil Labs dynamic reference image mapper. Ensure you have Python 3.7 or higher installed. ```bash pip install pupil-labs-dynamic-rim ``` -------------------------------- ### Control Recording with Pupil Remote (Python) Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/developer/network-api/index.md Demonstrates sending commands to Pupil Remote to start and stop recordings. It sends 'R' to start and 'r' to stop, printing the received responses. ```Python # start recording pupil_remote.send_string('R') print(pupil_remote.recv_string()) sleep(5) pupil_remote.send_string('r') print(pupil_remote.recv_string()) ``` -------------------------------- ### Vis Polyline Plugin Example - Pupil Player Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/software/pupil-player/index.md Shows an example of using the Vis Polyline plugin in conjunction with the Vis Circle plugin to visualize gaze position sequences over a single world frame. ```Python # Example usage of Vis Polyline and Vis Circle plugins (conceptual) # Load Vis Circle for current gaze point # plugin_manager.load_plugin('Vis Circle', color=(0, 0, 255, 100), fill=True, radius=5) # Load Vis Polyline to show gaze history # plugin_manager.load_plugin('Vis Polyline', color=(255, 0, 0, 255), line_thickness=2) ``` -------------------------------- ### Neon Player Workflow Example Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-player/index.md This snippet outlines a typical workflow for using Neon Player, including loading recordings, activating plugins like 'Vis Circle' and exporters, playback controls, setting trim marks, and exporting data and videos. ```text - Start **Neon Player** and load a recording. - Open a Plugin - From the `Plugin Manager` GUI menu (which can be found in the Sidebar), toggle any Plugin of your choice. The `Vis Circle` plugin is activated by default. - Playback - press the play button or `space` bar on your keyboard to view the video playback with visualization overlay, or drag the playhead in the seek bar to scrub through the dataset. - Set trim marks - drag the green rounded rectangle at the beginning and end of the seekbar to set the trim marks. This will set the start and end frame for the exporter and for other plugins. - Export Video & Raw Data - From the `Plugin Manager` view, load the `World Video Exporter` plugin and the `Raw Data Exporter` plugin. Press `e` on your keyboard or the ⬇ `download icon` button in the left hand side of the window to start the export. - View the exported data in the `exports` directory. ``` -------------------------------- ### Disable Google ARCore Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/MRTK3-template-project/index.md Disables the Google ARCore plug-in in Unity's XR Plug-in Management settings to avoid conflicts. ```Unity Edit -> Project Settings -> XR Plug-in Management -> Google ARCore ``` -------------------------------- ### Python Plugin Initialization and Session Settings Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-player/plugin-api/index.md Demonstrates how to initialize a custom plugin, manage user preferences via keyword arguments, and persist settings across application sessions using the get_init_dict method. It highlights the importance of calling the superclass constructor and the expected format for session data. ```Python from plugin import Plugin class MyCustomPlugin(Plugin): def __init__(self, g_pool, my_custom_setting=5): self._my_custom_setting = my_custom_setting def get_init_dict(self): return {"my_custom_setting": self._my_custom_setting} # return {} # on next launch, recover plugin with default settings # raise NotImplementedError # on next launch, do not recover plugin ``` -------------------------------- ### Download Pupil Geometry CAD Files Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/diy/index.md Access and download the CAD files for Pupil eye tracker camera mounts. These files can be modified to suit custom camera setups or other sensors, following the provided license. ```Git git clone https://github.com/pupil-labs/pupil-geometry ``` -------------------------------- ### Control Pupil Service via Notifications Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/developer/network-api/index.md Provides a comprehensive example of controlling Pupil Service by sending notifications. It demonstrates starting eye processes, setting calibration methods, and stopping the service, all using a helper function for sending and receiving messages. ```python import zmq, msgpack, time # create a zmq REQ socket to talk to Pupil Service ctx = zmq.Context() pupil_remote = ctx.socket(zmq.REQ) pupil_remote.connect('tcp://localhost:50020') # convenience function def send_recv_notification(n): pupil_remote.send_string(f"notify.{n['subject']}", flags=zmq.SNDMORE) pupil_remote.send(msgpack.dumps(n)) return pupil_remote.recv_string() # set start eye windows n = {'subject':'eye_process.should_start.0','eye_id': 0, 'args':{}} print(send_recv_notification(n)) n = {'subject':'eye_process.should_start.1','eye_id':1, 'args':{}} print(send_recv_notification(n)) time.sleep(2) # set calibration method to hmd calibration n = {'subject':'start_plugin','name':'HMD_Calibration', 'args':{}} print(send_recv_notification(n)) time.sleep(2) # close Pupil Service n = {'subject':'service_process.should_stop'} print(send_recv_notification(n)) ``` -------------------------------- ### Use MRTK3 Template Project for Neon XR Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-xr/index.md Utilize the MRTK3 template project for a streamlined development experience with Neon XR. This project leverages MRTK3 as its foundation and includes demo scenes to help you get started quickly with building XR applications that incorporate eye-tracking. ```Unity // Example usage within the MRTK3 template project // This would typically involve interacting with MRTK3 UI elements using gaze input provided by Neon XR. ``` -------------------------------- ### Setup Nerfstudio Dependencies Source: https://github.com/pupil-labs/pupil-docs/blob/master/alpha-lab/nerfs/index.md This snippet outlines the core dependencies required for Nerfstudio, a package used for creating and visualizing Neural Radiance Fields (NeRFs). It highlights the reliance on PyTorch for deep learning, PyQt for the graphical user interface, and ZeroMQ, ThreeJS, and ReactJS for real-time rendering capabilities. ```Python import torch import PyQt import ZeroMQ import ThreeJS import ReactJS ``` -------------------------------- ### Install Dependencies Source: https://github.com/pupil-labs/pupil-docs/blob/master/alpha-lab/undistort/index.md Installs the necessary Python libraries, OpenCV and NumPy, for image processing and numerical operations required for video and gaze data undistortion. ```bash pip install opencv-python numpy ``` -------------------------------- ### Python Plugin Initialization and Session Settings Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/developer/plugin-api/index.md Demonstrates how to initialize a Pupil plugin and manage persistent session settings using `__init__` and `get_init_dict`. The `__init__` method accepts custom settings with default values, and `get_init_dict` returns a dictionary to be saved for restoring the plugin's state on subsequent launches. ```Python from plugin import Plugin class MyCustomPlugin(Plugin): def __init__(self, g_pool, my_custom_setting=5): self._my_custom_setting = my_custom_setting def get_init_dict(self): return {"my_custom_setting": self._my_custom_setting} # return {} # on next launch, recover plugin with default settings # raise NotImplementedError # on next launch, do not recover plugin ``` -------------------------------- ### Basic Plugin Structure in Python Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/neon-player/plugin-api/index.md Demonstrates the fundamental structure for creating a new plugin by inheriting from the `Plugin` class. This serves as a starting point for developing custom functionalities within Neon Player. ```Python from plugin import Plugin class MyCustomPlugin(Plugin): pass ``` -------------------------------- ### Install Pupil Labs Plugin in PsychoPy Builder Source: https://github.com/pupil-labs/pupil-docs/blob/master/neon/data-collection/psychopy/index.md Instructions for installing the Pupil Labs plugin within the PsychoPy Builder interface. This plugin enables the use of the Neon eyetracker in PsychoPy experiments. ```English 1. Open PsychoPy Builder 2. Click on the "Tools" menu and select "Plugins/package Manager" 3. Find "Pupil Labs" in the plugin list and click "Install" 4. Close all PsychoPy windows and re-open Builder ``` -------------------------------- ### Post-Hoc Object Detection with pl-neon-recording (Python) Source: https://github.com/pupil-labs/pupil-docs/blob/master/alpha-lab/map-onto-anything/index.md This Python script demonstrates using the pl-neon-recording library for post-hoc analysis with an object detection model (YOLO). It includes setup instructions and highlights specific lines for easier navigation. ```py # /// script # requires-python = ">=3.10" # dependencies = [ # "opencv-python", # "pupil-labs-video", # "pupil-labs-neon-recording", # "ultralytics", # ] # /// from pathlib import Path import cv2 import pupil_labs.neon_recording as nr from tqdm import tqdm ``` -------------------------------- ### Python Implementations for Network Time Sync Source: https://github.com/pupil-labs/pupil-docs/blob/master/core/software/pupil-capture/index.md Provides example Python implementations for the Pupil Time Sync protocol, allowing multiple nodes to find a common clock master and synchronize their time. These examples are found in the pupil-helpers repository. ```Python See the [pupil-helpers](https://github.com/pupil-labs/pupil-helpers/tree/62ea54001fd051528bf24537bbd4f5f89e3391e8/network_time_sync) for example Python implementations. ```