### Install Neuracore with Example Support

Source: https://github.com/neuracoreai/neuracore/blob/main/README.md

Install Neuracore with the 'examples' extra to run provided examples. This includes dependencies needed for example scripts.

```bash
pip install neuracore[examples]
```

--------------------------------

### Setup Neuracore Development Environment

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/contribution_guide.md

Steps to clone the Neuracore repository and install the necessary dependencies for development, including optional ML packages.

```bash
git clone https://github.com/neuracoreai/neuracore
cd neuracore
pip install -e .[dev,ml]
```

--------------------------------

### Install BiGym Environment

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Clones the BiGym repository and installs it locally. This is a prerequisite for running BiGym examples.

```bash
git clone https://github.com/chernyadev/bigym.git
cd bigym
pip install .
```

--------------------------------

### Install Neuracore with Examples

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Installs Neuracore and its example dependencies using conda and pip. Requires Git LFS.

```bash
conda create -n neuracore_examples python=3.10
conda activate neuracore_examples
pip install "neuracore[examples]"
```

--------------------------------

### Neuracore Basic Usage Example

Source: https://github.com/neuracoreai/neuracore/blob/main/README.md

Demonstrates basic Neuracore functionalities including login, robot connection, dataset creation, data logging (joint positions, RGB images), and starting a cloud training run. Ensure you have a neuracore.com account and replace placeholder paths.

```python
import neuracore as nc # pip install neuracore
import time

# ensure you have an account at neuracore.com
nc.login()

# Connect to a robot with URDF
nc.connect_robot(
    robot_name="MyRobot", 
    urdf_path="/path/to/robot.urdf",
)

# Create a dataset for recording
nc.create_dataset(
    name="My Robot Dataset",
    description="Example dataset with multiple data types"
)

# Recording and streaming data
nc.start_recording()
t = time.time()
nc.log_joint_positions(positions={'joint1': 0.5, 'joint2': -0.3}, timestamp=t)
nc.log_rgb(name="top_camera", rgb=image_array, timestamp=t)
# Stop recording, the dataset is automatically uploaded to the cloud
nc.stop_recording()

# Kick off cloud training
job_data = nc.start_training_run(
    name="MyTrainingJob",
    dataset_name="My Robot Dataset",
    algorithm_name="diffusion_policy",
    num_gpus=5,
    frequency=50,
    ...
)

# Load a trained model locally
policy = nc.policy(
    train_run_name="MyTrainingJob",
    ...
)

# Get model inputs
nc.log_joint_positions(positions={'joint1': 0.5, 'joint2': -0.3})
nc.log_rgb(name="top_camera", rgb=image_array)
# Model Inference
predictions = policy.predict(timeout=5)
```

--------------------------------

### Build Docker Image and Run Example

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/ros_example/README.md

Builds the Docker image for the ROS2 example and runs it in interactive mode. This sets up the environment for Neuracore ROS2 integration.

```bash
docker build --file examples/ros_example/Dockerfile   -t ros_example:latest .
neuracore login
docker run -it --rm -v  ~/.neuracore:/root/.neuracore --network host ros_example:latest
```

--------------------------------

### Algorithm File Organization Example

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/contribution_guide.md

Illustrates the recommended directory structure for a basic custom algorithm, including the main model file, helper modules, and requirements.

```bash
algorithms/my_algorithm/
├── __init__.py               # Empty or importing from my_algorithm.py
├── my_algorithm.py           # Main model class definition
├── modules.py                # Helper modules and components
└── requirements.txt          # Optional: additional dependencies
```

--------------------------------

### Install Neuracore with Import Support

Source: https://github.com/neuracoreai/neuracore/blob/main/README.md

Install Neuracore with the 'import' extra for bulk dataset importing. This is useful for managing large datasets.

```bash
pip install neuracore[import]
```

--------------------------------

### Install Dependencies

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/getting_started_with_neuracore.ipynb

Installs Neuracore and Bigym, along with Rust for safetensors compilation. This may take a few minutes.

```python
# Installing neuracore
!pip install "neuracore[ml,mjcf]==11.0.0" > /dev/null 2>&1

# Installing rust --- Why? Safetensors 0.3.3 (required by Bigym) has no pre-built wheel on Colab, therefore it builds from source. That needs Rust.
!curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y -q > /dev/null 2>&1
import os
os.environ["PATH"] = os.path.expanduser("~/.cargo/bin") + os.pathsep + os.environ.get("PATH", "")

# Installing bigym --- Commit at the time of creation: 79420b0abad6fa7d7dfd98569cb485f73a2c8e3b
!git clone https://github.com/NeuracoreAI/bigym.git /content/bigym > /dev/null 2>&1
!pip install -e /content/bigym > /dev/null 2>&1
```

--------------------------------

### Start Local Training via Command Line

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/getting_started_with_neuracore.ipynb

Execute a local training script directly from the command line. This method is suitable for training on your own GPU.

```bash
python -m neuracore.ml.train algorithm=diffusion_policy dataset_name="Getting started Bigym Example"
```

--------------------------------

### Byte Unit Examples for Storage and Bandwidth Limits

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Demonstrates how to specify storage and bandwidth limits using raw bytes or suffixed units like GB and MB.

```bash
--storage-limit 500000000
--storage-limit 2gb
--bandwidth-limit 50mb
```

--------------------------------

### Record ALOHA Robot Demonstrations

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Runs the ALOHA data collection example to record robot demonstrations. Ensure MUJOCO_GL is set to 'egl' if visualizations are glitchy.

```bash
python example_data_collection_vx300s.py --record --num_episodes=1
```

--------------------------------

### Auto-Start Daemon from Script

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Example of how to use the Neuracore library within a Python script. The data daemon will automatically start in the background if it's not already running when a function like `start_recording` is called.

```python
import neuracore as nc

def main():
    nc.login()

    # The daemon starts automatically when needed
    nc.start_recording()
    # ...
    nc.stop_recording()
```

--------------------------------

### Install Neuracore with ML Support

Source: https://github.com/neuracoreai/neuracore/blob/main/README.md

Install Neuracore with the 'ml' extra for training and machine learning development. This includes additional dependencies for ML tasks.

```bash
pip install neuracore[ml]
```

--------------------------------

### Pending Changelog Summary Example

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/contribution_guide.md

An example of how to write a human-readable summary for significant changes in the 'pending-changelog.md' file, which will be used in GitHub release notes.

```markdown
## Summary

This release adds multi-GPU training and improves streaming performance by 40%.
```

--------------------------------

### Install Neuracore Basic Package

Source: https://github.com/neuracoreai/neuracore/blob/main/README.md

Install the core Neuracore package for data logging and visualization. Ensure you have Python 3.10+.

```bash
pip install neuracore
```

--------------------------------

### Record BiGym Robot Demonstrations

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Runs the BiGym data collection example to record robot demonstrations. Ensure MUJOCO_GL is set to 'egl' if visualizations are glitchy.

```bash
python example_data_collection_bigym.py --record --num_episodes=1
```

--------------------------------

### Install ffmpeg for Video Encoding

Source: https://github.com/neuracoreai/neuracore/blob/main/README.md

Install the ffmpeg binary on Debian/Ubuntu systems for faster video processing. This is recommended for recording and playback.

```bash
sudo apt-get update && sudo apt-get install -y ffmpeg
```

--------------------------------

### Training with YAML Configuration File

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/training.md

Runs training using arguments defined in a YAML configuration file, allowing for organized and reusable training setups.

```yaml
# @package _global_

training_name: my_experiment
training_name_auto_increment: true

dataset_name: my_dataset
algorithm_name: diffusion_policy

epochs: 100
batch_size: auto
frequency: 10
validation_split: 0.2

input_data_types:
  - JOINT_POSITIONS
  - RGB_IMAGES

output_data_types:
  - JOINT_TARGET_POSITIONS

algorithm:
  lr: 0.0005
  hidden_dim: 1024
```

```bash
python -m neuracore.ml.train user=my_experiment
```

--------------------------------

### Install Neuracore Daemon

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Install the Neuracore package using pip. This command installs the package in editable mode. It is recommended to also install ffmpeg for optimal video performance.

```bash
pip install -e .
sudo apt-get update && sudo apt-get install -y ffmpeg
```

--------------------------------

### Troubleshoot Daemon Startup Issues

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Provides commands to troubleshoot common daemon startup problems. It includes checking status, stopping, and relaunching the daemon, as well as listing and getting profile information.

```bash
neuracore data-daemon status
neuracore data-daemon stop
neuracore data-daemon launch
```

```bash
neuracore data-daemon profile list
```

```bash
neuracore data-daemon profile get
```

```bash
neuracore data-daemon profile get <name>
```

--------------------------------

### Start Cloud Training Job via Python

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/getting_started_with_neuracore.ipynb

Initiate a cloud training job using the `nc.start_training_run` function. Specify job name, dataset, algorithm, and resource allocation.

```python
job_data = nc.start_training_run(
  name="MyTrainingJob",
  dataset_name="Getting started Bigym Example",
  algorithm_name="diffusion_policy",
  num_gpus=1,
  frequency=20,
  ...
)
```

--------------------------------

### Run ROS2 Example for Asynchronous Predictions

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/ros_example/README.md

Runs the ROS2 example to demonstrate asynchronous predictions from a trained Neuracore model. Ensure a trained policy is available in Neuracore before executing.

```bash
docker run -it --rm -v  ~/.neuracore:/root/.neuracore --network host ros_example:latest ros2 launch ros_example prediction.py training_run_name:=MY_RUN_NAME
```

--------------------------------

### Install ML Dependencies for Local Deployment

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Installs additional packages required for local model deployment with Neuracore.

```bash
pip install "neuracore[ml]"
```

--------------------------------

### Run Server Model Deployment

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Starts a model endpoint on the Neuracore servers and visualizes its performance. Requires a collected dataset, a finished training run, and an active server endpoint.

```bash
python example_server_endpoint.py
```

--------------------------------

### Launch Neuracore Daemon with Default Profile

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Launches the Neuracore Data Daemon using the default profile. This is the simplest way to start the daemon.

```bash
neuracore data-daemon launch
```

--------------------------------

### Example CrossEmbodimentDescription Structure

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/model_construction.md

This is a basic example of the CrossEmbodimentDescription structure, mapping robot names to their respective embodiment details.

```python
{
    "franka": {
        ...
    },
    "ur5": {
        ...
    }
}
```

--------------------------------

### Basic Local Training with Diffusion Policy

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/training.md

Starts a local training job using the Diffusion Policy algorithm. It's recommended to provide a `training_name` for better experiment tracking.

```bash
python -m neuracore.ml.train algorithm=diffusion_policy dataset_name="my_dataset" training_name="my_experiment"
```

--------------------------------

### Fix Setuptools Version for ML Dependencies

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/commandline.md

If you encounter 'No module named 'pkg_resources'' after installing ML dependencies, install an older version of setuptools.

```bash
pip install "setuptools<82"
```

--------------------------------

### Check FFmpeg Installation

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Checks if FFmpeg is installed and available in the system's PATH. This is used to determine if the daemon will use the FFmpeg backend for video encoding.

```bash
ffmpeg -version
```

--------------------------------

### Update a Data Daemon Profile

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Example of updating a specific named profile with new storage and offline settings.

```bash
neuracore data-daemon profile update laptop --storage-limit 2gb --offline
```

--------------------------------

### Login and Select Organization

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/commandline.md

Log in to Neuracore and select your organization. This is a one-time setup per machine or when switching organizations.

```bash
neuracore login
neuracore select-org --org-name "<ORG_NAME>"
```

--------------------------------

### Run Daemon as a Separate Process

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Illustrates how the `neuracore data-daemon launch` command internally starts the daemon process by executing the runner entry point script. This is a high-level overview of the daemon's execution mechanism.

```text
python -m neuracore.data_daemon.runner_entry
```

--------------------------------

### Start Cloud Training Job with Python

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/training.md

Initiates a cloud training job using the Neuracore Python API. Ensure you have logged in and specified dataset and robot specifications.

```python
import neuracore as nc
from neuracore_types import DataType, RobotDataSpec

nc.login()
dataset = nc.get_dataset("My Dataset")
robot_id = dataset.robot_ids[0]
full_spec = dataset.get_full_embodiment_description(robot_id)

input_robot_data_spec: RobotDataSpec = {
    robot_id: {
        DataType.JOINT_POSITIONS: full_spec.get(DataType.JOINT_POSITIONS, []),
        DataType.RGB_IMAGES: full_spec.get(DataType.RGB_IMAGES, []),
    }
}
output_robot_data_spec: RobotDataSpec = {
    robot_id: {
        DataType.JOINT_TARGET_POSITIONS: full_spec.get(DataType.JOINT_TARGET_POSITIONS, []),
    }
}

job_data = nc.start_training_run(
    name="MyTrainingJob",
    dataset_name="My Dataset",
    algorithm_name="diffusion_policy",
    frequency=50,
    num_gpus=1,
    gpu_type="NVIDIA_TESLA_V100",
    input_robot_data_spec=input_robot_data_spec,
    output_robot_data_spec=output_robot_data_spec,
    algorithm_config={"batch_size": 32, "epochs": 100, "output_prediction_horizon": 50},
)
# Use name_auto_increment=True to auto-use MyTrainingJob_1, MyTrainingJob_2, etc. if the name already exists.
```

--------------------------------

### Get a Data Daemon Profile Description

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Command to retrieve and display the configuration details of a specific data daemon profile.

```bash
neuracore data-daemon profile get high-bandwidth
```

--------------------------------

### Setting Runtime Path Environment Variables

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Provides examples for setting environment variables that control the location of daemon runtime artifacts like the PID file, database, and recordings directory. Recommended for containerized or development environments.

```bash
export NEURACORE_DAEMON_DB_PATH=/workspaces/neuracore/data_daemon_state.db
export NEURACORE_DAEMON_RECORDINGS_ROOT=/workspaces/neuracore/recordings
```

--------------------------------

### Conventional Commits Example

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/contribution_guide.md

Demonstrates the required commit message format for contributing to the Neuracore project, using prefixes like 'feat', 'fix', or 'chore'.

```bash
<prefix>: <description>
Valid prefixes: feat, fix, chore, docs, ci, test, refactor, style, perf
```

--------------------------------

### Defining Input and Output CrossEmbodiment Descriptions

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/model_construction.md

This example demonstrates the explicit indexed structure for input and output specifications within a CrossEmbodimentDescription. It highlights how different robots (Franka and UR5) contribute to a shared tensor space, with empty index slots being padded.

```python
input_cross_embodiment_description: CrossEmbodimentDescription = {
    "franka": {
        JOINT_POSITIONS: {
            0: "f_base",
            1: "f_joint_1",
            2: "f_joint_2",
            3: "f_joint_3",
            4: "f_joint_4",
            5: "f_joint_5",
            6: "f_wrist",
        },
        RGB_IMAGES: {
            0: "f_wrist_camera",
            1: "f_overhead_camera",
        },
    },
    "ur5": {
        JOINT_POSITIONS: {
            0: "ur5_base",
            1: "ur5_1",
            2: "ur5_2",
            3: "ur5_3",
            4: "ur5_4",
            #  Missing Index 5 to make sure wrist is at index 6
            6: "ur5_wrist",
        },
        RGB_IMAGES: {
            # Index 0 is missing as there is no wrist camera
            1: "ur5_overhead_camera", 
        },
    },
}

output_cross_embodiment_description: CrossEmbodimentDescription = input_cross_embodiment_description
```

--------------------------------

### Example EmbodimentDescription for Joints and Images

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/model_construction.md

Illustrates how to define the data structure for a single robot, specifying indices and names for joints and camera inputs. This ensures consistent tensor positions and semantic meaning for features across different robots.

```python
{
    JOINTS: {
        0: "arm_joint_1",
        1: "arm_joint_2",
        2: "arm_joint_3",
        3: "arm_joint_4",
        4: "arm_wrist_joint",
    },
    RGB_IMAGES: {
        0: "wrist_camera",
        1: "overhead_camera"
    }
}
```

--------------------------------

### Basic Data Logging

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/tutorial.md

Log various data types including joint positions, velocities, camera images, language instructions, and custom data. Ensure you create a dataset before starting recording.

```python
import time

# Create a dataset for recording
nc.create_dataset(
    name="My Robot Dataset",
    description="Example dataset with multiple data types"
)

# Start recording
nc.start_recording()

# Log various data types with timestamps
t = time.time()
nc.log_joint_positions(positions={'joint1': 0.5, 'joint2': -0.3}, timestamp=t)
nc.log_joint_velocities(velocities={'joint1': 0.1, 'joint2': -0.05}, timestamp=t)
nc.log_joint_target_positions(target_positions={'joint1': 0.6, 'joint2': -0.2}, timestamp=t)

# Log camera data
nc.log_rgb(name="top_camera", rgb=image_array, timestamp=t)

# Log language instructions
nc.log_language(
    name="instruction",
    language="Pick up the red cube",
    timestamp=t,
)

# Log custom data
custom_sensor_data = np.array([1.2, 3.4, 5.6])
nc.log_custom_1d("force_sensor", custom_sensor_data, timestamp=t)

# Stop recording
nc.stop_recording()
```

--------------------------------

### Launch Cloud Training with Detailed Configuration

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/getting_started_with_neuracore.ipynb

This snippet demonstrates launching a cloud training job with specific input and output descriptions, algorithm configurations, and resource settings. It utilizes free Neuracore credits for a limited duration.

```python
from neuracore_types import CrossEmbodimentDescription

dataset = nc.get_dataset(DATASET_NAME)
robot_id = dataset.robot_ids[0]

# Set policy input space order as indexed JOINT_POSITIONS and RGB_IMAGES names
input_cross_embodiment_description: CrossEmbodimentDescription = {
    robot_id: {
        DataType.JOINT_POSITIONS: {
            index: name for index, name in enumerate(JOINT_NAMES)
        },
        DataType.RGB_IMAGES: {0: "head"},
    }
}

# Set policy output space order as indexed JOINT_TARGET_POSITIONS names
output_cross_embodiment_description: CrossEmbodimentDescription = {
    robot_id: {
        DataType.JOINT_TARGET_POSITIONS: {
            index: name for index, name in enumerate(JOINT_ACTUATORS)
        },
    }
}

# This launches a cloud training job which you can track on the UI web dashboard
JOBNAME = "Bigym example"
nc.start_training_run(
    name=JOBNAME,
    dataset_name=DATASET_NAME,
    algorithm_name="CNNMLP",
    frequency=FREQUENCY,
    num_gpus=1,
    gpu_type="NVIDIA_TESLA_T4",
    input_cross_embodiment_description=input_cross_embodiment_description,
    output_cross_embodiment_description=output_cross_embodiment_description,
    algorithm_config={"batch_size": "auto", "epochs": 5, "output_prediction_horizon": 10}
)

print(f'Training job "{JOBNAME}" launched')
```

--------------------------------

### Create and Use a Named Profile for the Daemon

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Demonstrates how to create a named profile, update its settings (storage limit, bandwidth limit, storage path, number of threads), and then launch the daemon using that profile. Profile names are positional arguments.

```bash
neuracore data-daemon profile create recording
neuracore data-daemon profile update recording --storage-limit 2gb --bandwidth-limit 50mb --storage-path /data/records --num-threads 4
neuracore data-daemon launch --profile recording
```

--------------------------------

### Command-line Training with Modalities

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/training.md

Initiates training with specified dataset, input/output modalities, and experiment name directly via the command line.

```bash
python -m neuracore.ml.train algorithm=pi0 dataset_name="my_multimodal_dataset" input_cross_embodiment_description={"my_robot": {"JOINT_POSITIONS": ["joint_1", "joint_2", ...], "RGB_IMAGES": ["wrist_camera"], "LANGUAGE": ["task_instruction"]}} output_cross_embodiment_description={"my_robot": {"JOINT_TARGET_POSITIONS": ["joint_1", "joint_2", ...]}} training_name="my_experiment"
```

--------------------------------

### Launch Local Policy Server

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/commandline.md

Launch a local policy server for inference validation. Requires input and output embodiment descriptions, job ID, and organization ID.

```bash
neuracore launch-server \
  --input_embodiment_description '{"RGB_IMAGES": {"0": "front_cam"}}' \
  --output_embodiment_description '{"JOINT_TARGET_POSITIONS": {"0": "arm"}}' \
  --job_id <RUN_ID> \
  --org_id <ORG_ID> \
  --port 8080
```

--------------------------------

### Language Instructions as Bytes with Source Mapping

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/dataset_importer.md

Configure multiple language instructions using BYTES format, specifying source names for each instruction.

```yaml
LANGUAGE:
  format:
    language_type: BYTES
  mapping:
    - name: instruction # name in neuracore
      source_name: language_instruction # name in input dataset
    - name: instruction2
      source_name: language_instruction_2
    - name: instruction3
      source_name: language_instruction_3
```

--------------------------------

### Import Packages

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/getting_started_with_neuracore.ipynb

Imports necessary libraries for Neuracore and Bigym. Ensure the runtime session has been restarted after dependency installation.

```python
# Headless OpenGL for Colab (no display server)
os.environ["MUJOCO_GL"] = "egl"

from typing import cast, Optional
import zipfile
import gdown
import time
from pathlib import Path

import numpy as np
import neuracore as nc
import bigym
from demonstrations.demo import Demo
from demonstrations.demo_store import DemoStore
from demonstrations.utils import Metadata
```

--------------------------------

### Custom 1D Data Arrays

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/dataset_importer.md

Configure custom 1D data arrays by specifying start and end indices for each named array.

```yaml
CUSTOM_1D:
  source: observation.state
  mapping:
    - name: custom_1d
      index_range:
        start: 0
        end: 10
    - name: sensor_readings
      index_range:
        start: 10
        end: 20
```

--------------------------------

### Configuring Storage Path and Threads via Environment Variables

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Illustrates setting environment variables for the storage path and number of worker threads. This is useful for customizing daemon behavior before launching.

```bash
export NCD_PATH_TO_STORE_RECORD=/mnt/data/records
export NCD_NUM_THREADS=4
neuracore data-daemon launch --background
```

--------------------------------

### Get Supported Output Data Types for NeuracoreModel

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/contribution_guide.md

Returns the set of data types that the NeuracoreModel produces as output. It specifically supports JOINT_TARGET_POSITIONS.

```python
    @staticmethod
    def get_supported_output_data_types() -> set[DataType]:
        """Return the data types supported by the model.

        Returns:
            set[DataType]: Set of supported output data types
        """
        return {DataType.JOINT_TARGET_POSITIONS}
```

--------------------------------

### Monitoring Training with TensorBoard

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/training.md

Launches TensorBoard to monitor training progress using the specified training run name.

```bash
neuracore training monitor --training-name my_experiment
```

```bash
neuracore training monitor
```

--------------------------------

### Launch Neuracore Data Daemon

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Launches the data daemon. Use `--profile` to specify a configuration profile and `--background` to run it in the background.

```bash
neuracore data-daemon launch [--profile <name>] [--background]
```

```bash
neuracore data-daemon launch
```

```bash
neuracore data-daemon launch --profile laptop
```

```bash
neuracore data-daemon launch --profile laptop --background
```

--------------------------------

### Monitor Local Training Runs with TensorBoard

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/commandline.md

Launch TensorBoard to monitor local training runs. This command requires the 'tensorboard' executable to be in your PATH.

```bash
neuracore training monitor --root <DIR> --port <PORT> --host <HOST> --bind-all
```

--------------------------------

### Login and Authentication

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/commandline.md

If authentication fails, re-run the login command and verify the configuration file exists and contains an API key.

```bash
neuracore login
```

--------------------------------

### Get Supported Input Data Types for NeuracoreModel

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/contribution_guide.md

Returns the set of data types that the NeuracoreModel supports as input during inference or training. Currently, it only supports JOINT_POSITIONS.

```python
    @staticmethod
    def get_supported_input_data_types() -> set[DataType]:
        """Return the data types supported by the model.

        Returns:
            set[DataType]: Set of supported input data types
        """
        return {DataType.JOINT_POSITIONS}
```

--------------------------------

### Run Neuracore Dataset Importer CLI

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/dataset_importer.md

Use the command-line interface to import datasets. Specify configuration, dataset directory, and robot description files.

```bash
neuracore importer import \
    --dataset-config path/to/config.yaml \
    --dataset-dir path/to/dataset \
    --robot-dir path/to/robot/description/files \
    [OPTIONS]
```

--------------------------------

### Connect to Local Policy Server

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/tutorial.md

This code demonstrates how to connect to a local policy server for model inference. Ensure you provide the correct training run name and embodiment descriptions.

```python
# Connect to a local policy server
policy = nc.policy_local_server(
    train_run_name="MyTrainingJob",
    input_embodiment_description=INPUT_EMBODIMENT_DESCRIPTION,
    output_embodiment_description=OUTPUT_EMBODIMENT_DESCRIPTION)

```

--------------------------------

### Inspect Training Run Details

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/commandline.md

Inspect the details of a specific training run. Use the --config flag to view algorithm configuration or --json for raw JSON output.

```bash
# Cloud (default)
neuracore training inspect --training-name <RUN_NAME_OR_ID>
# Add --config to print algorithm config, or --json for raw JSON.

# Local
neuracore training inspect --local --training-name <RUN_NAME>
```

--------------------------------

### Launch Training Job via Python API

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Launches a training job using the Neuracore Python API. Requires a pre-collected dataset.

```bash
python example_launch_training.py \
   --name 'My Training Job' \
   --algorithm_name 'CNNMLP' \
   --dataset_name 'Example Dataset'
```

--------------------------------

### Download Bigym Demonstrations Subset

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/getting_started_with_neuracore.ipynb

Downloads a subset of Bigym demonstration data specifically for the ReachTarget environment. This is used for quick testing and demonstration purposes in the notebook.

```python
#@markdown ## **Download Bigym demonstrations**
#@markdown This cell only downloads the **Bigym demonstration data** for the **ReachTarget** Bigym environment.
#@markdown
#@markdown We use a **small subset** of the original Bigym demos (ReachTarget env only) for the sake of running this Notebook quickly.

demo_store = DemoStore()

# Override cache path so we can use e.g. Google Drive. Mount Drive first if using /content/drive/MyDrive/. 
BIGYM_DEMO_BASE_PATH = Path("/content/bigym_demos")
SUBSET_DEMO_FILENAME = "demonstrations_subset_v2_v0.9.0"
demo_store._cache_path = BIGYM_DEMO_BASE_PATH / SUBSET_DEMO_FILENAME

```

--------------------------------

### Manage Neuracore Daemon Profiles

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Provides commands for managing named profiles for the Neuracore Data Daemon. This includes creating, updating, getting, listing, and deleting profiles. Note that profile names are positional arguments.

```bash
neuracore data-daemon profile create <name>
neuracore data-daemon profile update [profile_name] [options...]
neuracore data-daemon profile get [profile_name]
neuracore data-daemon profile list
neuracore data-daemon profile delete <name>
```

--------------------------------

### Parallel Gripper Target Open Amounts Configuration

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/dataset_importer.md

Configure target gripper open amounts for robot actions, with options to invert and normalize the values.

```yaml
PARALLEL_GRIPPER_TARGET_OPEN_AMOUNTS:
  source: action.gripper_target
  format:
    invert_gripper_amount: true # Convert from close amount to open amount
    normalize: # Optional: normalize the data to be between 0 and 1
      min: -3.2
      max: 3.2
  mapping:
    - name: gripper_target
      index: 7
```

--------------------------------

### Language Instruction as String

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/dataset_importer.md

Configure a single language instruction using STRING format.

```yaml
LANGUAGE:
  source: language_instruction
  format:
    language_type: STRING
  mapping:
    - name: instruction
```

--------------------------------

### Local Policy Deployment and Inference in Bigym

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/getting_started_with_neuracore.ipynb

Loads a trained Neuracore model locally and runs it within the Bigym ReachTarget environment. Records rollouts to a dataset for dashboard inspection. Includes setup for Google Colab and downloading a pre-trained model if needed.

```python
# @markdown ### **Try it yourself: deploy your first policy locally**
# @markdown This cell shows how to load a trained Neuracore model and run it directly in the Bigym `ReachTarget` environment.
# @markdown **Run this cell** to test your policy, or load a publicly available pre-trained model with `USE_PRETRAINED_MODEL = True`.
# @markdown
# @markdown **After running:** the rollout will be recorded to Neuracore so you can inspect it in the dashboard in the newly created `DATASET_NAME+"_Inference"` Dataset.

from neuracore_types import EmbodimentDescription

# For Google Colab only: pre-warm the data collection daemon with a long timeout before recording
from neuracore.data_daemon.lifecycle.daemon_os_control import ensure_daemon_running
ensure_daemon_running(timeout_s=30)

# Input ordering must match what was used at training time
INPUT_EMBODIMENT_DESCRIPTION: EmbodimentDescription = {
    DataType.JOINT_POSITIONS: {
        index: name for index, name in enumerate(JOINT_NAMES)
    },
    DataType.RGB_IMAGES: {0: "head"},
}

# Output ordering must match what was used at training time
OUTPUT_EMBODIMENT_DESCRIPTION: EmbodimentDescription = {
    DataType.JOINT_TARGET_POSITIONS: {
        index: name for index, name in enumerate(JOINT_ACTUATORS)
    },
}

robot = nc.connect_robot(robot_name="Mujoco UnitreeH1 Example")

# Create a dataset for collecting episodes at inference time
nc.create_dataset(
    name=DATASET_NAME+"_Inference",
    description="Inference rollouts for the Getting started Bigym example",
)

USE_PRETRAINED_MODEL = True
if USE_PRETRAINED_MODEL:
    # Load our public model pre-trained on 10 ReachTarget demo episodes for 50 epochs, with CNNMLP algorithm (~80% success rate).
    MODEL_PATH = "/content/bigym_pretrained_model.zip"
    if not Path(MODEL_PATH).exists():
        gdown.download("https://drive.google.com/uc?id=1vyuwkjZ87YiPr6zgS11SwkPu3u3912pw", MODEL_PATH, quiet=False)

    policy = nc.policy(
        model_file=MODEL_PATH,
        input_embodiment_description=INPUT_EMBODIMENT_DESCRIPTION,
        output_embodiment_description=OUTPUT_EMBODIMENT_DESCRIPTION,
    )

else:
    policy = nc.policy(
        train_run_name=JOBNAME,
        input_embodiment_description=INPUT_EMBODIMENT_DESCRIPTION,
        output_embodiment_description=OUTPUT_EMBODIMENT_DESCRIPTION,
    )


NUM_EPISODES = 3
RECORD = True

success_count = 0
try:
    for episode_idx in range(NUM_EPISODES):
        success = run_episode(
            episode_idx=episode_idx,
            env=env,
            policy=policy,
            record=RECORD
        )

        if success:
            success_count += 1

        print(f"Inference success rate: {success_count}/{episode_idx + 1}")

except KeyboardInterrupt:
    print("\nInterrupted by user.")
    if RECORD:
        nc.cancel_recording()
finally:
    print(f"\nFinished running {NUM_EPISODES} episodes → {success_count} succeeded.")
    policy.disconnect()
```

--------------------------------

### Download and Cache Bigym Demos

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/getting_started_with_neuracore.ipynb

This snippet downloads the ReachTarget demo subset from Google Drive if it's not already cached. It handles directory creation, downloading, extraction, and caching status updates.

```python
if demo_store._cache_path.exists() and demo_store.cached:
    print("Bigym demos already cached on disk; skipping download.")
else:
    print("Downloading ReachTarget demo subset from Google Drive (one-time per session)...")
    demo_store._cache_path.parent.mkdir(parents=True, exist_ok=True)
    subset_zip_url = "https://drive.google.com/uc?id=1kLBOYRC489_fcMKCwFc4XOmBGIdkfsbV"
    subset_zip_path = BIGYM_DEMO_BASE_PATH / "demonstrations_subset_v2_v0.9.0.zip"
    gdown.download(subset_zip_url, str(subset_zip_path), quiet=True)
    with zipfile.ZipFile(subset_zip_path, "r") as zf:
        zf.extractall(demo_store._cache_path.parent)
    subset_zip_path.unlink(missing_ok=True)
    demo_store.cached = True
    print("ReachTarget demos ready.")
```

--------------------------------

### Run Release Workflow (Dry Run)

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/contribution_guide.md

Instructions to initiate the Neuracore release process using GitHub Actions, including a recommended dry run to preview changes before a live release.

```bash
1. Go to **Actions** → **Release** → **Run workflow**
2. Check **dry_run** to preview (recommended)
3. Review the summary, then run again without dry_run
```

--------------------------------

### Run Local Model Deployment with BiGym

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Deploys and runs a trained model locally using the BiGym environment. Requires a collected dataset and a finished training run.

```bash
python example_local_endpoint_bigym.py
```

--------------------------------

### Run Local Model Deployment with ALOHA

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Deploys and runs a trained model locally using the ALOHA environment. Requires a collected dataset and a finished training run.

```bash
python example_local_endpoint.py
```

--------------------------------

### Select Organization

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/commandline.md

Ensure you have selected the correct organization before listing runs.

```bash
neuracore select-org
```

--------------------------------

### Verify Daemon Database Path

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Prints the path to the daemon's database. This helps in troubleshooting migration issues by ensuring the daemon is using the expected database.

```bash
echo "$NEURACORE_DAEMON_DB_PATH"
```

--------------------------------

### List All Data Daemon Profiles

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Command to list all available profiles for the data daemon.

```bash
neuracore data-daemon profile list
```

--------------------------------

### Monitor Training Metrics

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/commandline.md

View local training metrics using the 'training monitor' command. Specify the training name if needed.

```bash
neuracore training monitor --training-name <RUN_NAME>
```

--------------------------------

### Local Training with Auto Batch Size Tuning

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/training.md

Initiates local training for a Diffusion Policy model, enabling automatic tuning of the batch size. Provide the dataset name and a training name.

```bash
python -m neuracore.ml.train algorithm=diffusion_policy batch_size=auto dataset_name="my_dataset" training_name="my_experiment"
```

--------------------------------

### Retrieve and Process Neuracore Dataset

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/getting_started_with_neuracore.ipynb

Fetches a dataset, synchronizes specified data types across embodiments, and generates a video of the first episode's head camera feed. Ensure necessary imports and variables like DATASET_NAME and FREQUENCY are defined.

```python
from neuracore_types import CrossEmbodimentUnion, DataType
import imageio
from IPython.display import Video, display

dataset = nc.get_dataset(DATASET_NAME)

data_types_to_sync = [DataType.JOINT_POSITIONS, DataType.RGB_IMAGES]
cross_embodiment_union: CrossEmbodimentUnion = {}
for robot_id in dataset.robot_ids:
    full_spec = dataset.get_full_embodiment_description(robot_id)
    cross_embodiment_union[robot_id] = {
        dt: full_spec[dt] for dt in data_types_to_sync if dt in full_spec
    }

synced = dataset.synchronize(
    frequency=FREQUENCY,
    cross_embodiment_union=cross_embodiment_union,
)
print(f"Dataset '{DATASET_NAME}': {len(dataset)} episodes")

# First episode summary + video from head camera
first_ep = synced[0]
steps_list = list(first_ep)

print(f"First episode: {len(steps_list)} steps, duration {first_ep.end_time - first_ep.start_time:.2f} s")

frames = [step[DataType.RGB_IMAGES]["head"].frame for step in steps_list]
video_path = "/tmp/first_episode.mp4"
imageio.mimsave(video_path, frames, fps=FREQUENCY)
print("First episode video (head camera):")
display(Video(video_path, embed=True, width=420))
```

--------------------------------

### Point Clouds Configuration

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/dataset_importer.md

Configure 3D point clouds with options for distance units.

```yaml
POINT_CLOUDS:
  source: observation
  format:
    distance_units: M  # M | MM
  mapping:
    - name: point_cloud
      source_name: point_cloud
```

--------------------------------

### Core Training Configuration Parameters

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/training.md

Defines the fundamental training parameters, including seed, epochs, horizons, splits, logging, checkpointing, device, batch size, and data types.

```yaml
# config/config.yaml
defaults:
  - algorithm: diffusion_policy
  - training: default
  - dataset: default

# Core training parameters
seed: 42
epochs: 100
output_prediction_horizon: 100
validation_split: 0.2
logging_frequency: 50
keep_last_n_checkpoints: 5
device: null  # e.g., "cuda:0", "mps", "cpu"

# Batch size (can be "auto" for automatic tuning or an integer)
batch_size: "auto"

# You can either specify input_data_types/output_data_types or
# input_cross_embodiment_description/output_cross_embodiment_description
input_data_types:
  - "JOINT_POSITIONS"
  - "RGB_IMAGES"

output_data_types:
  - "JOINT_TARGET_POSITIONS"

# Dict[str, Dict[DataType, List[str]], e.g., {"my_robot": {"JOINT_POSITIONS": ["joint_1", "joint_2", ...]}}
# You can also pass in an empty dict {} to use all available data for all robots
input_cross_embodiment_description: null
output_cross_embodiment_description: null
```

--------------------------------

### Edit Dataset and Recording Metadata

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Demonstrates how to edit metadata for datasets and recordings, including name, description, tags, notes, and status.

```bash
python example_flag_data.py
```

--------------------------------

### Parallel Gripper Open Amounts Configuration

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/dataset_importer.md

Configure gripper open amounts with options to invert the amount and normalize values between 0 and 1.

```yaml
PARALLEL_GRIPPER_OPEN_AMOUNTS:
  source: observation.state
  format:
    invert_gripper_amount: true # Convert from close amount to open amount
    normalize:  # Optional: normalize the data to be between 0 and 1
      min: -3.2
      max: 3.2
  mapping:
    - name: gripper
      index: 6
```

--------------------------------

### Create a New Data Daemon Profile

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/data_daemon.md

Command to create a new named profile for the data daemon configuration.

```bash
neuracore data-daemon profile create laptop
```

--------------------------------

### Retrieve and Visualize Dataset Episode

Source: https://github.com/neuracoreai/neuracore/blob/main/examples/README.md

Streams data from Neuracore, synchronizes joint positions and camera streams, and replays the first episode of a dataset locally.

```bash
python example_retrieve_dataset.py
```

--------------------------------

### Visual Joint Positions (Gripper Input)

Source: https://github.com/neuracoreai/neuracore/blob/main/docs/dataset_importer.md

Configure visual joint positions populated from gripper open amounts for standard visual joints. Includes inversion and offset.

```yaml
# Example 1: Populate from gripper open amounts for standard visual joints
VISUAL_JOINT_POSITIONS:
  source: observation.state
  format:
    visual_joint_input_type: GRIPPER
  mapping:
    - name: finger_joint1
      index: 6
      inverted: true
      offset: -0.7853981633974483
    - name: finger_joint2
      index: 6
```