HoloAssist

https://holoassist.github.io/
Admin
HoloAssist is a large-scale egocentric human interaction dataset designed to facilitate research on...
Tokens:1,731
Snippets:18
Trust Score:9.7
Update:4 months ago
Show doc for...
Tokens:
Raw
### Raw Annotation Structure

Source: https://holoassist.github.io/data_links/README.md

Defines the basic structure for all raw annotations, including ID, label, start/end times, type, and attributes.

```json
{
    "id": int, original label id,
    "label": "Narration", "Conversation", "Fine grained action",  or "Coarse grained action", 
    "start": start time in seconds, 
    "end": end time in seconds, 
    "type":"range",
    "attributes":{
        Different from different label task. See below.
    },

}
```

--------------------------------

### HoloAssist Dataset Directory Structure

Source: https://holoassist.github.io/data_links/README.md

Illustrates the hierarchical organization of data files within a typical recording session folder, including modalities like depth, eye tracking, hand tracking, head pose, IMU, and video.

```tree
  .
  	├── R007-7July-DSLR
	│   └── Export_py
	│       	│── AhatDepth
					│   ├── 000000.png
					│   ├── 000001.png
					│   ├── ...
					│   ├── AhatDepth_synced.txt
					│   ├── Instrinsics.txt
					│   ├── Pose_sync.txt
					│   └── Timing_sync.txt
				├── Eyes
				│   └── Eyes_sync.txt 
			├── Hands
			│   ├── Left_sync.txt
			│   └── Right_sync.txt 
		├── Head
		│   └── Head_sync.txt 
		├── IMU
		│   ├── Accelerometer_sync.txt
		│   ├── Gyroscope_sync.txt
		│   └── Magnetometer_sync.txt
		├── Video
		│   ├── Pose_sync.txt
		│   ├── Instrinsincs.txt
		│   └── VideoMp4Timing.txt
		├── Video_pitchshift.mp4
		└── Video_compress.mp4
  ├── R012-7July-Nespresso/
  ├── R013-7July-Nespresso/
  ├── R014-7July-DSLR/
  └── ...

```

--------------------------------

### Fine Grained Action Annotation Attributes

Source: https://holoassist.github.io/data_links/README.md

Defines attributes for 'Fine grained action' labels, covering action correctness, explanations, and descriptive elements.

```json
    "id": int, original label id,
    "label": "Fine grained action",  
    "start": start time in seconds, 
    "end": end time in seconds, 
    "type":"range",
    "attributes": {
        "Action Correctness":"Correct Action",
        "Incorrect Action Explanation":"none",
        "Incorrect Action Corrected by":"none",
        "Verb":"approach",
        "Adjective":"none",
        "Noun":"gopro",
        "adverbial":"none"
    },


```

--------------------------------

### Coarse Grained Action Annotation Attributes

Source: https://holoassist.github.io/data_links/README.md

Outlines attributes for 'Coarse grained action' labels, including an action sentence and descriptive components.

```json
    "id": int, original label id,
    "label": "Coarse grained action",  
    "start": start time in seconds, 
    "end": end time in seconds, 
    "type":"range",
    "attributes": {
        "Action sentence":"The student changes the battery for the GoPro.",
        "Verb":"exchange",
        "Adjective":"none",
        "Noun":"battery"
    },


```

--------------------------------

### Narration Annotation Attributes

Source: https://holoassist.github.io/data_links/README.md

Specifies the attributes for 'Narration' labels, including a long-form description field for detailed explanations.

```json
    "id": int, original label id,
    "label": "Narration",  
    "start": start time in seconds, 
    "end": end time in seconds, 
    "type":"range",
    "attributes": {
        "Long-form description": "Use multiple sentences and make this as long as is necessary to be exhaustive. There are a finite number of scenarios across all videos, so make sure to call out the distinctive changes between videos, in particular, mistakes that the task performer makes in the learning process that are either self-corrected or corrected by the instructor."
    },


```

--------------------------------

### Conversation Annotation Attributes

Source: https://holoassist.github.io/data_links/README.md

Details the attributes for 'Conversation' labels, including purpose, transcription, and confidence.

```json
    "id": int, original label id,
    "label": "Narration",  
    "start": start time in seconds, 
    "end": end time in seconds, 
    "type":"range",
    "attributes": {
        "Conversation Purpose":"instructor-start-conversation_other",
        "Transcription":"*unintelligible*",
        "Transcription Confidence":"low-confidence-transcription",
    },


```
Tokens:
Raw