FiftyOne

https://docs.voxel51.com/
Admin
FiftyOne is an open-source tool for building high-quality datasets and improving computer vision...
Tokens:2,188,301
Snippets:22,531
Trust Score:9.8
Update:2 months ago
Show doc for...
Tokens:
Raw
### Launch FiftyOne Quickstart via CLI

Source: https://docs.voxel51.com/api/fiftyone.core.cli.html

Examples of launching the quickstart session with different configurations.

```bash
# Launch the quickstart
fiftyone quickstart

# Launch the quickstart with a video dataset
fiftyone quickstart --video

# Launch the quickstart as a remote session
fiftyone quickstart --remote
```

--------------------------------

### Setup Example Data

Source: https://docs.voxel51.com/cheat_sheets/filtering_cheat_sheet.html

Initial setup for dataset filtering examples.

```python
 1from bson import ObjectId
 2
 3import fiftyone as fo
 4import fiftyone.zoo as foz
 5from fiftyone import ViewField as F
 6
 7ds = foz.load_zoo_dataset("quickstart")
 8
 9# Tag a few random samples
10ds.take(3).tag_labels("potential_mistake", label_fields="predictions")
11
12# Grab a few label IDs
13label_ids = [
14    dataset.first().ground_truth.detections[0].id,
15    dataset.last().predictions.detections[0].id,
16]
17ds.select_labels(ids=label_ids).tag_labels("error")
18
19len_filter = F("label").strlen() < 3
20id_filter = F("_id").is_in([ObjectId(_id) for _id in label_ids])
```

--------------------------------

### Setup Keypoint Dataset

Source: https://docs.voxel51.com/cheat_sheets/filtering_cheat_sheet.html

Initial setup for keypoint filtering examples.

```python
 1import fiftyone as fo
 2from fiftyone import ViewField as F
 3
 4ds = fo.Dataset()
 5ds.add_samples(
 6    [
 7        fo.Sample(
 8            filepath="image1.jpg",
 9            predictions=fo.Keypoints(
10                keypoints=[
11                    fo.Keypoint(
12                        label="person",
13                        points=[(0.1, 0.1), (0.1, 0.9), (0.9, 0.9), (0.9, 0.1)],
14                        confidence=[0.7, 0.8, 0.95, 0.99],
15                    )
16                ]
17            )
18        ),
19        fo.Sample(filepath="image2.jpg"),
20    ]
21)
22
23ds.default_skeleton = fo.KeypointSkeleton(
24    labels=["nose", "left eye", "right eye", "left ear", "right ear"],
25    edges=[[0, 1, 2, 0], [0, 3], [0, 4]],
26)
```

--------------------------------

### Launch FiftyOne Quickstart

Source: https://docs.voxel51.com/cli/index.html

Starts a FiftyOne quickstart session with optional flags for video datasets, remote access, and connection timeouts.

```bash
fiftyone quickstart [-h] [-v] [-p PORT] [-A ADDRESS] [-r] [-a] [-w WAIT]
```

```bash
fiftyone quickstart
```

```bash
fiftyone quickstart --video
```

```bash
fiftyone quickstart --remote
```

--------------------------------

### Quick Start Setup and Inference

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/fiftyone_locate_anything.html

Registers the model source, downloads weights, and applies detection inference to a dataset.

```python
import fiftyone as fo
import fiftyone.zoo as foz

# 1. Register the model source (one time per environment)
foz.register_zoo_model_source(
    "https://github.com/Burhan-Q/fiftyone-locate-anything",
    overwrite=True,
)

# 2. Download the model weights (~4 GB, one time)
foz.download_zoo_model(
    "https://github.com/Burhan-Q/fiftyone-locate-anything",
    model_name="nvidia/LocateAnything-3B",
)

# 3. Load and run inference
dataset = foz.load_zoo_dataset("quickstart")

model = foz.load_zoo_model(
    "nvidia/LocateAnything-3B",
    operation="detect",
    classes=["person", "car", "dog"],
)
dataset.apply_model(model, label_field="detections")

session = fo.launch_app(dataset)
```

--------------------------------

### Install FiftyOne

Source: https://docs.voxel51.com/getting_started/object_detection/01_loading_datasets.html

Install the FiftyOne library to get started with dataset loading and management.

```python
!pip install fiftyone

```

--------------------------------

### Curation Prompt for Quickstart Dataset

Source: https://docs.voxel51.com/agents/skills_ecosystem/fiftyone-dataset-curation.html

Example prompt to initiate curation on the loaded quickstart dataset.

```text
"Curate the quickstart dataset: check for quality issues and near-duplicates"
```

--------------------------------

### Setup Qdrant Service and Client

Source: https://docs.voxel51.com/integrations/qdrant.html

Commands to pull and run a local Qdrant Docker container and install the required Python client.

```bash
docker pull qdrant/qdrant
docker run -p 6333:6333 qdrant/qdrant

pip install qdrant-client
```

--------------------------------

### Setup Video Dataset

Source: https://docs.voxel51.com/cheat_sheets/filtering_cheat_sheet.html

Initial setup for video frame filtering examples.

```python
1import fiftyone.zoo as foz
2from fiftyone import ViewField as F
3
4ds = foz.load_zoo_dataset("quickstart-video")
5num_objects = F("detections.detections").length()
```

--------------------------------

### setup()

Source: https://docs.voxel51.com/api/fiftyone.utils.bdd.html

Performs any necessary setup before exporting the first sample in the dataset.

```APIDOC
setup()

    Performs any necessary setup before exporting the first sample in the dataset.
```

--------------------------------

### Load Quickstart Dataset

Source: https://docs.voxel51.com/api/fiftyone.core.expressions.html

Loads the 'quickstart' dataset from FiftyOne Zoo. This is a common starting point for exploring FiftyOne's capabilities.

```python
import fiftyone.zoo as foz
from fiftyone import ViewField as F

dataset = foz.load_zoo_dataset("quickstart")

```

--------------------------------

### Development Environment Setup Commands

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/gemma_vllm_metrics.html

Commands for installing dependencies and building the project in Python and JavaScript environments.

```bash
# Python
pip install -r requirements.txt
pytest

# JS (hybrid panel frontend)
npm install
npm run build   # one shot build, outputs dist/index.umd.js
npm run dev     # watch mode
```

--------------------------------

### Install Replicate Client

Source: https://docs.voxel51.com/tutorials/monocular_depth_estimation.html

Install the necessary Python client for Replicate.

```bash
!pip install replicate
```

--------------------------------

### Install FiftyOne

Source: https://docs.voxel51.com/dataset_zoo/datasets_hf/afo_aerial_floating_objects.html

Install the FiftyOne library to manage and visualize the dataset.

```bash
pip install -U fiftyone
```

--------------------------------

### Install Dependencies

Source: https://docs.voxel51.com/getting_started/manufacturing/06_augmentation.html

Install the required versions of albumentations and fiftyone.

```bash
!pip install -U albumentations==1.4.24 fiftyone
```

--------------------------------

### Install FiftyOne and Ultralytics

Source: https://docs.voxel51.com/tutorials/yolov8.html

Install the necessary libraries for FiftyOne and Ultralytics to begin.

```bash
!pip install fiftyone ultralytics

```

--------------------------------

### Install FiftyOne and Albumentations

Source: https://docs.voxel51.com/integrations/albumentations.html

Install the required packages via pip.

```bash
$ pip install -U fiftyone albumentations
```

--------------------------------

### setup()

Source: https://docs.voxel51.com/api/fiftyone.utils.data.ingestors.html

Performs any necessary setup before importing the first sample in the dataset. This method is called when the importer's context manager interface is entered.

```APIDOC
## setup()

### Description
Performs any necessary setup before importing the first sample in the dataset. This method is called when the importer’s context manager interface is entered, `DatasetImporter.__enter__()`.

### Method
setup()
```

--------------------------------

### Install FiftyOne and Labelbox

Source: https://docs.voxel51.com/tutorials/labelbox_annotation.html

Install the necessary Python packages to enable the integration.

```bash
!pip install fiftyone labelbox
```

--------------------------------

### Load and Visualize QuickstartGeoDataset

Source: https://docs.voxel51.com/api/fiftyone.zoo.datasets.base.html

Demonstrates how to load the QuickstartGeoDataset from the zoo and launch the FiftyOne App to visualize it.

```python
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-geo")

session = fo.launch_app(dataset)
```

--------------------------------

### Detectron2 and PyTorch Setup

Source: https://docs.voxel51.com/tutorials/detectron2.html

Verify PyTorch and CUDA versions and check the Detectron2 installation. This setup is crucial for compatibility.

```python
import torch, detectron2
!nvcc --version
TORCH_VERSION = ".".join(torch.__version__.split(".")[:2])
CUDA_VERSION = torch.__version__.split("+")[-1]
print("torch: ", TORCH_VERSION, "; cuda: ", CUDA_VERSION)
print("detectron2:", detectron2.__version__)

```

--------------------------------

### Load and launch the quickstart dataset

Source: https://docs.voxel51.com/dataset_zoo/datasets/quickstart.html

Use these commands to download the dataset and open the FiftyOne App to visualize the data.

```python
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

session = fo.launch_app(dataset)
```

```bash
fiftyone zoo datasets load quickstart

fiftyone app launch quickstart
```

--------------------------------

### Install FiftyOne, scikit-learn, and umap-learn

Source: https://docs.voxel51.com/tutorials/dimension_reduction.html

Install the necessary libraries for FiftyOne, scikit-learn, and umap-learn. This is a prerequisite for running the dimensionality reduction examples.

```bash
!pip install -U fiftyone scikit-learn umap-learn

```

--------------------------------

### Load and Launch Quickstart Video Dataset via CLI

Source: https://docs.voxel51.com/dataset_zoo/datasets/quickstart_video.html

Use the FiftyOne CLI to load the dataset and launch the app directly from the terminal.

```bash
fiftyone zoo datasets load quickstart-video

fiftyone app launch quickstart-video
```

--------------------------------

### fiftyone.utils.quickstart.quickstart

Source: https://docs.voxel51.com/api/fiftyone.utils.quickstart.html

Runs the FiftyOne quickstart, which loads a dataset and launches the App.

```APIDOC
## quickstart(video=False, port=None, address=None, remote=False)

### Description
Runs the FiftyOne quickstart. This method loads an interesting dataset from the Dataset Zoo, launches the App, and prints some suggestions for exploring the dataset.

### Parameters
- **video** (bool) - Optional - Whether to launch a video dataset. Defaults to False.
- **port** (int) - Optional - The port number to serve the App. If None, fiftyone.config.default_app_port is used.
- **address** (str) - Optional - The address to serve the App. If None, fiftyone.config.default_app_address is used.
- **remote** (bool) - Optional - Whether this is a remote session, and opening the App should not be attempted. Defaults to False.

### Returns
- **dataset** (fiftyone.core.dataset.Dataset) - The dataset that was loaded.
- **session** (fiftyone.core.session.Session) - The session instance for the App that was launched.
```

--------------------------------

### Video Tracking Quickstart

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/molmo_point.html

Registers the model source, loads a dataset, and applies the tracking model to generate keypoints.

```python
import fiftyone as fo
import fiftyone.zoo as foz

foz.register_zoo_model_source(
    "https://github.com/harpreetsahota204/molmo_point",
    overwrite=True,
)

dataset = foz.load_zoo_dataset("quickstart-video")
dataset.compute_metadata()

model = foz.load_zoo_model("allenai/MolmoPoint-8B", media_type="video")
model.operation = "tracking"
model.prompt = ["person", "car", "dog"]

dataset.apply_model(
    model,
    label_field="tracking_keypoints",
    batch_size=1,
    num_workers=2,
)

session = fo.launch_app(dataset)
```

--------------------------------

### Install NuScenes dependencies

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/rerun_plugin.html

Required packages for running the NuScenes integration example.

```bash
pip install nuscenes-devkit rerun-sdk open3d matplotlib Pillow
```

--------------------------------

### Install FiftyOne and dependencies

Source: https://docs.voxel51.com/getting_started/model_dataset_zoo/01_intro.html

Required packages for running the FiftyOne Zoo examples.

```bash
pip install fiftyone torch torchvision
```

```bash
!pip install fiftyone
!pip install torch torchvision
```

--------------------------------

### Create a Sample

Source: https://docs.voxel51.com/user_guide/using_datasets.html

Initialize a new FiftyOne sample with a required filepath.

```python
1import fiftyone as fo
2
3sample = fo.Sample(filepath="/path/to/image.png")
4
5print(sample)
```

```text
<Sample: {
    'id': None,
    'media_type': 'image',
    'filepath': '/path/to/image.png',
    'tags': [],
    'metadata': None,
    'created_at': None,
    'last_modified_at': None,
}>
```

--------------------------------

### Performing COCO-style Detection Evaluation

Source: https://docs.voxel51.com/integrations/coco.html

Example of running COCO-style evaluation on the quickstart dataset and visualizing the results.

```python
 1import fiftyone as fo
 2import fiftyone.zoo as foz
 3from fiftyone import ViewField as F
 4
 5dataset = foz.load_zoo_dataset("quickstart")
 6print(dataset)
 7
 8# Evaluate the objects in the `predictions` field with respect to the
 9# objects in the `ground_truth` field
10results = dataset.evaluate_detections(
11    "predictions",
12    gt_field="ground_truth",
13    method="coco",
14    eval_key="eval",
15)
16
17# Get the 10 most common classes in the dataset
18counts = dataset.count_values("ground_truth.detections.label")
19classes = sorted(counts, key=counts.get, reverse=True)[:10]
20
21# Print a classification report for the top-10 classes
22results.print_report(classes=classes)
23
24# Print some statistics about the total TP/FP/FN counts
25print("TP: %d" % dataset.sum("eval_tp"))
26print("FP: %d" % dataset.sum("eval_fp"))
27print("FN: %d" % dataset.sum("eval_fn"))
28
29# Create a view that has samples with the most false positives first, and
30# only includes false positive boxes in the `predictions` field
31view = (
32    dataset
33    .sort_by("eval_fp", reverse=True)
34    .filter_labels("predictions", F("eval") == "fp")
35)
36
37# Visualize results in the App
38session = fo.launch_app(view=view)
```

--------------------------------

### Install Label Studio

Source: https://docs.voxel51.com/integrations/labelstudio.html

Install the Label Studio package and launch the server.

```bash
pip install label-studio

# Launch it!
label-studio
```

--------------------------------

### Generate label schemas

Source: https://docs.voxel51.com/api/fiftyone.core.collections.html

Example of generating and printing the label schema for a dataset using the quickstart dataset.

```python
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")
dataset.compute_metadata()

fo.pprint(dataset.generate_label_schemas(scan_samples=False))
```

--------------------------------

### Load Quickstart Dataset

Source: https://docs.voxel51.com/tutorials/gemini_vision.html

Initializes the FiftyOne quickstart dataset and launches the application session.

```python
dataset = foz.load_zoo_dataset("quickstart")
session = fo.launch_app(dataset, auto=False)
```

--------------------------------

### Load quickstart image dataset

Source: https://docs.voxel51.com/user_guide/using_datasets.html

Load a pre-built dataset from the FiftyOne Zoo and launch the App.

```python
 1import fiftyone as fo
 2import fiftyone.zoo as foz
 3
 4dataset = foz.load_zoo_dataset("quickstart")
 5
 6print(dataset.count("ground_truth.detections"))  # 1232
 7print(dataset.count("predictions.detections"))  # 5620
 8print(dataset.count_values("ground_truth.detections.label"))
 9# {'dog': 15, 'airplane': 24, 'dining table': 15, 'hot dog': 5, ...}
10
11session = fo.launch_app(dataset)
```

--------------------------------

### Start Qdrant Server

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/audio_retrieval.html

Launch a local Qdrant vector database instance using Docker.

```bash
docker run -p "6333:6333" -p "6334:6334" -d qdrant/qdrant
```

--------------------------------

### Get Plugin Information via CLI

Source: https://docs.voxel51.com/api/fiftyone.core.cli.html

Commands to retrieve details about locally installed plugins by name or directory path.

```bash
# Prints information about a plugin
fiftyone plugins info <name>

# Prints information about a plugin in a given directory
fiftyone plugins info <dir>
```

--------------------------------

### Install FiftyOne and dependencies

Source: https://docs.voxel51.com/getting_started/manufacturing/01_intro.html

Install the core library along with Hugging Face and Google Drive utilities.

```bash
!pip install fiftyone huggingface_hub gdown
```

--------------------------------

### Setup Milvus Server and Client

Source: https://docs.voxel51.com/integrations/milvus.html

Commands to deploy a Milvus standalone instance via Docker and install the required Python client.

```bash
wget https://github.com/milvus-io/milvus/releases/download/v2.2.11/milvus-standalone-docker-compose.yml -O docker-compose.yml
sudo docker compose up -d

pip install pymilvus
```

--------------------------------

### Install the Few-Shot Learning Plugin

Source: https://docs.voxel51.com/labs/labs_ecosystem/few_shot_learning.html

Use the FiftyOne CLI to download the plugin from the official labs repository.

```bash
fiftyone plugins download \
    https://github.com/voxel51/labs \
    --plugin-names @51labs/few_shot_learning
```

--------------------------------

### DatasetExporter.setup

Source: https://docs.voxel51.com/api/fiftyone.utils.data.exporters.html

Performs necessary setup before exporting the first sample. This method is called when entering the exporter's context manager.

```APIDOC
## setup()

### Description
Performs any necessary setup before exporting the first sample in the dataset. This method is called when the exporter’s context manager interface is entered, `DatasetExporter.__enter__()`.
```

--------------------------------

### Brain Model Management Functions

Source: https://docs.voxel51.com/api/fiftyone.brain.internal.html

Utility functions for listing, downloading, installing, loading, finding, getting, and deleting brain models.

```APIDOC
## Brain Model Management

### Description
Functions for managing brain models, including listing, downloading, and loading.

### Functions
- `list_models()`
- `list_downloaded_models()`
- `is_model_downloaded(model_name)`
- `download_model(model_name, **kwargs)`
- `install_model_requirements(model_name, **kwargs)`
- `ensure_model_requirements(model_name, **kwargs)`
- `load_model(model_name, **kwargs)`
- `find_model(model_name, **kwargs)`
- `get_model(model_name, **kwargs)`
- `delete_model(model_name, **kwargs)`
```

--------------------------------

### Load Dataset with Predictions

Source: https://docs.voxel51.com/getting_started/model_evaluation/01_basic_evaluation.html

Loads the 'quickstart' dataset from the FiftyOne Zoo, which includes pre-loaded model predictions. This is a convenient way to start with a dataset ready for evaluation.

```python
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

# View summary info about the dataset
print(dataset)

```

--------------------------------

### Load and launch quickstart-geo dataset

Source: https://docs.voxel51.com/dataset_zoo/datasets/quickstart_geo.html

Use these commands to load the dataset into FiftyOne and launch the application to visualize the data.

```python
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-geo")

session = fo.launch_app(dataset)
```

```bash
fiftyone zoo datasets load quickstart-geo

fiftyone app launch quickstart-geo
```

--------------------------------

### Get Filtered Fields

Source: https://docs.voxel51.com/api/fiftyone.core.stages.html

Identifies fields containing arrays that have been filtered by the stage. For example, it returns `"predictions.detections"` if the `"predictions"` field is filtered.

```python
stage.get_filtered_fields(sample_collection)
```

```python
stage.get_filtered_fields(sample_collection, frames=True)
```

--------------------------------

### Get Filtered Fields

Source: https://docs.voxel51.com/api/fiftyone.core.stages.html

Identifies fields containing arrays that have been filtered by the stage. For example, if a stage filters detections within a 'predictions' field, this method would return 'predictions.detections'.

```python
stage.get_filtered_fields(sample_collection)
```

--------------------------------

### Load and visualize the Quickstart 3D dataset

Source: https://docs.voxel51.com/dataset_zoo/datasets/quickstart_3d.html

Use these commands to download the dataset from the zoo and launch the FiftyOne App for visualization.

```python
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-3d")

session = fo.launch_app(dataset)
```

```bash
fiftyone zoo datasets load quickstart-3d

fiftyone app launch quickstart-3d
```

--------------------------------

### Install Image to Video Plugin

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/img_to_video.html

Use the FiftyOne CLI to download and install the plugin from the GitHub repository.

```bash
fiftyone plugins download https://github.com/danielgural/img_to_video_plugin
```

--------------------------------

### Custom Prompt Example for olmOCR-2

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/olmocr_2.html

Define a detailed custom prompt to guide the olmOCR-2 model in extracting text, preserving layout, converting equations to LaTeX, and tables to HTML.

```python
custom_prompt = """
Extract all text from this document. 
Focus on preserving the original layout and formatting.
Convert any equations to LaTeX and tables to HTML format.
"""

model = foz.load_zoo_model(
    "allenai-olmOCR-2",
    custom_prompt=custom_prompt
)

```

--------------------------------

### Load and Launch Quickstart Video Dataset

Source: https://docs.voxel51.com/dataset_zoo/datasets/quickstart_video.html

Load the dataset using the Python API and launch the FiftyOne App to visualize it.

```python
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-video")

session = fo.launch_app(dataset)
```

--------------------------------

### Quick Start: Load Dataset and Apply Moondream3

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/moondream3.html

Load a dataset from the FiftyOne Hub, extract object labels, and then apply the Moondream3 model for general captioning. This provides a basic end-to-end example.

```python
import fiftyone as fo
import fiftyone.zoo as foz
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub(
    "Voxel51/GQA-Scene-Graph",
    max_samples=50,
    overwrite=True
    )

#adding a list of objects which are present in the image
sample_objects = dataset.values("detections.detections.label")

sample_level_objects =  [list(set(obj)) for obj in sample_objects]

dataset.set_values("sample_level_objects", sample_level_objects)
# Load Moondream3
model = foz.load_zoo_model("moondream/moondream3-preview")

# Apply different operations
model.operation = "caption"
dataset.apply_model(model, label_field="md_captions")

```

--------------------------------

### Compute Array Length

Source: https://docs.voxel51.com/api/fiftyone.core.expressions.html

Use the `length()` method to get the number of elements in an array-resolving expression. If the expression is null or missing, zero is returned. This example filters for samples with at least 15 detections.

```python
import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F

dataset = foz.load_zoo_dataset("quickstart")

# Only contains samples with at least 15 predicted objects
view = dataset.match(F("predictions.detections").length() >= 15)

print(dataset.count())
print(view.count())
```

--------------------------------

### Manage FiftyOne Configuration via CLI

Source: https://docs.voxel51.com/api/fiftyone.core.cli.html

Examples of accessing and inspecting the FiftyOne configuration file.

```bash
# Print your entire config
fiftyone config

# Print a specific config field
fiftyone config <field>

# Print the location of your config on disk (if one exists)
fiftyone config --locate
```

--------------------------------

### Set Field with Conditional Logic

Source: https://docs.voxel51.com/api/fiftyone.core.expressions.html

Use `set_field` with `map` and `re_match` to conditionally update a field based on a regular expression match. This example renames labels starting with 'b' to 'b' and others to 'other'.

```python
view = dataset.set_field(
    "predictions.detections",
    F("detections").map(
        F().set_field(
            "label",
            F("label").re_match("^b").if_else("b", "other"),
        )
    )
)

print(view.count_values("predictions.detections.label"))
```

--------------------------------

### Install Qdrant Python Client

Source: https://docs.voxel51.com/integrations/qdrant.html

Install the required Python client library.

```bash
pip install qdrant-client
```

--------------------------------

### UI-TARS Agent Actions and Custom Prompts

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/ui_tars.html

Utilize the 'agentic' operation mode for full agent actions with reasoning. You can also set a custom system prompt to guide the agent's behavior, for example, focusing on accessibility and usability.

```python
# Full agentic mode with reasoning
model.operation = "agentic"
dataset.apply_model(model, prompt_field="instructions", label_field="agentic_output")

# Custom system prompts
model.system_prompt = "You are a GUI testing assistant. Focus on accessibility and usability."
```

--------------------------------

### Initialize Inference Environment

Source: https://docs.voxel51.com/integrations/ultralytics.html

Set up the environment and load a sample dataset for inference.

```python
 1# Suppress Ultralytics logging
 2import os; os.environ["YOLO_VERBOSE"] = "False"
 3
 4import fiftyone as fo
 5import fiftyone.zoo as foz
 6import fiftyone.utils.ultralytics as fou
 7
 8from ultralytics import YOLO
 9
10# Load an example dataset
11dataset = foz.load_zoo_dataset("quickstart", max_samples=25)
12dataset.select_fields().keep_fields()
```

--------------------------------

### Load and Apply Microsoft Kosmos-2.5 Model in FiftyOne

Source: https://docs.voxel51.com/model_zoo/models/microsoft_kosmos_2_5.html

This example demonstrates how to register a remote model source, load a dataset, load the 'microsoft/kosmos-2.5' model, and apply it to the dataset for predictions. Ensure the kosmos2_5 plugin is installed and review its documentation before use.

```python
import fiftyone as fo
import fiftyone.zoo as foz

foz.register_zoo_model_source("https://github.com/harpreetsahota204/kosmos2_5")

dataset = foz.load_zoo_dataset(
    "coco-2017",
    split="validation",
    dataset_name=fo.get_default_dataset_name(),
    max_samples=50,
    shuffle=True,
)

model = foz.load_zoo_model("microsoft/kosmos-2.5")

dataset.apply_model(model, label_field="predictions")

session = fo.launch_app(dataset)
```

--------------------------------

### Install FiftyOne

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/emoji_search.html

Upgrade your FiftyOne installation to the latest version before installing the plugin.

```bash
pip install fiftyone --upgrade

```

--------------------------------

### Setting Up Video Dataset for Summary Fields

Source: https://docs.voxel51.com/user_guide/using_datasets.html

Initializes a video dataset and prepares fields for summary field creation.

```python
1import fiftyone as fo
2import fiftyone.zoo as foz
3from fiftyone import ViewField as F
4
5dataset = foz.load_zoo_dataset("quickstart-video")
6dataset.set_field("frames.detections.detections.confidence", F.rand()).save()
7
8session = fo.launch_app(dataset)
```

--------------------------------

### Install FiftyOne MCP Server

Source: https://docs.voxel51.com/agents/using_agents.html

Install the package into the same Python environment where FiftyOne is installed.

```bash
pip install fiftyone-mcp-server
```

--------------------------------

### Install the plugin for local development

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/fo_video_sensor_data_sync.html

Clone the repository and create a symbolic link in the FiftyOne plugins directory to enable local development.

```bash
git clone https://github.com/Burhan-Q/fo-video-sensor-data-sync
mkdir -p "$(fiftyone config plugins_dir)/@Burhan-Q"
ln -s "$(pwd)/fo-video-sensor-data-sync" "$(fiftyone config plugins_dir)/@Burhan-Q/fo-video-sensor-data-sync"
```

--------------------------------

### Install OpenCLIP from Source

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/zero_shot_prediction.html

Install the `open-clip` library from its source repository if you prefer to build it from the latest code.

```bash
pip install git+https://github.com/mlfoundations/open_clip.git

```

--------------------------------

### Install FiftyOne

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/depth_pro_plugin.html

Install FiftyOne using pip. This is a prerequisite for installing the Depth Pro plugin.

```bash
pip install -U fiftyone
```

--------------------------------

### Install the YouTube Panel Plugin

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/youtube_panel_plugin.html

Use the FiftyOne CLI to download and install the plugin from the GitHub repository.

```bash
fiftyone plugins download https://github.com/jacobmarks/fiftyone-youtube-panel-plugin
```

--------------------------------

### Manually install OS-Atlas dependencies

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/os_atlas.html

Install the required Python packages manually if automatic installation is not used.

```bash
pip install huggingface-hub transformers torch torchvision qwen-vl-utils accelerate
```

--------------------------------

### start()

Source: https://docs.voxel51.com/api/fiftyone.core.utils.html

Starts the progress bar.

```APIDOC
## start()

### Description
Starts the progress bar.
```

--------------------------------

### Install video2dataset Plugin from GitHub

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/video2dataset.html

Installs the video2dataset plugin by downloading it from its GitHub repository. This command installs the entire plugin.

```bash
fiftyone plugins download https://github.com/Parva101/video2dataset
```

--------------------------------

### Complete Qwen3-VL Workflow

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/qwen3vl_video.html

A full example demonstrating model registration, dataset loading, metadata computation, and various analysis operations.

```python
import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone.utils.huggingface import load_from_hub

# Register model source
foz.register_zoo_model_source(
    "https://github.com/harpreetsahota204/qwen3vl_video",
    overwrite=True
)

# Load dataset
dataset = load_from_hub(
    "harpreetsahota/random_short_videos",
    dataset_name="random_short_videos"
)

# Compute metadata (required!)
dataset.compute_metadata()

# Load model
model = foz.load_zoo_model("Qwen/Qwen3-VL-8B-Instruct")

# Run comprehensive analysis
model.operation = "comprehensive"
dataset.apply_model(model, label_field="comprehensive", skip_failures=True)

# Add descriptions
model.operation = "description"
dataset.apply_model(model, label_field="desc", skip_failures=True)

# Extract text with OCR
model.operation = "ocr"
dataset.apply_model(model, label_field="ocr", skip_failures=True)

# Detect temporal events
model.operation = "temporal_localization"
dataset.apply_model(model, label_field="events", skip_failures=True)

# Custom analysis
model.operation = "custom"
model.custom_prompt = """Analyze for safety concerns:
{
  "potential_hazards": [{"start": "mm:ss.ff", "end": "mm:ss.ff", "description": "..."}],
  "safety_rating": "low/medium/high"
}
"""
dataset.apply_model(model, label_field="safety", skip_failures=True)

# Launch app
session = fo.launch_app(dataset)
```

--------------------------------

### Install FiftyOne with Documentation Dependencies

Source: https://docs.voxel51.com/contribute/index.html

Install the FiftyOne project with the necessary dependencies for working on and building the documentation. This step is crucial for documentation contributions.

```bash
# Install with docs dependencies

```

--------------------------------

### Install Transformers Library

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/moondream2.html

Install the transformers library with a specific version constraint. Ensure you have transformers<=4.49.0 installed before running the model.

```bash
pip install transformers<=4.49.0
# Add other dependencies as needed

```

--------------------------------

### Install FiftyOne Tile Plugin

Source: https://docs.voxel51.com/plugins/plugins_ecosystem/fiftyone_tile.html

Install the plugin from the GitHub repository using the FiftyOne CLI.

```bash
fiftyone plugins download https://github.com/mmoollllee/fiftyone-tile/
```

--------------------------------

### Generate Integer Range with E.range()

Source: https://docs.voxel51.com/api/fiftyone.core.expressions.html

Creates an array expression representing a sequence of integers. If only `start` is provided, it generates a range from 0 up to (but not including) `start`. If both `start` and `stop` are provided, it generates a range from `start` (inclusive) to `stop` (exclusive).

```python
import fiftyone as fo
from fiftyone import ViewField as F

dataset = fo.Dataset()
dataset.add_samples(
    [
        fo.Sample(filepath="image1.jpg", tags=["a", "b", "c"]),
        fo.Sample(filepath="image2.jpg", tags=["y", "z"]),
    ]
)

```

--------------------------------

### Install Required Packages

Source: https://docs.voxel51.com/tutorials/zero_shot_classification.html

Install PyTorch, FiftyOne, Transformers, and OpenCLIP. Ensure you have PyTorch installed with CUDA support if you plan to use a GPU.

```bash
pip install -U torch torchvision fiftyone transformers timm open_clip_torch

```

--------------------------------

### Launch Few-Shot Learning Demo

Source: https://docs.voxel51.com/labs/labs_ecosystem/few_shot_learning.html

Execute the demo script to initialize the few-shot learning plugin with a sample dataset.

```bash
cd /path/to/fiftyone-labs
python plugins/few_shot_learning/run_demo.py
```
Tokens:
Raw