MiVOLO

https://github.com/wildchlamydia/mivolo
Admin
MiVOLO is a multi-input transformer-based model for accurate age and gender estimation from facial...
Tokens:6,753
Snippets:38
Trust Score:7.2
License:Apache-2.0
Update:6 months ago
Show doc for...
Tokens:
Raw
### Install MiVOLO and Dependencies

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Installs the MiVOLO library and its required dependencies using pip. Ensure PyTorch 1.13+ is installed beforehand.

```bash
pip install -r requirements.txt
pip install .
```

--------------------------------

### Command Line Demo

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Instructions and examples for running inference on images and videos using the command-line interface, including downloading models and specifying input/output paths.

```APIDOC
## Command Line Demo

Run inference on images or videos from the command line.

```bash
# Download required models
# Note: Replace with actual download links if available and valid
# wget https://drive.google.com/file/d/1CGNCkZQNj5WkP3rLpENWAOgrBQkUWRdw/view -O models/yolov8x_person_face.pt
# wget https://drive.google.com/file/d/11i8pKctxz3wVkDBlWKvhYIh7kpVFXSZ4/view -O models/mivolo_imbd.pth.tar

# Process single image
python3 demo.py \
    --input "photo.jpg" \
    --output "output" \
    --detector-weights "models/yolov8x_person_face.pt" \
    --checkpoint "models/mivolo_imbd.pth.tar" \
    --device "cuda:0" \
    --with-persons \
    --draw

# Process YouTube video
python3 demo.py \
    --input "https://www.youtube.com/shorts/pVh32k0hGEI" \
    --output "output" \
    --detector-weights "models/yolov8x_person_face.pt" \
    --checkpoint "models/mivolo_imbd.pth.tar" \
    --device "cuda:0" \
    --with-persons \
    --draw

# Process local video file
python3 demo.py \
    --input "video.mp4" \
    --output "output" \
    --detector-weights "models/yolov8x_person_face.pt" \
    --checkpoint "models/mivolo_imbd.pth.tar" \
    --device "cuda:0" \
    --with-persons \
    --draw
```
```

--------------------------------

### Get Bounding Boxes and Track Objects

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Demonstrates how to retrieve bounding box indices for specific categories like 'face' and 'person', and how to iterate through detected bounding boxes. It also shows how to perform video tracking to maintain object IDs across frames.

```python
face_indices = results.get_bboxes_inds("face")
person_indices = results.get_bboxes_inds("person")

for idx in face_indices:
    bbox = results.get_bbox_by_ind(idx)  # Returns [x1, y1, x2, y2]
    x1, y1, x2, y2 = bbox.cpu().numpy()
    print(f"Face at: ({x1}, {y1}) to ({x2}, {y2})")

# For video tracking (maintains object IDs across frames)
for frame in video_frames:
    results = detector.track(frame)
    persons, faces = results.get_results_for_tracking()
    # persons = {track_id: (age, gender), ...}
```

--------------------------------

### Mivolo Command Line Demo

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Provides command-line instructions for running Mivolo inference on single images, YouTube videos, and local video files. It includes steps for downloading required models and specifying input/output paths, model weights, and device.

```bash
# Download required models
wget https://drive.google.com/file/d/1CGNCkZQNj5WkP3rLpENWAOgrBQkUWRdw/view \
    -O models/yolov8x_person_face.pt
wget https://drive.google.com/file/d/11i8pKctxz3wVkDBlWKvhYIh7kpVFXSZ4/view \
    -O models/mivolo_imbd.pth.tar

# Process single image
python3 demo.py \
    --input "photo.jpg" \
    --output "output" \
    --detector-weights "models/yolov8x_person_face.pt" \
    --checkpoint "models/mivolo_imbd.pth.tar" \
    --device "cuda:0" \
    --with-persons \
    --draw

# Process YouTube video
python3 demo.py \
    --input "https://www.youtube.com/shorts/pVh32k0hGEI" \
    --output "output" \
    --detector-weights "models/yolov8x_person_face.pt" \
    --checkpoint "models/mivolo_imbd.pth.tar" \
    --device "cuda:0" \
    --with-persons \
    --draw

# Process local video file
python3 demo.py \
    --input "video.mp4" \
    --output "output" \
    --detector-weights "models/yolov8x_person_face.pt" \
    --checkpoint "models/mivolo_imbd.pth.tar" \
    --device "cuda:0" \
    --with-persons \
    --draw
```

--------------------------------

### Export MiVOLO Model to TorchScript

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Demonstrates how to initialize the MiVOLO model and export it to a traced TorchScript file for production deployment. This process requires a dummy input tensor matching the expected model dimensions.

```python
import torch
from mivolo.model.mi_volo import MiVOLO

model = MiVOLO(
    ckpt_path="models/mivolo_imbd.pth.tar",
    device="cuda:0",
    half=False,
    use_persons=True,
    disable_faces=False
)

example_input = torch.randn(1, 6, 224, 224).to("cuda:0")

traced_model = torch.jit.trace(model.model, example_input)
traced_model.save("mivolo_traced.pt")

loaded_model = torch.jit.load("mivolo_traced.pt")
output = loaded_model(example_input)
```

--------------------------------

### Image Preprocessing Utilities

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Details image preprocessing steps using Mivolo utilities, including letterbox resizing to maintain aspect ratio and preparing batches of image crops for model input with ImageNet normalization. Shows concatenation for multi-input models.

```python
import cv2
import numpy as np
from mivolo.data.misc import prepare_classification_images, class_letterbox

# Letterbox resize maintaining aspect ratio
image = cv2.imread("face_crop.jpg")
resized = class_letterbox(image, new_shape=(224, 224), color=(0, 0, 0))

# Prepare batch of crops for model input
face_crops = [cv2.imread("face1.jpg"), cv2.imread("face2.jpg"), None]  # None = padding
body_crops = [cv2.imread("body1.jpg"), cv2.imread("body2.jpg"), cv2.imread("body3.jpg")]

# ImageNet normalization
from timm.data import IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD

face_tensor = prepare_classification_images(
    face_crops,
    target_size=224,
    mean=IMAGENET_DEFAULT_MEAN,
    std=IMAGENET_DEFAULT_STD,
    device="cuda:0"
)
# Output shape: (3, 3, 224, 224)

body_tensor = prepare_classification_images(
    body_crops,
    target_size=224,
    mean=IMAGENET_DEFAULT_MEAN,
    std=IMAGENET_DEFAULT_STD,
    device="cuda:0"
)

# Concatenate for multi-input model
model_input = torch.cat((face_tensor, body_tensor), dim=1)
# Output shape: (3, 6, 224, 224)
```

--------------------------------

### Mivolo Model Validation

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Shows how to evaluate Mivolo model performance on benchmark datasets like UTKFace and Lagenda. Includes commands for specifying dataset paths, split, batch size, checkpoint, and device, as well as options for drawing error histograms.

```bash
# Validate on UTKFace dataset
python3 eval_pretrained.py \
    --dataset_images /path/to/utk/images \
    --dataset_annotations /path/to/utk/annotation \
    --dataset_name utk \
    --split valid \
    --batch-size 512 \
    --checkpoint models/mivolo_imbd.pth.tar \
    --half \
    --with-persons \
    --device "cuda:0" \
    --results-file results.csv

# Supported datasets: utk, imdb, lagenda, fairface, adience, agedb, cacd

# Generate error histogram by age
python3 eval_pretrained.py \
    --dataset_images /path/to/lagenda/images \
    --dataset_annotations /path/to/lagenda/annotation \
    --dataset_name lagenda \
    --split valid \
    --checkpoint models/mivolo_imbd.pth.tar \
    --with-persons \
    --draw-hist
```

--------------------------------

### Image Preprocessing Utilities

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Utilities for preparing images for model input, including letterboxing for aspect ratio preservation and applying ImageNet normalization.

```APIDOC
## Image Preprocessing Utilities

Prepare images for model input with proper normalization and resizing.

```python
import cv2
import numpy as np
from mivolo.data.misc import prepare_classification_images, class_letterbox

# Letterbox resize maintaining aspect ratio
image = cv2.imread("face_crop.jpg")
resized = class_letterbox(image, new_shape=(224, 224), color=(0, 0, 0))

# Prepare batch of crops for model input
face_crops = [cv2.imread("face1.jpg"), cv2.imread("face2.jpg"), None]  # None = padding
body_crops = [cv2.imread("body1.jpg"), cv2.imread("body2.jpg"), cv2.imread("body3.jpg")]

# ImageNet normalization
from timm.data import IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD

face_tensor = prepare_classification_images(
    face_crops,
    target_size=224,
    mean=IMAGENET_DEFAULT_MEAN,
    std=IMAGENET_DEFAULT_STD,
    device="cuda:0"
)
# Output shape: (3, 3, 224, 224)

body_tensor = prepare_classification_images(
    body_crops,
    target_size=224,
    mean=IMAGENET_DEFAULT_MEAN,
    std=IMAGENET_DEFAULT_STD,
    device="cuda:0"
)

# Concatenate for multi-input model
# model_input = torch.cat((face_tensor, body_tensor), dim=1)
# Output shape: (3, 6, 224, 224)
```
```

--------------------------------

### TorchScript Export

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Exports the MiVOLO model to a TorchScript format for efficient production deployment and inference.

```APIDOC
## POST /export/torchscript

### Description
Converts the MiVOLO model instance into a serialized TorchScript file (.pt) for high-performance inference.

### Method
POST

### Endpoint
/export/torchscript

### Parameters
#### Request Body
- **ckpt_path** (string) - Required - Path to the model weights file (.pth.tar)
- **device** (string) - Optional - Device to load model on (e.g., "cuda:0")
- **half** (boolean) - Optional - Whether to use FP16 precision

### Request Example
{
  "ckpt_path": "models/mivolo_imbd.pth.tar",
  "device": "cuda:0",
  "half": false
}

### Response
#### Success Response (200)
- **status** (string) - Success message indicating the file path of the saved model.
```

--------------------------------

### MiVOLO Model Class for Direct Inference

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Provides a low-level API for direct model inference using preprocessed input tensors. Allows fine-grained control over model parameters like FP16 usage and feature extraction options. Outputs raw predictions that need parsing.

```python
import torch
import numpy as np
from mivolo.model.mi_volo import MiVOLO
from mivolo.structures import PersonAndFaceResult

# Initialize model
model = MiVOLO(
    ckpt_path="models/mivolo_imbd.pth.tar",
    device="cuda:0",
    half=True,  # Use FP16 for faster inference
    use_persons=True,  # Enable body feature extraction
    disable_faces=False,  # Enable face feature extraction
    verbose=True
)

# Print model metadata
print(f"Input size: {model.input_size}")
print(f"Age range: {model.meta.min_age} - {model.meta.max_age}")
print(f"Using persons: {model.meta.use_person_crops}")
print(f"Using faces: {model.meta.use_face_crops}")

# Warmup for consistent inference times
model.warmup(batch_size=1, steps=10)

# Direct inference with preprocessed input tensor
# Input shape: (batch, 6, H, W) for face+body or (batch, 3, H, W) for face-only
input_tensor = torch.randn(1, 6, 224, 224).to("cuda:0")
output = model.inference(input_tensor)

# Parse output: [gender_male_prob, gender_female_prob, normalized_age]
gender_probs = output[:, :2].softmax(-1)
age_normalized = output[:, 2]
age = age_normalized * (model.meta.max_age - model.meta.min_age) + model.meta.avg_age
print(f"Predicted age: {age.item():.1f}")
```

--------------------------------

### Data Reader Utilities

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Utilities for reading and parsing dataset annotations and determining input media types.

```APIDOC
## GET /data/annotations

### Description
Reads and parses CSV annotation files to extract bounding boxes and demographic metadata for training or evaluation.

### Method
GET

### Endpoint
/data/annotations

### Parameters
#### Query Parameters
- **annotation_file** (string) - Required - Path to the CSV annotation file
- **images_dir** (string) - Required - Directory containing source images
- **ignore_without_gt** (boolean) - Optional - Filter out entries without ground truth

### Response
#### Success Response (200)
- **bboxes_per_image** (object) - Dictionary mapping image paths to lists of PictureInfo objects
- **annot_type** (string) - The detected type of the annotation file
```

--------------------------------

### PersonAndFaceResult Class

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Demonstrates how to use the PersonAndFaceResult class to manage and access detection results, associate faces with persons, and set attributes like age and gender.

```APIDOC
## PersonAndFaceResult Class

Data structure for holding detection results with age/gender annotations.

```python
from mivolo.structures import PersonAndFaceResult
import cv2

# Results are typically created by detector
# results = detector.predict(image)

# Associate faces with their corresponding person bodies
results.associate_faces_with_persons()

# Access the face-to-person mapping
for face_idx, person_idx in results.face_to_person_map.items():
    if person_idx is not None:
        print(f"Face {face_idx} belongs to person {person_idx}")

# Persons without detected faces
unassigned = results.unassigned_persons_inds

# Set age/gender for detected objects
results.set_age(face_idx, 25.5)
results.set_gender(face_idx, "female", 0.95)

# Collect image crops for model input
image = cv2.imread("photo.jpg")
crops = results.collect_crops(image)

# Get paired face and body crops
(body_inds, body_crops), (face_inds, face_crops) = crops.get_faces_with_bodies(
    use_persons=True,
    use_faces=True
)

# Plot annotated image with results
annotated = results.plot(
    ages=True,
    genders=True,
    conf=True,
    gender_probs=False
)
cv2.imwrite("annotated.jpg", annotated)
```
```

--------------------------------

### Process Dataset Annotations and Input Types

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Provides utility functions to read CSV annotation files, iterate over image metadata, and identify file types (image vs video) for processing pipelines.

```python
from mivolo.data.data_reader import (
    read_csv_annotation_file,
    get_all_files,
    get_input_type,
    InputType,
    PictureInfo
)

bboxes_per_image, annot_type = read_csv_annotation_file(
    annotation_file="annotations/train.csv",
    images_dir="/path/to/images",
    ignore_without_gt=True
)

for image_path, picture_infos in bboxes_per_image.items():
    for info in picture_infos:
        print(f"Image: {image_path}")
        print(f"  Age: {info.age}, Gender: {info.gender}")
        print(f"  Face bbox: {info.bbox}")
        print(f"  Person bbox: {info.person_bbox}")
        print(f"  Has GT: {info.has_gt()}")

image_files = get_all_files("/path/to/images")

input_type = get_input_type("input.mp4")
if input_type == InputType.Video:
    print("Processing video")
elif input_type == InputType.Image:
    print("Processing image(s)")
```

--------------------------------

### PersonAndFaceResult Class Usage

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Illustrates the usage of the PersonAndFaceResult class for managing detection results, associating faces with persons, accessing mappings, setting attributes like age and gender, collecting image crops, and plotting annotated images.

```python
from mivolo.structures import PersonAndFaceResult
import cv2

# Results are typically created by detector
# results = detector.predict(image)

# Associate faces with their corresponding person bodies
results.associate_faces_with_persons()

# Access the face-to-person mapping
for face_idx, person_idx in results.face_to_person_map.items():
    if person_idx is not None:
        print(f"Face {face_idx} belongs to person {person_idx}")

# Persons without detected faces
unassigned = results.unassigned_persons_inds

# Set age/gender for detected objects
results.set_age(face_idx, 25.5)
results.set_gender(face_idx, "female", 0.95)

# Collect image crops for model input
image = cv2.imread("photo.jpg")
crops = results.collect_crops(image)

# Get paired face and body crops
(body_inds, body_crops), (face_inds, face_crops) = crops.get_faces_with_bodies(
    use_persons=True,
    use_faces=True
)

# Plot annotated image with results
annotated = results.plot(
    ages=True,
    genders=True,
    conf=True,
    gender_probs=False
)
cv2.imwrite("annotated.jpg", annotated)
```

--------------------------------

### Model Validation

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Guidance on evaluating model performance using benchmark datasets like UTKFace and Lagenda, including commands for validation and generating error histograms.

```APIDOC
## Model Validation

Evaluate model performance on benchmark datasets.

```bash
# Validate on UTKFace dataset
python3 eval_pretrained.py \
    --dataset_images /path/to/utk/images \
    --dataset_annotations /path/to/utk/annotation \
    --dataset_name utk \
    --split valid \
    --batch-size 512 \
    --checkpoint models/mivolo_imbd.pth.tar \
    --half \
    --with-persons \
    --device "cuda:0" \
    --results-file results.csv

# Supported datasets: utk, imdb, lagenda, fairface, adience, agedb, cacd

# Generate error histogram by age
python3 eval_pretrained.py \
    --dataset_images /path/to/lagenda/images \
    --dataset_annotations /path/to/lagenda/annotation \
    --dataset_name lagenda \
    --split valid \
    --checkpoint models/mivolo_imbd.pth.tar \
    --with-persons \
    --draw-hist
```
```

--------------------------------

### Video Processing with Temporal Tracking

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

Processes videos frame by frame, applying temporal tracking for smoothed age and gender predictions. Outputs a video file with annotations.

```python
import cv2
from mivolo.predictor import Predictor

class Config:
    detector_weights = "models/yolov8x_person_face.pt"
    checkpoint = "models/mivolo_imbd.pth.tar"
    device = "cuda:0"
    with_persons = True
    disable_faces = False
    draw = True

predictor = Predictor(Config(), verbose=True)

# Process video with tracking
fourcc = cv2.VideoWriter_fourcc(*"XVID")
out = cv2.VideoWriter("output.avi", fourcc, 30.0, (1920, 1080))

for detected_objects_history, frame in predictor.recognize_video("input.mp4"):
    # detected_objects_history contains smoothed predictions per tracked ID
    # {track_id: [(age1, gender1), (age2, gender2), ...]}
    out.write(frame)

out.release()
```

--------------------------------

### Predictor Class for Image/Video Inference

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

The main inference API for age and gender estimation on images and videos. It utilizes a YOLOv8 detector and the MiVOLO transformer model. Supports face-only, body-only, or combined analysis. Outputs detected objects and an annotated image.

```python
import cv2
import numpy as np
from mivolo.predictor import Predictor

class Config:
    def __init__(self):
        self.detector_weights = "models/yolov8x_person_face.pt"
        self.checkpoint = "models/mivolo_imbd.pth.tar"
        self.device = "cuda:0"
        self.with_persons = True  # Use body information
        self.disable_faces = False  # Use face information
        self.draw = True  # Draw results on image

config = Config()
predictor = Predictor(config, verbose=True)

# Process single image
image = cv2.imread("photo.jpg")
detected_objects, annotated_image = predictor.recognize(image)

# Access results for each detected person/face
for i in range(detected_objects.n_objects):
    age = detected_objects.ages[i]
    gender = detected_objects.genders[i]
    gender_score = detected_objects.gender_scores[i]
    print(f"Object {i}: Age={age:.1f}, Gender={gender} ({gender_score:.2%})")

# Save annotated result
cv2.imwrite("output.jpg", annotated_image)
```

--------------------------------

### Detector Class for Face and Person Detection

Source: https://context7.com/wildchlamydia/mivolo/llms.txt

A YOLOv8-based detector for identifying faces and persons within images. Supports configurable confidence and IoU thresholds, and includes optional tracking capabilities. Useful for pre-processing images before feeding them to the MiVOLO model.

```python
import cv2
from mivolo.model.yolo_detector import Detector

# Initialize detector
detector = Detector(
    weights="models/yolov8x_person_face.pt",
    device="cuda:0",
    half=True,
    verbose=False,
    conf_thresh=0.4,  # Detection confidence threshold
    iou_thresh=0.7    # NMS IoU threshold
)

# Detect faces and persons in image
image = cv2.imread("group_photo.jpg")
results = detector.predict(image)

print(f"Detected {results.n_faces} faces and {results.n_persons} persons")
```
Tokens:
Raw