MMHuman3D

https://mmhuman3d.readthedocs.io/en/latest/
Admin
MMHuman3D is an open-source toolbox for 3D human pose estimation, mesh recovery, and human body...
Tokens:20,760
Snippets:218
Trust Score:9.6
Update:6 months ago
Show doc for...
Tokens:
Raw
### Install MMCV-Full with CUDA 10.2 and PyTorch 1.8.0

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Example command to install mmcv-full compatible with CUDA 10.2 and PyTorch 1.8.0.

```bash
pip install "mmcv-full>=1.3.17,<=1.5.3" -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html
```

--------------------------------

### From-scratch setup script

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

A complete script to create a conda environment and install all dependencies from scratch.

```bash
# Create conda environment
conda create -n open-mmlab python=3.8 -y
conda activate open-mmlab

# Install ffmpeg
conda install ffmpeg

# Install PyTorch
conda install pytorch==1.8.0 torchvision cudatoolkit=10.2 -c pytorch -y

# Install PyTorch3D
conda install -c fvcore -c iopath -c conda-forge fvcore iopath -y
conda install -c bottler nvidiacub -y
conda install pytorch3d -c pytorch3d -y
# Alternatively from source in case of dependency conflicts
# git clone https://github.com/facebookresearch/pytorch3d.git
# cd pytorch3d
# pip install .
# cd ..

# Install mmcv-full
pip install "mmcv-full>=1.3.17,<1.6.0" -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.0/index.html

# Optional: install mmdetection & mmpose & mmtracking
pip install "mmdet<=2.25.1"
pip install "mmpose<=0.28.1"
pip install "mmcls<=0.23.2" "mmtrack<=0.13.0"

# Install mmhuman3d
git clone https://github.com/open-mmlab/mmhuman3d.git
cd mmhuman3d
pip install -v -e .
```

--------------------------------

### Install PyTorch 1.8.0 with CUDA 10.2

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Example command to install PyTorch version 1.8.0 with CUDA toolkit version 10.2 using conda.

```bash
conda install pytorch=1.8.0 torchvision cudatoolkit=10.2 -c pytorch
```

--------------------------------

### Install MMHuman3D

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Clone the repository and install the package in editable mode.

```bash
git clone https://github.com/open-mmlab/mmhuman3d.git
cd mmhuman3d
```

```bash
pip install -v -e .  # or "python setup.py develop"
```

--------------------------------

### Install MMCV-Full for GPU (Specific Versions)

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Install mmcv-full for GPU environments, specifying compatible PyTorch and CUDA versions. Ensure the versions in the URL match your setup.

```bash
pip install "mmcv-full>=1.3.17,<=1.5.3" -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html
```

--------------------------------

### Install MMPose

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Install MMPose via pip or build from source for code modification.

```bash
pip install "mmpose<=0.28.1"
```

```bash
git clone https://github.com/open-mmlab/mmpose.git -b v0.28.1
cd mmpose
pip install -r requirements.txt
pip install -v -e .
```

--------------------------------

### Install MMTracking

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Install MMTracking via pip or build from source for code modification.

```bash
pip install "mmcls<=0.23.2" "mmtrack<=0.13.0"
```

```bash
git clone https://github.com/open-mmlab/mmtracking.git -b v0.13.0
cd mmtracking
pip install -r requirements/build.txt
pip install -v -e .  # or "python setup.py develop"
```

--------------------------------

### Install PyTorch3D from Source

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Clone the PyTorch3D repository and install it from source using pip. This method is an alternative if conda installation leads to dependency conflicts.

```bash
git clone https://github.com/facebookresearch/pytorch3d.git
cd pytorch3d
pip install .
cd ..
```

--------------------------------

### Install MMCV-Full for CPU

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Install the mmcv-full package for CPU-only environments. Replace {torch_version} with your specific PyTorch version.

```bash
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cpu/{torch_version}/index.html
```

--------------------------------

### Install MMDetection

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Install MMDetection via pip or build from source for code modification.

```bash
pip install "mmdet<=2.25.1"
```

```bash
git clone https://github.com/open-mmlab/mmdetection.git -b v2.25.1
cd mmdetection
pip install -r requirements/build.txt
pip install -v -e .
```

--------------------------------

### Run Single-Person Inference

Source: https://mmhuman3d.readthedocs.io/en/latest/getting_started.html

Command template and example for estimating SMPL parameters for a single person.

```bash
python demo/estimate_smpl.py \
    ${MMHUMAN3D_CONFIG_FILE} \
    ${MMHUMAN3D_CHECKPOINT_FILE} \
    --single_person_demo \
    --det_config ${MMDET_CONFIG_FILE} \
    --det_checkpoint ${MMDET_CHECKPOINT_FILE} \
    --input_path ${VIDEO_PATH_OR_IMG_PATH} \
    [--show_path ${VIS_OUT_PATH}] \
    [--output ${RESULT_OUT_PATH}] \
    [--smooth_type ${SMOOTH_TYPE}] \
    [--speed_up_type ${SPEED_UP_TYPE}] \
    [--draw_bbox] \
```

```bash
python demo/estimate_smpl.py \
    configs/hmr/resnet50_hmr_pw3d.py \
    data/checkpoints/resnet50_hmr_pw3d.pth \
    --single_person_demo \
    --det_config demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
    --det_checkpoint https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
    --input_path  demo/resources/single_person_demo.mp4 \
    --show_path vis_results/single_person_demo.mp4 \
    --output demo_result \
    --smooth_type savgol \
    --speed_up_type deciwatch \
    --draw_bbox
```

--------------------------------

### Install PyTorch3D via Conda

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Install the PyTorch3D library using the conda package manager.

```bash
conda install pytorch3d -c pytorch3d
```

--------------------------------

### Run Multi-Person Inference

Source: https://mmhuman3d.readthedocs.io/en/latest/getting_started.html

Command template and example for estimating SMPL parameters for multiple people using tracking.

```bash
python demo/estimate_smpl.py \
    ${MMHUMAN3D_CONFIG_FILE} \
    ${MMHUMAN3D_CHECKPOINT_FILE} \
    --multi_person_demo \
    --tracking_config ${MMTRACKING_CONFIG_FILE} \
    --input_path ${VIDEO_PATH_OR_IMG_PATH} \
    [--show_path ${VIS_OUT_PATH}] \
    [--output ${RESULT_OUT_PATH}] \
    [--smooth_type ${SMOOTH_TYPE}] \
    [--speed_up_type ${SPEED_UP_TYPE}] \
    [--draw_bbox]
```

```bash
python demo/estimate_smpl.py \
    configs/hmr/resnet50_hmr_pw3d.py \
    data/checkpoints/resnet50_hmr_pw3d.pth \
    --multi_person_demo \
    --tracking_config demo/mmtracking_cfg/deepsort_faster-rcnn_fpn_4e_mot17-private-half.py \
    --input_path  demo/resources/multi_person_demo.mp4 \
    --show_path vis_results/multi_person_demo.mp4 \
    --smooth_type savgol \
    --speed_up_type deciwatch \
    [--draw_bbox]
```

--------------------------------

### Install ffmpeg with Conda

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Install ffmpeg using conda, which also builds the libx264 library automatically.

```bash
conda install ffmpeg
```

--------------------------------

### Verify PyTorch3D Installation (Linux)

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Verify the PyTorch3D installation on Linux by running a Python script that imports key components and prints their versions or types. This checks basic functionality.

```bash
echo "import pytorch3d;print(pytorch3d.__version__);     from pytorch3d.renderer import MeshRenderer;print(MeshRenderer);    from pytorch3d.structures import Meshes;print(Meshes);    from pytorch3d.renderer import cameras;print(cameras);    from pytorch3d.transforms import Transform3d;print(Transform3d);"|python
```

--------------------------------

### Visualize 3D Keypoints

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_keypoints.html

Examples for visualizing single and multi-person 3D keypoint data.

```python
visualize_kp3d(kp3d=kp3d, data_source='smplx', output_path='some_video.mp4')
```

```python
kp3d = np.concatenate([kp3d_1[:, np.newaxis], kp3d_2[:, np.newaxis]], axis=1)
```

--------------------------------

### Install PyTorch3D Dependencies

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Install necessary dependencies for PyTorch3D from conda channels. This includes fvcore, iopath, and nvidiacub.

```bash
conda install -c fvcore -c iopath -c conda-forge fvcore iopath -y
conda install -c bottler nvidiacub -y
```

--------------------------------

### Create Conda Environment

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Create and activate a conda virtual environment for managing project dependencies. Python 3.8 is used in this example.

```bash
conda create -n open-mmlab python=3.8 -y
conda activate open-mmlab
```

--------------------------------

### Compile MMCV from Source

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Compile mmcv from source to install the mmcv-full package, which includes CUDA operations. This is an alternative to installing pre-built packages.

```bash
git clone https://github.com/open-mmlab/mmcv.git -b v1.5.3
cd mmcv
MMCV_WITH_OPS=1 pip install -e .

```

--------------------------------

### Visualize SMPL with HMR/SPIN Camera

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_smpl.html

Render SMPL poses using HMR/SPIN camera parameters. These examples demonstrate passing poses/betas, vertices, or 2D keypoints.

```python
import pickle
from mmhuman3d.core.visualization.visualize_smpl import visualize_smpl_hmr
gender = 'female'
focal_length = 5000
det_width = 224
det_height = 224

# you can pass smpl poses & betas & gender
body_model_config = dict(
    type='smpl', model_path=model_path, gender=gender)
visualize_smpl_hmr(
    poses=poses,
    betas=betas,
    bbox=bbox,
    body_model_config=body_model_config,
    focal_length=focal_length,
    det_width=det_width,
    det_height=det_height,
    T=cam_translation,
    output_path='hmr_demo.mp4',
    origin_frames=image_folder,
    resolution=(1024, 1024))

# or you can pass verts
body_model_config = dict(
    type='smpl', model_path=model_path, gender=gender)
visualize_smpl_hmr(
    verts=verts,
    bbox=bbox,
    focal_length=focal_length,
    body_model_config=body_model_config,
    det_width=det_width,
    det_height=det_height,
    T=cam_translation,
    output_path='hmr_demo.mp4',
    origin_frames=image_folder,
    resolution=(1024, 1024))

# you can also pass kp2d in replace of bbox.
body_model_config = dict(
    type='smpl', model_path=model_path, gender=gender)
visualize_smpl_hmr(
    verts=verts,
    body_model_config=body_model_config,
    kp2d=kp2d,
    focal_length=focal_length,
    det_width=det_width,
    det_height=det_height,
    T=cam_translation,
    output_path='hmr_demo.mp4',
    origin_frames=image_folder,
    resolution=(1024, 1024))
```

--------------------------------

### Visualize 3D Keypoints

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_keypoints.html

Use this function to visualize 3D keypoints. Specify the output path for a video file or a folder for individual frames. Set output_path to None if return_array is True to get an array of frames.

```python
visualize_kp3d(kp3d=kp3d, data_source='smplx', output_path='some_video.mp4')
```

--------------------------------

### Install PyTorch and torchvision

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Install PyTorch and torchvision using conda, specifying the desired torch_version and cudatoolkit version. Ensure your compilation and runtime CUDA versions match.

```bash
conda install pytorch={torch_version} torchvision cudatoolkit={cu_version} -c pytorch
```

--------------------------------

### Select Keypoints by Shape

Source: https://mmhuman3d.readthedocs.io/en/latest/human_data.html

Selects a portion of keypoints data based on a target shape. Use `shape=[10, -1, -1]` to get the first 10 frames.

```python
first_ten_frames = human_data.get_value_in_shape('keypoints2d', shape=[10, -1, -1])
```

--------------------------------

### Get Compressed Keypoints Value

Source: https://mmhuman3d.readthedocs.io/en/latest/human_data.html

Retrieves the compressed raw value of keypoints. Accessing via `[]` returns keypoints padded with zeros.

```python
keypoints2d_human_data = human_data.get_raw_value('keypoints2d')
print(keypoints2d_human_data.shape)  # [frame_num, 127, 3]
keypoints2d_human_data = human_data['keypoints2d']
print(keypoints2d_human_data.shape)  # [frame_num, 190, 3]
```

--------------------------------

### Build Cameras via Registry

Source: https://mmhuman3d.readthedocs.io/en/latest/cameras.html

Shows how to use the build_cameras utility to initialize cameras using dictionaries, supporting both direct matrix input and parameter-based initialization.

```python
from mmhuman3d.core.cameras import build_cameras

# Initialize a perspective camera with given K, R, T matrix.
# It is recommended that the batches of K, R, T either the same or be 1.
K = torch.eye(4, 4)[None]
R = torch.eye(3, 3)[None]
T = torch.zeros(10, 3)

height, width = 1000
cam1 = build_cameras(
    dict(
        type='PerspectiveCameras',
        K=K,
        R=R,
        T=T,
        in_ndc=True,
        image_size=(height, width),
        convention='opencv',
        ))

# This is the same as:
cam2 = PerspectiveCameras(
        K=K,
        R=R,
        T=T,
        in_ndc=True,
        image_size=1000, # single number represents square images.
        convention='opencv',
        )
assert cam1.K.shape == cam2.K.shape == (10, 4, 4)
assert cam1.R.shape == cam2.R.shape == (10, 3, 3)
assert cam1.T.shape == cam2.T.shape == (10, 3)

# Initialize a perspective camera with specific `image_size`, `principal_points`, `focal_length`.
# `in_ndc = False` means the intrinsic matrix `K` defined in screen space. The `focal_length` and `principal_point` in `K` is defined in scale of pixels. This `principal_points` is (500, 500) pixels and `focal_length` is 1000 pixels.
cam = build_cameras(
    dict(
        type='PerspectiveCameras',
        in_ndc=False,
        image_size=(1000, 1000),
        principal_points=(500, 500),
        focal_length=1000,
        convention='opencv',
        ))

assert (cam.K[0] == torch.Tensor([[1000., 0.,  500.,    0.],
                                  [0., 1000.,  500.,    0.],
                                  [0.,    0.,    0.,    1.],
                                  [0.,    0.,    1.,    0.]]).view(4, 4)).all()

# Initialize a weakperspective camera with given K, R, T. weakperspective camera support `in_ndc = True` only.
cam = build_cameras(
    dict(
        type='WeakPerspectiveCameras',
        K=K,
        R=R,
        T=T,
        image_size=(1000, 1000)
        ))

# If no `K`, `R`, `T` information provided
# Initialize a `in_ndc` perspective camera with default matrix.
cam = build_cameras(
    dict(
        type='PerspectiveCameras',
        in_ndc=True,
        image_size=(1000, 1000),
        ))
# Then convert it to screen. This operation requires `image_size`.
cam.to_screen_()
```

--------------------------------

### Run Webcam Demo

Source: https://mmhuman3d.readthedocs.io/en/latest/getting_started.html

Execute the webcam demo script to estimate SMPL parameters from a camera feed or video file. Use --output to save visualizations, --synchronous for temporal alignment, or --cam-id to specify a video file for offline processing.

```bash
python demo/webcam_demo.py

```

--------------------------------

### Initialize and Slice PerspectiveCameras

Source: https://mmhuman3d.readthedocs.io/en/latest/cameras.html

Demonstrates initializing a camera with K, R, T matrices and performing slicing or batch concatenation operations.

```python
from mmhuman3d.core.cameras import PerspectiveCameras
import torch
K = torch.eye(4, 4)[None]
R = torch.eye(3, 3)[None]
T = torch.zeros(100, 3)
# Batch of K, R, T should all be the same or some of them could be 1. The final batch size will be the biggest one.
cam = PerspectiveCameras(K=K, R=R, T=T)
assert cam.R.shape == (100, 3, 3)
assert cam.K.shape == (100, 4, 4)
assert cam.T.shape == (100, 3)
assert (cam[:10].K == cam.K[:10]).all()
```

--------------------------------

### Import Camera Transformation Utilities

Source: https://mmhuman3d.readthedocs.io/en/latest/human_data.html

Import the necessary functions for transforming annotations from world space to OpenCV camera space.

```python
from mmhuman3d.models.body_models.utils import transform_to_camera_frame, batch_transform_to_camera_frame
```

--------------------------------

### Get camera plane normal

Source: https://mmhuman3d.readthedocs.io/en/latest/cameras.html

Retrieves the normalized normal tensor pointing out of the camera plane from the camera center.

```python
normals = cameras.get_camera_plane_normals()
```

--------------------------------

### Train Model on Single/Multiple GPUs

Source: https://mmhuman3d.readthedocs.io/en/latest/getting_started.html

Initiate model training using the train script. Provide the configuration file path and working directory. The --no-validate flag prevents evaluation during training. Use --gpus to specify the number of GPUs.

```bash
python tools/train.py ${CONFIG_FILE} ${WORK_DIR} --no-validate

```

```bash
python tools/train.py ${CONFIG_FILE} ${WORK_DIR} --gpus 1 --no-validate

```

--------------------------------

### Convert Datasets Command

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Use this script to convert raw datasets into the required npz format for training.

```bash
python tools/convert_datasets.py \
  --datasets <dataset-name> \
  --root_path data/datasets \
  --output_path data/preprocessed_datasets
```

--------------------------------

### Visualize SMPL with VIBE Camera

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_smpl.html

Render SMPL poses using VIBE camera parameters. You can provide either predicted camera parameters or original camera parameters.

```python
import pickle
from mmhuman3d.core.visualization.visualize_smpl import visualize_smpl_vibe
with open('vibe_output.pkl', 'rb') as f:
    d = pickle.load(f, encoding='latin1')
poses = d[1]['pose']
orig_cam = d[1]['orig_cam']
pred_cam = d[1]['pred_cam']
bbox = d[1]['bboxes']
gender = 'female'

# pass pred_cam & bbox
body_model_config = dict(
    type='smpl', model_path=model_path, gender=gender)
visualize_smpl_vibe(
    poses=poses,
    betas=betas,
    body_model_config=body_model_config,
    pred_cam=pred_cam,
    bbox=bbox,
    output_path='vibe_demo.mp4',
    origin_frames='sample_video.mp4',
    resolution=(1024, 1024))

# or pass orig_cam
body_model_config = dict(
    type='smpl', model_path=model_path, gender=gender)
visualize_smpl_vibe(
    poses=poses,
    betas=betas,
    body_model_config=body_model_config,
    orig_cam=orig_cam,
    output_path='vibe_demo.mp4',
    origin_frames='sample_video.mp4',
    resolution=(1024, 1024))
```

--------------------------------

### Train Model with Slurm

Source: https://mmhuman3d.readthedocs.io/en/latest/getting_started.html

Train models on a Slurm cluster using the provided script. Configure the Slurm partition, job name, configuration file, working directory, and GPU count. Optional arguments like --resume-from and --no-validate can be used.

```bash
./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} ${CONFIG_FILE} ${WORK_DIR} ${GPU_NUM} --no-validate

```

```bash
./tools/slurm_train.sh my_partition my_job configs/hmr/resnet50_hmr_pw3d.py work_dirs/hmr 8 --no-validate

```

--------------------------------

### Define a new keypoints convention

Source: https://mmhuman3d.readthedocs.io/en/latest/customize_keypoints_convention.html

Create a new file containing a list of keypoint names specific to the dataset convention.

```python
AGORA_KEYPOINTS = [
  'pelvis',
  'left_hip',
  'right_hip'
  ...
]
```

--------------------------------

### Visualize SMPL and SMPL-X Poses

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_smpl.html

Use these snippets to visualize pose tensors or arrays. The configuration dictionary specifies the model type as either 'smpl' or 'smplx'.

```python
from mmhuman3d.core.visualization.visualize_smpl import visualize_smpl_pose
body_model_config = dict(
    type='smpl', model_path=model_path)
visualize_smpl_pose(
    poses=poses,
    output_path='smpl.mp4',
    resolution=(1024, 1024))
```

```python
body_model_config = dict(
    type='smplx', model_path=model_path)
visualize_smpl_pose(
    poses=poses,
    body_model_config=body_model_config,
    output_path='smplx.mp4',
    resolution=(1024, 1024))
```

--------------------------------

### Register a new convention in the factory

Source: https://mmhuman3d.readthedocs.io/en/latest/customize_keypoints_convention.html

Import the new convention module and add its keypoints list to the KEYPOINTS_FACTORY dictionary.

```python
# add import
from mmhuman3d.core.conventions.keypoints_mapping import (
    agora,
    ...
)

# add to factory
KEYPOINTS_FACTORY = {
    'agora': agora.AGORA_KEYPOINTS,
    ...
}
```

--------------------------------

### Visualize 2D Keypoints on Backgrounds

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_keypoints.html

Overlay keypoints onto numpy array backgrounds or existing video/image frames.

```python
from mmhuman3d.core.conventions.keypoints_mapping import convert_kps
from mmhuman3d.core.visualization.visualize_keypoints2d import visualize_kp2d

background = np.random.randint(low=0, high=255, shape=(10, 1024, 1024, 4))
# multi_person, shape is (num_person, num_joints, 2)
out_image = visualize_kp2d(
    kp2d=kp2d, image_array=background, data_source='coco_wholebody', return_array=True)
```

```python
frame_list = ['im1.png', 'im2.png', ...]
visualize_kp2d(
    kp2d_coco_wholebody,
    data_source='coco_wholebody',
    output_path='some_video.mp4',
    resolution=(1024, 1024),
    frame_list=frame_list)

origin_frames = 'some_folder'
visualize_kp2d(
    kp2d_coco_wholebody,
    data_source='coco_wholebody',
    output_path='some_video.mp4',
    resolution=(1024, 1024),
    origin_frames=origin_frames)

origin_frames = 'some.mp4'
array = visualize_kp2d(
    kp2d_coco_wholebody,
    data_source='coco_wholebody',
    output_path='some_video.mp4',
    resolution=(1024, 1024),
    return_array=True,
    origin_frames=origin_frames)
```

--------------------------------

### Convert Camera Conventions

Source: https://mmhuman3d.readthedocs.io/en/latest/cameras.html

Converts camera parameters (K, R, T) between different conventions (e.g., 'opencv' to 'pytorch3d') and NDC spaces. Handles optional inputs for K, R, and T, setting defaults if None.

```python
from mmhuman3d.core.conventions.cameras import convert_cameras
import torch

K = torch.eye(4, 4)[None]
R = torch.eye(3, 3)[None]
T = torch.zeros(10, 3)
height, width = 1080, 1920
K, R, T = convert_cameras(
    K=K,
    R=R,
    T=T,
    in_ndc_src=False,
    in_ndc_dst=True,
    resolution_src=(height, width),
    convention_src='opencv',
    convention_dst='pytorch3d')
```

--------------------------------

### Visualize SMPL with OpenCV Camera

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_smpl.html

Render SMPL poses using standard OpenCV intrinsic (K) and extrinsic (R, T) matrices.

```python
from mmhuman3d.core.visualization.visualize_smpl import visualize_smpl_calibration
body_model_config = dict(
    type='smpl', model_path=model_path, gender=gender)
visualize_smpl_calibration(
    poses=poses,
    betas=betas,
    transl=transl,
    body_model_config=body_model_config,
    K=K,
    R=R,
    T=T,
    output_path='opencv.mp4',
    origin_frames='bg_video.mp4',
    resolution=(1024, 1024))
```

--------------------------------

### Dump and Load HumanData by Pickle

Source: https://mmhuman3d.readthedocs.io/en/latest/human_data.html

Uses `dump_by_pickle` and `load_by_pickle` for file operations when `numpy.savez_compressed()` would raise an error due to large instance size.

```python
# This is a conceptual example, actual dump/load methods are not shown here.
# Use dump_by_pickle and load_by_pickle for large HumanData instances.
```

--------------------------------

### Verify PyTorch3D Torus Generation (Linux)

Source: https://mmhuman3d.readthedocs.io/en/latest/install.html

Verify PyTorch3D's ability to generate a torus mesh and print its vertices on Linux. This confirms CUDA device compatibility and mesh generation capabilities.

```bash
echo "import torch;device=torch.device('cuda');    from pytorch3d.utils import torus;    Torus = torus(r=10, R=20, sides=100, rings=100, device=device);    print(Torus.verts_padded());"|python
```

--------------------------------

### Preprocessed Dataset Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Expected directory layout for preprocessed datasets within the MMHuman3D project.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    ├── datasets
    └── preprocessed_datasets
        ├── curated_fits_train.npz
        ├── ehf_val.npz
        ├── ffhq_flame_train.npz
        ├── freihand_test.npz
        ├── freihand_train.npz
        ├── freihand_val.npz
        ├── h36m_smplx_train.npz
        ├── pw3d_test.npz
        ├── spin_smplx_train.npz
        └── stirling_ESRC3D_HQ.npz
```

--------------------------------

### LSP Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Expected file hierarchy for the LSP dataset within the MMHuman3D project.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    └── datasets
        └── lsp
            ├── images
            |  ├── im0001.jpg
            |  ├── im0002.jpg
            |  └── ...
            └── joints.mat
```

--------------------------------

### ExPose Dataset Names

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

List of dataset names required for ExPose training.

```text
h36m, EHF, FreiHand, 3DPW, stirling, spin_in_smplx, ffhq, ExPose_curated_fits
```

--------------------------------

### Evaluate Model on Single/Multiple GPUs

Source: https://mmhuman3d.readthedocs.io/en/latest/getting_started.html

Run model evaluation using the test script. Specify the configuration file, working directory, checkpoint, and desired metrics. This command is used for evaluating performance on a single GPU or multiple GPUs.

```bash
python tools/test.py ${CONFIG} --work-dir=${WORK_DIR} ${CHECKPOINT} --metrics=${METRICS}

```

```bash
python tools/test.py configs/hmr/resnet50_hmr_pw3d.py --work-dir=work_dirs/hmr work_dirs/hmr/latest.pth --metrics pa-mpjpe mpjpe

```

--------------------------------

### Dump and Load HumanData to NPZ

Source: https://mmhuman3d.readthedocs.io/en/latest/human_data.html

Saves the HumanData instance to a compressed .npz file using `dump()` and loads it back using `load()`. Suitable for most compression needs.

```python
# save
human_data.dump('./dumped_human_data.npz')
# load
another_human_data = HumanData()
another_human_data.load('./dumped_human_data.npz')
```

--------------------------------

### Visualize 2D Keypoints

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_keypoints.html

Basic usage for visualizing 2D keypoints by specifying the data source and output resolution.

```python
from mmhuman3d.core.visualization.visualize_keypoints2d import visualize_kp2d

visualize_kp2d(
    kp2d_coco_wholebody,
    data_source='coco_wholebody',
    output_path='some_video.mp4',
    resolution=(1024, 1024))
```

--------------------------------

### Configure HumanData Strict Mode

Source: https://mmhuman3d.readthedocs.io/en/latest/human_data.html

Manage custom keys by toggling strict mode during initialization or on an existing instance.

```python
human_data = HumanData.new(key_strict=False)
human_data['video_path'] = 'test.mp4'
```

```python
human_data = HumanData.fromfile('human_data.npz')
key_strict = human_data.get_key_strict()
human_data.set_key_strict(not key_strict)
```

--------------------------------

### Human3.6M MoShed Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Expected file system layout for the MoShed data directory.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    └── datasets
        └── h36m_mosh
            ├── annot
            ├── S1
            |   ├── images
            |   |    ├── Directions 1_cam0_aligned.pkl
            |   |    ├── Directions 1_cam1_aligned.pkl
            |   |    └── ...
            ├── S5
            ├── S6
            ├── S7
            ├── S8
            ├── S9
            └── S11
```

--------------------------------

### LSPET Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Expected file hierarchy for the LSPET dataset.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    └── datasets
        └── lspet
            ├── im00001.jpg
            ├── im00002.jpg
            ├── im00003.jpg
            ├── ...
            └── joints.mat
```

--------------------------------

### Convert keypoints between conventions

Source: https://mmhuman3d.readthedocs.io/en/latest/customize_keypoints_convention.html

Use the convert_kps function to map keypoints between different conventions and generate a retention mask.

```python
new_kps, mask = convert_kps(smplx_keypoints, src='coco', dst='NEW_CONVENTION')
```

```python
new_kps, mask = convert_kps(smplx_keypoints, src='NEW_CONVENTION', dst='human_data')
```

--------------------------------

### Visualize T-pose

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_smpl.html

Visualize a T-pose SMPL model when specific pose data is unavailable or global orientation is missing.

```python
import torch
from mmhuman3d.core.visualization.visualize_smpl import visualize_T_pose
body_model_config = dict(
    type='smpl', model_path=model_path)
visualize_T_pose(
    num_frames=100,
    body_model_config=body_model_config,
    output_path='smpl_tpose.mp4',
    orbit_speed=(1, 0.5),
    resolution=(1024, 1024))
```

--------------------------------

### Set Keypoints Data in Compressed Mode

Source: https://mmhuman3d.readthedocs.io/en/latest/human_data.html

Allows editing keypoints data when in 'keypoints_compressed' mode. Data can be set either as padded or directly as compressed.

```python
padded_keypoints2d = np.zeros(shape=[100, 190, 3])
human_data['keypoints2d'] = padded_keypoints2d  # [frame_num, 190, 3]
compressed_keypoints2d = np.zeros(shape=[100, 127, 3])
human_data.set_raw_value('keypoints2d', compressed_keypoints2d)  # [frame_num, 127, 3]
```

--------------------------------

### MPI-INF-3DHP Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Expected file hierarchy for the MPI-INF-3DHP dataset.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    └── datasets
        └── mpi_inf_3dhp
            ├── mpi_inf_3dhp_test_set
            │   ├── TS1
            │   ├── TS2
            │   ├── TS3
            │   ├── TS4
            │   ├── TS5
            │   └── TS6
            ├── S1
            │   ├── Seq1
            │   └── Seq2
            ├── S2
            │   ├── Seq1
            │   └── Seq2
            ├── S3
            │   ├── Seq1
            │   └── Seq2
            ├── S4
            │   ├── Seq1
            │   └── Seq2
            ├── S5
            │   ├── Seq1
            │   └── Seq2
            ├── S6
            │   ├── Seq1
            │   └── Seq2
            ├── S7
            │   ├── Seq1
            │   └── Seq2
            └── S8
                ├── Seq1
                └── Seq2
```

--------------------------------

### MPII Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Expected file hierarchy for the MPII dataset.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    └── datasets
        └── mpii
            |── train.h5
            └── images
                |── 000001163.jpg
                |── 000003072.jpg
                └── ...
```

--------------------------------

### HybrIK Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Expected file system layout for HybrIK annotation files.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    └── datasets
        └── hybrik_data
            ├── Sample_5_train_Human36M_smpl_leaf_twist_protocol_2.json
            ├── Sample_20_test_Human36M_smpl_protocol_2.json
            ├── 3DPW_test_new.json
            ├── annotation_mpi_inf_3dhp_train_v2.json
            └── annotation_mpi_inf_3dhp_test.json
```

--------------------------------

### Preprocessed Dataset Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

The expected file system layout for preprocessed npz files within the project directory.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    ├── datasets
    └── preprocessed_datasets
        ├── coco_2014_train.npz
        ├── h36m_train.npz or h36m_mosh_train.npz (if mosh is available)
        ├── lspet_train.npz
        ├── lsp_train.npz
        ├── mpi_inf_3dhp_train.npz
        ├── mpii_train.npz
        └── pw3d_test.npz
```

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    ├── datasets
    └── preprocessed_datasets
        ├── spin_coco_2014_train.npz
        ├── h36m_train.npz or h36m_mosh_train.npz (if mosh is available)
        ├── spin_lsp_train.npz
        ├── spin_lspet_train.npz
        ├── spin_mpi_inf_3dhp_train.npz
        ├── spin_mpii_train.npz
        └── spin_pw3d_test.npz
```

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    ├── datasets
    └── preprocessed_datasets
        ├── vibe_insta_variety.npz
        ├── vibe_mpi_inf_3dhp_train.npz
        └── vibe_pw3d_test.npz
```

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    ├── datasets
    └── preprocessed_datasets
        ├── hybriK_coco_2017_train.npz
        ├── hybrik_h36m_train.npz
        ├── hybrik_mpi_inf_3dhp_train.npz
        └── hybrik_pw3d_test.npz
```

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    ├── datasets
    └── preprocessed_datasets
        ├── h36m_mosh_train.npz
        ├── h36m_train.npz
        ├── mpi_inf_3dhp_train.npz
        ├── eft_mpii.npz
        ├── eft_lspet.npz
        ├── eft_coco_all.npz
        └── pw3d_test.npz
```

--------------------------------

### Human3.6M Dataset Configuration

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Configuration dictionary for the H36mConverter, including options for image extraction and MoShed data.

```python
h36m_p1=dict(
    type='H36mConverter',
    modes=['train', 'valid'],
    protocol=1,
    extract_img=True, # set to true to extract images from raw videos
    prefix='h36m'),
```

```python
h36m_p1=dict(
    type='H36mConverter',
    modes=['train', 'valid'],
    protocol=1,
    extract_img=True,  # this is to specify you want to extract images from videos
    prefix='h36m'),
```

```python
h36m_p1=dict(
    type='H36mConverter',
    modes=['train', 'valid'],
    protocol=1,
    extract_img=True,  # this is to specify you want to extract images from videos
    mosh_dir='data/datasets/h36m_mosh', # supply the directory to the mosh if available
    prefix='h36m'),
```

--------------------------------

### MPI-INF-3DHP Configuration

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Configuration snippet for enabling image extraction in the MPI-INF-3DHP dataset.

```python
mpi_inf_3dhp=dict(
  type='MpiInf3dhpConverter',
  modes=['train', 'test'],
  extract_img=True),  # this is to specify you want to extract images from videos
```

--------------------------------

### Human3.6M Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Expected file system layout for the processed Human3.6M dataset.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    └── datasets
        └── h36m
            ├── annot
            ├── S1
            |   ├── images
            |   |    |── S1_Directions_1.54138969
            |   |    |  ├── S1_Directions_1.54138969_00001.jpg
            |   |    |  ├── S1_Directions_1.54138969_00006.jpg
            |   |    |  └── ...
            |   |    └── ...
            |   ├── MyPoseFeatures
            |   |    |── D2Positions
            |   |    └── D3_Positions_Mono
            |   ├── MySegmentsMat
            |   |    └── ground_truth_bs
            |   └── Videos
            |        |── Directions 1.54138969.mp4
            |        |── Directions 1.55011271.mp4
            |        └── ...
            ├── S5
            ├── S6
            ├── S7
            ├── S8
            ├── S9
            ├── S11
            └── metadata.xml
```

--------------------------------

### Convert between weak perspective and perspective

Source: https://mmhuman3d.readthedocs.io/en/latest/cameras.html

Requires zmean values. If perspective is not in NDC, resolution must be provided.

```python
from mmhuman3d.core.conventions.cameras import (
    convert_perspective_to_weakperspective,
    convert_weakperspective_to_perspective)

K = convert_perspective_to_weakperspective(
    K, zmean, in_ndc=False, resolution, convention='opencv')
K = convert_weakperspective_to_perspective(
    K, zmean, in_ndc=False, resolution, convention='pytorch3d')
```

--------------------------------

### AGORA Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/preprocess_dataset.html

Required folder structure for the AGORA dataset.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
   └── datasets
       └── agora
           ├── camera_dataframe # smplx annotations
           │   ├── train_0_withjv.pkl
           │   ├── validation_0_withjv.pkl
           │   └── ...
           ├── camera_dataframe_smpl # smpl annotations
           │   ├── train_0_withjv.pkl
           │   ├── validation_0_withjv.pkl
           │   └── ...
           ├── images
           │   ├── train
           │   │   ├── ag_trainset_3dpeople_bfh_archviz_5_10_cam00_00000_1280x720.png
           │   │   ├── ag_trainset_3dpeople_bfh_archviz_5_10_cam00_00001_1280x720.png
           │   │   └── ...
           │   ├── validation
           │   └── test
           ├── smpl_gt
           │   ├── trainset_3dpeople_adults_bfh
           │   │   ├── 10004_w_Amaya_0_0.mtl
           │   │   ├── 10004_w_Amaya_0_0.obj
           │   │   ├── 10004_w_Amaya_0_0.pkl
           │   │   └── ...
           │   └── ...
           └── smplx_gt
                   ├── 10004_w_Amaya_0_0.obj
                   ├── 10004_w_Amaya_0_0.pkl
                   └── ...
```

--------------------------------

### Visualize 2D Keypoints with Masking

Source: https://mmhuman3d.readthedocs.io/en/latest/visualize_keypoints.html

Use a mask to exclude invalid keypoints when converting between different conventions.

```python
from mmhuman3d.core.conventions.keypoints_mapping import convert_kps
from mmhuman3d.core.visualization.visualize_keypoints2d import visualize_kp2d

kp2d_smpl, mask = convert_kps(kp2d_coco_wholebody, src='coco_wholebody', dst='smpl')
visualize_kp2d(
    kp2d_smpl,
    mask=mask,
    output_path='some_video.mp4',
    resolution=(1024, 1024))
```

--------------------------------

### Visualize MultiHumanData mapping

Source: https://mmhuman3d.readthedocs.io/en/latest/human_data.html

Conceptual representation of how frame_range pointers map images to multiple human data instances.

```text
image_0  ----> human_0      <--- frame_range[0][0]
         -       .
          -      .
           --> human_(n-1)  <--- frame_range[0][0] + (n-1)
            -> human_n      <--- frame_range[0][1]
    .
    .
    .


image_n  ----> human_0     <--- frame_range[n][0]
         -       .
          -      .
           --> human_(n-1)  <--- frame_range[n][0] + (n-1)
            -> human_n     <--- frame_range[n][1]
```

--------------------------------

### Body Model Directory Structure

Source: https://mmhuman3d.readthedocs.io/en/latest/getting_started.html

Required file structure for body models within the MMHuman3D project directory.

```text
mmhuman3d
├── mmhuman3d
├── docs
├── tests
├── tools
├── configs
└── data
    └── body_models
        ├── J_regressor_extra.npy
        ├── J_regressor_h36m.npy
        ├── smpl_mean_params.npz
        └── smpl
            ├── SMPL_FEMALE.pkl
            ├── SMPL_MALE.pkl
            └── SMPL_NEUTRAL.pkl
```

--------------------------------

### Project 3D coordinates to screen

Source: https://mmhuman3d.readthedocs.io/en/latest/cameras.html

Projects 3D points into screen space and extracts the XY coordinates.

```python
points_xydepth = cameras.transform_points_screen(points)
points_xy = points_xydepth[..., :2]
```

--------------------------------

### Rename Body Model File

Source: https://mmhuman3d.readthedocs.io/en/latest/getting_started.html

Rename the neutral SMPL model file to the required format.

```bash
mv basicModel_neutral_lbs_10_207_0_v1.0.0.pkl SMPL_NEUTRAL.pkl
```

--------------------------------

### Convert between 3x3 and 4x4 intrinsic matrices

Source: https://mmhuman3d.readthedocs.io/en/latest/cameras.html

Utility functions for resizing camera intrinsic matrices.

```python
from mmhuman3d.core.conventions.cameras import (convert_K_3x3_to_4x4,
                                                convert_K_4x4_to_3x3)

K = convert_K_3x3_to_4x4(K, is_perspective=True)
K = convert_K_4x4_to_3x3(K, is_perspective=True)
```
Tokens:
Raw