Pose2Sim

https://perfanalytics.github.io/pose2sim/
Admin
Pose2Sim provides a workflow for 3D markerless kinematics (human or animal), offering a free and...
Tokens:12,060
Snippets:154
Trust Score:9.7
Update:4 months ago
Show doc for...
Tokens:
Raw
### Navigate to Demo Folder and Start IPython

Source: https://perfanalytics.github.io/pose2sim

Change directory to the demo folder and start an IPython session for interactive use.

```bash
cd <Demo_SinglePerson_path>
ipython
```

--------------------------------

### Install Pose2Sim (Latest Stable)

Source: https://perfanalytics.github.io/pose2sim

Installs the latest stable version of Pose2Sim using uv. Ensure your environment is activated.

```bash
uv pip install pose2sim --upgrade
```

--------------------------------

### Install uv and Create Environment (Windows)

Source: https://perfanalytics.github.io/pose2sim

Installs the 'uv' package manager and creates a Python virtual environment for Pose2Sim on Windows. Ensure you have PowerShell.

```powershell
# Install uv
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
# Create uv environment
uv venv "$env:USERPROFILE\.venv\pose2sim" --python 3.13 # or 3.11, or 3.12 
# Activate the uv environment
& "$env:USERPROFILE\.venv\pose2sim\Scripts\Activate.ps1"
```

--------------------------------

### Install uv and Create Environment (Linux/MacOS)

Source: https://perfanalytics.github.io/pose2sim

Installs the 'uv' package manager and creates a Python virtual environment for Pose2Sim on Linux or MacOS. Ensure you have curl.

```bash
# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
# Create uv environment
uv venv ~/.venv/pose2sim --python 3.13 # or 3.11, or 3.12 
# Activate the uv environment
source ~/.venv/pose2sim/bin/activate
```

--------------------------------

### Install Pose2Sim (Developer Version)

Source: https://perfanalytics.github.io/pose2sim

Installs Pose2Sim from a local clone of the repository for development purposes. This allows for testing and editing the bleeding-edge version.

```bash
git clone --depth 1 https://github.com/perfanalytics/pose2sim.git
cd pose2sim
uv pip install -e .
```

--------------------------------

### Find Pose2Sim Installation Path

Source: https://perfanalytics.github.io/pose2sim

Use this command to find the installation path of the pose2sim package.

```bash
uv pip show pose2sim # to find <pose2sim_path>
```

--------------------------------

### Install ONNX Runtime GPU Support

Source: https://perfanalytics.github.io/pose2sim

Installs the GPU-enabled version of ONNX Runtime. This replaces the CPU-only version if it was previously installed.

```bash
uv pip uninstall onnxruntime
uv pip install onnxruntime-gpu
```

--------------------------------

### Run OpenSim Scale and IK Tools via API

Source: https://perfanalytics.github.io/pose2sim

Utilize the OpenSim API to run scaling and inverse kinematics tools from the command line. Ensure the setup file paths are absolute and relevant parameters like time range and output files are correctly configured.

```python
import opensim
opensim.ScaleTool("<PATH TO YOUR SCALING OR IK SETUP FILE>.xml").run()
opensim.InverseKinematicsTool("<PATH TO YOUR SCALING OR IK SETUP FILE>.xml").run()
```

--------------------------------

### Install PyTorch with CUDA Support

Source: https://perfanalytics.github.io/pose2sim

Installs PyTorch, Torchvision, and Torchaudio with CUDA support for faster GPU inference. The index URL specifies the CUDA version (e.g., cu124).

```bash
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
```

--------------------------------

### Run Kinematics with Pose2Sim

Source: https://perfanalytics.github.io/pose2sim

Execute the Pose2Sim kinematics function from an IPython environment. Ensure your environment is activated and the Pose2Sim library is installed.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.kinematics()
```

--------------------------------

### RTMLib Custom Detection and Pose Models

Source: https://perfanalytics.github.io/pose2sim

Use this dictionary to manually select detection and pose models for RTMLib. Ensure input sizes are in square brackets and in the opposite order from the model path. This example uses YOLOX for detection and RTMPose for pose estimation.

```python
{"det_class":"YOLOX", "det_model":"https://…/yolox_m_8xb8-300e_humanart-c2c7a14a.zip", "det_input_size":[640,640], "pose_class":"RTMPose", "pose_model":"https://…/rtmpose-m_simcc-body7_pt-body7-halpe26_700e-256x192-4d3e73dd_20230605.zip", "pose_input_size":[192,256]}
```

--------------------------------

### Verify GPU Setup in Python

Source: https://perfanalytics.github.io/pose2sim

Checks if PyTorch can detect the GPU and if ONNX Runtime recognizes CUDA as an available execution provider. Expected output includes 'True' and 'CUDAExecutionProvider'.

```python
import torch; import onnxruntime as ort
print(torch.cuda.is_available(), ort.get_available_providers())
# Should print "True ['CUDAExecutionProvider', ...]"
```

--------------------------------

### RTMPose for Animal Pose Estimation

Source: https://perfanalytics.github.io/pose2sim

Example for animal pose estimation using RTMPose. Note that marker augmentation will not work, and a custom OpenSim skeleton is needed for Inverse Kinematics (IK).

```python
{"pose_class":"RTMPose", "pose_model":"https://…/rtmpose-m_simcc-ap10k_pt-aic-coco_210e-256x256-7a041aa1_20230206.zip", "pose_input_size":[256,256]}
```

--------------------------------

### Run Calibration Tool

Source: https://perfanalytics.github.io/pose2sim

Initiate the calibration process by setting 'calibration_type' to 'calculate' in Config.toml and calling the calibration function.

```python
Pose2Sim.calibration()
```

--------------------------------

### Run OpenPose for Video Analysis

Source: https://perfanalytics.github.io/pose2sim

Launch OpenPose from the command line to process video files and generate pose estimations in JSON format. Recommended for Pose2Sim due to extensive testing with the BODY_25B model.

```bash
bin\OpenPoseDemo.exe --model_pose BODY_25B --video <PATH_TO_TRIAL_DIR>\videos\cam01.mp4 --write_json <PATH_TO_TRIAL_DIR>\pose\pose_cam01_json
```

--------------------------------

### Run Pose Estimation with Pose2Sim

Source: https://perfanalytics.github.io/pose2sim

Initiate pose estimation for videos using the Pose2Sim library. This command should be run from your project folder after activating the appropriate environment. Results are saved in the 'pose' folder.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.poseEstimation()
```

--------------------------------

### Execute Pose2Sim Pipeline Steps

Source: https://perfanalytics.github.io/pose2sim

This code snippet demonstrates the sequential execution of the Pose2Sim pipeline for end-to-end video processing.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.calibration()
Pose2Sim.poseEstimation()
Pose2Sim.synchronization()
Pose2Sim.personAssociation()
Pose2Sim.triangulation()
Pose2Sim.filtering()
Pose2Sim.markerAugmentation()
Pose2Sim.kinematics()
```

--------------------------------

### Run Batch Processing Analysis

Source: https://perfanalytics.github.io/pose2sim

Initiate batch processing by navigating to the demo directory and running Pose2Sim. This method is suitable for running numerous analyses with varying parameters.

```bash
cd <Demo_Batch_path>
ipython
from Pose2Sim import Pose2Sim
Pose2Sim.runAll()
```

--------------------------------

### Run Pose2Sim Triangulation

Source: https://perfanalytics.github.io/pose2sim

Execute the triangulation process using the Pose2Sim library. Ensure your environment is activated and you are in the project folder.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.triangulation()
```

--------------------------------

### Run Camera Calibration

Source: https://perfanalytics.github.io/pose2sim

Initiates the camera calibration process. Ensure you are in your project folder and have activated your environment before running.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.calibration()
```

--------------------------------

### Run Multi-Person Analysis

Source: https://perfanalytics.github.io/pose2sim

Execute multi-person analysis by navigating to the demo directory and running Pose2Sim. Ensure 'multi_person = true' is set in your Config.toml.

```bash
cd <Demo_MultiPerson_path>
ipython
from Pose2Sim import Pose2Sim
Pose2Sim.runAll()
```

--------------------------------

### Run All Pose2Sim Stages

Source: https://perfanalytics.github.io/pose2sim

Execute all stages of the Pose2Sim pipeline. This can be customized by specifying which stages to run.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.runAll()
# or: 
# Pose2Sim.runAll(do_calibration=True, do_poseEstimation=True, do_synchronization=True, do_personAssociation=True, do_triangulation=True, do_filtering=True, do_markerAugmentation=True, do_kinematics=True)
```

--------------------------------

### Synchronize Cameras with Pose2Sim

Source: https://perfanalytics.github.io/pose2sim

Run camera synchronization using Pose2Sim. This function computes mean vertical speed for keypoints to find the optimal time offset for correlation across cameras. It's recommended to use this when cameras are not natively synchronized.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.synchronization()
```

--------------------------------

### Balanced Mode with Body_with_feet Pose Model

Source: https://perfanalytics.github.io/pose2sim

This configuration is equivalent to mode='balanced' and uses the body_with_feet pose model. It specifies the detection and pose models, input sizes, and classes.

```python
mode = "{'det_class':'YOLOX',
       'det_model':'https://huggingface.co/datasets/DavidPagnon/rtmlib_models/resolve/main/mmpose/rtmposev1/onnx_sdk/yolox_m_8xb8-300e_humanart-c2c7a14a.onnx',
       'det_input_size':[640, 640],
       'pose_class':'RTMPose',
       'pose_model':'https://huggingface.co/datasets/DavidPagnon/rtmlib_models/resolve/main/mmpose/rtmposev1/onnx_sdk/rtmpose-m_simcc-body7_pt-body7-halpe26_700e-256x192-4d3e73dd_20230605.onnx',
       'pose_input_size':[192,256]}"
```

--------------------------------

### Pose Estimation with Updated Parameters

Source: https://perfanalytics.github.io/pose2sim

Perform pose estimation by passing a configuration dictionary with updated parameters. Ensure the project directory is correctly specified.

```python
config_dict = {"project": {"project_dir": "<Demo_SinglePerson_path>"}, 
               "pose": {"pose_model": "whole_body", "overwrite_pose": True}}
Pose2Sim.poseEstimation(config_dict)
```

--------------------------------

### Configure Custom Pose Models in Pose2Sim

Source: https://perfanalytics.github.io/pose2sim

Specify custom detection and pose models for Pose2Sim by providing a dictionary with model paths or URLs and input sizes. This allows for flexibility beyond default models.

```python
{
    "detection_model": "path/to/your/detection_model.onnx",
    "pose_model": "path/to/your/pose_model.onnx",
    "input_sizes": [640, 640]
}
```

--------------------------------

### Perform Pose Estimation with Updated Config Dictionary

Source: https://perfanalytics.github.io/pose2sim

Execute pose estimation using a dynamically updated configuration dictionary, allowing for whole-body pose estimation and overwriting existing pose data.

```python
import toml
config_dict = toml.load("<Demo_SinglePerson_path>/Config.toml")
config_dict.get("project").update({"project_dir":"<Demo_SinglePerson_path>"})
config_dict.get("pose").update({"pose_model": "whole_body","overwrite_pose": True})
Pose2Sim.poseEstimation(config_dict)
```

--------------------------------

### Compress and Resize Video with ffmpeg

Source: https://perfanalytics.github.io/pose2sim

Use ffmpeg to compress videos by increasing the CRF value and decrease resolution. This reduces storage use with minimal impact on accuracy. Ensure the output file is specified.

```bash
ffmpeg -i original_vid.MP4 \
  -vcodec libx264 -crf 30 -preset fast \
  -vf "scale='min(1280,iw)':'min(720,ih)':force_original_aspect_ratio=decrease" 
  -movflags +faststart \
  lighter_vid.mp4
```

--------------------------------

### DeepSort Tracking Parameters

Source: https://perfanalytics.github.io/pose2sim

Configure DeepSort hyperparameters for person tracking. These parameters control aspects like maximum age of tracks, number of initializations, and overlap thresholds.

```python
{"max_age":30, "n_init":3, "nms_max_overlap":0.8, "max_cosine_distance":0.3, "nn_budget":200, "max_iou_distance":0.8}
```

--------------------------------

### Animal Pose Estimation Configuration

Source: https://perfanalytics.github.io/pose2sim

This configuration is for animal pose estimation using RTMPose. Marker augmentation is not supported, and a custom OpenSim skeleton is required for kinematic analysis.

```python
mode = "{'pose_class':'RTMPose',
       'pose_model':'https://huggingface.co/datasets/DavidPagnon/rtmlib_models/resolve/main/mmpose/rtmposev1/onnx_sdk/rtmpose-m_simcc-ap10k_pt-aic-coco_210e-256x256-7a041aa1_20230206.onnx',
       'pose_input_size':[256,256]}"
```

--------------------------------

### Run MediaPipe BlazePose and Save Detections

Source: https://perfanalytics.github.io/pose2sim

Execute the BlazePose script to run pose detection and save results in OpenPose (JSON) or DeepLabCut (h5/csv) formats. Use '-dJs' for JSON output.

```bash
Blazepose_runsave -i input_file -dJs
```

--------------------------------

### One-Stage RTMO Model Configuration

Source: https://perfanalytics.github.io/pose2sim

This configuration uses a one-stage RTMO model. Marker augmentation will not work, and kinematic analysis will require a custom OpenSim skeleton.

```python
mode = "{'pose_class':'RTMO', 
       'pose_model':'https://huggingface.co/datasets/DavidPagnon/rtmlib_models/resolve/main/mmpose/rtmo/onnx_sdk/rtmo-m_16xb16-600e_body7-640x640-39e78cc4_20231211.onnx', 
       'pose_input_size':[640, 640]}"
```

--------------------------------

### Visualize Custom Pose Model Hierarchy

Source: https://perfanalytics.github.io/pose2sim

Visualize the skeleton's hierarchy by setting pose_model to CUSTOM in Config.toml and using this Python script to load and render the model structure.

```python
config_path = r'path_to_Config.toml'
import toml, anytree
config = toml.load(config_path)
pose_model = config.get('pose').get('pose_model')
model = anytree.importer.DictImporter().import_(config.get('pose').get(pose_model))
for pre, _, node in anytree.RenderTree(model): 
    print(f'{pre}{node.name} id={node.id}')
```

--------------------------------

### Run Marker Augmentation in Pose2Sim

Source: https://perfanalytics.github.io/pose2sim

Execute the marker augmentation process. Ensure your environment is activated and you are in the project folder. This function uses the Stanford LSTM model to estimate marker positions.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.markerAugmentation()
```

--------------------------------

### Translate DeepLabCut H5 to OpenPose JSON

Source: https://perfanalytics.github.io/pose2sim

Use this script to convert 2D coordinates from a DeepLabCut model into OpenPose JSON format. Ensure camera folder names match the calibration file and end with '_json'.

```bash
DLC_to_OpenPose -i input_h5_file
```

--------------------------------

### Apply Filtering to 3D Coordinates

Source: https://perfanalytics.github.io/pose2sim

This snippet shows how to initiate the filtering process for 3D coordinates using the Pose2Sim library. Ensure you have activated your environment and are running IPython.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.filtering()
```

--------------------------------

### BibTeX Entries for Pose2Sim Citations

Source: https://perfanalytics.github.io/pose2sim

Use these BibTeX entries when citing Pose2Sim in your publications. They cover different aspects of the Pose2Sim workflow.

```bibtex
@Article{Pagnon_2022_JOSS, 
    AUTHOR = {Pagnon, David and Domalain, Mathieu and Reveret, Lionel}, 
    TITLE = {Pose2Sim: An open-source Python package for multiview markerless kinematics}, 
    JOURNAL = {Journal of Open Source Software}, 
    YEAR = {2022},
    DOI = {10.21105/joss.04362}, 
    URL = {https://joss.theoj.org/papers/10.21105/joss.04362}
   }
```

```bibtex
@Article{Pagnon_2022_Accuracy,
    AUTHOR = {Pagnon, David and Domalain, Mathieu and Reveret, Lionel},
    TITLE = {Pose2Sim: An End-to-End Workflow for 3D Markerless Sports Kinematics—Part 2: Accuracy},
    JOURNAL = {Sensors},
    YEAR = {2022},
    DOI = {10.3390/s22072712},
    URL = {https://www.mdpi.com/1424-8220/22/7/2712}
  }
```

```bibtex
@Article{Pagnon_2021_Robustness,
    AUTHOR = {Pagnon, David and Domalain, Mathieu and Reveret, Lionel},
    TITLE = {Pose2Sim: An End-to-End Workflow for 3D Markerless Sports Kinematics—Part 1: Robustness},
    JOURNAL = {Sensors},
    YEAR = {2021},
    DOI = {10.3390/s21196530},
    URL = {https://www.mdpi.com/1424-8220/21/19/6530}
  }
```

--------------------------------

### Translate AlphaPose JSON to OpenPose Format

Source: https://perfanalytics.github.io/pose2sim

Convert single JSON files from AlphaPose into OpenPose frame-by-frame files using this utility script. This is necessary for integrating AlphaPose detections with Pose2Sim.

```bash
AlphaPose_to_OpenPose -i input_alphapose_json_file
```

--------------------------------

### Associate Persons Across Cameras with Pose2Sim

Source: https://perfanalytics.github.io/pose2sim

Perform person association across camera views using Pose2Sim. This is crucial when multi-person tracking is enabled, as it measures the intersection of 3D rays to associate individuals. Skip this if only one person is present.

```python
from Pose2Sim import Pose2Sim
Pose2Sim.personAssociation()
```

--------------------------------

### RTMO One-Stage Pose Estimation

Source: https://perfanalytics.github.io/pose2sim

Configure RTMO for one-stage pose estimation. This requires setting 'pose_model' to 'Body'. The input size should match the model's expected dimensions.

```python
{"pose_class":"RTMO", "pose_model":"https://…/rtmo-m_16xb16-600e_body7-640x640-39e78cc4_20231211.zip", "pose_input_size":[640,640]}
```
Tokens:
Raw