### Install Manotorch Package Source: https://github.com/lixiny/manotorch/blob/master/README.md Install the manotorch package locally to import and use it in other projects. ```shell pip install . ``` -------------------------------- ### Install Conda Dependencies Source: https://github.com/lixiny/manotorch/blob/master/README.md Create a new conda environment or update an existing one using the provided environment.yaml file. ```shell conda env create -f environment.yaml ``` ```shell conda env update -f environment.yaml ``` -------------------------------- ### Get Watertight Mesh Faces Source: https://context7.com/lixiny/manotorch/llms.txt Obtain the full set of faces for a watertight hand mesh, including 14 additional triangles to close the wrist boundary. This is useful for rendering and volumetric operations. ```python import torch from manotorch.manolayer import ManoLayer mano_layer = ManoLayer(side="right", mano_assets_root="assets/mano") open_faces = mano_layer.th_faces # (1538, 3) closed_faces = mano_layer.get_mano_closed_faces() # (1552, 3) print(f"Open faces: {open_faces.shape}") # torch.Size([1538, 3]) print(f"Closed faces: {closed_faces.shape}") # torch.Size([1552, 3]) # Use with a renderer (e.g., trimesh / pytorch3d) import trimesh pose = torch.zeros(1, 48) shape = torch.zeros(1, 10) output = mano_layer(pose, shape) mesh = trimesh.Trimesh( vertices=output.verts[0].detach().numpy(), faces=closed_faces.numpy(), ) mesh.export("hand_closed.obj") ``` -------------------------------- ### Get Beta-Dependent Rotation Centre Source: https://context7.com/lixiny/manotorch/llms.txt Calculate the 3-D rotation center for a given set of shape betas. This center is used to determine how global root rotations affect the hand mesh and requires `center_idx` to be `None`. ```python import torch from manotorch.manolayer import ManoLayer mano_layer = ManoLayer( side="right", center_idx=None, # must be None to get a non-zero centre mano_assets_root="assets/mano", ) batch_size = 2 betas = torch.rand(batch_size, 10) rot_center = mano_layer.get_rotation_center(betas) # (B, 3) print(rot_center) ``` -------------------------------- ### Initialize and Compare Yana's and manotorch MANO Layers Source: https://github.com/lixiny/manotorch/blob/master/scripts/test_compatibility.ipynb Initializes MANO layers from Yana's manopth and the manotorch library, generates hand vertices and joints, converts units to meters, and asserts their similarity. ```python yana_mano_layer = YanaMano.ManoLayer( center_idx=None, side="right", mano_root="../assets/mano/models/", use_pca=True, flat_hand_mean=True, ncomps=ncomps ) # hand's vertices and joints. The unit set by Yana is millimeters. V1, J1 = yana_mano_layer(random_pose, random_shape) # change to meter, V1 = V1 / 1000.0 J1 = J1 / 1000.0 this_mano_layer = ThisMano.ManoLayer( rot_mode="axisang", center_idx=None, side="right", mano_assets_root="../assets/mano", use_pca=True, flat_hand_mean=True, ncomps=ncomps ) out = this_mano_layer(random_pose, random_shape) V2, J2 = out.verts, out.joints assert torch.allclose(V1, V2, atol=1e-6) # less than 0.001 mm is acceptable assert torch.allclose(J1, J2, atol=1e-6) ``` -------------------------------- ### Import MANO Libraries Source: https://github.com/lixiny/manotorch/blob/master/scripts/test_compatibility.ipynb Imports necessary libraries and MANO model implementations from different sources. Ensure the MANO and manopth repositories are cloned into the thirdparty directory. ```python import os, sys module_path = os.path.abspath(os.path.join('..')) if module_path not in sys.path: sys.path.append(module_path) import torch import numpy as np import thirdparty.manopth.manopth.manolayer as YanaMano import thirdparty.MANO.mano as OmidMano import manotorch.manolayer as ThisMano ``` -------------------------------- ### Minimal Manotorch Usage Source: https://github.com/lixiny/manotorch/blob/master/README.md Demonstrates the basic initialization and forward pass of the ManoLayer with random pose and shape parameters. Ensure MANO pickle data is downloaded and placed in the assets/mano folder. ```python import torch from manotorch.manolayer import ManoLayer, MANOOutput # Select number of principal components for pose space ncomps = 15 # initialize layers mano_layer = ManoLayer(use_pca=True, flat_hand_mean=False, ncomps=ncomps) batch_size = 2 # Generate random shape parameters random_shape = torch.rand(batch_size, 10) # Generate random pose parameters, including 3 values for global axis-angle rotation random_pose = torch.rand(batch_size, 3 + ncomps) # The mano_layer's output contains: """ MANOOutput = namedtuple( "MANOOutput", [ "verts", "joints", "center_idx", "center_joint", "full_poses", "betas", "transforms_abs", ], ) """ # forward mano layer mano_output: MANOOutput = mano_layer(random_pose, random_shape) # retrieve 778 vertices, 21 joints and 16 SE3 transforms of each articulation # verts and joints in meters. verts = mano_output.verts # (B, 778, 3), root(center_joint) relative joints = mano_output.joints # (B, 21, 3), root relative transforms_abs = mano_output.transforms_abs # (B, 16, 4, 4), root relative ``` -------------------------------- ### Run Axis Layer Forward Kinematics Source: https://github.com/lixiny/manotorch/blob/master/README.md Execute the script to demonstrate the axis layer's forward kinematics, which is used for calculating twist-spread-bend axes in a canonical pose. ```shell python scripts/simple_app.py --mode axis ``` -------------------------------- ### ManoTorch Training Loop Sketch Source: https://context7.com/lixiny/manotorch/llms.txt Demonstrates a basic training loop using the HandPoseNet model, Adam optimizer, and MSELoss. Includes forward pass, loss calculation, and backpropagation. Adjust the anatomy loss weight (λ) as needed. ```python # Training loop sketch model = HandPoseNet() optimizer = torch.optim.Adam(model.parameters(), lr=1e-4) criterion = nn.MSELoss() dummy_images = torch.randn(4, 3, 224, 224) gt_joints = torch.randn(4, 21, 3) pred_verts, pred_joints, ana_loss = model(dummy_images) joint_loss = criterion(pred_joints, gt_joints) total_loss = joint_loss + 0.01 * ana_loss # anatomy weight λ = 0.01 optimizer.zero_grad() total_loss.backward() optimizer.step() print(f"joint_loss: {joint_loss.item():.4f} | anatomy_loss: {ana_loss.item():.4f}") ``` -------------------------------- ### Run Hand Composition with Euler Angles Source: https://github.com/lixiny/manotorch/blob/master/README.md Execute the script to demonstrate composing a hand model from specified Euler angles for each joint, showing deterministic hand construction. ```shell # transform order of right hand # 15-14-13- # \ #* 3-- 2 -- 1 -----0 < NOTE: demo on this finger # 6 -- 5 -- 4 ----/ # 12 - 11 - 10 --/ # 9-- 8 -- 7 --/ # the ID: 1 joints have been rotated by pi/6 around spread-axis, and pi/2 around bend-axis # the ID: 2, 3 joints have been rotated by pi/2 around bend-axis python scripts/simple_compose.py ``` -------------------------------- ### Run Anchor Layer for Contact Interaction Source: https://github.com/lixiny/manotorch/blob/master/README.md Execute the script to demonstrate the anchor layer's functionality, which derives coarse palm vertices for modeling hand-object contact. ```shell python scripts/simple_app.py --mode anchor ``` -------------------------------- ### Load and Compare Omid's MANO Model Source: https://github.com/lixiny/manotorch/blob/master/scripts/test_compatibility.ipynb Loads Omid's MANO model, generates hand vertices and joints, and asserts their similarity to the manotorch output. ```python omid_mano_model = OmidMano.load(model_path="../assets/mano/models", is_rhand= True, num_pca_comps=ncomps, batch_size=batch_size, flat_hand_mean=True) output = omid_mano_model( global_orient=random_pose[:,:3], hand_pose=random_pose[:,3:], transl = None, betas=random_shape, return_verts=True, return_tips = True ) V3, J3 = output.vertices, output.joints assert torch.allclose(V2, V3, atol=1e-6) # less than 0.001 mm is acceptable ``` -------------------------------- ### AxisLayerFK.compose: Build MANO Pose from Euler Angles Source: https://context7.com/lixiny/manotorch/llms.txt Builds the full MANO axis-angle pose vector from anatomy-aligned Euler angles. Enables intuitive, anatomically grounded hand pose construction. ```python import torch from math import pi from manotorch.manolayer import ManoLayer, MANOOutput from manotorch.axislayer import AxisLayerFK mano_layer = ManoLayer(rot_mode="axisang", side="right", center_idx=9, mano_assets_root="assets/mano", flat_hand_mean=True) axis_layer = AxisLayerFK(side="right", mano_assets_root="assets/mano") # Joint index layout (right hand): # 15-14-13-\ # \ # 3-- 2 -- 1 -----0 # 6 -- 5 -- 4 ----/ # 12 - 11 - 10 --/ # 9-- 8 -- 7 --/ composed_ee = torch.zeros(1, 16, 3) # (B, 16, [twist, spread, bend]) # Bend index finger: MCP (id=1) spread + bend, PIP (id=2,3) bend only composed_ee[:, 1] = torch.tensor([0, pi / 6, pi / 2]) composed_ee[:, 2] = torch.tensor([0, 0, pi / 2]) composed_ee[:, 3] = torch.tensor([0, 0, pi / 2]) # compose returns axis-angle pose (B, 16, 3) → reshape to (B, 48) composed_aa = axis_layer.compose(composed_ee) # (B, 16, 3) pose_flat = composed_aa.reshape(1, -1) # (B, 48) zero_shape = torch.zeros(1, 10) output: MANOOutput = mano_layer(pose_flat, zero_shape) verts = output.verts # (1, 778, 3) joints = output.joints # (1, 21, 3) print(verts.shape) # torch.Size([1, 778, 3]) ``` -------------------------------- ### Clone Manotorch Repository Source: https://github.com/lixiny/manotorch/blob/master/README.md Clone the manotorch repository to your local machine. ```shell git clone https://github.com/lixiny/manotorch.git cd manotorch ``` -------------------------------- ### Generate Palm Anchors with AnchorLayer Source: https://context7.com/lixiny/manotorch/llms.txt Derives anchor points from MANO mesh vertices for compact palm surface representation. Requires ManoLayer and AnchorLayer, and MANOOutput for vertex data. ```python import torch from manotorch.manolayer import ManoLayer, MANOOutput from manotorch.anchorlayer import AnchorLayer mano_layer = ManoLayer(rot_mode="axisang", side="right", mano_assets_root="assets/mano") anchor_layer = AnchorLayer(anchor_root="assets/anchor") batch_size = 2 pose = torch.zeros(batch_size, 48) shape = torch.zeros(batch_size, 10) output: MANOOutput = mano_layer(pose, shape) verts = output.verts # (B, 778, 3) anchors = anchor_layer(verts) # (B, N_anchors, 3) print(anchors.shape) # torch.Size([2, 32, 3]) — 32 palm anchor points per sample ``` -------------------------------- ### ManoLayer Core Forward Pass (Axis-Angle) Source: https://context7.com/lixiny/manotorch/llms.txt Initialize and use ManoLayer for axis-angle pose representation. Ensure the `mano_assets_root` points to your MANO model files. ```python import torch from manotorch.manolayer import ManoLayer, MANOOutput # ── Axis-angle mode (full 48-dim pose: 3 global + 45 finger) ────────────────── mano_layer = ManoLayer( rot_mode="axisang", # "axisang" | "quat" side="right", # "right" | "left" center_idx=9, # root-relative output; None → absolute coords mano_assets_root="assets/mano", use_pca=False, flat_hand_mean=True, ) batch_size = 4 pose = torch.zeros(batch_size, 48) # (B, 3+45) shape = torch.zeros(batch_size, 10) # (B, 10) beta parameters output: MANOOutput = mano_layer(pose, shape) verts = output.verts # (B, 778, 3) – root-relative meters joints = output.joints # (B, 21, 3) – SNAP joint order transforms_abs = output.transforms_abs # (B, 16, 4, 4) SE(3) per joint center_joint = output.center_joint # (B, 1, 3) full_poses = output.full_poses # (B, 48) betas = output.betas # (B, 10) print(verts.shape) # torch.Size([4, 778, 3]) print(joints.shape) # torch.Size([4, 21, 3]) print(transforms_abs.shape) # torch.Size([4, 16, 4, 4]) ``` -------------------------------- ### Rotation Utility Functions Source: https://context7.com/lixiny/manotorch/llms.txt `manotorch.utils.geometry` provides a complete set of differentiable rotation conversion utilities compatible with arbitrary leading batch dimensions. ```APIDOC ## Rotation Utility Functions ### Description `manotorch.utils.geometry` provides a complete set of differentiable rotation conversion utilities compatible with arbitrary leading batch dimensions. ### Usage ```python import torch from manotorch.utils.geometry import ( axis_angle_to_matrix, matrix_to_euler_angles, euler_angles_to_matrix, axis_angle_to_quaternion, quaternion_to_matrix, matrix_to_quaternion, quaternion_to_axis_angle, rotation_to_axis_angle, ) # Axis-angle → rotation matrix aa = torch.tensor([[0.0, 0.0, 1.5708]]) # 90° around Z R = axis_angle_to_matrix(aa) # (1, 3, 3) print(R.round(decimals=4)) # tensor([[[ 0.0000, -1.0000, 0.0000], # [ 1.0000, 0.0000, 0.0000], # [ 0.0000, 0.0000, 1.0000]]]) # Rotation matrix → euler angles (XYZ Tait-Bryan) ee = matrix_to_euler_angles(R, convention="XYZ") print(ee) # tensor([[0.0000, 0.0000, 1.5708]]) # Euler → matrix (round-trip) R2 = euler_angles_to_matrix(ee, convention="XYZ") print(torch.allclose(R, R2, atol=1e-5)) # True # Quaternion conversions q = axis_angle_to_quaternion(aa) # (1, 4) real-first R3 = quaternion_to_matrix(q) # (1, 3, 3) aa2 = quaternion_to_axis_angle(q) # (1, 3) print(torch.allclose(aa, aa2, atol=1e-5)) # True ``` ``` -------------------------------- ### ManoLayer Core Forward Pass (PCA Pose Mode) Source: https://context7.com/lixiny/manotorch/llms.txt Utilize ManoLayer with PCA-compressed pose representation. The `ncomps` parameter determines the dimensionality of the compressed pose. ```python ncomps = 15 mano_pca = ManoLayer( rot_mode="axisang", use_pca=True, flat_hand_mean=False, ncomps=ncomps, side="right", mano_assets_root="assets/mano", ) random_pose = torch.rand(batch_size, 3 + ncomps) # (B, 18) random_shape = torch.rand(batch_size, 10) out_pca: MANOOutput = mano_pca(random_pose, random_shape) print(out_pca.verts.shape) # torch.Size([4, 778, 3]) ``` -------------------------------- ### Generate Random Pose and Shape Parameters Source: https://github.com/lixiny/manotorch/blob/master/scripts/test_compatibility.ipynb Generates random pose and shape parameters for batch processing. Sets random seeds for reproducibility. ```python batch_size = 10 ncomps = 15 seed = 23 np.random.seed(seed) torch.manual_seed(seed) random_shape = torch.rand(batch_size, 10) # Generate random pose parameters, including 3 values for global axis-angle rotation random_pose = torch.rand(batch_size, 3 + ncomps) ``` -------------------------------- ### End-to-End Hand Pose Network with ManoTorch Source: https://context7.com/lixiny/manotorch/llms.txt Defines a HandPoseNet module that integrates ManoLayer, AxisLayerFK, and AnatomyConstraintLossEE for end-to-end training. Requires ManoTorch and PyTorch. Ensure MANO assets are available at the specified path. ```python import torch import torch.nn as nn from manotorch.manolayer import ManoLayer, MANOOutput from manotorch.axislayer import AxisLayerFK from manotorch.anatomy_loss import AnatomyConstraintLossEE class HandPoseNet(nn.Module): def __init__(self): super().__init__() # Lightweight encoder backbone (replace with ResNet / ViT etc.) self.encoder = nn.Sequential( nn.Flatten(), nn.Linear(224 * 224 * 3, 512), nn.ReLU(), ) self.pose_head = nn.Linear(512, 48) # global (3) + finger (45) self.shape_head = nn.Linear(512, 10) # betas self.mano_layer = ManoLayer(rot_mode="axisang", side="right", center_idx=9, mano_assets_root="assets/mano", flat_hand_mean=True) self.axis_layer = AxisLayerFK(side="right", mano_assets_root="assets/mano") self.anatomy_loss = AnatomyConstraintLossEE(reduction="mean") self.anatomy_loss.setup() def forward(self, images): # images: (B, 3, 224, 224) feat = self.encoder(images) pose = self.pose_head(feat) # (B, 48) shape = self.shape_head(feat) # (B, 10) output: MANOOutput = self.mano_layer(pose, shape) verts = output.verts # (B, 778, 3) joints = output.joints # (B, 21, 3) T_g_p = output.transforms_abs # (B, 16, 4, 4) _, _, ee = self.axis_layer(T_g_p) # (B, 16, 3) ana_loss = self.anatomy_loss(ee) # scalar return verts, joints, ana_loss ``` -------------------------------- ### Adjust and Reorder Joints for Compatibility Source: https://github.com/lixiny/manotorch/blob/master/scripts/test_compatibility.ipynb Defines constants and a function to add tip joints and reorder joints to match Yana's MANO definition. This is necessary because Omid's MANO model has different joint definitions. ```python OURS_TIP_IDS = { 'thumb': 745, 'index': 317, 'middle': 444, 'ring': 556, 'pinky': 673, } REORDER_IDX = [0, 13, 14, 15, 16, 1, 2, 3, 17, 4, 5, 6, 18, 10, 11, 12, 19, 7, 8, 9, 20] def add_tips(vertices, joints, joint_ids = None): if joint_ids is None: joint_ids = torch.tensor(list(OURS_TIP_IDS.values()), dtype=torch.long) extra_joints = torch.index_select(vertices, 1, joint_ids) joints = torch.cat([joints, extra_joints], dim=1) return joints J3_ = J3[:, :16] # remove the tips in Omid's definition tips = V3[:, [745, 317, 444, 556, 673]] # get the tips from Yana's definition J3_ = torch.cat([J3_, tips], 1) new_J3 = J3_[:, REORDER_IDX] # reorder the joints to match Yana's definition assert torch.allclose(J2, new_J3, atol=1e-6) # less than 0.001 mm is acceptable ``` -------------------------------- ### ManoLayer Core Forward Pass (Quaternion Mode) Source: https://context7.com/lixiny/manotorch/llms.txt Initialize and use ManoLayer with quaternion pose representation. The input pose tensor size will differ based on the quaternion format. ```python mano_quat = ManoLayer( rot_mode="quat", side="right", mano_assets_root="assets/mano", ) quat_pose = torch.zeros(batch_size, 64) # (B, 16 * 4) out_quat: MANOOutput = mano_quat(quat_pose, shape) print(out_quat.joints.shape) # torch.Size([4, 21, 3]) ``` -------------------------------- ### Run Anatomy Loss for Pose Correction Source: https://github.com/lixiny/manotorch/blob/master/README.md Execute the script to demonstrate the pose correction capabilities using the anatomy loss, which penalizes abnormal rotations based on Euler angles for robustness. ```shell python scripts/simple_anatomy_loss.py ``` -------------------------------- ### Optimise Hand Pose with MANO Layer Source: https://context7.com/lixiny/manotorch/llms.txt Optimises a hand pose to minimise anatomy violation using the MANO layer and Adam optimizer. Requires MANOOutput and axis_layer for pose processing. ```python composed_aa = torch.randn(1, 15, 3, requires_grad=True) # 15 finger joints global_aa = torch.zeros(1, 1, 3) optimizer = torch.optim.Adam([composed_aa], lr=1e-3) for step in range(100): pose_flat = torch.cat([global_aa, composed_aa], dim=1).reshape(1, -1) output: MANOOutput = mano_layer(pose_flat, torch.zeros(1, 10)) # Assuming mano_layer is defined elsewhere T_g_p = output.transforms_abs # (1, 16, 4, 4) _, _, ee = axis_layer(T_g_p) # (1, 16, 3) # Assuming axis_layer is defined elsewhere loss = anatomy_loss(ee) # scalar # Assuming anatomy_loss is defined elsewhere optimizer.zero_grad() loss.backward() optimizer.step() if step % 20 == 0: print(f"step {step:3d} | anatomy loss: {loss.item():.5f}") ``` -------------------------------- ### AxisLayerFK: Absolute Transforms to Anatomy-Aligned Euler Angles Source: https://context7.com/lixiny/manotorch/llms.txt Converts MANO global transforms into anatomy-aligned Euler angles (twist, spread, bend). Requires MANO global transforms as input. ```python import torch from manotorch.manolayer import ManoLayer, MANOOutput from manotorch.axislayer import AxisLayerFK mano_layer = ManoLayer( rot_mode="axisang", side="right", center_idx=9, mano_assets_root="assets/mano", use_pca=False, flat_hand_mean=True, ) axis_layer = AxisLayerFK(side="right", mano_assets_root="assets/mano") batch_size = 2 pose = torch.zeros(batch_size, 48) shape = torch.zeros(batch_size, 10) output: MANOOutput = mano_layer(pose, shape) T_g_p = output.transforms_abs # (B, 16, 4, 4) — MANO global transforms T_g_a, R_anatomy, ee = axis_layer(T_g_p) # T_g_a : (B, 16, 4, 4) — global transforms in anatomy-aligned frame # R_anatomy : (B, 16, 3, 3) — anatomy-aligned rotation matrices # ee : (B, 16, 3) — euler angles [twist, spread, bend] in radians print(ee.shape) # torch.Size([2, 16, 3]) print(ee[0, 1]) # euler angles for index MCP joint, batch 0 # Channels: [twist(X), spread(Y), bend(Z)] ``` -------------------------------- ### Differentiable Rotation Conversions Source: https://context7.com/lixiny/manotorch/llms.txt Provides differentiable utilities for converting between axis-angle, rotation matrices, Euler angles, and quaternions. Compatible with arbitrary leading batch dimensions. ```python import torch from manotorch.utils.geometry import ( axis_angle_to_matrix, matrix_to_euler_angles, euler_angles_to_matrix, axis_angle_to_quaternion, quaternion_to_matrix, matrix_to_quaternion, quaternion_to_axis_angle, rotation_to_axis_angle, ) # Axis-angle → rotation matrix aa = torch.tensor([[0.0, 0.0, 1.5708]]) # 90° around Z R = axis_angle_to_matrix(aa) # (1, 3, 3) print(R.round(decimals=4)) # tensor([[[ 0.0000, -1.0000, 0.0000], # [ 1.0000, 0.0000, 0.0000], # [ 0.0000, 0.0000, 1.0000]]]) # Rotation matrix → euler angles (XYZ Tait-Bryan) ee = matrix_to_euler_angles(R, convention="XYZ") print(ee) # tensor([[0.0000, 0.0000, 1.5708]]) # Euler → matrix (round-trip) R2 = euler_angles_to_matrix(ee, convention="XYZ") print(torch.allclose(R, R2, atol=1e-5)) # True # Quaternion conversions q = axis_angle_to_quaternion(aa) # (1, 4) real-first R3 = quaternion_to_matrix(q) # (1, 3, 3) aa2 = quaternion_to_axis_angle(q) # (1, 3) print(torch.allclose(aa, aa2, atol=1e-5)) # True ``` -------------------------------- ### AnchorLayer — Palm Anchor Interpolation Source: https://context7.com/lixiny/manotorch/llms.txt `AnchorLayer` derives a set of anchor points interpolated from the MANO mesh vertices that give a compact, semantically meaningful representation of the palm surface. These anchors are used as contact descriptors in hand-object interaction tasks, providing coarse spatial coverage of the palm and finger pads. ```APIDOC ## AnchorLayer — Palm Anchor Interpolation ### Description `AnchorLayer` derives a set of anchor points interpolated from the MANO mesh vertices that give a compact, semantically meaningful representation of the palm surface. These anchors are used as contact descriptors in hand-object interaction tasks, providing coarse spatial coverage of the palm and finger pads. ### Usage ```python import torch from manotorch.manolayer import ManoLayer, MANOOutput from manotorch.anchorlayer import AnchorLayer mano_layer = ManoLayer(rot_mode="axisang", side="right", mano_assets_root="assets/mano") anchor_layer = AnchorLayer(anchor_root="assets/anchor") batch_size = 2 pose = torch.zeros(batch_size, 48) shape = torch.zeros(batch_size, 10) output: MANOOutput = mano_layer(pose, shape) verts = output.verts # (B, 778, 3) anchors = anchor_layer(verts) # (B, N_anchors, 3) print(anchors.shape) # torch.Size([2, 32, 3]) — 32 palm anchor points per sample ``` ``` -------------------------------- ### Upsample Mesh with UpSampleLayer Source: https://context7.com/lixiny/manotorch/llms.txt Subdivides a triangle mesh by inserting new vertices at edge midpoints, quadrupling face count. Useful for progressive mesh refinement. Requires ManoLayer for initial mesh data. ```python import torch from manotorch.manolayer import ManoLayer, MANOOutput from manotorch.upsamplelayer import UpSampleLayer mano_layer = ManoLayer(rot_mode="axisang", side="right", mano_assets_root="assets/mano") upsample_layer = UpSampleLayer() pose = torch.zeros(1, 48) shape = torch.zeros(1, 10) output: MANOOutput = mano_layer(pose, shape) verts = output.verts # (1, 778, 3) faces = mano_layer.th_faces # (1538, 3) # faces must be provided per sample in a list/batch faces_batch = faces.unsqueeze(0).repeat(1, 1, 1) # (1, 1538, 3) upsampled_verts, upsampled_faces = upsample_layer(verts, faces_batch) print(upsampled_verts.shape) # torch.Size([1, 2316, 3]) — 778 + 1538 new edge midpoints print(upsampled_faces.shape) # torch.Size([1, 6152, 3]) — 1538 × 4 # Second subdivision level verts2, faces2 = upsample_layer(upsampled_verts, upsampled_faces) print(verts2.shape) # torch.Size([1, 8468, 3]) ``` -------------------------------- ### ManoLayer - Core MANO Forward Pass Source: https://context7.com/lixiny/manotorch/llms.txt The ManoLayer is the central differentiable module that accepts pose and shape parameters to output hand mesh data. It supports different rotation modes (axis-angle, quaternion) and PCA-compressed poses. ```APIDOC ## ManoLayer - Core MANO Forward Pass ### Description This section details the usage of the `ManoLayer` for performing the core MANO forward pass. It accepts pose and shape parameters and returns a `MANOOutput` named tuple containing vertices, joints, and transformations. ### Initialization ```python import torch from manotorch.manolayer import ManoLayer, MANOOutput # Axis-angle mode mano_layer = ManoLayer( rot_mode="axisang", # "axisang" | "quat" side="right", # "right" | "left" center_idx=9, # root-relative output; None → absolute coords mano_assets_root="assets/mano", use_pca=False, flat_hand_mean=True, ) # PCA pose mode ncomps = 15 mano_pca = ManoLayer( rot_mode="axisang", use_pca=True, flat_hand_mean=False, ncomps=ncomps, side="right", mano_assets_root="assets/mano", ) # Quaternion mode mano_quat = ManoLayer( rot_mode="quat", side="right", mano_assets_root="assets/mano", ) ``` ### Usage ```python # Example for axis-angle mode batch_size = 4 pose = torch.zeros(batch_size, 48) # (B, 3+45) shape = torch.zeros(batch_size, 10) # (B, 10) beta parameters output: MANOOutput = mano_layer(pose, shape) verts = output.verts # (B, 778, 3) joints = output.joints # (B, 21, 3) transforms_abs = output.transforms_abs # (B, 16, 4, 4) print(verts.shape) print(joints.shape) print(transforms_abs.shape) # Example for PCA pose mode random_pose = torch.rand(batch_size, 3 + ncomps) # (B, 18) random_shape = torch.rand(batch_size, 10) out_pca: MANOOutput = mano_pca(random_pose, random_shape) print(out_pca.verts.shape) # Example for Quaternion mode quat_pose = torch.zeros(batch_size, 64) # (B, 16 * 4) out_quat: MANOOutput = mano_quat(quat_pose, shape) print(out_quat.joints.shape) ``` ### Parameters - **rot_mode**: (str) Rotation parameterization mode. Options: "axisang", "quat". - **side**: (str) Hand side. Options: "right", "left". - **center_idx**: (int or None) Index of the joint to re-center outputs around. If None, outputs are absolute. - **mano_assets_root**: (str) Path to the MANO assets directory. - **use_pca**: (bool) Whether to use PCA-compressed pose representation. - **flat_hand_mean**: (bool) Whether to use a flattened hand mean. - **ncomps**: (int) Number of principal components to use for PCA pose representation (only if `use_pca` is True). ### Returns - **MANOOutput**: A named tuple containing: - **verts**: (torch.Tensor) Hand mesh vertices (B, 778, 3). - **joints**: (torch.Tensor) Hand joints (B, 21, 3). - **transforms_abs**: (torch.Tensor) Absolute SE(3) transformation matrices per joint (B, 16, 4, 4). - **center_joint**: (torch.Tensor) The re-centered root joint (B, 1, 3). - **full_poses**: (torch.Tensor) Full pose coefficients (B, 48). - **betas**: (torch.Tensor) Shape parameters (B, 10). ``` -------------------------------- ### MANO Publication Citation Source: https://github.com/lixiny/manotorch/blob/master/README.md Citation for the original MANO publication, providing reference to the foundational hand modeling work. ```bibtex @article{MANO:SIGGRAPHASIA:2017, title = {Embodied Hands: Modeling and Capturing Hands and Bodies Together}, author = {Romero, Javier and Tzionas, Dimitrios and Black, Michael J.}, journal = {ACM Transactions on Graphics, (Proc. SIGGRAPH Asia)}, publisher = {ACM}, month = nov, year = {2017}, url = {http://doi.acm.org/10.1145/3130800.3130883}, month_numeric = {11} } ``` -------------------------------- ### AnatomyConstraintLossEE: Enforce Joint Angular Limits Source: https://context7.com/lixiny/manotorch/llms.txt Penalizes biomechanically infeasible poses by enforcing per-joint angular limits on anatomy-aligned Euler angles. Configures limits for various joint types. ```python import torch from math import pi from manotorch.manolayer import ManoLayer, MANOOutput from manotorch.axislayer import AxisLayerFK from manotorch.anatomy_loss import AnatomyConstraintLossEE mano_layer = ManoLayer(rot_mode="axisang", side="right", center_idx=9, mano_assets_root="assets/mano", flat_hand_mean=True) axis_layer = AxisLayerFK(side="right", mano_assets_root="assets/mano") anatomy_loss = AnatomyConstraintLossEE(reduction="mean") # Configure per-joint-type angular limits [twist, spread, bend] anatomy_loss.setup( thumb_cmc= ["+-:45", "+:45,-:15", "+:45,-:0"], thumb_mcp= ["+-:0", "+-:10", "+:90,-:0"], thumb_pip= ["+-:0", "+-:0", "+:90,-:0"], finger_mcp= ["+-:0", "+-:5", "+:90,-:0"], finger_pip= ["+-:0", "+-:0", "+:90,-:0"], finger_dip= ["+-:0", "+-:0", "+:90,-:0"], ) ``` -------------------------------- ### ManoLayer Rotation Center Source: https://context7.com/lixiny/manotorch/llms.txt When center_idx is set, the rotation center is always (0,0,0). ```python mano_layer_centered = ManoLayer(side="right", center_idx=9, mano_assets_root="assets/mano") print(mano_layer_centered.get_rotation_center(betas)) ``` -------------------------------- ### ManoLayer.get_mano_closed_faces — Watertight Mesh Faces Source: https://context7.com/lixiny/manotorch/llms.txt Retrieves the face tensor for the MANO mesh, including 14 additional triangles to close the wrist boundary, creating a watertight mesh suitable for rendering. ```APIDOC ## ManoLayer.get_mano_closed_faces — Watertight Mesh Faces ### Description This method returns the face tensor for the MANO mesh, augmented with 14 extra triangles to close the wrist opening. This results in a watertight mesh suitable for rendering and volumetric computations. ### Usage ```python import torch from manotorch.manolayer import ManoLayer mano_layer = ManoLayer(side="right", mano_assets_root="assets/mano") open_faces = mano_layer.th_faces # (1538, 3) closed_faces = mano_layer.get_mano_closed_faces() # (1552, 3) print(f"Open faces: {open_faces.shape}") print(f"Closed faces: {closed_faces.shape}") # Example with trimesh import trimesh pose = torch.zeros(1, 48) shape = torch.zeros(1, 10) output = mano_layer(pose, shape) mesh = trimesh.Trimesh( vertices=output.verts[0].detach().numpy(), faces=closed_faces.numpy(), ) mesh.export("hand_closed.obj") ``` ### Returns - **closed_faces**: (torch.Tensor) A tensor of shape (1552, 3) representing the watertight mesh faces. ``` -------------------------------- ### UpSampleLayer — Mesh Upsampling Source: https://context7.com/lixiny/manotorch/llms.txt `UpSampleLayer` subdivides a triangle mesh by inserting new vertices at each edge midpoint, quadrupling the face count at each call. It operates on batched vertex tensors and is fully differentiable, making it useful for progressive mesh refinement inside a network. ```APIDOC ## UpSampleLayer — Mesh Upsampling ### Description `UpSampleLayer` subdivides a triangle mesh by inserting new vertices at each edge midpoint, quadrupling the face count at each call. It operates on batched vertex tensors and is fully differentiable, making it useful for progressive mesh refinement inside a network. ### Usage ```python import torch from manotorch.manolayer import ManoLayer, MANOOutput from manotorch.upsamplelayer import UpSampleLayer mano_layer = ManoLayer(rot_mode="axisang", side="right", mano_assets_root="assets/mano") upsample_layer = UpSampleLayer() pose = torch.zeros(1, 48) shape = torch.zeros(1, 10) output: MANOOutput = mano_layer(pose, shape) verts = output.verts # (1, 778, 3) faces = mano_layer.th_faces # (1538, 3) # faces must be provided per sample in a list/batch faces_batch = faces.unsqueeze(0).repeat(1, 1, 1) # (1, 1538, 3) upsampled_verts, upsampled_faces = upsample_layer(verts, faces_batch) print(upsampled_verts.shape) # torch.Size([1, 2316, 3]) — 778 + 1538 new edge midpoints print(upsampled_faces.shape) # torch.Size([1, 6152, 3]) — 1538 × 4 # Second subdivision level verts2, faces2 = upsample_layer(upsampled_verts, upsampled_faces) print(verts2.shape) # torch.Size([1, 8468, 3]) ``` ``` -------------------------------- ### ManoLayer.get_rotation_center — Beta-Dependent Rotation Centre Source: https://context7.com/lixiny/manotorch/llms.txt Calculates the beta-dependent rotation center for the MANO hand model. This point, in the MANO reference frame, determines the center of rotation for the global root pose based on shape parameters. ```APIDOC ## ManoLayer.get_rotation_center — Beta-Dependent Rotation Centre ### Description This method computes the 3-D rotation center of the hand mesh in the MANO reference frame. This center is dependent on the hand's shape parameters (betas) and the joint regressor, and it dictates the pivot point for perturbations of the global root rotation. ### Usage ```python import torch from manotorch.manolayer import ManoLayer mano_layer = ManoLayer( side="right", center_idx=None, # must be None to get a non-zero centre mano_assets_root="assets/mano", ) batch_size = 2 betas = torch.rand(batch_size, 10) rot_center = mano_layer.get_rotation_center(betas) # (B, 3) print(rot_center) ``` ### Parameters - **betas**: (torch.Tensor) The shape parameters of the MANO model, with shape (B, 10). ### Returns - **rot_center**: (torch.Tensor) The calculated rotation center for each sample in the batch, with shape (B, 3). ``` -------------------------------- ### CPF Citation Source: https://github.com/lixiny/manotorch/blob/master/README.md Citation for the CPF paper, where manotorch was originally developed, useful for academic referencing. ```bibtex @inproceedings{yang2021cpf, title = {{CPF}: Learning a Contact Potential Field to Model the Hand-Object Interaction}, author = {Yang, Lixin and Zhan, Xinyu and Li, Kailin and Xu, Wenqiang and Li, Jiefeng and Lu, Cewu}, booktitle = {ICCV}, year = {2021} } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.