HandRefiner

https://github.com/wenquanlu/handrefiner
Admin
HandRefiner is a lightweight post-processing solution that uses a diffusion-based conditional...
Tokens:2,282
Snippets:37
Trust Score:7.4
License:MIT
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install Mediapipe and Download Hand Landmark Model

Source: https://github.com/wenquanlu/handrefiner/blob/main/docs/installation.md

Installs Mediapipe version 0.10.0 and downloads the hand landmark model task file. Ensure you are in the 'HandRefiner/' directory before running.

```bash
pip install -q mediapipe==0.10.0
cd preprocessor
wget https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task
```

--------------------------------

### Dataset Structure Example

Source: https://github.com/wenquanlu/handrefiner/blob/main/training/README.md

The dataset needs to be structured with separate directories for images, masks, poses, and a prompt JSON file.

```bash
|- dataset1
|   |- image
|   |- mask
|   |- pose
|   |- prompt.json
```

--------------------------------

### Clone MeshGraphormer Repository and Install Dependencies

Source: https://github.com/wenquanlu/handrefiner/blob/main/docs/meshgraphormer.md

Clone the MeshGraphormer repository recursively and install the required Python package.

```bash
git clone --recursive https://github.com/microsoft/MeshGraphormer.git
cd MeshGraphormer
pip install ./manopth/.
```

--------------------------------

### Navigate Out of MeshGraphormer Directory

Source: https://github.com/wenquanlu/handrefiner/blob/main/docs/meshgraphormer.md

Exit the MeshGraphormer directory after completing the setup steps.

```bash
cd ..
```

--------------------------------

### Prompt JSON Structure Example

Source: https://github.com/wenquanlu/handrefiner/blob/main/training/README.md

Each prompt JSON file should contain the image name, text prompt, and dataset identifier.

```json
{"jpg": "image name", "txt": "text prompt", "dataset": "dataset identifier (RHD|synthesisai)"}
```

--------------------------------

### Create Models Directory

Source: https://github.com/wenquanlu/handrefiner/blob/main/docs/meshgraphormer.md

Create a directory to store pretrained models.

```bash
mkdir -p models  # pre-trained models
```

--------------------------------

### Download Pretrained Models and Code Modifications

Source: https://github.com/wenquanlu/handrefiner/blob/main/docs/meshgraphormer.md

Copy necessary scripts and download pretrained models. Ensure you are in the 'HandRefiner/MeshGraphormer' directory before running.

```bash
cp ../scripts/download_models.sh scripts/download_models.sh
cp ../scripts/_gcnn.py src/modeling/_gcnn.py
cp ../scripts/_mano.py src/modeling/_mano.py
cp ../scripts/config.py src/modeling/data/config.py
bash scripts/download_models.sh
```

--------------------------------

### Prompt File Format for Hand Refinement

Source: https://github.com/wenquanlu/handrefiner/blob/main/docs/manual.md

Specifies the JSON format for a prompt file used in multi-image rectification. Each entry maps a filename to a descriptive text prompt.

```json
{
"img": "img1.jpg", 
"txt": "a woman making a hand gesture"
}

```

```json
{
"img": "img2.jpg", 
"txt": "a man making a hand gesture"
}

```

```json
{
"img": "img3.jpg", 
"txt": "a man making a thumbs up gesture"
}

```

--------------------------------

### Test Multiple Image Rectifications

Source: https://github.com/wenquanlu/handrefiner/blob/main/docs/installation.md

Processes multiple images for rectification using a prompt file. Specify input directory, output directory, strength, weights, prompt file, and seed.

```bash
python handrefiner.py --input_dir test --out_dir output --strength 0.55 --weights models/inpaint_depth_control.ckpt --prompt_file test/test.json --seed 1
```

--------------------------------

### Test Single Image Rectification

Source: https://github.com/wenquanlu/handrefiner/blob/main/docs/installation.md

Runs HandRefiner on a single image for rectification. Specify input image, output directory, strength, weights, prompt, and seed.

```bash
python handrefiner.py --input_img test/1.jpg --out_dir output --strength 0.55 --weights models/inpaint_depth_control.ckpt --prompt "a man facing the camera, making a hand gesture, indoor" --seed 1
```
Tokens:
Raw