Pixel Difference Networks

https://github.com/zhuoinoulu/pidinet
Admin
Pixel Difference Networks (PiDiNet) is a PyTorch implementation for efficient edge detection using...
Tokens:6,732
Snippets:48
Trust Score:4.8
Update:5 months ago
Show doc for...
Tokens:
Raw
### Instantiate and Run PiDiNet Model

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Demonstrates how to instantiate PiDiNet models of different sizes for training or inference. Shows how to pass arguments for configuration and how to get model outputs.

```python
import torch, argparse
import models   # models/__init__.py exposes all factory functions

# Simulate CLI args
args = argparse.Namespace(config='carv4', sa=True, dil=True, gpu='0')

# ── Training-time model (uses custom PDC ops) ──────────────────────────────
model = models.pidinet(args)          # 60 base channels
# model = models.pidinet_small(args)  # 30 base channels
# model = models.pidinet_tiny(args)   # 20 base channels

model.eval()
x = torch.randn(1, 3, 256, 256)
with torch.no_grad():
    outputs = model(x)               # list of 5 tensors (4 side + 1 fused)
print([o.shape for o in outputs])
# [torch.Size([1,1,256,256])] × 5

# ── Check model size ───────────────────────────────────────────────────────
from utils import get_model_parm_nums
print('Model size: %.2f MB' % get_model_parm_nums(model))   # ~0.71 MB

# ── Inference-time converted model (vanilla CNN only) ─────────────────────
from models.convert_pidinet import convert_pidinet

checkpoint = torch.load('trained_models/table5_pidinet.pth', map_location='cpu')
converted_model = models.pidinet_converted(args)
converted_model.load_state_dict(convert_pidinet(checkpoint['state_dict'], args.config))
converted_model.eval()
with torch.no_grad():
    outputs_c = converted_model(x)
# Outputs are numerically identical to the original model
```

--------------------------------

### Run Matlab Evaluation Script

Source: https://github.com/zhuoinoulu/pidinet/blob/master/README.md

This demonstrates how to initiate the evaluation script within a Matlab environment. Ensure Matlab is installed and the evaluation code is correctly set up.

```bash
matlab -nosplash -nodisplay -nodesktop

# after entering the Matlab environment, 
>>> eval_bsds
```

--------------------------------

### Download Datasets

Source: https://github.com/zhuoinoulu/pidinet/blob/master/README.md

Use these wget commands to download the necessary datasets for training and evaluation. Ensure you have sufficient disk space.

```bash
wget http://mftp.mmcheng.net/liuyun/rcf/data/HED-BSDS.tar.gz
wget http://mftp.mmcheng.net/liuyun/rcf/data/PASCAL.tar.gz
wget http://mftp.mmcheng.net/liuyun/rcf/data/NYUD.tar.gz
```

--------------------------------

### Train PiDiNet Model in Bash

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Command to initiate the training process for the PiDiNet model. Requires specifying model, configuration, dataset, and saving directories. Training involves parameters like learning rate, epochs, and GPU usage.

```bash
python main.py \
  --model pidinet --config carv4 --sa --dil \
  --resume --iter-size 24 -j 4 --gpu 0 \
  --epochs 20 --lr 0.005 --lr-type multistep --lr-steps 10-16 --wd 1e-4 \
  --savedir /results/table5_pidinet \
  --datadir /data/BSDS500 --dataset BSDS
```

--------------------------------

### Initialize Dataset Loaders in Python

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Demonstrates initializing different dataset loaders like BSDS_VOCLoader, NYUD_Loader, Multicue_Loader, and Custom_Loader. Training loaders apply ImageNet normalization and threshold continuous annotation maps into ternary labels.

```python
from edge_dataloader import (
    BSDS_Loader, BSDS_VOCLoader,
    Multicue_Loader, NYUD_Loader, Custom_Loader
)
from torch.utils.data import DataLoader

# ── BSDS500 + PASCAL VOC augmentation (default training setup) ────────────
train_ds = BSDS_VOCLoader(root='/data/BSDS500', split='train',
                           threshold=0.3, ablation=False)
test_ds  = BSDS_VOCLoader(root='/data/BSDS500', split='test', threshold=0.3)
train_dl = DataLoader(train_ds, batch_size=1, num_workers=4, shuffle=True)
for img, label in train_dl:
    print(img.shape, label.shape)   # (1,3,H,W)  (1,1,H,W) with values {0,1,2}
    break

# ── NYUDv2 RGB ────────────────────────────────────────────────────────────
nyud_ds = NYUD_Loader(root='/data/NYUD', split='test', setting=['image'])
print(len(nyud_ds))

# ── Multicue boundary, set 1 ──────────────────────────────────────────────
mc_ds = Multicue_Loader(root='/data/Multicue', split='train',
                         threshold=0.3, setting=['boundary', '1'])

# ── Custom images (inference only) ────────────────────────────────────────
custom_ds = Custom_Loader(root='/path/to/my_images')
custom_dl = DataLoader(custom_ds, batch_size=1, shuffle=False)
for img, filename in custom_dl:
    print(img.shape, filename)   # (1,3,H,W)  ['my_photo']
    break
```

--------------------------------

### Test PiDiNet FPS on GPU

Source: https://github.com/zhuoinoulu/pidinet/blob/master/README.md

Use this command to test the Frames Per Second (FPS) performance of PiDiNet models on a GPU. Ensure you have the necessary data and model configurations.

```bash
python throughput.py --model pidinet_converted --config carv4 --sa --dil -j 1 --gpu 0 --datadir /path/to/BSDS500 --dataset BSDS
```

--------------------------------

### Train PiDiNet Model

Source: https://github.com/zhuoinoulu/pidinet/blob/master/README.md

Command to train the PiDiNet model. Checkpoints will be saved to the specified directory. Adjust parameters like learning rate, epochs, and dataset paths as needed.

```bash
python main.py --model pidinet --config carv4 --sa --dil --resume --iter-size 24 -j 4 --gpu 0 --epochs 20 --lr 0.005 --lr-type multistep --lr-steps 10-16 --wd 1e-4 --savedir /path/to/table5_pidinet --datadir /path/to/BSDS500 --dataset BSDS
```

--------------------------------

### Configure PiDiNet Model Architecture

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Functions to retrieve ordered lists of PDC operator functions for training or string keys for the converted vanilla model during inference. Pre-defined configurations like 'carv4' (cd–ad–rd–cv repeated) and 'baseline' are available. Lists all available network configurations.

```python
from models.config import config_model, config_model_converted

# Runtime PDC functions — used during training
pdcs_train = config_model('carv4')
print(len(pdcs_train))        # 16
print(pdcs_train[0])          # <function createConvFunc.<locals>.func at 0x...>

# String labels — used when building the converted (vanilla) model
pdcs_infer = config_model_converted('carv4')
print(pdcs_infer[:4])         # ['cd', 'ad', 'rd', 'cv']

# List all available configurations
from models.config import nets
print(list(nets.keys()))
# ['baseline', 'c-v15', 'a-v15', 'r-v15', 'cvvv4', 'avvv4', 'rvvv4',
#  'cccv4', 'aaav4', 'rrrv4', 'c16', 'a16', 'r16', 'carv4']
```

--------------------------------

### Train Tiny PiDiNet on NYUD Dataset in Bash

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Command to train a smaller variant of PiDiNet ('pidinet_tiny') on the NYUD dataset, specifically using HHA input. Includes parameters for learning rate, epochs, and a lambda value.

```bash
python main.py \
  --model pidinet_tiny --config carv4 --sa --dil \
  --iter-size 24 -j 4 --gpu 0 \
  --epochs 14 --lr 0.005 --lr-type multistep --lr-steps 8-12 --wd 1e-4 \
  --savedir /results/nyud_tiny \
  --datadir /data/NYUD --dataset NYUD-hha --lmbda 1.3
```

--------------------------------

### Benchmark Full PiDiNet FPS in Bash

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Measures the inference throughput (frames per second) of the full PiDiNet model using the converted variant for realistic deployment numbers. Uses `time.perf_counter` and `torch.cuda.synchronize`.

```bash
python throughput.py \
  --model pidinet_converted --config carv4 --sa --dil \
  -j 1 --gpu 0 \
  --datadir /data/BSDS500 --dataset BSDS
```

--------------------------------

### Run PiDiNet on Custom Images in Bash

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Command to process custom images using the converted PiDiNet model. This is useful for inference on user-provided image folders.

```bash
python main.py \
  --model pidinet_converted --config carv4 --sa --dil -j 4 --gpu 0 \
  --savedir /results/custom_out \
  --datadir /path/to/my_images --dataset Custom \
  --evaluate /results/table5_pidinet/save_models/checkpoint_019.pth \
  --evaluate-converted
```

--------------------------------

### Generate Edge Maps with Original PiDiNet Model in Bash

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Command to generate edge maps using a pre-trained PiDiNet model checkpoint. Skips training and directly evaluates the model on the specified dataset.

```bash
python main.py \
  --model pidinet --config carv4 --sa --dil -j 4 --gpu 0 \
  --savedir /results/table5_pidinet \
  --datadir /data/BSDS500 --dataset BSDS \
  --evaluate /results/table5_pidinet/save_models/checkpoint_019.pth
```

--------------------------------

### Generate Edge Maps with Original Model

Source: https://github.com/zhuoinoulu/pidinet/blob/master/README.md

Use this command to generate edge maps using the trained original PiDiNet model. Specify the savedir and datadir paths.

```bash
python main.py --model pidinet --config carv4 --sa --dil -j 4 --gpu 0 --savedir /path/to/table5_pidinet --datadir /path/to/BSDS500 --dataset BSDS --evaluate /path/to/table5_pidinet/save_models/checkpointxxx.pth
```

--------------------------------

### Generate Edge Maps with Converted PiDiNet Model in Bash

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Command to generate edge maps using the converted (vanilla CNN) PiDiNet model. This involves an initial conversion step before running the evaluation.

```bash
python main.py \
  --model pidinet_converted --config carv4 --sa --dil -j 4 --gpu 0 \
  --savedir /results/table5_pidinet \
  --datadir /data/BSDS500 --dataset BSDS \
  --evaluate /results/table5_pidinet/save_models/checkpoint_019.pth \
  --evaluate-converted
```

--------------------------------

### Generate Edge Maps for Custom Images

Source: https://github.com/zhuoinoulu/pidinet/blob/master/README.md

This command generates edge maps for your own images using a specified PiDiNet model and checkpoint. Adjust paths and model configurations as needed.

```bash
python main.py --model pidinet_converted --config carv4 --sa --dil -j 4 --gpu 0 --savedir /path/to/savedir --datadir /path/to/custom_images --dataset Custom --evaluate /path/to/table5_pidinet/save_models/checkpointxxx.pth --evaluate-converted
```

--------------------------------

### Benchmark PiDiNet-tiny FPS in Bash

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Measures the inference throughput (frames per second) of the 'pidinet_tiny' model using the converted variant. This benchmark is useful for comparing performance of smaller model architectures.

```bash
python throughput.py \
  --model pidinet_tiny_converted --config carv4 --sa --dil \
  -j 2 --gpu 0 \
  --datadir /data/BSDS500 --dataset BSDS
```

--------------------------------

### Convert PiDiNet Checkpoint Weights

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Converts PDC, CDC, and ADC kernel weights from a saved state dictionary into standard convolution kernels. This allows loading into a converted PiDiNet model for deployment without custom operations.

```python
import torch
from models.convert_pidinet import convert_pidinet
import models, argparse

args = argparse.Namespace(config='carv4', sa=True, dil=True, gpu='')

# Load original checkpoint
ckpt = torch.load('trained_models/table5_pidinet.pth', map_location='cpu')
original_state = ckpt['state_dict']

# Convert weights
converted_state = convert_pidinet(original_state, 'carv4')

# Verify kernel shapes changed for rd layers (3x3 → 5x5)
for k in original_state:
    if 'conv1.weight' in k:
        orig_shape = tuple(original_state[k].shape)
        conv_shape = tuple(converted_state[k].shape)
        if orig_shape != conv_shape:
            print(f'{k}: {orig_shape} → {conv_shape}')
# block?_3.conv1.weight: (C, 1, 3, 3) → (C, 1, 5, 5)  (rd layers)

# Load into converted model and run
model = models.pidinet_converted(args)
model.load_state_dict(converted_state)
model.eval()
x = torch.randn(1, 3, 512, 512)
with torch.no_grad():
    out = model(x)
print(out[-1].shape)   # torch.Size([1, 1, 512, 512])
```

--------------------------------

### Generate Edge Maps with Converted Model

Source: https://github.com/zhuoinoulu/pidinet/blob/master/README.md

This command generates edge maps using a converted PiDiNet model, which should produce identical results to the original model. The conversion process transforms PiDiNet into a vanilla CNN.

```bash
python main.py --model pidinet_converted --config carv4 --sa --dil -j 4 --gpu 0 --savedir /path/to/table5_pidinet --datadir /path/to/BSDS500 --dataset BSDS --evaluate /path/to/table5_pidinet/save_models/checkpointxxx.pth --evaluate-converted
```

--------------------------------

### Benchmark PiDiNet-tiny-l FPS in Bash

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Measures the inference throughput (frames per second) of the 'pidinet_tiny_l' model variant, which omits attention and dilation modules. This provides FPS numbers for a more streamlined model.

```bash
python throughput.py \
  --model pidinet_tiny_converted --config carv4 \
  -j 2 --gpu 0 \
  --datadir /data/BSDS500 --dataset BSDS
```

--------------------------------

### createConvFunc(op_type)

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Factory function to create pixel difference convolution functions for various operation types ('cv', 'cd', 'ad', 'rd'). The returned function mimics `torch.nn.functional.conv2d`.

```APIDOC
## `createConvFunc(op_type)` — PDC operator factory

Creates a pixel difference convolution function for one of four operation types: `'cv'` (vanilla), `'cd'` (central difference), `'ad'` (angular difference), or `'rd'` (radial difference). The returned callable has the same signature as `torch.nn.functional.conv2d` and is used directly inside the custom `Conv2d` layer.

```python
from models.ops import createConvFunc
import torch

# Create a central-difference conv function
cd_func = createConvFunc('cd')

# Manually apply it to a feature map (B=1, C=8, H=32, W=32)
x       = torch.randn(1, 8, 32, 32)
weight  = torch.randn(8, 1, 3, 3)          # depthwise: out=8, in/groups=1
output  = cd_func(x, weight, bias=None, stride=1, padding=1, groups=8)
print(output.shape)   # torch.Size([1, 8, 32, 32])

# Compare all four operators on the same input
for op in ['cv', 'cd', 'ad', 'rd']:
    fn  = createConvFunc(op)
    pad = 2 if op == 'rd' else 1            # rd expands kernel to 5x5 internally
    out = fn(x, weight, bias=None, stride=1, padding=pad, groups=8)
    print(f'{op}: {out.shape}, mean={out.mean():.4f}')
```
```

--------------------------------

### Conv2d

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

A `nn.Module` that serves as a drop-in replacement for `nn.Conv2d`. It integrates a PDC function with the learnable weight tensor, supporting all standard `nn.Conv2d` arguments plus a `pdc` argument for the convolution function.

```APIDOC
## `Conv2d` — PDC-aware convolution layer (`models/ops.py`)

A drop-in `nn.Module` replacement for `nn.Conv2d` that stores a PDC function alongside the learnable weight tensor. All standard `nn.Conv2d` arguments are supported; the only addition is the leading `pdc` positional argument that accepts a function returned by `createConvFunc`.

```python
from models.ops import Conv2d, createConvFunc
import torch, torch.nn as nn

cd  = createConvFunc('cd')
ad  = createConvFunc('ad')

# Depthwise PDC layer (groups == in_channels)
layer_cd = Conv2d(cd, in_channels=16, out_channels=16,
                  kernel_size=3, padding=1, groups=16, bias=False)
layer_ad = Conv2d(ad, in_channels=16, out_channels=32,
                  kernel_size=3, padding=1, bias=False)

x = torch.randn(2, 16, 64, 64)
y = layer_ad(layer_cd(x))
print(y.shape)   # torch.Size([2, 32, 64, 64])

# Count parameters — same as equivalent nn.Conv2d
total = sum(p.numel() for p in layer_cd.parameters())
print(f'layer_cd params: {total}')   # 144  (16 * 1 * 3 * 3)
```
```

--------------------------------

### config_model(config) / config_model_converted(config)

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Functions to retrieve ordered lists of PDC operator functions or string keys defining the convolution types for PiDiNet's 16 blocks. Supports pre-defined configurations like 'carv4', 'baseline', and single-type configs.

```APIDOC
## `config_model(config)` / `config_model_converted(config)` — architecture configuration (`models/config.py`)

Returns an ordered list of 16 PDC operator functions (or string keys for the converted variant) that define the convolution type used in each of the 16 blocks of PiDiNet. Pre-defined configurations include `'carv4'` (the best-performing config: cd–ad–rd–cv repeated four times), `'baseline'` (all vanilla), and single-type configs such as `'c16'`, `'a16'`, `'r16'`.

```python
from models.config import config_model, config_model_converted

# Runtime PDC functions — used during training
pdcs_train = config_model('carv4')
print(len(pdcs_train))        # 16
print(pdcs_train[0])          # <function createConvFunc.<locals>.func at 0x...>

# String labels — used when building the converted (vanilla) model
pdcs_infer = config_model_converted('carv4')
print(pdcs_infer[:4])         # ['cd', 'ad', 'rd', 'cv']

# List all available configurations
from models.config import nets
print(list(nets.keys()))
# ['baseline', 'c-v15', 'a-v15', 'r-v15', 'cvvv4', 'avvv4', 'rvvv4',
#  'cccv4', 'aaav4', 'rrrv4', 'c16', 'a16', 'r16', 'carv4']
```
```

--------------------------------

### Compute Class-Balanced Edge Loss

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Calculates the class-balanced binary cross-entropy loss for edge detection, weighting positive and negative edge pixels appropriately. Ambiguous pixels are ignored to prevent them from dominating the gradient.

```python
import torch
from utils import cross_entropy_loss_RCF

# Simulate a batch: 1 image, 1 label map (H=4, W=4)
# Labels: 0=non-edge, 1=edge, 2=ignore
pred  = torch.sigmoid(torch.randn(1, 1, 4, 4))
label = torch.zeros(1, 1, 4, 4)
label[0, 0, 1, 1] = 1.0   # one edge pixel
label[0, 0, 2, 2] = 2.0   # one ambiguous pixel (ignored)

beta = 1.1   # slightly upweight negatives (paper default)
loss = cross_entropy_loss_RCF(pred, label, beta)
print(f'loss: {loss.item():.4f}')

# In the training loop (main.py pattern):
outputs = model(image)   # list of 5 tensors
total_loss = sum(cross_entropy_loss_RCF(o, label, args.lmbda) for o in outputs)
total_loss = total_loss / args.iter_size   # gradient accumulation
total_loss.backward()
```

--------------------------------

### Create Pixel Difference Convolution Function

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

Factory function to create callable convolution operators for 'cv', 'cd', 'ad', or 'rd' types. The returned function mimics `torch.nn.functional.conv2d` and is used within custom `Conv2d` layers. Note that 'rd' requires larger padding due to its internal 5x5 kernel expansion.

```python
from models.ops import createConvFunc
import torch

# Create a central-difference conv function
cd_func = createConvFunc('cd')

# Manually apply it to a feature map (B=1, C=8, H=32, W=32)
x       = torch.randn(1, 8, 32, 32)
weight  = torch.randn(8, 1, 3, 3)          # depthwise: out=8, in/groups=1
output  = cd_func(x, weight, bias=None, stride=1, padding=1, groups=8)
print(output.shape)   # torch.Size([1, 8, 32, 32])

# Compare all four operators on the same input
for op in ['cv', 'cd', 'ad', 'rd']:
    fn  = createConvFunc(op)
    pad = 2 if op == 'rd' else 1            # rd expands kernel to 5x5 internally
    out = fn(x, weight, bias=None, stride=1, padding=pad, groups=8)
    print(f'{op}: {out.shape}, mean={out.mean():.4f}')
```

--------------------------------

### PDC-Aware Convolution Layer

Source: https://context7.com/zhuoinoulu/pidinet/llms.txt

A `nn.Module` that replaces `nn.Conv2d`, storing a PDC function and learnable weights. It supports standard `nn.Conv2d` arguments, with an additional `pdc` argument for the function returned by `createConvFunc`. The parameter count is identical to an equivalent `nn.Conv2d` layer.

```python
from models.ops import Conv2d, createConvFunc
import torch, torch.nn as nn

cd  = createConvFunc('cd')
ad  = createConvFunc('ad')

# Depthwise PDC layer (groups == in_channels)
layer_cd = Conv2d(cd, in_channels=16, out_channels=16,
                  kernel_size=3, padding=1, groups=16, bias=False)
layer_ad = Conv2d(ad, in_channels=16, out_channels=32,
                  kernel_size=3, padding=1, bias=False)

x = torch.randn(2, 16, 64, 64)
y = layer_ad(layer_cd(x))
print(y.shape)   # torch.Size([2, 32, 64, 64])

# Count parameters — same as equivalent nn.Conv2d
total = sum(p.numel() for p in layer_cd.parameters())
print(f'layer_cd params: {total}')   # 144  (16 * 1 * 3 * 3)
```
Tokens:
Raw