### Initialize Baseline Model Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Setup and run the baseline graph convolution model for action recognition. ```python from model.baseline import Model import torch # Initialize baseline model baseline = Model( num_class=120, num_point=25, num_person=2, graph='graph.ntu_rgb_d.Graph', graph_args={'labeling_mode': 'spatial'}, in_channels=3, drop_out=0, adaptive=True, num_set=3 # Number of adjacency subsets ) # Input: (batch, channels, time, joints, persons) x = torch.randn(32, 3, 64, 25, 2) output = baseline(x) # Shape: (32, 120) # Training command for baseline # python main.py --config config/nturgbd120-cross-subject/default.yaml \ # --model model.baseline.Model \ # --work-dir work_dir/ntu120/csub/baseline --device 0 ``` -------------------------------- ### Install Torchlight Source: https://github.com/uason-chen/ctr-gcn/blob/main/README.md Installs the torchlight library, which is a dependency for this project. ```bash pip install -e torchlight ``` -------------------------------- ### Train Baseline Model on NTU RGB+D 120 Source: https://github.com/uason-chen/ctr-gcn/blob/main/README.md Example command to train the provided baseline model on the NTU RGB+D 120 dataset using cross-subject split on GPU 0. ```bash python main.py --config config/nturgbd120-cross-subject/default.yaml --model model.baseline.Model--work-dir work_dir/ntu120/csub/baseline --device 0 ``` -------------------------------- ### Train Custom Model Source: https://github.com/uason-chen/ctr-gcn/blob/main/README.md Example command to train a custom model. Place your model file (e.g., `your_model.py`) in the `./model` directory and specify it using the `--model` argument. ```bash python main.py --config config/nturgbd120-cross-subject/default.yaml --model model.your_model.Model --work-dir work_dir/ntu120/csub/your_model --device 0 ``` -------------------------------- ### Ensemble Modality Results Source: https://github.com/uason-chen/ctr-gcn/blob/main/README.md Example command to ensemble results from different modalities (joint, bone, motion) for CTRGCN on NTU RGB+D 120 cross-subject. ```bash python ensemble.py --datasets ntu120/xsub --joint-dir work_dir/ntu120/csub/ctrgcn --bone-dir work_dir/ntu120/csub/ctrgcn_bone --joint-motion-dir work_dir/ntu120/csub/ctrgcn_motion --bone-motion-dir work_dir/ntu120/csub/ctrgcn_bone_motion ``` -------------------------------- ### CTR-GCN Configuration File Structure Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Example YAML configuration file for CTR-GCN training. Defines data feeder arguments, model architecture, and optimization hyperparameters. ```yaml # config/nturgbd120-cross-subject/default.yaml work_dir: ./work_dir/ntu120/xsub/ctrgcn_joint # Data feeder configuration feeder: feeders.feeder_ntu.Feeder train_feeder_args: data_path: data/ntu120/NTU120_CSub.npz split: train debug: False random_choose: False random_shift: False random_move: False window_size: 64 normalization: False random_rot: True p_interval: [0.5, 1] vel: False bone: False test_feeder_args: data_path: data/ntu120/NTU120_CSub.npz split: test window_size: 64 p_interval: [0.95] vel: False bone: False debug: False # Model configuration model: model.ctrgcn.Model model_args: num_class: 120 num_point: 25 num_person: 2 graph: graph.ntu_rgb_d.Graph graph_args: labeling_mode: 'spatial' # Optimization settings weight_decay: 0.0004 base_lr: 0.1 lr_decay_rate: 0.1 step: [35, 55] warm_up_epoch: 5 ``` -------------------------------- ### Train CTRGCN on NTU RGB+D 120 Source: https://github.com/uason-chen/ctr-gcn/blob/main/README.md Example command to train the CTRGCN model on the NTU RGB+D 120 dataset using cross-subject split on GPU 0. ```bash python main.py --config config/nturgbd120-cross-subject/default.yaml --work-dir work_dir/ntu120/csub/ctrgcn --device 0 ``` -------------------------------- ### Train CTRGCN with Bone Modality Source: https://github.com/uason-chen/ctr-gcn/blob/main/README.md Example command to train CTRGCN on NTU RGB+D 120 cross-subject using the bone modality. Bone modality is enabled by setting `bone=True` in feeder arguments. ```bash python main.py --config config/nturgbd120-cross-subject/default.yaml --train_feeder_args bone=True --test_feeder_args bone=True --work-dir work_dir/ntu120/csub/ctrgcn_bone --device 0 ``` -------------------------------- ### Execute Data Preparation Pipeline Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Shell commands to process raw NTU RGB+D skeleton files into training-ready formats. ```bash # Step 1: Extract raw skeleton data from .skeleton files cd data/ntu # or data/ntu120 # Get skeleton data for each performer python get_raw_skes_data.py # Step 2: Remove bad/noisy skeletons python get_raw_denoised_data.py # Step 3: Transform skeleton to center of first frame python seq_transformation.py # Output: NTU60_CS.npz, NTU60_CV.npz (or NTU120_CSub.npz, NTU120_CSet.npz) ``` -------------------------------- ### Configure Training Settings Source: https://context7.com/uason-chen/ctr-gcn/llms.txt YAML-based configuration for training parameters including device, batch size, and epoch count. ```yaml device: [0] batch_size: 64 test_batch_size: 64 num_epoch: 65 nesterov: True ``` -------------------------------- ### Initialize Training Data Feeder for NTU RGB+D 120 Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Initializes a Feeder object for training data. Supports various augmentation techniques like random rotation and temporal cropping. Set `bone=True` for bone modality or `vel=True` for motion modality. ```python train_feeder = Feeder( data_path='data/ntu120/NTU120_CSub.npz', label_path=None, split='train', random_choose=False, random_shift=False, random_move=False, random_rot=True, # Enable random rotation augmentation window_size=64, # Temporal window size normalization=False, debug=False, use_mmap=False, p_interval=[0.5, 1], # Random temporal crop interval bone=False, # Set True for bone modality vel=False # Set True for motion/velocity modality ) ``` ```python # Create DataLoader train_loader = DataLoader( dataset=train_feeder, batch_size=64, shuffle=True, num_workers=8, drop_last=True ) ``` ```python # Iterate through batches for data, label, index in train_loader: print(f"Data shape: {data.shape}") # (64, 3, 64, 25, 2) print(f"Labels shape: {label.shape}") # (64,) break ``` ```python # Enable bone modality (relative joint positions) bone_feeder = Feeder( data_path='data/ntu120/NTU120_CSub.npz', split='train', window_size=64, random_rot=True, p_interval=[0.5, 1], bone=True, # Enable bone modality vel=False ) ``` -------------------------------- ### Initialize CTR-GCN Model Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Instantiates the full CTR-GCN architecture for skeleton-based action recognition. Requires defining the number of classes, joints, and persons to match the target dataset. ```python from model.ctrgcn import Model import torch # Initialize CTR-GCN model for NTU RGB+D 120 dataset model = Model( num_class=120, # Number of action classes num_point=25, # Number of skeleton joints (25 for NTU) num_person=2, # Maximum number of persons per frame graph='graph.ntu_rgb_d.Graph', # Graph structure class graph_args={'labeling_mode': 'spatial'}, in_channels=3, # Input channels (x, y, z coordinates) drop_out=0, # Dropout rate adaptive=True # Use adaptive graph learning ) # Input shape: (batch_size, channels, temporal_frames, joints, persons) # Example: 64 samples, 3 coords, 64 frames, 25 joints, 2 persons x = torch.randn(64, 3, 64, 25, 2) output = model(x) # Shape: (64, 120) - class logits # Get predicted action class predictions = torch.argmax(output, dim=1) print(f"Predicted actions: {predictions}") ``` -------------------------------- ### Test Trained Models Source: https://github.com/uason-chen/ctr-gcn/blob/main/README.md Command to test models trained and saved in a specified work directory. Requires the config file path, work directory, phase set to 'test', and the path to the trained weights. ```bash python main.py --config /config.yaml --work-dir --phase test --save-score True --weights /xxx.pt --device 0 ``` -------------------------------- ### Initialize Multi-Scale Temporal Convolution Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Sets up temporal convolutions with multiple dilation rates to capture dependencies across different time scales. ```python from model.ctrgcn import MultiScale_TemporalConv import torch # Initialize multi-scale temporal convolution ms_tcn = MultiScale_TemporalConv( in_channels=64, out_channels=64, kernel_size=3, stride=1, dilations=[1, 2, 3, 4], # Multiple dilation rates residual=True, residual_kernel_size=1 ) # Input: (batch, channels, time, vertices) x = torch.randn(32, 64, 64, 25) output = ms_tcn(x) print(f"Output shape: {output.shape}") # (32, 64, 64, 25) ``` -------------------------------- ### Train CTR-GCN with Joint Modality Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Trains the CTR-GCN model on NTU RGB+D 120 dataset using the joint modality. Specify the configuration file, working directory, and device. ```bash python main.py \ --config config/nturgbd120-cross-subject/default.yaml \ --work-dir work_dir/ntu120/csub/ctrgcn \ --device 0 ``` -------------------------------- ### Train Baseline Model Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Trains a baseline model instead of CTR-GCN. Specify the model path using the `--model` argument. ```bash python main.py \ --config config/nturgbd120-cross-subject/default.yaml \ --model model.baseline.Model \ --work-dir work_dir/ntu120/csub/baseline \ --device 0 ``` -------------------------------- ### Generate NTU RGB+D Dataset Source: https://github.com/uason-chen/ctr-gcn/blob/main/README.md Scripts to process raw skeleton data for NTU RGB+D datasets, including extracting performer data, removing bad skeletons, and transforming skeleton data. ```bash cd ./data/ntu # or cd ./data/ntu120 # Get skeleton of each performer python get_raw_skes_data.py # Remove the bad skeleton python get_raw_denoised_data.py # Transform the skeleton to the center of the first frame python seq_transformation.py ``` -------------------------------- ### Import NTU RGB+D Data Feeder Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Imports the necessary classes for loading and preprocessing skeleton data from the NTU RGB+D dataset. ```python from feeders.feeder_ntu import Feeder from torch.utils.data import DataLoader ``` -------------------------------- ### Train CTR-GCN with Bone Modality Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Trains the CTR-GCN model using the bone modality. This requires setting `bone=True` for both training and testing feeder arguments. ```bash python main.py \ --config config/nturgbd120-cross-subject/default.yaml \ --train_feeder_args bone=True \ --test_feeder_args bone=True \ --work-dir work_dir/ntu120/csub/ctrgcn_bone \ --device 0 ``` -------------------------------- ### Initialize CTRGC Module Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Configures the core topology refinement module for dynamic graph learning. The module computes correlations between joints per channel to refine the graph structure. ```python from model.ctrgcn import CTRGC import torch # Initialize CTRGC module ctrgc = CTRGC( in_channels=64, # Input feature channels out_channels=64, # Output feature channels rel_reduction=8, # Reduction ratio for relation channels mid_reduction=1 # Reduction ratio for middle channels ) # Input feature map: (batch, channels, time, vertices) x = torch.randn(32, 64, 64, 25) # Pre-defined adjacency matrix (optional, can be learned) A = torch.randn(25, 25) # Spatial graph adjacency # Forward pass with topology refinement output = ctrgc(x, A=A, alpha=1.0) print(f"Output shape: {output.shape}") # (32, 64, 64, 25) ``` -------------------------------- ### Define Graph Structure Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Initialize the skeleton graph topology and access adjacency matrices for NTU RGB+D data. ```python from graph.ntu_rgb_d import Graph import numpy as np # Initialize graph for NTU RGB+D skeleton graph = Graph(labeling_mode='spatial') # Access adjacency matrix # Shape: (3, 25, 25) for 3 subsets (self, inward, outward) adjacency = graph.A print(f"Adjacency shape: {adjacency.shape}") # Graph properties print(f"Number of joints: {graph.num_node}") # 25 print(f"Self links: {graph.self_link[:5]}") # [(0,0), (1,1), ...] print(f"Inward edges: {graph.inward[:5]}") # Parent-child connections print(f"Outward edges: {graph.outward[:5]}") # Child-parent connections # Joint indices for NTU RGB+D (0-indexed): # 0: Base of spine, 1: Middle of spine, 2: Neck, 3: Head # 4: Left shoulder, 5: Left elbow, 6: Left wrist, 7: Left hand # 8: Right shoulder, 9: Right elbow, 10: Right wrist, 11: Right hand # ... ``` -------------------------------- ### Train CTRGCN on NW-UCLA Source: https://github.com/uason-chen/ctr-gcn/blob/main/README.md Command to train CTRGCN on the NW-UCLA dataset. Requires modifying `data_path` in feeder arguments to specify 'bone', 'motion', or 'bone motion'. ```bash python main.py --config config/ucla/default.yaml --work-dir work_dir/ucla/ctrgcn_xxx --device 0 ``` -------------------------------- ### Train CTR-GCN with Motion Modality Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Trains the CTR-GCN model using the motion (velocity) modality. This requires setting `vel=True` for both training and testing feeder arguments. ```bash python main.py \ --config config/nturgbd120-cross-subject/default.yaml \ --train_feeder_args vel=True \ --test_feeder_args vel=True \ --work-dir work_dir/ntu120/csub/ctrgcn_motion \ --device 0 ``` -------------------------------- ### Apply Data Augmentation Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Perform temporal cropping, rotation, and spatial transformations on skeleton sequences. ```python from feeders.tools import valid_crop_resize, random_rot, random_move import numpy as np import torch # Sample skeleton data: (C, T, V, M) = (3, 300, 25, 2) data = np.random.randn(3, 300, 25, 2).astype(np.float32) valid_frames = 250 # Number of valid frames # Temporal crop and resize to fixed window cropped = valid_crop_resize( data_numpy=data, valid_frame_num=valid_frames, p_interval=[0.5, 1], # Random crop 50-100% of valid frames window=64 # Target temporal size ) print(f"Cropped shape: {cropped.shape}") # (3, 64, 25, 2) # Random 3D rotation augmentation rotated = random_rot(data, theta=0.3) # Max rotation: 0.3 radians print(f"Rotated shape: {rotated.shape}") # Random spatial transformation (scale, translate, rotate) transformed = random_move( data, angle_candidate=[-10., -5., 0., 5., 10.], scale_candidate=[0.9, 1.0, 1.1], transform_candidate=[-0.2, -0.1, 0.0, 0.1, 0.2] ) ``` -------------------------------- ### Ensemble Four Modalities Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Combines predictions from four modalities (joint, bone, joint-motion, bone-motion) using weighted summation for optimal accuracy. Specify the directory for each modality's results. ```bash python ensemble.py \ --dataset ntu120/xsub \ --joint-dir work_dir/ntu120/csub/ctrgcn \ --bone-dir work_dir/ntu120/csub/ctrgcn_bone \ --joint-motion-dir work_dir/ntu120/csub/ctrgcn_motion \ --bone-motion-dir work_dir/ntu120/csub/ctrgcn_bone_motion ``` -------------------------------- ### Test Trained CTR-GCN Model Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Evaluates a trained CTR-GCN model using a saved checkpoint. This command computes Top-1 and Top-5 accuracy and can save prediction scores. ```bash python main.py \ --config work_dir/ntu120/csub/ctrgcn/config.yaml \ --work-dir work_dir/ntu120/csub/ctrgcn \ --phase test \ --save-score True \ --weights work_dir/ntu120/csub/ctrgcn/runs-65-12345.pt \ --device 0 ``` -------------------------------- ### Ensemble Two Modalities (Joint + Bone) Source: https://context7.com/uason-chen/ctr-gcn/llms.txt Ensembles predictions from two modalities (joint and bone). The `--alpha` parameter controls the weighting between modalities. ```bash python ensemble.py \ --dataset ntu120/xsub \ --joint-dir work_dir/ntu120/csub/ctrgcn \ --bone-dir work_dir/ntu120/csub/ctrgcn_bone \ --alpha 1.0 ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.