### Install OC-SORT and Dependencies Source: https://context7.com/noahcao/oc_sort/llms.txt Instructions for cloning the repository, installing Python packages, and setting up Docker for OC-SORT. ```shell git clone https://github.com/noahcao/OC_SORT.git cd OC_SORT pip3 install -r requirements.txt python3 setup.py develop ``` ```shell pip3 install cython pip3 install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI' ``` ```shell pip3 install cython_bbox pandas xmltodict ``` ```shell docker build -t ocsort:latest . docker run --gpus all -it --rm \ -v $PWD/pretrained:/workspace/OC_SORT/pretrained \ -v $PWD/datasets:/workspace/OC_SORT/datasets \ ocsort:latest ``` -------------------------------- ### Install OC-SORT and Dependencies Source: https://github.com/noahcao/oc_sort/blob/master/docs/INSTALL.md Clone the repository, install required Python packages, and set up the project in develop mode. Ensure pycocotools is installed separately. ```shell git clone https://github.com/noahcao/OC_SORT.git cd OC_SORT pip3 install -r requirements.txt python3 setup.py develop ``` ```shell pip3 install cython; pip3 install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI' ``` ```shell pip3 install cython_bbox pandas xmltodict ``` -------------------------------- ### Run ByteTrack Demo Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ncnn/cpp/README.md Execute the compiled ByteTrack demo with a video file. The example shows running at 5 FPS on a specific CPU. ```shell ./bytetrack palace.mp4 ``` -------------------------------- ### OpenCV and NCNN Integration for Examples Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ncnn/cpp/CMakeLists.txt Configures the build environment for NCNN examples, searching for OpenCV and setting up include paths. It conditionally builds examples based on OpenCV availability or NCNN_SIMPLEOCV. ```cmake if(NCNN_PIXEL) find_package(OpenCV QUIET COMPONENTS opencv_world) # for opencv 2.4 on ubuntu 16.04, there is no opencv_world but OpenCV_FOUND will be TRUE if("${OpenCV_LIBS}" STREQUAL "") set(OpenCV_FOUND FALSE) endif() if(NOT OpenCV_FOUND) find_package(OpenCV QUIET COMPONENTS core highgui imgproc imgcodecs videoio) endif() if(NOT OpenCV_FOUND) find_package(OpenCV QUIET COMPONENTS core highgui imgproc) endif() if(OpenCV_FOUND OR NCNN_SIMPLEOCV) if(OpenCV_FOUND) message(STATUS "OpenCV library: ${OpenCV_INSTALL_PATH}") message(STATUS " version: ${OpenCV_VERSION}") message(STATUS " libraries: ${OpenCV_LIBS}") message(STATUS " include path: ${OpenCV_INCLUDE_DIRS}") if(${OpenCV_VERSION_MAJOR} GREATER 3) set(CMAKE_CXX_STANDARD 11) endif() endif() include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../src) include_directories(${CMAKE_CURRENT_BINARY_DIR}/../src) include_directories(include) include_directories(/usr/local/include/eigen3) ncnn_add_example(squeezenet) ncnn_add_example(squeezenet_c_api) ncnn_add_example(fasterrcnn) ncnn_add_example(rfcn) ncnn_add_example(yolov2) ncnn_add_example(yolov3) if(OpenCV_FOUND) ncnn_add_example(yolov4) endif() ncnn_add_example(yolov5) ncnn_add_example(yolox) ncnn_add_example(mobilenetv2ssdlite) ncnn_add_example(mobilenetssd) ncnn_add_example(squeezenetssd) ncnn_add_example(shufflenetv2) ncnn_add_example(peleenetssd_seg) ncnn_add_example(simplepose) ncnn_add_example(retinaface) ncnn_add_example(yolact) ncnn_add_example(nanodet) ncnn_add_example(scrfd) ncnn_add_example(scrfd_crowdhuman) ncnn_add_example(rvm) file(GLOB My_Source_Files src/*.cpp) add_executable(bytetrack ${My_Source_Files}) if(OpenCV_FOUND) target_include_directories(bytetrack PRIVATE ${OpenCV_INCLUDE_DIRS}) target_link_libraries(bytetrack PRIVATE ncnn ${OpenCV_LIBS}) elseif(NCNN_SIMPLEOCV) target_compile_definitions(bytetrack PUBLIC USE_NCNN_SIMPLEOCV) target_link_libraries(bytetrack PRIVATE ncnn) endif() # add test to a virtual project group set_property(TARGET bytetrack PROPERTY FOLDER "examples") else() message(WARNING "OpenCV not found and NCNN_SIMPLEOCV disabled, examples won't be built") endif() else() message(WARNING "NCNN_PIXEL not enabled, examples won't be built") endif() ``` -------------------------------- ### Install Eigen Library Source: https://github.com/noahcao/oc_sort/blob/master/deploy/TensorRT/cpp/README.md Steps to download, extract, build, and install the Eigen library. Ensure you have a version compatible with the project's requirements. ```shell unzip eigen-3.3.9.zip cd eigen-3.3.9 mkdir build cd build cmake .. sudo make install ``` -------------------------------- ### Run Demo Tracking with OC-SORT Source: https://github.com/noahcao/oc_sort/blob/master/README.md Execute the tracker on a demo video using a specified configuration and pre-trained model. Ensure you have the necessary dependencies installed and models downloaded. ```shell python3 tools/demo_track.py --demo_type video -f exps/example/mot/yolox_dancetrack_test.py -c pretrained/ocsort_dance_model.pth.tar --path videos/dance_demo.mp4 --fp16 --fuse --save_result --out_path demo_out.mp4 ``` -------------------------------- ### C++ OC-SORT Tracker Usage Source: https://context7.com/noahcao/oc_sort/llms.txt Example demonstrating how to use the OCSort C++ shared library. It includes initializing the tracker with parameters, preparing detection data as an Eigen matrix, and updating the tracker per frame to get tracked object IDs. ```cpp #include "OCSort.hpp" #include // Input: Eigen matrix, columns = [x1, y1, x2, y2, confidence, class_id] Eigen::Matrix detections(2, 6); detections << 100, 150, 200, 250, 0.91, 0, 300, 100, 420, 220, 0.85, 0; ocsort::OCSort tracker( /*det_thresh=*/0.6f, /*max_age=*/30, /*min_hits=*/3, /*iou_threshold=*/0.3f, /*delta_t=*/3, /*asso_func=*/"iou", /*inertia=*/0.2f, /*use_byte=*/false ); // Per-frame update; returns vector of row vectors [x1,y1,x2,y2,ID,class,conf] std::vector results = tracker.update(detections); for (auto& row : results) { int id = static_cast(row[4]); float conf = row[6]; printf("ID=%d box=[%.1f,%.1f,%.1f,%.1f] conf=%.2f\n", id, row[0], row[1], row[2], row[3], conf); } // → ID=1 box=[100.0,150.0,200.0,250.0] conf=0.91 // → ID=2 box=[300.0,100.0,420.0,220.0] conf=0.85 ``` -------------------------------- ### NCNN Example Macro Definition Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ncnn/cpp/CMakeLists.txt Defines a macro to add an executable for an NCNN example. It links against ncnn and optionally OpenCV, setting include directories and compile definitions as needed. ```cmake macro(ncnn_add_example name) add_executable(${name} ${name}.cpp) if(OpenCV_FOUND) target_include_directories(${name} PRIVATE ${OpenCV_INCLUDE_DIRS}) target_link_libraries(${name} PRIVATE ncnn ${OpenCV_LIBS}) elseif(NCNN_SIMPLEOCV) target_compile_definitions(${name} PUBLIC USE_NCNN_SIMPLEOCV) target_link_libraries(${name} PRIVATE ncnn) endif() # add test to a virtual project group set_property(TARGET ${name} PROPERTY FOLDER "examples") endmacro() ``` -------------------------------- ### Build ByteTrack with ncnn Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ncnn/cpp/README.md Copy the necessary source files and the optimized ncnn model files into the ncnn/examples directory, then build the project using CMake and Make. ```shell cd ncnn/build/examples cmake .. make ``` -------------------------------- ### Build ByteTrack C++ Demo Source: https://github.com/noahcao/oc_sort/blob/master/deploy/TensorRT/cpp/README.md Compiles the ByteTrack C++ demo application using CMake. Ensure TensorRT and CUDA paths are correctly set in the CMakeLists.txt file. ```shell cd /deploy/TensorRT/cpp mkdir build cd build cmake .. make ``` -------------------------------- ### Run ONNXRuntime Demo Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ONNXRuntime/README.md Execute this command from the ONNXRuntime deployment directory to run the inference demo. This is useful for verifying the ONNX model conversion. ```shell cd /deploy/ONNXRuntime python3 onnx_inference.py ``` -------------------------------- ### Run ByteTrack C++ Demo Source: https://github.com/noahcao/oc_sort/blob/master/deploy/TensorRT/cpp/README.md Executes the compiled ByteTrack C++ demo. Specify the path to the serialized TensorRT engine file and the input video file. ```shell ./bytetrack ../../../../YOLOX_outputs/yolox_s_mix_det/model_trt.engine -i ../../../../videos/palace.mp4 ``` -------------------------------- ### Run TensorRT Demo for Object Tracking Source: https://github.com/noahcao/oc_sort/blob/master/deploy/TensorRT/python/README.md Executes the ByteTrack TensorRT demo using a converted model for real-time video tracking. This command enables TensorRT acceleration and saves the results. ```shell cd python3 tools/demo_track.py video -f exps/example/mot/yolox_s_mix_det.py --trt --save_result ``` -------------------------------- ### Build and Run OC-SORT Docker Image Source: https://github.com/noahcao/oc_sort/blob/master/docs/INSTALL.md Build the OC-SORT Docker image and then run a container, mounting necessary directories and configuring display and device access for real-time video processing. ```shell docker build -t ocsort:latest . ``` ```shell mkdir -p pretrained && \ mkdir -p YOLOX_outputs && \ xhost +local: && \ docker run --gpus all -it --rm \ -v $PWD/pretrained:/workspace/OC_SORT/pretrained \ -v $PWD/datasets:/workspace/OC_SORT/datasets \ -v $PWD/YOLOX_outputs:/workspace/OC_SORT/YOLOX_outputs \ -v /tmp/.X11-unix/:/tmp/.X11-unix:rw \ --device /dev/video0:/dev/video0:mwr \ --net=host \ -e XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR \ -e DISPLAY=$DISPLAY \ --privileged \ ocsort:latest ``` -------------------------------- ### CLI: Demo tracking on webcam with YOLOX Source: https://context7.com/noahcao/oc_sort/llms.txt Command-line interface for running OCSORT end-to-end on a webcam feed using the YOLOX detector. This command shows how to specify the camera ID and adjust tracking thresholds, including the option to use ByteTrack. ```shell python3 tools/demo_track.py \ --demo_type webcam \ -f exps/example/mot/yolox_dancetrack_test.py \ -c pretrained/ocsort_dance_model.pth.tar \ --camid 0 \ --track_thresh 0.5 \ --use_byte ``` -------------------------------- ### CLI: Demo tracking on video with YOLOX Source: https://context7.com/noahcao/oc_sort/llms.txt Command-line interface for running OCSORT end-to-end on video files, webcams, or image folders using the YOLOX detector. This command demonstrates tracking on a sample video, saving the annotated output, and specifies various tracking parameters. ```shell python3 tools/demo_track.py \ --demo_type video \ -f exps/example/mot/yolox_dancetrack_test.py \ -c pretrained/ocsort_dance_model.pth.tar \ --path videos/dance_demo.mp4 \ --fp16 --fuse \ --save_result \ --out_path demo_out.mp4 \ --track_thresh 0.6 \ --iou_thresh 0.3 \ --inertia 0.2 \ --deltat 3 ``` -------------------------------- ### Run TensorRT Demo Source: https://github.com/noahcao/oc_sort/blob/master/docs/DEPLOY.md Run the tracking demo on a video using the TensorRT-enabled model. This command includes options to save the results. ```python python3 tools/demo_track.py video -f exps/example/mot/yolox_s_mix_det.py --trt --save_result ``` -------------------------------- ### OCSort.__init__ Source: https://context7.com/noahcao/oc_sort/llms.txt Instantiates a new OC-SORT tracker with customizable hyper-parameters for detection thresholds, track management, association metrics, and motion modeling. ```APIDOC ## OCSort.__init__ — Instantiate the tracker Constructs a new OC-SORT tracker with all tuneable hyper-parameters. `det_thresh` is the primary detection score cut-off; detections below it but above 0.1 are used in a second-round BYTE association pass. `delta_t` controls how many frames back to look when estimating motion direction. `asso_func` selects the similarity metric (`"iou"`, `"giou"`, `"ciou"`, `"diou"`, or `"ct_dist"`). `inertia` (`vdc_weight`) weights the velocity-direction cost against the IoU cost. ```python from trackers.ocsort_tracker.ocsort import OCSort # Standard MOT17 / pedestrian tracking settings tracker = OCSort( det_thresh=0.6, # high-score detection threshold max_age=30, # frames before a lost track is deleted min_hits=3, # frames before a new track is confirmed iou_threshold=0.3, # minimum IoU to consider a match delta_t=3, # look-back window for velocity estimation asso_func="iou", # association cost: iou | giou | ciou | diou | ct_dist inertia=0.2, # weight of velocity-direction consistency term use_byte=False, # enable BYTE two-round low-score association ) # DanceTrack / non-rigid motion settings (lower thresholds, GIoU) tracker_dance = OCSort( det_thresh=0.5, max_age=30, min_hits=1, iou_threshold=0.2, delta_t=3, asso_func="giou", inertia=0.2, use_byte=True, ) ``` ``` -------------------------------- ### CMake Configuration for libocsort Source: https://context7.com/noahcao/oc_sort/llms.txt This CMakeLists.txt excerpt sets up the C++ shared library for OC-SORT, finding necessary packages like Eigen3 and OpenCV, and defining the library and test executable. ```cmake cmake_minimum_required(VERSION 3.10) project(libocsort) set(CMAKE_CXX_STANDARD 17) find_package(Eigen3 REQUIRED) find_package(OpenCV REQUIRED) file(GLOB SRC_LIST src/*.cpp) add_library(${PROJECT_NAME} SHARED ${SRC_LIST}) target_include_directories(${PROJECT_NAME} PUBLIC include) target_link_libraries(${PROJECT_NAME} Eigen3::Eigen) add_executable(test mutilthread.cpp detector/inference.cpp) target_link_libraries(test PUBLIC Eigen3::Eigen ${PROJECT_NAME} ${OpenCV_LIBS}) ``` -------------------------------- ### Build Mixed Training Sets Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md Create mixed training sets for MOT17 and MOT20 datasets by running the specified script. This is an optional step for specific training scenarios. ```python python3 tools/mix_data_{ablation/mot17/mot20}.py ``` -------------------------------- ### CLI: Demo Tracking Source: https://context7.com/noahcao/oc_sort/llms.txt Command-line interface for running OCSORT tracking on video files, image folders, or webcam streams using the YOLOX detector. Results are saved as annotated videos and MOT-format files. ```APIDOC ## CLI: Demo tracking on a video Run OC-SORT end-to-end on a video file or webcam using the YOLOX detector. The `--demo_type` flag switches between `image` (folder of frames), `video`, and `webcam`. Results are saved as an annotated MP4 and a MOT-format `.txt` file. ```shell # Track on a provided demo video (DanceTrack example) python3 tools/demo_track.py \ --demo_type video \ -f exps/example/mot/yolox_dancetrack_test.py \ -c pretrained/ocsort_dance_model.pth.tar \ --path videos/dance_demo.mp4 \ --fp16 --fuse \ --save_result \ --out_path demo_out.mp4 \ --track_thresh 0.6 \ --iou_thresh 0.3 \ --inertia 0.2 \ --deltat 3 # Track on webcam (camera id 0) python3 tools/demo_track.py \ --demo_type webcam \ -f exps/example/mot/yolox_dancetrack_test.py \ -c pretrained/ocsort_dance_model.pth.tar \ --camid 0 \ --track_thresh 0.5 \ --use_byte # Track on a folder of images python3 tools/demo_track.py \ --demo_type image \ -f exps/default/yolox_x.py \ -c pretrained/yolox_x.pth.tar \ --path /data/MOT17/train/MOT17-02-FRCNN/img1 \ --save_result ``` ``` -------------------------------- ### CLI: Evaluation on MOT Benchmarks Source: https://context7.com/noahcao/oc_sort/llms.txt Command-line interface for evaluating OCSORT performance on MOT17/MOT20 benchmarks. This runs the full detection and tracking pipeline and computes standard metrics. ```APIDOC ## CLI: Evaluation on MOT17 / MOT20 benchmarks Full evaluation loop that runs the YOLOX detector + OC-SORT tracker over a benchmark split and computes HOTA, MOTA, IDF1, and auxiliary metrics via the built-in `motmetrics` integration. ```shell # ── MOT17 half-val (ablation model) ───────────────────────────────────────── python3 tools/run_ocsort.py \ -f exps/example/mot/yolox_x_ablation.py \ -c pretrained/bytetrack_ablation.pth.tar \ -b 1 -d 1 --fp16 --fuse \ --expn mot17_halfval # Expected: HOTA ≈ 66.5 ``` ``` -------------------------------- ### Run ONNX Inference Source: https://github.com/noahcao/oc_sort/blob/master/docs/DEPLOY.md Execute inference on a video using the converted ONNX model with ONNX Runtime. Ensure you are in the correct directory. ```shell cd $OCSORT_HOME/deploy/ONNXRuntime python onnx_inference.py ``` -------------------------------- ### Convert Input Video for TensorRT C++ Demo Source: https://github.com/noahcao/oc_sort/blob/master/deploy/TensorRT/cpp/README.md Converts an input video to a format suitable for the TensorRT C++ demo if frame loss is observed. This script is part of the ByteTrack project's tools. ```shell cd python3 tools/convert_video.py ``` -------------------------------- ### Simulate Video Loop with OCSort Source: https://context7.com/noahcao/oc_sort/llms.txt This snippet demonstrates how to integrate OCSort into a video processing loop. It reads frames from a video, processes detections (simulated here), updates the tracker, and visualizes the results with bounding boxes and IDs. Ensure your detector output matches the expected format. ```python import cv2 import torch import numpy as np # Assuming 'tracker' and 'model_input_size' are initialized elsewhere # tracker = OCSort(...) # model_input_size = (width, height) cap = cv2.VideoCapture("videos/dance_demo.mp4") while True: ret, frame = cap.read() if not ret: break img_h, img_w = frame.shape[:2] # --- detector output: shape (N, 6) with columns [x1,y1,x2,y2,obj_conf,cls_conf] # (replace with your actual detector call) raw_dets = torch.zeros((0, 6)) # empty frame example if raw_dets is not None and len(raw_dets): # OCSort.update handles both 5-col [x1,y1,x2,y2,score] and # 6-col [x1,y1,x2,y2,obj_conf,cls_conf] tensors automatically online_targets = tracker.update( output_results=raw_dets, img_info=[img_h, img_w], img_size=model_input_size, ) else: online_targets = tracker.update( output_results=np.empty((0, 5)), img_info=[img_h, img_w], img_size=model_input_size, ) # online_targets: np.ndarray of shape (M, 5) → [x1, y1, x2, y2, id] for t in online_targets: x1, y1, x2, y2, tid = t w, h = x2 - x1, y2 - y1 if w * h > 100: # filter tiny boxes cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0,255,0), 2) cv2.putText(frame, f"ID:{int(tid)}", (int(x1), int(y1)-5), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0,255,0), 1) cap.release() # Expected output: annotated frames with stable bounding boxes and persistent IDs ``` -------------------------------- ### CLI: Evaluation on MOT17 benchmark Source: https://context7.com/noahcao/oc_sort/llms.txt Command-line interface for evaluating OCSORT with the YOLOX detector on the MOT17 benchmark. This command runs the full evaluation loop, computing metrics like HOTA, MOTA, and IDF1, and specifies the experiment name for tracking results. ```shell # ── MOT17 half-val (ablation model) ───────────────────────────────────────── python3 tools/run_ocsort.py \ -f exps/example/mot/yolox_x_ablation.py \ -c pretrained/bytetrack_ablation.pth.tar \ -b 1 -d 1 --fp16 --fuse \ --expn mot17_halfval # Expected: HOTA ≈ 66.5 ``` -------------------------------- ### CLI: Demo tracking on image folder with YOLOX Source: https://context7.com/noahcao/oc_sort/llms.txt Command-line interface for running OCSORT end-to-end on a folder of images using the YOLOX detector. This command specifies the input image directory and the path to the detector's configuration and weights. ```shell python3 tools/demo_track.py \ --demo_type image \ -f exps/default/yolox_x.py \ -c pretrained/yolox_x.pth.tar \ --path /data/MOT17/train/MOT17-02-FRCNN/img1 \ --save_result ``` -------------------------------- ### Generate ONNX File Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ncnn/cpp/README.md Use this command to convert a PyTorch model to ONNX format. Ensure you are in the ByteTrack home directory and specify the correct model and configuration files. ```shell cd python3 tools/export_onnx.py -f exps/example/mot/yolox_s_mix_det.py -c pretrained/bytetrack_s_mot17.pth.tar ``` -------------------------------- ### Generate ncnn Param and Bin Files Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ncnn/cpp/README.md Convert the generated ONNX file to ncnn's param and bin formats using the onnx2ncnn tool. Place the ONNX file in the ncnn/build/tools/onnx directory before running. ```shell cd ncnn/build/tools/onnx ./onnx2ncnn bytetrack_s.onnx bytetrack_s.param bytetrack_s.bin ``` -------------------------------- ### Build Test Executable with YOLO Inference Source: https://github.com/noahcao/oc_sort/blob/master/deploy/OCSort/cpp/CMakeLists.txt Creates an executable for testing, including multi-threading and YOLO inference components. Links against Eigen3, the project's shared library, and OpenCV. ```cmake # note:test with yolo inference add_executable(test mutilthread.cpp ../detector/inference.h ../detector/inference.cpp) target_link_libraries(test PUBLIC Eigen3::Eigen ${PROJECT_NAME} ${OpenCV_LIBS}) ``` -------------------------------- ### Instantiate OC-SORT Tracker Source: https://context7.com/noahcao/oc_sort/llms.txt Constructs a new OC-SORT tracker with tuneable hyper-parameters. `det_thresh` is the detection score cut-off. `delta_t` controls the velocity estimation look-back window. `inertia` weights the velocity-direction cost against the IoU cost. ```python from trackers.ocsort_tracker.ocsort import OCSort # Standard MOT17 / pedestrian tracking settings tracker = OCSort( det_thresh=0.6, # high-score detection threshold max_age=30, # frames before a lost track is deleted min_hits=3, # frames before a new track is confirmed iou_threshold=0.3, # minimum IoU to consider a match delta_t=3, # look-back window for velocity estimation asso_func="iou", # association cost: iou | giou | ciou | diou | ct_dist inertia=0.2, # weight of velocity-direction consistency term use_byte=False, # enable BYTE two-round low-score association ) ``` ```python # DanceTrack / non-rigid motion settings (lower thresholds, GIoU) tracker_dance = OCSort( det_thresh=0.5, max_age=30, min_hits=1, iou_threshold=0.2, delta_t=3, asso_func="giou", inertia=0.2, use_byte=True, ) ``` -------------------------------- ### Evaluate on MOT20 Test Set Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md Use this command to evaluate the model on the MOT20 test set. The zipped output files should be submitted to the MOTChallenge system. This command requires a model checkpoint, an experiment configuration file, and specific flags for MOT20. ```shell # MOT20 python3 tools/run_ocsort.py -f exps/example/mot/yolox_x_mix_mot20_ch.py -c pretrained/bytetrack_x_mot20.tar -b 1 -d 1 --fp16 --fuse --track_thresh 0.4 --mot20 --expn $exp_name ``` -------------------------------- ### Create Mixed Training Set for MOT17 Source: https://context7.com/noahcao/oc_sort/llms.txt Combine multiple datasets into a single mixed training set for MOT17, facilitating comprehensive model training. ```shell python3 tools/mix_data_test_mot17.py ``` -------------------------------- ### ONNX Runtime Inference for Detection and Tracking Source: https://context7.com/noahcao/oc_sort/llms.txt Perform inference using an ONNX-exported YOLOX model with ONNX Runtime. This script runs the full detect-and-track pipeline and saves the annotated video. ```python import onnxruntime from deploy.ONNXRuntime.onnx_inference import Predictor, imageflow_demo import argparse args = argparse.Namespace( model = "ocsort.onnx", video_path = "videos/dance_demo.mp4", output_dir = "demo_output", score_thr = 0.1, nms_thr = 0.7, input_shape = "800,1440", with_p6 = False, track_thresh = 0.6, iou_thresh = 0.3, track_buffer = 30, match_thresh = 0.8, min_box_area = 10, mot20 = False, ) predictor = Predictor(args) # loads ONNX session imageflow_demo(predictor, args) # runs tracker and writes annotated video to demo_output/ ``` -------------------------------- ### Convert PyTorch Model to ONNX Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ONNXRuntime/README.md Use this command to convert your PyTorch model to ONNX format. Ensure you are in the ByteTrack home directory and specify the output name, configuration file, and model checkpoint. ```shell cd python3 tools/export_onnx.py --output-name bytetrack_s.onnx -f exps/example/mot/yolox_s_mix_det.py -c pretrained/bytetrack_s_mot17.pth.tar ``` -------------------------------- ### Compile OC-SORT with CMake Source: https://github.com/noahcao/oc_sort/blob/master/deploy/OCSort/cpp/Readme.md CMakeLists.txt for compiling the OC-SORT library and a test executable. Ensure Eigen3 and OpenCV are correctly configured. ```cmake cmake_minimum_required(VERSION 3.10) project(libocsort) set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /O2") set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_RELEASE} /O2") set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS_RELEASE} /O2") set(CMAKE_CXX_STANDARD 17) # Linker external library SET(Eigen3_DIR "C:/eigen-3.4.0/build") find_package(Eigen3 REQUIRED) set(OpenCV_DIR "C:/opencv") find_package(OpenCV REQUIRED) # add_subdirectory(src) set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON) file(GLOB SRC_LIST src/*.cpp) add_library(${PROJECT_NAME} SHARED ${SRC_LIST}) target_include_directories(${PROJECT_NAME} PUBLIC include) target_link_libraries(${PROJECT_NAME} Eigen3::Eigen) # note:test with yolo inference add_executable(test mutilthread.cpp ) target_link_libraries(test PUBLIC Eigen3::Eigen ${PROJECT_NAME} ${OpenCV_LIBS}) # note:test with MOT # add_executable(test testMOT.cpp) # target_link_libraries(test PUBLIC Eigen3::Eigen ${PROJECT_NAME} ${OpenCV_LIBS}) ``` -------------------------------- ### Build Unit Test Executable Source: https://github.com/noahcao/oc_sort/blob/master/deploy/OCSort/cpp/CMakeLists.txt Creates an executable for unit tests that focus on bug fixes and do not have an OpenCV dependency. Links against Eigen3 and the project's shared library. ```cmake # unit tests for bug fixes (no OpenCV dependency) add_executable(test_fixes test_fixes.cpp) target_link_libraries(test_fixes PUBLIC Eigen3::Eigen ${PROJECT_NAME}) ``` -------------------------------- ### Evaluate on KITTI Test Set Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md Use this command to evaluate OC-SORT on the KITTI test set using public detection results from PermaTrack. The output .txt files should be submitted to the KITTI evaluation server. ```shell python tools/run_ocsort_public.py --hp --out_path kitti_test --dataset kitti --raw_results_path exps/permatrack_kitti_test ``` -------------------------------- ### Process Frame with Object Detection and OC-SORT Source: https://github.com/noahcao/oc_sort/blob/master/deploy/OCSort/cpp/Readme.md Processes a video frame using object detection and the OC-SORT tracker. It draws bounding boxes and labels for detected objects and tracked instances. ```cpp void processFrame(const cv::Mat& frame, std::vector& output, ocsort::OCSort& tracker) { std::vector> data; // Iterate over the output vector starting from index 1 (skipping the first element) for (int i = 1; i < output.size(); ++i) { Detection detection = output[i]; cv::Rect box = detection.box; std::vector row; for (;;) { // Push the coordinates, confidence, and class ID of the detection to the row vector row.push_back(output[i].box.x); row.push_back(output[i].box.y); row.push_back(output[i].box.x + output[i].box.width); row.push_back(output[i].box.y + output[i].box.height); row.push_back(output[i].confidence); row.push_back(output[i].class_id); } data.push_back(row); // Add the row vector to the data vector // Update the tracker with the data converted to an Eigen matrix std::vector res = tracker.update(Vector2Matrix(data)); cv::rectangle(frame, box, cv::Scalar(0, 255, 0), 2); // Draw a rectangle around the detection // Add text indicating the class name and confidence to the frame std::string classString = detection.className + '(' + std::to_string(detection.confidence).substr(0, 4) + ')'; cv::putText(frame, classString, cv::Point(box.x + 5, box.y + box.height - 10), cv::FONT_HERSHEY_DUPLEX, 0.5, cv::Scalar(0, 255, 0), 1, 0); // Process the results from the tracker for (auto j : res) { int ID = int(j[4]); int Class = int(j[5]); float conf = j[6]; // Add ID text and draw a rectangle for each tracked object cv::putText(frame, cv::format("ID:%d", ID), cv::Point(j[0], j[1] - 5), 0, 0.5, cv::Scalar(0, 0, 255), 1, cv::LINE_AA); cv::rectangle(frame, cv::Rect(j[0], j[1], j[2] - j[0] + 1, j[3] - j[1] + 1), cv::Scalar(0, 0, 255), 1); } data.clear(); // Clear the data vector for the next iteration } } ``` -------------------------------- ### Train Ablation Model (MOT17 half train and CrowdHuman) Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md Train an ablation model using MOT17 (half train) and CrowdHuman datasets. This command specifies the experiment configuration, number of devices, batch size, mixed precision, and pretrained weights. ```shell python3 tools/train.py -f exps/example/mot/yolox_x_ablation.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth ``` -------------------------------- ### KalmanBoxTracker State Estimation Source: https://context7.com/noahcao/oc_sort/llms.txt This snippet illustrates the `KalmanBoxTracker` class, which uses a Kalman filter for per-object state estimation. It shows initialization with a bounding box, prediction of future states, updating with new observations, and handling missed frames. The `observations` attribute stores all received detections. ```python import numpy as np from trackers.ocsort_tracker.ocsort import KalmanBoxTracker # Initialize with a detection [x1, y1, x2, y2, score] bbox = np.array([100.0, 150.0, 200.0, 280.0, 0.92]) trk = KalmanBoxTracker(bbox, delta_t=3) print("Initial state:", trk.get_state()) # → [[100. 150. 200. 280.]] # Simulate 5 frames for frame in range(5): predicted = trk.predict() print(f"Frame {frame} prediction: {predicted}") if frame < 3: # Object is observed — update with a slightly shifted box new_bbox = np.array([100.0 + frame*5, 150.0 + frame*3, 200.0 + frame*5, 280.0 + frame*3, 0.88]) trk.update(new_bbox) print(f" velocity direction: {trk.velocity}") else: # Object is occluded — ORU kicks in on re-detection trk.update(None) print(f" time_since_update: {trk.time_since_update}") print("Observations recorded:", list(trk.observations.keys())) # → Observations recorded: [0, 1, 2] (frames where detections arrived) ``` -------------------------------- ### Run OC-SORT on MOT20 Test Set Source: https://context7.com/noahcao/oc_sort/llms.txt Execute OC-SORT for tracking on the MOT20 test set, including specific parameters for MOT20. This command is for evaluation purposes. ```shell python3 tools/run_ocsort.py \ -f exps/example/mot/yolox_x_mix_mot20_ch.py \ -c pretrained/bytetrack_x_mot20.tar \ -b 1 -d 1 --fp16 --fuse \ --track_thresh 0.4 --mot20 \ --expn mot20_test --test ``` -------------------------------- ### Optimize Threading in ByteTrack Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ncnn/cpp/README.md Modify the 'num_threads' variable in the bytetrack.cpp file to tune the running speed based on your CPU cores. ```cpp yolox.opt.num_threads = 20; ``` -------------------------------- ### Configure Input Dimensions in ByteTrack C++ Source: https://github.com/noahcao/oc_sort/blob/master/deploy/TensorRT/cpp/README.md Defines the input width and height for the TensorRT engine. Adjust these values based on the specific ByteTrack model being used (e.g., bytetrack_s, bytetrack_m). ```cpp static const int INPUT_W = 1088; static const int INPUT_H = 608; ``` -------------------------------- ### Evaluate on MOT17 Test Set Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md Use this command to evaluate the model on the MOT17 test set. The zipped output files should be submitted to the MOTChallenge system. This command requires a model checkpoint and an experiment configuration file. ```shell # MOT17 python3 tools/run_ocsort.py -f exps/example/mot/yolox_x_mix_det.py -c pretrained/bytetrack_x_mot17.pth.tar -b 1 -d 1 --fp16 --fuse --expn $exp_name ``` -------------------------------- ### Convert PyTorch Model to ONNX Source: https://github.com/noahcao/oc_sort/blob/master/docs/DEPLOY.md Use this script to convert your PyTorch model to ONNX format. You may want to use a smaller model for faster inference. ```python python deploy/scripts/export_onnx.py --output-name ocsort.onnx -f exps/example/mot/yolox_x_mix_det.py -c pretrained/bytetrack_x_mot17.pth.tar ``` -------------------------------- ### Convert Model to TensorRT Source: https://github.com/noahcao/oc_sort/blob/master/docs/DEPLOY.md Convert the ByteTrack model checkpoint to a TensorRT-compatible format. Ensure you have downloaded the necessary checkpoint. ```python # you have to download checkpoint bytetrack_s_mot17.pth.tar from model zoo of ByteTrack python3 deploy/scripts/trt.py -f exps/example/mot/yolox_s_mix_det.py -c pretrained/bytetrack_s_mot17.pth.tar ``` -------------------------------- ### Optimize ncnn Model Source: https://github.com/noahcao/oc_sort/blob/master/deploy/ncnn/cpp/README.md Use ncnnoptimize to further optimize the generated param and bin files for better performance. This step is performed in the ncnn/build/tools/onnx directory. ```shell ../ncnnoptimize bytetrack_s.param bytetrack_s.bin bytetrack_s_op.param bytetrack_s_op.bin 65536 ``` -------------------------------- ### Run OC-SORT on MOT17 Test Set Source: https://context7.com/noahcao/oc_sort/llms.txt Execute OC-SORT for tracking on the MOT17 test set using a specified configuration and checkpoint. This command is for evaluation purposes. ```shell python3 tools/run_ocsort.py \ -f exps/example/mot/yolox_x_mix_det.py \ -c pretrained/bytetrack_x_mot17.pth.tar \ -b 1 -d 1 --fp16 --fuse \ --expn mot17_test --test ``` -------------------------------- ### Train MOT20 Test Model Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md Train a model for MOT20 testing, using MOT20 train and CrowdHuman datasets. Note that box clipping modifications might be necessary in the source code before running this command. ```shell python3 tools/train.py -f exps/example/mot/yolox_x_mix_mot20_ch.py -d 8 -b 48 --fp16 -o -c pretrained/yolox_x.pth ``` -------------------------------- ### Optional: Gaussian Process Regression Interpolation Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md This command performs optional offline interpolation using Gaussian Process Regression, building upon existing linear interpolation results. Specify paths for raw results, linear interpolation, and the save path. ```shell python3 tools/gp_interpolation.py $raw_results_path $linear_interp_path $save_path ``` -------------------------------- ### Build Shared Library Source: https://github.com/noahcao/oc_sort/blob/master/deploy/OCSort/cpp/CMakeLists.txt Compiles the project into a shared library. It uses file(GLOB ...) to find all .cpp files in the src directory. ```cmake set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON) file(GLOB SRC_LIST src/*.cpp) add_library(${PROJECT_NAME} SHARED ${SRC_LIST}) target_include_directories(${PROJECT_NAME} PUBLIC include) target_link_libraries(${PROJECT_NAME} Eigen3::Eigen) ``` -------------------------------- ### Evaluate on DanceTrack Test Set Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md Use this command to evaluate the model on the DanceTrack test set. The outputs should be submitted to the DanceTrack evaluation site. This command requires a model checkpoint and an experiment configuration file. ```shell python tools/run_ocsort_dance.py -f exps/example/mot/yolox_dancetrack_test.py -c pretrained/bytetrack_dance_model.pth.tar -b 1 -d 1 --fp16 --fuse --test --expn $exp_name ``` -------------------------------- ### Convert Dataset to COCO Format Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md Use this script to convert datasets like DanceTrack to COCO format for compatibility with OC-SORT. Replace 'dance' with the appropriate dataset name. ```python python3 tools/convert_dance_to_coco.py ``` -------------------------------- ### Run OC-SORT on DanceTrack Validation Set Source: https://context7.com/noahcao/oc_sort/llms.txt Execute OC-SORT for tracking on the DanceTrack validation set. This command is used for evaluating performance on this specific dataset. ```shell python tools/run_ocsort_dance.py \ -f exps/example/mot/yolox_dancetrack_val.py \ -c pretrained/bytetrack_dance_model.pth.tar \ -b 1 -d 1 --fp16 --fuse \ --expn dance_val ``` -------------------------------- ### Update OC-SORT Tracker with Frame Data Source: https://context7.com/noahcao/oc_sort/llms.txt The main per-frame entry point for the tracker. Accepts detector output, image metadata, and model input resolution. Returns tracked objects as `[x1, y1, x2, y2, track_id]`. Pass an empty array for frames with no detections to maintain internal state consistency. ```python import numpy as np import cv2 import torch from trackers.ocsort_tracker.ocsort import OCSort from yolox.data.data_augment import preproc from yolox.utils import postprocess # ── Setup ─────────────────────────────────────────────────────────────────── tracker = OCSort(det_thresh=0.6, max_age=30, min_hits=3, iou_threshold=0.3) model_input_size = (800, 1440) # (H, W) the detector expects ``` -------------------------------- ### Find and Link External Libraries Source: https://github.com/noahcao/oc_sort/blob/master/deploy/OCSort/cpp/CMakeLists.txt Locates and links Eigen3 and OpenCV libraries. Ensure Eigen3_DIR and OpenCV_DIR are set correctly. ```cmake SET(Eigen3_DIR "C:/eigen-3.4.0/build") find_package(Eigen3 REQUIRED) set(OpenCV_DIR "C:/opencv") find_package(OpenCV REQUIRED) ``` -------------------------------- ### Run OC-SORT with Public Detections on KITTI Source: https://context7.com/noahcao/oc_sort/llms.txt Utilize OC-SORT with pre-existing public detections for the KITTI test set. This allows for association-only comparisons without re-running a detector. ```shell python tools/run_ocsort_public.py \ --hp \ --out_path kitti_test \ --dataset kitti \ --raw_results_path exps/permatrack_kitti_test ``` -------------------------------- ### Vectorized Pairwise Cost Functions for OCSORT Source: https://context7.com/noahcao/oc_sort/llms.txt Demonstrates the usage of vectorized association cost functions like IoU, GIoU, CIoU, DIoU, and CT distance. These functions accept detection and track bounding boxes and return a similarity matrix. They are useful for custom tracking implementations where pairwise costs are needed. ```python import numpy as np from trackers.ocsort_tracker.association import ( iou_batch, giou_batch, diou_batch, ciou_batch, ct_dist ) # 3 detections vs 2 tracks, all in [x1,y1,x2,y2] format dets = np.array([[50, 50, 150, 150], [200, 200, 350, 350], [400, 10, 500, 100]], dtype=float) tracks = np.array([[60, 60, 160, 160], [210, 190, 360, 360]], dtype=float) iou = iou_batch(dets, tracks) giou = giou_batch(dets, tracks) diou = diou_batch(dets, tracks) ciou = ciou_batch(dets, tracks) ctd = ct_dist(dets, tracks) print("IoU matrix:\n", iou) # [[0.6944 0. ] # [0. 0.7042] # [0. 0. ]] print("GIoU matrix:\n", giou) # penalises enclosing-box area gaps print("CIoU matrix:\n", ciou) # adds aspect-ratio consistency penalty # Use in a custom tracker from trackers.ocsort_tracker.association import linear_assignment cost = -ciou # minimise negative similarity → maximise CIoU matches = linear_assignment(cost) print("Matched pairs (det_idx, trk_idx):", matches) # → [[0 0] [1 1]] ``` -------------------------------- ### Convert PyTorch Model to TensorRT Source: https://github.com/noahcao/oc_sort/blob/master/deploy/TensorRT/python/README.md Converts a PyTorch ByteTrack model to a TensorRT model. Ensure you are in the ByteTrack home directory and have the necessary configuration and model files. ```shell cd python3 tools/trt.py -f exps/example/mot/yolox_s_mix_det.py -c pretrained/bytetrack_s_mot17.pth.tar ``` -------------------------------- ### Export YOLOX Model to ONNX Format Source: https://context7.com/noahcao/oc_sort/llms.txt Export a trained YOLOX model to the ONNX format for deployment. This allows for inference without PyTorch dependencies. ```shell python3 deploy/scripts/export_onnx.py \ -f exps/example/mot/yolox_dancetrack_test.py \ -c pretrained/ocsort_dance_model.pth.tar \ --output-name ocsort.onnx \ --input h 800 --input-w 1440 ``` -------------------------------- ### Optional: Linear Interpolation Post-processing Source: https://github.com/noahcao/oc_sort/blob/master/docs/GET_STARTED.md This command performs optional offline linear interpolation on existing tracking results. Specify the path to the results and the desired save path. ```shell python3 tools/interpolation.py $result_path $save_path ```