### Install degirum-tools Source: https://github.com/degirum/pysdkexamples/blob/main/examples/specialized/multi_object_tracking_video_file.ipynb Ensures the degirum-tools package and other necessary dependencies are installed. Run this before proceeding with other examples. ```python # make sure degirum-tools package and other dependencies are installed !pip show degirum-tools || pip install degirum-tools ``` -------------------------------- ### Install Required Packages Source: https://github.com/degirum/pysdkexamples/blob/main/examples/multimodel/sound_classification_and_object_detection_asynchronous.ipynb Installs the `degirum-tools` package and optionally `pyaudio` with its dependencies. Uncomment the `pyaudio` installation lines if needed. ```python # make sure degirum-tools package is installed !pip show degirum-tools || pip install degirum-tools # to install pyaudio package, uncomment the following lines #!apt install libasound2-dev portaudio19-dev libportaudio2 libportaudiocpp0 #!pip show pyaudio || pip install pyaudio ``` -------------------------------- ### Install DeGirum Package Source: https://github.com/degirum/pysdkexamples/blob/main/examples/basic/basic_pysdk_demo_image.ipynb Ensures the DeGirum package is installed. Run this before proceeding with other examples. ```python # make sure degirum package is installed !pip show degirum || pip install degirum ``` -------------------------------- ### Start Jupyter Lab Server Source: https://github.com/degirum/pysdkexamples/blob/main/README.md Starts the Jupyter Lab server from the PySDKExample repository root directory. This command is used to launch the Jupyter Lab interface for running example notebooks. ```bash jupyter lab ``` -------------------------------- ### Install DeGirum PySDK and Tools Source: https://context7.com/degirum/pysdkexamples/llms.txt Install the core SDK and the tools companion package using pip. ```python pip install degirum degirum_tools ``` -------------------------------- ### Install DeGirum and FiftyOne Source: https://github.com/degirum/pysdkexamples/blob/main/examples/specialized/object_detection_dataset.ipynb Installs the necessary DeGirum tools and FiftyOne libraries. Run this cell first to ensure all dependencies are met. ```python # make sure degirum-tools package is installed !pip show degirum-tools || pip install degirum-tools !pip show fiftyone || pip install fiftyone ``` -------------------------------- ### Install Python Dependencies for DeGirum PySDK Source: https://github.com/degirum/pysdkexamples/blob/main/README.md Installs the necessary Python dependencies for the DeGirum PySDK examples. This command should be executed after cloning the repository and navigating into its directory. ```bash pip install -r requirements.txt ``` -------------------------------- ### Set up Video Processing Pipeline Source: https://github.com/degirum/pysdkexamples/blob/main/examples/dgstreams/person_pose_detection_pipelined_video_stream.ipynb Creates a pipeline of gizmos for video processing: a `VideoSourceGizmo` for input, `AiSimpleGizmo` for person detection, `AiObjectDetectionCroppingGizmo` to crop detected persons, another `AiSimpleGizmo` for pose detection on cropped images, a `CropCombiningGizmo` to merge results, and a `VideoDisplayGizmo` for output. The pipeline is then started using `dgstreams.Composition`. ```python # create gizmos source = dgstreams.VideoSourceGizmo(video_source) # video source person = dgstreams.AiSimpleGizmo(person_det_model) # person detector crop = dgstreams.AiObjectDetectionCroppingGizmo( # cropper [person_det_model.label_dictionary[0]], crop_extent=10.0 ) pose = dgstreams.AiSimpleGizmo(pose_det_model) # pose detector combiner = dgstreams.CropCombiningGizmo() # combiner display = dgstreams.VideoDisplayGizmo( # display "Poses", show_ai_overlay=True, show_fps=True ) # create pipeline and composition, then start it dgstreams.Composition( source >> person >> crop >> pose >> combiner[1] >> display, person >> combiner[0] ).start() ``` -------------------------------- ### Install degirum-tools Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/zone_annotation.ipynb Ensures the degirum-tools package is installed. Run this before using other degirum_tools functionalities. ```python # make sure degirum-tools packages are installed !pip show degirum-tools || pip install degirum-tools ``` -------------------------------- ### Install degirum-tools Package Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/parking_management.ipynb Ensures the degirum-tools package is installed. Run this before proceeding with other DeGirum functionalities. ```python # make sure degirum and degirum-tools packages are installed !pip show degirum-tools || pip install degirum-tools ``` -------------------------------- ### Start Video Composition Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/smart_nvr.ipynb Initiates a video processing pipeline, starting from camera sources, through a detector, and finally to a display. Ensure all components (cam_source, detector, display) are properly initialized before calling start(). ```python dgstreams.Composition(cam_source >> detector >> display).start() ``` -------------------------------- ### Install degirum-tools Package Source: https://github.com/degirum/pysdkexamples/blob/main/examples/benchmarks/single_model_performance_test.ipynb Ensures the degirum-tools package is installed before proceeding. Use this in environments where the package might not be pre-installed. ```python # make sure degirum-tools package is installed !pip show degirum-tools || pip install degirum-tools ``` -------------------------------- ### Unified AI Inference with DeGirum PySDK Source: https://github.com/degirum/pysdkexamples/blob/main/examples/google_colab/pysdk_intel_hello_world.ipynb Defines model configurations for Intel hardware, specifies source images for different model types, and sets up interactive widgets to run AI inference. This example uses `degirum_tools.Display` to visualize inference results. ```python import ipywidgets as widgets from IPython.display import display import degirum as dg, degirum_tools # Define configurations model_configurations = { "Intel": { "zoo_url": "degirum/models_openvino", "model_names": { "classification": ["yolov8s_silu_imagenet--224x224_quant_openvino_multidevice_1"], "detection": ["yolov8n_relu6_coco--640x640_quant_openvino_multidevice_1"], "keypoint_detection": ["yolov8n_relu6_widerface_kpts--640x640_quant_openvino_multidevice_1"], "segmentation": ["yolov8n_relu6_coco_seg--640x640_quant_openvino_multidevice_1"], }, }, } source_images = { "classification": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Cat.jpg", "detection": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/bikes.jpg", "keypoint_detection": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Mask1.jpg", "segmentation": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/ThreePersons.jpg", } # Create widgets model_type_dropdown = widgets.Dropdown( options=list(source_images.keys()), value="detection", description="Model Type:", ) run_button = widgets.Button( description="Run Inference", button_style="success", # Optional: 'success', 'info', 'warning', 'danger' ) output = widgets.Output() # Define the inference function def run_inference(button_click): with output: output.clear_output() # Clear previous output hardware_option = "Intel" model_type = model_type_dropdown.value inference_host_address = "@cloud" zoo_url = model_configurations[hardware_option]["zoo_url"] model_name = model_configurations[hardware_option]["model_names"][model_type][0] source_image = source_images[model_type] print(f"Running inference on hardware: {hardware_option}") print(f"Model type: {model_type}") print(f"Model name: {model_name}") # Load and run inference model = dg.load_model( model_name=model_name, inference_host_address=inference_host_address, zoo_url=zoo_url, token=degirum_tools.get_token() ) inference_result = model(source_image) print("Inference result:") print(inference_result) with degirum_tools.Display() as display: display.show_image(inference_result) # Link the Run button to the inference function run_button.on_click(run_inference) # Display widgets and output display(model_type_dropdown, run_button, output) ``` -------------------------------- ### Initialize Video Source Source: https://github.com/degirum/pysdkexamples/blob/main/examples/dgstreams/dgstreams_demo.ipynb Sets up the video source for streaming. This can be a local camera, RTSP stream, YouTube video, or a video file. ```python import degirum_tools.streams as dgstreams # video_source: video source for inference # camera index for local camera # URL of RTSP stream # URL of YouTube Video # path to video file (mp4 etc) video_source = "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/example_video.mp4" ``` -------------------------------- ### Configure Inference and Video Parameters Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/parking_management.ipynb Sets up parameters for DeGirum inference, including hardware location, model zoo URL, model name, zone definition file, video source, and output video path. ```python # hw_location: where you want to run inference # "@cloud" to use DeGirum cloud # "@local" to run on local machine # IP address for AI server inference # vehicle_model_zoo_url: url/path for vehicle detection model zoo # Use cloud_zoo_url for @cloud, @local, and AI server inference options. # Use '' for an AI server serving models from a local folder. # Use a path to a JSON file for a single model zoo in case of @local inference. # vehicle_model_name: name of the model for person detection. # zones_json_name: path to zone JSON file # video_source: video source for inference # camera index for local camera # URL of RTSP stream # URL of YouTube Video # path to video file (mp4 etc) # output_video: annotated video destination # degirum_cloud_token: your token for accessing the DeGirum cloud platform hw_location = "@cloud" vehicle_model_zoo_url = "https://cs.degirum.com/degirum/visdrone" vehicle_model_name = "yolov8s_relu6_visdrone--640x640_float_openvino_cpu_1" zones_json_name = "parking_zones.json" video_source = "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Parking.mp4" output_video = "temp/Parking_annotated.mp4" ``` -------------------------------- ### Create Gizmos for Stream Processing Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/smart_nvr.ipynb Sets up the necessary gizmos for processing the video stream: a VideoSourceGizmo to read the video, an AiSimpleGizmo to perform inference using the loaded model, and a VideoDisplayGizmo for visualizing the results. ```python # # create gizmos # # video source gizmo cam_source = dgstreams.VideoSourceGizmo(video_source) # detection gizmo detector = dgstreams.AiSimpleGizmo(model) # local display gizmo (just for debugging) display = dgstreams.VideoDisplayGizmo(window_name, show_ai_overlay=True) ``` -------------------------------- ### Set up AI Video Streaming Pipeline Source: https://github.com/degirum/pysdkexamples/blob/main/examples/dgstreams/rtsp_smart_camera.ipynb Loads the AI model and sets up a pipeline of gizmos for video processing and streaming. This includes video source, AI detection, RTSP streaming, and an optional local display for debugging. ```python import degirum as dg, degirum_tools, time from degirum_tools import streams as dgstreams # load model model = dg.load_model( model_name, hw_location, model_zoo_url, degirum_tools.get_token(), overlay_show_probabilities=True, overlay_line_width=1, ) # create gizmos # video source gizmo cam_source = dgstreams.VideoSourceGizmo(video_source) # detection gizmo detector = dgstreams.AiSimpleGizmo(model) # video streamer gizmo streamer = dgstreams.VideoStreamerGizmo(f"rtsp://localhost:8554{url_path}", show_ai_overlay=True) # local display gizmo (just for debugging) display = dgstreams.VideoDisplayGizmo(show_ai_overlay=True) # start media server to serve RTSP streams with degirum_tools.MediaServer(): # connect gizmos into pipeline and start composition dgstreams.Composition(cam_source >> detector >> streamer, detector >> display).start() # # You can access the WebRTC stream at the following URL: http://localhost:8888/my-ai-stream/ # You can access the RTSP stream at the following URL: rtsp://localhost:8554/my-ai-stream/ # ``` -------------------------------- ### Install DeGirum Packages Source: https://github.com/degirum/pysdkexamples/blob/main/examples/basic/pysdk_hello_world.ipynb Install the necessary DeGirum packages using pip. These are required for using the PySDK and its associated tools. ```python %pip install degirum %pip install degirum_tools ``` -------------------------------- ### Configure Inference and Model Parameters Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/person_count_video.ipynb Sets up parameters for inference, including the hardware location, model zoo URLs, model names for person, head, and face detection, and the video source. ```python # hw_location: where you want to run inference # "@cloud" to use DeGirum cloud # "@local" to run on local machine # IP address for AI server inference # person_model_zoo_url: url/path for person model zoo # Use cloud_zoo_url for @cloud, @local, and AI server inference options. # Use '' for an AI server serving models from a local folder. # Use a path to a JSON file for a single model zoo in case of @local inference. # person_model_name: name of the model for person detection. # head_model_zoo_url: URL/path for the head model zoo. # head_model_name: name of the model for head detection. # face_model_zoo_url: URL/path for the face model zoo. # face_model_name: name of the model for face detection. # video_source: video source for inference # camera index for local camera # URL of RTSP stream # URL of YouTube Video # path to video file (mp4 etc) # degirum_cloud_token: your token for accessing the DeGirum cloud platform hw_location = "@cloud" model_zoo_url = "degirum/public" person_model_name = "yolov8m_relu6_person--960x960_float_openvino_cpu_1" head_model_name = "yolov8s_relu6_human_head--960x960_float_openvino_cpu_1" face_model_name = "yolov8s_relu6_face--640x640_float_openvino_cpu_1" ``` -------------------------------- ### Configure Inference and Video Source Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/car_wrong_direction_detection.ipynb Sets up essential parameters for inference, including the hardware location, model details, video source, and configuration for line crossing detection, clip duration, storage, and notifications. Adjust these variables based on your deployment environment and requirements. ```python # hw_location: where you want to run inference # "@cloud" to use DeGirum cloud # "@local" to run on local machine # IP address for AI server inference # model_zoo_url: url/path for model zoo # cloud_zoo_url: valid for @cloud, @local, and ai server inference options # '': ai server serving models from local folder # path to json file: single model zoo in case of @local inference # model_name: name of the model for running AI inference # video_source: video source for inference # camera index for local camera # URL of RTSP stream # URL of YouTube Video # path to video file (mp4 etc) # cross_line: line that marks the lane crossing. Format: [x_start, y_start, x_end, y_end]. # It should be oriented so cars moving in wrong direction would cross it from left to right # when looking towards line end. # clip_duration: duration of the video clip to save, in frames # storage_config: configuration for storing the results in S3-compatible storage # notification_config: Apprise-compatible configuration for sending notifications # (see https://github.com/caronc/apprise for details) import degirum as dg, degirum_tools hw_location = "@cloud" model_zoo_url = "degirum/public" model_name = "yolo_v5n_car_det--512x512_quant_n2x_orca1_1" video_source = ( "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Traffic.mp4" ) cross_line = [(800, 180, 900, 80)] # [x_start, y_start, x_end, y_end] clip_duration = 30 # frames storage_config = degirum_tools.ObjectStorageConfig( endpoint="./temp", # endpoint url, or path to local folder for local storage access_key="", # access key for S3-compatible storage secret_key="", # secret key for S3-compatible storage bucket="car_wrong_direction", # bucket name for S3-compatible storage or subdirectory name for local storage ) notification_config = degirum_tools.notification_config_console ``` -------------------------------- ### Start Degirum Streams Composition Source: https://github.com/degirum/pysdkexamples/blob/main/examples/dgstreams/multi_camera_multi_model_detection.ipynb Creates and starts a Degirum Streams composition using a given pipeline configuration. Ensure the pipeline is correctly defined before calling this. ```python dgstreams.Composition(*pipeline).start() ``` -------------------------------- ### Clone DeGirum PySDKExamples Repository Source: https://github.com/degirum/pysdkexamples/blob/main/README.md Clones the DeGirum PySDKExamples repository from GitHub. This command is typically run in a terminal or command prompt to obtain the example scripts. ```bash git clone https://github.com/DeGirum/PySDKExamples.git ``` -------------------------------- ### Configure Inference Parameters Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/stop_sign_violation_detection.ipynb Set up essential parameters for inference, including hardware location, model details, video source, and detection zones/lines. ```python # hw_location: where you want to run inference # "@cloud" to use DeGirum cloud # "@local" to run on local machine # IP address for AI server inference # model_zoo_url: url/path for model zoo # cloud_zoo_url: valid for @cloud, @local, and ai server inference options # '': ai server serving models from local folder # path to json file: single model zoo in case of @local inference # model_name: name of the model for running AI inference # video_source: video source for inference # camera index for local camera # URL of RTSP stream # URL of YouTube Video # path to video file (mp4 etc) # stop_zone: zone in which front of the car is detected near the stop line # stop_line: line that is considered as stop line hw_location = "@cloud" model_zoo_url = "degirum/public" model_name = "yolo_v5n_car_det--512x512_quant_n2x_orca1_1" video_source = "https://raw.githubusercontent.com/Leonnorblad/DetStopViolation/main/videos/example.mp4" stop_zone = [ [[677, 751], [911, 614], [1093, 652], [860, 804]], ] stop_line = [(860, 804, 1093, 652)] ``` -------------------------------- ### Load Model and Run Inference Source: https://github.com/degirum/pysdkexamples/blob/main/examples/basic/pysdk_hello_world.ipynb Loads a DeGirum AI model from a specified zoo URL and performs inference on a given image source. This is a standalone example for direct use. ```python import degirum as dg, degirum_tools hw_location = "@cloud" # "@cloud" for cloud inference, "@local" for local inference, or IP address for AI server inference model_zoo_url = "degirum/models_hailort" # models_hailort, models_tflite, models_openvino, models_n2x, models_rknn model_name = "yolov8n_relu6_coco--640x640_quant_hailort_hailo8_1" image_source = "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Mask1.jpg" # load object detection AI model model = dg.load_model( model_name=model_name, inference_host_address=hw_location, zoo_url=model_zoo_url, token=degirum_tools.get_token() ) # perform AI model inference on given image source inference_result = model(image_source) # show results of inference print(inference_result) # numeric results with degirum_tools.Display() as display: display.show_image(inference_result) ``` -------------------------------- ### Configure Inference Parameters Source: https://github.com/degirum/pysdkexamples/blob/main/examples/benchmarks/object_detection_multiplexing_multiple_streams.ipynb Set up inference parameters including hardware location, model zoo URL, model name, and input video file paths. Adjust these settings based on your deployment environment and desired model. ```python # hw_location: where you want to run inference # "@cloud" to use DeGirum cloud # "@local" to run on local machine # IP address for AI server inference # model_zoo_url: url/path for model zoo # cloud_zoo_url: valid for @cloud, @local, and ai server inference options # '': ai server serving models from local folder # path to json file: single model zoo in case of @local inference # model_name: name of the model for running AI inference # input_filenames: paths to video files for inference # offload_preprocessing: True to do image preprocessing outside of inference call # do_image_compression: True to do JPEG compression before sending image for inference hw_location = "@cloud" model_zoo_url = "degirum/public" model_name = "mobilenet_v2_ssd_coco--300x300_quant_n2x_orca1_1" input_filenames = [ "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Traffic.mp4", "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Traffic.mp4", "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Traffic.mp4", "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Traffic.mp4", ] offload_preprocessing = True # do image preprocessing outside of inference call do_image_compression = True # do JPEG compression before sending image for inference ``` -------------------------------- ### Instantiate Object Tracker Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/parking_management.ipynb Initializes an ObjectTracker for tracking objects across video frames. Configure 'class_list' and 'track_buffer' as needed. 'show_overlay' controls visualization. ```python # Instantiate an ObjectTracker Analyzer tracker = degirum_tools.ObjectTracker( class_list=class_list, track_buffer=track_buffer, trail_depth=track_buffer, show_overlay=False ) ``` -------------------------------- ### Define and Link Inference Function Source: https://github.com/degirum/pysdkexamples/blob/main/examples/basic/pysdk_hello_world.ipynb Defines the inference logic, including model loading and execution, and links it to a button click event. This snippet is part of a larger interactive application setup. ```python def run_inference(button_click): with output: output.clear_output() # Clear previous output hardware_option = hardware_dropdown.value model_type = model_type_dropdown.value inference_host_address = "@cloud" zoo_url = model_configurations[hardware_option]["zoo_url"] model_name = model_configurations[hardware_option]["model_names"][model_type][0] source_image = source_images[model_type] print(f"Running inference on hardware: {hardware_option}") print(f"Model type: {model_type}") print(f"Model name: {model_name}") # Load and run inference model = dg.load_model( model_name=model_name, inference_host_address=inference_host_address, zoo_url=zoo_url, token=degirum_tools.get_token() ) inference_result = model(source_image) print("Inference result:") print(inference_result) with degirum_tools.Display() as display: display.show_image(inference_result) # Link the Run button to the inference function run_button.on_click(run_inference) # Display widgets and output display(hardware_dropdown, model_type_dropdown, run_button, output) ``` -------------------------------- ### Configure AI Models for Different Hardware Source: https://github.com/degirum/pysdkexamples/blob/main/examples/basic/pysdk_hello_world.ipynb Define model configurations including zoo URLs and model names for various AI tasks and hardware platforms. This setup allows for flexible model selection and deployment. ```python import ipywidgets as widgets from IPython.display import display import degirum as dg, degirum_tools # Define configurations model_configurations = { "Hailo": { "zoo_url": "degirum/models_hailort", "model_names": { "classification": ["yolov8s_silu_imagenet--224x224_quant_hailort_hailo8_1"], "detection": ["yolov8n_relu6_coco--640x640_quant_hailort_hailo8_1"], "keypoint_detection": ["yolov8n_relu6_widerface_kpts--640x640_quant_hailort_hailo8_1"], "segmentation": ["yolov8n_relu6_coco_seg--640x640_quant_hailort_hailo8_1"], }, }, "Google": { "zoo_url": "degirum/models_tflite", "model_names": { "classification": ["yolov8s_silu_imagenet--224x224_quant_tflite_edgetpu_1"], "detection": ["yolov8n_relu6_coco--640x640_quant_tflite_edgetpu_1"], "keypoint_detection": ["yolov8n_relu6_widerface_kpts--640x640_quant_tflite_edgetpu_1"], "segmentation": ["yolov8n_relu6_coco_seg--640x640_quant_tflite_edgetpu_1"], }, }, "Intel": { "zoo_url": "degirum/models_openvino", "model_names": { "classification": ["yolov8s_silu_imagenet--224x224_float_openvino_multidevice_1"], "detection": ["yolov8n_relu6_coco--640x640_float_openvino_multidevice_1"], "keypoint_detection": ["yolov8n_relu6_widerface_kpts--640x640_float_openvino_multidevice_1"], "segmentation": ["yolov8n_relu6_coco_seg--640x640_float_openvino_multidevice_1"], }, }, "DeGirum": { "zoo_url": "degirum/models_n2x", "model_names": { "classification": ["yolov8s_silu_imagenet--224x224_quant_n2x_orca1_1"], "detection": ["yolov8n_relu6_coco--640x640_quant_n2x_orca1_1"], "keypoint_detection": ["yolov8n_relu6_widerface_kpts--640x640_quant_n2x_orca1_1"], "segmentation": ["yolov8n_relu6_coco_seg--640x640_quant_n2x_orca1_1"], }, }, "RockChip": { "zoo_url": "degirum/models_rknn", "model_names": { "classification": ["yolov8s_silu_imagenet--224x224_quant_rknn_rk3588_1"], "detection": ["yolov8n_relu6_coco--640x640_quant_rknn_rk3588_1"], "keypoint_detection": ["yolov8n_relu6_widerface_kpts--640x640_quant_rknn_rk3588_1"], "segmentation": ["yolov8n_relu6_coco_seg--640x640_quant_rknn_rk3588_1"], }, }, } source_images = { "classification": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Cat.jpg", "detection": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/bikes.jpg", "keypoint_detection": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Mask1.jpg", "segmentation": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/ThreePersons.jpg", } # Create widgets hardware_dropdown = widgets.Dropdown( options=model_configurations.keys(), value="Hailo", description="Hardware:", ) model_type_dropdown = widgets.Dropdown( options=list(source_images.keys()), value="detection", description="Model Type:", ) run_button = widgets.Button( description="Run Inference", button_style="success", # Optional: 'success', 'info', 'warning', 'danger' ) output = widgets.Output() ``` -------------------------------- ### Import DeGirum Libraries and Setup NMS Options Source: https://github.com/degirum/pysdkexamples/blob/main/examples/specialized/advanced_tiling_strategies.ipynb Imports necessary DeGirum modules and defines base Non-Maximum Suppression (NMS) options for object detection post-processing. These settings are used across various tiling strategies. ```python # imports and variables used in most cells import degirum as dg import degirum_tools as dgt from degirum_tools import TileExtractorPseudoModel, TileModel, LocalGlobalTileModel, BoxFusionTileModel, BoxFusionLocalGlobalTileModel from degirum_tools import NmsBoxSelectionPolicy, NmsOptions # Base NMS options. nms_options = NmsOptions( threshold=0.6, use_iou=True, box_select=NmsBoxSelectionPolicy.MOST_PROBABLE, ) ``` -------------------------------- ### Video Stream Inference with degirum_tools.predict_stream() Source: https://context7.com/degirum/pysdkexamples/llms.txt Run inference over every frame of a video source and get a lazy iterator of results. Use `degirum_tools.Display` for real-time annotated display. Supports video files, webcams, RTSP, and YouTube URLs. ```python import degirum as dg, degirum_tools model = dg.load_model( model_name="yolo_v5s_coco--512x512_quant_n2x_orca1_1", inference_host_address="@cloud", zoo_url="degirum/public", token=degirum_tools.get_token(), ) video_source = "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/example_video.mp4" # video_source = 0 # local webcam # video_source = "rtsp://..." # RTSP stream inference_results = degirum_tools.predict_stream(model, video_source) # Press 'x' or 'q' to stop with degirum_tools.Display("AI Camera") as display: for result in inference_results: display.show(result) ``` -------------------------------- ### Create Object Tracker and Zone Counter Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/person_age_gender_detection.ipynb Initializes an ObjectTracker to associate detected persons across frames and a ZoneCounter to count objects within a defined area. This setup is crucial for analyzing object trajectories and presence within specific regions. ```python # # Create analyzers # # create object tracker tracker = degirum_tools.ObjectTracker( class_list=[person_det_model.label_dictionary[0]], track_thresh=0.35, track_buffer=100, match_thresh=0.9999, trail_depth=20, anchor_point=degirum_tools.AnchorPoint.CENTER, ) # create zone counter counting_zone = [[[300, 10], [500, 10], [500, 710], [300, 710]]] zone_counter = degirum_tools.ZoneCounter( counting_zone, triggering_position=degirum_tools.AnchorPoint.CENTER, annotation_color=(255, 0, 0), ) ``` -------------------------------- ### Configure Inference Options and Parameters Source: https://github.com/degirum/pysdkexamples/blob/main/examples/specialized/object_in_zone_counting_video_stream.ipynb Set up inference parameters including hardware location, model details, video source, polygon zones, and target object classes. Adjust these variables to suit your specific use case. ```python # hw_location: where you want to run inference # "@cloud" to use DeGirum cloud # "@local" to run on local machine # IP address for AI server inference # model_zoo_url: url/path for model zoo # cloud_zoo_url: valid for @cloud, @local, and ai server inference options # '': ai server serving models from local folder # path to json file: single model zoo in case of @local inference # model_name: name of the model for running AI inference # video_source: video source for inference # camera index for local camera # URL of RTSP stream # URL of YouTube Video # path to video file (mp4 etc) # polygon_zones: zones in which objects need to be counted # class_list: list of classes to be counted # per_class_display: Boolean to specify if per class counts are to be displayed hw_location = "@cloud" model_zoo_url = "degirum/public" model_name = "yolo_v5s_coco--512x512_quant_n2x_orca1_1" video_source = "https://github.com/DeGirum/PySDKExamples/raw/main/images/Traffic.mp4" polygon_zones = [ [[265, 260], [730, 260], [870, 450], [120, 450]], [[400, 100], [610, 100], [690, 200], [320, 200]], ] class_list = ["car", "motorbike", "truck"] per_class_display = True window_name="AI Camera" ``` -------------------------------- ### Processing and Visualizing Latency Times Source: https://github.com/degirum/pysdkexamples/blob/main/examples/benchmarks/object_detection_multiplexing_multiple_streams.ipynb This snippet calculates and prints a histogram of latency times in milliseconds. It converts collected start and end times into a NumPy array for efficient calculation. Use this to analyze the performance and identify potential bottlenecks in the inference pipeline. ```python # process latency times end_times = numpy.array(end_times) start_times = numpy.array(start_times) latency_times_ms = (end_times - start_times) * 1000 print("\nLatency Histogram") latency_hist = numpy.histogram(latency_times_ms) for hval, bin in zip(latency_hist[0], latency_hist[1]): print(f"{bin:4.0f} ms: {hval:4}") ``` -------------------------------- ### Load AI Model for Inference Source: https://github.com/degirum/pysdkexamples/blob/main/examples/google_colab/pysdk_google_hello_world.ipynb This code snippet demonstrates how to load a pre-trained AI model from a specified model zoo URL for inference. It configures the hardware location, model name, and image source. Use this for direct model loading and inference setup. ```python import degirum as dg, degirum_tools hw_location = "@cloud" # "@cloud" for cloud inference, "@local" for local inference, or IP address for AI server inference model_zoo_url = "degirum/google" model_name = "yolov8n_relu6_coco--512x512_quant_tflite_edgetpu_1" image_source = "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Mask1.jpg" # load object detection AI model model = dg.load_model( model_name=model_name, inference_host_address=hw_location, zoo_url=model_zoo_url, token=degirum_tools.get_token() ) ``` -------------------------------- ### Configure Inference Parameters Source: https://github.com/degirum/pysdkexamples/blob/main/examples/specialized/object_in_zone_counting_video_file.ipynb Set up inference parameters including hardware location, model details, video source, polygon zones, and output path. Adjust these variables to suit your specific use case. ```python # hw_location: where you want to run inference # "@cloud" to use DeGirum cloud # "@local" to run on local machine # IP address for AI server inference # model_zoo_url: url/path for model zoo # cloud_zoo_url: valid for @cloud, @local, and ai server inference options # '': ai server serving models from local folder # path to json file: single model zoo in case of @local inference # model_name: name of the model for running AI inference # video_source: video source for inference # camera index for local camera # URL of RTSP stream # URL of YouTube Video # path to video file (mp4 etc) # polygon_zones: zones in which objects need to be counted # class_list: list of classes to be counted # per_class_display: Boolean to specify if per class counts are to be displayed # ann_path: path to save annotated video hw_location = "@cloud" model_zoo_url = "degirum/public" model_name = "yolo_v5s_coco--512x512_quant_n2x_orca1_1" video_source = "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Traffic.mp4" polygon_zones = [ [[265, 260], [730, 260], [870, 450], [120, 450]], [[400, 100], [610, 100], [690, 200], [320, 200]], ] class_list = ["car", "motorbike", "truck"] per_class_display = True ann_path = "temp/object_in_zone_counting_video_file.mp4" ``` -------------------------------- ### Initialize DeGirum Model and Components Source: https://github.com/degirum/pysdkexamples/blob/main/examples/applications/car_wrong_direction_detection.ipynb Loads the specified DeGirum model and initializes the object tracker, line counter, event detector, and event notifier. This setup configures the core logic for detecting and reacting to cars crossing the defined line in the wrong direction. ```python # load model model = dg.load_model( model_name=model_name, inference_host_address=hw_location, zoo_url=model_zoo_url, token=degirum_tools.get_token(), overlay_color=[(255, 0, 0)], overlay_line_width=1, overlay_show_labels=False, ) # bbox anchor point for object tracker and line counter anchor = degirum_tools.AnchorPoint.CENTER # create object tracker object_tracker = degirum_tools.ObjectTracker( track_thresh=0.35, track_buffer=100, match_thresh=0.9999, trail_depth=20, anchor_point=anchor, ) # create line crossing counter line_counter = degirum_tools.LineCounter( cross_line, anchor, accumulate=False, show_overlay=True, annotation_color=(255, 255, 0), ) event_name = "car_moving_wrong_direction" # create event detector: car crosses a line linecross_detector = degirum_tools.EventDetector( f""" Trigger: {event_name} when: LineCount with: directions: [right] is greater than: 0 during: [1, frame] for at least: [1, frame] """, show_overlay=False, ) # create event notifier: car crosses line in wrong direction annotation_pos = cross_line[0][:2] linecross_notifier = degirum_tools.EventNotifier( "Wrong Direction", event_name, message="{time}: {result.events_detected} ({url})", annotation_pos=annotation_pos, annotation_color=(255, 0, 0), annotation_cool_down=1.0, notification_config=notification_config, clip_save=True, clip_duration=clip_duration, clip_pre_trigger_delay=clip_duration // 2, storage_config=storage_config, ) ``` -------------------------------- ### Initialize Object Tracker and Selector Source: https://github.com/degirum/pysdkexamples/blob/main/examples/specialized/hand_tracking_and_control.ipynb Sets up an object tracker for continuous tracking and an object selector to focus on the most relevant detected object (e.g., the largest hand). ```python # create object tracker tracker = degirum_tools.ObjectTracker( track_thresh=0.35, track_buffer=100, match_thresh=0.9999, anchor_point=degirum_tools.AnchorPoint.CENTER, ) # create object selector to track only one hand selector = degirum_tools.ObjectSelector( top_k=1, use_tracking=True, selection_strategy=degirum_tools.ObjectSelectionStrategies.LARGEST_AREA, ) ``` -------------------------------- ### Interactive AI Inference with PySDK Widgets Source: https://github.com/degirum/pysdkexamples/blob/main/examples/google_colab/pysdk_deepx_hello_world.ipynb Sets up interactive widgets to select model type, run inference on a selected model and image, and display results using DeGirum PySDK. This example is useful for experimenting with different models and image sources in a live environment. ```python import ipywidgets as widgets from IPython.display import display import degirum as dg, degirum_tools # Define configurations model_configurations = { "DEEPX": { "zoo_url": "degirum/deepx", "model_names": { "classification": ["mobilenetv2--224x224_quant_deepx_m1a_1"], "detection": ["yolov8n_coco--640x640_quant_deepx_m1a_1"], "keypoint_detection": ["yolov8n_relu6_widerface_kpts--640x640_quant_deepx_m1a_1"] }, }, } source_images = { "classification": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Cat.jpg", "detection": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/bikes.jpg", "keypoint_detection": "https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Mask1.jpg" } # Create widgets model_type_dropdown = widgets.Dropdown( options=list(source_images.keys()), value="classification", description="Model Type:", ) run_button = widgets.Button( description="Run Inference", button_style="success", # Optional: 'success', 'info', 'warning', 'danger' ) output = widgets.Output() # Define the inference function def run_inference(button_click): with output: output.clear_output() # Clear previous output hardware_option = "DEEPX" model_type = model_type_dropdown.value inference_host_address = "@cloud" zoo_url = model_configurations[hardware_option]["zoo_url"] model_name = model_configurations[hardware_option]["model_names"][model_type][0] source_image = source_images[model_type] print(f"Running inference on hardware: {hardware_option}") print(f"Model type: {model_type}") print(f"Model name: {model_name}") # Load and run inference model = dg.load_model( model_name=model_name, inference_host_address=inference_host_address, zoo_url=zoo_url, token=degirum_tools.get_token() ) inference_result = model(source_image) print("Inference result:") print(inference_result) with degirum_tools.Display() as display: display.show_image(inference_result) # Link the Run button to the inference function run_button.on_click(run_inference) # Display widgets and output display(model_type_dropdown, run_button, output) ``` -------------------------------- ### Initialize DeGirum Pipeline Components Source: https://github.com/degirum/pysdkexamples/blob/main/examples/dgstreams/multi_camera_multi_model_detection.ipynb Loads AI models, creates video source gizmos, resizers, detectors, a result combiner, and a display gizmo. It also asserts that all models share the same input configuration. ```python import degirum as dg, degirum_tools from degirum_tools import streams as dgstreams # create PySDK AI model objects models = [ dg.load_model( model_name=model_name, inference_host_address=hw_location, zoo_url=model_zoo_url, token=degirum_tools.get_token(), overlay_line_width=2, ) for model_name in model_names ] # check that all models have the same input configuration assert all( type(model._preprocessor) == type(models[0]._preprocessor) and model.model_info.InputH == models[0].model_info.InputH and model.model_info.InputW == models[0].model_info.InputW for model in models[1:] ) # create video source gizmos; # stop_composition_on_end=True to stop whole composition when one (shorter) video source ends sources = [ dgstreams.VideoSourceGizmo(src, stop_composition_on_end=True) for src in video_sources ] # create image resizer gizmos, one per video source # (we use separate resizers to do resize only once per source to improve performance) resizers = [dgstreams.AiPreprocessGizmo(models[0]) for _ in video_sources] # create multi-input detector gizmos, one per model detectors = [dgstreams.AiSimpleGizmo(model, inp_cnt=len(video_sources)) for model in models] # create result combiner gizmo to combine results from all detectors into single result combiner = dgstreams.AiResultCombiningGizmo(len(models)) # create multi-window video multiplexing display gizmo win_captions = [f"Stream #{i}: {str(src)}" for i, src in enumerate(video_sources)] display = dgstreams.VideoDisplayGizmo( win_captions, show_ai_overlay=True, show_fps=True, multiplex=True ) # connect all gizmos in the pipeline # source[i] -> resizer[i] -> detector[j] -> combiner -> display pipeline = ( # each source is connected to corresponding resizer (source >> resizer for source, resizer in zip(sources, resizers)), # each resizer is connected to every detector ( resizer >> detector[ri] for detector in detectors for ri, resizer in enumerate(resizers) ), # each detector is connected to result combiner (detector >> combiner[di] for di, detector in enumerate(detectors)), # result combiner is connected to display combiner >> display, ) ``` -------------------------------- ### Load Model, Create Zone Counter, and Run Inference Source: https://github.com/degirum/pysdkexamples/blob/main/examples/specialized/object_in_zone_counting_video_stream.ipynb This snippet loads the specified model, initializes a ZoneCounter with defined parameters, attaches it to the model, and then runs inference on the video stream. Results are displayed in a window. ```python import degirum as dg, degirum_tools # load model model = dg.load_model( model_name=model_name, inference_host_address=hw_location, zoo_url=model_zoo_url, token=degirum_tools.get_token(), overlay_color=[(255,0,0)] ) # create zone counter zone_counter = degirum_tools.ZoneCounter( polygon_zones, class_list=class_list, per_class_display=per_class_display, triggering_position=degirum_tools.AnchorPoint.CENTER, window_name=window_name, # attach display window for interactive zone adjustment ) # attach zone counter to model degirum_tools.attach_analyzers(model, [zone_counter]) # run inference inference_results=degirum_tools.predict_stream(model, video_source,) # display results with degirum_tools.Display(window_name) as display: for inference_result in inference_results: display.show(inference_result) ```