### Start ClearML Agent for Remote Experiment Execution Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md Transform any machine into a ClearML agent by running this command. The agent listens to specified queues for incoming tasks, recreates the experiment environment (including installed packages and uncommitted changes), and executes the tasks while continuously reporting scalars, plots, and other metrics back to the ClearML experiment manager. The `--docker` flag enables containerized execution. ```bash clearml-agent daemon --queue [--docker] ``` -------------------------------- ### Setup and Run Local Hyperparameter Optimization with ClearML Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md To perform hyperparameter optimization locally, first install Optuna if you plan to use it. Then, run the provided ClearML HPO script (`utils/loggers/clearml/hpo.py`). This script clones an existing ClearML training task (which must have been run at least once) and automatically reruns it with varied hyperparameters. You can modify `task.execute_locally()` to `task.execute()` within the script to queue it for remote execution. ```bash # To use optuna, install it first, otherwise you can change the optimizer to just be RandomSearch pip install optuna python utils/loggers/clearml/hpo.py ``` -------------------------------- ### Install ClearML for YOLOv5 Integration Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md Installs ClearML version 1.2.0 or newer, specifically enabling automatic integration and experiment tracking for YOLOv5 training scripts. This ensures that all subsequent YOLOv5 training runs are captured by the ClearML experiment manager. ```bash pip install clearml>=1.2.0 ``` -------------------------------- ### Install Comet ML Library Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md Instructions to install the Comet ML Python library using pip. ```shell pip install comet_ml ``` -------------------------------- ### Install ClearML Python Package Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md Installs the `clearml` Python package using pip, which is the foundational step for enabling ClearML's experiment tracking and MLOps capabilities within your Python environment. ```bash pip install clearml ``` -------------------------------- ### Train YOLOv5 with ClearML Default Tracking Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md Runs the YOLOv5 `train.py` script with standard parameters. With ClearML installed, this command automatically captures comprehensive experiment details, including source code, installed packages, hyperparameters, model files, console output, and various performance metrics, storing them in the ClearML experiment manager. ```bash python train.py --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache ``` -------------------------------- ### Train YOLOv5 with COCO formatted dataset Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Start a training session using a COCO formatted dataset. This involves defining the dataset paths in a `data.yml` file and then initiating training with the specified data and pretrained weights. ```yaml # data.yml train_json_path: "train.json" train_image_dir: "train_image_dir/" val_json_path: "val.json" val_image_dir: "val_image_dir/" ``` ```bash $ yolov5 train --data data.yaml --weights yolov5s.pt ``` -------------------------------- ### Install YOLOv5 Python Package Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Install the YOLOv5 object detection library using pip. This package is compatible with Python 3.7 and newer versions. ```console pip install yolov5 ``` -------------------------------- ### Perform Object Detection Inference with YOLOv5 in Python Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Demonstrates how to load pre-trained or custom YOLOv5 models, configure inference parameters like confidence and IoU thresholds, perform detection on images, and process or visualize the results. Includes examples for different input sizes and augmentations. ```python import yolov5 # load pretrained model model = yolov5.load('yolov5s.pt') # or load custom model model = yolov5.load('train/best.pt') # set model parameters model.conf = 0.25 # NMS confidence threshold model.iou = 0.45 # NMS IoU threshold model.agnostic = False # NMS class-agnostic model.multi_label = False # NMS multiple labels per box model.max_det = 1000 # maximum number of detections per image # set image img = 'https://github.com/ultralytics/yolov5/raw/master/data/images/zidane.jpg' # perform inference results = model(img) # inference with larger input size results = model(img, size=1280) # inference with test time augmentation results = model(img, augment=True) # parse results predictions = results.pred[0] boxes = predictions[:, :4] # x1, y1, x2, y2 scores = predictions[:, 4] categories = predictions[:, 5] # show detection bounding boxes on image results.show() # save results into "results/" folder results.save(save_dir='results/') ``` -------------------------------- ### Execute Comet Optimizer Sweep with Base Configuration Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This command runs the `hpo.py` script to start a hyperparameter optimization sweep using Comet. It requires a JSON configuration file that defines the sweep parameters. This is the fundamental way to initiate a Comet Optimizer sweep. ```shell python utils/loggers/comet/hpo.py \ --comet_optimizer_config "utils/loggers/comet/optimizer_config.json" ``` -------------------------------- ### YOLOv5 Python Package Dependencies (requirements.txt) Source: https://github.com/fcakyon/yolov5-pip/blob/main/requirements.txt This snippet lists all Python packages required for the YOLOv5 project, including minimum version specifications and functional categories like base operations, logging, plotting, and export capabilities. It is designed to be used with `pip install -r requirements.txt` to set up the project environment. ```Python # YOLOv5 requirements # Usage: pip install -r requirements.txt # Base ------------------------------------------------------------------------ gitpython>=3.1.30 matplotlib>=3.3 numpy>=1.18.5 opencv-python>=4.1.1 Pillow>=7.1.2 psutil # system resources PyYAML>=5.3.1 requests>=2.23.0 scipy>=1.4.1 thop>=0.1.1 # FLOPs computation torch>=1.7.0 # see https://pytorch.org/get-started/locally (recommended) torchvision>=0.8.1 tqdm>=4.64.0 ultralytics>=8.0.100 # protobuf<=3.20.1 # https://github.com/ultralytics/yolov5/issues/8012 # Logging --------------------------------------------------------------------- tensorboard>=2.4.1 # clearml>=1.2.0 # comet # Plotting -------------------------------------------------------------------- pandas>=1.1.4 seaborn>=0.11.0 # Export ---------------------------------------------------------------------- # coremltools>=6.0 # CoreML export # onnx>=1.10.0 # ONNX export # onnx-simplifier>=0.4.1 # ONNX simplifier # nvidia-pyindex # TensorRT export # nvidia-tensorrt # TensorRT export # scikit-learn<=1.1.2 # CoreML quantization # tensorflow>=2.4.0 # TF exports (-cpu, -aarch64, -macos) # tensorflowjs>=3.9.0 # TF.js export # openvino-dev # OpenVINO export # Deploy ---------------------------------------------------------------------- setuptools>=65.5.1 # Snyk vulnerability fix # tritonclient[all]~=2.24.0 # Extras ---------------------------------------------------------------------- # ipython # interactive notebook # mss # screenshots # albumentations>=1.0.3 # pycocotools>=2.0.6 # COCO mAP # CLI fire # AWS boto3>=1.19.1 # coco to yolov5 conversion sahi>=0.11.10 # huggingface huggingface-hub>=0.12.0,<0.25.0 # roboflow roboflow>=0.2.29 ``` -------------------------------- ### Initialize ClearML SDK Credentials Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md Executes the `clearml-init` command to connect the ClearML SDK to a ClearML server. This interactive process prompts the user to enter API credentials, establishing the link required for experiment tracking and data management. ```bash clearml-init ``` -------------------------------- ### Upload Dataset to ClearML using Sync Command Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md Navigate to the dataset's root directory and use the `clearml-data sync` command to upload and version your dataset in ClearML. This command acts as a convenient shorthand for creating, adding files, and closing a dataset, ensuring all data is tracked and reproducible. ```bash cd coco128 clearml-data sync --project YOLOv5 --name coco128 --folder . ``` -------------------------------- ### Train YOLOv5 with Neptune.AI experiment tracking Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Visualize your training experiments via Neptune.AI. This integration requires `neptune-client>=0.10.10` and allows tracking metrics and artifacts by specifying your Neptune project and token. ```bash $ yolov5 train --data data.yaml --weights yolov5s.pt --neptune_project NAMESPACE/PROJECT_NAME --neptune_token YOUR_NEPTUNE_TOKEN ``` -------------------------------- ### Run YOLOv5 Training with ClearML Dataset Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md Execute custom YOLOv5 model training using a dataset versioned in ClearML. Specify the dataset by its ClearML ID using the `clearml://` prefix in the `--data` argument. This command also includes standard training parameters such as image size, batch size, epochs, and pre-trained weights. ```bash python train.py --img 640 --batch 16 --epochs 3 --data clearml:// --weights yolov5s.pt --cache ``` -------------------------------- ### Train YOLOv5 with Custom ClearML Project and Task Names Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md Executes the YOLOv5 training script while specifying custom project and task names for ClearML. This allows for better organization and categorization of experiments within the ClearML UI, using `--project` and `--name` arguments. ```bash python train.py --project my_project --name my_training --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache ``` -------------------------------- ### Train YOLOv5 with Dataset from Comet Artifacts Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This command initiates YOLOv5 training using a dataset previously saved as a Comet Artifact. The `--data` flag points to a local YAML file (e.g., `artifact.yaml`) that contains the Comet Artifact resource URL, allowing the training script to fetch the dataset. ```shell python train.py \ --img 640 \ --batch 16 \ --epochs 5 \ --data artifact.yaml \ --weights yolov5s.pt ``` -------------------------------- ### Upload Dataset to ClearML using Individual Commands Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md For more granular control, you can upload a dataset to ClearML by executing individual commands: `clearml-data create` to initialize a new dataset, `clearml-data add` to include files from the current directory, and `clearml-data close` to finalize the dataset version. Optionally, use `--parent ` to base this version on an existing one, preventing duplicate file uploads. ```bash # Optionally add --parent if you want to base # this version on another dataset version, so no duplicate files are uploaded! clearml-data create --name coco128 --project YOLOv5 clearml-data add --files . clearml-data close ``` -------------------------------- ### Train YOLOv5 model with custom data.yaml Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Finetune one of the pretrained YOLOv5 models using your custom `data.yaml`. This command specifies the data configuration, initial weights, batch size, and image size for training. ```bash $ yolov5 train --data data.yaml --weights yolov5s.pt --batch-size 16 --img 640 yolov5m.pt 8 yolov5l.pt 4 yolov5x.pt 2 ``` -------------------------------- ### Run YOLOv5 Training Script with Comet Integration Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md Execute the YOLOv5 training script with specified image size, batch size, epochs, data configuration, and pre-trained weights. Comet will automatically log metrics and parameters. ```shell python train.py --img 640 --batch 16 --epochs 5 --data coco128.yaml --weights yolov5s.pt ``` -------------------------------- ### Directly Call YOLOv5 Training, Validation, Detection, and Export Functions Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Illustrates how to import and execute core YOLOv5 functionalities such as training, validation, detection, and model export directly from the Python package. Shows how to pass various arguments to these functions. ```python from yolov5 import train, val, detect, export # from yolov5.classify import train, val, predict # from yolov5.segment import train, val, predict train.run(imgsz=640, data='coco128.yaml') val.run(imgsz=640, data='coco128.yaml', weights='yolov5s.pt') detect.run(imgsz=640) export.run(imgsz=640, weights='yolov5s.pt') from yolov5 import detect img_url = 'https://github.com/ultralytics/yolov5/raw/master/data/images/zidane.jpg' detect.run(source=img_url, weights="yolov5s6.pt", conf_thres=0.25, imgsz=640) ``` -------------------------------- ### Train and predict with YOLOv5 instance segmentation model Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Perform training, validation, and prediction tasks using the YOLOv5 instance segmentation model. This includes commands for training a segmentation model with specified image size, weights, and epochs, as well as predicting on new images. ```bash $ yolov5 segment train --img 640 --weights yolov5s-seg.pt --epochs 1 ``` ```bash $ yolov5 segment predict --img 640 --weights yolov5s-seg.pt --source images/ ``` -------------------------------- ### Train YOLOv5 and upload weights to Huggingface Hub Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Automatically upload your trained YOLOv5 model weights to Huggingface Hub. This requires a Huggingface model ID and a write token for authentication. ```bash $ yolov5 train --data data.yaml --weights yolov5s.pt --hf_model_id username/modelname --hf_token YOUR-HF-WRITE-TOKEN ``` -------------------------------- ### Configure Comet Credentials via Configuration File Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md Create a .comet.config file in your working directory to set Comet API key and project name. The project name defaults to 'yolov5'. ```ini [comet] api_key= project_name= # This will default to 'yolov5' ``` -------------------------------- ### Train YOLOv5 with Roboflow Universe dataset Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Train your YOLOv5 model using datasets from Roboflow Universe. Requires `roboflow>=0.2.29` and a Roboflow API token. The dataset URL must follow a specific format. ```bash $ yolov5 train --data DATASET_UNIVERSE_URL --weights yolov5s.pt --roboflow_token YOUR_ROBOFLOW_TOKEN ``` -------------------------------- ### Configure ClearML for Remote YOLOv5 Training Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/clearml/README.md This Python code snippet demonstrates how to modify the YOLOv5 training script (`training.py`) to enable remote execution via ClearML. By adding `loggers.clearml.task.execute_remotely(queue='my_queue')` after the ClearML logger instantiation, the script will package and send the task to the specified ClearML queue instead of running locally, allowing an agent to pick it up. ```python # ... # Loggers data_dict = None if RANK in {-1, 0}: loggers = Loggers(save_dir, weights, opt, hyp, LOGGER) # loggers instance if loggers.clearml: loggers.clearml.task.execute_remotely(queue='my_queue') # <------ ADD THIS LINE # Data_dict is either None is user did not choose for ClearML dataset or is filled in by ClearML data_dict = loggers.clearml.data_dict # ... ``` -------------------------------- ### Export YOLOv5 model weights to various formats Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Export your fine-tuned YOLOv5 weights to multiple formats such as `torchscript`, `onnx`, `coreml`, `pb`, `tflite`, and `tfjs`. This command allows for model deployment across different platforms. ```bash $ yolov5 export --weights yolov5s.pt --include torchscript,onnx,coreml,pb,tfjs ``` -------------------------------- ### Configure Comet Credentials via Environment Variables Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md Set Comet API key and project name as environment variables for authentication and project association. The project name defaults to 'yolov5'. ```shell export COMET_API_KEY= export COMET_PROJECT_NAME= # This will default to 'yolov5' ``` -------------------------------- ### Execute Comet Optimizer Sweep with Custom Training Arguments Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This command extends the basic sweep execution by allowing additional `train.py` arguments to be passed directly. It demonstrates how to customize training parameters like save period and bounding box interval within the sweep. This enables fine-grained control over the training process during optimization. ```shell python utils/loggers/comet/hpo.py \ --comet_optimizer_config "utils/loggers/comet/optimizer_config.json" \ --save-period 1 \ --bbox_interval 1 ``` -------------------------------- ### Perform YOLOv5 object detection inference Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Run inference using the `yolov5 detect` command on various sources, including webcams, image files, video files, directories, glob patterns, and different types of network streams. Models are automatically downloaded from the latest YOLOv5 release, and results are saved to `runs/detect`. ```bash $ yolov5 detect --source 0 # webcam file.jpg # image file.mp4 # video path/ # directory path/*.jpg # glob rtsp://170.93.143.139/rtplive/470011e600ef003a004ee33696235daa # rtsp stream rtmp://192.168.1.105/live/test # rtmp stream http://112.50.243.8/PLTV/88888888/224/3221225900/1.m3u8 # http stream ``` -------------------------------- ### Enable Model Checkpoint Logging to Comet Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md Configure the YOLOv5 training script to save and log model checkpoints to Comet at a specified interval using the `--save-period` argument. ```shell python train.py \ --img 640 \ --batch 16 \ --epochs 5 \ --data coco128.yaml \ --weights yolov5s.pt \ --save-period 1 ``` -------------------------------- ### Configure Advanced Comet Logging Options via Environment Variables Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md Set various Comet logging behaviors using environment variables, including online/offline mode, model name, confusion matrix logging, image upload limits, per-class metrics, checkpoint filename, and batch-level metrics. ```shell export COMET_MODE=online # Set whether to run Comet in 'online' or 'offline' mode. Defaults to online export COMET_MODEL_NAME= #Set the name for the saved model. Defaults to yolov5 export COMET_LOG_CONFUSION_MATRIX=false # Set to disable logging a Comet Confusion Matrix. Defaults to true export COMET_MAX_IMAGE_UPLOADS= # Controls how many total image predictions to log to Comet. Defaults to 100. export COMET_LOG_PER_CLASS_METRICS=true # Set to log evaluation metrics for each detected class at the end of training. Defaults to false export COMET_DEFAULT_CHECKPOINT_FILENAME= # Set this if you would like to resume training from a different checkpoint. Defaults to 'last.pt' export COMET_LOG_BATCH_LEVEL_METRICS=true # Set this if you would like to log training metrics at the batch level. Defaults to false. export COMET_LOG_PREDICTIONS=true # Set this to false to disable logging model predictions ``` -------------------------------- ### Train and predict with YOLOv5 image classifier Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Perform training, validation, and prediction tasks using the YOLOv5 image classifier. This includes commands for training a classification model with specified image size, data, weights, and epochs, as well as predicting on new images. ```bash $ yolov5 classify train --img 640 --data mnist2560 --weights yolov5s-cls.pt --epochs 1 ``` ```bash $ yolov5 classify predict --img 640 --weights yolov5s-cls.pt --source images/ ``` -------------------------------- ### Configure YOLOv5 to Use Comet Artifact Dataset Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This YAML snippet shows how to configure a dataset YAML file to point to a dataset stored as a Comet Artifact. The `path` variable should be set to the Artifact resource URL, including the workspace name, artifact name, and version/alias. ```yaml # contents of artifact.yaml file path: "comet:///:" ``` -------------------------------- ### Train YOLOv5 and upload weights/datasets to AWS S3 Source: https://github.com/fcakyon/yolov5-pip/blob/main/README.md Automatically upload trained weights and datasets to AWS S3, with optional Neptune.AI artifact tracking integration. Requires AWS access keys to be set as environment variables and can include S3 data directory in `data.yaml`. ```bash export AWS_ACCESS_KEY_ID=YOUR_KEY export AWS_SECRET_ACCESS_KEY=YOUR_KEY ``` ```bash $ yolov5 train --data data.yaml --weights yolov5s.pt --s3_upload_dir YOUR_S3_FOLDER_DIRECTORY --upload_dataset ``` ```yaml # data.yml train_json_path: "train.json" train_image_dir: "train_image_dir/" val_json_path: "val.json" val_image_dir: "val_image_dir/" yolo_s3_data_dir: s3://bucket_name/data_dir/ ``` -------------------------------- ### Log Class-Level Metrics to Comet ML for YOLOv5 Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This command enables logging of class-level metrics such as mAP, precision, recall, and F1 score for each class to Comet ML. This is achieved by setting the `COMET_LOG_PER_CLASS_METRICS` environment variable to `true`. ```shell env COMET_LOG_PER_CLASS_METRICS=true python train.py \ --img 640 \ --batch 16 \ --epochs 5 \ --data coco128.yaml \ --weights yolov5s.pt ``` -------------------------------- ### Log YOLOv5 Model Predictions to Comet ML Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This command logs YOLOv5 model predictions, including images, ground truth labels, and bounding boxes, to Comet ML. The `bbox_interval` flag controls the frequency of logged predictions, set here to every 2nd batch per epoch. Note that the default YOLOv5 validation dataloader batch size is 32. ```shell python train.py \ --img 640 \ --batch 16 \ --epochs 5 \ --data coco128.yaml \ --weights yolov5s.pt \ --bbox_interval 2 ``` -------------------------------- ### Resume Interrupted YOLOv5 Training Run with Comet ML Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This command allows resuming an interrupted YOLOv5 training run using Comet ML. By providing the Comet Run Path (e.g., `comet:////`), the run restores its state, including model checkpoints, hyperparameters, and dataset artifacts, and continues logging to the existing experiment. ```shell python train.py \ --resume "comet://" ``` -------------------------------- ### Upload YOLOv5 Dataset to Comet Artifacts Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This command uploads a YOLOv5 dataset to Comet Artifacts for storage and versioning. The dataset must be organized according to YOLOv5 documentation, and its configuration YAML file must follow the `coco128.yaml` format. The `--upload_dataset` flag triggers the upload. ```shell python train.py \ --img 640 \ --batch 16 \ --epochs 5 \ --data coco128.yaml \ --weights yolov5s.pt \ --upload_dataset ``` -------------------------------- ### Run Comet Optimizer Sweep in Parallel with Multiple Workers Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This command utilizes the `comet optimizer` CLI to run a hyperparameter sweep across multiple parallel workers. It requires specifying the number of workers and the HPO script along with its configuration file. This significantly speeds up the optimization process by distributing the workload. ```shell comet optimizer -j utils/loggers/comet/hpo.py \ utils/loggers/comet/optimizer_config.json" ``` -------------------------------- ### Control Number of Prediction Images Logged to Comet ML Source: https://github.com/fcakyon/yolov5-pip/blob/main/yolov5/utils/loggers/comet/README.md This command sets the maximum number of validation images logged to Comet ML when logging predictions from YOLOv5. The `COMET_MAX_IMAGE_UPLOADS` environment variable is used to specify the limit, here set to 200 images. ```shell env COMET_MAX_IMAGE_UPLOADS=200 python train.py \ --img 640 \ --batch 16 \ --epochs 5 \ --data coco128.yaml \ --weights yolov5s.pt \ --bbox_interval 1 ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.