### Install 3LC and 3LC Examples Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/2-modify-tables/1-weight-coreset.html Installs the necessary 3LC libraries and example code from GitHub. This is a prerequisite for running the subsequent code examples. ```bash %pip install 3lc %pip install git+https://github.com/3lc-ai/3lc-examples.git ``` -------------------------------- ### Install Dependencies for 3LC Examples Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/create-semantic-segmentation-table.html Installs necessary libraries for running 3LC examples, including the 3LC library itself, example-specific repositories, and Hugging Face utilities. ```python %pip install 3lc %pip install git+https://github.com/3lc-ai/3lc-examples.git %pip install huggingface-hub ``` -------------------------------- ### Install 3LC and Hugging Face dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/huggingface-coco128-segformer-collection.html Installs the necessary 3LC packages and example tools required to run the segmentation workflow. ```bash %pip install 3lc[huggingface] %pip install git+https://github.com/3lc-ai/3lc-examples.git ``` -------------------------------- ### Install 3LC Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/2-modify-tables/add-instance-embeddings/1-train-crop-model.html Installs the necessary 3LC libraries, example tools, and the timm library for model architectures. ```bash %pip install 3lc[pacmap] %pip install git+https://github.com/3lc-ai/3lc-examples.git %pip install timm ``` -------------------------------- ### Install 3LC and dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/pytorch-cifar10-collect-metrics.html Installs the required 3LC library, timm for model architectures, and example utilities via pip. ```bash %pip install 3lc %pip install timm %pip install git+https://github.com/3lc-ai/3lc-examples.git ``` -------------------------------- ### Install 3LC and Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/2-modify-tables/split-tables.html Installs the 3LC library, the 3LC examples repository, and Seaborn for data visualization. These are essential dependencies for running the provided examples. ```python %pip install 3lc %pip install git+https://github.com/3lc-ai/3lc-examples.git %pip install seaborn ``` -------------------------------- ### Start 3LC Object Service Source: https://docs.3lc.ai/3lc/latest/getting-started/index.html Launch the 3LC Object Service. Options include running with a text-based UI, disabling it, or disabling public examples. ```bash 3lc service ``` ```bash 3lc service --no-tui ``` ```bash 3lc service --no-public-examples ``` -------------------------------- ### Install 3LC and Hugging Face dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/huggingface-ade20k-segformer-finetuning.html Installs the required 3LC library with Hugging Face support and the necessary example utilities via pip. ```bash %pip install 3lc[huggingface] "transformers<=4.56.0" %pip install git+https://github.com/3lc-ai/3lc-examples.git ``` -------------------------------- ### Initialize 3LC Project and Install Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/keypoints/create-custom-keypoints-table.html Sets up the environment variables for the 3LC project and installs necessary packages including the 3LC SDK and kagglehub for dataset retrieval. ```python PROJECT_NAME = "3LC Tutorials - 2D Keypoints" DATASET_NAME = "AnimalPose" TABLE_NAME = "initial" DOWNLOAD_PATH = "../../transient_data" %pip install 3lc %pip install kagglehub ``` -------------------------------- ### Install 3LC and SAM dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/2-modify-tables/transform-bbs-to-segs.html Installs the necessary Python packages including 3LC, the 3LC examples repository, the Segment Anything Model, and matplotlib for visualization. ```bash %pip install 3lc %pip install git+https://github.com/3lc-ai/3lc-examples.git %pip install git+https://github.com/facebookresearch/segment-anything %pip install matplotlib ``` -------------------------------- ### Install 3LC Library Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/create-fhibe-table.html Installs the 3LC library using pip. This is a prerequisite for running the subsequent code in the notebook. ```bash %pip install -q 3lc ``` -------------------------------- ### Setup and Install 3LC Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/instance-segmentation/create-table-from-coco-segmentation.html Configures the environment variables for the 3LC project and installs the necessary Python package. ```python PROJECT_NAME = "3LC Tutorials - COCO128" DATASET_NAME = "COCO128" TABLE_NAME = "initial-segmentation" DATA_PATH = "../../../data" %pip install 3lc ``` -------------------------------- ### Install 3LC Library Source: https://docs.3lc.ai/3lc/latest/getting-started/deployment-examples/remote-training-local-service-s3.html Installs the 3LC Python library using pip. This is the first step for both training VMs and team users. ```bash pip install 3lc ``` -------------------------------- ### Configure Project Environment and Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/instance-segmentation/create-custom-instance-segmentation-table.html Initializes project constants and installs necessary libraries including 3LC, matplotlib, and kaggle API tools. This setup is required to manage data paths and enable 3LC functionality. ```python PROJECT_NAME = "3LC Tutorials - Cell Segmentation" DATASET_NAME = "Sartorius Cell Segmentation" TABLE_NAME = "initial" DOWNLOAD_PATH = "../../../transient_data" %pip install -q 3lc %pip install -q matplotlib %pip install -q kaggle %pip install -q git+https://github.com/3lc-ai/3lc-examples.git ``` -------------------------------- ### Install Required Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/lightning-balloons-segformer-training.html Installs the necessary 3LC, PyTorch Lightning, and visualization libraries via pip. ```bash %pip install 3lc[huggingface] %pip install pytorch-lightning %pip install matplotlib ``` -------------------------------- ### Configure 3LC Project and Install Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/pytorch-augmentation-explorer.html Sets up the environment variables for the 3LC project and installs the necessary package via pip. ```python PROJECT_NAME = "3LC Tutorials - Augmentation Explorer" DATASET_NAME = "COCO128" TABLE_NAME = "images-only" RUN_NAME = "register-augmented-samples" RUN_DESCRIPTION = "Inspecting augmentations on COCO-128" DATA_PATH = "../../data" BATCH_SIZE = 32 EPOCHS = 10 INSTALL_DEPENDENCIES = True if INSTALL_DEPENDENCIES: %pip install 3lc ``` -------------------------------- ### Initialize Project and Install Dependencies for 3LC Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/obbs/create-table-from-yolo-obb.html Sets up the project configuration variables and installs the required 3LC library. This is the prerequisite step for interacting with 3LC tables. ```python PROJECT_NAME = "3LC Tutorials - YOLO OBB" DATASET_NAME = "YOLO-OBB-Dataset" TABLE_NAME = "initial-obb" DATA_PATH = "../../../data" %pip install 3lc import tlc ``` -------------------------------- ### Setup and Train YOLO Pose Estimation Model Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/ultralytics-animalpose-yolo11-pose-training.html Configures the project environment, installs necessary 3LC and Ultralytics dependencies, performs stratified data splitting on keypoint tables, and executes the training process. ```python PROJECT_NAME = "3LC Tutorials - 2D Keypoints" RUN_NAME = "train-yolon-animalpose" RUN_DESCRIPTION = "Train yolo11n-pose on the AnimalPose dataset" DATASET_NAME = "AnimalPose" TABLE_NAME = "initial" NUM_WORKERS = 0 EPOCHS = 10 %pip install 3lc-ultralytics %pip install git+https://github.com/3lc-ai/3lc-examples.git import tlc from tlc_ultralytics import YOLO, Settings from tlc_tools.split import split_table initial_table = tlc.Table.from_names(TABLE_NAME, DATASET_NAME, PROJECT_NAME) def split_by(table_row): return table_row["keypoints_2d"]["instances_additional_data"]["label"][0] train_val_test = split_table( initial_table, splits={"train": 0.8, "val_test": 0.2}, split_strategy="stratified", split_by=split_by, random_seed=42, shuffle=False, ) model = YOLO("yolo11n-pose.pt") settings = Settings( project_name=PROJECT_NAME, run_name=RUN_NAME, run_description=RUN_DESCRIPTION, collect_loss=True, image_embeddings_dim=2, ) model.train( tables={"train": train_val_test["train"], "val": train_val_test["val_test"]}, epochs=EPOCHS, workers=NUM_WORKERS, ) ``` -------------------------------- ### Install Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/obbs/create-custom-obb-table.html Installs the necessary Python packages, `gdown` for downloading the dataset and `3lc` for data manipulation and model training. ```python %pip install -q gdown %pip install -q 3lc ``` -------------------------------- ### Install 3LC with NGrok Support Source: https://docs.3lc.ai/3lc/latest/python-package/object-service/object-service-deployment-guide.html Installs the 3LC Python package including the necessary dependencies for NGrok integration. ```bash pip install 3lc[pyngrok] ``` -------------------------------- ### Install 3LC Package Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/3d/write-mammoth-table.html Installs the 3LC package using pip. This is a prerequisite for using 3LC functionalities in the notebook. ```python if INSTALL_DEPENDENCIES: %pip --quiet install 3lc ``` -------------------------------- ### Configure 3LC Project and Environment Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/huggingface-cifar100-collect-embeddings.html Sets up the project metadata, model parameters, and installs necessary dependencies including 3LC, UMAP, and Hugging Face transformers. ```python PROJECT_NAME = "3LC Tutorials - CIFAR-100" RUN_NAME = "Collect Image Embeddings" DESCRIPTION = "Collect image embeddings from ViT model on CIFAR-100" DEVICE = None TRAIN_DATASET_NAME = "hf-cifar-100-train" TEST_DATASET_NAME = "hf-cifar-100-test" MODEL = "google/vit-base-patch16-224" BATCH_SIZE = 32 NUM_WORKERS = 4 INSTALL_DEPENDENCIES = True if INSTALL_DEPENDENCIES: %pip -q install 3lc[umap,huggingface] "transformers<=4.56.0" ``` -------------------------------- ### Install SuperGradients and 3LC Source: https://docs.3lc.ai/3lc/latest/python-package/integrations/supergradients/supergradients.html Installs the SuperGradients library and the 3LC library. It addresses potential version conflicts with the 'termcolor' package by recommending separate installation steps. ```bash pip install super-gradients pip install 3lc ``` -------------------------------- ### Install Dependencies for 3LC and Detectron2 Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/detectron2-balloons-detection-finetuning.html Installs the necessary libraries including PyTorch, Detectron2, and the 3LC SDK. Note that specific versions are required based on the system architecture and CUDA version. ```python if INSTALL_DEPENDENCIES: %pip install torch==1.10.1+cu111 torchvision==0.11.2+cu111 -f https://download.pytorch.org/whl/cu111/torch_stable.html %pip install detectron2 -f "https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.10/index.html" %pip install 3lc %pip install opencv-python %pip install matplotlib %pip install numpy==1.24.4 ``` -------------------------------- ### Install Dependencies for Detectron2 and 3LC Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/detectron2-coco128-detection-collection.html Installs the required versions of PyTorch, Detectron2, 3LC, and utility libraries for the environment. ```bash %pip install --force-reinstall torch==1.10.1+cu111 torchvision==0.11.2+cu111 -f https://download.pytorch.org/whl/cu111/torch_stable.html %pip install detectron2 -f "https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.10/index.html" %pip install 3lc[pacmap] %pip install opencv-python %pip install matplotlib %pip install numpy==1.24.4 ``` -------------------------------- ### Configure Project and Install Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/keypoints/create-table-from-coco-keypoints.html Sets up the project environment variables and installs the required 3LC Python package to enable data table creation. ```python PROJECT_NAME = "3LC Tutorials - COCO128" DATASET_NAME = "COCO128" TABLE_NAME = "initial-keypoints" DATA_PATH = "../../../data" %pip install 3lc ``` -------------------------------- ### Install Required Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/detectron2-coco128-segmentation-collection.html Installs the necessary libraries including PyTorch, Detectron2, 3LC, and utility packages like OpenCV and Matplotlib to support the metrics collection pipeline. ```python if INSTALL_DEPENDENCIES: %pip install torch==1.10.1+cu111 torchvision==0.11.2+cu111 -f https://download.pytorch.org/whl/cu111/torch_stable.html %pip install detectron2 -f "https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.10/index.html" %pip install 3lc[pacmap] %pip install opencv-python %pip install matplotlib %pip install numpy==1.24.4 ``` -------------------------------- ### Install dependencies for 3LC and SuperGradients Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/supergradients-animalpose-yolonas-training.html Installs the necessary Python packages including 3LC, super-gradients, and specific versions of dependencies required for the pose estimation workflow. ```bash %pip install 3lc %pip install super-gradients %pip install termcolor==3.1.0 %pip install git+https://github.com/3lc-ai/3lc-examples.git ``` -------------------------------- ### Install dependencies for 3LC segmentation workflows Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/instance-segmentation/create-instance-segmentations-from-image-masks.html Installs the necessary Python packages including the 3LC SDK, Hugging Face Hub, and utility libraries required for data processing and visualization. ```bash %pip install 3lc %pip install huggingface-hub %pip install git+https://github.com/3lc-ai/3lc-examples.git %pip install matplotlib ``` -------------------------------- ### Install Detectron2 and 3LC Dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/detectron2-balloons-segmentation-finetuning.html Installs necessary libraries including PyTorch with CUDA support, Detectron2, the 3LC library, OpenCV, Matplotlib, and NumPy. This step ensures all required tools are available for the training process. ```python %pip install torch==1.10.1+cu111 torchvision==0.11.2+cu111 -f https://download.pytorch.org/whl/cu111/torch_stable.html %pip install detectron2 -f "https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.10/index.html" %pip install 3lc %pip install opencv-python %pip install matplotlib %pip install numpy==1.24.4 ``` -------------------------------- ### Initialize and Use the Url Class Source: https://docs.3lc.ai/3lc/latest/python-package/apidocs/tlcurl/tlcurl.url.html Demonstrates how to instantiate the Url class with various schemes and paths. It shows how the class handles automatic scheme detection and alias expansion. ```python file_url = Url("/path/to/file") assert file_url.scheme == Scheme.FILE s3_url = Url("s3://bucket/path/to/object") assert s3_url.scheme == Scheme.S3 # Aliases are expanded when the URL is used alias_url = Url("/data.csv") UrlAliasRegistry.instance().register_url_alias(token="", path="/path/to/data") ``` -------------------------------- ### Initialize and Access OptionLoader Instance Source: https://docs.3lc.ai/3lc/latest/python-package/apidocs/tlcconfig/tlcconfig.option_loader.html Demonstrates how to instantiate the OptionLoader and retrieve the global singleton instance for the application. ```python from tlc.option_loader import OptionLoader # Initialize a new loader loader = OptionLoader(config_file="config.yaml", initialize=True) # Access the global singleton instance instance = OptionLoader.instance() ``` -------------------------------- ### Create 3LC Image Tables using various schemas Source: https://docs.3lc.ai/3lc/latest/user-guide/column-types/computer-vision/image.html Demonstrates how to initialize a 3LC table with image data. Includes examples using built-in URL schemas, explicit schema definitions, and direct PIL Image object integration. ```python import tlc # Using Builtin schema table = tlc.Table.from_dict( data={"image": ["path/to/image0.png", "path/to/image1.png", ...]}, structure={"image": tlc.ImageUrlSchema()}, ) ``` ```python import tlc # Using explicit Schema definition table = tlc.Table.from_dict( data={"image": ["path/to/image0.png", "path/to/image1.png", ...]}, structure={"image": tlc.Schema(value=tlc.StringValue(string_role=tlc.STRING_ROLE_IMAGE_URL))}, ) ``` ```python import tlc from PIL import Image # Using PIL Image objects table = tlc.Table.from_dict( data={"image": [Image.open("path/to/image0.png"), Image.open("path/to/image1.png"), ...]}, structure={"image": tlc.PILImage}, ) ``` -------------------------------- ### Run Training with Main.py Script Source: https://docs.3lc.ai/3lc/latest/python-package/integrations/supergradients/supergradients.html This command demonstrates how to initiate the training process using the `main.py` script with a specified YAML recipe configuration file. ```bash python main.py --config-name=my_recipe.yaml ``` -------------------------------- ### Install 3LC and dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/2-modify-tables/add-embeddings.html Installs the necessary Python packages including 3LC with PaCMAP support, example tools, and the Hugging Face transformers library. ```bash %pip install 3lc[pacmap] %pip install git+https://github.com/3lc-ai/3lc-examples.git %pip install transformers ``` -------------------------------- ### Get URL Extension Source: https://docs.3lc.ai/3lc/latest/python-package/apidocs/tlcurl/tlcurl.url.html Retrieves the file extension from the URL. For example, 'example.json' would return '.json'. ```python property extension: str """ Get the extension of the URL. Example: ``` Url("example.json").extension == ".json" ``` Returns: The extension of the URL. """ pass ``` -------------------------------- ### Install 3LC with UMAP and PACMAP dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/2-modify-tables/dimensionality-reduction-toy-example.html Installs the 3LC library with necessary dependencies for UMAP and PACMAP dimensionality reduction algorithms. This command ensures that all required packages are available for the subsequent code examples. ```bash %pip install "3lc[umap,pacmap]" ``` -------------------------------- ### Configuration: System Options Source: https://docs.3lc.ai/3lc/latest/python-package/apidocs/tlcconfig/tlcconfig.options.html General system configuration options like display progress and logging. ```APIDOC ## Configuration: System Options ### Description Settings for controlling system behavior such as progress bar visibility and log file paths. ### Keys - `tlc.display-progress` (int) - 0 or 1, toggles progress bars. - `logfile` (string) - Path to the system log file. ### Request Example { "tlc.display-progress": 1, "logfile": "/var/log/3lc/system.log" } ``` -------------------------------- ### Get URL Name Source: https://docs.3lc.ai/3lc/latest/python-package/apidocs/tlcurl/tlcurl.url.html Retrieves the name component of the URL. This could be a filename or a directory name. For example, 'C:/folder/file.txt' returns 'file.txt', and 'C:/folder' returns 'folder'. ```python property name: str """ Get the name of the URL. Example: ``` Url("C:/folder/file.txt").name == "file.txt" Url("C:/folder").name == "folder" ``` Returns: The name of the URL. """ pass ``` -------------------------------- ### Register MetricsCollectionHook in Detectron2 Trainer Source: https://docs.3lc.ai/3lc/latest/python-package/apidocs/tlc/tlc.integration.detectron2.metrics_collection_hook.html Demonstrates how to instantiate and register the MetricsCollectionHook within a Detectron2 training workflow. This setup enables automated metric collection on a specified dataset starting at a defined iteration. ```python test_hook = MetricsCollectionHook( dataset_name="my_test_dataset", metrics_collectors=MyMetricsCollector(), collection_start_iteration=1000, collection_frequency=100, ) trainer.register_hooks([train_hook, test_hook]) trainer.train() ``` -------------------------------- ### Manage Configuration Values Source: https://docs.3lc.ai/3lc/latest/python-package/apidocs/tlcconfig/tlcconfig.option_loader.html Shows how to retrieve and update configuration options programmatically using the OptionLoader instance. ```python loader = OptionLoader.instance() # Get a specific configuration value value = loader.get_value(MyOptionClass) # Set a configuration value loader.set_value(MyOptionClass, "new_value") # Validate the current configuration state loader.validate() ``` -------------------------------- ### Export Configuration to YAML Source: https://docs.3lc.ai/3lc/latest/python-package/apidocs/tlcconfig/tlcconfig.option_loader.html Provides examples for serializing the current configuration state into YAML format for storage or inspection. ```python loader = OptionLoader.instance() # Get YAML string representation yaml_str = loader.to_yaml(doc_strings=True, detail=True) # Write current configuration to a file loader.write_to_yaml_file("settings.yaml", overwrite=True) ``` -------------------------------- ### Install 3LC Library (Shell) Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/create-table-from-image-folder.html This command installs the 3LC library using pip. Ensure you have pip installed and configured correctly in your environment before running this command. ```shell %pip install 3lc ``` -------------------------------- ### Launch 3LC Object Service with NGrok Source: https://docs.3lc.ai/3lc/latest/python-package/object-service/object-service-deployment-guide.html Starts the 3LC Object Service with the --ngrok flag to automatically generate a public URL for remote access. ```bash 3lc service --ngrok ``` -------------------------------- ### Example ID Schema Source: https://docs.3lc.ai/3lc/latest/python-package/apidocs/tlc/tlc.core.builtins.schemas.schemas.html Schema for representing unique example identifiers. ```APIDOC ## ExampleIdSchema ### Description A schema for example ID values. Example ID is a unique identifier for an example. It is used to identify examples across different tables. ### Method N/A (Schema Definition) ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **display_name** (str) - Default: 'Example ID' * **description** (str) - Default: '' * **writable** (bool) - Default: False * **computable** (bool) - Default: False ### Request Example ```json { "display_name": "Example ID", "description": "", "writable": false, "computable": false } ``` ### Response #### Success Response (200) * **Schema Definition** - Represents an example ID. #### Response Example ```json { "example": "ExampleIdSchema" } ``` ``` -------------------------------- ### Initialize 3LC Training Environment Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/2-modify-tables/add-instance-embeddings/1-train-crop-model.html Imports required modules, sets configuration constants, and detects the available hardware device for training. ```python import tlc from tlc_tools.augment_bbs.finetune_on_crops import train_model from tlc_tools.common import infer_torch_device from tlc_tools.split import split_table EPOCHS = 10 TMP_PATH = "../../../transient_data" MODEL_NAME = "efficientnet_b0" NUM_WORKERS = 0 MODEL_CHECKPOINT = TMP_PATH + "/instance_classifier.pth" DEVICE = infer_torch_device() print(f"Using device: {DEVICE}") ``` -------------------------------- ### Setup Detectron2 Project Configuration Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/detectron2-balloons-segmentation-finetuning.html Defines project-specific configurations for fine-tuning a Detectron2 instance segmentation model, including dataset names, paths, model configuration, and training parameters. These variables are essential for setting up the training environment. ```python PROJECT_NAME = "3LC Tutorials - Balloons" RUN_NAME = "Fine-tune segmentation model" DESCRIPTION = "Train a balloon segmenter using detectron2" TRAIN_DATASET_NAME = "balloons-train-seg" VAL_DATASET_NAME = "balloons-val-seg" TMP_PATH = "../transient_data" DATA_PATH = "../../data" MODEL_CONFIG = "COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml" MAX_ITERS = 200 BATCH_SIZE = 2 MAX_DETECTIONS_PER_IMAGE = 30 SCORE_THRESH_TEST = 0.5 MASK_FORMAT = "bitmask" ``` -------------------------------- ### Create Table and Run URLs Source: https://docs.3lc.ai/3lc/latest/python-package/urls.html Demonstrates how to generate standard URLs for tables and runs using the tlc.Url utility methods. These methods require parameters like table/run names, dataset names, and project names to construct the correct path. ```python import tlc table_url = tlc.Url.create_table_url(table_name, dataset_name, project_name) run_url = tlc.Url.create_run_url(run_name, project_name) ``` -------------------------------- ### Install 3LC dependency Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/create-table-from-torch-dataset.html Installs the 3LC library using pip to enable dataset conversion functionality. ```bash %pip install -q 3lc ``` -------------------------------- ### Initialize 3LC run and configure Detectron2 Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/detectron2-balloons-detection-finetuning.html Initializes a 3LC tracking run and sets up the Detectron2 configuration object with model parameters and solver settings. ```python run = tlc.init( project_name=PROJECT_NAME, run_name=RUN_NAME, description=DESCRIPTION, if_exists="overwrite", ) cfg = get_cfg() cfg.merge_from_file(model_zoo.get_config_file(MODEL_CONFIG)) cfg.DATASETS.TRAIN = (TRAIN_DATASET_NAME,) cfg.DATASETS.TEST = (VAL_DATASET_NAME,) cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(MODEL_CONFIG) cfg.SOLVER.BASE_LR = 0.00025 cfg.MODEL.DEVICE = "cuda" os.makedirs(cfg.OUTPUT_DIR, exist_ok=True) ``` -------------------------------- ### Initialize 3LC run and configure Detectron2 Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/detectron2-coco128-detection-collection.html Initializes a 3LC run and sets up the Detectron2 configuration object with model parameters and training settings. ```python run = tlc.init( PROJECT_NAME, run_name=RUN_NAME, description=DESCRIPTION, if_exists="overwrite", ) cfg = get_cfg() cfg.merge_from_file(model_zoo.get_config_file(MODEL_CONFIG)) cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(MODEL_CONFIG) cfg.DATASETS.TRAIN = (TRAIN_DATASET_NAME,) cfg.OUTPUT_DIR = TMP_PATH cfg.DATALOADER.NUM_WORKERS = 0 cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 512 cfg.MODEL.ROI_HEADS.NUM_CLASSES = 80 cfg.TEST.DETECTIONS_PER_IMAGE = MAX_DETECTIONS_PER_IMAGE cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = SCORE_THRESH_TEST cfg.MODEL.DEVICE = "cuda" cfg.DATALOADER.FILTER_EMPTY_ANNOTATIONS = False config = { "model_config": MODEL_CONFIG, "test.detections_per_image": MAX_DETECTIONS_PER_IMAGE, "model.roi_heads.score_thresh_test": SCORE_THRESH_TEST, } run.set_parameters(config) ``` -------------------------------- ### Configure and Initialize 3LC Training Environment Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/3-training-and-metrics/huggingface-mrcp-bert-finetuning.html Sets up global configuration variables, installs necessary dependencies, and initializes a 3LC run to track the fine-tuning process. ```python PROJECT_NAME = "3LC Tutorials - Hugging Face BERT" RUN_NAME = "finetuning-run" DESCRIPTION = "Fine-tune BERT on MRPC" TRAIN_DATASET_NAME = "hugging-face-train" VAL_DATASET_NAME = "hugging-face-val" CHECKPOINT = "bert-base-uncased" # Install dependencies %pip install accelerate scikit-learn 3lc[huggingface] "transformers<=4.56.0" # Initialize 3LC Run import tlc run = tlc.init( project_name=PROJECT_NAME, run_name=RUN_NAME, description=DESCRIPTION, if_exists="overwrite" ) ``` -------------------------------- ### Convert starts and lengths to COCO RLE Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/instance-segmentation/create-custom-instance-segmentation-table.html Converts a list of starts and lengths into a binary mask and encodes it into the COCO RLE format. ```python def starts_lengths_to_coco_rle(starts_lengths, image_height, image_width): s = np.array(starts_lengths, dtype=int) starts = s[0::2] - 1 lengths = s[1::2] mask = np.zeros(image_height * image_width, dtype=np.uint8) for start, length in zip(starts, lengths): mask[start : start + length] = 1 mask = mask.reshape(image_height, image_width) rle = mask_utils.encode(np.asfortranarray(mask)) return rle["counts"].decode("utf-8") ``` -------------------------------- ### Install 3LC and dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/2-modify-tables/add-instance-embeddings/2-add-instance-metrics.html Installs the required 3LC packages, including PaCMAP for dimensionality reduction, along with utility libraries and timm for model support. ```bash %pip install -q 3lc[pacmap] %pip install -q git+https://github.com/3lc-ai/3lc-examples.git %pip install -q timm ``` -------------------------------- ### Initialize 3LC Integration for Recipes Source: https://docs.3lc.ai/3lc/latest/python-package/integrations/supergradients/supergradients.html Demonstrates how to import the 3LC integration module in a main entrypoint script to register datasets and callbacks for use in SuperGradients YAML recipes. ```python import omegaconf import hydra from super_gradients import Trainer, init_trainer ``` -------------------------------- ### Install 3LC and dependencies Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/2-modify-tables/add-classification-metrics.html Installs the necessary Python packages including 3LC, transformers, and dimensionality reduction tools required for metric computation. ```bash %pip install 3lc %pip install git+https://github.com/3lc-ai/3lc-examples.git %pip install transformers %pip install pacmap %pip install joblib ``` -------------------------------- ### Install 3LC Library Source: https://docs.3lc.ai/3lc/latest/examples/tutorials/1-create-tables/create-custom-table.html Installs the 3LC library, which is a dependency for creating and managing custom tables and datasets. This is typically the first step before utilizing any 3LC functionalities. ```bash %pip install 3lc ```