### Example of a different facility setup

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/11-performing-network-analyses/part6_solve_location_allocation.ipynb

Demonstrates adding facilities with different attribute names for capacity andేవ.

```python
facilities_alt = {"features": [{"geometry": {"x": -119.67, "y": 36.74}, "attributes": {"MaxCapacity": 100, "ేవ": 1}}], "spatialReference": {"wkid": 4326}}
la_layer.add_facilities(facilities_alt, "MaxCapacity", "ేవ")
```

--------------------------------

### Example of a different demand point setup

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/11-performing-network-analyses/part6_solve_location_allocation.ipynb

Demonstrates adding demand points with a different attribute name for demand.

```python
demand_points_alt = {"features": [{"geometry": {"x": -119.68, "y": 36.75}, "attributes": {"ServiceDemand": 10}}], "spatialReference": {"wkid": 4326}}
la_layer.add_demand_points(demand_points_alt, "ServiceDemand")
```

--------------------------------

### Install Libraries

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/04_gis_analysts_data_scientists/detect_super_blooms_using_satellite_image_classification.ipynb

Installs necessary libraries for the project. Run this command in your environment.

```bash
pip install -q arcgis
pip install -q matplotlib
pip install -q pandas
pip install -q scikit-learn
pip install -q seaborn
pip install -q tensorflow
pip install -q tensorflow_datasets
pip install -q Pillow
pip install -q opencv-python
pip install -q rasterio
pip install -q geopandas
pip install -q pyarrow
pip install -q pygeos
pip install -q shapely
pip install -q scikit-image
pip install -q tqdm
```

--------------------------------

### Install MaXDeepLab

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/panoptic_segmentation_with_maxdeeplab.ipynb

Install the MaXDeepLab package to enable panoptic segmentation capabilities. This is a prerequisite for running the subsequent examples.

```python
from arcgis.gis import GIS
from arcgis.learn import MaXDeepLab

# Connect to your ArcGIS Enterprise portal or ArcGIS Online
gis = GIS('home')

# Download and install MaXDeepLab
MaXDeepLab.install_maxdeeplab()
```

--------------------------------

### Basic Network Analysis Setup

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/11-performing-network-analyses/part1_introduction_to_network_analysis.ipynb

Demonstrates the initial steps to set up and access a network dataset for analysis. Ensure the necessary libraries are imported.

```python
from arcgis.gis import GIS
gis = GIS('home')
from arcgis.networkanalyst import NetworkAnalyst
nxanalyst = NetworkAnalyst(gis)
```

--------------------------------

### Get Available Geocoders

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/09-finding-places-with-geocoding/part1_what_is_geocoding.ipynb

Retrieve a list of registered geocoders with the GIS. This example shows how to get the first geocoder, which typically uses the Esri World Geocoding Service.

```python
from arcgis.geocoding import Geocoder, get_geocoders

my_geocoder = get_geocoders(gis)[0]
my_geocoder
```

--------------------------------

### Configure and Start Finetuning

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Sets up the finetuning configuration and initiates the training process. This includes defining optimizer, learning rate, and number of epochs.

```python
from sam_lora import SAM_LoRA
from torch.optim import Adam

sam_lora = SAM_LoRA(model_path="sam_vit_heavy_4k.pth", lora_path="sam_lora_vit_heavy_4k.pth")
optimizer = Adam(sam_lora.parameters(), lr=1e-4)
sam_lora.finetune(dataloader=dataloader, optimizer=optimizer, epochs=10)
```

--------------------------------

### Configure and start finetuning

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Sets up the training configuration and initiates the finetuning process. This includes defining optimizer, learning rate, and number of epochs.

```python
from samlora import Trainer

trainer = Trainer(
    lora_model,
    dataloader,
    optimizer_cls='AdamW',
    learning_rate=1e-4,
    num_epochs=10,
    batch_size=2,
    save_dir="output"
)
trainer.train()
```

--------------------------------

### Example REST API Search Query (Fiddler)

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/03-the-gis/accessing-and-creating-content.ipynb

This is an example of a REST API query that might be observed using a tool like Fiddler when searching for feature services. Note the parameters like 'q', 'start', 'num', and 'token'.

```python
https://geosaurus.maps.arcgis.com/sharing/rest/search?
    f=json
    &q=owner:arcgis_python AND (type:"feature service") accountid:xxxxxxxxxxxxxxxx
    &start=1
    &num=100
    &sortFields=avgRating
    &sortOrder=desc
    &token=<my_web_gis_token>   
```

--------------------------------

### Create Trainer and Start Training

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Instantiate the `Trainer` with the model, training arguments, and dataset. Then, start the finetuning process.

```python
from transformers import Trainer

trainer = Trainer(
    model=model,
    tokenizer=processor,
    args=training_args,
    train_dataset=train_dataset, # Assuming train_dataset is already created
    eval_dataset=val_dataset,   # Assuming val_dataset is already created
    data_collator=lambda data: data[0] # SamLoRA uses a custom data collator
)

trainer.train()
```

--------------------------------

### Create Trainer and Start Finetuning

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Instantiate the Trainer with the SamLoRA model, training arguments, and dataset. Then, start the finetuning process.

```python
from transformers import Trainer
from segment_anything_lora import LoraTrainer

# Assuming you have your dataset prepared as 'train_dataset'
# train_dataset = CocoDataset(image_dir='path/to/images', annotations_file='path/to/annotations.json', transform=ResizeLongestSide(sam.image_size))

trainer = LoraTrainer(
    model=lora_model,
    args=training_args,
    train_dataset=train_dataset,
    # optim="adamw_torch", # Uncomment if you want to use adamw_torch
)

trainer.train()

```

--------------------------------

### Access Imagery Layer

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/how_change_detection_works.ipynb

Get a specific imagery layer item from the search results. This example uses the first result.

```python
imagery_layer = search_results[0]
print(f"Found imagery layer: {imagery_layer.title}")
```

--------------------------------

### Activate Conda Environment

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/01-getting-started/understanding-conda.ipynb

Activate a Conda environment to start using its installed packages and Python interpreter. This command is specific to macOS and Linux.

```bash
source activate gis-enviro
```

--------------------------------

### Set up environment and data paths

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/point_cloud_classification_using_randlanet.ipynb

Configure the environment and specify the paths for training data, model checkpoints, and output logs.

```python
data_path = Path("../data/S3DIS/stanford_indoor3d/")
model_path = Path("../models/randlanet/")
log_path = Path("../logs/randlanet/")

# Create directories if they don't exist
model_path.mkdir(parents=True, exist_ok=True)
log_path.mkdir(parents=True, exist_ok=True)
```

--------------------------------

### Initialize Change Detection Model

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/04_gis_analysts_data_scientists/multi_class_change_detection_using_segmentation_deep_learning_models.ipynb

Initializes a deep learning model for change detection. This example uses a pre-trained model as a starting point.

```python
from arcgis.learn import ChangeDetectionModel

# Initialize the ChangeDetectionModel
# We use a pre-trained model 'unet' as the architecture
# The number of classes is inferred from the dataset
model = ChangeDetectionModel(dataset, unet)

print('Change Detection Model initialized.')
```

--------------------------------

### Configure and start training

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Configure the training arguments and start the finetuning process using the prepared dataset and the SamLoRA model.

```python
from transformers import TrainingArguments, Trainer

# Define training arguments
training_args = TrainingArguments(
    output_dir="./sam-lora-finetuned",
    per_device_train_batch_size=2,
    num_train_epochs=10,
    learning_rate=1e-4,
    fp16=True,
    logging_steps=10,
    save_steps=100,
    evaluation_strategy="steps",
    eval_steps=100,
    load_best_model_at_end=True,
)

# Initialize Trainer
trainer = Trainer(
    model=lora_model,
    args=training_args,
    train_dataset=dataset,  # Replace with your actual dataset
    eval_dataset=dataset,   # Replace with your actual dataset
)

# Start training
trainer.train()
```

--------------------------------

### Get Track Properties

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/multi_object_tracking_using_object_tracker.ipynb

Retrieve the properties associated with each track. This can include information like track ID, start time, and end time.

```python
# Get track properties
track_properties = tracker.get_track_properties()
```

--------------------------------

### Access an imagery layer

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/how_change_detection_works.ipynb

Get a specific imagery layer item from the search results. This example assumes the first result is the desired layer.

```python
imagery_layer = search_results[0].layers[0]
```

--------------------------------

### Set Device to CPU for Initial Setup

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/utilize_multiple_gpus_to_train_model.ipynb

Before distributing to multiple GPUs, set the learner's device to 'cpu'. This is a common practice to avoid potential issues during the initial setup phase.

```python
learn.dls.device = 'cpu'
```

--------------------------------

### Initialize ObjectTracker

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/multi_object_tracking_using_object_tracker.ipynb

Initializes the ObjectTracker with a specified maximum number of objects and a minimum confidence score. This setup is crucial before starting the tracking process.

```python
from arcgis.gis import GIS
from arcgis.learn import ObjectTracker

gis = GIS('home')

tracker = ObjectTracker(max_objects=100, min_confidence=0.5)
```

--------------------------------

### Set up Training Environment

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/point_cloud_object_detection_using_second.ipynb

Initializes the training environment, including the optimizer, learning rate scheduler, and data loaders, based on the configuration.

```python
from mmcv.runner import get_optimizer
from mmcv.utils import get_logger
from mmdet3d.datasets import build_dataloader

# Build data loaders	rain_dataloader = build_dataloader(
    train_dataset,
    **model_cfg.data.train_dataloader)
val_dataloader = build_dataloader(
    val_dataset,
    **model_cfg.data.val_dataloader, 
    shuffle=False)

# Build optimizer
optimizer = get_optimizer(model, model_cfg.optimizer)

# Get logger
logger = get_logger('mmdet3d')
logger.setLevel('INFO')

print('Training environment setup complete.')
```

--------------------------------

### Import Libraries and Download Data

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/05-working-with-the-spatially-enabled-dataframe/part2_data_io_reading_data.ipynb

Imports necessary libraries and downloads the required data for the guide. This is a prerequisite for running the subsequent code examples.

```python
import os
import sys

import pandas as pd
from arcgis.gis import GIS
from arcgis.features import GeoAccessor, GeoDeserializer

from IPython.display import display

# connect to your GIS
gis = GIS('home')

# download the data
item_id = 'c7140ae3d7ae4fd0817181461019aa75'
item = gis.content.get(item_id)

data_path = os.path.join(os.path.expanduser('~'), 'arcgis')

# download and extract the data
item.download(data_path)

# extract the zip file
zip_path = os.path.join(data_path, 'data.zip')

import zipfile
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
    zip_ref.extractall(data_path)

# add the data path to the system path
sys.path.append(data_path)
```

--------------------------------

### Install necessary libraries

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Install the required libraries for finetuning SAM with SamLoRA. Ensure you have a compatible environment.

```bash
pip install -U git+https://github.com/Pix2PixHD/SAM-LoRA.git
```

--------------------------------

### Initialize Object Detector

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/04_gis_analysts_data_scientists/automate_road_surface_investigation_using_deep_learning.ipynb

Initializes an ObjectDetector model. You can specify a pre-trained model or start from scratch. This example uses a pre-trained model for faster results.

```python
model = ObjectDetector(training_df, batch_size=4, epochs=5, learning_rate=0.001, patience=3, factor=0.5, monitor='val_loss')
```

--------------------------------

### Configure and start training

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Configures the training parameters, including optimizer, learning rate, and number of epochs, then starts the finetuning process.

```python
# Configure training parameters
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4)
num_epochs = 10

# Training loop
model.train()
for epoch in range(num_epochs):
    for images, masks in dataloader:
        images = images.to(model.device)
        masks = masks.to(model.device)

        optimizer.zero_grad()
        loss = model(images, masks)
        loss.backward()
        optimizer.step()

    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {loss.item():.4f}")

print("Training finished!")

```

--------------------------------

### Install Required Libraries

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Install necessary libraries for SamLoRA finetuning. Ensure you have a compatible environment.

```bash
pip install -U "git+https://github.com/facebookresearch/segment-anything.git"
```

```bash
pip install -U "git+https://github.com/ybelkada/segment-anything-lora.git"
```

--------------------------------

### Model Initialization and Training Example

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/point_cloud_classification_using_randlanet.ipynb

Initializes the RandLA-Net model, sets up the optimizer and loss function, and starts the training process. Requires a DataLoader for the dataset.

```python
import torch
import torch.nn as nn

# Assuming you have a DataLoader named 'train_loader'
# from your_dataloader_module import train_loader

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

model = PointTransformer(input_channels=3, num_classes=10).to(device) # Example: 3 input channels, 10 classes

criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# Dummy train_loader for demonstration
class DummyDataLoader:
    def __init__(self):
        self.data = [(
            torch.randn(1, 1024, 3),  # xyz
            torch.randn(1, 1024, 3),  # features
            torch.randint(0, 10, (1, 1024)) # labels
        )]
    def __iter__(self):
        return iter(self.data)

train_loader = DummyDataLoader()

# Train the model for a few epochs (example)
num_epochs = 2
for epoch in range(num_epochs):
    print(f'Epoch {epoch+1}/{num_epochs}')
    train(model, train_loader, criterion, optimizer, device)

print('Training finished.')

```

--------------------------------

### Get Track Properties

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/multi_object_tracking_using_object_tracker.ipynb

Retrieve properties of the tracks, such as start and end times, and the number of features in each track. This provides summary statistics for each identified track.

```python
# Get track properties
track_properties = tracker.get_track_properties(source_layer=source, tracked_features=tracked_features)
```

--------------------------------

### Start Finetuning

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Begin the finetuning process by calling the `train()` method on the `Trainer` object. This will train the SamLoRA adapter on your dataset.

```python
trainer.train()
```

--------------------------------

### Set up environment and import libraries

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/04_gis_analysts_data_scientists/automatic_road_extraction_using_deep_learning.ipynb

Initializes the environment and imports necessary libraries for geospatial analysis and deep learning.

```python
from arcgis.gis import GIS
from arcgis.learn import extract_roads
import os
import time
import matplotlib.pyplot as plt

# Connect to your ArcGIS Enterprise portal or ArcGIS Online
gis = GIS('home')

# Set environment variables
arcgis_output_url = os.getenv('ARCGIS_OUTPUT_URL')
arcgis_output_username = os.getenv('ARCGIS_OUTPUT_USERNAME')
arcgis_output_password = os.getenv('ARCGIS_OUTPUT_PASSWORD')

# Connect to the output portal
out_gis = GIS(arcgis_output_url, arcgis_output_username, arcgis_output_password)

print('Successfully connected to ArcGIS Enterprise.')
```

--------------------------------

### Perform Prediction with the Loaded Model

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/utilize_multiple_gpus_to_train_model.ipynb

Uses the loaded deep learning model to predict on new data. This example shows how to get predictions for a dataset.

```python
predictions = loaded_model.predict(data.test_ds)
```

--------------------------------

### Make predictions with the trained model

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/utilize_multiple_gpus_to_train_model.ipynb

Use the trained model to make predictions on new data. This example shows how to get predictions for the validation set.

```python
interp = ClassificationInterpretation.from_learner(learn)

# Get predictions for the validation set
preds, targs = learn.get_preds()

# You can also use interp.plot_top_losses(9) to visualize the top losses.
```

--------------------------------

### Main Execution Block

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/point_cloud_classification_using_sqn.ipynb

Sets up the training and testing process, including device selection, model instantiation, optimizer configuration, and iterating through epochs.

```python
def main():
    use_cuda = not False and torch.cuda.is_available()
    device = torch.device("cuda" if use_cuda else "cpu")

    model = SQN(input_channels=3, output_channels=40).to(device)
    optimizer = optim.Adam(model.parameters(), lr=0.001)

    for epoch in range(1, 10 + 1):
        train(model, device, train_loader, optimizer, epoch)
        test(model, device, test_loader)

if __name__ == '__main__':
    main()

```

--------------------------------

### Start Finetuning Process

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Execute the finetuning process using the initialized trainer. The model will be trained on the provided dataset.

```python
trainer.train(
    "sam_lora_finetuned.pth"
)
```

--------------------------------

### Get Common Time Zones with pytz

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/16-introduction-to-data-engineering-in-python/part5_time_series_analysis_with_pandas.ipynb

Retrieves the last 10 common time zones available in the pytz library. Ensure pytz is installed.

```python
import pytz
pytz.common_timezones[-10:]
```

--------------------------------

### Configure and Start Finetuning

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Sets up the training arguments and initiates the finetuning process using the Trainer class. This includes defining hyperparameters and the training dataset.

```python
from transformers import TrainingArguments, Trainer
from datasets import Dataset

# Assuming you have your dataset loaded into a Hugging Face Dataset object
# For example:
# train_dataset = MyDataset(image_dir='path/to/images', mask_dir='path/to/masks')
# train_dataset = Dataset.from_dict({'image': [...], 'mask': [...]})

# Define training arguments
training_args = TrainingArguments(
    output_dir="./sam_lora_finetuned",
    num_train_epochs=10,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=2,
    learning_rate=2e-4,
    fp16=True,
    logging_steps=50,
    save_strategy="epoch",
    report_strategy="epoch",
    # Add other relevant arguments as needed
)

# Initialize the Trainer
trainer = Trainer(
    model=sam,  # The SAM model with LoRA adapter loaded
    args=training_args,
    train_dataset=train_dataset, # Your prepared dataset
    # data_collator can be defined if needed
)

# Start finetuning
trainer.train()
```

--------------------------------

### Initialize Object Detection Model

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/04_gis_analysts_data_scientists/detecting_swimming_pools_using_satellite_image_and_deep_learning.ipynb

Initialize an object detection model using the ArcGIS API for Python. This example uses a pre-trained model as a starting point.

```python
model = object_detection(data_path='pool_dataset', 
                         images='images', 
                         annotations='annotations', 
                         model_name='fasterrcnn_resnet50_fpn', 
                         batch_size=4, 
                         epochs=10, 
                         learning_rate=0.001, 
                         train_split=0.8, 
                         save_best_model=True, 
                         monitor='val_loss', 
                         early_stopping_patience=5)

print("Model initialized and ready for training.")
```

--------------------------------

### Example Usage

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/point_cloud_classification_using_sqn.ipynb

Demonstrates a complete workflow from data loading to model evaluation.

```python
# Example usage:
filepath = 'path/to/your/point_cloud_data.csv'
X_train, X_test, y_train, y_test, scaler = load_and_preprocess_data(filepath)

input_dim = X_train.shape[1] # Number of features (x, y, z, intensity)
num_classes = len(np.unique(y_train))

model = SQN(input_dim, num_classes)
train_sqn(model, X_train, y_train)

accuracy = evaluate_sqn(model, X_test, y_test)
```

--------------------------------

### Example of doccano export format (JSON)

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/labeling-text-using-doccano.ipynb

This JSON structure represents exported data from doccano, showing text and its associated labels with start and end positions.

```json
[
  {
    "id": 1,
    "text": "This is a sample text.",
    "label": [
      [10, 16, "PERSON"]
    ]
  }
]
```

--------------------------------

### Wildcard Search for Content

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/03-the-gis/accessing-and-creating-content.ipynb

Perform wildcard searches using '*' for multiple characters or '?' for a single character in the query title. This example searches for titles starting with 'USA'.

```python
search_result_USA = gis.content.search(query="title:USA*")                  
search_result_USA
```

--------------------------------

### Start the finetuning process

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Initiate the finetuning process using the configured SamLoRA trainer. This will train the model on your prepared dataset.

```python
trainer.train()

```

--------------------------------

### Access and Query Feature Layer

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/05_content_publishers/pdf_table_to_pdf_map.ipynb

Retrieves a feature layer item from ArcGIS Online and queries it to get specific features. This example fetches all counties in California.

```python
counties_item = gis.content.get('48f9af87daa241c4b267c5931ad3b226')
counties_flayer = counties_item.layers[0]
counties_fset = counties_flayer.query("STATE_NAME='California'")
```

--------------------------------

### Main execution block

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/point_cloud_classification_using_sqn.ipynb

Sets up the dataset, dataloaders, model, loss function, and optimizer, then initiates the training and evaluation process.

```python
if __name__ == '__main__':
    # Configuration
    DATA_DIR = 'path/to/your/point_cloud_data' # Replace with your data directory
    NUM_POINTS = 1024
    K_NEIGHBORS = 20
    NUM_CLASSES = 10 # Example: number of classes in your dataset
    BATCH_SIZE = 16
    LEARNING_RATE = 0.001
    EPOCHS = 50

    # Device selection
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    print(f'Using device: {device}')

    # Create datasets and dataloaders
    train_dataset = PointCloudDataset(DATA_DIR, num_points=NUM_POINTS, k=K_NEIGHBORS, train=True, augment=True)
    test_dataset = PointCloudDataset(DATA_DIR, num_points=NUM_POINTS, k=K_NEIGHBORS, train=False, augment=False)

    train_loader = DataLoader(train_dataset, batch_size=BATCH_SIZE, shuffle=True, num_workers=4)
    test_loader = DataLoader(test_dataset, batch_size=BATCH_SIZE, shuffle=False, num_workers=4)

    # Initialize model, loss function, and optimizer
    model = SQN(num_classes=NUM_CLASSES, k=K_NEIGHBORS).to(device)
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.Adam(model.parameters(), lr=LEARNING_RATE, weight_decay=1e-4)

    # Train the model
    train_model(model, train_loader, criterion, optimizer, device, EPOCHS)

    # Evaluate the model
    evaluate_model(model, test_loader, device)
```

--------------------------------

### Prepare Dataset for Finetuning

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Load and preprocess a dataset for finetuning SAM. This example uses a dummy dataset and demonstrates how to format it for the model.

```python
from datasets import Dataset
import torch

data = {
    "pixel_values": [torch.randn(3, 1024, 1024)],
    "labels": [torch.randint(0, 2, (1024, 1024))],
}
dataset = Dataset.from_dict(data)
dataset.set_format("torch")
```

--------------------------------

### Set cloud removal parameters

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/04_gis_analysts_data_scientists/cloud_removal_using_pretrained_deep_learning_model_and_raster_function.ipynb

Modify the parameters of the 'cloud_removal' raster function. For example, setting the 'output_cloud_mask' parameter to True to also get the cloud mask.

```python
rf.output_cloud_mask = True
cloud_removed_imagery_with_mask = cloud_removed_imagery.apply_function(rf)
```

--------------------------------

### Setup Environment for Deep Learning

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/04_gis_analysts_data_scientists/automate_road_surface_investigation_using_deep_learning.ipynb

This code block is intended for setting up the necessary environment for deep learning tasks. Ensure all required libraries are installed before execution.

```python
import os
import sys

import matplotlib.pyplot as plt
import numpy as np

from IPython.display import display

from arcgis.gis import GIS
from arcgis.learn import * 
from arcgis.apps import * 
from arcgis.features import * 
from arcgis.geometry import * 
from arcgis.raster import * 
from arcgis.network import *
from arcgis.raster.functions import * 
from arcgis.mapping import WebMap

from arcgis.learn.utils import * 

# Set the environment for the notebook
from arcgis.learn.utils import _set_env
_set_env()

# Set the environment for the notebook
from arcgis.learn.utils import _set_env
_set_env()
```

--------------------------------

### Initialize and Start Finetuning

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Initialize the SamLoRA trainer with the configuration and dataset, then start the finetuning process. This will train the LoRA adapters for the SAM model.

```python
import torch
from samlora.trainer import SamLoRATrainer

# Assuming config and datasets are already defined as above
trainer = SamLoRATrainer(config)
trainer.train()

print("Finetuning complete. LoRA adapters saved to:", config.output_dir)
```

--------------------------------

### Install necessary libraries

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Install the required libraries for SamLoRA finetuning, including PyTorch, Transformers, and other dependencies.

```bash
pip install torch torchvision torchaudio
pip install transformers
pip install bitsandbytes accelerate
pip install git+https://github.com/facebookresearch/segment-anything.git
pip install git+https://github.com/ybelkada/segment-anything-lora.git
```

--------------------------------

### Loading Data for Multi-GPU Training

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/utilize_multiple_gpus_to_train_model.ipynb

Prepare your dataset for training. This example shows how to load image data using the `prepare_data` function, which is compatible with multi-GPU setups.

```python
# Load data from a folder
data = prepare_data(path='path/to/your/data', transform_mode='image')

# Display information about the dataset
data
```

--------------------------------

### Query Feature Layer for Recent Disasters

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/05_content_publishers/hey_gis_give_me_a_map_of_the_recent_natural_disasters.ipynb

Query the feature layer to retrieve recent disaster events. This example filters by a date range to get data from the last 30 days.

```python
query_result = feature_layer.query(where="1=1", time_filter="last 30 days", as_df=True)
query_result.head()
```

--------------------------------

### Prepare Data for Training (Alternative)

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/retraining_windows_doors_extraction_model.ipynb

Demonstrates an alternative way to prepare data by creating a new LPSULayer from a folder. This is often the first step in retraining.

```python
from arcgis.learn import LPSULayer

# Create a new LPSULayer from a folder
my_model = LPSULayer.from_folder("/path/to/your/training/data")
```

--------------------------------

### NumPy Binary Universal Functions

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/16-introduction-to-data-engineering-in-python/part2_introduction_to_numpy.ipynb

Illustrates the setup for applying NumPy universal functions to multiple arrays. The example shows two arrays, `arr1` and `arr2`, ready for binary operations.

```python
print('Array 1:', arr1)
print('Array 2:', arr2)
```

--------------------------------

### Basic Image Captioning Model Setup

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/how_image_captioning_works.ipynb

This snippet outlines the fundamental structure of an image captioning model, often employing an encoder-decoder architecture. It's a starting point for understanding how image features are processed and translated into text.

```python
import torch
import torch.nn as nn

class EncoderCNN(nn.Module):
    def __init__(self, embed_size):
        super(EncoderCNN, self).__init__()
        # ... CNN layers ...
        self.fc = nn.Linear(..., embed_size) # Map features to embedding size
        self.embed_size = embed_size

    def forward(self, images):
        # ... forward pass through CNN ...
        features = self.fc(features)
        return features

class DecoderRNN(nn.Module):
    def __init__(self, embed_size, hidden_size, vocab_size, num_layers):
        super(DecoderRNN, self).__init__()
        # ... RNN layers ...
        self.embedding = nn.Embedding(vocab_size, embed_size)
        self.lstm = nn.LSTM(embed_size, hidden_size, num_layers, batch_first=True)
        self.linear = nn.Linear(hidden_size, vocab_size)
        self.dropout = nn.Dropout(p=0.5)

    def forward(self, features, captions):
        # ... forward pass through decoder ...
        embeddings = self.embedding(captions)
        # Concatenate CNN features with embeddings
        inputs = torch.cat((features.unsqueeze(1), embeddings), 1)
        hiddens, _ = self.lstm(inputs)
        outputs = self.linear(hiddens)
        return outputs

class CNN2RNN(nn.Module):
    def __init__(self, embed_size, hidden_size, vocab_size, num_layers):
        super(CNN2RNN, self).__init__()
        self.encoder = EncoderCNN(embed_size)
        self.decoder = DecoderRNN(embed_size, hidden_size, vocab_size, num_layers)
        
    def forward(self, images, captions):
        features = self.encoder(images)
        outputs = self.decoder(features, captions)
        return outputs
```

--------------------------------

### Download Pretrained SamLoRA Weights

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Download the pretrained SamLoRA weights to use as a starting point for finetuning. This helps in achieving better results faster.

```bash
wget https://huggingface.co/laituan293/SamLoRA-weights/resolve/main/sam_lora_vit_large_patch16_224.pth
```

--------------------------------

### Format Prompt for Few-Shot Classification

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/leveraging_llms_for_real_time_decision_making_in_calamities.ipynb

Formats the prompt for few-shot classification by combining a system prompt, user prompt, and examples. It dynamically constructs the prompt structure, including class labels, to guide the LLM's response.

```python
def format_prompt(self):
    # add the labels. Classes for the classification
    global SYSTEM_PROMPT
    payload = {
        "user_prompt": self.prompt,
        "sentence": self.examples[0][0],
        "answer": f"<<{self.examples[0][1]}>>",
        "classes": ",".join(self.labels),
    }
    SYSTEM_PROMPT = Template(SYSTEM_PROMPT).substitute(**payload)
    self.prompt = self.prompt + f"\n\nClass labels are specified below:\n{','.join(self.labels)}. Your out should select from provided labels."
    system_prompt = f"{SYSTEM_PROMPT}\n\n{self.prompt}\n\n{end_seq}"
    self.prompt = [{"role": "system", "content": system_prompt}]
    for idx, i in enumerate(self.examples):
        if idx == 0:
            pass
        else:
            self.prompt += [{"role": "user", "content": i[0]}]
            self.prompt += [
                {
                    "role": "assistant",
                    "content": f"<<{i[1]}>>",
                }
            ]
```

--------------------------------

### Start Finetuning Process

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Execute the finetuning process using the initialized trainer. This will train the SamLoRA model on the specified dataset.

```python
trainer.train()
print("Finetuning process completed.")
```

--------------------------------

### Get Tracked Features with Time Range, Object IDs, and Output Layer (Yet Another Alternative)

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/multi_object_tracking_using_object_tracker.ipynb

This example further illustrates the flexibility in calling get_tracked_features by varying the order of parameters for time range, object IDs, and output layer.

```python
tracked_features = tracker.get_tracked_features(time_range=time_range, output_layer=output_layer, object_ids=object_ids)
```

--------------------------------

### Install necessary libraries

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Installs the required libraries for finetuning SAM with SamLoRA. Ensure you have a compatible environment.

```bash
pip install -U "git+https://github.com/SysCV/sam-lora.git"
```

--------------------------------

### Get Tracked Features with Time Range, Object IDs, and Output Layer (Final Alternative)

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/multi_object_tracking_using_object_tracker.ipynb

The final example showcasing parameter combinations for retrieving tracked features, demonstrating the various ways to specify time range, object IDs, and output layer.

```python
tracked_features = tracker.get_tracked_features(output_layer=output_layer, object_ids=object_ids, time_range=time_range)
```

--------------------------------

### Configure and Start Finetuning

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/finetune-sam-using-samlora.ipynb

Configures the training parameters and starts the finetuning process for the SAM model with SamLoRA. This includes setting up the optimizer, scheduler, and training loop.

```python
from torch.optim import Adam
from torch.optim.lr_scheduler import StepLR

# Assuming dataset and dataloader are prepared
train_dataset = MyDataset(image_dir='path/to/images', annotation_dir='path/to/annotations')
train_dataloader = torch.utils.data.DataLoader(train_dataset, batch_size=4, shuffle=True)

# Initialize model and SamLoRA adapter (as shown in the first snippet)
# ...

# Finetune only the LoRA parameters
for param in sam.parameters():
    param.requires_grad = False

# Enable gradients for LoRA parameters
for name, param in sam.named_parameters():
    if 'lora' in name or 'prompt_encoder' in name or 'mask_decoder' in name:
        param.requires_grad = True

# Optimizer and Scheduler
optimizer = Adam([p for p in sam.parameters() if p.requires_grad], lr=1e-4, weight_decay=0)
scheduler = StepLR(optimizer, step_size=10, gamma=0.1)

num_epochs = 10

for epoch in range(num_epochs):
    sam.train()
    total_loss = 0
    for images, masks, points, labels in train_dataloader:
        images = images.to(device)
        masks = masks.to(device)
        points = points.to(device)
        labels = labels.to(device)

        optimizer.zero_grad()

        # Forward pass
        # This part requires a custom training loop that handles SAM's input/output
        # and calculates the appropriate loss (e.g., IoU loss, Dice loss)
        # Example placeholder:
        # loss = sam(images, masks, points, labels)
        loss = torch.tensor(0.0) # Placeholder loss

        loss.backward()
        optimizer.step()
        total_loss += loss.item()

    scheduler.step()
    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {total_loss/len(train_dataloader):.4f}")

# Save the finetuned LoRA weights
# ... (implementation depends on how LoRA weights are managed)
```

--------------------------------

### Verify ArcGIS API for Python Installation in Google Colab

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/01-getting-started/install-and-set-up.ipynb

Check if the ArcGIS API for Python and its dependencies are installed correctly by listing all installed packages. This command is useful for troubleshooting installation issues.

```bash
!pip list
```

--------------------------------

### Set up configuration

Source: https://github.com/esri/arcgis-python-api/blob/master/guide/14-deep-learning/panoptic_segmentation_with_maxdeeplab.ipynb

Configure the parameters for training and inference. This includes paths to data, model settings, and training hyperparameters.

```python
config_file = "/home/user/.config/maxdeeplab/maxdeeplab.conf"
config = ConfigParser(config_file)

# Set random seed for reproducibility
random_seed = config.get_int("random_seed")
random.seed(random_seed)
np.random.seed(random_seed)
torch.manual_seed(random_seed)
torch.cuda.manual_seed_all(random_seed)

# Set device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")

# Create output directory if it doesn't exist
output_dir = Path(config.get("output_dir"))
output_dir.mkdir(parents=True, exist_ok=True)

# Load dataset configuration
dataset_config = config.get_section("dataset")

# Load model configuration
model_config = config.get_section("model")

# Load training configuration
train_config = config.get_section("training")

# Load inference configuration
inference_config = config.get_section("inference")
```

--------------------------------

### Install Required Libraries

Source: https://github.com/esri/arcgis-python-api/blob/master/samples/04_gis_analysts_data_scientists/detecting_mussel_farms_using_deep_learning.ipynb

Installs the necessary Python libraries for the project. Run this cell if you don't have the libraries installed.

```python
import sys
import os

from IPython.display import display

import arcgis
from arcgis.gis import GIS
from arcgis.learn import ObjectDetector
from arcgis.core import FeatureCollection, data

import pandas as pd
import numpy as np

import matplotlib.pyplot as plt

from PIL import Image

from IPython.core.display import HTML

# Set up environment variables
os.environ['ARCGIS_PYTHON_API_ENABLE_PROFILING'] = 'false'

# Check arcgis version
print(f"ArcGIS API for Python: {arcgis.__version__}")

# Connect to ArcGIS Online
gis = GIS('home')

```