### PromptEngine - Jinja2 Template Prompt Generation (Python)

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Manages Jinja2 templates for generating structured LLM prompts. Supports various prompt types like context building, diffusion transformation, and refinement. Requires prompt template definitions and context data.

```python
from src.llm.prompt_engine import PromptEngine, PromptTemplate

# Initialize engine with default templates
engine = PromptEngine()  # Uses src/templates/ directory

# Render diffusion transformation prompt
context_data = {
    "caption": "A clear sunny day on an urban street with multiple vehicles and pedestrians."
}

prompt = engine.render_prompt(
    template=PromptTemplate.DIFFUSION_MODEL_TRANSFORMATION_PROMPT,
    overall_caption_context=context_data
)
print(prompt)

# Render refinement prompt with evaluation feedback
refinement_prompt = engine.render_prompt(
    template=PromptTemplate.REFINER_PROMPT,
    previous_prompt="rainy urban street",
    evaluation_results={"iou_score": 0.65, "missing_classes": ["pedestrian"]},
    iteration=2
)
```

--------------------------------

### GPT4oCaptioner: Vision API Captioning with GPT-4o

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Leverages OpenAI's GPT-4o vision capabilities for high-quality, detailed captions of driving scenes. Suitable for generating ground-truth descriptions and processing entire datasets.

```python
import os
os.environ["OPENAI_API_KEY"] = "your-api-key"

from src.context.gpt4o_captioner import GPT4oCaptioner

# Initialize captioner
captioner = GPT4oCaptioner()

# Caption a single image with default prompt
caption = captioner.caption_image("dataset/rgb_front/0001.png")
print(f"Caption: {caption}")

# Caption with custom prompt
detailed_caption = captioner.caption_image(
    image_path="dataset/rgb_front/0001.png",
    prompt="Analyze this autonomous driving scene. Describe: 1) Weather conditions, 2) Road type and condition, 3) All visible vehicles with positions, 4) Pedestrians and their activities, 5) Traffic signs and lights."
)

# Process entire dataset
captions = captioner.caption_dataset(
    dataset_root="dataset/weather-0",
    output_file="captions/gpt4o_captions.json"
)

# Output includes metadata:
# {
#   "model_id": "gpt-4o-vision",
#   "total_images": 100,
#   "captions": { ... }
# }
```

--------------------------------

### LLaVACaptioner: Image Captioning with LLaVA Model

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Generates detailed captions for autonomous driving scenes using the LLaVA model. Supports 4-bit quantization for efficient inference and can caption single images or entire datasets, outputting structured JSON.

```python
from src.context.llava_captioner import LLaVACaptioner

# Initialize captioner with custom prompt
captioner = LLaVACaptioner(
    model_id="llava-hf/llava-1.5-7b-hf",
    prompt="Describe this autonomous driving scene in detail, including all vehicles, pedestrians, road conditions, weather.",
    device="cuda",
    use_4bit=True  # Enable 4-bit quantization for memory efficiency
)

# Caption a single image
caption = captioner.caption_image(
    image_path="dataset/rgb_front/0001.png",
    prompt="Describe the weather conditions and road surface in this driving scene."
)
print(f"Caption: {caption}")

# Caption an entire dataset
captions_dict = captioner.caption_dataset(
    dataset_root="dataset/weather-0",
    output_file="captions/llava_captions.json"
)

# Output format:
# {
#   "model_id": "llava-hf/llava-1.5-7b-hf",
#   "total_images": 100,
#   "captions": {
#     "data/route_0/rgb_front/0001.png": "The scene shows a clear day...",
#     ...
#   }
# }
```

--------------------------------

### CarlaFrameDataset - PyTorch Dataset Handler (Python)

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Provides a PyTorch Dataset interface for CARLA simulator data, loading paired RGB and semantic segmentation images. It automatically pairs images across different conditions and can be used with PyTorch's DataLoader.

```python
from src.data.dataset import CarlaFrameDataset
from torch.utils.data import DataLoader

# Initialize dataset
dataset = CarlaFrameDataset(
    dataset_root="dataset/domain_0/weather-0",
    transform=None,       # Optional custom RGB transform
    seg_transform=None    # Optional segmentation transform
)

print(f"Total frames: {len(dataset)}")

# Access single frame
(rgb_tensor, seg_tensor), (rgb_path, seg_path) = dataset[0]
print(f"RGB shape: {rgb_tensor.shape}")
print(f"Segmentation shape: {seg_tensor.shape}")
print(f"RGB path: {rgb_path}")

# Use with DataLoader
dataloader = DataLoader(dataset, batch_size=4, shuffle=True)

for (rgb_batch, seg_batch), (rgb_paths, seg_paths) in dataloader:
    # Process batch
    print(f"Batch RGB shape: {rgb_batch.shape}")
    break
```

--------------------------------

### OpenAIClient - Text and Image Analysis (Python)

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Provides a unified interface for OpenAI's chat and vision APIs. Handles text generation for prompts and image analysis for multimodal understanding. Requires the 'OPENAI_API_KEY' environment variable and specific client/config classes.

```python
import os
os.environ["OPENAI_API_KEY"] = "your-api-key"

from src.llm.openai_client import OpenAIClient
from src.llm.config import LLMConfig
from src.llm.base import LLMRequest

# Configure LLM
config = LLMConfig()
config.openai_model = "gpt-4o"
config.temperature = 0.7
config.max_tokens = 2000

# Initialize client
client = OpenAIClient(config)

# Send text request
request = LLMRequest(
    messages=[
        {"role": "system", "content": "You are an expert in autonomous driving scenario generation."},
        {"role": "user", "content": "Generate a diffusion prompt for transforming a clear day scene into a foggy morning scene."}
    ],
    temperature=0.7,
    max_tokens=500
)

response = client.send_request(request)
print(f"Response: {response.content}")
print(f"Tokens used: {response.usage['total_tokens']}")

# Analyze images with vision API
image_response = client.analyze_images(
    image_paths=["original.png", "generated.png"],
    prompt="Compare these two driving scenes. What are the main differences in weather, lighting, and road conditions?",
    image_types=["png", "png"]
)
print(f"Analysis: {image_response.content}")
```

--------------------------------

### Orchestrate Domain Expansion Pipeline with FeedbackLoop

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

The FeedbackLoop class manages the end-to-end domain expansion pipeline, coordinating caption generation, prompt creation, image augmentation, and semantic evaluation. It implements a closed-loop system for iterative refinement of generated images based on semantic thresholds.

```python
from src.feedback_loop.loop import FeedbackLoop

# Initialize the feedback loop with all components
feedback_loop = FeedbackLoop(
    dataset_root="/path/to/carla/dataset",  # Root of CARLA dataset with weather-*/data/routes structure
    llm_provider="openai",                   # LLM provider for prompt generation
    diffusion_model_id="runwayml/stable-diffusion-v1-5",
    segmentation_model_id="nvidia/segformer-b5-finetuned-cityscapes-1024-1024",
    max_iterations=5,                        # Max refinement iterations per frame
    semantic_threshold=0.75,                 # Threshold to accept generated image
    output_dir="outputs/augmented",
    save_all_iterations=True,                # Save intermediate results
    prompt_mode="advanced"                   # "basic" or "advanced" prompt generation
)

# Process specific frames
results = feedback_loop.run(
    num_frames=10,        # Number of frames to process
    start_idx=0,          # Starting frame index
    frame_indices=None    # Or provide specific indices: [0, 5, 10, 15]
)

# Process a single frame with custom save directory
single_result = feedback_loop.process_frame(
    frame_idx=0,
    save_dir="outputs/custom_frame"
)

# Results contain iteration details, best scores, and paths
print(f"Best score: {single_result['best_score']}")
print(f"Best iteration: {single_result['best_iteration']}")
print(f"Best image: {single_result['best_image_path']}")
```

--------------------------------

### Text-Image Similarity with ClipScoreModel

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Computes CLIP-based similarity scores between text descriptions and images using the LongCLIP model variant. This allows for evaluating how well an image matches a given text prompt, even for longer descriptions.

```python
from src.evaluation.clip_score import ClipScoreModel
from PIL import Image

# Initialize CLIP model
clip_model = ClipScoreModel(
    model_id="zer0int/LongCLIP-L-Diffusers",
    device="cuda"
)

# Calculate text-image similarity
score = clip_model.calculate_text_image_similarity(
    text="A rainy urban street scene with wet roads, pedestrians with umbrellas, and cars with headlights on",
    image="outputs/augmented/rainy_scene.png"
)
print(f"CLIP similarity score: {score:.4f}")

# Compare multiple prompts to find best match
prompts = [
    "sunny day with clear sky",
    "rainy evening with wet roads",
    "foggy morning with low visibility"
]

image = "generated_image.png"
for prompt in prompts:
    score = clip_model.calculate_text_image_similarity(prompt, image)
    print(f"'{prompt}': {score:.4f}")
```

--------------------------------

### BLIPConditionalCaptioner: Conditional Caption Generation

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Uses Salesforce's BLIP model for conditional text generation, employing multiple starter phrases to produce diverse captions and selecting the most comprehensive one. Supports single image captioning and dataset processing.

```python
from src.context.blip_conditional_captioner import BLIPConditionalCaptioner

# Initialize with BLIP model
captioner = BLIPConditionalCaptioner(
    model_id="Salesforce/blip-image-captioning-large"
)

# Caption with specific starter phrase
caption = captioner.caption_image_with_starter(
    image_path="dataset/rgb_front/0001.png",
    starter_phrase="A street scene showing"
)

# Use multi-starter approach (automatically selects best caption)
best_caption = captioner.caption_image_multi_starter("dataset/rgb_front/0001.png")

# Default starter phrases used:
# - "a street scene showing"
# - "an autonomous driving view of"
# - "a road with"
# - "the driving scene includes"
# - "this urban scene shows"

# Process dataset with both modes
captions = captioner.caption_dataset(
    dataset_root="dataset/weather-0",
    output_file="captions/blip_captions.json",
    use_multi_starter=True
)
```

--------------------------------

### Semantic Segmentation with SegFormer

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Performs semantic segmentation on images using the SegFormer model fine-tuned on Cityscapes. It outputs raw segmentation maps with CARLA class IDs and colorized visualizations, mapping Cityscapes classes to CARLA IDs.

```python
from src.segmentation.segformer import SegFormer
from PIL import Image

# Initialize SegFormer model
segformer = SegFormer()  # Uses nvidia/segformer-b5-finetuned-cityscapes-1024-1024

# Segment an image
carla_seg_map, colored_seg_map = segformer("dataset/rgb_front/0001.png")

# Save outputs
Image.fromarray(carla_seg_map).save("segmentation_mask.png")
Image.fromarray(colored_seg_map).save("segmentation_colored.png")
```

--------------------------------

### Save LLM Context JSON

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Saves Large Language Model (LLM) context to a JSON file. This function takes a frame path, JSON content, and a folder name to store the context, likely for later analysis or model training.

```python
from carlaframe.dataset import CarlaFrameDataset

rgb_path = "/path/to/rgb/frame.png"
CarlaFrameDataset.save_llm_context_json(
    frame_path=rgb_path,
    json_content='{"caption": "A sunny day scene", "objects": ["car", "pedestrian"]}',
    folder_name='llm_context'
)
```

--------------------------------

### Synthesize Augmented Images with DiffusionAugmenter

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

The DiffusionAugmenter class facilitates the generation of augmented driving scene images using Stable Diffusion with ControlNet. It supports various pipeline types, including inpainting and segmentation-guided generation, to maintain scene structure during domain transformations.

```python
from src.diffusion.diffusion_augmenter import DiffusionAugmenter
from src.diffusion.factory import PipelineType

# Initialize augmenter with specific pipeline typeaugmenter = DiffusionAugmenter(
    pipeline_type=PipelineType.STABLE_DIFFUSION_INPAINTING_CONTROLNET_REFINING_SEG,
    model_id="runwayml/stable-diffusion-v1-5",
    device="cuda",
    output_dir="outputs/augmented",
    use_canny=True  # Enable canny edge detection for structure preservation
)

# Prepare prompt data (from LLM or manual)
prompt_data = {
    "prompt": "a rainy urban street scene with wet roads and reflections, autonomous driving view",
    "negative_prompt": "blurry, distorted, unrealistic, artifacts, low quality",
    "refine_prompt": "photorealistic rainy city street, evening lighting"
}

# Augment a single image
augmented_image, output_path, comparison_grid = augmenter.augment_image(
    image_path="dataset/weather-0/data/route_0/rgb_front/0001.png",
    prompt_data=prompt_data,
    strength=0.75,                # How much to modify the original
    guidance_scale=7.5,           # How closely to follow the prompt
    num_inference_steps=50,       # Denoising steps
    seed=42,                      # For reproducibility
    save_output=True,
    return_grid=True              # Returns side-by-side comparison
)

# Available pipeline types:
# - PipelineType.STABLE_DIFFUSION_CONTROLNET
# - PipelineType.STABLE_DIFFUSION_CONTROLNET_IMG2IMG_CANNY
```

--------------------------------

### Evaluate Generated Image Against Original (Python)

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Evaluates a generated image against an original image and its semantic map. It calculates various metrics like IoU, VQA score, and class preservation, saving visualizations of the comparison. Dependencies include an 'evaluator' object with an 'evaluate' method.

```python
results = evaluator.evaluate(
    original_image="dataset/rgb_front/0001.png",
    generated_image="outputs/augmented/0001_aug.png",
    original_semantic_map="dataset/seg_front/0001.png",  # Ground truth segmentation
    critical_classes=["pedestrian", "vehicle", "road"],
    target_caption="A rainy urban street with wet roads and evening lighting",
    save_visualization=True,
    output_dir="outputs/evaluation"
)

# Access evaluation results
print(f"Weighted Score: {results['weighted_score']:.4f}")
print(f"Above Threshold: {results['above_threshold']}")
print(f"IoU Scores: {results['iou_scores']}")
print(f"VQA Score: {results['vqa_score']}")
print(f"Class Preservation: {results['class_preservation']}")
```

--------------------------------

### Calculate Domain Distance Matrices (FID & CMMD)

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Calculates distance matrices between generated domains using Frechet Inception Distance (FID) and CLIP Maximum Mean Discrepancy (CMMD) metrics. This helps in prioritizing diverse domains for testing and avoiding redundancy.

```python
from diversity_matrix.diversity import (
    calculate_fid,
    calculate_clip_mmd,
    calculate_domain_distance_matrix_fid,
    calculate_domain_distance_matrix_clip_mmd,
    plot_distance_matrix,
    plot_combined_distance_matrices
)

# Calculate FID between two domains
fid_score = calculate_fid(
    real_dir="dataset/domain_0/rgb_front",
    generated_dir="dataset/domain_1/rgb_front",
    max_images=100
)
print(f"FID Score: {fid_score:.2f}")

# Calculate CMMD between domains
cmmd_score = calculate_clip_mmd(
    real_dir="dataset/domain_0/rgb_front",
    generated_dir="dataset/domain_1/rgb_front"
)
print(f"CMMD Score: {cmmd_score:.4f}")

# Generate full distance matrix across all domains
fid_matrix = calculate_domain_distance_matrix_fid(
    base_dir="dataset/",  # Contains domain_0/, domain_1/, etc.
    max_images=100
)

cmmd_matrix = calculate_domain_distance_matrix_clip_mmd(base_dir="dataset/")

# Visualize matrices
plot_distance_matrix(
    distance_matrix=fid_matrix,
    metric_name="FID",
    save_path="diversity_matrix_fid.png",
    label_mapping={"domain_0": "Clear Day", "domain_1": "Rainy", "domain_2": "Foggy"}
)

# Plot combined comparison
plot_combined_distance_matrices(
    clip_matrix=cmmd_matrix,
    fid_matrix=fid_matrix,
    save_path="diversity_combined.png"
)
```

--------------------------------

### Train Risk Model for Failure Prediction

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Trains predictive models to identify domains likely to cause ADS failures. It utilizes composite features from trajectory analysis and MAE metrics to predict infraction counts, aiding in risk assessment.

```python
from risk_analysis.train_risk_model import get_data, train_risk_model

# Load and merge feature datasets
df = get_data()
# Returns DataFrame with columns:
# - domain_id: Domain identifier
# - percentage: Data sampling percentage (10, 25, 50, 75)
# - *_MAE: Mean Absolute Error features
# - LJ_mean: Longitudinal jerk mean
# - LI_mean: Lateral instability mean
# - BEC: Brake event count
# - TRC: Throttle release count
# - SRR: Steer reversal rate

# Example of training the model (actual training call not shown in provided text)
# train_risk_model(df)
```

--------------------------------

### Train Risk Prediction Model

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Trains a risk prediction model. The output is an interpretation report detailing top predictors at different data percentages, indicating feature importance for risk assessment. This function is crucial for understanding and prioritizing safety-critical scenarios.

```python
train_risk_model()
# Outputs: analysis_outputs_by_failure_type/model_interpretation_report.txt
```

--------------------------------

### SemanticEvaluator: Semantic Coherence Validation

Source: https://context7.com/bubabi/zero-shot-domain-expansion-for-testing-av/llms.txt

Validates semantic consistency between generated and original scenes using segmentation-based IoU scoring and VQA-based text-image similarity. Computes a weighted evaluation score for critical classes.

```python
from src.evaluation.semantic_evaluator import SemanticEvaluator

# Initialize evaluator
evaluator = SemanticEvaluator(
    segmentation_model_id="nvidia/segformer-b5-finetuned-cityscapes-1024-1024",
    device="cuda",
    semantic_threshold=0.75,
    critical_classes=["pedestrian", "vehicle", "road", "traffic_light", "traffic_sign", "pole"]
)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.