### Fine-Tune StreetCLIP for Downstream Tasks

Source: https://context7.com/geolocal/streetclip/llms.txt

Initialize a fine-tuning setup using StreetCLIP's pretrained weights. This example freezes the text encoder and adds a classification head to the visual encoder for domain-specific tasks.

```python
import torch
from torch import nn
from transformers import CLIPModel, CLIPProcessor
from torch.utils.data import DataLoader

# Load base model
model = CLIPModel.from_pretrained("geolocal/StreetCLIP")
processor = CLIPProcessor.from_pretrained("geolocal/StreetCLIP")

# Freeze text encoder; fine-tune only visual encoder
for param in model.text_model.parameters():
    param.requires_grad = False

# Add a classification head on top of visual features (e.g., 5 building quality classes)
class GeoClassifier(nn.Module):
    def __init__(self, clip_model, num_classes=5):
        super().__init__()
        self.clip = clip_model
        self.classifier = nn.Linear(768, num_classes)

    def forward(self, pixel_values):
        features = self.clip.get_image_features(pixel_values=pixel_values)
        features = features / features.norm(dim=-1, keepdim=True)
        return self.classifier(features)

classifier = GeoClassifier(model, num_classes=5)
optimizer = torch.optim.AdamW(
    filter(lambda p: p.requires_grad, classifier.parameters()),
    lr=1e-5
)
loss_fn = nn.CrossEntropyLoss()

# Training loop skeleton
# for images, labels in DataLoader(your_dataset):
#     inputs = processor(images=images, return_tensors="pt", padding=True)
#     logits = classifier(inputs["pixel_values"])
#     loss = loss_fn(logits, labels)
#     optimizer.zero_grad(); loss.backward(); optimizer.step()
print("Fine-tuning setup ready. Trainable params:",
      sum(p.numel() for p in classifier.parameters() if p.requires_grad))
```

--------------------------------

### Get Started with StreetCLIP Model

Source: https://github.com/geolocal/streetclip/blob/main/README.md

Load the StreetCLIP model and processor, process an image and text choices, and get similarity scores. Ensure you have Pillow, requests, and transformers installed.

```python
from PIL import Image
import requests

from transformers import CLIPProcessor, CLIPModel

model = CLIPModel.from_pretrained("geolocal/StreetCLIP")
processor = CLIPProcessor.from_pretrained("geolocal/StreetCLIP")

url = "https://huggingface.co/geolocal/StreetCLIP/resolve/main/sanfrancisco.jpeg"
image = Image.open(requests.get(url, stream=True).raw)

choices = ["San Jose", "San Diego", "Los Angeles", "Las Vegas", "San Francisco"]
inputs = processor(text=choices, images=image, return_tensors="pt", padding=True)

outputs = model(**inputs)
logits_per_image = outputs.logits_per_image # this is the image-text similarity score
probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities
```

--------------------------------

### Run StreetCLIP Inference on GPU

Source: https://context7.com/geolocal/streetclip/llms.txt

Move the model to a CUDA device for faster inference on large batches. Ensure PyTorch is installed with CUDA support.

```python
import torch
from PIL import Image
import requests
from transformers import CLIPModel, CLIPProcessor

device = "cuda" if torch.cuda.is_available() else "cpu"

model = CLIPModel.from_pretrained("geolocal/StreetCLIP").to(device)
processor = CLIPProcessor.from_pretrained("geolocal/StreetCLIP")
model.eval()

url = "https://huggingface.co/geolocal/StreetCLIP/resolve/main/sanfrancisco.jpeg"
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")
labels = ["San Francisco", "New York", "London", "Paris", "Sydney"]

inputs = processor(text=labels, images=image, return_tensors="pt", padding=True)
inputs = {k: v.to(device) for k, v in inputs.items()}

with torch.no_grad():
    outputs = model(**inputs)

probs = outputs.logits_per_image.softmax(dim=1).squeeze().cpu()
for label, prob in zip(labels, probs):
    print(f"{label}: {prob:.2%}")
```

--------------------------------

### Image Preprocessing for StreetCLIP

Source: https://context7.com/geolocal/streetclip/llms.txt

Applies resizing, center cropping, tensor conversion, and normalization to images, matching preprocessor_config.json settings. Requires Pillow and PyTorch.

```python
from PIL import Image
from torchvision import transforms

preprocess = transforms.Compose([
    transforms.Resize(336, interpolation=transforms.InterpolationMode.BICUBIC),
    transforms.CenterCrop(336),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.48145466, 0.4578275,  0.40821073],
        std= [0.26862954, 0.26130258, 0.27577711]
    ),
])

image = Image.open("street_photo.jpg").convert("RGB")
tensor = preprocess(image).unsqueeze(0)   # shape: (1, 3, 336, 336)
print(tensor.shape)  # torch.Size([1, 3, 336, 336])
```

--------------------------------

### StreetCLIP Image Preprocessor Configuration

Source: https://context7.com/geolocal/streetclip/llms.txt

The preprocessor performs center-cropping and normalization to match StreetCLIP's training distribution. These parameters are loaded automatically via CLIPProcessor but can be applied manually for custom pipelines.

```python
from torchvision import transforms
from PIL import Image
```

--------------------------------

### Load StreetCLIP Model and Processor

Source: https://context7.com/geolocal/streetclip/llms.txt

Load the pretrained StreetCLIP model and its associated CLIPProcessor from the Hugging Face Hub. The processor handles image preprocessing and text tokenization.

```python
from transformers import CLIPModel, CLIPProcessor

# Load model and processor from Hugging Face Hub
model = CLIPModel.from_pretrained("geolocal/StreetCLIP")
processor = CLIPProcessor.from_pretrained("geolocal/StreetCLIP")

print(model.config.vision_config.image_size)   # 336
print(model.config.text_config.max_position_embeddings)  # 77
```

--------------------------------

### Zero-Shot City Classification with StreetCLIP

Source: https://context7.com/geolocal/streetclip/llms.txt

Narrow down geolocation to the city level by providing candidate city names. The model returns a probability for each city against the input image.

```python
from PIL import Image
import requests
import torch
from transformers import CLIPModel, CLIPProcessor

model = CLIPModel.from_pretrained("geolocal/StreetCLIP")
processor = CLIPProcessor.from_pretrained("geolocal/StreetCLIP")
model.eval()

# Load a street-level image (example: San Francisco)
url = "https://huggingface.co/geolocal/StreetCLIP/resolve/main/sanfrancisco.jpeg"
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")

cities = ["San Jose", "San Diego", "Los Angeles", "Las Vegas", "San Francisco", "Seattle"]

inputs = processor(text=cities, images=image, return_tensors="pt", padding=True)

with torch.no_grad():
    outputs = model(**inputs)

probs = outputs.logits_per_image.softmax(dim=1).squeeze()
ranked = sorted(zip(cities, probs.tolist()), key=lambda x: x[1], reverse=True)

for city, prob in ranked:
    print(f"{city}: {prob:.2%}")
# Example output:
# San Francisco: 61.34%
# Los Angeles: 14.21%
# Seattle: 9.87%
# ...
```

--------------------------------

### Zero-Shot Country Classification with StreetCLIP

Source: https://context7.com/geolocal/streetclip/llms.txt

Classify a street-level image by passing a list of candidate country names. The model scores each label against the image and returns a probability distribution.

```python
from PIL import Image
import requests
import torch
from transformers import CLIPModel, CLIPProcessor

model = CLIPModel.from_pretrained("geolocal/StreetCLIP")
processor = CLIPProcessor.from_pretrained("geolocal/StreetCLIP")
model.eval()

# Load a street-level image (example: Nagasaki, Japan)
url = "https://huggingface.co/geolocal/StreetCLIP/resolve/main/nagasaki.jpg"
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")

# Candidate country labels
countries = ["China", "South Korea", "Japan", "Philippines", "Taiwan", "Vietnam", "Cambodia"]

inputs = processor(text=countries, images=image, return_tensors="pt", padding=True)

with torch.no_grad():
    outputs = model(**inputs)

logits_per_image = outputs.logits_per_image          # shape: (1, num_labels)
probs = logits_per_image.softmax(dim=1).squeeze()    # shape: (num_labels,)

results = {country: round(prob.item(), 4) for country, prob in zip(countries, probs)}
best_match = max(results, key=results.get)

print(results)
# Example output: {'China': 0.05, 'South Korea': 0.07, 'Japan': 0.72, ...}
print(f"Predicted country: {best_match}")
# Predicted country: Japan
```

--------------------------------

### Hierarchical Linear Probing (Country → City)

Source: https://context7.com/geolocal/streetclip/llms.txt

Predicts the country first, then uses that prediction to narrow down city candidates. This mimics the evaluation methodology used in the StreetCLIP paper.

```python
from PIL import Image
import requests
import torch
from transformers import CLIPModel, CLIPProcessor

model = CLIPModel.from_pretrained("geolocal/StreetCLIP")
processor = CLIPProcessor.from_pretrained("geolocal/StreetCLIP")
model.eval()

CITY_MAP = {
    "United States": ["New York", "Los Angeles", "Chicago", "San Francisco", "Seattle"],
    "Japan":         ["Tokyo", "Osaka", "Nagasaki", "Kyoto", "Hiroshima"],
    "France":        ["Paris", "Lyon", "Marseille", "Bordeaux", "Nice"],
}

url = "https://huggingface.co/geolocal/StreetCLIP/resolve/main/nagasaki.jpg"
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")

def predict_top(image, labels):
    inputs = processor(text=labels, images=image, return_tensors="pt", padding=True)
    with torch.no_grad():
        outputs = model(**inputs)
    probs = outputs.logits_per_image.softmax(dim=1).squeeze()
    best_idx = probs.argmax().item()
    return labels[best_idx], probs[best_idx].item()

# Step 1: Country prediction
predicted_country, country_prob = predict_top(image, list(CITY_MAP.keys()))
print(f"Country: {predicted_country} ({country_prob:.2%})")
# Country: Japan (78.43%)

# Step 2: City prediction within the predicted country
city_candidates = CITY_MAP[predicted_country]
predicted_city, city_prob = predict_top(image, city_candidates)
print(f"City: {predicted_city} ({city_prob:.2%})")
# City: Nagasaki (54.11%)
```

--------------------------------

### Batch Inference on Multiple Images

Source: https://context7.com/geolocal/streetclip/llms.txt

Processes a batch of street-level images against the same candidate labels in one forward pass for efficient inference. The output probabilities are shaped (num_images, num_labels).

```python
import torch
import requests
from PIL import Image
from transformers import CLIPModel, CLIPProcessor

model = CLIPModel.from_pretrained("geolocal/StreetCLIP")
processor = CLIPProcessor.from_pretrained("geolocal/StreetCLIP")
model.eval()

image_urls = [
    "https://huggingface.co/geolocal/StreetCLIP/resolve/main/nagasaki.jpg",
    "https://huggingface.co/geolocal/StreetCLIP/resolve/main/sanfrancisco.jpeg",
]
images = [Image.open(requests.get(u, stream=True).raw).convert("RGB") for u in image_urls]
labels = ["United States", "Japan", "France", "Germany", "Brazil"]

inputs = processor(text=labels, images=images, return_tensors="pt", padding=True)

with torch.no_grad():
    outputs = model(**inputs)

probs = outputs.logits_per_image.softmax(dim=1)   # shape: (2, 5) 

for i, image_probs in enumerate(probs):
    best = labels[image_probs.argmax().item()]
    print(f"Image {i+1}: {best} ({image_probs.max().item():.2%})")
# Image 1: Japan (71.22%)
# Image 2: United States (65.48%)
```

--------------------------------

### BibTeX Citation for StreetCLIP Paper

Source: https://github.com/geolocal/streetclip/blob/main/README.md

Use this BibTeX entry to cite the StreetCLIP preprint in academic work.

```bibtex
@misc{haas2023learning,
      title={Learning Generalized Zero-Shot Learners for Open-Domain Image Geolocalization},
      author={Lukas Haas and Silas Alberti and Michal Skreta},
      year={2023},
      eprint={2302.00275},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
  }
```

--------------------------------

### Extracting Image and Text Embeddings

Source: https://context7.com/geolocal/streetclip/llms.txt

Uses StreetCLIP as a feature extractor to obtain L2-normalized image or text embeddings for similarity search, clustering, or downstream fine-tuning. Ensure embeddings are L2 normalized for cosine similarity calculations.

```python
import torch
import requests
from PIL import Image
from transformers import CLIPModel, CLIPProcessor

model = CLIPModel.from_pretrained("geolocal/StreetCLIP")
processor = CLIPProcessor.from_pretrained("geolocal/StreetCLIP")
model.eval()

url = "https://huggingface.co/geolocal/StreetCLIP/resolve/main/sanfrancisco.jpeg"
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")
text = ["A street in San Francisco", "A street in Tokyo"]

image_inputs = processor(images=image, return_tensors="pt")
text_inputs  = processor(text=text, return_tensors="pt", padding=True)

with torch.no_grad():
    image_features = model.get_image_features(**image_inputs)   # (1, 768)
    text_features  = model.get_text_features(**text_inputs)     # (2, 768)

# L2 normalize for cosine similarity
image_features = image_features / image_features.norm(dim=-1, keepdim=True)
text_features  = text_features  / text_features.norm(dim=-1, keepdim=True)

similarity = (image_features @ text_features.T).squeeze()
for label, score in zip(text, similarity.tolist()):
    print(f"{label}: {score:.4f}")
# A street in San Francisco: 0.3121
# A street in Tokyo: 0.1893
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.