### Install Backend Dependencies and Run Server

Source: https://github.com/abi/screenshot-to-code/blob/main/README.md

Install backend dependencies using Poetry and start the FastAPI backend server with hot-reloading.

```bash
poetry install
poetry env activate
poetry run uvicorn main:app --reload --port 7001
```

--------------------------------

### Install Frontend Dependencies and Run Server

Source: https://github.com/abi/screenshot-to-code/blob/main/README.md

Install frontend dependencies using Yarn and start the Vite development server.

```bash
cd frontend
yarn
yarn dev
```

--------------------------------

### Docker Setup for Screenshot-to-Code

Source: https://github.com/abi/screenshot-to-code/blob/main/README.md

Use this snippet to set up and run the screenshot-to-code application using Docker. Ensure you have Docker installed and replace 'sk-your-key' with your actual OpenAI API key.

```bash
echo "OPENAI_API_KEY=sk-your-key" > .env
docker-compose up -d --build
```

--------------------------------

### Install Backend Dependencies

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

Installs all project dependencies using Poetry. Ensure you are in the backend directory.

```bash
cd backend
poetry install
```

--------------------------------

### Install Playwright Dependencies

Source: https://github.com/abi/screenshot-to-code/blob/main/README.md

Install the Chromium browser used by the screenshot preview tool. On Linux, this command also installs required system libraries.

```bash
poetry run playwright install chromium
```

--------------------------------

### Install pytest-xdist for Parallel Testing

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

Installs the pytest-xdist plugin, required for running tests in parallel. Use the --with dev flag for development dependencies.

```bash
poetry install --with dev pytest-xdist
```

--------------------------------

### Example Backend Test Function

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

A basic pytest test function demonstrating URL normalization. It asserts that a URL without a protocol is correctly prefixed with 'https://'.

```python
import pytest
from routes.screenshot import normalize_url

def test_url_normalization():
    assert normalize_url("example.com") == "https://example.com"
```

--------------------------------

### Configure Backend API Keys

Source: https://github.com/abi/screenshot-to-code/blob/main/README.md

Set up environment variables for API keys required by the backend. Ensure you have at least one model provider key (OpenAI, Anthropic, or Gemini). Gemini and Replicate are strongly recommended for best results.

```bash
cd backend
echo "OPENAI_API_KEY=sk-your-key" > .env
echo "ANTHROPIC_API_KEY=your-key" >> .env
echo "GEMINI_API_KEY=your-key" >> .env
echo "REPLICATE_API_KEY=r8_your-key" >> .env
```

--------------------------------

### Backend Test for Prompt Summary/Preview

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/prompt-history-refactor.md

Unit tests for prompt preview logging, replacing previous summary logging.

```python
backend/tests/test_prompt_summary.py
```

--------------------------------

### Visualize Prompts

Source: https://github.com/abi/screenshot-to-code/blob/main/backend/README.md

Use the `print_prompt_summary` function from `utils.py` to visualize prompt messages. Ensure `utils` is imported.

```python
from utils import print_prompt_summary
print_prompt_summary(prompt_messages)
```

--------------------------------

### Lint Frontend with pnpm

Source: https://github.com/abi/screenshot-to-code/blob/main/CLAUDE.md

Runs the linter for the frontend project using pnpm. This command should be executed if changes affect the frontend code.

```bash
cd frontend && pnpm lint
```

--------------------------------

### Run Backend Tests with Coverage Report

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

Runs all backend tests and generates a coverage report for the specified module. Use the --cov flag.

```bash
poetry run pytest --cov=routes
```

--------------------------------

### Tool Execution Lifecycle

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/agent-tool-calling-flow.md

Details the steps involved in executing a tool call, from emitting lifecycle events to handling tool results and streaming code previews for file creation.

```python
# Execution lifecycle per tool call:
# 1. Emit `toolStart` (unless already emitted from streamed args).
# 2. If `create_file`, stream preview code chunks while args are still arriving.
# 3. Execute tool in runtime.
# 4. If tool returns `updated_content`, emit `setCode`.
# 5. Emit `toolResult` with `{ name, output, ok }`.
```

--------------------------------

### Model Selection Logic

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/variant-system.md

Demonstrates how to configure the list of models for variant generation based on the availability of API keys. The system cycles through these models to produce the desired number of variants.

```python
models = [claude_model, Llm.GPT_4_1_NANO_2025_04_14]
```

```python
models = [claude_model, Llm.CLAUDE_4_5_SONNET_2025_09_29]
```

```python
models = [Llm.GPT_4O_2024_11_20]
```

--------------------------------

### Run Backend Tests with Verbose Output

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

Runs all backend tests with verbose output enabled for more detailed logging. Use the -vv flag.

```bash
poetry run pytest -vv
```

--------------------------------

### Run Specific Backend Test Method

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

Executes a single test method within a specific class and file. Provide the full path to the method.

```bash
poetry run pytest tests/test_screenshot.py::TestNormalizeUrl::test_url_without_protocol
```

--------------------------------

### Run Tests

Source: https://github.com/abi/screenshot-to-code/blob/main/backend/README.md

Execute the test suite using Pytest. This command is run using Poetry.

```bash
poetry run pytest
```

--------------------------------

### Run Specific Backend Test File

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

Executes tests only within a specified file. Replace 'tests/test_screenshot.py' with your target file.

```bash
poetry run pytest tests/test_screenshot.py
```

--------------------------------

### Activate Poetry Virtual Environment

Source: https://github.com/abi/screenshot-to-code/blob/main/CLAUDE.md

Activates the Poetry virtual environment for the backend Python project. Use this command to ensure all subsequent Python commands run within the correct isolated environment.

```bash
cd backend && poetry env activate
```

--------------------------------

### Backend Prompt Assembly Logic

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/prompt-history-refactor.md

Python logic for assembling prompts using explicit role history, including system messages and prior conversation turns.

```python
# For update generation, builds:
#   system message
#   followed by all provided explicit history messages
```

--------------------------------

### Use Triple-Quoted f-strings for Interpolated Prompts

Source: https://github.com/abi/screenshot-to-code/blob/main/CLAUDE.md

Shows the recommended way to format interpolated multi-line prompts using a single triple-quoted f-string, which is cleaner than concatenating string fragments.

```python
user_input = "some data"

prompt = f"""This prompt includes interpolated data:
{user_input}
"""
```

--------------------------------

### Run Screenshot-to-Code Evals

Source: https://github.com/abi/screenshot-to-code/blob/main/Evaluation.md

Execute the screenshot-to-code evaluation script. Ensure the OPENAI_API_KEY is set and configure stack and model variables in the script.

```bash
OPENAI_API_KEY=sk-... python run_evals.py
```

--------------------------------

### Run Backend Type Checking with Pyright

Source: https://github.com/abi/screenshot-to-code/blob/main/CLAUDE.md

Performs static type checking on the backend code using pyright. The policy is to ensure no new warnings are introduced in the changed files.

```bash
cd backend && poetry run pyright
```

--------------------------------

### Backend Test for Prompt Assembly

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/prompt-history-refactor.md

Updated backend unit tests to expect explicit role history instead of index parity inference.

```python
backend/tests/test_prompts.py
```

--------------------------------

### Use Triple-Quoted Strings for Prompts

Source: https://github.com/abi/screenshot-to-code/blob/main/CLAUDE.md

Demonstrates the preferred method for defining multi-line prompt text using triple-quoted strings in Python.

```python
prompt = """This is a multi-line
prompt.
"""
```

--------------------------------

### Run Specific Backend Test Class

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

Targets and runs all tests within a specific class in a test file. Specify the file and class name.

```bash
poetry run pytest tests/test_screenshot.py::TestNormalizeUrl
```

--------------------------------

### Run All Backend Tests

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

Executes all tests in the backend directory using pytest. Ensure you are in the backend directory.

```bash
cd backend
poetry run pytest
```

--------------------------------

### Typical Per-Turn Streaming Sequence

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/agent-tool-calling-flow.md

Outlines the common sequence of frontend websocket messages emitted during a typical agent turn, including thinking, assistant updates, tool calls, and results.

```text
# Typical per-turn stream sequence:
# 1. thinking/assistant deltas
# 2. tool call deltas (optional)
# 3. `toolStart`
# 4. `setCode` previews (for `create_file`, optional)
# 5. `toolResult`
# 6. next model turn starts, repeat
```

--------------------------------

### Run Type Checker

Source: https://github.com/abi/screenshot-to-code/blob/main/backend/README.md

Execute the Pyright type checker to ensure code quality and catch type errors. This command is run using Poetry.

```bash
poetry run pyright
```

--------------------------------

### Project Commit Storage

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/commits-and-variants.md

Illustrates how commits are stored in the project's flat record, using a record keyed by commit hash and a 'head' pointer to track the currently active commit.

```typescript
commits: Record<CommitHash, Commit>
head: CommitHash | null  // Current active commit
```

--------------------------------

### Basic CSS Styling

Source: https://github.com/abi/screenshot-to-code/blob/main/frontend/src/tests/fixtures/simple_page.html

Applies basic styling to the body element for consistent padding and font.

```css
body {
  font-family: Arial, sans-serif;
  padding: 24px;
}
```

--------------------------------

### Button Styling with CSS

Source: https://github.com/abi/screenshot-to-code/blob/main/frontend/src/tests/fixtures/simple_page.html

Defines styles for a button element, including background color, border, text color, and padding.

```css
.button {
  background: #2563eb;
  border: none;
  border-radius: 8px;
  color: #fff;
  padding: 12px 20px;
  font-size: 16px;
}
```

--------------------------------

### Backend Request Parsing Functions

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/prompt-history-refactor.md

Python functions for normalizing and parsing raw prompt content and history from incoming requests.

```python
def parse_prompt_content(raw_prompt):
    ...
def parse_prompt_history(raw_history):
    ...
```

--------------------------------

### Commit Type Definitions

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/commits-and-variants.md

Defines the possible types for commits in the application: 'ai_create' for initial generation, 'ai_edit' for user-instructed updates, and 'code_create' for importing existing code.

```typescript
type CommitType = "ai_create" | "ai_edit" | "code_create";
```

--------------------------------

### OpenAI Tool Result Appending

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/agent-tool-calling-flow.md

Demonstrates how OpenAI provider appends tool results to the request history for continuation. It adds prior assistant output and function call outputs.

```python
# Append prior assistant output items (`turn.assistant_turn`) to request history.
# Append one `function_call_output` per tool result:
# - `{"type":"function_call_output","call_id":...,"output": json_string}`
```

--------------------------------

### Run Backend Tests in Parallel

Source: https://github.com/abi/screenshot-to-code/blob/main/TESTING.md

Executes backend tests concurrently using pytest-xdist. The '-n auto' flag automatically determines the number of workers.

```bash
poetry run pytest -n auto
```

--------------------------------

### Core Tool-Calling Loop in AgentEngine

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/agent-tool-calling-flow.md

This is the main loop for handling tool calls within the agent engine. It manages streaming state, processes provider turns, executes tool calls, and continues the conversation with tool results, including a guardrail for the maximum number of tool turns.

```python
await session.stream_turn(on_event)

# on_event handles streamed deltas:
#   assistant_delta -> websocket `assistant`
#   thinking_delta -> websocket `thinking`
#   tool_call_delta -> `_handle_streamed_tool_delta(...)`

# Branch by tool calls.
if turn.tool_calls is empty: finalize and return.
else execute each tool call, emit tool lifecycle messages, and collect results.

# Continue conversation with tool results.
session.append_tool_results(turn, executed_tool_calls)
# Next loop iteration sends another model turn with updated history.

# Guardrail.
# Maximum 20 tool turns; raises if exceeded.
```

--------------------------------

### Frontend Update History Item with Images

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/images-in-update-history.md

Frontend can send update history items with text and an array of base64 encoded images. The backend automatically processes this into multipart content for AI models.

```typescript
{
  text: "Update instructions",
  images: ["data:image/png;base64,img1", "data:image/png;base64,img2"]
}
```

--------------------------------

### Gemini Tool Result Appending

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/agent-tool-calling-flow.md

Shows how the Gemini provider appends tool results, preserving the original model content and adding tool response parts for reliable continuation.

```python
# Append exact original model content (`turn.assistant_turn`).
# Append `role="tool"` content with `Part.from_function_response(...)` per tool.
```

--------------------------------

### Sidebar UI Condition

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/commits-and-variants.md

Demonstrates a dual-condition UI rendering logic in Sidebar.tsx, showing the update interface when either the app state is 'CODE_READY' or the selected variant is complete.

```typescript
// Show update UI when either condition is true
{(appState === AppState.CODE_READY || isSelectedVariantComplete) && (
  <UpdateInterface />
)}
```

--------------------------------

### Python Asyncio Gather for Parallel Tasks

Source: https://github.com/abi/screenshot-to-code/blob/main/plan.md

This code demonstrates the current backend approach using `asyncio.gather` to wait for all parallel variant generation tasks to complete before proceeding. It's used when all variants must finish before any results are processed.

```python
completions = await asyncio.gather(*tasks, return_exceptions=True)
```

--------------------------------

### Anthropic Tool Result Appending

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/agent-tool-calling-flow.md

Illustrates Anthropic's method for appending tool results, involving assistant message blocks (text, tool_use) and user message blocks (tool_result).

```python
# Append assistant message blocks:
# - optional text block
# - tool_use blocks (`id`, `name`, `input`)
# Append user message with tool_result blocks:
# - `tool_use_id`, serialized result content, `is_error`
```

--------------------------------

### Backend Test for Request Parsing

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/prompt-history-refactor.md

Unit tests for the new request parsing module in the backend.

```python
backend/tests/test_request_parsing.py
```

--------------------------------

### Python Non-Blocking Variant Processing

Source: https://github.com/abi/screenshot-to-code/blob/main/plan.md

This proposed backend logic iterates through tasks, sending a 'variantComplete' message as soon as each individual variant finishes. This allows the frontend to receive and display results progressively, rather than waiting for all tasks.

```python
async def process_variants():
    for index, task in enumerate(tasks):
        try:
            completion = await task
            await send_message("variantComplete", index)
            # Allow frontend to interact immediately
        except Exception as e:
            await send_message("variantFailed", index)
```

--------------------------------

### Shared Media Asset Store Structure

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/prompt-history-refactor.md

Defines the structure for storing media assets (images and videos) by their IDs in a shared map.

```typescript
type PromptAsset = {
  id: string;
  type: "image" | "video";
  dataUrl: string;
};

assetsById: Record<string, PromptAsset>;
```

--------------------------------

### Independent Variant Processing in Python

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/commits-and-variants.md

Backend Python code snippet showing how each variant is processed independently using asyncio tasks, allowing immediate processing and sending of results to the frontend.

```python
# Process each variant independently
async def process_variant_completion(index: int, task: asyncio.Task):
    completion = await task  # Wait for THIS variant only
    
    # Process images immediately
    processed_html = await perform_image_generation(...)
    
    # Send to frontend immediately
    await send_message("setCode", processed_html, index)
    await send_message("variantComplete", "Variant generation complete", index)
```

--------------------------------

### UI Update Logic Condition

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/commits-and-variants.md

TypeScript code defining the condition for showing the update interface, which is true if all variants are done (AppState.CODE_READY) or if the currently selected variant has completed.

```typescript
// UI shows update interface when either:
const canUpdate = 
  appState === AppState.CODE_READY ||           // All variants done
  isSelectedVariantComplete;                    // Selected variant done
```

--------------------------------

### Frontend WebSocket Event Handlers

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/commits-and-variants.md

Shows new WebSocket event handlers in App.tsx for updating variant status upon completion or error, enabling real-time feedback.

```typescript
// New WebSocket events
onVariantComplete: (variantIndex) => {
  updateVariantStatus(commit.hash, variantIndex, 'complete');
}
onVariantError: (variantIndex, error) => {
  updateVariantStatus(commit.hash, variantIndex, 'cancelled');
}
```

--------------------------------

### TypeScript Variant State Interface

Source: https://github.com/abi/screenshot-to-code/blob/main/plan.md

Defines the structure for tracking the state of individual variants in the frontend. This includes the generated code, its current status (pending, generating, complete, failed, cancelled), and optional generation time.

```typescript
interface VariantState {
  code: string;
  status: 'pending' | 'generating' | 'complete' | 'failed' | 'cancelled';
  generationTime?: number;
}
```

--------------------------------

### Request History Payload Structure

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/prompt-history-refactor.md

Illustrates the simplified structure of the explicit role-based history sent in edit requests to the backend.

```plaintext
- history[i].role: "user" or "assistant"
- history[i].text: textual instruction or generated code
- history[i].images: data URLs for image inputs for that message
- history[i].videos: data URLs for video inputs for that message
```

--------------------------------

### Commit Data Structure

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/commits-and-variants.md

Defines the structure of a commit object, including its hash, parent hash, creation date, commitment status, variants, selected variant index, type, and inputs.

```typescript
type Commit = {
  hash: CommitHash;
  parentHash: CommitHash | null;
  dateCreated: Date;
  isCommitted: boolean;
  variants: Variant[];
  selectedVariantIndex: number;
  type: CommitType;
  inputs: any; // Type-specific inputs
}

type Variant = {
  code: string;
  status: VariantStatus;
}

type VariantStatus = "generating" | "complete" | "cancelled";
```

--------------------------------

### WebSocket Response Protocol

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/commits-and-variants.md

Defines the structure of WebSocket responses from the backend, including types for streaming code chunks, status updates, final code, completion, and errors, each associated with a variant index.

```typescript
type WebSocketResponse = {
  type: "chunk" | "status" | "setCode" | "variantComplete" | "variantError";
  value: string;
  variantIndex: number;
}
```

--------------------------------

### Variant History Message Structure

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/prompt-history-refactor.md

Defines the structure for messages within a variant's explicit history, including role, text, and media asset IDs.

```typescript
type VariantHistoryMessage = {
  role: "user" | "assistant";
  text: string;
  imageAssetIds: string[];
  videoAssetIds: string[];
};
```

--------------------------------

### WebSocket Message Types

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/variant-system.md

Defines the types of messages that can be exchanged between the backend and frontend via WebSockets. These messages are used for status updates, code transmission, and completion notifications.

```typescript
"variantCount" | "chunk" | "status" | "setCode" | "variantComplete" | "variantError"
```

--------------------------------

### Cancel Generating Variants in TypeScript

Source: https://github.com/abi/screenshot-to-code/blob/main/design-docs/commits-and-variants.md

This snippet demonstrates how to cancel variants that are currently generating when a user initiates an update. It iterates through existing variants, checks their status, and sends a 'cancel_variant' message via a WebSocket connection if the variant is generating and not the selected one.

```typescript
// Cancel generating variants when user updates
currentCommit.variants.forEach((variant, index) => {
  if (index !== selectedVariantIndex && variant.status === 'generating') {
    wsRef.current.send(JSON.stringify({
      type: "cancel_variant",
      variantIndex: index
    }));
  }
});
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.