### Bash CLI Usage Examples for CUA

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

Provides command-line interface (CLI) examples for running the CUA sample application with different computer environments. These commands demonstrate how to launch the interactive CLI with options for visualizing steps, specifying environments like Playwright, Docker, Browserbase, and Scrapybara, and setting initial URLs or input queries.

```bash
# Local Playwright browser (default)
python cli.py --show --computer local-playwright --start-url https://bing.com

# Docker Linux desktop environment
python cli.py --show --computer docker --debug

# Remote Browserbase browser
python cli.py --computer browserbase --input "Search for Python tutorials"

# Scrapybara remote browser
python cli.py --computer scrapybara-browser

# Scrapybara Ubuntu desktop
python cli.py --computer scrapybara-ubuntu
```

--------------------------------

### Install Dependencies for CUA Sample App

Source: https://github.com/openai/openai-cua-sample-app/blob/main/README.md

This snippet installs the necessary Python dependencies for the Computer Using Agent sample app. It assumes you have Python 3 and pip installed. The `requirements.txt` file should contain all necessary packages, including OpenAI and Playwright.

```shell
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

--------------------------------

### Docker Setup for CUA Sample App

Source: https://github.com/openai/openai-cua-sample-app/blob/main/README.md

Provides shell commands to build and run a Docker container for the CUA sample app. This is necessary for using the 'docker' computer environment. It includes instructions for stopping the container and handling potential naming conflicts.

```shell
docker build -t cua-sample-app .
docker run --rm -t --name cua-sample-app -p 5900:5900 --dns=1.1.1.3 -e DISPLAY=:99 cua-sample-app
```

```shell
docker rm -f cua-sample-app
```

--------------------------------

### Run Simple CUA Loop Example

Source: https://github.com/openai/openai-cua-sample-app/blob/main/README.md

This command executes a basic implementation of the Computer Using Agent loop, showing the fundamental interaction between the agent and the computer environment. It's useful for understanding the core mechanics of CUA.

```shell
python simple_cua_loop.py
```

--------------------------------

### Run Weather Example with CUA

Source: https://github.com/openai/openai-cua-sample-app/blob/main/README.md

This command runs a specific example demonstrating the Computer Using Agent's ability to interact with a weather application. It utilizes the CUA framework to automate tasks within a browser environment.

```shell
python -m examples.weather_example
```

--------------------------------

### Programmatic Agent Usage Example

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

Demonstrates the complete programmatic usage of an agent, including setting up a local computer environment, running single and multi-turn conversations, and processing responses. It uses the `Agent` class and `LocalPlaywrightComputer` for interaction.

```python
from agent import Agent
from computers import LocalPlaywrightComputer

def main():
    with LocalPlaywrightComputer() as computer:
        agent = Agent(
            computer=computer,
            acknowledge_safety_check_callback=lambda msg: True  # Auto-acknowledge
        )

        # Single task
        items = [{"role": "user", "content": "what is the weather in sf"}]
        response_items = agent.run_full_turn(
            items,
            debug=True,
            show_images=True
        )

        final_answer = response_items[-1]["content"][0]["text"]
        print(f"Answer: {final_answer}")

        # Multi-turn conversation
        items.extend(response_items)
        items.append({"role": "user", "content": "Now check Boston"})
        more_responses = agent.run_full_turn(items)

        return items

if __name__ == "__main__":
    conversation = main()

```

--------------------------------

### Custom Remote Browser Computer Implementation

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

Provides an example of implementing a custom computer environment by extending `BasePlaywrightComputer` to connect to a remote browser session. This allows for specialized configurations like custom viewport sizes and remote debugging connections.

```python
from playwright.sync_api import Browser, Page
from computers.shared.base_playwright import BasePlaywrightComputer

class CustomRemoteBrowser(BasePlaywrightComputer):
    def __init__(self, api_key: str, session_id: str = None):
        super().__init__()
        self.api_key = api_key
        self.session_id = session_id

    def get_dimensions(self) -> tuple[int, int]:
        return (1920, 1080)  # Custom viewport size

    def _get_browser_and_page(self) -> tuple[Browser, Page]:
        # Connect to remote browser service
        cdp_url = f"wss://remote-service.com/sessions/{self.session_id}"

        browser = self._playwright.chromium.connect_over_cdp(cdp_url)
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        width, height = self.get_dimensions()
        page.set_viewport_size({"width": width, "height": height})

        return browser, page

# Register in computers/config.py
from .contrib.custom_remote import CustomRemoteBrowser

computers_config = {
    # ... existing configs
    "custom-remote": CustomRemoteBrowser,
}

# Usage
from computers.config import computers_config

ComputerClass = computers_config["custom-remote"]
with ComputerClass(api_key="abc123", session_id="xyz789") as computer:
    agent = Agent(computer=computer)
    # ... use agent

```

--------------------------------

### Run CUA Sample App with Local Playwright

Source: https://github.com/openai/openai-cua-sample-app/blob/main/README.md

This command executes the Computer Using Agent sample application using a local browser controlled by Playwright. It will open a local browser window to interact with. Stop the execution with CTRL+C. Ensure Playwright dependencies are installed.

```shell
python cli.py --computer local-playwright
```

--------------------------------

### Configure New Computer in Python

Source: https://github.com/openai/openai-cua-sample-app/blob/main/README.md

Shows how to register a new computer by adding its class to the `computers_config` dictionary. This step makes the computer accessible and configurable within the application's settings. It requires modifying the `computers/config.py` file.

```python
# ... existing computers_config
"your_computer_name": YourComputerName,
```

--------------------------------

### Run CUA Sample App with Different Computer Environments (CLI)

Source: https://github.com/openai/openai-cua-sample-app/blob/main/README.md

Demonstrates how to execute the CUA sample app using various computer environments via the command-line interface. This allows users to test the app in different contexts.

```shell
python cli.py --show --computer <computer-option>
```

```shell
python cli.py --show --computer docker
```

--------------------------------

### Local Playwright Computer: Browser Automation (Python)

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

Demonstrates browser automation using Playwright, with options for headless (no visible window) and visible browser modes. It covers navigating to URLs, waiting, clicking, typing, sending keypresses, scrolling, taking screenshots, and basic navigation controls.

```python
from computers import LocalPlaywrightComputer

# Headless mode (no visible browser window)
with LocalPlaywrightComputer(headless=True) as computer:
    computer.goto("https://example.com")
    computer.wait(2000)  # Wait 2 seconds
    computer.click(512, 384)  # Click center of 1024x768 viewport
    computer.type("Hello World")
    computer.keypress(["ENTER"])
    computer.scroll(512, 400, 0, -100)  # Scroll up 100 pixels

    # Take screenshot
    screenshot_b64 = computer.screenshot()

    # Get current URL
    current_url = computer.get_current_url()
    print(f"Currently at: {current_url}")

    # Navigation
    computer.back()
    computer.forward()

# Visible browser mode (default)
with LocalPlaywrightComputer(headless=False) as computer:
    computer.goto("https://github.com")
    dimensions = computer.get_dimensions()  # (1024, 768)
    env = computer.get_environment()  # "browser"

```

--------------------------------

### Docker Computer: Linux Desktop Automation (Python)

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

Enables automation of a Linux desktop environment running inside a Docker container via X11 display. This snippet shows how to interact with the desktop, including clicking, double-clicking, typing, mouse movements, dragging, scrolling, and capturing screenshots.

```python
from computers import DockerComputer

# Prerequisites: Docker container must be running
# docker build -t cua-sample-app .
# docker run --rm -it --name cua-sample-app -p 5900:5900 -e DISPLAY=:99 cua-sample-app

with DockerComputer(
    container_name="cua-sample-app",
    display=":99",
    port_mapping="5900:5900"
) as computer:
    dimensions = computer.get_dimensions()  # Actual container display size
    env = computer.get_environment()  # "linux"

    # Click at coordinates
    computer.click(640, 360, button="left")
    computer.double_click(640, 360)

    # Type text
    computer.type("echo 'Hello from Docker'")
    computer.keypress(["ENTER"])

    # Mouse operations
    computer.move(100, 200)
    computer.drag([{"x": 100, "y": 100}, {"x": 200, "y": 200}, {"x": 300, "y": 150}])

    # Scrolling (uses mouse wheel clicks)
    computer.scroll(640, 360, 0, -5)  # Scroll up 5 clicks

    # Get screenshot
    screenshot = computer.screenshot()

```

--------------------------------

### Add Computer Import in Python

Source: https://github.com/openai/openai-cua-sample-app/blob/main/README.md

Demonstrates how to import a newly created computer class into the main application's initialization file. This ensures the new computer is recognized and available for use within the application's framework. It assumes the new computer class is in a file named 'your_computer_name.py' within the same directory.

```python
# ... existing computer imports
from .your_computer_name import YourComputerName
```

--------------------------------

### Implement New Computer Class in Python

Source: https://github.com/openai/openai-cua-sample-app/blob/main/README.md

Defines the basic structure for a new computer class in Python. This includes placeholder methods for initialization, screenshotting, and clicking, serving as a template for custom computer implementations. It requires no external dependencies beyond standard Python.

```python
class YourComputerName:
    def __init__(self):
        pass

    def screenshot(self):
        # TODO: implement
        pass

    def click(self, x, y):
        # TODO: implement
        pass

    # ... add other methods as needed
```

--------------------------------

### Python Computer Protocol Interface Definition

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

Defines the abstract `Computer` protocol interface that all computer environments must implement for CUA compatibility. It includes methods for environment details, screenshots, user interactions like clicking and typing, and waiting. This interface ensures a standardized way for agents to interact with diverse computer environments.

```python
from computers import Computer
from typing import Literal, List, Dict
import time

class CustomComputer:
    def get_environment(self) -> Literal["windows", "mac", "linux", "browser"]:
        return "browser"

    def get_dimensions(self) -> tuple[int, int]:
        return (1024, 768)

    def screenshot(self) -> str:
        # Return base64-encoded PNG screenshot
        return base64_image_string

    def click(self, x: int, y: int, button: str = "left") -> None:
        # Execute click at coordinates
        pass

    def double_click(self, x: int, y: int) -> None:
        pass

    def scroll(self, x: int, y: int, scroll_x: int, scroll_y: int) -> None:
        pass

    def type(self, text: str) -> None:
        pass

    def wait(self, ms: int = 1000) -> None:
        time.sleep(ms / 1000)

    def move(self, x: int, y: int) -> None:
        pass

    def keypress(self, keys: List[str]) -> None:
        # keys like ["CTRL", "C"] or ["ENTER"]
        pass

    def drag(self, path: List[Dict[str, int]]) -> None:
        # path like [{"x": 100, "y": 200}, {"x": 300, "y": 400}]
        pass

    def get_current_url(self) -> str:
        # For browser environments only
        return "https://example.com"
```

--------------------------------

### Interact with OpenAI Responses API for CUA

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

This Python code illustrates direct API calls to OpenAI's Responses endpoint for Conversational User Agent (CUA) tasks. It shows how to make a single request with a computer tool, process the response including 'computer_call' actions and 'message' types, and construct a subsequent request to handle 'computer_call_output'. It relies on the 'utils' library and environment variable for API keys.

```python
from utils import create_response
import os

os.environ["OPENAI_API_KEY"] = "sk-..."

# Single request with computer tool
response = create_response(
    model="computer-use-preview",
    input=[
        {"role": "user", "content": "Click on the search button"}
    ],
    tools=[
        {
            "type": "computer-preview",
            "display_width": 1024,
            "display_height": 768,
            "environment": "browser"
        }
    ],
    truncation="auto"
)

# Response contains output items
output_items = response["output"]
for item in output_items:
    if item["type"] == "computer_call":
        action = item["action"]
        print(f"Action: {action['type']}")
        print(f"Args: {action}")
    elif item["type"] == "message":
        print(f"Message: {item['content'][0]['text']}")

# Handle computer_call_output in next request
if output_items[-1]["type"] == "computer_call":
    next_input = [
        {"role": "user", "content": "Click on the search button"},
        *output_items,
        {
            "type": "computer_call_output",
            "call_id": output_items[-1]["call_id"],
            "acknowledged_safety_checks": [],
            "output": {
                "type": "input_image",
                "image_url": "data:image/png;base64,iVBORw0KGgo...",
                "current_url": "https://example.com"
            }
        }
    ]

    next_response = create_response(
        model="computer-use-preview",
        input=next_input,
        tools=[{"type": "computer-preview", "display_width": 1024, "display_height": 768, "environment": "browser"}],
        truncation="auto"
    )
```

--------------------------------

### Simple CUA Loop: Core Request-Response Pattern (Python)

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

Implements the fundamental request-response cycle of the CUA. It utilizes LocalPlaywrightComputer for browser interactions, handles computer calls, executes actions, and manages safety checks. The output includes screenshots and current URLs for browser environments.

```python
from computers import LocalPlaywrightComputer
from utils import create_response, check_blocklisted_url

def acknowledge_safety_check(message: str) -> bool:
    response = input(f"Safety Check: {message}\nAcknowledge? (y/n): ")
    return response.strip() == "y"

def handle_item(item, computer):
    if item["type"] == "message":
        print(item["content"][0]["text"])

    if item["type"] == "computer_call":
        action = item["action"]
        action_type = action["type"]
        action_args = {k: v for k, v in action.items() if k != "type"}
        print(f"{action_type}({action_args})")

        # Execute action on computer
        getattr(computer, action_type)(**action_args)

        # Get screenshot after action
        screenshot_base64 = computer.screenshot()

        # Handle safety checks
        pending_checks = item.get("pending_safety_checks", [])
        for check in pending_checks:
            if not acknowledge_safety_check(check["message"]):
                raise ValueError(f"Safety check failed: {check['message']}")

        # Build response with screenshot
        call_output = {
            "type": "computer_call_output",
            "call_id": item["call_id"],
            "acknowledged_safety_checks": pending_checks,
            "output": {
                "type": "input_image",
                "image_url": f"data:image/png;base64,{screenshot_base64}",
            }
        }

        # Add current URL for browser environments
        if computer.get_environment() == "browser":
            current_url = computer.get_current_url()
            call_output["output"]["current_url"] = current_url
            check_blocklisted_url(current_url)

        return [call_output]

    return []

# Main loop
with LocalPlaywrightComputer() as computer:
    dimensions = computer.get_dimensions()
    tools = [{
        "type": "computer-preview",
        "display_width": dimensions[0],
        "display_height": dimensions[1],
        "environment": computer.get_environment(),
    }]

    items = []
    while True:
        user_input = input("> ")
        items.append({"role": "user", "content": user_input})

        while True:
            response = create_response(
                model="computer-use-preview",
                input=items,
                tools=tools,
                truncation="auto"
            )

            if "output" not in response:
                raise ValueError("No output from model")

            items += response["output"]

            for item in response["output"]:
                items += handle_item(item, computer)

            # Break when assistant finishes
            if items[-1].get("role") == "assistant":
                break

```

--------------------------------

### Define Custom Function Tools for Agent

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

This Python snippet demonstrates how to define custom tools for an agent, including functions for navigation ('back', 'goto') and data retrieval ('get_weather'). These tools can route to computer methods or return hardcoded values, enabling complex interactions within a conversational agent. It utilizes the 'agent' and 'computers' libraries.

```python
from agent.agent import Agent
from computers import LocalPlaywrightComputer

# Define custom function tools
tools = [
    {
        "type": "function",
        "name": "back",
        "description": "Go back to the previous page.",
        "parameters": {},
    },
    {
        "type": "function",
        "name": "goto",
        "description": "Go to a specific URL.",
        "parameters": {
            "type": "object",
            "properties": {
                "url": {
                    "type": "string",
                    "description": "Fully qualified URL to navigate to.",
                },
            },
            "additionalProperties": False,
            "required": ["url"],
        },
    },
    {
        "type": "function",
        "name": "get_weather",
        "description": "Get weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and state, e.g. San Francisco, CA",
                },
                "unit": {"type": "string", "enum": ["c", "f"]},
            },
            "required": ["location", "unit"],
        },
    }
]

with LocalPlaywrightComputer() as computer:
    agent = Agent(computer=computer, tools=tools)

    items = [
        {
            "role": "developer",
            "content": "Use back() and goto() functions to navigate. Use get_weather() to retrieve weather data."
        }
    ]

    while True:
        user_input = input("> ")
        items.append({"role": "user", "content": user_input})

        # Agent will route back() and goto() to computer methods
        # get_weather() will return hardcoded "success" response
        output_items = agent.run_full_turn(items, show_images=True)
        items += output_items
```

--------------------------------

### Check URL Safety with Blocklisting

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

This Python code demonstrates how to use the `check_blocklisted_url` function from the `utils` library to validate URLs against a predefined list of malicious domains. If a URL is found on the blocklist, a `ValueError` is raised. This functionality is also automatically applied by `BasePlaywrightComputer` and `Agent.handle_item()` in browser environments.

```python
from utils import check_blocklisted_url

# Blocked domains include:
# - maliciousbook.com
# - evilvideos.com
# - darkwebforum.com
# - shadytok.com
# - suspiciouspins.com

try:
    check_blocklisted_url("https://example.com")  # OK
    check_blocklisted_url("https://maliciousbook.com")  # Raises ValueError
    check_blocklisted_url("https://subdomain.evilvideos.com")  # Raises ValueError
except ValueError as e:
    print(f"Blocked: {e}")

# Automatically applied in BasePlaywrightComputer via route interception
# and in Agent.handle_item() for browser environments
```

--------------------------------

### Python Agent Class Implementation for CUA

Source: https://context7.com/openai/openai-cua-sample-app/llms.txt

Implements the core agent loop for the Computer Using Agent (CUA) system. This class handles processing user requests, executing computer actions via a `Computer` instance, and managing the conversation flow with OpenAI's API. It includes safety callback functionality for user confirmation of potentially sensitive actions.

```python
from agent.agent import Agent
from computers import LocalPlaywrightComputer

def acknowledge_safety_callback(message: str) -> bool:
    response = input(f"Safety Warning: {message}\nProceed? (y/n): ")
    return response.lower().strip() == "y"

# Initialize agent with a computer environment
with LocalPlaywrightComputer() as computer:
    agent = Agent(
        model="computer-use-preview",
        computer=computer,
        tools=[],  # Optional: additional function tools
        acknowledge_safety_check_callback=acknowledge_safety_callback
    )

    # Run a single agent turn
    items = [{"role": "user", "content": "Search for weather in San Francisco"}]

    response_items = agent.run_full_turn(
        input_items=items,
        print_steps=True,    # Print actions as they execute
        show_images=True,    # Display screenshots during execution
        debug=False          # Enable debug output
    )

    # Access final assistant response
    final_response = response_items[-1]["content"][0]["text"]
    print(f"Agent: {final_response}")

    # Continue conversation by appending to items
    items.extend(response_items)
    items.append({"role": "user", "content": "Now check New York weather"})
    more_responses = agent.run_full_turn(items)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.