DroidRun

https://docs.droidrun.ai/v3/
Admin
DroidRun is an AI-powered automation framework for controlling Android and iOS devices with...
Tokens:14,003
Snippets:101
Trust Score:7.7
Update:1 year ago
Show doc for...
Tokens:
Raw
### Install LlamaIndex LLM Integrations

Source: https://docs.droidrun.ai/v3/guides/cli

Installs additional LlamaIndex LLM integrations if you need to use a provider not included in DroidRun's extras. This example shows how to install the OpenRouter integration.

```sh
pip install llama-index-llms-openrouter
```

--------------------------------

### Install DroidRun and Dependencies

Source: https://docs.droidrun.ai/v3/guides/cli

Installs DroidRun along with optional dependencies for various LLM providers like Google, Anthropic, OpenAI, DeepSeek, and Ollama. Ensure you have Python and pip installed.

```sh
pip install 'droidrun[google,anthropic,openai,deepseek,ollama,dev]'
```

--------------------------------

### Setup DroidRun Portal and Ping Device

Source: https://docs.droidrun.ai/v3/guides/cli

Sets up the DroidRun Portal on your Android device and verifies the connection. This involves downloading and installing the APK, followed by a ping command to check device accessibility.

```sh
droidrun setup
droidrun ping
```

--------------------------------

### Manage Android Applications

Source: https://docs.droidrun.ai/v3/concepts/android-tools

Provides examples for managing Android applications using AdbTools. This includes starting an app by package name, listing installed packages (optionally including system apps), and installing an APK file.

```python
# App management examples
await tools.start_app("com.android.settings")
packages = await tools.list_packages()
all_packages = await tools.list_packages(include_system_apps=True)
await tools.install_app("/path/to/app.apk", reinstall=True)
```

--------------------------------

### Install Droidrun with OpenAI Support

Source: https://docs.droidrun.ai/v3/guides/openailike

Installs the Droidrun framework with the necessary dependencies for OpenAI-like LLM integration.

```shell
pip install 'droidrun[openai]'
```

--------------------------------

### Start Application

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Starts an application on the iOS device using its bundle identifier.

```APIDOC
## Start Application

### Description
Starts an application on the iOS device using its bundle identifier.

### Method
`start_app`

### Endpoint
`/start_app`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **package** (str) - Required - The bundle identifier of the application to start (e.g., "com.apple.MobileSMS").
- **activity** (str) - Optional - Activity name (not used on iOS).

### Request Example
```json
{
  "package": "com.apple.Maps"
}
```

### Response
#### Success Response (200)
- **str** - A result message indicating the success or failure of starting the application.

### Response Example
```json
{
  "message": "Successfully started application com.apple.Maps."
}
```
```

--------------------------------

### Install Droidrun with Google Support

Source: https://docs.droidrun.ai/v3/guides/gemini

Installs the droidrun framework with necessary dependencies for Google AI integration. This command ensures that all required libraries for connecting to Google services, such as Gemini, are included.

```shell
pip install 'droidrun[google]'
```

--------------------------------

### Run Ollama Server and Pull Models

Source: https://docs.droidrun.ai/v3/guides/ollama

Commands to start the Ollama local LLM server and download specific language models. Ensure Ollama is installed and running before proceeding with model downloads.

```shell
ollama serve

```

```shell
ollama pull qwen2.5vl

```

```shell
ollama pull gemma3

```

```shell
ollama pull deepseek-r1

```

```shell
ollama pull llama4
```

--------------------------------

### App Management Tools

Source: https://docs.droidrun.ai/v3/concepts/android-tools

Tools for managing applications on the Android device, including starting, listing, and installing apps.

```APIDOC
## App Management Tools

### `start_app(package: str, activity: str = "")`

Start application by package name and optional activity.

### Method
POST

### Endpoint
/tools/start_app

### Parameters
#### Request Body
- **package** (string) - Required - The package name of the application to start.
- **activity** (string) - Optional - The specific activity to launch within the application.

### Request Example
```json
{
  "package": "com.android.settings"
}
```

### Response
#### Success Response (200)
- **message** (string) - A status message indicating the result of starting the app.

#### Response Example
```json
{
  "message": "Successfully started application com.android.settings."
}
```

---

### `list_packages(include_system_apps: bool = False)`

List installed packages. Optionally include system apps.

### Method
GET

### Endpoint
/tools/list_packages

### Parameters
#### Query Parameters
- **include_system_apps** (boolean) - Optional - Whether to include system applications in the list. Defaults to false.

### Request Example
```json
/tools/list_packages?include_system_apps=true
```

### Response
#### Success Response (200)
- **packages** (array of strings) - A list of installed package names.

#### Response Example
```json
{
  "packages": ["com.android.settings", "com.example.app"]
}
```

---

### `install_app(apk_path: str, reinstall: bool = False, grant_permissions: bool = True)`

Install APK file with optional reinstall and permission granting.

### Method
POST

### Endpoint
/tools/install_app

### Parameters
#### Request Body
- **apk_path** (string) - Required - The file path to the APK to install.
- **reinstall** (boolean) - Optional - Whether to reinstall the app if it's already installed. Defaults to false.
- **grant_permissions** (boolean) - Optional - Whether to automatically grant permissions during installation. Defaults to true.

### Request Example
```json
{
  "apk_path": "/path/to/app.apk",
  "reinstall": true
}
```

### Response
#### Success Response (200)
- **message** (string) - A status message indicating the result of the app installation.

#### Response Example
```json
{
  "message": "Successfully installed application from /path/to/app.apk."
}
```
```

--------------------------------

### GET /list_packages

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Retrieves a list of all installed packages on the device.

```APIDOC
## GET /list_packages

### Description
Lists all installed application packages on the device. You can choose to include system applications in the results.

### Method
GET

### Endpoint
/list_packages

### Parameters
#### Query Parameters
- **include_system_apps** (boolean) - Optional - If true, system applications will be included in the list. Defaults to false.

### Request Example
```json
{
  "include_system_apps": true
}
```

### Response
#### Success Response (200)
- **packages** (List[string]) - A list of package names installed on the device.

#### Response Example
```json
{
  "packages": [
    "com.example.app1",
    "com.android.settings",
    "com.google.android.youtube"
  ]
}
```
```

--------------------------------

### Control Application Launch

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Functions to start an application by its package name and optionally a specific activity, and to install applications onto the device.

```python
def start_app(package: str, activity: str | None = None) -> str:
    """Start an app on the device."""
    pass

def install_app(path: str) -> str:
    """Install an app on the device."""
    pass
```

--------------------------------

### Start Application (Python)

Source: https://docs.droidrun.ai/v3/sdk/base-tools

The 'start_app' function launches a specified application identified by its package and activity name. It returns a string indicating the launch status.

```python
def start_app(package: str, activity: str = "") -> str:
    pass
```

--------------------------------

### Install Droidrun with Ollama Support

Source: https://docs.droidrun.ai/v3/guides/ollama

Installs the Droidrun framework with the necessary dependencies to support Ollama integration. This command ensures that all required Python packages for connecting Droidrun to Ollama are available in your environment.

```shell
pip install 'droidrun[ollama]'
```

--------------------------------

### POST /install_app

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Installs an application on the device. Supports optional reinstallation and permission granting.

```APIDOC
## POST /install_app

### Description
Installs an application from a given APK path onto the device. It can optionally reinstall the app if it already exists and grant all necessary permissions.

### Method
POST

### Endpoint
/install_app

### Parameters
#### Request Body
- **apk_path** (string) - Required - Path to the APK file to install.
- **reinstall** (boolean) - Optional - If true, the app will be reinstalled if it already exists on the device. Defaults to false.
- **grant_permissions** (boolean) - Optional - If true, all requested permissions for the app will be granted. Defaults to true.

### Request Example
```json
{
  "apk_path": "/path/to/your/app.apk",
  "reinstall": true,
  "grant_permissions": true
}
```

### Response
#### Success Response (200)
- **result** (string) - A confirmation message indicating the success of the installation.

#### Response Example
```json
{
  "result": "App installed successfully."
}
```
```

--------------------------------

### List Installed Packages

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Retrieves a list of installed application package names on the iOS device.

```APIDOC
## List Installed Packages

### Description
Retrieves a list of installed application package names on the iOS device.

### Method
`list_packages`

### Endpoint
`/list_packages`

### Parameters
#### Path Parameters
None

#### Query Parameters
- **include_system_apps** (bool) - Optional - Whether to include system applications in the list. Defaults to true.

#### Request Body
None

### Request Example
```json
{
  "include_system_apps": false
}
```

### Response
#### Success Response (200)
- **List[str]** - A list of strings, where each string is a package name (bundle identifier) of an installed application.

### Response Example
```json
{
  "packages": [
    "com.apple.Maps",
    "com.apple.MobileSMS",
    "com.example.MyApp"
  ]
}
```
```

--------------------------------

### List Installed Packages (Python)

Source: https://docs.droidrun.ai/v3/sdk/base-tools

The 'list_packages' function retrieves a list of all installed packages on the device. It can optionally include system applications.

```python
def list_packages(include_system_apps: bool = False) -> List[str]:
    pass
```

--------------------------------

### Droidrun Agent with OpenAI-like LLM Example

Source: https://docs.droidrun.ai/v3/guides/openailike

A Python script demonstrating how to set up and run a DroidAgent using an OpenAI-compatible LLM. It loads ADB tools, configures the LLM (OpenAILike), creates the agent with a specified goal, and executes it. Handles potential output and success status.

```python
import asyncio
from llama_index.llms.openai_like import OpenAILike
from droidrun import DroidAgent, AdbTools

async def main():
    # Load adb tools for the first connected device
    tools = AdbTools()

    # Set up the OpenAI-like LLM (uses env vars for API key and base by default)
    llm = OpenAILike(
        model="gpt-3.5-turbo",  # or "gpt-4o", "gpt-4", etc.
        api_base="http://localhost:1234/v1",  # For local endpoints
        is_chat_model=True, # droidrun requires chat model support
        api_key="YOUR API KEY"
    )

    # Create the DroidAgent
    agent = DroidAgent(
        goal="Open Settings and check battery level",
        llm=llm,
        tools=tools,
        vision=False,        # Set to True if your model supports vision
        reasoning=False,     # Optional: enable planning/reasoning
    )

    # Run the agent
    result = await agent.run()
    print(f"Success: {result['success']}")
    if result.get('output'):
        print(f"Output: {result['output']}")

if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### Install LLM Provider Package - Bash

Source: https://docs.droidrun.ai/v3/concepts/models

This command installs the necessary LlamaIndex package for a specified LLM provider. Replace '[PROVIDER]' with the actual provider name (e.g., 'openai', 'anthropic'). This is a prerequisite for using a particular LLM provider with DroidRun.

```bash
pip install llama-index-llms-[PROVIDER]
```

--------------------------------

### Start Application on iOS Device

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Launches a specified application on the iOS device using its bundle identifier. An optional activity name can be provided, though it's not typically used on iOS.

```python
def start_app(package: str, activity: str = "") -> str:
    """Start an app on the iOS device."""
    pass
```

--------------------------------

### List Installed Packages on iOS

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Retrieves a list of installed application package names (bundle identifiers) on the iOS device. System apps can be included or excluded based on the `include_system_apps` flag.

```python
def list_packages(include_system_apps: bool = True) -> List[str]:
    """"""
```

--------------------------------

### Install App on Android Device (Python)

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Installs an application from an APK file onto an Android device. Supports reinstallation and automatic permission granting. Requires the path to the APK file as input.

```Python
def install_app(
    apk_path: str,
    reinstall: bool = False,
    grant_permissions: bool = True
) -> str:
    # Function implementation here
```

--------------------------------

### Python Example: Droidrun Agent with Ollama LLM

Source: https://docs.droidrun.ai/v3/guides/ollama

A Python script demonstrating how to set up and run a Droidrun agent using Ollama as the LLM backend. It initializes ADB tools, configures the Ollama LLM with a specified model and parameters, and then runs the agent with a defined goal. Requires Ollama to be running and accessible.

```python
import asyncio
from llama_index.llms.ollama import Ollama
from droidrun import DroidAgent, AdbTools

async def main():
    # load adb tools for the first connected device
    tools = AdbTools()

    # Set up the Ollama LLM with a modern model
    llm = Ollama(
        model="qwen2.5vl",                  # or "gemma3", "deepseek", "llama4", etc.
        base_url="http://localhost:11434",  # default Ollama endpoint
        context_window=8192,                # limit the max context window to prevent running out of memory. 
        request_timeout=120.0               # increase the request timeout
    )

    # Create the DroidAgent
    agent = DroidAgent(
        goal="Open Settings and check battery level",
        llm=llm,
        tools=tools,
        vision=False,         # Optional: enable vision. use vision=True only in conjunction with a vision model
        reasoning=True,       # Optional: enable planning/reasoning. Read more about the agent configuration in Core-Concepts/Agent
    )

    # Run the agent
    result = await agent.run()
    print(f"Success: {result['success']}")
    if result.get('output'):
        print(f"Output: {result['output']}")

if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### Run Command with OpenAILike Provider

Source: https://docs.droidrun.ai/v3/guides/cli

Executes a command on an Android device using the OpenAILike LLM provider. Requires setting the OPENAI_API_KEY environment variable and specifying the provider, model, and API base URL.

```sh
export OPENAI_API_KEY=<your-api-key>
droidrun run \
  --provider OpenAILike \
  --model qwen/qwen2.5-vl-72b-instruct:free \
  --api_base https://openrouter.ai/api/v1 \
  "Open the settings app"
```

--------------------------------

### Run Command with Gemini Provider

Source: https://docs.droidrun.ai/v3/guides/cli

Executes a command on an Android device using the Gemini LLM provider. Requires setting the GOOGLE_API_KEY environment variable and specifying the provider and model name.

```sh
export GOOGLE_API_KEY=<your-google-api-key>
droidrun run \
  --provider GoogleGenAI \
  --model gemini-2.5-flash \
  "Open the settings app" 
```

--------------------------------

### List Installed Packages on Android (Python)

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Retrieves a list of installed package names on an Android device. Optionally includes system applications. Returns a list of strings, where each string is a package name.

```Python
def list_packages(include_system_apps: bool = False) -> List[str]:
    # Function implementation here
```

--------------------------------

### Python: Droidrun Agent with Gemini LLM

Source: https://docs.droidrun.ai/v3/guides/gemini

Demonstrates a basic Droidrun agent setup using Google's Gemini LLM for Android automation. It initializes ADB tools, configures the Gemini LLM with an API key and model, and creates a DroidAgent to execute a specified goal, with optional vision and reasoning capabilities.

```python
import asyncio
from llama_index.llms.google_genai import GoogleGenAI
from droidrun import DroidAgent, AdbTools

async def main():
    # load adb tools for the first connected device
    tools = AdbTools()

    # Set up the Gemini LLM
    llm = GoogleGenAI(
        api_key="YOUR_GEMINI_API_KEY",  # Replace with your Gemini API key
        model="gemini-2.5-flash",  # or "gemini-2.5-pro" for enhanced reasoning
    )

    # Create the DroidAgent
    agent = DroidAgent(
        goal="Open Settings and check battery level",
        llm=llm,
        tools=tools,
        vision=True,         # Set to True for vision models, False for text-only
        reasoning=False,      # Optional: enable planning/reasoning
    )

    # Run the agent
    result = await agent.run()
    print(f"Success: {result['success']}")
    if result.get('output'):
        print(f"Output: {result['output']}")

if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### Run Command with Ollama Provider

Source: https://docs.droidrun.ai/v3/guides/cli

Executes a command on an Android device using the Ollama LLM provider for local model execution. Requires specifying the provider and model name. The base URL defaults to http://localhost:11434.

```sh
droidrun run \
  --provider Ollama \
  --model qwen2.5vl:3b \
  --base_url http://localhost:11434 \
  "Open the settings app"
```

--------------------------------

### Run Command with DeepSeek Provider

Source: https://docs.droidrun.ai/v3/guides/cli

Executes a command on an Android device using the DeepSeek LLM provider. Requires setting the DEEPSEEK_API_KEY environment variable and specifying the provider and model name. Note that vision capabilities are not supported.

```sh
export DEEPSEEK_API_KEY=<your-deepseek-api-key>
droidrun run \
  --provider DeepSeek \
  --model deepseek-chat \
  "Open the settings app"
```

--------------------------------

### GET /ping

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Tests the TCP connection to the device.

```APIDOC
## GET /ping

### Description
Tests the TCP connection to the device by sending a ping request via the `/ping` endpoint. This is useful for verifying connectivity.

### Method
GET

### Endpoint
/ping

### Parameters
No parameters required.

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **ping_result** (Dict[str, Any]) - A dictionary containing the result of the ping test.

#### Response Example
```json
{
  "ping_result": {
    "status": "ok",
    "message": "Device is reachable."
  }
}
```
```

--------------------------------

### Get Clickable UI Elements on iOS

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Retrieves all clickable UI elements from the iOS device screen using the accessibility API. The output is a list of dictionaries, where each dictionary represents a UI element.

```python
def get_state() -> List[Dict[str, Any]]:
    """Get all clickable UI elements from the iOS device using accessibility API."""
    pass
```

--------------------------------

### GET /get_state

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Fetches both the accessibility tree and the phone state in a single call.

```APIDOC
## GET /get_state

### Description
Retrieves a comprehensive snapshot of the device's current state by fetching both the accessibility tree and general phone state information simultaneously.

### Method
GET

### Endpoint
/get_state

### Parameters
#### Query Parameters
- **serial** (string) - Optional - The serial number of the device to query. If not provided, the default device is used.

### Request Example
```json
{
  "serial": "emulator-5554"
}
```

### Response
#### Success Response (200)
- **a11y_tree** (Dict[str, Any]) - The accessibility tree data of the device.
- **phone_state** (Dict[str, Any]) - The general state information of the phone.

#### Response Example
```json
{
  "a11y_tree": {
    "nodes": [...]
  },
  "phone_state": {
    "battery": 85,
    "signal": "4G"
  }
}
```
```

--------------------------------

### GET /get_memory

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Retrieves all previously stored memory items.

```APIDOC
## GET /get_memory

### Description
Retrieves a list of all information items that have been previously stored using the `/remember` endpoint.

### Method
GET

### Endpoint
/get_memory

### Parameters
No parameters required.

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **memory_items** (List[string]) - A list containing all stored memory items.

#### Response Example
```json
{
  "memory_items": [
    "The user prefers dark mode.",
    "Device model is Pixel 6."
  ]
}
```
```

--------------------------------

### GET /get_a11y_tree

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Retrieves only the accessibility tree of the device.

```APIDOC
## GET /get_a11y_tree

### Description
Fetches the accessibility tree data for the device using the `/a11y_tree` endpoint. This provides detailed information about UI elements and their properties for accessibility purposes.

### Method
GET

### Endpoint
/a11y_tree

### Parameters
No parameters required.

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **a11y_tree** (Dict[str, Any]) - A dictionary containing the accessibility tree data.

#### Response Example
```json
{
  "a11y_tree": {
    "nodes": [
      {
        "id": "1",
        "text": "Home screen",
        "children": [...] 
      }
    ]
  }
}
```
```

--------------------------------

### Get UI Element State

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Retrieves all clickable UI elements from the iOS device's screen using the accessibility API.

```APIDOC
## Get UI Element State

### Description
Retrieves all clickable UI elements from the iOS device's screen using the accessibility API.

### Method
`get_state`

### Endpoint
`/get_state`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **List[Dict[str, Any]]** - A list of dictionaries, where each dictionary represents a clickable UI element extracted from the device screen.

### Response Example
```json
[
  {
    "index": 0,
    "label": "Settings",
    "type": "Button",
    "frame": {
      "x": 10, "y": 10, "width": 100, "height": 50
    }
  },
  {
    "index": 1,
    "label": "App Store",
    "type": "Button",
    "frame": {
      "x": 120, "y": 10, "width": 100, "height": 50
    }
  }
]
```
```

--------------------------------

### Get Current Tool State (Python)

Source: https://docs.droidrun.ai/v3/sdk/base-tools

The 'get_state' function retrieves the current state of a tool. This is useful for understanding the tool's current configuration or status.

```python
def get_state() -> Dict[str, Any]:
    pass
```

--------------------------------

### GET /get_phone_state

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Retrieves only the general phone state information.

```APIDOC
## GET /get_phone_state

### Description
Fetches the general state of the phone using the `/phone_state` endpoint. This includes information such as battery level, network status, and other relevant device details.

### Method
GET

### Endpoint
/phone_state

### Parameters
No parameters required.

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **phone_state** (Dict[str, Any]) - A dictionary containing the phone state information.

#### Response Example
```json
{
  "phone_state": {
    "battery": 75,
    "network": "WiFi",
    "carrier": "Example Carrier",
    "screen_on": true
  }
}
```
```

--------------------------------

### Perform Drag Gesture on iOS Screen

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Simulates a drag gesture on the iOS device screen, moving from a specified start coordinate to an end coordinate over a defined duration.

```python
def drag(
    start_x: int,
    start_y: int,
    end_x: int,
    end_y: int,
    duration_ms: int = 3000
) -> bool:
    """Drag from the given start coordinates to the given end coordinates."""
    pass
```

--------------------------------

### Perform Drag Gesture

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Drags from the given start coordinates to the given end coordinates with a specified duration.

```APIDOC
## Perform Drag Gesture

### Description
Drags from the given start coordinates to the given end coordinates with a specified duration.

### Method
`drag`

### Endpoint
`/drag`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **start_x** (int) - Required - Starting X coordinate.
- **start_y** (int) - Required - Starting Y coordinate.
- **end_x** (int) - Required - Ending X coordinate.
- **end_y** (int) - Required - Ending Y coordinate.
- **duration_ms** (int) - Optional - Duration of drag in milliseconds.

### Request Example
```json
{
  "start_x": 200,
  "start_y": 600,
  "end_x": 400,
  "end_y": 600,
  "duration_ms": 1000
}
```

### Response
#### Success Response (200)
- **bool** - Indicates whether the drag gesture was successful or not.

### Response Example
```json
{
  "success": true
}
```
```

--------------------------------

### Get Memory Items using Python IOSTools

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Retrieves all stored memory items. This function returns a list of strings, where each string represents a stored memory item. It has no input arguments.

```python
def get_memory() -> List[str]:
    # Implementation details to retrieve memory items
    pass
```

--------------------------------

### Analyze Android Device State

Source: https://docs.droidrun.ai/v3/concepts/android-tools

Shows how to retrieve and analyze the state of an Android device using AdbTools. This involves getting the comprehensive device state, including the accessibility tree and phone state, and capturing screenshots.

```python
# State analysis examples
state = await tools.get_state()
a11y_tree = state["a11y_tree"]
phone_state = state["phone_state"]

format, image_data = await tools.take_screenshot()
```

--------------------------------

### Perform Swipe Gesture

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Performs a straight-line swipe gesture on the device screen. A hold (long press) can be simulated by setting start and end coordinates to the same values and increasing the duration.

```APIDOC
## Perform Swipe Gesture

### Description
Performs a straight-line swipe gesture on the device screen. To perform a hold (long press), set the start and end coordinates to the same values and increase the duration as needed.

### Method
`swipe`

### Endpoint
`/swipe`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **start_x** (int) - Required - Starting X coordinate.
- **start_y** (int) - Required - Starting Y coordinate.
- **end_x** (int) - Required - Ending X coordinate.
- **end_y** (int) - Required - Ending Y coordinate.
- **duration_ms** (int) - Optional - Duration of swipe in milliseconds (not used in iOS API).

### Request Example
```json
{
  "start_x": 100,
  "start_y": 500,
  "end_x": 100,
  "end_y": 100,
  "duration_ms": 500
}
```

### Response
#### Success Response (200)
- **bool** - Indicates whether the swipe gesture was successful or not.

### Response Example
```json
{
  "success": true
}
```
```

--------------------------------

### Get Android Accessibility Tree (Python)

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Fetches only the accessibility tree data from the Android device using the dedicated '/a11y_tree' endpoint. Returns a dictionary containing the accessibility tree structure.

```Python
def get_a11y_tree() -> Dict[str, Any]:
    # Function implementation here
```

--------------------------------

### Perform Swipe Gesture on iOS Screen

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Executes a straight-line swipe gesture on the iOS device screen between specified coordinates. A long press can be simulated by setting identical start and end coordinates with an increased duration.

```python
def swipe(
    start_x: int,
    start_y: int,
    end_x: int,
    end_y: int,
    duration_ms: int = 300
) -> bool:
    """Performs a straight-line swipe gesture on the device screen."""
    pass
```

--------------------------------

### Get Android Phone State (Python)

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Retrieves only the phone state information from the Android device using the '/phone_state' endpoint. Returns a dictionary containing the current phone state details.

```Python
def get_phone_state() -> Dict[str, Any]:
    # Function implementation here
```

--------------------------------

### Get Android Device State (Python)

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Retrieves both the accessibility tree and phone state information in a single API call. Supports specifying a device serial number. Returns a dictionary containing 'a11y_tree' and 'phone_state'.

```Python
def get_state(serial: Optional[str] = None) -> Dict[str, Any]:
    # Function implementation here
```

--------------------------------

### IOSTools Initialization

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Initializes the IOSTools instance with the device URL and an optional list of bundle identifiers.

```APIDOC
## IOSTools Initialization

### Description
Initializes the IOSTools instance with the device URL and an optional list of bundle identifiers.

### Method
`__init__`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **url** (str) - Required - iOS device URL.
- **bundle_identifiers** (List[str]) - Optional - List of bundle identifiers to include in the list of packages.

### Request Example
```json
{
  "url": "http://localhost:8100",
  "bundle_identifiers": ["com.example.app1", "com.example.app2"]
}
```

### Response
None

### Response Example
None
```

--------------------------------

### Initialize IOSTools for iOS Device Control

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Initializes the IOSTools instance for controlling an iOS device. It requires the device URL and an optional list of bundle identifiers to specify which packages to consider.

```python
def __init__(url: str, bundle_identifiers: List[str] = []) -> None:
    """Initialize the IOSTools instance."""
    pass
```

--------------------------------

### Initialize Droid Agent and Tools

Source: https://docs.droidrun.ai/v3/concepts/android-tools

This snippet shows how to initialize the AdbTools for a specific device and then create a DroidAgent with these tools. The agent is configured with a goal and a language model.

```python
from droidrun import DroidAgent, AdbTools

# Load tools for a device
tools = AdbTools(serial="device_serial")

# Create agent with tools
agent = DroidAgent(
    goal="Your task",
    llm=llm,
    tools=tools
)
```

--------------------------------

### DroidAgent Configuration Parameters

Source: https://docs.droidrun.ai/v3/concepts/agent

Defines the constructor for the DroidAgent, outlining parameters such as the agent's goal, the language model (LLM) to use, available tools, persona settings, step limits, timeouts, and options for enabling vision, reasoning, reflection, tracing, and debugging.

```python
def __init__(
    self, 
    goal: str,                                  # The goal for the agent to reach
    llm: LLM,                                   # Language model to use
    tools: Tools,                               # Loaded tools
    personas: List[AgentPersona] = [DEFAULT],   # Experimental: custom system prompt for agent
    max_steps: int = 15,                        # Maximum steps the agent takes
    timeout: int = 1000,                        # Global Timeout
    vision: bool = False,                       # Whether the agent shall also utilize screenshots
    reasoning: bool = False,                    # Enable reasoning
    reflection: bool = False,                   # Enable reflection
    enable_tracing: bool = False,               # Enable tracing (this requires arize phoenix)
    debug: bool = False,                        # Enable additional debug logs
    save_trajectories: str = "none",            # Trajectory saving level: "none" (no saving), "step" (save per step), "action" (save per action)
    *args,
    **kwargs
)
```

--------------------------------

### Initialize DroidAgent Wrapper (Python)

Source: https://docs.droidrun.ai/v3/sdk/droid-agent

Initializes the DroidAgent, a wrapper class that orchestrates task planning and execution on an Android device. It requires a goal, an LLM, and tools, with optional parameters for controlling reasoning, reflection, tracing, and debugging.

```python
class DroidAgent(Workflow):
    def __init__(
        self,
        goal: str,
        llm: LLM,
        tools: Tools,
        personas: List[AgentPersona] = [DEFAULT],
        max_steps: int = 15,
        timeout: int = 1000,
        vision: bool = False,
        reasoning: bool = False,
        reflection: bool = False,
        enable_tracing: bool = False,
        debug: bool = False,
        save_trajectories: str = "none",
        excluded_tools: List[str] = None,
        *args,
        **kwargs
    ):
        # ... implementation details ...
        pass
```

--------------------------------

### Describe Available Tools (Python)

Source: https://docs.droidrun.ai/v3/sdk/base-tools

The 'describe_tools' function generates a dictionary mapping tool names to their callable functions for a given Tools instance. It allows excluding specific tools.

```python
def describe_tools(
    tools: Tools,
    exclude_tools: Optional[List[str]] = None
) -> Dict[str, Callable[..., Any]]:
    pass
```

--------------------------------

### Press Key on iOS Device

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Simulates a key press on the iOS device using predefined keycodes. Supported keycodes include HOME, ACTION, and CAMERA.

```python
def press_key(keycode: int) -> str:
    """Press a key on the iOS device."""
    pass
```

--------------------------------

### DroidAgent Planning Mode Initialization

Source: https://docs.droidrun.ai/v3/concepts/agent

Initializes the DroidAgent in Planning Mode for multi-step tasks that require navigation and decision-making. This mode has medium reasoning and medium speed, with reasoning enabled.

```python
# Multi-step tasks requiring navigation and decision-making
agent = DroidAgent(
    goal="Set up a new alarm for 7 AM with custom ringtone and label 'Work'",
    llm=llm,
    tools=tools,
    reasoning=True
)
```

--------------------------------

### Navigate Back

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Performs the back navigation action on the iOS device.

```APIDOC
## Navigate Back

### Description
Performs the back navigation action on the iOS device.

### Method
`back`

### Endpoint
`/back`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **str** - A result message indicating the success or failure of the back navigation.

### Response Example
```json
{
  "message": "Navigated back successfully."
}
```
```

--------------------------------

### DroidAgent Reflection Mode Initialization

Source: https://docs.droidrun.ai/v3/concepts/agent

Initializes the DroidAgent in Reflection Mode for complex, multi-app workflows that necessitate verification and adaptive planning. This mode offers high reasoning but lower speed, with both reasoning and reflection enabled.

```python
# Complex, multi-app workflows requiring verification and adaptive planning
agent = DroidAgent(
    goal="Find the cheapest hotel in Manhattan for next weekend, compare prices across multiple booking apps, and share the best option with my team on Slack",
    llm=llm,
    tools=tools,
    reasoning=True,
    reflection=True
)
```

--------------------------------

### Press Key

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Presses a specified key on the iOS device.

```APIDOC
## Press Key

### Description
Presses a specified key on the iOS device.

### Method
`press_key`

### Endpoint
`/press_key`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **keycode** (int) - Required - The iOS keycode to press. Keycodes include: 0 (HOME), 4 (ACTION), 5 (CAMERA).

### Request Example
```json
{
  "keycode": 0
}
```

### Response
#### Success Response (200)
- **str** - A result message indicating the success or failure of the key press.

### Response Example
```json
{
  "message": "Successfully pressed HOME key."
}
```
```

--------------------------------

### Take Screenshot

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Captures a screenshot of the current iOS device screen. The screenshot is added to context and stored for potential GIF creation.

```APIDOC
## Take Screenshot

### Description
Captures a screenshot of the current iOS device screen. The screenshot is added to context and stored for potential GIF creation.

### Method
`take_screenshot`

### Endpoint
`/take_screenshot`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **Tuple[str, bytes]** - A tuple containing a message string and the raw bytes of the screenshot image.

### Response Example
```json
{
  "message": "Screenshot taken successfully.",
  "screenshot_data": "<base64_encoded_image_data>"
}
```
```

--------------------------------

### Simulate Key Press by Keycode (Python)

Source: https://docs.droidrun.ai/v3/sdk/base-tools

The 'press_key' function simulates a key press using a specified keycode. It returns a string indicating the result of the key press.

```python
def press_key(keycode: int) -> str:
    pass
```

--------------------------------

### Tap UI Element by Index (Python)

Source: https://docs.droidrun.ai/v3/sdk/base-tools

The 'tap_by_index' function simulates a tap action on a specific UI element identified by its index. It returns a string indicating the result of the action.

```python
def tap_by_index(index: int) -> str:
    pass
```

--------------------------------

### Capture UI Screenshots with ui_action Decorator (Python)

Source: https://docs.droidrun.ai/v3/sdk/base-tools

The 'ui_action' decorator is used to automatically capture screenshots and UI states for methods that modify the user interface. This helps in debugging and state tracking.

```python
def ui_action(func):
    pass
```

--------------------------------

### Initialize AdbTools

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Initializes an AdbTools instance for controlling an Android device. Supports specifying the device serial number and configuring TCP communication.

```python
def __init__(
    serial: str | None = None,
    use_tcp: bool = False,
    tcp_port: int = 8080
) -> None:
```

--------------------------------

### Enable Vision in DroidAgent Configuration

Source: https://docs.droidrun.ai/v3/concepts/agent

This code snippet demonstrates how to initialize a DroidAgent with vision capabilities enabled. Setting `vision=True` allows the agent to process visual information, which is crucial for tasks involving image or video content analysis and visual verification.

```python
agent = DroidAgent(
    goal="Open up TikTok and describe the content of the video you are seeing",
    llm=llm,
    tools=tools,
    vision=True
)
```

--------------------------------

### Navigate Back on iOS Device

Source: https://docs.droidrun.ai/v3/sdk/ios-tools

Performs a back navigation action on the iOS device. This is equivalent to pressing the back button or swiping back if applicable.

```python
def back() -> str:
    """"""
```

--------------------------------

### POST /take_screenshot

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Captures a screenshot of the current device screen.

```APIDOC
## POST /take_screenshot

### Description
Takes a screenshot of the device's current screen. The screenshot is added to the message context and also stored for potential GIF creation.

### Method
POST

### Endpoint
/take_screenshot

### Parameters
No parameters required.

### Request Example
```json
{}
```

### Response
#### Success Response (200)
- **screenshot_path** (string) - The file path to the saved screenshot.
- **screenshot_data** (bytes) - The raw binary data of the screenshot.

#### Response Example
```json
{
  "screenshot_path": "/path/to/screenshot.png",
  "screenshot_data": "...binary image data..."
}
```
```

--------------------------------

### Simulate Back Button Press (Python)

Source: https://docs.droidrun.ai/v3/sdk/base-tools

The 'back' function simulates a press of the device's back button. It returns a string representing the action's result.

```python
def back() -> str:
    pass
```

--------------------------------

### POST /complete

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Marks the current task as finished, indicating success or failure.

```APIDOC
## POST /complete

### Description
Signals the completion of a task, specifying whether it was successful and providing a reason if necessary.

### Method
POST

### Endpoint
/complete

### Parameters
#### Request Body
- **success** (boolean) - Required - Indicates whether the task was successful.
- **reason** (string) - Optional - A message explaining the reason for success or failure.

### Request Example
```json
{
  "success": true,
  "reason": "Task completed successfully."
}
```

### Response
#### Success Response (200)
- **message** (string) - A confirmation message.

#### Response Example
```json
{
  "message": "Task completion acknowledged."
}
```
```

--------------------------------

### DroidAgent Direct Execution Mode Initialization

Source: https://docs.droidrun.ai/v3/concepts/agent

Initializes the DroidAgent in Direct Execution mode, suitable for simple, single-action tasks. This mode is characterized by low reasoning and high speed, with reasoning set to False.

```python
# Simple tasks
agent = DroidAgent(
    goal="Take a screenshot of the current screen",
    llm=llm,
    tools=tools,
    reasoning=False
)
```

--------------------------------

### POST /remember

Source: https://docs.droidrun.ai/v3/sdk/adb-tools

Stores a piece of information to be recalled later for context.

```APIDOC
## POST /remember

### Description
Stores specific information that the system should remember and use to maintain context in future interactions. This is useful for critical facts, observations, or user preferences.

### Method
POST

### Endpoint
/remember

### Parameters
#### Request Body
- **information** (string) - Required - The information to be stored and remembered.

### Request Example
```json
{
  "information": "The user prefers dark mode."
}
```

### Response
#### Success Response (200)
- **confirmation** (string) - A message confirming that the information has been stored.

#### Response Example
```json
{
  "confirmation": "Information stored successfully."
}
```
```

--------------------------------

### Take Device Screenshot (Python)

Source: https://docs.droidrun.ai/v3/sdk/base-tools

The 'take_screenshot' function captures the current screen of the device. It returns a tuple containing the screenshot path and its byte data.

```python
def take_screenshot() -> Tuple[str, bytes]:
    pass
```
Tokens:
Raw