### Start Activity and Manage Object Store - Python Source: https://context7.com/dtmilano/androidviewclient/llms.txt Demonstrates how to start Android activities using package and class names, or by providing a URI. It also shows how to manage objects within the object store, including listing, clearing, and removing specific objects. ```Python from com.dtmilano.android.viewclient import ViewClient vc = ViewClient(*ViewClient.connectToDeviceOrExit(), useuiautomatorhelper=True) helper = vc.uiAutomatorHelper # Start activity by package and class helper.target_context.start_activity( pkg='com.android.settings', cls='.Settings' ) # Start activity with URI helper.target_context.start_activity( pkg='com.android.browser', cls='.BrowserActivity', uri='https://example.com' ) # Object Store management (for tracking found elements) # List stored objects objects = helper.object_store.list() print(f"Stored objects: {objects}") # Clear all stored objects helper.object_store.clear() # Remove specific object helper.object_store.remove(oid=123) ``` -------------------------------- ### MCP Server Configuration Examples Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/MCP_CONFIGURATION.md Provides example configurations for the MCP server in different scenarios: production, development, and remote device connection. ```APIDOC ## Complete Examples ### Example 1: Production Setup (kiro-cli) **File:** `~/.kiro/settings/mcp.json` ```json { "mcpServers": { "culebratester2-mcp": { "command": "culebra-mcp", "args": [], "env": { "CULEBRATESTER2_URL": "http://localhost:9987", "CULEBRATESTER2_TIMEOUT": "30" }, "disabled": false, "autoApprove": [ "getDeviceInfo", "dumpUiHierarchy", "takeScreenshot", "getCurrentPackage" ] } } } ``` **Use case:** Daily use with kiro-cli for Android automation ### Example 2: Development Setup (Workspace) **File:** `.kiro/settings/mcp.json` (in AndroidViewClient workspace) ```json { "mcpServers": { "culebratester2-mcp": { "command": "python3", "args": ["-m", "com.dtmilano.android.mcp.server"], "env": { "ANDROID_VIEW_CLIENT_HOME": "${workspaceFolder}", "PYTHONPATH": "${workspaceFolder}/src", "CULEBRATESTER2_URL": "http://localhost:9987", "CULEBRATESTER2_TIMEOUT": "30", "CULEBRATESTER2_DEBUG": "1" }, "disabled": false, "autoApprove": [ "getDeviceInfo", "dumpUiHierarchy", "getCurrentPackage" ] } } } ``` **Use case:** Developing or debugging the MCP server itself ### Example 3: Remote Device **File:** `~/.kiro/settings/mcp.json` ```json { "mcpServers": { "culebratester2-mcp": { "command": "culebra-mcp", "args": [], "env": { "CULEBRATESTER2_URL": "http://192.168.1.100:9987", "CULEBRATESTER2_TIMEOUT": "60" }, "disabled": false, "autoApprove": [ "getDeviceInfo", "getCurrentPackage" ] } } } ``` **Use case:** Connecting to CulebraTester2 running on a remote device or emulator ``` -------------------------------- ### Direct ADB Operations with AdbClient Source: https://context7.com/dtmilano/androidviewclient/llms.txt Provides examples of using the AdbClient class for direct device operations, including executing shell commands, retrieving device properties, managing key events, starting activities, and taking screenshots. ```python from com.dtmilano.android.viewclient import ViewClient device, serialno = ViewClient.connectToDeviceOrExit() output = device.shell('pm list packages') print(output) ``` ```python sdk_version = device.getProperty('ro.build.version.sdk') model = device.getProperty('ro.product.model') print(f"SDK: {sdk_version}, Model: {model}") ``` ```python display = device.getDisplayInfo() print(f"Width: {display['width']}, Height: {display['height']}") print(f"Density: {display['density']}") ``` ```python device.press('HOME') device.press('BACK') device.press('ENTER') ``` ```python device.longPress('POWER', duration=1.0) ``` ```python device.startActivity(component='com.example.app/.MainActivity') ``` ```python device.startActivity(package='com.android.settings') ``` ```python device.forceStop('com.example.app') ``` ```python image = device.takeSnapshot(reconnect=True) image.save('/tmp/screenshot.png', 'PNG') ``` ```python cropped = device.takeSnapshot(box=(100, 200, 500, 800)) cropped.save('/tmp/cropped.png', 'PNG') ``` -------------------------------- ### Start Application Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/mcp.md Starts an Android application, optionally specifying an activity. ```APIDOC ## POST /tools/startApp ### Description Starts an Android application. If `activityName` is not provided, the default activity will be launched. ### Method POST ### Endpoint /tools/startApp ### Parameters #### Request Body - **packageName** (string) - Required - The package name of the app (e.g., "com.example.app") - **activityName** (string) - Optional - The activity name to start (e.g., ".MainActivity") ### Request Example ```json { "packageName": "com.example.app", "activityName": ".MainActivity" } ``` ### Response #### Success Response (200) - **status** (string) - Indicates the success status of the operation. #### Response Example ```json { "status": "success" } ``` ``` -------------------------------- ### Perform System Actions Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/index.md Examples of interacting with the device, such as pressing hardware keys or pausing execution. ```python ViewClient.sleep(2.0) vc.pressKeyCode(66) # KEYCODE_ENTER ``` -------------------------------- ### Start CulebraTester2 on Device and Configure MCP Server Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/mcp.md This snippet covers the initial setup for CulebraTester2 on an Android device, including installation, instrumentation, port forwarding, and configuring the AI assistant's settings file (mcp.json) to connect to the MCP server. ```bash adb install -r culebratester2.apk ``` ```bash adb shell am instrument -w com.dtmilano.android.culebratester2/.CulebraTester2Instrumentation ``` ```bash adb forward tcp:9987 tcp:9987 ``` ```json { "mcpServers": { "culebratester2": { "command": "culebra-mcp", "env": { "CULEBRATESTER2_URL": "http://localhost:9987" } } } } ``` -------------------------------- ### Simple Interaction Example Source: https://github.com/dtmilano/androidviewclient/wiki/Home This script demonstrates how to find a UI element by its content description and touch it to open an application. ```APIDOC ## POST /interact ### Description Finds a UI element by its content description and touches it. ### Method POST ### Endpoint /interact ### Parameters #### Query Parameters - **serialno** (string) - Optional - The serial number of the device to connect to. #### Request Body - **content_description** (string) - Required - The content description of the UI element to find. ### Request Example ```python #! /usr/bin/env python # -*- coding: utf-8 -*- """ Copyright (C) 2019 Diego Torres Milano Created on Jun 7, 2019 @author: diego """ from com.dtmilano.android.viewclient import ViewClient vc = ViewClient(*ViewClient.connectToDeviceOrExit()) vc.dump(window=-1) vc.findViewWithContentDescriptionOrRaise(u'''Chrome''').touch() ``` ### Response #### Success Response (200) - **status** (string) - Indicates the success of the operation. #### Response Example ```json { "status": "success" } ``` ``` -------------------------------- ### Simple Dump Example Source: https://github.com/dtmilano/androidviewclient/wiki/Home This script demonstrates how to dump the current view hierarchy of an Android device. ```APIDOC ## GET /dump ### Description Dumps the current view hierarchy of the Android device. ### Method GET ### Endpoint /dump ### Parameters #### Query Parameters - **serialno** (string) - Optional - The serial number of the device to connect to. ### Request Example ```python #! /usr/bin/env python ''' Copyright (C) 2012 Diego Torres Milano Created on Feb 3, 2012 @author: diego ''' import sys import os try: sys.path.insert(0, os.path.join(os.environ['ANDROID_VIEW_CLIENT_HOME'], 'src')) except: pass from com.dtmilano.android.viewclient import ViewClient ViewClient(*ViewClient.connectToDeviceOrExit()).traverse() ``` ### Response #### Success Response (200) - **view_hierarchy** (string) - A string representation of the device's view hierarchy. #### Response Example ``` android.widget.FrameLayout com.android.launcher3.Workspace com.google.android.apps.nexuslauncher:id/workspace android.widget.FrameLayout com.google.android.apps.nexuslauncher:id/search_container_workspace android.widget.TextView com.google.android.apps.nexuslauncher:id/clock Wednesday, May 29 android.widget.LinearLayout com.google.android.apps.nexuslauncher:id/title_weather_content android.widget.TextView com.google.android.apps.nexuslauncher:id/title_weather_text 57°F android.widget.ImageView com.google.android.apps.nexuslauncher:id/all_apps_handle android.view.ViewGroup com.google.android.apps.nexuslauncher:id/layout android.widget.TextView Phone android.widget.TextView Messages android.widget.TextView Play Store android.widget.TextView Chrome android.widget.TextView Culebra Tester android.widget.FrameLayout com.google.android.apps.nexuslauncher:id/search_container_hotseat ``` ``` -------------------------------- ### POST /start_activity Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Starts a specific Android activity on the target device. ```APIDOC ## POST /start_activity ### Description Starts a specified Android activity using the provided package and class names. ### Method POST ### Endpoint /start_activity ### Parameters #### Request Body - **pkg** (string) - Required - The package name of the application. - **cls** (string) - Required - The Activity class name. - **uri** (string) - Optional - An optional URI to pass to the activity. ### Request Example { "pkg": "com.example.app", "cls": ".MainActivity" } ### Response #### Success Response (200) - **status** (string) - The API execution status. #### Response Example { "status": "success" } ``` -------------------------------- ### Calculator Test Example - Python Source: https://context7.com/dtmilano/androidviewclient/llms.txt A comprehensive example demonstrating automated testing of a calculator application. It covers connecting to the device, finding UI elements, performing clicks, and waiting for window updates. ```Python #!/usr/bin/env python3 from com.dtmilano.android.viewclient import ViewClient # Connect with UiAutomatorHelper kwargs1 = {'verbose': False, 'ignoresecuredevice': False, 'ignoreversioncheck': False} device, serialno = ViewClient.connectToDeviceOrExit(**kwargs1) kwargs2 = {'useuiautomatorhelper': True, 'autodump': False} vc = ViewClient(device, serialno, **kwargs2) helper = vc.uiAutomatorHelper # Click digit 3 obj_ref = helper.until.find_object(body={ 'clazz': 'android.widget.ImageButton', 'res': 'com.google.android.calculator:id/digit_3', 'desc': '3' }) response = helper.ui_device.wait(oid=obj_ref.oid) helper.ui_object2.click(oid=response['oid']) helper.ui_device.wait_for_window_update() # Click multiply obj_ref = helper.until.find_object(body={ 'res': 'com.google.android.calculator:id/op_mul', 'desc': 'multiply' }) response = helper.ui_device.wait(oid=obj_ref.oid) helper.ui_object2.click(oid=response['oid']) helper.ui_device.wait_for_window_update() # Click digit 2 obj_ref = helper.until.find_object(body={ 'res': 'com.google.android.calculator:id/digit_2', 'desc': '2' }) response = helper.ui_device.wait(oid=obj_ref.oid) helper.ui_object2.click(oid=response['oid']) helper.ui_device.wait_for_window_update() ``` -------------------------------- ### MCP Server Tool: Start Android Application Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/mcp.md Launches a specified Android application identified by its package name and optionally its main activity. This is used to start applications for testing or interaction. ```python # Example usage (assuming mcp_client is an initialized MCP client) # success = mcp_client.startApp("com.example.app", ".MainActivity") # print(success) ``` -------------------------------- ### Start CulebraTester2 Server Source: https://github.com/dtmilano/androidviewclient/wiki/CulebraTester2-public Command to start the CulebraTester2 server. Ensure the server is running before executing other commands. ```bash $ ./culebratester2 start-server ``` -------------------------------- ### Start Culebra GUI Source: https://github.com/dtmilano/androidviewclient/wiki/culebra Launches the Culebra GUI, providing a graphical interface for interacting with and automating Android views. ```bash culebra -G ``` -------------------------------- ### POST /adb/startActivity Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/genindex.html Starts an activity on the connected Android device using the AdbClient. ```APIDOC ## POST /adb/startActivity ### Description Initiates the launch of a specific activity on the target Android device. ### Method POST ### Endpoint /adb/startActivity ### Parameters #### Request Body - **component** (string) - Required - The component name of the activity to start. ### Request Example { "component": "com.example.app/.MainActivity" } ### Response #### Success Response (200) - **status** (string) - Indicates the success status of the activity launch. #### Response Example { "status": "success" } ``` -------------------------------- ### POST /target_context/start_activity Source: https://context7.com/dtmilano/androidviewclient/llms.txt Starts a new Android activity on the connected device by specifying the package and class name, with optional URI data. ```APIDOC ## POST /target_context/start_activity ### Description Starts an Android activity using the target_context interface. ### Method POST ### Endpoint /target_context/start_activity ### Parameters #### Request Body - **pkg** (string) - Required - The package name of the application. - **cls** (string) - Required - The class name of the activity. - **uri** (string) - Optional - Data URI to pass to the activity. ### Request Example { "pkg": "com.android.settings", "cls": ".Settings" } ### Response #### Success Response (200) - **status** (string) - Operation success status. ``` -------------------------------- ### Environment Variable Setup for AndroidViewClient Source: https://github.com/dtmilano/androidviewclient/wiki/culebra Demonstrates how to set environment variables required for running AndroidViewClient scripts, including ANDROID_VIEW_CLIENT_HOME and PYTHONPATH. ```bash export ANDROID_VIEW_CLIENT_HOME=/home/user/workspace/AndroidViewClient ``` -------------------------------- ### Start Android Activity Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Initiates an Android Activity. Can be started by component name or package name. Supports specifying flags and URIs. This is a method of the AdbClient class. ```python startActivity(_component=None_, _flags=None_, _uri=None_, _package=None_) ``` -------------------------------- ### Install AndroidViewClient via pip Source: https://github.com/dtmilano/androidviewclient/blob/master/README.md Use the pip package manager to install or upgrade the AndroidViewClient library in your Python environment. ```bash pip3 install androidviewclient --upgrade ``` -------------------------------- ### Start Android App with androidviewclient Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/mcp.md Starts a specified Android application. It can optionally launch a specific activity within the app. The function returns a JSON string indicating the success status of the operation. ```python from com.dtmilano.android.mcp import Mcp mcp = Mcp() mcp.tools.startApp(packageName='com.example.app', activityName='.MainActivity') ``` -------------------------------- ### Example JSON UI Hierarchy Output Source: https://github.com/dtmilano/androidviewclient/wiki/CulebraTester2-public An example of the JSON output generated by the dump command, representing the Android window hierarchy. This structured format facilitates programmatic access to UI elements. ```json { "id": "hierarchy", "text": "Window Hierarchy", "children": [ { "id": 0, "parent": -1, "text": "", "package": "com.google.android.apps.nexuslauncher", "checkable": false, "clickable": false, "index": 0, "content_description": "", "focusable": false, "resource_id": "", "enabled": true, "password": false, "long_clickable": false, "long_text": "android.widget.FrameLayout__ id=0 parent=-1", "clazz": "android.widget.FrameLayout", "scrollable": false, "selected": false, "checked": false, "focused": false, "bounds": [ 0, 0, 1080, 1878 ], "children": [ { "id": 1, "parent": 0, "text": "", "package": "com.google.android.apps.nexuslauncher", "checkable": false, "clickable": false, "index": 0, ... } } ] } ``` -------------------------------- ### Test Fixture Setup and Teardown Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Defines methods for setting up and tearing down test fixtures for CulebraTestCase. Includes class-level and instance-level setup/teardown hooks. ```Python def setUp(self): # Hook method for setting up the test fixture before exercising it. pass def setUpClass(self): # Hook method for setting up class fixture before running tests in the class. pass def tearDown(self): # Hook method for deconstructing the test fixture after testing it. pass def tearDownClass(self): # Hook method for deconstructing the class fixture after running all tests in the class. pass ``` -------------------------------- ### Pinch and Gesture Operations - Python Source: https://context7.com/dtmilano/androidviewclient/llms.txt Provides examples of performing multi-touch gestures like pinch-to-zoom and custom two-pointer gestures on UI elements. It also shows how to retrieve view bounds for gesture calculations. ```Python from com.dtmilano.android.viewclient import ViewClient vc = ViewClient(*ViewClient.connectToDeviceOrExit(), useuiautomatorhelper=True) helper = vc.uiAutomatorHelper # Find a zoomable view (e.g., map or image) map_ref = helper.ui_device.find_object(resource_id='com.example:id/map_view') # Pinch in (zoom out) helper.ui_object.pinch_in(oid=map_ref.oid, percentage=50, steps=50) # Pinch out (zoom in) helper.ui_object.pinch_out(oid=map_ref.oid, percentage=50, steps=50) # Custom two-pointer gesture helper.ui_object.perform_two_pointer_gesture( oid=map_ref.oid, startPoint1=(200, 400), startPoint2=(600, 400), endPoint1=(400, 400), endPoint2=(400, 400), steps=50 ) # Get view bounds for gesture calculations bounds = helper.ui_object.get_bounds(oid=map_ref.oid) left, top, right, bottom = bounds center_x = (left + right) // 2 center_y = (top + bottom) // 2 ``` -------------------------------- ### Dump Command with Verbose Output (Bash) Source: https://github.com/dtmilano/androidviewclient/wiki/Secure-mode Example of using the 'dump' tool provided with AndroidViewClient with the '--verbose' option. This command provides extra information about the device, including its security and debuggable status, and indicates if AndroidViewClient can operate. ```bash $ dump --verbose ``` -------------------------------- ### Configure CulebraTester2 MCP Server via JSON Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/MCP_CONFIGURATION.md Examples of MCP server configuration blocks for Kiro. These snippets demonstrate how to define the command, arguments, and environment variables required to launch the server in different environments. ```json { "mcpServers": { "culebratester2-mcp": { "command": "culebra-mcp", "args": [] } } } ``` ```json { "mcpServers": { "culebratester2-mcp": { "command": "python3", "args": ["-m", "com.dtmilano.android.mcp.server"], "env": { "ANDROID_VIEW_CLIENT_HOME": "${workspaceFolder}", "PYTHONPATH": "${workspaceFolder}/src" } } } } ``` -------------------------------- ### Device Information and Dumpsys using ViewClient Source: https://context7.com/dtmilano/androidviewclient/llms.txt Shows how to retrieve detailed device information such as display size, physical dimensions, and dumpsys output using the ViewClient's UiAutomatorHelper. It also covers getting the current activity name and PID. ```python from com.dtmilano.android.viewclient import ViewClient vc = ViewClient(*ViewClient.connectToDeviceOrExit(), useuiautomatorhelper=True) helper = vc.uiAutomatorHelper display = helper.device.display_real_size() print(f"Display: {display.x} x {display.y}") ``` ```python physical = helper.device.display_physical_size() print(f"Physical: {physical['physical_width']}in x {physical['physical_height']}in") print(f"Screen diagonal: {physical['screen']}in") ``` ```python wifi_info = helper.device.dumpsys('wifi') activity_info = helper.device.dumpsys('activity', arg1='top') ``` ```python activity = helper.device.get_top_activity_name() print(f"Current activity: {activity}") ``` ```python pkg, activity_class, pid = helper.device.get_top_activity_name_and_pid() print(f"Package: {pkg}, Activity: {activity_class}, PID: {pid}") ``` -------------------------------- ### Configure MCP Server for Development Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/MCP_CONFIGURATION.md Development-specific configuration using a local Python module. This setup includes additional environment variables for workspace paths and debug mode. ```json { "mcpServers": { "culebratester2-mcp": { "command": "python3", "args": ["-m", "com.dtmilano.android.mcp.server"], "env": { "ANDROID_VIEW_CLIENT_HOME": "${workspaceFolder}", "PYTHONPATH": "${workspaceFolder}/src", "CULEBRATESTER2_URL": "http://localhost:9987", "CULEBRATESTER2_TIMEOUT": "30", "CULEBRATESTER2_DEBUG": "1" }, "disabled": false, "autoApprove": [ "getDeviceInfo", "dumpUiHierarchy", "getCurrentPackage" ] } } } ``` -------------------------------- ### Interact with Android UI Elements using Python Source: https://github.com/dtmilano/androidviewclient/wiki/Home This Python script demonstrates how to use AndroidViewClient to connect to a device, dump the UI hierarchy, and interact with a specific UI element by its content description. In this example, it touches the 'Chrome' shortcut to launch the application. ```python #! /usr/bin/env python # -*- coding: utf-8 -*- """ Copyright (C) 2019 Diego Torres Milano Created on Jun 7, 2019 @author: diego """ from com.dtmilano.android.viewclient import ViewClient vc = ViewClient(*ViewClient.connectToDeviceOrExit()) vc.dump(window=-1) vc.findViewWithContentDescriptionOrRaise(u'''Chrome''').touch() ``` -------------------------------- ### Import AndroidViewClient Modules via Shebang Source: https://github.com/dtmilano/androidviewclient/wiki/Shebang This example shows how a properly configured shebang line allows for clean imports of AndroidViewClient modules in a Python script executed by monkeyrunner. ```python #! /usr/bin/env shebang monkeyrunner -plugin $ANDROID_VIEW_CLIENT_HOME/bin/androidviewclient-$ANDROID_VIEW_CLIENT_VERSION.jar @! from com.dtmilano.android.viewclient import ViewClient ``` -------------------------------- ### Set ANDROID_VIEW_CLIENT_HOME Environment Variable Source: https://github.com/dtmilano/androidviewclient/wiki/Home This command sets the ANDROID_VIEW_CLIENT_HOME environment variable, which is crucial for Python to locate the AndroidViewClient modules. This is a common step for manual installations. ```bash export ANDROID_VIEW_CLIENT_HOME=/path/to/androidviewclient/ ``` -------------------------------- ### Get Device Serial Number (Python) Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Returns the unique serial number of the connected Android device. This is essential for identifying specific devices in multi-device setups. ```python serialno ``` -------------------------------- ### ViewClient Initialization Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Demonstrates how to initialize the ViewClient with various configuration options. ```APIDOC ## ViewClient Constructor ### Description Initializes the ViewClient, which acts as a client for the Android ViewServer. It can optionally start the ViewServer on the target device or emulator. ### Parameters - **device** (object) - Required - The device object. - **serialno** (str) - Required - The serial number of the device. - **adb** (object) - Optional - The adb client object. - **autodump** (bool) - Optional - Whether to automatically dump the view hierarchy. - **forceviewserveruse** (bool) - Optional - Force the use of ViewServer. - **localport** (int) - Optional - Local port to use for ViewServer. - **remoteport** (int) - Optional - Remote port on the device for ViewServer. - **startviewserver** (bool) - Optional - Whether to start the ViewServer. - **ignoreuiautomatorkilled** (bool) - Optional - Ignore if UiAutomator is killed. - **compresseddump** (bool) - Optional - Use compressed dump format. - **useuiautomatorhelper** (bool) - Optional - Use UiAutomatorHelper backend. - **debug** (dict) - Optional - Debugging options. ``` -------------------------------- ### Run dump Tool Help Source: https://github.com/dtmilano/androidviewclient/wiki/Home This command displays the help information for the 'dump' tool, which is part of the AndroidViewClient distribution. It shows available command-line options for the tool. ```bash $ dump --help ``` -------------------------------- ### Initialize ViewClient Helper for Lightweight Automation Source: https://context7.com/dtmilano/androidviewclient/llms.txt Shows how to instantiate a standalone UiAutomatorHelper instance to perform device operations without a full ViewClient. This is ideal for lightweight scripts requiring direct device communication. ```python from com.dtmilano.android.viewclient import ViewClient helper = ViewClient.view_client_helper(hostname='localhost', port=9987, kato=False) display = helper.device.display_real_size() print(f"Screen: {display.x}x{display.y}") obj = helper.ui_device.find_object(text='Settings') if obj: helper.ui_object2.click(oid=obj.oid) helper.ui_device.take_screenshot(filename='/tmp/screen.png') ``` -------------------------------- ### GET /device/dump_window_hierarchy Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the current window hierarchy of the device. ```APIDOC ## GET /device/dump_window_hierarchy ### Description Returns the current UI window hierarchy in the requested format. ### Method GET ### Endpoint /device/dump_window_hierarchy ### Parameters #### Query Parameters - **_format** (string) - Optional - The output format (e.g., 'JSON' or 'XML'). Defaults to 'JSON'. ### Response #### Success Response (200) - **hierarchy** (object/string) - The window hierarchy data. ``` -------------------------------- ### Interact with UI Elements using ViewClient Source: https://context7.com/dtmilano/androidviewclient/llms.txt Demonstrates finding UI elements by text, resource ID, or class, and performing actions like clicking, setting text, and retrieving text content. It also covers dumping object information. ```python button_ref = helper.ui_device.find_object(text='Sign In') helper.ui_object2.click(oid=button_ref.oid) ``` ```python menu_item = helper.ui_device.find_object(text='Options') helper.ui_object2.long_click(oid=menu_item.oid) ``` ```python input_ref = helper.ui_device.find_object(clazz='android.widget.EditText') helper.ui_object2.clear(oid=input_ref.oid) helper.ui_object2.set_text(oid=input_ref.oid, text='Hello World!') ``` ```python text_ref = helper.ui_device.find_object(resource_id='com.example:id/title') text_value = helper.ui_object2.get_text(oid=text_ref.oid) print(f"Title: {text_value}") ``` ```python icon_ref = helper.ui_device.find_object(clazz='android.widget.ImageButton') desc = helper.ui_object2.get_content_description(oid=icon_ref.oid) print(f"Icon description: {desc}") ``` ```python obj_dump = helper.ui_object2.dump(oid=button_ref.oid) print(f"Object dump: {obj_dump}") ``` -------------------------------- ### GET /ui_object/exists Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Checks if a specific UI element exists on the screen based on its object ID. ```APIDOC ## GET /ui_object/exists ### Description Verifies the existence of a UI element identified by its OID. ### Method GET ### Endpoint /ui_object/exists ### Parameters #### Query Parameters - **oid** (int) - Required - The search condition reference (oid). ### Request Example { "oid": 12345 } ### Response #### Success Response (200) - **exists** (boolean) - True if the element exists, false otherwise. #### Response Example { "exists": true } ``` -------------------------------- ### Get Screenshot Number Count Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the current count of screenshots taken. This is a property of the AdbClient class. ```python screenshot_number ``` -------------------------------- ### MCP Server for AI Automation Source: https://context7.com/dtmilano/androidviewclient/llms.txt Details on running the MCP server and a list of available tools for AI assistants to interact with Android devices, including device info, UI hierarchy, element finding, and automation actions. ```bash # Running the MCP server (command line) # export CULEBRATESTER2_URL=http://localhost:9987 # python -m com.dtmilano.android.mcp.server ``` ```plaintext # MCP Tools available: # - getDeviceInfo(): Get display dimensions and device info # - dumpUiHierarchy(): Dump current UI hierarchy as XML # - findElementByText(text): Find element by text content # - findElementByResourceId(resourceId): Find element by resource ID # - clickElement(elementId): Click on found element # - enterText(elementId, text): Enter text into element # - pressBack(): Press Android BACK button # - pressHome(): Press Android HOME button # - takeScreenshot(): Capture screen as base64 image # - startApp(packageName, activityName): Launch application # - clickAtCoordinates(x, y): Click at screen coordinates # - longClickAtCoordinates(x, y): Long click at coordinates # - swipeGesture(startX, startY, endX, endY, steps): Perform swipe # - wakeDevice(): Turn screen on # - sleepDevice(): Turn screen off # - pressRecentApps(): Show recent apps # - getCurrentPackage(): Get current foreground package # - forceStopApp(packageName): Force stop application # - longClickElement(elementId): Long click on element # - clearText(elementId): Clear text from input field ``` -------------------------------- ### Manage UI Element References with ObjectStore Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/mcp.md Demonstrates how to use the ObjectStore class to cache, retrieve, and verify UI element metadata. This is essential for maintaining state between UI operations in the MCP server. ```python from com.dtmilano.android.mcp.object_store import ObjectStore # Initialize store store = ObjectStore() # Store an object reference store.store(123, {"selector": {"text": "Login"}, "type": "text"}) # Check existence and retrieve if store.exists(123): metadata = store.get(123) print(metadata) # Clear store store.clear() ``` -------------------------------- ### Test Fixture Setup/Teardown (Python) Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/index.md Provides hook methods for setting up and tearing down test fixtures before and after test execution, respectively. These are standard methods for test case management. ```python setUp() ``` ```python tearDown() ``` ```python setUpClass() ``` ```python tearDownClass() ``` -------------------------------- ### Get View Width - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the width of an Android View. This method returns the current width of the view. ```Python def getWidth(self): """Gets the width.""" pass ``` -------------------------------- ### Get View Height - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the height of an Android View. This method returns the current height of the view. ```Python def getHeight(self): """Gets the height.""" pass ``` -------------------------------- ### Device Interaction API Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/index.md Methods for controlling the Android device, including starting activities, taking screenshots, and unlocking the screen. ```APIDOC ## POST /device/startActivity ### Description Starts an Activity on the connected device. If a package is specified instead of a component, the main activity for that package is resolved and launched. ### Method POST ### Parameters #### Request Body - **component** (string) - Optional - The component name to start. - **package** (string) - Optional - The package name to resolve the main activity. - **flags** (int) - Optional - Intent flags. - **uri** (string) - Optional - URI for the intent. ## GET /device/snapshot ### Description Takes a snapshot of the current device screen and returns it as a PIL Image. ### Method GET ### Parameters #### Query Parameters - **reconnect** (boolean) - Optional - Reconnects the device after taking the screenshot. - **box** (tuple) - Optional - A tuple (left, top, right, bottom) to crop the screenshot. ### Response #### Success Response (200) - **image** (binary) - The captured screen image. ``` -------------------------------- ### Get View Text - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the text content of an Android View. This is commonly used for views like TextView. ```Python def getText(self): """Gets the text attribute. @return: the text attribute or C{None} if not defined""" pass ``` -------------------------------- ### Connect to ADB Server Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Establishes a connection to the ADB server at a specified hostname and port, with an optional timeout. Returns a connection object. ```python com.dtmilano.android.adb.adbclient.connect(_hostname_, _port_, _timeout=15_) ``` -------------------------------- ### Get View Parent - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the parent view of the current Android View. This is useful for navigating up the view hierarchy. ```Python def getParent(self): """Gets the parent.""" pass ``` -------------------------------- ### AdbClient: Property and Image Utilities Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Provides utility functions for retrieving device properties by key and for image analysis. The `imageInScreen` method checks for the presence of a sub-image within a larger screen capture, while `imageToData` extracts text and bounding boxes from an image, useful for OCR tasks. ```python # Get a specific property property_value = adb_client.getProperty('ro.product.model') # Check if an image is on the screen (assuming 'screen_image' and 'template_image' are defined) # is_present = AdbClient.imageInScreen(screen_image, template_image) # Extract data from an image (assuming 'screenshot_image' is defined) # extracted_data = adb_client.imageToData(screenshot_image) ``` -------------------------------- ### Get View ID - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the ID of an Android View. This method returns the unique identifier assigned to the view. ```Python def getId(self): """Gets the L{View} Id @return: the L{View} C{Id} or C{None} if not defined @see: L{getUniqueId()}""" pass ``` -------------------------------- ### Get View Content Description - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the content description of an Android View. This is often used for accessibility purposes. ```Python def getContentDescription(self): """Gets the content description.""" pass ``` -------------------------------- ### Run culebra Tool Help Source: https://github.com/dtmilano/androidviewclient/wiki/Home This command displays the help information for the 'culebra' tool, a command-line utility provided by AndroidViewClient. It lists the available options for interacting with Android devices. ```bash $ culebra --help ``` -------------------------------- ### Configure Multiple MCP Servers for Devices Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/MCP_CONFIGURATION.md This JSON configuration allows setting up multiple MCP server instances, each targeting a different device. It specifies the command to run, arguments, environment variables (like device URLs), and a disabled flag. This is useful for testing on multiple devices simultaneously. ```json { "mcpServers": { "culebratester2-device1": { "command": "culebra-mcp", "args": [], "env": { "CULEBRATESTER2_URL": "http://localhost:9987" }, "disabled": false }, "culebratester2-device2": { "command": "culebra-mcp", "args": [], "env": { "CULEBRATESTER2_URL": "http://localhost:9988" }, "disabled": false } } } ``` -------------------------------- ### Get View Class - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the class name of an Android View. This can be helpful for identifying the type of a UI element. ```Python def getClass(self): """Gets the L{View} class @return: the L{View} class or C{None} if not defined""" pass ``` -------------------------------- ### Get Root Node (Python) Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the root node of the view hierarchy. This is a fundamental element for navigating and interacting with the UI structure. ```python root ``` -------------------------------- ### Get Root View Node (Python) Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the root node of the current view hierarchy. This is the top-level element from which all other views can be accessed. ```python getRoot() ``` -------------------------------- ### Get View Y Coordinate - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the Y coordinate of an Android View. This represents the vertical position of the view's top-left corner. ```Python def getY(self): """Gets the View Y coordinate""" pass ``` -------------------------------- ### Find Views by Attribute Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/index.md Demonstrates how to locate views in the hierarchy based on specific attributes and values, such as identifying all buttons. ```python buttons = v.findViewsWithAttribute("class", "android.widget.Button") ``` -------------------------------- ### Get View Visibility - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the visibility status of an Android View. This indicates whether the view is visible, hidden, or gone. ```Python def getVisibility(self): """Gets the View visibility""" pass ``` -------------------------------- ### Configure MCP Server Command and Arguments Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/MCP_CONFIGURATION.md This JSON snippet demonstrates how to specify the command and arguments for running the MCP server. It shows an alternative to directly calling `culebra-mcp` by using `python3 -m com.dtmilano.android.mcp.server`, which can resolve 'Command Not Found' errors. ```json { "command": "python3", "args": ["-m", "com.dtmilano.android.mcp.server"] } ``` -------------------------------- ### Get View Tag - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the tag associated with an Android View. Tags can be used to store arbitrary data with a view. ```Python def getTag(self): """Gets the tag.""" pass ``` -------------------------------- ### Get View Children - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves a list of child views for a given Android View. This is useful for traversing the view hierarchy. ```Python def getChildren(self): """Gets the children of this L{View}.""" pass ``` -------------------------------- ### Check AndroidViewClient Directory Contents Source: https://github.com/dtmilano/androidviewclient/wiki/Home This command lists the contents of the AndroidViewClient directory, typically used to confirm that the ANDROID_VIEW_CLIENT_HOME variable points to the correct location. The output should show key files and directories like LICENSE, README.md, setup.py, and src. ```bash cd $ANDROID_VIEW_CLIENT_HOME ls -l ``` -------------------------------- ### Device Interaction API Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Provides methods for interacting with an Android device, such as getting display size, running dumpsys, and waiting for toasts. ```APIDOC ## GET /device/display_real_size ### Description Retrieves the real display size of the device. ### Method GET ### Endpoint /device/display_real_size ### Parameters None ### Request Example None ### Response #### Success Response (200) - **width** (int) - The width of the display in pixels. - **height** (int) - The height of the display in pixels. #### Response Example { "width": 1080, "height": 1920 } ## GET /device/dumpsys ### Description Executes the dumpsys command for a specified service on the device. ### Method GET ### Endpoint /device/dumpsys ### Parameters #### Query Parameters - **service** (string) - Required - The name of the service to run dumpsys on. - **kwargs** (object) - Optional - Additional arguments for the dumpsys command. ### Request Example None ### Response #### Success Response (200) - **output** (string) - The output of the dumpsys command. #### Response Example { "output": "Window Manager Service ..." } ## POST /device/wait_for_new_toast ### Description Waits for a new toast message to appear on the device. ### Method POST ### Endpoint /device/wait_for_new_toast ### Parameters #### Request Body - **timeout** (int) - Optional - The maximum time in milliseconds to wait for the toast. Defaults to 10000. ### Request Example { "timeout": 5000 } ### Response #### Success Response (200) - **text** (string) - The text content of the toast message. #### Response Example { "text": "Operation successful" } ``` -------------------------------- ### Retrieve Object Information Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/index.md Methods to dump object content, get text values, or retrieve content descriptions for a given OID. ```python def dump(oid): pass def get_text(oid): pass def get_content_description(oid): pass ``` -------------------------------- ### Initialize ViewClient Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/index.md Initializes the ViewClient to interface with an Android device. It supports multiple backends like ViewServer and UiAutomator. ```python from com.dtmilano.android.viewclient import ViewClient vc = ViewClient(device, serialno, autodump=True) ``` -------------------------------- ### Connect to Android Device Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Establishes a connection to an Android device using ADB. Exits the script if the connection fails. ```python def connectToDeviceOrExit(timeout=60, verbose=False, serialno=None): # Logic to connect to device or exit on failure device, serial = connect(serialno=serialno, timeout=timeout) if not device: sys.exit(1) return device, serial ``` -------------------------------- ### Get View Attributes Map - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves a map containing attribute-value pairs for an Android View. This can be useful for inspecting view properties. ```Python def map(self): """The map that contains the C{attr},C{value} pairs""" pass ``` -------------------------------- ### Traverse View Hierarchy Source: https://github.com/dtmilano/androidviewclient/blob/master/sphinx/index.md Shows how to walk through the view tree and print node information, optionally using a transform function to format the output. ```python v.traverse(transform=ViewClient.traverseShowClassIdAndText) ``` -------------------------------- ### Get View Coordinates - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the coordinates of an Android View on the screen. The coordinates are returned as a tuple representing the top-left and bottom-right corners. ```Python def getCoords(self): """Gets the coords of the View @return: A tuple containing the View’s coordinates ((L, T), (R, B))""" pass ``` -------------------------------- ### Get SDK Version (Python) Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Returns the Android SDK version of the device. This information can be crucial for ensuring compatibility or conditional logic in automation scripts. ```python getSdkVersion() ``` -------------------------------- ### Connect to Android Device with ViewClient Source: https://context7.com/dtmilano/androidviewclient/llms.txt Establishes a connection to an Android device using ADB. It returns a ViewClient instance configured for either standard ViewServer or UiAutomatorHelper backends. ```python from com.dtmilano.android.viewclient import ViewClient # Connect to a device (auto-discovers if no serial provided) device, serialno = ViewClient.connectToDeviceOrExit( timeout=60, verbose=True, ignoresecuredevice=False, ignoreversioncheck=False, serialno=None ) # Create ViewClient with default options vc = ViewClient(device, serialno) # Create ViewClient with UiAutomatorHelper backend (recommended for Android 4.1+) vc = ViewClient(device, serialno, useuiautomatorhelper=True) # One-liner connection and ViewClient creation vc = ViewClient(*ViewClient.connectToDeviceOrExit()) ``` -------------------------------- ### Get View Parent Object - Python Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves the parent object of the current Android View. This provides direct access to the parent view instance. ```Python def parent(self): """The parent of this View""" pass ``` -------------------------------- ### List Windows (Python) Source: https://github.com/dtmilano/androidviewclient/blob/master/docs/index.html Retrieves a list of all available windows on the device. An optional sleep parameter can be used to wait before fetching the list, useful when changes are expected. ```python list(_sleep=1_) ```