### Install App Source: https://context7.com/codeskyblue/uiautodev/llms.txt Installs an application on the device from a URL or local path. ```APIDOC ## POST /api/{platform}/{serial}/command/installApp ### Description Installs an application on the device from a URL or local path. ### Method POST ### Endpoint /api/{platform}/{serial}/command/installApp ### Parameters #### Path Parameters - **platform** (string) - Required - The platform of the device (e.g., android, ios). - **serial** (string) - Required - The serial number of the device. #### Request Body - **url** (string) - Required - The URL or local path to the application file. ### Request Example ```json { "url": "https://example.com/app.apk" } ``` ### Response #### Success Response (200) - **success** (boolean) - Indicates if the installation was successful. - **id** (string) - The ID of the installation task, if applicable. #### Response Example ```json { "success": true, "id": null } ``` ``` -------------------------------- ### Install Application on Device Source: https://context7.com/codeskyblue/uiautodev/llms.txt Installs an application from a provided URL or local path onto the target device. Requires a JSON payload containing the source URL. ```bash curl -X POST http://localhost:20242/api/android/emulator-5554/command/installApp -H "Content-Type: application/json" -d '{"url": "https://example.com/app.apk"}' ``` -------------------------------- ### Install UIAutodev Package Source: https://context7.com/codeskyblue/uiautodev/llms.txt Install the UIAutodev package using pip. Includes an optional flag for HarmonyOS support. ```bash pip install uiautodev pip install "uiautodev[harmony]" ``` -------------------------------- ### GET /api/info Source: https://context7.com/codeskyblue/uiautodev/llms.txt Retrieves general information about the running UIAutodev server instance. ```APIDOC ## GET /api/info ### Description Returns information about the running UIAutodev server including version, platform, and available drivers. ### Method GET ### Endpoint /api/info ### Response #### Success Response (200) - **version** (string) - Server version - **platform** (string) - Host OS - **drivers** (array) - List of supported device drivers #### Response Example { "version": "0.5.0", "platform": "Linux", "drivers": ["android", "ios", "harmony"] } ``` -------------------------------- ### Launch UIAutodev Local Server Source: https://context7.com/codeskyblue/uiautodev/llms.txt Commands to start the UIAutodev server with various configurations such as custom ports, offline mode, or headless operation. ```bash uiauto.dev uiauto.dev server --port 8080 --host 0.0.0.0 uiautodev server --offline uiauto.dev server --no-browser uiauto.dev server --force ``` -------------------------------- ### Backup Application Source: https://context7.com/codeskyblue/uiautodev/llms.txt Downloads the APK file of an installed application from the device to the local machine for backup. ```bash curl "http://localhost:20242/api/android/emulator-5554/backupApp?packageName=com.example.app" --output app_backup.apk ``` -------------------------------- ### Automate HarmonyOS Device Interactions Source: https://context7.com/codeskyblue/uiautodev/llms.txt Provides an example of managing HarmonyOS devices, including capturing screenshots and launching applications with specific ability parameters. ```python harmony_provider = HarmonyProvider() harmony_devices = harmony_provider.list_devices() if harmony_devices: harmony_driver = harmony_provider.get_device_driver(harmony_devices[0].serial) harmony_screenshot = harmony_driver.screenshot() harmony_screenshot.save("harmony_screen.png") harmony_driver.app_launch("com.example.app", page_name="MainAbility") ``` -------------------------------- ### GET /api/{platform}/list Source: https://context7.com/codeskyblue/uiautodev/llms.txt Lists all connected devices for a specific mobile platform. ```APIDOC ## GET /api/{platform}/list ### Description Lists all connected devices for a specific platform (android, ios, or harmony). ### Method GET ### Endpoint /api/{platform}/list ### Parameters #### Path Parameters - **platform** (string) - Required - The target platform (android, ios, or harmony) ### Response #### Success Response (200) - **serial** (string) - Device serial identifier - **model** (string) - Device model name - **status** (string) - Current connection status #### Response Example [ { "serial": "emulator-5554", "model": "sdk_gphone64_x86_64", "status": "device" } ] ``` -------------------------------- ### Initialize App Inspector Status Check (JavaScript) Source: https://github.com/codeskyblue/uiautodev/blob/master/uiautodev/static/demo.html This script checks if the App Inspector Chrome Extension is installed by verifying the presence of `chrome.runtime`. It updates a message element on the page to indicate the installation status. This code is intended to run in a browser environment. ```JavaScript window.onload = function() { let message = "App Inspector is installed."; if (chrome.runtime === undefined) { message = "Please install the App Inspector Chrome Extension to use this feature."; } document.getElementById('message').innerHTML = `${message}`; } ``` -------------------------------- ### Get Current Application Information Source: https://context7.com/codeskyblue/uiautodev/llms.txt Retrieves the currently active application details on an iOS or Android device. Returns the package name, activity, and process ID. ```bash curl http://localhost:20242/api/ios/00008030-001234567890/command/currentApp ``` -------------------------------- ### Get Current Foreground App Source: https://context7.com/codeskyblue/uiautodev/llms.txt Retrieves information about the application currently running in the foreground on the specified device. ```bash curl http://localhost:20242/api/android/emulator-5554/command/currentApp ``` -------------------------------- ### GET /api/{platform}/{serial}/hierarchy Source: https://context7.com/codeskyblue/uiautodev/llms.txt Dumps the UI hierarchy tree of the device's current screen. ```APIDOC ## GET /api/{platform}/{serial}/hierarchy ### Description Dumps the UI hierarchy tree of the device's current screen, supporting both JSON and XML formats. ### Method GET ### Endpoint /api/{platform}/{serial}/hierarchy ### Parameters #### Path Parameters - **platform** (string) - Required - The target platform - **serial** (string) - Required - The device serial #### Query Parameters - **format** (string) - Optional - Output format (json or xml) ### Response #### Success Response (200) - **name** (string) - Element class name - **bounds** (array) - Element coordinates - **children** (array) - Nested UI elements ``` -------------------------------- ### Get Current Activity Source: https://context7.com/codeskyblue/uiautodev/llms.txt Fetches the currently focused activity on the Android device. ```bash curl http://localhost:20242/api/android/emulator-5554/current_activity ``` -------------------------------- ### Get Current Activity Source: https://context7.com/codeskyblue/uiautodev/llms.txt Returns the currently focused activity on the Android device. ```APIDOC ## GET /api/android/{serial}/current_activity ### Description Returns the currently focused activity on the Android device. ### Method GET ### Endpoint /api/android/{serial}/current_activity ### Parameters #### Path Parameters - **serial** (string) - Required - The serial number of the Android device. ### Response #### Success Response (200) - **activity** (string) - The name of the current activity. #### Response Example ``` com.android.settings/.Settings ``` ``` -------------------------------- ### Get Current App on iOS Source: https://context7.com/codeskyblue/uiautodev/llms.txt Retrieves information about the currently active application on an iOS device. ```APIDOC ## GET /api/ios/{serial}/command/currentApp ### Description Retrieves information about the currently active application on an iOS device. ### Method GET ### Endpoint /api/ios/{serial}/command/currentApp ### Parameters #### Path Parameters - **serial** (string) - Required - The serial number of the iOS device. ### Response #### Success Response (200) - **package** (string) - The package name of the current app. - **activity** (string) - The activity name of the current app. - **pid** (integer) - The process ID of the current app. #### Response Example ```json { "package": "com.android.settings", "activity": ".Settings", "pid": 12345 } ``` ``` -------------------------------- ### Python API Usage Source: https://context7.com/codeskyblue/uiautodev/llms.txt Utilize UIAutodev drivers directly within Python scripts for building custom automation workflows. This section provides examples of importing and using the provider and command type classes. ```APIDOC ## Python API Usage Use UIAutodev drivers directly in Python scripts for custom automation workflows. ### Importing Providers and Command Types ```python from uiautodev.provider import AndroidProvider, IOSProvider, HarmonyProvider from uiautodev.command_types import TapRequest, FindElementRequest # Example usage (conceptual): # android_provider = AndroidProvider() # tap_command = TapRequest(x=100, y=200) # android_provider.execute_command(tap_command) ``` ``` -------------------------------- ### GET /api/{platform}/features Source: https://context7.com/codeskyblue/uiautodev/llms.txt Retrieves a list of supported features and endpoints for a specified platform (e.g., Android, iOS). This is useful for understanding the capabilities available for a particular device type. ```APIDOC ## GET /api/{platform}/features ### Description Returns the list of supported features/endpoints for a specific platform. ### Method GET ### Endpoint /api/{platform}/features ### Parameters #### Path Parameters - **platform** (string) - Required - The target platform (e.g., 'android', 'ios'). ### Request Example ```bash curl http://localhost:20242/api/android/features ``` ### Response #### Success Response (200) - **screenshot** (boolean) - Indicates if screenshot functionality is supported. - **hierarchy** (boolean) - Indicates if UI hierarchy retrieval is supported. - **shell** (boolean) - Indicates if shell command execution is supported. - **backupApp** (boolean) - Indicates if app backup functionality is supported. - **current_activity** (boolean) - Indicates if retrieving the current activity is supported. #### Response Example ```json { "screenshot": true, "hierarchy": true, "shell": true, "backupApp": true, "current_activity": true } ``` ``` -------------------------------- ### Send Message to App Inspector Extension (JavaScript) Source: https://github.com/codeskyblue/uiautodev/blob/master/uiautodev/static/demo.html This script sends a message to the App Inspector Chrome Extension with the ID 'fjbboaelofjaabjmlphndicacmapbalm' to request URL information. It handles the response, logging any errors or displaying the description from the extension. This code requires the App Inspector extension to be installed and accessible. ```JavaScript document.getElementById('test').addEventListener('click', function() { const extensionId = "fjbboaelofjaabjmlphndicacmapbalm"; chrome.runtime.sendMessage(extensionId, {url: "/info"}, function(response) { console.log(response); if (response.error) { console.error(response.error); } else { document.getElementById('message').innerHTML = response.data.description; } }); }); ``` -------------------------------- ### Initialize Python API Drivers Source: https://context7.com/codeskyblue/uiautodev/llms.txt Demonstrates how to import and initialize the core provider classes for Android, iOS, and HarmonyOS to build custom automation scripts. ```python from uiautodev.provider import AndroidProvider, IOSProvider, HarmonyProvider from uiautodev.command_types import TapRequest, FindElementRequest ``` -------------------------------- ### Retrieve Server Information Source: https://context7.com/codeskyblue/uiautodev/llms.txt Fetches metadata about the running server instance, including version, platform, and active drivers. ```bash curl http://localhost:20242/api/info ``` ```json { "version": "0.5.0", "description": "client for https://uiauto.dev", "platform": "Linux", "code_language": "Python", "cwd": "/home/user/project", "drivers": ["android", "ios", "harmony"] } ``` -------------------------------- ### Retrieve Supported Platform Features Source: https://context7.com/codeskyblue/uiautodev/llms.txt Fetches a list of supported capabilities and endpoints for a specific mobile platform. This is useful for verifying if features like screenshotting or hierarchy dumping are available. ```bash curl http://localhost:20242/api/android/features ``` -------------------------------- ### Automate Android Device Interactions Source: https://context7.com/codeskyblue/uiautodev/llms.txt Demonstrates how to connect to an Android device, capture screenshots, dump UI hierarchy, and perform common actions like tapping, text input, and app management. ```python android_provider = AndroidProvider() devices = android_provider.list_devices() if devices: driver = android_provider.get_device_driver(devices[0].serial) screenshot = driver.screenshot(0) screenshot.save("android_screen.png") xml_data, hierarchy = driver.dump_hierarchy() driver.tap(500, 800) driver.app_launch("com.android.settings") driver.send_keys("Hello World") ``` -------------------------------- ### Execute CLI Automation Commands Source: https://context7.com/codeskyblue/uiautodev/llms.txt Provides common CLI commands for device management, including listing devices, performing touch interactions, app lifecycle management, and system navigation. ```bash uiauto.dev android list uiauto.dev android tap x=500 y=800 uiauto.dev android tap x=0.5 y=0.5 isPercent=true uiauto.dev android currentApp uiauto.dev android appLaunch package=com.android.settings uiauto.dev android appTerminate package=com.android.settings uiauto.dev android home uiauto.dev android back uiauto.dev android wakeUp uiauto.dev android getWindowSize uiauto.dev android dump uiauto.dev android installApp url=https://example.com/app.apk uiauto.dev version uiauto.dev self-update uiauto.dev shutdown ``` -------------------------------- ### Automate iOS Device Interactions Source: https://context7.com/codeskyblue/uiautodev/llms.txt Shows the basic workflow for connecting to an iOS device, capturing a screenshot, and performing basic UI interactions like tapping and returning to the home screen. ```python ios_provider = IOSProvider() ios_devices = ios_provider.list_devices() if ios_devices: ios_driver = ios_provider.get_device_driver(ios_devices[0].serial) ios_screenshot = ios_driver.screenshot() ios_screenshot.save("ios_screen.png") ios_driver.tap(200, 400) ios_driver.home() ``` -------------------------------- ### List Connected Devices Source: https://context7.com/codeskyblue/uiautodev/llms.txt Lists all connected devices for a specific platform. Supported platforms are android, ios, and harmony. ```bash curl http://localhost:20242/api/android/list curl http://localhost:20242/api/ios/list curl http://localhost:20242/api/harmony/list ``` -------------------------------- ### CLI Commands Source: https://context7.com/codeskyblue/uiautodev/llms.txt Execute device commands directly from the command line for automation scripts and quick testing. This section covers various commands for device management, app interaction, and UI automation. ```APIDOC ## CLI Commands Execute device commands directly from the command line for automation scripts and quick testing. ### List Connected Devices ```bash uiauto.dev android list uiauto.dev ios list ``` ### Tap Actions Tap at specific coordinates or using percentage-based coordinates. ```bash # Tap at coordinates (pixels) uiauto.dev android tap x=500 y=800 # Tap using percentage coordinates uiauto.dev android tap x=0.5 y=0.5 isPercent=true ``` ### Application Management Commands for interacting with applications on the device. ```bash # Get current app package name uiauto.dev android currentApp # Launch an application uiauto.dev android appLaunch package=com.android.settings # Terminate an application uiauto.dev android appTerminate package=com.android.settings # Install app from URL uiauto.dev android installApp url=https://example.com/app.apk ``` ### Device Control Commands for basic device control. ```bash # Press the home button uiauto.dev android home # Press the back button uiauto.dev android back # Wake up the device uiauto.dev android wakeUp ``` ### Information Retrieval Commands to get information about the device and its UI. ```bash # Get window size uiauto.dev android getWindowSize # Dump UI hierarchy uiauto.dev android dump ``` ### Utility Commands Commands for version checking, updating, and server control. ```bash # Print version uiauto.dev version # Update to latest version uiauto.dev self-update # Shutdown running server uiauto.dev shutdown ``` ``` -------------------------------- ### Execute Device Commands Source: https://context7.com/codeskyblue/uiautodev/llms.txt Performs various system-level operations such as navigation, volume control, and app lifecycle management. This is a unified interface for common device interactions. ```bash curl -X POST http://localhost:20242/api/android/emulator-5554/command/home curl -X POST http://localhost:20242/api/android/emulator-5554/command/appLaunch -H "Content-Type: application/json" -d '{"package": "com.android.settings", "stop": false}' ``` -------------------------------- ### Environment Variables Source: https://context7.com/codeskyblue/uiautodev/llms.txt Configure UIAutodev behavior through environment variables. These variables allow customization of the driver, enabling mock mode, and other settings. ```APIDOC ## Environment Variables Configure UIAutodev behavior through environment variables. ### UIAUTODEV_USE_ADB_DRIVER Use ADB driver instead of uiautomator2. This can be faster but offers fewer features. ```bash export UIAUTODEV_USE_ADB_DRIVER=1 ``` ### UIAUTODEV_MOCK Enable mock mode for testing without real devices. This is useful for development and testing purposes. ```bash export UIAUTODEV_MOCK=1 ``` ``` -------------------------------- ### Execute Device Command Source: https://context7.com/codeskyblue/uiautodev/llms.txt Executes various device commands including navigation, volume control, and app management through a unified command interface. ```APIDOC ## POST /api/{platform}/{serial}/command/{command} ### Description Executes various device commands including navigation, volume control, and app management through a unified command interface. ### Method POST ### Endpoint /api/{platform}/{serial}/command/{command} ### Parameters #### Path Parameters - **platform** (string) - Required - The platform of the device (e.g., android, ios). - **serial** (string) - Required - The serial number of the device. - **command** (string) - Required - The command to execute (e.g., home, back, wakeUp, volumeUp, appLaunch, sendKeys). #### Request Body - **package** (string) - Optional - The package name for app-related commands. - **stop** (boolean) - Optional - Whether to stop the app before launching (for appLaunch command). - **text** (string) - Optional - The text to send for sendKeys command. ### Request Example ```json { "package": "com.android.settings", "stop": false } ``` ### Response #### Success Response (200) - **width** (integer) - The width of the device screen (for getWindowSize command). - **height** (integer) - The height of the device screen (for getWindowSize command). #### Response Example ```json {"width": 1080, "height": 2400} ``` ``` -------------------------------- ### Connect to HarmonyOS MJPEG WebSocket Source: https://context7.com/codeskyblue/uiautodev/llms.txt Establishes a WebSocket connection to stream HarmonyOS device screens using the MJPEG format. Each received binary frame represents a complete JPEG image. ```python import asyncio import websockets async def connect_harmony(): uri = "ws://localhost:20242/ws/harmony/mjpeg/HARMONY-SERIAL" async with websockets.connect(uri) as ws: while True: data = await ws.recv() if isinstance(data, bytes): with open("frame.jpg", "wb") as f: f.write(data) print(f"Received frame: {len(data)} bytes") asyncio.run(connect_harmony()) ``` -------------------------------- ### Perform Tap Action Source: https://context7.com/codeskyblue/uiautodev/llms.txt Sends a tap command to the device. Supports both absolute pixel coordinates and percentage-based positioning. ```bash curl -X POST http://localhost:20242/api/android/emulator-5554/command/tap \ -H "Content-Type: application/json" \ -d '{"x": 500, "y": 800, "isPercent": false}' curl -X POST http://localhost:20242/api/android/emulator-5554/command/tap \ -H "Content-Type: application/json" \ -d '{"x": 0.5, "y": 0.5, "isPercent": true}' ``` -------------------------------- ### Find UI Elements Source: https://context7.com/codeskyblue/uiautodev/llms.txt Searches for UI elements in the current screen hierarchy using locators like ID, text, or class name. Supports setting a timeout for the search operation. ```bash curl -X POST http://localhost:20242/api/android/emulator-5554/command/findElements -H "Content-Type: application/json" -d '{"by": "id", "value": "com.example:id/button", "timeout": 10.0}' ``` -------------------------------- ### Capture Device Screenshot Source: https://context7.com/codeskyblue/uiautodev/llms.txt Captures a screenshot from a specific device and saves it as a JPEG file. Requires the platform, serial number, and display ID. ```bash curl http://localhost:20242/api/android/emulator-5554/screenshot/0 --output screenshot.jpg curl http://localhost:20242/api/ios/00008030-001234567890/screenshot/0 --output ios_screenshot.jpg curl http://localhost:20242/api/harmony/HDC-SERIAL/screenshot/0 --output harmony_screenshot.jpg ``` -------------------------------- ### Execute Shell Command Source: https://context7.com/codeskyblue/uiautodev/llms.txt Executes a shell command directly on the Android device via ADB. ```APIDOC ## POST /api/android/{serial}/shell ### Description Executes a shell command directly on the Android device via ADB. ### Method POST ### Endpoint /api/android/{serial}/shell ### Parameters #### Path Parameters - **serial** (string) - Required - The serial number of the Android device. #### Request Body - **command** (string) - Required - The shell command to execute. ### Request Example ```json { "command": "pm list packages -3" } ``` ### Response #### Success Response (200) - **output** (string) - The standard output of the command. - **error** (string) - The standard error of the command, if any. #### Response Example ```json { "output": "package:com.example.app1\npackage:com.example.app2", "error": null } ``` ``` -------------------------------- ### Click UI Element Source: https://context7.com/codeskyblue/uiautodev/llms.txt Locates a specific UI element and performs a click action on its center point. Includes automatic retry logic until the specified timeout is reached. ```bash curl -X POST http://localhost:20242/api/android/emulator-5554/command/clickElement -H "Content-Type: application/json" -d '{"by": "text", "value": "Submit", "timeout": 10.0}' ``` -------------------------------- ### Dump UI Hierarchy Source: https://context7.com/codeskyblue/uiautodev/llms.txt Retrieves the UI hierarchy tree of the current screen. Supports JSON and XML formats for parsing element properties. ```bash curl http://localhost:20242/api/android/emulator-5554/hierarchy curl "http://localhost:20242/api/android/emulator-5554/hierarchy?format=xml" ``` -------------------------------- ### POST /api/{platform}/{serial}/command/tap Source: https://context7.com/codeskyblue/uiautodev/llms.txt Performs a tap action on the device at specified coordinates. ```APIDOC ## POST /api/{platform}/{serial}/command/tap ### Description Performs a tap action on the device at specified coordinates, supporting both absolute pixel coordinates and percentage-based positioning. ### Method POST ### Endpoint /api/{platform}/{serial}/command/tap ### Request Body - **x** (number) - Required - X coordinate - **y** (number) - Required - Y coordinate - **isPercent** (boolean) - Required - Whether coordinates are percentages ### Request Example { "x": 500, "y": 800, "isPercent": false } ### Response #### Success Response (200) - **status** (string) - Operation result ``` -------------------------------- ### WebSocket /ws/harmony/mjpeg/{serial} Source: https://context7.com/codeskyblue/uiautodev/llms.txt Establishes a WebSocket connection for streaming HarmonyOS device screens in MJPEG format. This is useful for real-time monitoring of the device's display. ```APIDOC ## WebSocket /ws/harmony/mjpeg/{serial} ### Description Establishes a WebSocket connection for HarmonyOS device screen streaming using MJPEG format. ### Method WebSocket ### Endpoint /ws/harmony/mjpeg/{serial} ### Parameters #### Path Parameters - **serial** (string) - Required - The serial number of the HarmonyOS device. ### Request Example ```python import asyncio import websockets async def connect_harmony(): uri = "ws://localhost:20242/ws/harmony/mjpeg/HARMONY-SERIAL" async with websockets.connect(uri) as ws: while True: # Receive MJPEG frames data = await ws.recv() if isinstance(data, bytes): # Each frame is a complete JPEG image with open("frame.jpg", "wb") as f: f.write(data) print(f"Received frame: {len(data)} bytes") asyncio.run(connect_harmony()) ``` ### Response #### Success Response (101) - Switching Protocols - The connection is upgraded to WebSocket. - MJPEG frames (JPEG images) are sent as binary data. #### Response Example (Binary MJPEG frame data) ``` -------------------------------- ### Configure Environment Variables Source: https://context7.com/codeskyblue/uiautodev/llms.txt Sets environment variables to modify the behavior of the UIAutodev server, such as switching drivers or enabling mock modes for testing. ```bash export UIAUTODEV_USE_ADB_DRIVER=1 export UIAUTODEV_MOCK=1 ``` -------------------------------- ### Backup App Source: https://context7.com/codeskyblue/uiautodev/llms.txt Downloads an APK file from the device for backup purposes. ```APIDOC ## GET /api/{platform}/{serial}/backupApp ### Description Downloads an APK file from the device for backup purposes. ### Method GET ### Endpoint /api/{platform}/{serial}/backupApp ### Parameters #### Path Parameters - **platform** (string) - Required - The platform of the device (e.g., android, ios). - **serial** (string) - Required - The serial number of the device. #### Query Parameters - **packageName** (string) - Required - The package name of the app to back up. ### Response #### Success Response (200) - The response will be the APK file content. ### Request Example ```bash curl "http://localhost:20242/api/android/emulator-5554/backupApp?packageName=com.example.app" \ --output app_backup.apk ``` ``` -------------------------------- ### WebSocket Video Stream Rendering Source: https://github.com/codeskyblue/uiautodev/blob/master/examples/harmony-video.html This snippet initializes a WebSocket connection to receive binary image data, converts it into a Blob, and renders it onto an HTML5 canvas while maintaining aspect ratio. ```javascript const ws = new WebSocket('ws://localhost:8899'); ws.binaryType = 'arraybuffer'; ws.onmessage = function(event) { const arrayBuffer = event.data; const blob = new Blob([arrayBuffer], {type: 'image/jpeg'}); const url = URL.createObjectURL(blob); const img = new Image(); img.onload = function() { const containerWidth = canvasContainer.offsetWidth; const containerHeight = canvasContainer.offsetHeight; const aspectRatio = img.width / img.height; if (containerWidth / containerHeight > aspectRatio) { canvas.height = containerHeight; canvas.width = Math.floor(containerHeight * aspectRatio); } else { canvas.width = containerWidth; canvas.height = Math.floor(containerWidth / aspectRatio); } context.clearRect(0, 0, canvas.width, canvas.height); context.drawImage(img, 0, 0, canvas.width, canvas.height); URL.revokeObjectURL(url); }; img.src = url; }; ``` -------------------------------- ### Execute Shell Command Source: https://context7.com/codeskyblue/uiautodev/llms.txt Runs arbitrary shell commands directly on the Android device via ADB. Returns the standard output and error stream. ```bash curl -X POST http://localhost:20242/api/android/emulator-5554/shell -H "Content-Type: application/json" -d '{"command": "pm list packages -3"}' ``` -------------------------------- ### Find Elements Source: https://context7.com/codeskyblue/uiautodev/llms.txt Searches for UI elements in the current hierarchy using various locator strategies. ```APIDOC ## POST /api/{platform}/{serial}/command/findElements ### Description Searches for UI elements in the current hierarchy using various locator strategies including ID, text, class name, and XPath. ### Method POST ### Endpoint /api/{platform}/{serial}/command/findElements ### Parameters #### Path Parameters - **platform** (string) - Required - The platform of the device (e.g., android, ios). - **serial** (string) - Required - The serial number of the device. #### Request Body - **by** (string) - Required - The locator strategy (e.g., id, text, className, xpath). - **value** (string) - Required - The value to search for. - **timeout** (float) - Optional - The timeout in seconds for the search. ### Request Example ```json { "by": "id", "value": "com.example:id/button", "timeout": 10.0 } ``` ### Response #### Success Response (200) - **count** (integer) - The number of elements found. - **value** (array) - An array of found UI elements. - **key** (string) - Unique identifier for the element. - **name** (string) - The name or class of the element. - **bounds** (array) - The bounding box of the element [x1, y1, x2, y2]. - **properties** (object) - Key-value pairs of element properties. - **children** (array) - Child elements of the current element. #### Response Example ```json { "count": 1, "value": [ { "key": "0-1-2", "name": "android.widget.Button", "bounds": [0.1, 0.3, 0.9, 0.35], "properties": { "resource-id": "com.example:id/button", "text": "Click Me", "clickable": "true" }, "children": [] } ] } ``` ``` -------------------------------- ### Canvas Coordinate Mapping and Event Handling Source: https://github.com/codeskyblue/uiautodev/blob/master/examples/harmony-video.html This snippet captures mouse events on the canvas, calculates the scaled coordinates relative to the target device screen size, and sends the action to the backend via WebSocket. ```javascript function calculateCoordinates(event) { const rect = canvas.getBoundingClientRect(); const mouseXCanvas = event.clientX - rect.left; const mouseYCanvas = event.clientY - rect.top; const scaleX = phoneScreenWidth / canvas.width; const scaleY = phoneScreenHeight / canvas.height; return { mouseX: Math.floor(mouseXCanvas * scaleX), mouseY: Math.floor(mouseYCanvas * scaleY) }; } canvas.addEventListener('mouseup', function(event) { upCoordinates = calculateCoordinates(event); if (downCoordinates.mouseX === upCoordinates.mouseX && downCoordinates.mouseY === upCoordinates.mouseY) { sendMouseEvent('normal', downCoordinates.mouseX, downCoordinates.mouseY); } else { sendMouseMoveEvent('move', downCoordinates.mouseX, downCoordinates.mouseY, upCoordinates.mouseX, upCoordinates.mouseY); } }); ``` -------------------------------- ### WebSocket /ws/android/scrcpy/{serial} Source: https://context7.com/codeskyblue/uiautodev/llms.txt Establishes a WebSocket connection for real-time screen mirroring and touch control using the scrcpy protocol for Android devices. Allows sending touch events, key events, and text, while receiving video streams. ```APIDOC ## WebSocket /ws/android/scrcpy/{serial} ### Description Establishes a WebSocket connection for real-time screen mirroring and touch control using scrcpy protocol. ### Method WebSocket ### Endpoint /ws/android/scrcpy/{serial} ### Parameters #### Path Parameters - **serial** (string) - Required - The serial number of the Android device. #### Query Parameters - **version** (string) - Optional - The scrcpy protocol version. ### Request Example ```python import asyncio import websockets import json async def connect_scrcpy(): uri = "ws://localhost:20242/ws/android/scrcpy/emulator-5554?version=2.7" async with websockets.connect(uri) as ws: # Receive video stream (binary H264 data) async def receive_video(): while True: data = await ws.recv() if isinstance(data, bytes): # Process H264 video frame print(f"Received {len(data)} bytes of video data") # Send touch events async def send_touch(): # Touch down at center await ws.send(json.dumps({ "type": "touchDown", "xP": 0.5, # 50% from left "yP": 0.5 # 50% from top })) await asyncio.sleep(0.1) # Touch up await ws.send(json.dumps({ "type": "touchUp", "xP": 0.5, "yP": 0.5 })) # Send key event (HOME = 3) await ws.send(json.dumps({ "type": "keyEvent", "data": {"eventNumber": 3} })) # Send text input await ws.send(json.dumps({ "type": "text", "detail": "Hello from WebSocket" })) # Ping to keep connection alive await ws.send(json.dumps({"type": "ping"})) await asyncio.gather(receive_video(), send_touch()) asyncio.run(connect_scrcpy()) ``` ### Response #### Success Response (101) - Switching Protocols - The connection is upgraded to WebSocket. - Data frames will be sent and received according to the scrcpy protocol. #### Response Example (No direct response body, but data frames are exchanged) ```json { "type": "touchDown", "xP": 0.5, "yP": 0.5 } ``` ``` -------------------------------- ### Connect to Android Scrcpy WebSocket Source: https://context7.com/codeskyblue/uiautodev/llms.txt Establishes a WebSocket connection for real-time screen mirroring and remote control. It handles binary H264 video streams and sends touch, key, and text input events. ```python import asyncio import websockets import json async def connect_scrcpy(): uri = "ws://localhost:20242/ws/android/scrcpy/emulator-5554?version=2.7" async with websockets.connect(uri) as ws: async def receive_video(): while True: data = await ws.recv() if isinstance(data, bytes): print(f"Received {len(data)} bytes of video data") async def send_touch(): await ws.send(json.dumps({"type": "touchDown", "xP": 0.5, "yP": 0.5})) await asyncio.sleep(0.1) await ws.send(json.dumps({"type": "touchUp", "xP": 0.5, "yP": 0.5})) await ws.send(json.dumps({"type": "keyEvent", "data": {"eventNumber": 3}})) await ws.send(json.dumps({"type": "text", "detail": "Hello from WebSocket"})) await ws.send(json.dumps({"type": "ping"})) await asyncio.gather(receive_video(), send_touch()) asyncio.run(connect_scrcpy()) ``` -------------------------------- ### Click Element Source: https://context7.com/codeskyblue/uiautodev/llms.txt Finds an element using the specified locator and clicks on its center point. ```APIDOC ## POST /api/{platform}/{serial}/command/clickElement ### Description Finds an element using the specified locator and clicks on its center point, with automatic retry until timeout. ### Method POST ### Endpoint /api/{platform}/{serial}/command/clickElement ### Parameters #### Path Parameters - **platform** (string) - Required - The platform of the device (e.g., android, ios). - **serial** (string) - Required - The serial number of the device. #### Request Body - **by** (string) - Required - The locator strategy (e.g., id, text, xpath). - **value** (string) - Required - The value to search for. - **timeout** (float) - Optional - The timeout in seconds for the click action. ### Request Example ```json { "by": "text", "value": "Submit", "timeout": 10.0 } ``` ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.