### AppScanner: Initialize and Search Installed Apps (Kotlin)

Source: https://context7.com/turbo1123/roubao/llms.txt

Demonstrates how to initialize the AppScanner, refresh the app list, perform smart searches using various strategies (pinyin, semantic, direct lookup), filter by category, and format results for LLM context. It relies on the Android context for initialization.

```kotlin
val appScanner = AppScanner(context)

val apps = appScanner.refreshApps()
println("Found ${apps.size} installed apps")

val cachedApps = appScanner.getApps()

val results = appScanner.searchApps(
    query = "weixin",      // Pinyin search
    topK = 5,
    includeSystem = false
)

results.forEach { result ->
    println("${result.app.appName} (${result.app.packageName})")
    println("  Score: ${result.score}")
    println("  Match type: ${result.matchType}")  // pinyin_exact, contains, semantic
    println("  Category: ${result.app.category}")
}

val socialApps = appScanner.searchApps("聊天", topK = 10)

val foodApps = appScanner.searchApps("点外卖", topK = 5)

val packageName = appScanner.findPackage("淘宝")
println("Package: $packageName")  // com.taobao.taobao

val videoApps = appScanner.getAppsByCategory("视频")
videoApps.forEach { println(it.appName) }

val categories = appScanner.getAllCategories()
println("Categories: $categories")

val formattedResults = appScanner.formatSearchResultsForLLM(results)
println(formattedResults)
```

--------------------------------

### Skills Configuration: JSON-based Intent Definitions (JSON)

Source: https://context7.com/turbo1123/roubao/llms.txt

An example structure for `skills.json`, defining user intents with keywords, parameters, and related applications. It supports multiple execution strategies like delegation and GUI automation, including DeepLink templates and priority settings.

```json
[
  {
    "id": "order_food",
    "name": "点外卖",
    "description": "帮你点外卖、订餐",
    "category": "外卖",
    "keywords": ["点外卖", "叫外卖", "外卖", "点餐", "订餐", "吃饭", "饿了"],
    "params": [
      {
        "name": "food",
        "type": "string",
        "description": "想吃什么",
        "required": false,
        "examples": ["汉堡", "猪脚饭", "麻辣烫"]
      }
    ],
    "related_apps": [
      {
        "package": "com.meituan.android.beam",
        "name": "小美",
        "type": "delegation",
        "deep_link": "beam://www.meituan.com/home",
        "priority": 100,
        "description": "美团 AI 助手，直接对话点餐"
      },
      {
        "package": "me.ele",
        "name": "饿了么",
        "type": "gui_automation",
        "priority": 60,
        "steps": ["打开饿了么", "搜索食物", "选择商家", "下单"]
      }
    ]
  },
  {
    "id": "navigate",
    "name": "导航",
    "description": "导航到目的地",
    "category": "出行",
    "keywords": ["导航", "去", "怎么走", "路线"],
    "params": [
      {
        "name": "destination",
        "type": "string",
        "description": "目的地",
        "required": true
      }
    ],
    "related_apps": [
      {
        "package": "com.autonavi.minimap",
        "name": "高德地图",
        "type": "delegation",
        "deep_link": "amap://search?keyword={destination}&sourceApplication=roubao",
        "priority": 100
      }
    ]
  },
  {
    "id": "post_xiaohongshu",
    "name": "发小红书",
    "description": "在小红书发布笔记",
    "category": "社交",
    "keywords": ["发小红书", "小红书", "发笔记"],
    "prompt_hint": "小红书笔记内容不能超过100字，请精炼表达",
    "related_apps": [
      {
        "package": "com.xingin.xhs",
        "name": "小红书",
        "type": "gui_automation",
        "priority": 100,
        "steps": ["打开小红书", "点击底部+号发布", "选择图片", "输入内容（不超过100字）", "发布"]
      }
    ]
  }
]
```

--------------------------------

### Initialize and Use SkillManager for Intent Matching and Execution (Kotlin)

Source: https://context7.com/turbo1123/roubao/llms.txt

Demonstrates how to initialize SkillManager, enable LLM intent matching, perform keyword-based and LLM-powered intent matching, execute matched skills, and detect fast paths for delegation. It also shows how to generate agent context and retrieve all available skills.

```kotlin
// Initialize SkillManager
val skillManager = SkillManager.init(
    context = applicationContext,
    toolManager = toolManager,
    appScanner = appScanner
)
skillManager.setVLMClient(vlmClient)  // Enable LLM intent matching

// Keyword-based intent matching
val match = skillManager.matchAvailableApp("帮我点份外卖")
match?.let {
    println("Matched skill: ${it.skill.config.name}")  // 点外卖
    println("Best app: ${it.app.name}")                 // 小美 or 饿了么
    println("Execution type: ${it.app.type}")          // DELEGATION or GUI_AUTOMATION
    println("Confidence: ${it.score}")                  // 0.9
}

// LLM-powered intent matching (more accurate)
val llmMatch = skillManager.matchAvailableAppWithLLM("附近有什么好吃的")
llmMatch?.let {
    println("LLM matched: ${it.skill.config.name}")    // 找美食
    println("Reasoning from LLM available in logs")
}

// Execute matched skill
val result = skillManager.execute(match!!)
when (result) {
    is SkillResult.Delegated -> {
        println("Opened ${result.app.name} via DeepLink: ${result.deepLink}")
    }
    is SkillResult.NeedAutomation -> {
        val plan = result.plan
        println("GUI automation needed for ${plan.app.name}")
        println("Steps: ${plan.app.steps?.joinToString(" -> ")}")
        println("Agent context:\n${plan.toAgentContext()}")
    }
    is SkillResult.Failed -> {
        println("Failed: ${result.error}")
    }
    is SkillResult.NoAvailableApp -> {
        println("No installed app for ${result.skillName}")
        println("Required: ${result.requiredApps}")
    }
}

// Fast path detection for high-confidence delegation
val fastPath = skillManager.shouldUseFastPath("导航去星巴克")
fastPath?.let {
    println("Can use fast path with ${it.app.name}")
    // Execute immediately via DeepLink
}

// Generate context for Agent
val agentContext = skillManager.generateAgentContextWithLLM("给妈妈发个微信")
println(agentContext)
// Output:
// 根据用户意图，已匹配到技能：
// 【发消息】(置信度: 85%)
// 推荐应用: 微信 🤖GUI自动化
// 操作步骤: 打开微信 → 搜索联系人 → 进入聊天 → 输入消息 → 发送

// Get all skills for UI display
val allSkills = skillManager.getAllSkills()
val categories = allSkills.map { it.config.category }.distinct()
println("Categories: $categories")  // [外卖, 出行, AI, 音乐, 视频, 社交, ...]

```

--------------------------------

### Register and Execute Atomic Tools with ToolManager (Kotlin)

Source: https://context7.com/turbo1123/roubao/llms.txt

Illustrates initializing ToolManager, executing various tools by name with parameters, and retrieving tool descriptions. Supported tools include app searching, deep linking, shell command execution, and HTTP requests. It also shows how to access individual tools directly.

```kotlin
import com.example.roubao.ToolManager
import com.example.roubao.ToolResult

// Initialize ToolManager (singleton pattern)
val toolManager = ToolManager.init(
    context = applicationContext,
    deviceController = deviceController,
    appScanner = appScanner
)

// Execute tools by name with parameters
val searchResult = toolManager.execute(
    toolName = "search_apps",
    params = mapOf(
        "query" to "外卖",
        "top_k" to 5,
        "include_system" to false
    )
)

when (searchResult) {
    is ToolResult.Success -> {
        val apps = searchResult.data as List<Map<String, Any>>
        apps.forEach { app ->
            println("Found: ${app["app_name"]} (${app["package_name"]})")
        }
    }
    is ToolResult.Error -> {
        println("Error: ${searchResult.error}")
    }
}

// DeepLink tool for fast delegation
val deepLinkResult = toolManager.execute(
    toolName = "deep_link",
    params = mapOf("uri" to "amap://search?keyword=麦当劳")
)

// Shell command execution
val shellResult = toolManager.execute(
    toolName = "shell",
    params = mapOf("command" to "input tap 500 800")
)

// HTTP requests for external APIs
val httpResult = toolManager.execute(
    toolName = "http",
    params = mapOf(
        "url" to "https://api.example.com/data",
        "method" to "GET"
    )
)

// Get tool descriptions for LLM context
val toolDescriptions = toolManager.getToolDescriptions()
println(toolDescriptions)
// Output:
// 工具: search_apps
// 说明: 在已安装应用中搜索，支持应用名、拼音、语义描述等多种方式
// 参数:
//   - query: string (必填) - 搜索词
//   - top_k: int (可选) - 返回结果数量

// Access individual tools directly
val searchTool = toolManager.searchAppsTool
val bestMatch = searchTool.findBestMatch("微信")
println("Best match package: $bestMatch")  // com.tencent.mm
```

--------------------------------

### Orchestrate Automation with MobileAgent (Kotlin)

Source: https://context7.com/turbo1123/roubao/llms.txt

Shows how to create a MobileAgent instance, set a stop callback, execute natural language instructions with configurable parameters, and observe the agent's state and execution logs. It also covers programmatic stopping and clearing logs.

```kotlin
// Create MobileAgent instance
val agent = MobileAgent(
    vlmClient = vlmClient,
    controller = deviceController,
    context = applicationContext
)

// Set stop callback for user cancellation
agent.onStopRequested = {
    agentJob?.cancel()
}

// Execute natural language instruction
val result = agent.runInstruction(
    instruction = "打开微信给张三发消息说明天下午开会",
    maxSteps = 25,
    useNotetaker = true
)

when {
    result.success -> println("Task completed: ${result.message}")
    else -> println("Task failed: ${result.message}")
}

// Observe agent state via StateFlow
lifecycleScope.launch {
    agent.state.collect {
        state ->
        println("Running: ${state.isRunning}")
        println("Step: ${state.currentStep}")
        println("Instruction: ${state.instruction}")

        state.executionSteps.forEach {
            step ->
            println("Step ${step.stepNumber}: ${step.action}")
            println("  Thought: ${step.thought}")
            println("  Description: ${step.description}")
            println("  Outcome: ${step.outcome}")  // A=success, B=partial, C=failed
        }
    }
}

// Observe execution logs
lifecycleScope.launch {
    agent.logs.collect {
        logs ->
        logs.takeLast(5).forEach { println(it) }
    }
}

// Stop execution programmatically
agent.stop()
agent.clearLogs()

// Example execution flow logged by agent:
// [肉包] 开始执行: 打开微信给张三发消息
// [肉包] 使用 OpenAI 兼容模式
// [肉包] 正在分析意图...
// [肉包] 已匹配到可用技能:
//        【发消息】推荐应用: 微信 🤖GUI自动化
// [肉包] 屏幕尺寸: 1080x2400
// [肉包] 已加载 50 个应用
// [肉包] ========== Step 1 ==========
// [肉包] 截图中...
// [肉包] Manager 规划中...
// [肉包] 计划: First, open WeChat app...
// [肉包] Executor 决策中...
// [肉包] 思考: Current screen is home. Need to open WeChat...
// [肉包] 动作: open_app
// [肉包] 执行动作: open_app
// [肉包] Reflector 反思中...
// [肉包] 结果: A - Successfully opened WeChat

```

--------------------------------

### Initialize and Use VLMClient for AI Phone Automation (Kotlin)

Source: https://context7.com/turbo1123/roubao/llms.txt

Demonstrates initializing the VLMClient with various providers like Qwen-VL, GPT-4V, and Claude, including custom local models. It shows single-turn and multi-turn predictions using screenshots and conversation memory, along with fetching available models from an API. This client is crucial for the AI's ability to understand screen content and interact with the device.

```kotlin
import android.graphics.Bitmap
import com.roubao.vlm.client.VLMClient
import com.roubao.vlm.client.VLMConfigs
import com.roubao.vlm.client.ConversationMemory

// Initialize VLM client with different providers
val qwenClient = VLMClient(
    apiKey = "your-dashscope-api-key",
    baseUrl = "https://dashscope.aliyuncs.com/compatible-mode/v1",
    model = "qwen-vl-max"
)

// Or use convenience factory methods
val gpt4vClient = VLMConfigs.gpt4v("your-openai-key")
// val qwenClient = VLMConfigs.qwenVL("your-aliyun-key") // Duplicate declaration, assuming one is intended
val claudeClient = VLMConfigs.claude("your-anthropic-key")
val customClient = VLMConfigs.custom(
    apiKey = "key",
    baseUrl = "http://localhost:8000/v1",
    model = "local-vlm"
)

// Single-turn prediction with screenshot
// Assuming 'controller' is an instance that provides screenshots
// val screenshot: Bitmap = controller.screenshotWithFallback().bitmap
// For demonstration, let's assume 'screenshot' is obtained elsewhere
val screenshot: Bitmap? = null // Placeholder for actual screenshot

if (screenshot != null) {
    val result = qwenClient.predict(
        prompt = "Describe what's on this screen and identify clickable elements",
        images = listOf(screenshot)
    )

    result.onSuccess { response ->
        println("VLM Response: $response")
    }.onFailure { error ->
        println("Error: ${error.message}")
    }
} else {
    println("Screenshot not available for prediction.")
}


// Multi-turn conversation with context
val memory = ConversationMemory.withSystemPrompt(
    "You are an Android automation agent. Decide the next action."
)
// memory.addUserMessage("Current screen shows a login form", screenshot) // Requires screenshot

// Assuming memory is populated and screenshot is available for context
// val contextResponse = qwenClient.predictWithContext(memory.toMessagesJson())
// contextResponse.onSuccess { response ->
//     memory.addAssistantMessage(response)
//     memory.stripLastUserImage() // Remove image to save tokens
// }

// Fetch available models from API
val modelsResult = VLMClient.fetchModels(
    baseUrl = "https://dashscope.aliyuncs.com/compatible-mode/v1",
    apiKey = "your-api-key"
)
modelsResult.onSuccess {
    println("Available models:")
    it.forEach { modelName -> println("- $modelName") }
}.onFailure {
    println("Failed to fetch models: ${it.message}")
}

```

--------------------------------

### Control Device Screen and Apps with Shizuku (Kotlin)

Source: https://context7.com/turbo1123/roubao/llms.txt

Demonstrates initializing DeviceController, binding the Shizuku service, checking privilege levels, capturing screenshots with fallback, performing touch interactions, typing text, simulating system keys, and launching applications. It handles sensitive screen detection and uses fallback bitmaps when necessary.

```kotlin
import com.example.roubao.DeviceController
import com.example.roubao.ShizukuPrivilegeLevel

// Initialize and bind Shizuku service
val controller = DeviceController(context)
controller.bindService()

// Check availability and privilege level
if (controller.isShizukuAvailable()) {
    val level = controller.getShizukuPrivilegeLevel()
    when (level) {
        ShizukuPrivilegeLevel.ROOT -> println("Running with root access")
        ShizukuPrivilegeLevel.ADB -> println("Running with ADB access")
        ShizukuPrivilegeLevel.NONE -> println("Shizuku not connected")
    }
}

// Screen capture with fallback handling
val screenshotResult = controller.screenshotWithFallback()
when {
    screenshotResult.isSensitive -> {
        // Payment/password screen detected, screenshot blocked
        println("Sensitive screen detected")
    }
    screenshotResult.isFallback -> {
        // Screenshot failed, using black placeholder
        println("Using fallback bitmap")
    }
    else -> {
        val bitmap = screenshotResult.bitmap
        println("Screenshot: ${bitmap.width}x${bitmap.height}")
    }
}

// Touch interactions
controller.tap(540, 960)                    // Single tap at coordinates
controller.doubleTap(540, 960)              // Double tap
controller.longPress(540, 960, 1500)        // Long press for 1.5 seconds
controller.swipe(540, 1500, 540, 500, 500)  // Swipe up with 500ms duration

// Text input (supports Chinese via clipboard)
controller.type("Hello World")              // ASCII text via input command
controller.type("你好世界")                  // Chinese via clipboard paste
controller.typeCharByChar("Complex text!")  // Character-by-character input

// System keys
controller.back()                           // Back button (keyevent 4)
controller.home()                           // Home button (keyevent 3)
controller.enter()                          // Enter key (keyevent 66)

// App launching
controller.openApp("com.tencent.mm")        // By package name
controller.openApp("微信")                   // By app name (uses AppScanner)
controller.openDeepLink("weixin://scanqrcode")  // Via DeepLink

// Get screen dimensions (handles orientation)
val (width, height) = controller.getScreenSize()
println("Screen size: ${width}x${height}")
```

--------------------------------

### Generate and Execute DeepLinks using DeepLinkTool (Kotlin)

Source: https://context7.com/turbo1123/roubao/llms.txt

Demonstrates how to access predefined DeepLink templates, generate DeepLinks with parameters, and execute them using a controller. This is useful for integrating with various popular Chinese applications.

```kotlin
val templates = DeepLinkTool.TEMPLATES

val amapSearch = DeepLinkTool.fromTemplate(
    templateName = "amap_search",
    params = mapOf("query" to "星巴克")
)
println(amapSearch)  // amap://search?keyword=星巴克&sourceApplication=roubao

// Available templates by category:

// Food delivery
val meituanHome = "imeituan://www.meituan.com/waimai/home"
val meituanSearch = "imeituan://www.meituan.com/search?q={query}"
val xiaomeiChat = "xiaomei://chat?message={query}"

// Transportation
val amapRoute = "amap://route/plan?sourceApplication=roubao&dname={destination}"
val didiCall = "diditaxi://"
val baiduMapRoute = "baidumap://map/direction?destination={destination}&mode=driving"

// Social & Payment
val weixinScan = "weixin://scanqrcode"
val weixinPay = "weixin://pay"
val alipayScan = "alipays://platformapi/startapp?appId=10000007"
val alipayPay = "alipays://platformapi/startapp?appId=20000056"

// Music
val neteaseSearch = "orpheus://search?keyword={query}"
val qqmusicSearch = "qqmusic://qq.com/ui/search?key={query}"

// Video
val douyinSearch = "snssdk1128://search?keyword={query}"
val bilibiliSearch = "bilibili://search?keyword={query}"

// AI Assistants (Delegation targets)
val doubaoChat = "doubao://chat?message={query}"
val tongyiChat = "tongyi://chat?message={query}"

// Execute DeepLink via controller
controller.openDeepLink("amap://search?keyword=附近美食")
```

--------------------------------

### VLMClient - Vision Language Model API

Source: https://context7.com/turbo1123/roubao/llms.txt

The VLMClient provides a unified interface to various vision-language models like GPT-4V, Qwen-VL, and Claude. It handles image encoding, API communication, and supports single-turn and multi-turn conversations.

```APIDOC
## VLMClient - Vision Language Model API

### Description
Provides a unified interface to interact with various Vision Language Models (VLMs) including OpenAI GPT-4V, Alibaba Qwen-VL, and Claude. It handles image encoding, API communication with retry logic, and supports both single-turn and context-aware multi-turn conversations.

### Method
Various (primarily POST for predictions, GET for fetching models)

### Endpoint
N/A (Client-side library, interacts with various model provider endpoints)

### Parameters
#### Initialization Parameters
- **apiKey** (string) - Required - API key for the VLM provider.
- **baseUrl** (string) - Required - Base URL for the VLM API endpoint.
- **model** (string) - Required - The specific VLM model to use (e.g., "qwen-vl-max").

#### Prediction Parameters
- **prompt** (string) - Required - The natural language prompt for the VLM.
- **images** (List<Bitmap>) - Optional - A list of images to be analyzed by the VLM.
- **memory** (ConversationMemory) - Optional - For multi-turn conversations, provides context.

### Request Example (Kotlin)
```kotlin
// Initialize VLM client with different providers
val qwenClient = VLMClient(
    apiKey = "your-dashscope-api-key",
    baseUrl = "https://dashscope.aliyuncs.com/compatible-mode/v1",
    model = "qwen-vl-max"
)

// Or use convenience factory methods
val gpt4vClient = VLMConfigs.gpt4v("your-openai-key")
val qwenClient = VLMConfigs.qwenVL("your-aliyun-key")
val claudeClient = VLMConfigs.claude("your-anthropic-key")
val customClient = VLMConfigs.custom(
    apiKey = "key",
    baseUrl = "http://localhost:8000/v1",
    model = "local-vlm"
)

// Single-turn prediction with screenshot
val screenshot: Bitmap = controller.screenshotWithFallback().bitmap
val result = qwenClient.predict(
    prompt = "Describe what\'s on this screen and identify clickable elements",
    images = listOf(screenshot)
)

// Multi-turn conversation with context
val memory = ConversationMemory.withSystemPrompt(
    "You are an Android automation agent. Decide the next action."
)
mory.addUserMessage("Current screen shows a login form", screenshot)
val contextResponse = qwenClient.predictWithContext(memory.toMessagesJson())
```

### Response
#### Success Response
- **response** (string) - The VLM's textual response to the prompt and images.
- **models** (List<string>) - List of available model names when fetching models.

#### Response Example (Prediction)
```json
{
  "response": "The screen shows a login form with fields for username and password. There is a 'Login' button."
}
```

#### Response Example (Fetch Models)
```json
{
  "models": ["qwen-vl-max", "gpt-4-vision-preview"]
}
```

### Error Handling
- **onFailure** (error) - Handles errors during API communication or prediction.

```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.