DeskMate

https://github.com/zhaohb/deskmate
Admin
DeskMate is a local-first desktop activity recorder for Windows that captures screen, audio, and...
Tokens:140,506
Snippets:864
Trust Score:9.3
License:MIT
Update:5 days ago
Show doc for...
Tokens:
Raw
### Install training extra and setup Intel XPU

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Install the 'training' extra and run the setup script for Intel Arc/Core-Ultra iGPU (XPU). Requires oneAPI/Level-Zero/MSVC toolchain; the script pulls torch 2.10.0+xpu and Level-Zero SDK headers.

```powershell
pip install "WHL[training]"
scripts\setup-intel-xpu.bat          :: pulls torch 2.10.0+xpu + Level-Zero SDK headers
```

--------------------------------

### Install from a built wheel (core and recommended combo)

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Use the same `[extras]` syntax on the wheel file. The first command installs only the core, the second installs the recommended combo (ocr-winrt, audio, vad, mcp).

```powershell
:: core only
pip install deskmate-0.1.0-py3-none-any.whl

:: recommended combo (same as Step 3)
pip install "deskmate-0.1.0-py3-none-any.whl[ocr-winrt,audio,vad,mcp]"
```

--------------------------------

### Run one-time XPU setup script

Source: https://github.com/zhaohb/deskmate/blob/main/docs/16-learning-training.md

Runs the setup script that installs Level-Zero SDK headers and the XPU build of torch. This is a one-time step after creating the environment.

```bat
scripts\setup-intel-xpu.bat
```

--------------------------------

### Start DeskMate browser UI

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Use 'deskmate ui' to start the recording daemon and open the /deskmate web interface. Use '--no-run-daemon' to view existing data without starting new recording.

```powershell
deskmate ui                  # record + API + auto-open /deskmate
deskmate ui --no-run-daemon  # view existing data only, don't start new recording
```

--------------------------------

### Run the one-time setup script for Intel XPU

Source: https://github.com/zhaohb/deskmate/blob/main/docs/16-learning-training.md

Executes the setup script that downloads Level-Zero SDK headers and force-installs the XPU torch wheel. Idempotent and one-time.

```bat
:: one-time, idempotent
scripts\setup-intel-xpu.bat
```

--------------------------------

### Install optional modules with pip extras

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Use `pip install -e ".[module-name]"` to install one module, or `pip install -e ".[module1,module2]"` for multiple modules. Commas separate module names with no spaces.

```powershell
pip install -e ".[module-name]"          # install one
pip install -e ".[module1,module2]"      # install several (comma-separated, NO spaces)
```

--------------------------------

### Install DeskMate with training extras

Source: https://github.com/zhaohb/deskmate/blob/main/docs/16-learning-training.md

Installs the training dependencies: torch, unsloth, transformers, peft, and accelerate. Required for actual training; without it, imports and data mining still work but training is gated.

```bash
pip install 'deskmate[training]'      # torch + unsloth + transformers + peft + accelerate
```

--------------------------------

### deskmate train-lora CLI examples

Source: https://github.com/zhaohb/deskmate/blob/main/docs/16-learning-training.md

Shows three usage patterns for the `train-lora` command: a dry-run preview, a training run with specified sources and epochs, and exporting the dataset to JSONL. Requires `pip install 'deskmate[training]'` for training.

```bash
# Preview the mined data without training (no torch needed)
deskmate train-lora --dry-run
```

```bash
# Train (requires: pip install 'deskmate[training]')
deskmate train-lora --epochs 3 --sources habits,apps,pipes,behavior,ask,profile
```

```bash
# Inspect the exact dataset a run would use, as JSONL, without training
deskmate train-lora --export ~/.deskmate/sft_preview.jsonl
```

--------------------------------

### Install onnx_cpu backend dependencies

Source: https://github.com/zhaohb/deskmate/blob/main/docs/04-audio.md

Install the audio and VAD extras for the default onnx_cpu backend. This is the most compatible option, running on CPU without special hardware.

```powershell
pip install -e ".[audio,vad]"
```

--------------------------------

### Install and launch MCP server

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Install the MCP extra and run the API and MCP server in separate terminals. The API must be running before the MCP server can proxy requests.

```powershell
pip install -e ".[mcp]"     # or add `mcp` to your extras combo
deskmate serve              # in one terminal — the API on :3030
deskmate mcp                # in another — the MCP stdio server
```

--------------------------------

### Install OpenVINO Whisper dependencies

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Run this PowerShell command to install the package with OpenVINO and VAD audio extras for Intel devices.

```powershell
pip install -e ".[audio-openvino,vad]"
```

--------------------------------

### Install training extra for CPU only

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Install the 'training' extra on CPU-only systems. The default torch is already CPU-only; suitable for validating the pipeline on a tiny base like Qwen/Qwen3-0.6B, not real ≥1B runs.

```powershell
pip install "WHL[training]"          :: the default torch is already CPU-only
```

--------------------------------

### Create training environment and install DeskMate

Source: https://github.com/zhaohb/deskmate/blob/main/docs/16-learning-training.md

Creates a conda environment named 'deskmate_train' with Python 3.10 and installs DeskMate with training extras. This pulls a CPU torch, which is replaced in the next step.

```bat
mamba create -n deskmate_train python=3.10 -y
mamba run -n deskmate_train pip install -e .[training]
```

--------------------------------

### POST /models/start

Source: https://github.com/zhaohb/deskmate/blob/main/docs/20-model-service.md

Starts the model service. Returns a 400 error if the executable is missing.

```APIDOC
## POST /models/start

### Description
Starts the model service. Returns a 400 error if the executable is missing.

### Method
POST

### Endpoint
/models/start

### Response
#### Success Response (200)
- **message** (string) - Confirmation that the service started.

#### Response Example
{
  "message": "Service started"
}

#### Error Response (400)
- **error** (string) - Description of the missing executable.

#### Error Example
{
  "error": "Executable not found"
}
```

--------------------------------

### Offline installation with pip download

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Pre-downloads dependencies for the specified extras into a `wheelhouse` folder, then installs on an offline machine using `--no-index` and `--find-links`.

```powershell
pip download "deskmate-0.1.0-py3-none-any.whl[ocr-winrt,audio,vad,mcp]" -d wheelhouse
:: then on the offline machine:
pip install --no-index --find-links wheelhouse "deskmate[ocr-winrt,audio,vad,mcp]"
```

--------------------------------

### Install almost all optional features from a wheel

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Installs every optional feature except `training`, `redact-onnx-dml`, and `dev`. Expect a large download due to heavy dependencies like PyTorch, OpenVINO, ONNX Runtime, Whisper, and RapidOCR.

```powershell
pip install "deskmate-0.1.0-py3-none-any.whl[ocr-tesseract,ocr-rapidocr,ocr-winrt,audio,audio-openvino,vad,speaker,redact-onnx,semantic,mcp,pipes]"
```

--------------------------------

### Launch DeskMate from training environment

Source: https://github.com/zhaohb/deskmate/blob/main/docs/16-learning-training.md

Starts the DeskMate UI from the 'deskmate_train' environment. A plain launch works; the toolchain is bootstrapped at train time.

```bat
mamba run -n deskmate_train deskmate ui
```

--------------------------------

### Install DeskMate with recommended features

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Installs DeskMate with core features plus built-in Windows OCR, audio transcription, voice activity detection, and MCP support. This is the recommended default combo for most users.

```powershell
pip install -e ".[ocr-winrt,audio,vad,mcp]"
```

--------------------------------

### Training completion summary output

Source: https://github.com/zhaohb/deskmate/blob/main/docs/16-learning-training.md

Example status box shown when training finishes, displaying sample count, epochs, total steps, average loss, and the adapter path. The adapter path defaults to ~/.deskmate/checkpoints/lora/final.

```text
训练完成 ✓ / Training complete ✓
Samples: 42        ← how many input→output pairs were used
Epochs: 3   Total steps: 33
Avg loss: 1.8423   ← see below
Adapter: C:\Users\<you>\.deskmate\checkpoints\lora\final
```

--------------------------------

### Verify XPU availability after setup

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Run this Python one-liner to confirm the XPU torch build is live. Expect output: 2.10.0+xpu True.

```powershell
python -c "import torch; print(torch.__version__, getattr(torch, 'xpu', None) is not None and torch.xpu.is_available())"
:: expect:  2.10.0+xpu True
```

--------------------------------

### GET /capture/ui

Source: https://github.com/zhaohb/deskmate/blob/main/docs/15-fusion-timeline.md

Serves the capture.html user interface.

```APIDOC
## GET /capture/ui

### Description
Serves the capture.html user interface.

### Method
GET

### Endpoint
/capture/ui

### Response
#### Success Response (200)
- **body** (html) - The capture.html page content.
```

--------------------------------

### Launch DeskMate UI

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Starts the DeskMate UI and begins recording. Your browser opens at http://127.0.0.1:3030/deskmate. On first run it auto-creates a config and data folder under C:\Users\<your-username>\.deskmate\. Stop with Ctrl + C in the terminal.

```powershell
deskmate ui
```

--------------------------------

### GET /power/apps

Source: https://github.com/zhaohb/deskmate/blob/main/docs/22-power-manager.md

Enumerates visible-window apps along with per-app throttle-ability and current eco state.

```APIDOC
## GET /power/apps

### Description
Enumerates visible-window apps along with per-app throttle-ability and current eco state.

### Method
GET

### Endpoint
/power/apps

### Response
#### Success Response (200)
- **apps** (array) - List of visible-window apps with their throttle-ability and eco state.

#### Response Example
{
  "apps": [
    {
      "pid": 1234,
      "name": "example.exe",
      "throttleable": true,
      "eco": false
    }
  ]
}
```

--------------------------------

### GET /power/ui

Source: https://github.com/zhaohb/deskmate/blob/main/docs/22-power-manager.md

Legacy standalone page entry that now 307-redirects to /deskmate (Battery Saver is a SPA view).

```APIDOC
## GET /power/ui

### Description
Legacy standalone page entry that now 307-redirects to /deskmate (Battery Saver is a SPA view).

### Method
GET

### Endpoint
/power/ui

### Response
#### Success Response (307)
- **Location** (string) - Redirect target: /deskmate

#### Response Example
(Redirect to /deskmate)
```

--------------------------------

### Clone repository and create virtual environment

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Clones the DeskMate repository and creates an isolated Python virtual environment. Activate the environment with .\.venv\Scripts\Activate.ps1 in every new terminal; you'll know it worked when you see (.venv) at the start of your prompt.

```powershell
git clone <repo-url>
cd deskmate

# Create an isolated Python environment (keeps your system clean)
python -m venv .venv
.\\.venv\\Scripts\\Activate.ps1
```

--------------------------------

### Live meeting translation configuration (TOML)

Source: https://github.com/zhaohb/deskmate/blob/main/docs/18-live-translation.md

Example configuration for live meeting translation into Chinese using endpoint chunk mode and balanced latency.

```toml
[audio]
enabled = true
chunk_mode = "endpoint"
translate_enabled = true
translate_target_lang = "zh"
translate_latency_mode = "balanced"

```

--------------------------------

### DeskMate MCP server configuration for Cursor

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Add this JSON block to ~/.cursor/mcp.json (global) or <project>/.cursor/mcp.json. The command points to the Python environment with the mcp extra installed.

```json
{
  "mcpServers": {
    "deskmate": {
      "command": "C:/path/to/env/python.exe",
      "args": ["-m", "deskmate", "mcp"],
      "env": { "DESKMATE_API": "http://127.0.0.1:3030" }
    }
  }
}
```

--------------------------------

### Split API and recorder into two processes

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Run the recorder and API as separate processes. The recorder runs alone, and the API runs without starting a new recording daemon.

```powershell
deskmate record
deskmate serve --no-run-daemon
```

--------------------------------

### GET /meetings

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/SKILL.md

Lists detected or recorded meetings, most recent first.

```APIDOC
## GET /meetings

### Description
Lists detected or recorded meetings, most recent first.

### Method
GET

### Endpoint
/meetings

### Parameters
#### Query Parameters
- **limit** (integer) - Optional - Maximum number of meetings to return (default 1)

### Response
#### Success Response (200)
- **meetings** (array) - List of meeting objects

Each meeting row has `id`, `name`, `note`, `started_at`, `ended_at`, `metadata`.

#### Response Example
{
  "meetings": [
    {
      "id": 1,
      "name": "Team Meeting",
      "note": "",
      "started_at": "2023-01-01T00:00:00Z",
      "ended_at": "2023-01-01T01:00:00Z",
      "metadata": {}
    }
  ]
}
```

--------------------------------

### GET /meetings/{id}

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/SKILL.md

Retrieves a specific meeting along with its transcript segments.

```APIDOC
## GET /meetings/{id}

### Description
Retrieves a specific meeting along with its transcript segments.

### Method
GET

### Endpoint
/meetings/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - Meeting ID

### Response
#### Success Response (200)
- **id** (string) - Meeting ID
- **name** (string) - Meeting name
- **note** (string) - Meeting note
- **started_at** (string) - Start timestamp
- **ended_at** (string) - End timestamp
- **metadata** (object) - Additional metadata
- **transcript** (array) - List of transcript segments

Each transcript segment has `text`, `speaker_name`, `start_time`, `end_time`.

#### Response Example
{
  "id": 1,
  "name": "Team Meeting",
  "note": "",
  "started_at": "2023-01-01T00:00:00Z",
  "ended_at": "2023-01-01T01:00:00Z",
  "metadata": {},
  "transcript": [
    {
      "text": "Hello everyone",
      "speaker_name": "Alice",
      "start_time": "2023-01-01T00:00:00Z",
      "end_time": "2023-01-01T00:00:05Z"
    }
  ]
}
```

--------------------------------

### GET /timeline/unified/breakdown

Source: https://github.com/zhaohb/deskmate/blob/main/docs/15-fusion-timeline.md

Returns per-source event counts since a specified time.

```APIDOC
## GET /timeline/unified/breakdown

### Description
Returns per-source event counts since a specified time.

### Method
GET

### Endpoint
/timeline/unified/breakdown

### Parameters
#### Query Parameters
- **since** (string) - Required - ISO timestamp to count events after.

### Response
#### Success Response (200)
- **counts** (object) - Per-source event counts.

#### Response Example
{
  "counts": {
    "screen": 10,
    "audio": 5,
    "input": 3,
    "clipboard": 2
  }
}
```

--------------------------------

### Run user-learning app

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/README.md

Runs the user-learning app with an 8-hour window and verbose output. Detects learning phases and generates a recap.

```cmd
python deskmate\apps\user-learning\app.py --hours 8 --verbose
```

--------------------------------

### Configure OpenVINO Whisper backend in config.toml

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Set the whisper_backend to openvino_genai and choose the OpenVINO device (NPU, GPU, CPU, or AUTO). The model auto-downloads from ModelScope on first use and falls back to CPU if the device fails.

```toml
[audio]
whisper_backend = "openvino_genai"   # default is "onnx_cpu"
openvino_device = "NPU"              # NPU | GPU | CPU | AUTO
```

--------------------------------

### Run video-export app with 5-minute window

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/README.md

Activates the support_qwen3.5 conda environment and runs the video-export app, which exports recent screenshots as a video clip. The --minutes 5 flag sets the time window, and --verbose enables detailed logging.

```cmd
cd /d C:\hongbo\UX\deskmate
conda activate support_qwen3.5
python deskmate\apps\video-export\app.py --minutes 5 --verbose
```

--------------------------------

### GET /capture/control

Source: https://github.com/zhaohb/deskmate/blob/main/docs/15-fusion-timeline.md

Retrieves the current pause/auto-resume state and per-source switch state for capture control.

```APIDOC
## GET /capture/control

### Description
Retrieves the current pause/auto-resume state and per-source switch state for capture control.

### Method
GET

### Endpoint
/capture/control

### Response
#### Success Response (200)
- **pause_until** (string) - Optional - ISO timestamp for auto-resume, null if not paused.
- **sources** (object) - Per-source switch state (screen, audio, input, clipboard).

#### Response Example
{
  "pause_until": null,
  "sources": {
    "screen": true,
    "audio": true,
    "input": true,
    "clipboard": true
  }
}
```

--------------------------------

### GET /meetings/{id}/transcript

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/SKILL.md

Retrieves the transcript for a specific meeting, including the full text and segments.

```APIDOC
## GET /meetings/{id}/transcript

### Description
Retrieves the transcript for a specific meeting, including the full text and segments.

### Method
GET

### Endpoint
/meetings/{id}/transcript

### Parameters
#### Path Parameters
- **id** (string) - Required - Meeting ID

### Response
#### Success Response (200)
- **text** (string) - Full transcript text
- **segments** (array) - List of transcript segments

Each transcript segment has `text`, `speaker_name`, `start_time`, `end_time`.

#### Response Example
{
  "text": "Hello everyone...",
  "segments": [
    {
      "text": "Hello everyone",
      "speaker_name": "Alice",
      "start_time": "2023-01-01T00:00:00Z",
      "end_time": "2023-01-01T00:00:05Z"
    }
  ]
}
```

--------------------------------

### Run todo-list app

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/README.md

Builds a unified todo list from email and meeting evidence. Outputs GitHub-style checkboxes and persists to the todos table via POST /todos unless --no-store is passed.

```cmd
python deskmate\apps\todo-list\app.py --hours 24 --verbose
```

--------------------------------

### Build a wheel for distribution

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Produces a distributable wheel at `dist\deskmate-0.1.0-py3-none-any.whl`. The wheel bundles DeskMate's own code and built-in apps but not third-party dependencies.

```powershell
pip install build
python -m build --wheel        :: output: dist\deskmate-0.1.0-py3-none-any.whl
```

--------------------------------

### GET /timeline/unified

Source: https://github.com/zhaohb/deskmate/blob/main/docs/15-fusion-timeline.md

Returns a fused feed of events from all sources, with optional filtering by time range, source, and limit.

```APIDOC
## GET /timeline/unified

### Description
Returns a fused feed of events from all sources, with optional filtering by time range, source, and limit.

### Method
GET

### Endpoint
/timeline/unified

### Parameters
#### Query Parameters
- **since** (string) - Optional - ISO timestamp to filter events after.
- **until** (string) - Optional - ISO timestamp to filter events before.
- **sources** (string) - Optional - Comma-separated list of source names to include.
- **limit** (integer) - Optional - Maximum number of events to return (≤1000).

### Response
#### Success Response (200)
- **data** (array) - List of Event objects.
- **total** (integer) - Total number of events.

#### Response Example
{
  "data": [
    {
      "ts": "2024-01-01T00:00:00Z",
      "source": "screen",
      "kind": "snapshot",
      "app_name": "ExampleApp",
      "window_title": "Example Window",
      "summary": "Example summary",
      "payload": {},
      "confidence": 0.95,
      "frame_id": "abc123"
    }
  ],
  "total": 1
}
```

--------------------------------

### Configure onnx_cpu backend in config.toml

Source: https://github.com/zhaohb/deskmate/blob/main/docs/04-audio.md

Configuration for the default onnx_cpu backend. The model is downloaded from HuggingFace on first use and cached under ~/.cache/huggingface/. The openvino_* keys are ignored by this backend.

```toml
[audio]
enabled = true
whisper_backend = "onnx_cpu"
whisper_model = "small"       # tiny | base | small | medium | large-v3
device = "cpu"                # faster-whisper device
compute_type = "int8"         # int8 | int8_float16 | float16 | float32
languages = ["zh"]            # [] = auto-detect
```

--------------------------------

### GET /models/status

Source: https://github.com/zhaohb/deskmate/blob/main/docs/20-model-service.md

Retrieves the current status of the model service. This endpoint is polled by the UI to display service state.

```APIDOC
## GET /models/status

### Description
Retrieves the current status of the model service. This endpoint is polled by the UI to display service state.

### Method
GET

### Endpoint
/models/status

### Response
#### Success Response (200)
- **status** (object) - The status object returned by `modelsvc.status(cfg)`.

#### Response Example
{
  "status": {}
}
```

--------------------------------

### POST /power/apps/eco

Source: https://github.com/zhaohb/deskmate/blob/main/docs/22-power-manager.md

Pushes a user-selected process onto E-cores. Returns 409 if unable, likely due to needing admin privileges.

```APIDOC
## POST /power/apps/eco

### Description
Pushes a user-selected process onto E-cores. Returns 409 if unable, likely due to needing admin privileges.

### Method
POST

### Endpoint
/power/apps/eco

### Parameters
#### Request Body
- **pid** (integer) - Required - The process ID to push onto E-cores.

### Request Example
{
  "pid": 1234
}

### Response
#### Success Response (200)
- **status** (string) - Success message.

#### Response Example
{
  "status": "ok"
}

#### Error Response (409)
- **error** (string) - Error message indicating inability to push (likely needs admin).

#### Error Example
{
  "error": "unable to push process"
}
```

--------------------------------

### GET /power/cores

Source: https://github.com/zhaohb/deskmate/blob/main/docs/22-power-manager.md

Retrieves per-core load grouped by P/E cores. This endpoint is retained in the backend but is not currently used by the frontend.

```APIDOC
## GET /power/cores

### Description
Retrieves per-core load grouped by P/E cores. This endpoint is retained in the backend but is not currently used by the frontend.

### Method
GET

### Endpoint
/power/cores

### Response
#### Success Response (200)
- **cores** (array) - List of core load information, grouped by P/E.

#### Response Example
{
  "cores": [
    {
      "type": "P",
      "load": 45
    },
    {
      "type": "E",
      "load": 20
    }
  ]
}
```

--------------------------------

### Run standup-update app

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/README.md

Generates a standup update (Yesterday / Today / Blockers) from the supplied time window. Uses rich prefetch and a single LLM pass.

```cmd
python deskmate\apps\standup-update\app.py --hours 24 --verbose
```

--------------------------------

### Run email-compose app (new draft and reply with send)

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/README.md

Drafts an email or reply for Gmail or Outlook. Optionally fetches a source message via --reply-to. Pass --send to actually deliver.

```cmd
:: New draft (review only)
python deskmate\apps\email-compose\app.py --provider gmail --to a@b.com --intent "需要本周确认预算" --verbose

:: Reply, then actually send
python deskmate\apps\email-compose\app.py --provider outlook --to a@b.com --intent "我同意方案 A" --reply-to AAMkAGI... --send --verbose
```

--------------------------------

### Activate virtual environment on Windows

Source: https://github.com/zhaohb/deskmate/blob/main/README.md

Run this command in the project folder to activate the virtual environment so that 'deskmate' is available in the terminal.

```powershell
.\\.venv\\Scripts\\Activate.ps1
```

--------------------------------

### GET /training/data

Source: https://github.com/zhaohb/deskmate/blob/main/docs/16-learning-training.md

Preview mined training data pairs without training. Returns sources, per-source breakdown, total count, and a sample of the data.

```APIDOC
## GET /training/data

### Description
Preview mined pairs: sources, per-source breakdown, total, and a sample.

### Method
GET

### Endpoint
/training/data

### Response
#### Success Response (200)
- **sources** (array) - List of source names.
- **breakdown** (object) - Per-source breakdown of mined pairs.
- **total** (integer) - Total number of mined pairs.
- **sample** (array) - Sample of the mined pairs.

#### Response Example
{
  "sources": ["habits", "apps"],
  "breakdown": {"habits": 10, "apps": 5},
  "total": 15,
  "sample": []
}
```

--------------------------------

### GET /search

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/SKILL.md

Searches screen captures (OCR text, accessibility text) and audio transcriptions. Supports filtering by content type, time range, app, window, text length, and speaker IDs.

```APIDOC
## GET /search

### Description
Searches screen captures (OCR text, accessibility text) and audio transcriptions. Supports filtering by content type, time range, app, window, text length, and speaker IDs.

### Method
GET

### Endpoint
/search

### Parameters
#### Query Parameters
- **q** (string) - Optional - Text search query (optional for OCR; required for audio/UI)
- **content_type** (string) - Optional - Filter by content type: "all" | "ocr" | "audio" | "ui"
- **limit** (integer) - Optional - Max results (default 20)
- **offset** (integer) - Optional - Pagination offset
- **start_time** (string) - Optional - ISO8601 start timestamp
- **end_time** (string) - Optional - ISO8601 end timestamp
- **app_name** (string) - Optional - Filter by app (e.g. "chrome.exe", "Cursor.exe")
- **window_name** (string) - Optional - Filter by window title
- **min_length** (integer) - Optional - Filter by minimum text length
- **max_length** (integer) - Optional - Filter by maximum text length
- **speaker_ids** (string) - Optional - Filter audio by speaker IDs (comma-separated)

### Response
#### Success Response (200)
- **data** (array) - List of ContentItem objects
- **pagination** (object) - Pagination details

Each ContentItem has `type` ("OCR" | "Audio" | "UI") and `content` with:
- OCR: text, app_name, window_name, browser_url, timestamp, frame_id
- Audio: transcription, device, timestamp, speaker_id, language
- UI: event_type, app_name, window_title, timestamp, data

#### Response Example
{
  "data": [
    {
      "type": "OCR",
      "content": {
        "text": "example",
        "app_name": "chrome.exe",
        "window_name": "Example",
        "browser_url": "https://example.com",
        "timestamp": "2023-01-01T00:00:00Z",
        "frame_id": 123
      }
    }
  ],
  "pagination": {"total": 1, "limit": 20, "offset": 0}
}
```

--------------------------------

### Configure always-learning whitelist in config.toml

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/README.md

Sets a whitelist of trusted sources for automatic learning detection. Entries are matched case-insensitively against URL, host, window title, app name, and on-screen text. Entries under 3 characters are refused. Editable live via GET/POST /config/learning/always-learning.

```toml
[learning]
always_learning = ["OpenVINO中文社区", "docs.openvino.ai", "space.bilibili.com/123456"]
```

--------------------------------

### Run day-recap app with 16-hour window

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/README.md

Runs the day-recap app, which queries screen and audio data to generate a structured day recap. The --hours 16 flag sets the time window, and --verbose enables detailed logging.

```cmd
python deskmate\apps\day-recap\app.py --hours 16 --verbose
```

--------------------------------

### GET /power/status

Source: https://github.com/zhaohb/deskmate/blob/main/docs/22-power-manager.md

Retrieves the current power source, eco active state, and eco thread count. This endpoint feeds the capsule and main card, and degrades to raw OS read when the daemon is absent.

```APIDOC
## GET /power/status

### Description
Retrieves the current power source, eco active state, and eco thread count. This endpoint feeds the capsule and main card, and degrades to raw OS read when the daemon is absent.

### Method
GET

### Endpoint
/power/status

### Response
#### Success Response (200)
- **power_source** (string) - The current power source (e.g., battery, AC).
- **eco_active** (boolean) - Whether eco mode is active.
- **eco_thread_count** (integer) - The number of eco threads.

#### Response Example
{
  "power_source": "battery",
  "eco_active": true,
  "eco_thread_count": 4
}
```

--------------------------------

### Configure openvino_genai backend in config.toml

Source: https://github.com/zhaohb/deskmate/blob/main/docs/04-audio.md

Configuration for the openvino_genai backend. The model is auto-downloaded from ModelScope on first use (or loaded from a local GenAI-IR directory) and cached under ~/.cache/modelscope/. The whisper_model, device, and compute_type keys are ignored by this backend.

```toml
[audio]
enabled = true
whisper_backend = "openvino_genai"
openvino_genai_model = "OpenVINO/whisper-medium-int8-ov"  # ModelScope id or local IR dir
openvino_device = "NPU"       # NPU | GPU | CPU | AUTO
languages = ["zh"]            # [] = auto-detect
```

--------------------------------

### GET /activity-summary

Source: https://github.com/zhaohb/deskmate/blob/main/deskmate/apps/SKILL.md

Retrieves an aggregated summary of user activity over a specified time range, including app usage, key texts, edited files, audio summaries, and guidance. This is the preferred endpoint for broad overviews and day recaps.

```APIDOC
## GET /activity-summary

### Description
Retrieves an aggregated summary of user activity over a specified time range, including app usage, key texts, edited files, audio summaries, and guidance. This is the preferred endpoint for broad overviews and day recaps.

### Method
GET

### Endpoint
/activity-summary

### Parameters
#### Query Parameters
- **start_time** (string) - Required - ISO8601 start timestamp
- **end_time** (string) - Required - ISO8601 end timestamp

### Response
#### Success Response (200)
- **apps** (array) - List of apps with usage minutes and frame counts
- **windows** (array) - List of windows with app name, window name, minutes, and browser URL
- **key_texts** (array) - List of key texts with text, app name, window name, and timestamp
- **edited_files** (array) - List of edited file paths with frame counts
- **audio_summary** (object) - Audio summary with segment count, speakers, and top transcriptions
- **snippets** (array) - List of snippets with source, text, and timestamp
- **data_status** (string) - Status of captured data: ok, no_capture_in_range, not_recording, or empty_but_recording
- **guidance** (object) - Guidance with next best query

#### Response Example
{
  "apps": [{"name": "Cursor.exe", "minutes": 42.5, "frame_count": 100, "first_seen": "2023-01-01T00:00:00Z", "last_seen": "2023-01-01T01:00:00Z"}],
  "windows": [{"app_name": "Cursor.exe", "window_name": "Main", "minutes": 42.5, "browser_url": "https://example.com"}],
  "key_texts": [{"text": "hello", "app_name": "Cursor.exe", "window_name": "Main", "timestamp": "2023-01-01T00:00:00Z"}],
  "edited_files": [{"path": "c:/proj/file.md", "frame_count": 10}],
  "audio_summary": {"segment_count": 5, "speakers": ["Speaker 1"], "top_transcriptions": ["Hello world"]},
  "snippets": [{"source": "screen", "text": "snippet", "timestamp": "2023-01-01T00:00:00Z"}],
  "data_status": "ok",
  "guidance": {"next_best_query": "What did I do?"}
}
```
Tokens:
Raw