Windrecorder

https://github.com/yuka-friends/windrecorder
Admin
Windrecorder is an open-source Windows app that records your screen to help you retrieve memory...
Tokens:18,972
Snippets:32
Trust Score:5.4
License:GPL-2.0
Update:1 year ago
Show doc for...
Tokens:
Raw
### Project Setup and Cloning

Source: https://github.com/yuka-friends/windrecorder/blob/main/__assets__/README-sc.md

Instructions on how to download the Windrecorder project using Git and prepare the installation.

```APIDOC
Project Setup:

1.  Navigate to your desired installation directory using your file manager.
2.  Open a command prompt in that directory by typing `cmd` in the file manager's path bar and pressing Enter.
3.  Clone the Windrecorder repository using the command:
    `git clone https://github.com/yuka-friends/Windrecorder`
4.  After cloning, navigate into the cloned directory (`cd Windrecorder`).
5.  Run the `install_update.bat` script to install the tool and configure its environment.
```

--------------------------------

### Installation Prerequisites

Source: https://github.com/yuka-friends/windrecorder/blob/main/__assets__/README-sc.md

Lists the essential software and tools required to install and run Windrecorder, including FFmpeg, Git, and Python.

```APIDOC
Installation Prerequisites:

1.  **FFmpeg**: Download from [FFmpeg-Builds](https://github.com/BtbN/FFmpeg-Builds/releases). Copy all files from the `bin` directory to `C:\\Windows\\System32` or another directory in your system's PATH.
2.  **Git**: Install from [git-scm.com](https://git-scm.com/download/win). Follow the default installation steps.
3.  **Python**: Install Python 3.11.7 (64-bit) from [python.org](https://www.python.org/ftp/python/3.11.7/python-3.11.7-amd64.exe). Ensure 'Add python.exe to PATH' is checked during installation. Python 3.12 is not currently supported.

**Note**: Ensure you have sufficient disk space for data storage.
```

--------------------------------

### Web Service Startup

Source: https://github.com/yuka-friends/windrecorder/blob/main/ocr_lib/chineseocr_lite_onnx/README.md

Command to navigate to the chineseocr_lite directory and start the web service using Python.

```Bash
cd chineseocr_lite## 进入chineseocr目录
python backend/main.py
```

--------------------------------

### Install and Update Windrecorder

Source: https://github.com/yuka-friends/windrecorder/blob/main/README.md

This batch script handles the installation of dependencies and initial configuration for the Windrecorder application. It should be run after cloning the repository.

```batch
@echo off

REM This script installs dependencies and configures Windrecorder.
REM Ensure ffmpeg, Git, and Python are installed and in PATH.

echo Installing dependencies...
REM Add commands here to install Python packages via pip, e.g., pip install -r requirements.txt

echo Configuration complete.

REM Add commands here to start the application or provide next steps.
```

--------------------------------

### Running Windrecorder

Source: https://github.com/yuka-friends/windrecorder/blob/main/__assets__/README-sc.md

Details on how to start the Windrecorder application and access its features via the system tray.

```APIDOC
Running Windrecorder:

1.  Execute the `start_app.bat` script located in the Windrecorder directory.
2.  The application will run in the system tray.
3.  Access its features and settings through the right-click context menu on the system tray icon.

**Troubleshooting**: If the command window closes immediately or the app doesn't appear, create an empty file named `hide_CLI_by_python.txt` in the Windrecorder root directory and try running `start_app.bat` again.
```

--------------------------------

### Setup Project Paths and Directories

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/i18n_create_synonyms_embedding_asset/create_synonyms_embedding_asset.ipynb

Configures the Python environment by setting the current working directory and adding a parent directory to the system path. This is crucial for importing modules from different project levels.

```python
# add vocabulary txt, one line for each word
VOCABULARY_TXT_FILEPATH = "synonyms_en.txt"
OUTPUT_VDB_FILENAME = "synonyms_en.index"

# Set workspace to Windrecorder dir
import os
import sys
current_dir = os.getcwd()
parent_parent_dir = os.path.dirname(os.path.dirname(current_dir))
sys.path.append(parent_parent_dir)
os.chdir("..")
os.chdir("..")
SUB_DIR = "extension\i18n_create_synonyms_embedding_asset"
```

--------------------------------

### Install or Update Windrecorder Environment

Source: https://github.com/yuka-friends/windrecorder/blob/main/README.md

Run this script to set up or update the necessary virtual environment for Windrecorder. This is required after moving the application directory or when setting up for the first time.

```batch
install_update.bat
```

--------------------------------

### OpenAI Compatible LLM Service Configuration

Source: https://github.com/yuka-friends/windrecorder/blob/main/__assets__/how_to_set_LLM_api_endpoint_en.md

Configuration details for LLM services that expose an OpenAI-compatible API endpoint. This includes specifying the base URL, API key, and model name required to interact with the service.

```APIDOC
Provider: OpenAI Platform
base url: https://api.openai.com/v1
api key: sk-AbCdEfGxXXXXXXXXXXXXXXXXX
modelname: gpt-4o
Description: Official OpenAI API. Requires an API key obtained from the OpenAI platform. Model names like 'gpt-4o' are common.
```

```APIDOC
Provider: DeepSeek
base url: https://api.deepseek.com/v1
api key: <your-deepseek-api-key>
modelname: deepseek-chat
Description: DeepSeek offers cost-effective, high-quality models. Note potential content restrictions. Requires an API key from DeepSeek.
```

```APIDOC
Provider: Groq
base url: https://api.groq.com/openai/v1
api key: <your-groq-api-key>
modelname: llama-3.1-70b-versatile
Description: Groq provides fast inference for open-source models. Requires an API key from Groq's console.
```

```APIDOC
Provider: LM Studio (Local Server)
base url: http://localhost:1234/v1
api key: lm-studio
modelname: cognitivecomputations/dolphin-2.9-llama3-8b-gguf
Description: For local LLM execution via LM Studio. The base URL is typically localhost. 'lm-studio' is often used as a placeholder API key. Local models may have lower intelligence or instruction compliance.
```

```APIDOC
Provider: Any OpenAI Compatible Service
base url: <service-provided-url>
api key: <service-provided-api-key>
modelname: <service-provided-model-name>
Description: Generic configuration for any LLM service that adheres to the OpenAI API specification. This is useful for custom or self-hosted solutions.
```

--------------------------------

### Start Windrecorder Application

Source: https://github.com/yuka-friends/windrecorder/blob/main/README.md

Execute this batch script to launch Windrecorder. The application will run in the system tray, accessible via its right-click menu. Ensure the virtual environment is set up correctly.

```batch
start_app.bat
```

--------------------------------

### Run OCR Benchmark

Source: https://github.com/yuka-friends/windrecorder/blob/main/__assets__/third_party_ocr_engine_benchmark_reference.md

Executes the OCR benchmark using the `windrecorder.ocr_manager` module and prints the formatted results. This script requires the `windrecorder` library and its dependencies to be installed.

```Python
from windrecorder.ocr_manager import format_print_benchmark, ocr_benchmark
format_print_benchmark(ocr_benchmark())
```

--------------------------------

### Version Update Warning

Source: https://github.com/yuka-friends/windrecorder/blob/main/__assets__/README-sc.md

Important information regarding potential issues with update detection and installation for versions prior to 0.0.5.

```APIDOC
Version Update Warning:

Versions of Windrecorder prior to `0.0.5` may have issues with automatic update detection or the `install_update.bat` script.

If you encounter problems updating, please perform a manual update via the command line:
1.  Open a command prompt in the Windrecorder root directory.
2.  Execute `git pull` to fetch the latest changes.
```

--------------------------------

### Setup Workspace and Import Libraries for Windrecorder

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/LLM_search_and_summary/_early_research_mvp.ipynb

Sets the current working directory to the Windrecorder project root and imports essential Python libraries including datetime, pandas, and custom modules from the windrecorder package for database management, daily data handling, and configuration.

```python
# Set workspace to Windrecorder dir
import os
import io
os.chdir(".."")
os.chdir(".."")
# ------------------------------------------------------------

import datetime
import pandas as pd

from windrecorder.db_manager import db_manager
from windrecorder.oneday import OneDay
from windrecorder.record_wintitle import get_wintitle_stat_in_day
from windrecorder.config import config
from windrecorder import llm
```

--------------------------------

### Hide Command Line Interface

Source: https://github.com/yuka-friends/windrecorder/blob/main/README.md

Create this text file in the application directory to prevent the command line window from appearing when starting Windrecorder. This is useful for a cleaner user experience if the CLI is not needed.

```text
hide_CLI_by_python.txt
```

--------------------------------

### Clone Windrecorder Repository

Source: https://github.com/yuka-friends/windrecorder/blob/main/README.md

This command downloads the Windrecorder project files from its GitHub repository to your local machine. It's the first step in setting up the application.

```bash
git clone https://github.com/yuka-friends/Windrecorder
```

--------------------------------

### Data Management and Portability

Source: https://github.com/yuka-friends/windrecorder/blob/main/__assets__/README-sc.md

Explains how Windrecorder stores data and how to move or copy the application to a new location or computer.

```APIDOC
Data Management:

-   All data, including videos, databases, and statistics, are stored within the Windrecorder directory.
-   To move or copy the application:
    1.  Delete the `.venv` directory.
    2.  Move or copy the entire Windrecorder folder.
    3.  Re-run `install_update.bat` in the new location to set up the virtual environment.
```

--------------------------------

### Load Image Embedding Model (CPU)

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/i18n_create_synonyms_embedding_asset/create_synonyms_embedding_asset.ipynb

Initializes an image embedding model, specifically configured to run on the CPU. This model is used for generating vector representations of text.

```python
from windrecorder import img_embed_manager

model_cpu = img_embed_manager.get_model(mode="cpu")
```

--------------------------------

### LLM Request and Output in Python

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/LLM_search_and_summary/_early_research_mvp.ipynb

This snippet demonstrates how to send a one-shot request to a Large Language Model (LLM) using the `llm` library. It includes specifying user content, a system prompt, and temperature settings. The response, containing success status and plain text, is then printed to the console.

```Python
success, plain_text = llm.request_llm_one_shot(
    user_content=prompt_mount + prompt_end,
    system_prompt=system_prompt,
    temperature=.3,
)

print(plain_text)
```

--------------------------------

### Read Vocabulary from Text File

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/i18n_create_synonyms_embedding_asset/create_synonyms_embedding_asset.ipynb

Reads a list of words from a specified text file using `file_utils`. It constructs the full file path using previously defined directory variables.

```python
from windrecorder import file_utils

words_list = file_utils.read_txt_as_list(os.path.join(parent_parent_dir, SUB_DIR, VOCABULARY_TXT_FILEPATH))
```

--------------------------------

### Embed Words and Add to Vector Database

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/i18n_create_synonyms_embedding_asset/create_synonyms_embedding_asset.ipynb

Iterates through a list of words, embeds each word into a vector using the loaded model, and adds the vector to a `VectorDatabase`. The database is initialized with a specified filename and directory.

```python
# embedding all words into vector
vdb = img_embed_manager.VectorDatabase(vdb_filename=os.path.join(parent_parent_dir, SUB_DIR, OUTPUT_VDB_FILENAME), db_dir="")

i = 0
all = len(words_list)
for word in words_list:
    print(f"{i}/{all}, {word=}")
    vector = img_embed_manager.embed_text(model=model_cpu, text_query=word, detach_numpy=False)
    vdb.add_vector(vector=vector, rowid=i)
    i += 1
```

--------------------------------

### Retrieve and Display Optimized Daily Activity Statistics

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/LLM_search_and_summary/_early_research_mvp.ipynb

Fetches daily window title statistics using `get_wintitle_stat_in_day`. It retrieves both raw data and data optimized for display, then shows the optimized version.

```python
df_day_activity_raw, day_time_sum = get_wintitle_stat_in_day(datetime_select, optimize_for_display=False)
df_day_activity_optimize_display, _ = get_wintitle_stat_in_day(datetime_select, optimize_for_display=True)

df_day_activity_optimize_display
```

--------------------------------

### Process and Clean Vocabulary File

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/i18n_create_synonyms_embedding_asset/create_synonyms_embedding_asset.ipynb

A utility function to preprocess a text file. It extracts the first word from each line (if a space exists), strips whitespace, and filters out words shorter than two characters before writing to an output file.

```python
# [optional] clean txt
# Dictionaries can be preprocessed as needed. Here we only take the words before the first space of each line, and then remove all words with less than 2 words.

def process_file(input_file, output_file):
    with open(input_file, 'r', encoding='utf8') as f:
        lines = f.readlines()

    processed_lines = []

    for line in lines:
        if ' ' in line:
            line = line.split(' ')[0]

        line = line.strip()

        if len(line) >= 2:
            processed_lines.append(line + '\n')

    with open(output_file, 'w', encoding='utf8') as f:
        f.writelines(processed_lines)


process_file(os.path.join(parent_parent_dir, SUB_DIR, "TOEFL.txt"), os.path.join(parent_parent_dir, SUB_DIR, "synonyms_en.txt"))
```

--------------------------------

### Recording Modes

Source: https://github.com/yuka-friends/windrecorder/blob/main/__assets__/README-sc.md

Describes the two primary recording modes available in Windrecorder: flexible screenshotting and direct video recording via FFmpeg.

```APIDOC
Recording Modes:

1.  **Automatic Flexible Screenshotting**:
    -   Captures screenshots every 3 seconds (default).
    -   Indexes changes in content or text, allowing real-time playback.
    -   Converts past screenshots into video segments every 15 minutes.
    -   Low system resource usage, suitable for most users.

2.  **Direct Video Recording via FFmpeg**:
    -   Records video in 15-minute segments.
    -   Indexes video clips after recording, introducing a potential 15-minute delay in data query.
    -   Medium system resource usage, provides smoother, complete activity recording.
```

--------------------------------

### Test Vector Search with Embedding Model

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/i18n_create_synonyms_embedding_asset/create_synonyms_embedding_asset.ipynb

Embeds a query word ("love") into a vector and then searches the vector database for the top `k` most similar vectors. It retrieves the corresponding words from the original list.

```python
# test querying result
vector_search = img_embed_manager.embed_text(model=model_cpu, text_query="love", detach_numpy=True)
prob_res = vdb.search_vector(vector=vector_search, k=5)
word_res = [words_list[i[0]] for i in prob_res]
print(word_res)
```

--------------------------------

### Generate LLM Prompt Context from Processed Data

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/LLM_search_and_summary/_early_research_mvp.ipynb

Converts processed pandas DataFrames into CSV formatted strings. These strings are then combined with system prompts and user instructions to create a comprehensive context for a large language model to summarize daily user activities.

```python
def convert_df_to_csv_str(df:pd.DataFrame):
    csv_buffer = io.StringIO()
    df.to_csv(csv_buffer, index=False)
    csv_string = csv_buffer.getvalue()
    return csv_string

prompt_day_wintitle_activity = convert_df_to_csv_str(df_day_activity_optimize_display)
prompt_day_records = convert_df_to_csv_str(day_records_clean_df)

prompt_mount = f"""
### Top {MAX_DAY_ACTIVITY_HEAD_TRUNCATION_FOR_LLM} activities by user screen time for the day, sorted by time, highest to lowest (CSV format):
---Start of screen time ranking csv---
{prompt_day_wintitle_activity}
---END of screen time ranking csv---

### Part of the user's specific activities on that day, sorted by time (CSV format) (ocr_content contains a lot of noise, please refer to it with very low weight and only focus on the key content information related to the activity title. Pay more attention to activity and only use ocr_content as an optional supplement.):
---Start of activities csv---
{prompt_day_records}
---END of activities csv---
"""

# prompt_mount = f"""
# ### Top {MAX_DAY_ACTIVITY_HEAD_TRUNCATION_FOR_LLM} activities by user screen time for the day, sorted by time, highest to lowest (CSV format):
# ---Start of screen time ranking csv---
# {prompt_day_wintitle_activity}
# ---END of screen time ranking csv---
# """

system_prompt = f"""
You are a user behavior analysis expert. Please generate a simple summary of the user's daily activities on that day based on the following computer activity records (screen time ranking, specific activity tracks) of the user ({config.user_name}) received on that day, describing what the user did and what content browsed on that day. If necessary, please retain the proper nouns or origin text of the activity content.
"""
prompt_end = """
Now, please summarize the user's daily activities in chronological order based on the above. Keep it concise, clear, simple, and insightful. Please output the results directly without adding additional instructions.
"""

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": prompt_mount + prompt_end}
]

```

--------------------------------

### Third-Party OCR Engine Integration

Source: https://github.com/yuka-friends/windrecorder/blob/main/__assets__/README-sc.md

Lists supported third-party OCR engines that can be integrated with Windrecorder for enhanced text recognition capabilities.

```APIDOC
Third-Party OCR Engines:

Windrecorder supports integration with various OCR engines to improve text recognition accuracy and performance. Supported engines include:

-   **Rapid OCR**: Based on Paddle OCR's onnxruntime version.
-   **WeChat OCR**: Known for high accuracy in Chinese and English recognition.
-   **Tesseract OCR**: Supports over 100 languages and multi-language recognition simultaneously.

Users can also contribute custom OCR support by following the guidelines provided in the project's extension documentation.
```

--------------------------------

### Save Vector Database to File

Source: https://github.com/yuka-friends/windrecorder/blob/main/extension/i18n_create_synonyms_embedding_asset/create_synonyms_embedding_asset.ipynb

Persists the current state of the vector database to its designated file. This ensures that the embedded data is stored for later use without re-computation.

```python
# save vector database to file
vdb.save_to_file()
```
Tokens:
Raw