### Quick Start: Clone, Setup, and Run Tests Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md A streamlined guide to get started quickly. This involves cloning the repository, setting up a virtual environment, installing test dependencies, and running tests using either the convenience script or pytest directly. ```bash # 1. Clone the repository (if not already done) git clone cd ha-llmvision ``` ```bash # 2. Create and activate virtual environment python -m venv .venv source .venv/bin/activate # On Windows: .venv\Scripts\activate ``` ```bash # 3. Install all test dependencies pip install -r requirements-test.txt ``` ```bash # 4. Run tests using the convenience script ./run_tests.sh # Run all unit tests ./run_tests.sh -c # Run with coverage report ./run_tests.sh -q # Quick run (quiet mode) ``` ```bash # Or use pytest directly pytest tests/ -v # Run all tests (integration tests auto-skipped) pytest tests/ -m "not integration" --cov=custom_components/llmvision --cov-report=html ``` -------------------------------- ### Setup Virtual Environment and Install Dependencies Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Commands to set up a Python virtual environment and install all necessary testing dependencies from the `requirements-test.txt` file. ```bash python -m venv .venv # Create virtual environment ``` ```bash source .venv/bin/activate # Activate (macOS/Linux) ``` ```bash .venv\Scripts\activate # Activate (Windows) ``` ```bash pip install -r requirements-test.txt # Install dependencies ``` -------------------------------- ### Verify Installation and Configuration Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Check pytest version, verify Python path includes 'custom_components', and run a basic test to ensure the setup is correct. ```bash pytest --version ``` ```python import sys; print('custom_components' in str(sys.path)) ``` ```bash pytest tests/test_const.py::TestConstants::test_domain -v ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Upgrade pip and install all test dependencies from the requirements-test.txt file. This ensures all necessary libraries for testing are available. ```bash pip install --upgrade pip pip install -r requirements-test.txt ``` -------------------------------- ### Azure OpenAI Provider Configuration Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Configuration example for Azure OpenAI. Requires API key, base URL, deployment name, and version. ```yaml # Azure OpenAI Configuration # - api_key: Your Azure OpenAI key # - azure_base_url: https://your-resource.openai.azure.com/ # - azure_deployment: your-deployment-name # - azure_version: 2025-04-01-preview # - default_model: gpt-4o-mini ``` -------------------------------- ### Setup and Run Integration Tests Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md To run integration tests, first set up a Home Assistant instance. Then, create `.instance` and `.token` files in the `tests/` directory with your HA URL and a long-lived access token, respectively. Finally, run the integration tests. ```bash http://localhost:8123 ``` ```bash pytest tests/test_api.py -v ``` -------------------------------- ### Custom OpenAI-Compatible Endpoint Configuration Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Configuration example for connecting to a custom OpenAI-compatible API endpoint. Requires an API key and the endpoint URL. ```yaml # Custom OpenAI-compatible endpoint # - api_key: Your API key # - custom_openai_endpoint: https://your-server.com/v1/chat/completions # - default_model: your-model-name ``` -------------------------------- ### OpenAI Provider Configuration Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Configuration example for the OpenAI provider. Requires an API key and allows setting default model and generation parameters. ```yaml # Provider configuration examples (via Home Assistant UI config flow) # OpenAI Configuration # - api_key: Your OpenAI API key # - default_model: gpt-4o-mini (default), gpt-4o, gpt-4-turbo # - temperature: 0.5 (default) # - top_p: 0.9 (default) ``` -------------------------------- ### Basic Pytest Unit Test Structure Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md A template for writing unit tests using pytest. Includes setup for mocks, testing a method, and testing an asynchronous function. Ensure pytest and unittest.mock are installed. ```python """Unit tests for module_name.py module.""" import pytest from unittest.mock import Mock, patch, AsyncMock class TestClassName: """Test ClassName class.""" @pytest.fixture def mock_dependency(self): """Create a mock dependency.""" return Mock() def test_method_name(self, mock_dependency): """Test method_name does something.""" # Arrange instance = ClassName(mock_dependency) # Act result = instance.method_name() # Assert assert result == expected_value @pytest.mark.asyncio async def test_async_method(self): """Test async method.""" result = await async_function() assert result is not None ``` -------------------------------- ### Anthropic Provider Configuration Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Configuration example for the Anthropic provider. Requires an API key and allows setting the default model. ```yaml # Anthropic Configuration # - api_key: Your Anthropic API key # - default_model: claude-3-7-sonnet-latest (default), claude-3-opus-latest ``` -------------------------------- ### AWS Bedrock Provider Configuration Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Configuration example for AWS Bedrock. Requires AWS access keys, region, and allows setting the default model. ```yaml # AWS Bedrock Configuration # - aws_access_key_id: Your AWS access key # - aws_secret_access_key: Your AWS secret key # - aws_region_name: us-east-1 # - default_model: us.amazon.nova-pro-v1:0 ``` -------------------------------- ### Google Gemini Provider Configuration Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Configuration example for the Google Gemini provider. Requires an API key and allows setting the default model. ```yaml # Google Gemini Configuration # - api_key: Your Google API key # - default_model: gemini-2.0-flash (default), gemini-1.5-pro ``` -------------------------------- ### Ollama Self-Hosted Provider Configuration Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Configuration example for a self-hosted Ollama instance. Requires IP address, port, and allows setting model, context window, and keep-alive settings. ```yaml # Ollama Configuration (self-hosted) # - ip_address: 192.168.1.100 # - port: 11434 # - https: false # - default_model: gemma3:4b (default), llava, llama3.2-vision # - context_window: 2048 # - keep_alive: 5 ``` -------------------------------- ### Common Pytest Fixtures Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Example fixtures available in conftest.py for mocking Home Assistant instances and config entries. These are useful for setting up test environments. ```python @pytest.fixture def mock_hass(): """Mock Home Assistant instance.""" ``` ```python @pytest.fixture def mock_config_entry(): """Mock config entry.""" ``` -------------------------------- ### Run Tests with Verbose Output Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Execute tests with verbose output to get more detailed information about the test execution process. This is useful for debugging. ```bash pytest tests/ -vv ``` -------------------------------- ### Get Recent Timeline Events via REST API Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Retrieves a list of recent timeline events from the LLM Vision integration using a GET request to the /api/llmvision/timeline/events endpoint. Requires authentication with a long-lived access token. ```bash curl -X GET "http://homeassistant.local:8123/api/llmvision/timeline/events?limit=10" \ -H "Authorization: Bearer YOUR_LONG_LIVED_ACCESS_TOKEN" \ -H "Content-Type: application/json" ``` -------------------------------- ### Get Specific Event by ID Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Retrieve details for a single event using its unique ID. This is useful for inspecting specific detections. ```bash curl -X GET "http://homeassistant.local:8123/api/llmvision/timeline/event/abc123-def456" \ -H "Authorization: Bearer YOUR_LONG_LIVED_ACCESS_TOKEN" ``` -------------------------------- ### Useful pytest Combinations Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Combine multiple pytest options for specific testing scenarios. Examples include running a specific test with coverage and verbose output, or running all tests except integration tests with coverage and HTML reporting. ```bash # Run specific test with coverage and verbose output pytest tests/test_memory.py::TestMemory::test_init_without_entry -v --cov=custom_components/llmvision ``` ```bash # Run all tests except integration tests with coverage pytest tests/ --ignore=tests/test_api.py --cov=custom_components/llmvision --cov-report=html ``` ```bash # Run failed tests first, stop on first failure, show output pytest tests/ --ff -x -s ``` -------------------------------- ### VS Code Debugger Configuration for Pytest Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Configure VS Code to launch and debug pytest tests for the current file. Ensure the Python extension is installed. ```json { "version": "0.2.0", "configurations": [ { "name": "Python: Pytest Current File", "type": "python", "request": "launch", "module": "pytest", "args": [ "${file}", "-v", "-s" ], "console": "integratedTerminal", "justMyCode": false } ] } ``` -------------------------------- ### Use pytest Debugging Features Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Leverage pytest's built-in debugging options to drop into a debugger on test failure or at the start of each test. ```bash pytest tests/ --pdb ``` ```bash pytest tests/ --trace ``` ```bash pytest tests/ -v -s ``` -------------------------------- ### Create Timeline Event with Current Timestamp Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Creates a timeline event using the current timestamp for start and end times. Useful for logging events that occur without a predefined duration. ```yaml service: llmvision.create_event data: title: "Gate opened" description: "Main gate was opened via automation" label: "Door" ``` -------------------------------- ### Create and Activate Virtual Environment Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Use these commands to create and activate a Python virtual environment for the project. Ensure you are in the project root directory. ```bash cd ha-llmvision python -m venv .venv ``` ```bash source .venv/bin/activate ``` ```cmd .venv\Scripts\activate ``` -------------------------------- ### Get Specific Event Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Retrieves details for a specific event using its unique ID. ```APIDOC ## GET /api/llmvision/timeline/event/{event_id} ### Description Retrieves the details of a specific event identified by its unique ID. ### Method GET ### Endpoint /api/llmvision/timeline/event/{event_id} ### Path Parameters - **event_id** (string) - Required - The unique identifier of the event. ### Request Example ```bash curl -X GET "http://homeassistant.local:8123/api/llmvision/timeline/event/abc123-def456" \ -H "Authorization: Bearer YOUR_LONG_LIVED_ACCESS_TOKEN" ``` ### Response #### Success Response (200) - **event** (object) - The event object containing details. - **event_id** (string) - Unique identifier for the event. - **timestamp** (string) - Timestamp of the event. - **camera_name** (string) - Name of the camera where the event occurred. - **label** (string) - Detected category label. - **title** (string) - Title of the event. - **description** (string) - Description of the event. - **key_frame** (string) - Path to the key frame image. #### Response Example ```json { "event": { "event_id": "abc123-def456", "timestamp": "2024-01-15T14:30:00Z", "camera_name": "camera.front_door", "label": "person", "title": "Person detected", "description": "A person was detected at the front door.", "key_frame": "/media/llmvision/snapshots/event_123.jpg" } } ``` ``` -------------------------------- ### Convenience Script Test Options Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md The `run_tests.sh` script offers several options for running tests. Use these shortcuts for common testing scenarios, including coverage, verbose output, quiet mode, and including integration tests. ```bash ./run_tests.sh # Run all unit tests ``` ```bash ./run_tests.sh -c # Run with coverage report ``` ```bash ./run_tests.sh -v # Verbose output ``` ```bash ./run_tests.sh -q # Quick/quiet mode ``` ```bash ./run_tests.sh -i # Include integration tests ``` ```bash ./run_tests.sh -c -v # Coverage with verbose output ``` ```bash ./run_tests.sh --help # Show all options ``` -------------------------------- ### Run All Tests Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Execute all unit tests by default. Use flags to explicitly include or exclude integration tests. ```bash pytest tests/ -v ``` ```bash pytest tests/ -m "not integration" -v ``` ```bash pytest tests/ -v --run-integration ``` -------------------------------- ### Run Tests with Script Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Execute all unit tests using the provided script. Options include running with coverage, verbose output, quiet mode, including integration tests, or displaying help. ```bash ./run_tests.sh ``` ```bash ./run_tests.sh -c ``` ```bash ./run_tests.sh -v ``` ```bash ./run_tests.sh -q ``` ```bash ./run_tests.sh -i ``` ```bash ./run_tests.sh --help ``` -------------------------------- ### Home Assistant Script for Image Analysis Source: https://github.com/valentinfrlch/ha-llmvision/wiki/Frontend This script analyzes the front door camera using LLM Vision and saves the response to an input text helper. Ensure the 'llmvision.image_analyzer' service and 'input_text.llmvision_response' helper are configured. ```yaml alias: Analyse Front Door sequence: - service: llmvision.image_analyzer data: provider: OpenAI detail: low temperature: 0.5 message: Is a person or a package at the front door? image_entity: - camera.front_door max_tokens: 60 response_variable: response - service: input_text.set_value metadata: {} data: value: "{{response.response_text}}" target: entity_id: input_text.llmvision_response description: "" icon: mdi:cube-scan ``` -------------------------------- ### Run Unit Tests Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Execute all unit tests in the project. These tests are designed to run in isolation and do not require external dependencies. ```bash pytest tests/ -m "not integration" -v ``` -------------------------------- ### Run Specific Test Files Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Target specific test files or multiple files for execution. ```bash pytest tests/test_memory.py -v ``` ```bash pytest tests/test_memory.py tests/test_calendar.py -v ``` -------------------------------- ### Run Local Tests with Pytest Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Execute tests locally before pushing changes. Use the quick test command to skip integration tests, or the full test command to include coverage reporting similar to CI. ```bash # Quick test pytest tests/ -m "not integration" -q ``` ```bash # Full test with coverage (like CI) pytest tests/ -m "not integration" --cov=custom_components/llmvision --cov-report=term ``` -------------------------------- ### Coverage Report Options Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Customize coverage reports to show missing lines, generate multiple formats, or set a minimum coverage threshold. ```bash pytest tests/ --cov=custom_components/llmvision --cov-report=term-missing ``` ```bash pytest tests/ --cov=custom_components/llmvision --cov-report=term --cov-report=html --cov-report=xml ``` ```bash pytest tests/ --cov=custom_components/llmvision --cov-fail-under=50 ``` -------------------------------- ### Clean Up Project Files Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Commands for cleaning up project artifacts. This includes deactivating and removing the virtual environment, deleting coverage files and HTML reports, and removing Python cache directories and compiled files. ```bash # Remove virtual environment deactivate rm -rf .venv ``` ```bash # Remove coverage files (if generated with --cov-report=html) rm -rf .coverage htmlcov/ ``` ```bash # Remove Python cache files find . -type d -name "__pycache__" -exec rm -rf {} + find . -type f -name "*.pyc" -delete ``` -------------------------------- ### Create a new feature branch Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/CONTRIBUTING.md Use this command to create a new branch for your feature development. Replace 'your-feature-name' with a descriptive name for your changes. ```bash git checkout -b feature/your-feature-name ``` -------------------------------- ### Analyze Video with LLM Vision Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Use the `llmvision.video_analyzer` service to analyze video files or Frigate events. The service extracts key frames for analysis. Specify the video file path or Frigate event ID, along with the AI provider and message. Options include limiting the number of frames analyzed and storing results in the timeline. ```yaml service: llmvision.video_analyzer data: provider: "01234567-89ab-cdef-0123-456789abcdef" model: "claude-3-7-sonnet-latest" message: "Describe what happens in this video. Focus on any suspicious activity." video_file: "/media/recordings/front_door_2024-01-15_14-30-00.mp4" max_frames: 5 target_width: 1280 max_tokens: 4000 store_in_timeline: true generate_title: true expose_images: true response_variable: video_analysis ``` ```yaml service: llmvision.video_analyzer data: provider: "01234567-89ab-cdef-0123-456789abcdef" model: "gpt-4o-mini" message: "What triggered this motion event? Describe any people, vehicles, or animals." event_id: "1712108310.968815-r28cdt" max_frames: 3 include_filename: true store_in_timeline: true response_variable: frigate_analysis ``` -------------------------------- ### Capture Image and Ask Question with LLM Vision Source: https://github.com/valentinfrlch/ha-llmvision/wiki/Scripts-using-LLM-Vision Use this script to analyze an image from a camera entity based on a prompt and send the response to a TTS entity. Requires camera and TTS entities to be configured. ```yaml alias: Example fields: prompt: selector: text: null name: prompt required: true sequence: - service: llmvision.image_analyzer metadata: {} data: max_tokens: 100 model: gpt-4o target_width: 1280 image_entity: - camera.front_door - camera.garage message: "{{ prompt }}" response_variable: response - service: tts.speak metadata: {} data: cache: true media_player_entity_id: media_player.entity_id message: "{{response.response_text}}" target: entity_id: tts.piper ``` -------------------------------- ### Run Tests with Different Output Formats Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Control the verbosity and output of test runs. Options include quiet mode, showing print statements, stopping on failure, and showing local variables. ```bash pytest tests/ -q ``` ```bash pytest tests/ -v -s ``` ```bash pytest tests/ -x ``` ```bash pytest tests/ -l ``` -------------------------------- ### Analyze Image with LLM Vision Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Use the `llmvision.image_analyzer` service to analyze static images. You can provide an image entity or a local file path. Configure the AI provider, model, and message for the analysis. Options include storing results in the timeline, generating titles, and using memory for context. ```yaml service: llmvision.image_analyzer data: provider: "01234567-89ab-cdef-0123-456789abcdef" # Config entry ID from integration model: "gpt-4o-mini" message: "Describe what you see in this image. Focus on people and vehicles." image_entity: - camera.front_door target_width: 1280 max_tokens: 3000 include_filename: false store_in_timeline: true generate_title: true use_memory: true expose_images: true response_variable: analysis_result ``` ```yaml service: llmvision.image_analyzer data: provider: "01234567-89ab-cdef-0123-456789abcdef" model: "gpt-4o" message: "Analyze this security camera image" image_file: "/config/www/snapshots/front_door.jpg" response_format: "json" structure: | { "type": "object", "properties": { "title": {"type": "string", "description": "Event title"}, "description": {"type": "string", "description": "Event description"}, "person_count": {"type": "integer"}, "vehicle_detected": {"type": "boolean"}, "confidence": {"type": "number", "minimum": 0, "maximum": 100} }, "required": ["title", "description", "person_count", "vehicle_detected", "confidence"], "additionalProperties": false } title_field: "title" description_field: "description" response_variable: structured_result ``` -------------------------------- ### Describe Camera Feed Script for LLM Vision Source: https://github.com/valentinfrlch/ha-llmvision/wiki/Specs-for-Extended-OpenAI-Conversation This script analyzes camera feeds using the llmvision.image_analyzer service. Configure provider, max_tokens, and temperature as needed. Ensure relevant camera entities are exposed in Home Assistant settings. ```yaml - spec: name: describe_camera_feed description: Get a description whats happening on security cameras around the house parameters: type: object properties: message: type: string description: The prompt for the image analyzer entity_ids: type: array description: List of camera entities items: type: string description: Entity id of the camera required: - message - entity_ids function: type: script sequence: - service: llmvision.image_analyzer data: provider: Ollama max_tokens: 100 temperature: 0.3 image_entity: "{{entity_ids}}" message: "{{message}}" response_variable: _function_result ``` -------------------------------- ### Generate Coverage Reports Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Run tests with coverage enabled and generate reports in different formats like terminal output or HTML. ```bash pytest tests/ --ignore=tests/test_api.py --cov=custom_components/llmvision --cov-report=term ``` ```bash pytest tests/ --ignore=tests/test_api.py --cov=custom_components/llmvision --cov-report=html ``` ```bash open htmlcov/index.html ``` ```bash xdg-open htmlcov/index.html ``` ```bash start htmlcov/index.html ``` -------------------------------- ### Detect Package Presence Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Detects the presence of a package at the front door and updates an input_boolean sensor. The AI response is parsed to determine a true or false value for the sensor. ```yaml service: llmvision.data_analyzer data: provider: "01234567-89ab-cdef-0123-456789abcdef" message: "Is there a package at the front door?" image_entity: - camera.front_door sensor_entity: input_boolean.package_detected ``` -------------------------------- ### Generate Code Coverage Reports Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Generate code coverage reports for specified components. Options include basic coverage, HTML reports, and terminal output for missing lines. The HTML report can be viewed in a browser. ```bash pytest tests/ --cov=custom_components/llmvision ``` ```bash pytest tests/ --cov=custom_components/llmvision --cov-report=html ``` ```bash pytest tests/ --cov=custom_components/llmvision --cov-report=term-missing ``` ```bash open htmlcov/index.html ``` -------------------------------- ### Run Tests with Debug Logging Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Check test logs by running tests with the DEBUG log level enabled. This provides detailed output that can help in diagnosing issues. ```bash pytest tests/ --log-cli-level=DEBUG ``` -------------------------------- ### Filter Events by Camera and Category Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Use this endpoint to retrieve events filtered by specific cameras and object categories. Ensure your access token is valid. ```bash curl -X GET "http://homeassistant.local:8123/api/llmvision/timeline/events?limit=20&cameras=camera.front_door,camera.driveway&categories=person,car&days=7" \ -H "Authorization: Bearer YOUR_LONG_LIVED_ACCESS_TOKEN" ``` -------------------------------- ### Create New Event via API Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Manually create a new event in the timeline using the API. This requires a valid JSON payload with event details. ```bash curl -X POST "http://homeassistant.local:8123/api/llmvision/timeline/events/new" \ -H "Authorization: Bearer YOUR_LONG_LIVED_ACCESS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "title": "Custom event", "description": "Event created via API", "camera_name": "camera.front_door", "label": "person", "key_frame": "/media/llmvision/snapshots/custom.jpg", "start": "2024-01-15T14:30:00", "end": "2024-01-15T14:31:00" }' ``` -------------------------------- ### Debug Tests Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Commands for debugging tests. This includes dropping into a debugger on failure, enabling verbose output with print statements, setting log levels, and displaying local variables on failure. ```bash pytest tests/ --pdb ``` ```bash pytest tests/ -vv -s ``` ```bash pytest tests/ --log-cli-level=DEBUG ``` ```bash pytest tests/ -l ``` -------------------------------- ### Update Test Dependencies Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Manage test dependencies using pip. Commands include upgrading all dependencies from a requirements file, upgrading a specific package, checking for outdated packages, and freezing current versions for reproducibility. ```bash # Update all dependencies to latest versions pip install --upgrade -r requirements-test.txt ``` ```bash # Update specific package pip install --upgrade pytest ``` ```bash # Check for outdated packages pip list --outdated ``` ```bash # Freeze current versions (for reproducibility) pip freeze > requirements-test-frozen.txt ``` -------------------------------- ### Run Specific Test Classes or Functions Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Isolate and run a particular test class or a single test function for focused testing. ```bash pytest tests/test_memory.py::TestMemory -v ``` ```bash pytest tests/test_memory.py::TestMemory::test_init_without_entry -v ``` -------------------------------- ### Optimizing Test Execution Speed with Pytest Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Commands to speed up test execution. Includes running tests in parallel using `pytest-xdist`, re-running only failed tests (`--lf`), running failed tests first (`--ff`), and skipping slow tests marked with `@pytest.mark.slow`. ```bash # Run tests in parallel (pytest-xdist is included in requirements-test.txt) pytest tests/ -n auto # Uses all CPU cores # Run only failed tests from last run pytest tests/ --lf # Run tests that failed first, then others pytest tests/ --ff # Skip slow tests (if marked with @pytest.mark.slow) pytest tests/ -m "not slow" ``` -------------------------------- ### Configure Python Path Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md The project's pytest.ini usually handles Python path configuration. If import errors occur, manually add the project root to PYTHONPATH. ```bash export PYTHONPATH="${PYTHONPATH}:$(pwd)" ``` -------------------------------- ### Analyze Live Camera Stream Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Analyzes a live camera stream for a specified duration, summarizing activity and identifying people, vehicles, or unusual events. Configure provider, model, message, image entities, duration, frame limits, and output options. ```yaml service: llmvision.stream_analyzer data: provider: "01234567-89ab-cdef-0123-456789abcdef" model: "gemini-2.0-flash" message: "Summarize the activity in this camera feed. Report any people, vehicles, or unusual events." image_entity: - camera.front_door - camera.driveway duration: 10 max_frames: 5 target_width: 1280 max_tokens: 3000 store_in_timeline: true generate_title: true use_memory: true expose_images: true response_variable: stream_result ``` -------------------------------- ### Configuring Coverage Report with pytest-cov Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Specify the package to include in the coverage report using `--cov` and `--cov-report` flags. Alternatively, configure the `source` and `omit` options in a `.coveragerc` file for persistent settings. ```bash # Specify the package explicitly pytest tests/ --cov=custom_components/llmvision --cov-report=term-missing # Or create .coveragerc file [run] source = custom_components/llmvision omit = */tests/* */__pycache__/* ``` -------------------------------- ### Create New Event Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Creates a new event manually via the API. ```APIDOC ## POST /api/llmvision/timeline/events/new ### Description Creates a new event in the timeline, typically for manual logging or integration purposes. ### Method POST ### Endpoint /api/llmvision/timeline/events/new ### Request Body - **title** (string) - Required - The title of the event. - **description** (string) - Optional - A detailed description of the event. - **camera_name** (string) - Required - The name of the camera associated with the event. - **label** (string) - Required - The category label for the event (e.g., 'person', 'car'). - **key_frame** (string) - Optional - Path to the key frame image for the event. - **start** (string) - Required - ISO 8601 formatted start timestamp of the event. - **end** (string) - Optional - ISO 8601 formatted end timestamp of the event. ### Request Example ```json { "title": "Custom event", "description": "Event created via API", "camera_name": "camera.front_door", "label": "person", "key_frame": "/media/llmvision/snapshots/custom.jpg", "start": "2024-01-15T14:30:00", "end": "2024-01-15T14:31:00" } ``` ### Response #### Success Response (200) - **event_id** (string) - The unique identifier of the newly created event. - **status** (string) - Confirmation message, e.g., 'created'. #### Response Example ```json { "event_id": "newevent789", "status": "created" } ``` ``` -------------------------------- ### Resolving Import Errors with Pip Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Commands to fix ModuleNotFoundError by ensuring the project root is active, the virtual environment is sourced, and test dependencies are reinstalled. Includes upgrading pip as a troubleshooting step. ```bash # Ensure you're in the project root directory cd ha-llmvision # Ensure virtual environment is activated source .venv/bin/activate # Reinstall all test dependencies pip install -r requirements-test.txt # If still having issues, try upgrading pip first pip install --upgrade pip pip install -r requirements-test.txt ``` -------------------------------- ### llmvision.image_analyzer Service Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Analyzes static images using AI vision models. Accepts images from local files or camera/image entities, sends them to the configured AI provider, and returns a text description or structured JSON response. Supports memory for context about known people/objects and can store results in the timeline. ```APIDOC ## llmvision.image_analyzer ### Description Analyzes static images using AI vision models. Accepts images from local files or camera/image entities, sends them to the configured AI provider, and returns a text description or structured JSON response. Supports memory for context about known people/objects and can store results in the timeline. ### Method CALL SERVICE ### Endpoint N/A (Home Assistant Service Call) ### Parameters #### Service Data - **provider** (string) - Required - Config entry ID from integration. - **model** (string) - Optional - The AI model to use. - **message** (string) - Required - The prompt for the AI model. - **image_entity** (list of entity IDs) - Optional - Camera or image entities to analyze. - **image_file** (string) - Optional - Path to a local image file. - **target_width** (integer) - Optional - Resize image to this width before sending. - **max_tokens** (integer) - Optional - Maximum tokens for the AI response. - **include_filename** (boolean) - Optional - Whether to include the filename in the response. - **store_in_timeline** (boolean) - Optional - Whether to store the analysis result in the timeline. - **generate_title** (boolean) - Optional - Whether to generate a title for the analysis. - **use_memory** (boolean) - Optional - Whether to use memory for context about known people/objects. - **expose_images** (boolean) - Optional - Whether to expose analyzed images. - **response_format** (string) - Optional - Desired response format ('text' or 'json'). - **structure** (yaml string) - Optional - Defines the structure for JSON output. - **title_field** (string) - Optional - Specifies the field for the title in structured JSON output. - **description_field** (string) - Optional - Specifies the field for the description in structured JSON output. ### Request Example ```yaml service: llmvision.image_analyzer data: provider: "01234567-89ab-cdef-0123-456789abcdef" model: "gpt-4o-mini" message: "Describe what you see in this image. Focus on people and vehicles." image_entity: - camera.front_door target_width: 1280 max_tokens: 3000 include_filename: false store_in_timeline: true generate_title: true use_memory: true expose_images: true response_variable: analysis_result ``` ### Response #### Success Response (200) - **title** (string) - The generated title for the analysis. - **response_text** (string) - The text description of the image content. - **key_frame** (string) - Path to the analyzed image file. - **structured_response** (object) - The structured JSON response if `response_format` is 'json'. #### Response Example ```json { "title": "Person at front door", "response_text": "A person wearing a blue jacket is standing at the front door, appearing to ring the doorbell.", "key_frame": "/media/llmvision/snapshots/front_door_1234567890.jpg" } ``` #### Structured JSON Response Example ```json { "structured_response": { "title": "Delivery person at door", "description": "A delivery person in brown uniform holding a package at the front door", "person_count": 1, "vehicle_detected": true, "confidence": 95 } } ``` ``` -------------------------------- ### Vertical Stack Card for LLM Vision Response Source: https://github.com/valentinfrlch/ha-llmvision/wiki/Frontend Use this card configuration to display a tile for triggering analysis and a markdown area for the LLM's response. Requires a 'vertical-stack-in-card' custom card and an 'input_text' helper. ```yaml type: custom:vertical-stack-in-card cards: - type: tile entity: script.analyze_front_door icon_tap_action: action: toggle vertical: false - type: markdown content: '{{states("input_text.llmvision_response")}}' ``` -------------------------------- ### REST API: Timeline Events Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Accesses timeline data programmatically via REST API endpoints. Requires Home Assistant authentication. ```APIDOC ## GET /api/llmvision/timeline/events ### Description Retrieves recent events from the LLM Vision timeline. Requires Home Assistant authentication using a long-lived access token. ### Method GET ### Endpoint `/api/llmvision/timeline/events` ### Query Parameters - **limit** (integer) - Optional - The maximum number of events to retrieve. ### Request Example ```bash curl -X GET "http://homeassistant.local:8123/api/llmvision/timeline/events?limit=10" \ -H "Authorization: Bearer YOUR_LONG_LIVED_ACCESS_TOKEN" \ -H "Content-Type: application/json" ``` ### Response #### Success Response (200) - **events** (list of objects) - A list of timeline events. - **uid** (string) - Unique identifier for the event. - **title** (string) - The title of the event. - **description** (string) - A description of the event. - **start** (string) - The start time of the event (ISO 8601 format). - **end** (string) - The end time of the event (ISO 8601 format). - **key_frame** (string) - The path to a key frame image associated with the event. - **camera_name** (string) - The name of the camera associated with the event. - **label** (string) - The label categorizing the event. #### Response Example ```json { "events": [ { "uid": "abc123-def456", "title": "Person at door", "description": "A person approached the front door and rang the doorbell", "start": "2024-01-15T14:30:00", "end": "2024-01-15T14:31:00", "key_frame": "/media/llmvision/snapshots/event_abc123.jpg", "camera_name": "camera.front_door", "label": "person" } ] } ``` ``` -------------------------------- ### Create Custom Timeline Event Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Manually creates a custom event in the LLM Vision timeline with a title, description, label, camera entity, image path, and specific start/end times. ```yaml service: llmvision.create_event data: title: "Visitor arrived" description: "John Smith arrived for scheduled meeting" label: "Person" camera_entity: camera.front_door image_path: "/media/llmvision/snapshots/visitor_john.jpg" start_time: "2024-01-15 14:30:00" end_time: "2024-01-15 14:31:00" ``` -------------------------------- ### Pytest Verbose Logging Commands Source: https://github.com/valentinfrlch/ha-llmvision/blob/main/README_TESTING.md Use these commands to control the verbosity of pytest output. The first command shows all log output, while the second focuses on warnings. ```bash # Show all log output pytest tests/ -v --log-cli-level=DEBUG ``` ```bash # Show warnings pytest tests/ -v -W all ``` -------------------------------- ### Analyze Live Camera Stream Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Analyzes a live camera stream for a specified duration and extracts information based on a provided message. The results can include summaries, titles, and key frames. ```APIDOC ## POST llmvision.stream_analyzer ### Description Analyzes a live camera stream for a specified duration, extracting information based on a provided message. Useful for summarizing activity, identifying objects, and detecting unusual events. ### Method POST ### Endpoint llmvision.stream_analyzer ### Parameters #### Request Body - **provider** (string) - Required - The unique identifier for the AI provider. - **model** (string) - Optional - The specific AI model to use for analysis. - **message** (string) - Required - The prompt or question for the AI model. - **image_entity** (list of strings) - Required - A list of camera entities to analyze. - **duration** (integer) - Optional - The duration in seconds to analyze the stream (default: 10). - **max_frames** (integer) - Optional - The maximum number of frames to process. - **target_width** (integer) - Optional - The target width for frame processing. - **max_tokens** (integer) - Optional - The maximum number of tokens for the AI response. - **store_in_timeline** (boolean) - Optional - Whether to store the analysis results in the timeline. - **generate_title** (boolean) - Optional - Whether to generate a title for the analysis. - **use_memory** (boolean) - Optional - Whether to use memory for context in the analysis. - **expose_images** (boolean) - Optional - Whether to expose analyzed images in the response. ### Request Example ```json { "provider": "01234567-89ab-cdef-0123-456789abcdef", "model": "gemini-2.0-flash", "message": "Summarize the activity in this camera feed. Report any people, vehicles, or unusual events.", "image_entity": [ "camera.front_door", "camera.driveway" ], "duration": 10, "max_frames": 5, "target_width": 1280, "max_tokens": 3000, "store_in_timeline": true, "generate_title": true, "use_memory": true, "expose_images": true } ``` ### Response #### Success Response (200) - **title** (string) - The generated title for the analysis. - **response_text** (string) - The detailed text response from the AI model. - **key_frame** (string) - The path to a key frame image from the analysis. #### Response Example ```json { "title": "Person walking dog", "response_text": "A person walking a medium-sized dog passes by on the sidewalk. No other activity detected.", "key_frame": "/media/llmvision/snapshots/front_door_stream_1234567890.jpg" } ``` ``` -------------------------------- ### Create Custom Timeline Event Source: https://context7.com/valentinfrlch/ha-llmvision/llms.txt Manually creates events in the LLM Vision timeline without AI analysis. This is useful for logging custom events or integrating with other systems. ```APIDOC ## POST llmvision.create_event ### Description Manually creates events in the LLM Vision timeline without AI analysis. Useful for logging custom events or integrating with other systems. ### Method POST ### Endpoint llmvision.create_event ### Parameters #### Request Body - **title** (string) - Required - The title of the event. - **description** (string) - Optional - A detailed description of the event. - **label** (string) - Optional - A label for categorizing the event. Available labels: Alarm, Bike, Bird, Bus, Camera, Car, Cat, Dog, Door, Key, Light, Lock, Motorcycle, Package, Person, Plant, Sensor, Tree, Truck, Van. - **camera_entity** (string) - Optional - The camera entity associated with the event. - **image_path** (string) - Optional - The path to an image associated with the event. - **start_time** (string) - Optional - The start time of the event in `YYYY-MM-DD HH:MM:SS` format. Defaults to the current time if not provided. - **end_time** (string) - Optional - The end time of the event in `YYYY-MM-DD HH:MM:SS` format. ### Request Example (Custom Visitor Event) ```yaml service: llmvision.create_event data: title: "Visitor arrived" description: "John Smith arrived for scheduled meeting" label: "Person" camera_entity: camera.front_door image_path: "/media/llmvision/snapshots/visitor_john.jpg" start_time: "2024-01-15 14:30:00" end_time: "2024-01-15 14:31:00" ``` ### Request Example (Gate Opened Event) ```yaml service: llmvision.create_event data: title: "Gate opened" description: "Main gate was opened via automation" label: "Door" ``` ### Response This service does not return a direct response. It creates an event in the LLM Vision timeline. ```