### Setup Development Environment

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/vscode_tasks.md

Installs or updates Python dependencies from requirements files. Essential for initial setup or after dependency changes.

```bash
./scripts/setup
```

--------------------------------

### Setup Project Dependencies

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/CLAUDE.md

Installs project dependencies using pip. This is automatically run in the devcontainer.

```bash
pip install -r requirements.txt
pip install -r requirements.test.txt
```

--------------------------------

### Run Integration Tests

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

After initial setup, run integration tests. This command will start Home Assistant with a fresh database if it's not already running.

```bash
source .env  # If using .env file for token
pytest tests/integration_tests
```

--------------------------------

### Copy Environment Example to .env

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

Copy the example environment file to .env to configure authentication tokens for integration tests. This file is gitignored.

```bash
cp .env.example .env
```

--------------------------------

### Start Home Assistant Development Instance

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

Run this script to start a development instance of Home Assistant with the custom component loaded. Access it at http://localhost:8123. The PYTHONPATH is automatically set to include custom_components.

```bash
scripts/develop
```

--------------------------------

### Install Dependencies and Run Linters

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/AGENTS.md

Commands to install project dependencies and format code using ruff.

```bash
scripts/setup              # Install dependencies from requirements.txt
scripts/lint              # Format with ruff and auto-fix issues
```

--------------------------------

### Start Home Assistant

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/CLAUDE.md

Starts the Home Assistant server within the devcontainer. Access it at http://localhost:8123.

```bash
hass -c config
```

--------------------------------

### Install/Update Dependencies

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

Run this script to install or update Python dependencies. This is also executed automatically during devcontainer setup.

```bash
scripts/setup
```

--------------------------------

### Example YAML for Importing Statistics

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/services.md

Demonstrates how to call the `import_statistics.import_from_json` service using YAML. Shows examples for both sensor and counter data types.

```yaml
service: import_statistics.import_from_json
data:
  entities:
    - id: sensor.temperature
      unit: "°C"
      values:
        - datetime: "01.01.2024 10:00"
          mean: 21.5
          min: 20.1
          max: 22.3
        - datetime: "01.01.2024 11:00"
          mean: 22.1
          min: 21.2
          max: 23.4
    - id: counter:energy
      unit: "kWh"
      values:
        - datetime: "01.01.2024 10:00"
          sum: 1234.56
          state: 1234.56
```

--------------------------------

### CSV Import Example

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/README.md

Example of a CSV file format for importing statistics, including required and optional columns for sensor data.

```csv
statistic_id,unit,start,min,max,mean
sensor.temperature,°C,2024-01-01 10:00,20.0,22.0,21.0
sensor.temperature,°C,2024-01-01 11:00,19.0,23.0,21.0
```

--------------------------------

### TSV Import with Deltas

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/README.md

Example of data format for TSV import with time deltas. Ensure statistic_id, unit, start, and delta columns are present.

```tsv
statistic_id	unit	start	delta
counter:energy	kWh	2024-01-01 10:00	10.5
counter:energy	kWh	2024-01-01 11:00	12.3
```

--------------------------------

### Configure VSCode Task Shortcut

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/vscode_tasks.md

Example of how to add a keyboard shortcut in VSCode's settings.json to run a specific task.

```json
{
  "key": "ctrl+shift+t",
  "command": "workbench.action.tasks.runTask",
  "args": "Run Tests"
}
```

--------------------------------

### Registering the Import Service

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/configuration.md

Demonstrates how the 'import_from_file' service is registered within the Home Assistant integration during its setup. It includes a placeholder for timezone URLs.

```python
hass.services.async_register(
    DOMAIN,
    "import_from_file",
    handle_import_from_file,
    description_placeholders={"pytz_url": "https://en.wikipedia.org/wiki/List_of_tz_database_time_zones"}
)
```

--------------------------------

### Example Debug Log Output

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/user/debug-logging.md

This is an example of the debug log output you might see when the import statistics integration is running. It includes INFO and DEBUG level messages related to service calls and data preparation.

```text
2026-01-24 18:31:37.399 INFO (MainThread) [custom_components.import_statistics.helpers] Service handle_export_statistics called
2026-01-24 18:31:37.399 INFO (MainThread) [custom_components.import_statistics.helpers] Exporting entities: ALL
2026-01-24 18:31:37.399 INFO (MainThread) [custom_components.import_statistics.helpers] Time range: AUTO to AUTO
2026-01-24 18:31:37.399 INFO (MainThread) [custom_components.import_statistics.helpers] Output file: xx.tsv
2026-01-24 18:31:37.399 INFO (MainThread) [custom_components.import_statistics.helpers] Fetching statistics from recorder API
2026-01-24 18:31:37.399 INFO (MainThread) [custom_components.import_statistics.helpers] No entities specified, fetching all statistics from database
2026-01-24 18:31:37.408 INFO (MainThread) [custom_components.import_statistics.helpers] Found 25 statistics in database
2026-01-24 18:31:37.446 DEBUG (MainThread) [custom_components.import_statistics.helpers] Global statistics time range determined: start=2025-06-29 05:00:00+00:00 end=2025-12-30 10:00:00+00:00
2026-01-24 18:31:37.463 INFO (SyncWorker_1) [custom_components.import_statistics.helpers] Preparing export data
2026-01-24 18:31:37.464 INFO (SyncWorker_1) [custom_components.import_statistics.helpers] Export contains both measurement (mean/min/max) and counter (sum/state) statistics. Measurement columns will be empty for counters and vice versa.
2026-01-24 18:31:37.465 DEBUG (SyncWorker_1) [custom_components.import_statistics.helpers] Export data prepared with columns: ['statistic_id', 'unit', 'start', 'min', 'max', 'mean', 'sum', 'state', 'delta']
```

--------------------------------

### Delta Import Example Data

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/user/counters.md

Illustrates the state of a counter after delta imports, showing calculated sums, states, and deltas. Highlights a spike and its correction.

```text
29.12.2025 13:00: sum=66, state=76 (delta: 15, corrected)
29.12.2025 14:00: sum=81, state=91 (delta: 15, corrected)
29.12.2025 15:00: sum=28, state=38 (delta: -53, **spike!** - negative value to compensate)
29.12.2025 16:00: sum=36, state=46 (delta: 8, unchanged)
```

--------------------------------

### Import Statistics Example

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-import-service.md

Shows how to import prepared statistics into the Home Assistant database. This function handles validation and processing for different statistic sources, such as the recorder or external domains.

```python
async def example_import(hass: HomeAssistant):
    stats = {
        "sensor.temperature": (
            {
                "statistic_id": "sensor.temperature",
                "source": "recorder",
                "unit_of_measurement": "°C",
                "unit_class": "temperature",
                "mean_type": StatisticMeanType.ARITHMETIC,
                "has_sum": False,
                "name": None,
            },
            [
                {"start": datetime(...), "min": 20, "max": 22, "mean": 21},
                {"start": datetime(...), "min": 19, "max": 23, "mean": 21},
            ]
        )
    }
    await import_stats(hass, stats)
```

--------------------------------

### VSCode Task Environment Sourcing Example

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/vscode_tasks.md

Illustrates how test-related VSCode tasks automatically source the .env file to load environment variables.

```bash
bash -lc 'set -a; [ -f .env ] && source .env; set +a; ...'
```

--------------------------------

### Export Inventory YAML Configuration

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/README.md

Example YAML configuration for exporting the statistics inventory. Specify the output filename.

```yaml
action: import_statistics.export_inventory
data:
  filename: statistics_inventory.csv
```

--------------------------------

### Prepare Delta Handling References

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-import-service.md

Fetches and validates delta conversion references from the database. Requires Home Assistant instance and a pandas DataFrame with statistic_id, start, and delta columns.

```python
from homeassistant.core import HomeAssistant
from typing import Any

async def prepare_delta_handling(
    hass: HomeAssistant,
    df: Any,  # pandas.DataFrame
) -> dict[str, dict]:
    # Implementation details omitted for brevity
    pass

# Example return structure:
# {
#     statistic_id: {
#         "reference": {
#             "start": datetime,
#             "sum": float | None,
#             "state": float | None,
#         },
#         "ref_type": DeltaReferenceType.OLDER_REFERENCE or DeltaReferenceType.NEWER_REFERENCE,
#     } or None (if error),
#     ...
# }
```

--------------------------------

### Statistics Inventory Table File Content Example

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/README.md

Example content of the table file generated during statistics inventory export. Each row represents a single statistic with its metadata.

```text
statistic_id,unit_of_measurement,source,category,type,samples_count,first_seen,last_seen,days_span
sensor.button_master_power,%,recorder,Active,Measurement,32949,2/12/2022 10:00,2/12/2026 13:00,1461.1
sensor.disk_free,GiB,recorder,Active,Measurement,35876,1/9/2022 16:00,2/12/2026 13:00,1494.9
sensor.disk_use,GiB,recorder,Active,Measurement,35876,1/9/2022 16:00,2/12/2026 13:00,1494.9
sensor.disk_use_percent,%,recorder,Active,Measurement,35876,1/9/2022 16:00,2/12/2026 13:00,1494.9
sensor.e3_tcu10_x07_buffer_main_temperature,°C,recorder,Active,Measurement,20326,9/29/2023 10:00,2/12/2026 13:00,867.2
sensor.e3_tcu10_x07_compressor_hours,h,recorder,Active,Counter,17638,1/20/2024 21:00,2/12/2026 13:00,753.7
sensor.e3_tcu10_x07_compressor_starts,,recorder,Active,Counter,17638,1/20/2024 21:00,2/12/2026 13:00,753.7
```

--------------------------------

### JSON Import/Export Example

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/README.md

Example structure for importing or exporting statistical data in JSON format. Includes entity ID, unit, and time-series values.

```json
{
  "entities": [
    {
      "id": "sensor.temperature",
      "unit": "°C",
      "values": [
        {
          "datetime": "2024-01-01 10:00",
          "min": 20.0,
          "max": 22.0,
          "mean": 21.0
        }
      ]
    }
  ]
}
```

--------------------------------

### Home Assistant Debug Configuration (Skip Pip Checks)

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/vscode_debugging.md

This configuration is similar to the Component Dev setup but skips pip dependency checks for faster startup. It's useful for repeated debug sessions when dependencies are already verified.

```json
{
    "name": "Home Assistant (skip pip)",
    "type": "debugpy",
    "request": "launch",
    "module": "homeassistant",
    "justMyCode": false,
    "args": [
        "--debug",
        "-c",
        "${workspaceFolder}/config",
        "--skip-pip"
    ],
    "env": {
        "PYTHONPATH": "${workspaceFolder}/custom_components"
    }
}
```

--------------------------------

### Clean Configuration Directory

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

Use this script to reset the configuration directory for a fresh start. It preserves configuration.yaml, CSV files, test files, and markdown files.

```bash
scripts/clean_config
```

--------------------------------

### JSON Mixed Format Example

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/plans/mixed-import-architecture.md

Demonstrates a JSON structure where each entity can have its own set of value keys, naturally supporting mixed data types.

```json
[
    {
        "id": "sensor.temperature",
        "unit": "°C",
        "values": [
            {"datetime": "26.01.2024 12:00", "min": 20, "max": 21, "mean": 20.5}
        ]
    },
    {
        "id": "counter.energy",
        "unit": "kWh",
        "values": [
            {"datetime": "26.01.2024 12:00", "sum": 10.5, "state": 100}
        ]
    }
]
```

--------------------------------

### Unit Test for Helper Functions

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/README.md

Example of a unit test for validating helper functions without requiring a running Home Assistant instance. Uses pandas DataFrame for input.

```python
# Unit test example (no HA needed)
import pandas as pd
from custom_components.import_statistics.helpers import are_columns_valid

df = pd.DataFrame({
    "statistic_id": ["sensor.temp"],
    "start": [datetime(...)],
    "unit": ["°C"],
    "min": [20.0],
    "max": [22.0],
    "mean": [21.0],
})

assert are_columns_valid(df) is True
```

--------------------------------

### Add Historical Data Before Sensor Existed

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/user/counters.md

Use this example when you have historical data from before a sensor was added to Home Assistant. The import will create new entries for the earlier dates.

```tsv
statistic_id start unit delta
sensor.imp_before 28.12.2025 09:00 kWh 10
sensor.imp_before 28.12.2025 10:00 kWh 20
sensor.imp_before 28.12.2025 11:00 kWh 30
```

--------------------------------

### Prepare Delta Handling Example

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-import-service.md

Demonstrates how to prepare data for delta handling, which is used to process time-series data where only the change (delta) is recorded. This function is useful when dealing with sensors that report cumulative values like energy meters.

```python
import pandas as pd
import datetime as dt
from homeassistant.core import HomeAssistant

async def example_delta_handling(hass: HomeAssistant):
    df = pd.DataFrame({
        "statistic_id": ["sensor.energy", "sensor.energy"],
        "start": [
            dt.datetime(2024, 1, 15, 10, 0, tzinfo=dt.timezone.utc),
            dt.datetime(2024, 1, 15, 11, 0, tzinfo=dt.timezone.utc),
        ],
        "unit": ["kWh", "kWh"],
        "delta": [10.5, 12.3],
    })
    
    refs = await prepare_delta_handling(hass, df)
    # refs = {
    #     "sensor.energy": {
    #         "reference": {"start": ..., "sum": 1234.0, "state": 1234.0},
    #         "ref_type": DeltaReferenceType.OLDER_REFERENCE
    #     }
    # }
```

--------------------------------

### Add New Consumption Data

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/user/counters.md

Shows how to import new consumption data for hours not yet present in the database. The example uses a TSV format for the import values.

```tsv
statistic_id start unit delta
sensor.imp_after 30.12.2025 09:00 kWh 10
sensor.imp_after 30.12.2025 10:00 kWh 20
sensor.imp_after 30.12.2025 11:00 kWh 30
```

--------------------------------

### Template Sensor Definitions

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/plans/mixed-import-architecture.md

Example configuration for defining template sensors in configuration.yaml. These are necessary for internal entities used in testing the mixed import functionality.

```yaml
template:
  - sensor:
      - name: "mixed_test_temp"
        state: "{{ states('sensor.some_temp_sensor') }}"
      - name: "mixed_test_energy"
        state: "{{ states('sensor.some_energy_sensor') }}"
```

--------------------------------

### Jeedom Exported CSV Format Example

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/misc/jeedom.md

This shows the raw format of a CSV file exported from Jeedom, which requires preparation before importing into Home Assistant.

```csv
2019-11-17 17:00:00;0,000000
2019-11-17 18:00:00;911,797500
2019-11-18 06:00:00;312,750000
2019-11-19 06:00:00;309,500000
2019-11-20 06:00:00;0,000000
....
2022-02-15 06:17:30;617,267500
2022-02-15 06:50:00;307,630000
...
2025-02-16 02:00:02;336,750000000
2025-02-16 03:03:02;388,550000000
2025-02-16 04:06:01;374,750000000
```

--------------------------------

### Setup Home Assistant Service

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/AGENTS.md

Registers services synchronously. Import and export handlers are asynchronous for delta processing and blocking I/O respectively. Data preparation and delta reference fetching utilize executors and async operations for HA compatibility.

```python
setup()
```

--------------------------------

### Correct Values in the Middle with Spike

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/user/counters.md

This example demonstrates correcting sensor values that result in a spike due to import file errors. It highlights how incorrect deltas can affect subsequent data points.

```tsv
statistic_id start unit delta
sensor:imp_inside_spike 29.12.2025 09:00 kWh 12
sensor:imp_inside_spike 29.12.2025 10:00 kWh 12
sensor:imp_inside_spike 29.12.2025 11:00 kWh 12
sensor:imp_inside_spike 29.12.2025 12:00 kWh 15
sensor:imp_inside_spike 29.12.2025 13:00 kWh 15
sensor:imp_inside_spike 29.12.2025 14:00 kWh 15
```

--------------------------------

### Timestamp Format Example

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/CLAUDE.md

Specifies the required timestamp format for CSV/TSV imports, which must be at the full hour.

```text
"%d.%m.%Y %H:%M" (e.g., "17.03.2024 02:00")
```

--------------------------------

### Run Integration Tests with .env

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

Source the .env file to load authentication tokens and then run integration tests. This is an alternative to using VSCode tasks.

```bash
source .env
pytest tests/integration_tests
```

--------------------------------

### Get Statistic Source Helper

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/README.md

Retrieves the source of a statistic ID, such as 'recorder' for entity-based or 'counter' for domain-based.

```python
from custom_components.import_statistics.helpers import get_source

source = get_source("sensor.temperature")  # "recorder"
source = get_source("counter:energy")      # "counter"
```

--------------------------------

### Run All Tests

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/CLAUDE.md

Executes the full test suite, including unit and integration tests with automatic dependency handling.

```bash
pytest
```

--------------------------------

### _resolve_time_range

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-export-service.md

Resolves the start and end times for statistics export. It can fetch the earliest or latest timestamps from the database if not explicitly provided.

```APIDOC
## _resolve_time_range()

### Description

Resolve start and end times, fetching from database if needed.

### Signature

```python
async def _resolve_time_range(
    hass: HomeAssistant,
    start_dt: dt.datetime | None,
    end_dt: dt.datetime | None,
    metadata: dict,
) -> tuple[dt.datetime, dt.datetime]
```

### Parameters

| Parameter | Type | Description |
|-----------|------|-------------|
| hass | HomeAssistant | Home Assistant instance |
| start_dt | datetime | Start time or None to fetch earliest |
| end_dt | datetime | End time or None to fetch latest |
| metadata | dict | Statistic metadata from `get_metadata()` |

### Return Value

Tuple of `(resolved_start_dt, resolved_end_dt)`: Both are timezone-aware UTC datetimes at full hours.

### Resolution Logic

If `start_dt` is None:
- Queries database for earliest statistic timestamp

If `end_dt` is None:
- Queries database for latest statistic timestamp
```

--------------------------------

### Export with Wildcard Pattern

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/services.md

Export statistics using a wildcard pattern for entities. This example also specifies counter fields for the export.

```yaml
service: import_statistics.export_statistics
data:
  filename: sensors.json
  entities:
    - "sensor.*"
  counter_fields: "sum"
```

--------------------------------

### Run All Tests

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/vscode_tasks.md

Executes all test suites (unit, integration_mock, integration) after sourcing the .env file. Use this after code changes or before opening a PR for a consistent environment.

```bash
bash -lc 'set -a; [ -f .env ] && source .env; set +a; ${command:python.interpreterPath} -m pytest tests/'
```

--------------------------------

### Check if Port 8123 is in Use (Linux/Mac)

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

Use this command to check if port 8123 is already occupied, which can prevent Home Assistant from starting.

```bash
lsof -i :8123
```

--------------------------------

### Configure PYTHONPATH for Complex Setups

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/vscode_debugging.md

Modify the PYTHONPATH environment variable in your launch configuration to include additional directories for Python module resolution.

```json
{
    "env": {
        "PYTHONPATH": "${workspaceFolder}/custom_components:${workspaceFolder}"
    }
}
```

--------------------------------

### Fix Home Assistant Startup Errors

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

This optional script can help eliminate common error messages that appear during Home Assistant startup. These errors are often related to missing packages like ffmpeg, libpcap, or libturbojpeg.

```bash
scripts/fix_ha_errors
```

--------------------------------

### Run Home Assistant and Tests

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/AGENTS.md

Commands to run Home Assistant with the custom component and execute tests.

```bash
scripts/develop           # Run Home Assistant with custom component (sets PYTHONPATH)
pytest                    # Run tests from tests/ directory
pytest -v tests/test_X.py # Run specific test file
```

--------------------------------

### Import Data Flow

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/architecture.md

Illustrates the process of importing statistics data, from file input to database storage, including type detection and handler selection.

```mermaid
graph TD
CSV/TSV/JSON Input File
         ↓
[prepare_data_to_import() or prepare_json_data_to_import()]
         ↓
DataFrame (with timezone-aware datetimes)
         ↓
[_validate_and_detect_delta()]
         ↓
Type Detection → SENSOR | COUNTER | MIXED | DELTA
         ↓
    ┌────┴─────┬──────────┬──────────┐
    ↓          ↓          ↓          ↓
 SENSOR    COUNTER     MIXED       DELTA
    ↓          ↓          ↓          ↓
    └────┬─────┴──────────┴──────────┘
         ↓
[Appropriate handler]
  handle_dataframe_no_delta()        (SENSOR/COUNTER)
  handle_dataframe_mixed()           (MIXED - splits then calls above)
  handle_dataframe_delta()           (DELTA - needs references)
         ↓
{statistic_id: (metadata, statistics_list)}
         ↓
[import_stats()]
         ↓
Home Assistant Recorder Database
```

--------------------------------

### Statistics Inventory Summary Output

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/README.md

Example content of the summary file generated during statistics inventory export. It provides aggregated information about the statistics.

```text
Total statistics: 257
Measurements: 194
Counters: 63
Total samples: 5038317
Global start: 2022-01-09 16:00:00
Global end: 2026-02-12 13:00:00
Active statistics: 224
Orphan statistics: 7
Deleted statistics: 26
External statistics: 0
```

--------------------------------

### Check if Port 8123 is in Use (Windows)

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

Use this command to check if port 8123 is already occupied on Windows, which can prevent Home Assistant from starting.

```bash
netstat -ano | findstr :8123
```

--------------------------------

### Export Selected Entities

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/README.md

Exports specific entities to a TSV file with a defined delimiter and decimal separator. Includes start and end times for the export range.

```yaml
action: import_statistics.export_statistics
data:
  filename: exported_statistics.tsv
  entities:
    - sensor.temperature
    - sensor.energy_consumption
    - sensor:ext_value
  start_time: "2025-12-22 12:00:00"
  end_time: "2025-12-25 12:00:00"
  delimiter: \	
  decimal: "."
  # timezone_identifier: Europe/Vienna  # Optional - defaults to HA timezone
```

--------------------------------

### Import Statistics Module Dependencies

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/architecture.md

Visualizes the internal module dependencies within the import_statistics integration and its connections to the Home Assistant core.

```mermaid
graph TB
    subgraph import_statistics["import_statistics"]
        const[const]
        helpers[helpers]
        import_service[import_service]
        import_service_helper[import_service_helper]
        import_service_delta_helper[import_service_delta_helper]
        delta_database_access[delta_database_access]
        export_service[export_service]
        export_service_helper[export_service_helper]
        init[__init__]
        config_flow[config_flow]
    end

    ha_core[Running HA]

    %% Internal dependencies
    helpers --> const
    import_service --> import_service_helper
    import_service --> delta_database_access
    import_service --> import_service_delta_helper
    import_service --> helpers
    import_service_helper --> helpers
    import_service_delta_helper --> helpers
    delta_database_access --> helpers
    export_service --> export_service_helper
    export_service --> helpers
    export_service_helper --> helpers
    init --> import_service
    init --> export_service

    %% Home Assistant Core dependencies
    init --> ha_core
    import_service --> ha_core
    export_service --> ha_core
```

--------------------------------

### Enable Debug Logging in configuration.yaml

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/user/debug-logging.md

Add this configuration to your Home Assistant configuration.yaml file to enable debug logging for the import_statistics custom component. Ensure Home Assistant is restarted or the logger is reloaded after changes.

```yaml
logger:
  default: info
  logs:
    custom_components.import_statistics: debug
```

--------------------------------

### Correct Values in the Middle of a Time Range

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/user/counters.md

Use this example to correct sensor values that were recorded incorrectly or when the sensor was offline. This ensures data continuity and accuracy.

```tsv
statistic_id start unit delta
sensor:imp_inside 29.12.2025 09:00 kWh 2
sensor:imp_inside 29.12.2025 10:00 kWh 2
sensor:imp_inside 29.12.2025 11:00 kWh 2
sensor:imp_inside 29.12.2025 12:00 kWh 5
sensor:imp_inside 29.12.2025 13:00 kWh 5
sensor:imp_inside 29.12.2025 14:00 kWh 5
```

--------------------------------

### get_unit_from_row()

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-helpers.md

Get and normalize the unit value from a given row in the import data. This function handles potential variations in how the unit is provided and ensures it is in a consistent format.

```APIDOC
## get_unit_from_row()

### Description
Get and normalize unit value from import row.

### Signature
```python
def get_unit_from_row(
    unit_from_row: Any,
    statistic_id: str
) -> str | None
```

### Parameters
- **unit_from_row** (Any) - The unit value as provided in the row.
- **statistic_id** (str) - The statistic ID, used for context in potential error messages.
```

--------------------------------

### Python Script Usage for Data Preparation

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/misc/jeedom.md

This command demonstrates how to use the provided Python scripts to convert Jeedom exported data into a format suitable for Home Assistant import.

```bash
python3 script.py <input_file> <output_file> <prefix>
```

--------------------------------

### Home Assistant Component Development Debug Configuration

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/vscode_debugging.md

Use this configuration to launch Home Assistant in debug mode with your custom components. It sets the configuration directory and includes your custom components in the Python path, enabling debugging of all code.

```json
{
    "name": "Home Assistant (Component Dev)",
    "type": "debugpy",
    "request": "launch",
    "module": "homeassistant",
    "justMyCode": false,
    "args": [
        "--debug",
        "-c",
        "${workspaceFolder}/config"
    ],
    "env": {
        "PYTHONPATH": "${workspaceFolder}/custom_components"
    }
}
```

--------------------------------

### Validate Timestamps Vectorized

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-helpers.md

Validates that all timestamps in a DataFrame's 'start' column are at the full hour (minutes and seconds are zero). Uses vectorized pandas operations for performance.

```python
def validate_timestamps_vectorized(
    df: pd.DataFrame
) -> None:
    """Validate all timestamps in DataFrame are at full hour."""
    # Validation
    # Checks that for all timestamps in df["start"]:
    # - Minutes == 0
    # - Seconds == 0
    pass
```

--------------------------------

### Missing Required Columns Error Message

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/errors.md

This error indicates that essential columns ('statistic_id', 'start', 'unit') are missing from the input file. The delimiter might also be incorrect.

```text
"The file must contain the columns 'statistic_id', 'start' and 'unit' (check delimiter). Number of found columns: {n}. Found columns: {list}"
```

--------------------------------

### WSL2 Performance Configuration

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

Configure WSL2 resource allocation on the Windows host to improve performance for development environments like Home Assistant and tests. Restart WSL and Docker Desktop after changes.

```ini
[wsl2]
memory=8GB
processors=8
swap=4GB
```

--------------------------------

### prepare_data_to_import()

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-import-helpers.md

Loads, validates, and prepares data from a CSV or TSV file for import into Home Assistant statistics. It handles file reading, parameter extraction, encoding validation, datetime parsing, DST adjustments, and data type detection.

```APIDOC
## prepare_data_to_import()

### Description
Loads, validates, and prepares data from a CSV or TSV file for import into Home Assistant statistics. It handles file reading, parameter extraction, encoding validation, datetime parsing, DST adjustments, and data type detection.

### Signature
```python
def prepare_data_to_import(
    file_path: str,
    call: ServiceCall,
    ha_timezone: str
) -> tuple[pd.DataFrame, str, str, ImportDataType]
```

### Parameters
#### Path Parameters
- **file_path** (str) - Required - Absolute path to CSV/TSV file

#### Request Body
- **call** (ServiceCall) - Required - Service call with `delimiter`, `decimal`, `datetime_format`, `timezone_identifier`
- **ha_timezone** (str) - Required - Home Assistant's configured timezone (fallback)

### Return Value
Tuple of `(df, timezone_id, datetime_format, data_type)`:
- **df** (pandas.DataFrame) - Validated DataFrame with timezone-aware datetime objects in `start` column
- **timezone_id** (str) - IANA timezone used for interpreting timestamps
- **datetime_format** (str) - Datetime format string used for parsing
- **data_type** (ImportDataType) - Detected type: SENSOR, COUNTER, MIXED, or DELTA

### Processing Steps
1. **Validate extension**: File must be `.csv` or `.tsv`
2. **Extract parameters**: Calls `handle_arguments()` to get delimiter, decimal, format, timezone
3. **Validate encoding**: Checks file is valid UTF-8
4. **Read file**: Uses pandas `read_csv()` with specified delimiter, decimal separator, and date format
5. **Validate datetime parsing**: Ensures `start` column was parsed to datetime objects
6. **Handle DST**: Localizes naive timestamps to user's timezone with DST handling
7. **Detect type**: Validates columns and detects data type (SENSOR/COUNTER/MIXED/DELTA)

### Raises
`HomeAssistantError`:
- Invalid extension: "Unsupported filename extension for 'data.json'..."
- File not found: "path /config/data.csv does not exist"
- Bad encoding: "File has encoding issues..."
- Invalid timestamp format: "Invalid timestamp format: ..."
- DST gap conflict: "Timestamp does not exist due to daylight saving time transition..."
- Column validation fails: "The file must contain the columns..."
- Unknown columns: "Unknown columns in file: ..."

### Constraints
- Timestamps must be at **full hour** (minutes=0, seconds=0)
- File must be UTF-8 encoded
- Timestamps in DST gaps (spring forward) are rejected
- All rows for same statistic_id must have same unit

### Example
```python
df, tz_id, dt_fmt, data_type = prepare_data_to_import(
    "/config/sensor_data.csv",
    ServiceCall(...),  # with delimiter=",", decimal=".", etc.
    "Europe/Vienna"
)

# df contains:
# statistic_id | unit | start                      | mean  | min   | max
# sensor.temp  | °C   | 2024-01-15 10:00:00+01:00 | 21.5  | 20.0  | 23.0
# sensor.temp  | °C   | 2024-01-15 11:00:00+01:00 | 22.1  | 21.0  | 23.5
```
```

--------------------------------

### Get Unit Class

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-helpers.md

Resolves the unit class (e.g., 'temperature', 'energy', 'power') for a given unit of measurement string. Returns None for unknown or unmapped units.

```python
assert get_unit_class("°C") == "temperature"
assert get_unit_class("kWh") == "energy"
assert get_unit_class("W") == "power"
assert get_unit_class("unknown_unit") is None
assert get_unit_class(None) is None
```

--------------------------------

### Import Statistics from JSON via API

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/README.md

This HTTP POST request demonstrates how to import statistics using the Home Assistant API. Ensure the Content-Type is set to application/json.

```http
POST https://<your-ha-url>/api/services/import_statistics/import_from_json
Content-Type: application/json

<JSON content>
```

--------------------------------

### Run Specific Test Suites

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/docs/dev/README.md

Execute only unit tests, integration tests with mocks, or full integration tests.

```bash
# Unit tests only (fast, no HA required)
pytest tests/unit_tests

# Integration tests with mocks only
pytest tests/integration_tests_mock

# Full integration tests (requires HA_TOKEN_DEV)
pytest tests/integration_tests
```

--------------------------------

### Verbose Output: Show Print Statements and Logging

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/tests/README.md

Runs tests with verbose output, including print statements and logging information.

```bash
pytest -v -s
```

--------------------------------

### Run Tests Matching a Pattern

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/tests/README.md

Executes tests in a specified directory that match a given keyword pattern.

```bash
pytest tests/unit_tests -k "handle_error"
```

--------------------------------

### Get Unit From Row

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-helpers.md

Extracts and normalizes a unit of measurement from a given row value. Handles None, NaN, empty strings, and specific string literals like 'nan' or 'None'.

```python
assert get_unit_from_row("°C", "sensor.temp") == "°C"
assert get_unit_from_row(None, "sensor.temp") is None
assert get_unit_from_row(float('nan'), "sensor.temp") is None
assert get_unit_from_row("", "sensor.temp") is None
assert get_unit_from_row("  ", "sensor.temp") is None

# Raises error:
get_unit_from_row("nan", "sensor.temp")  # String literal "nan"
get_unit_from_row("None", "sensor.temp")  # String literal "None"
```

--------------------------------

### Get Statistic Source Domain

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/api-reference-helpers.md

Determines the source domain ('recorder' for internal, or the actual domain for external statistics) based on the statistic ID format. Use for parsing statistic IDs.

```python
assert get_source("sensor.temperature") == "recorder"
assert get_source("counter:energy") == "counter"
assert get_source("myapp:custom_metric") == "myapp"
```

--------------------------------

### Module Organization Structure

Source: https://github.com/klausj1/homeassistant-statistics/blob/main/_autodocs/architecture.md

Overview of the directory and file structure for the custom_components/import_statistics integration.

```tree
custom_components/import_statistics/
├── __init__.py                           # Entry point, service registration
├── const.py                              # Constants (domain, attribute names, defaults)
├── config_flow.py                        # Config flow (minimal, no user configuration)
│
├── helpers.py                            # Core validation/conversion utilities
├── import_service.py                     # Import service handlers
├── import_service_helper.py              # CSV/TSV/JSON loading & DataFrame processing
├── import_service_delta_helper.py        # Delta-to-absolute conversion (pure calculation)
├── delta_database_access.py              # Async DB queries for delta references
│
├── export_service.py                     # Export service handlers
├── export_service_helper.py              # Export formatting & file writing
├── export_database_access.py             # Database time range queries
├── export_inventory_service.py           # Inventory export handler
├── export_inventory_helper.py            # Inventory formatting
└── export_inventory_database_access.py   # Inventory data fetching
```