### Basic Logging Setup and Usage

Source: https://github.com/kovashikawa/bls_data/blob/main/bls_logging/README.md

Set up the logging system once at application start and then get a logger for your module. Use the logger for informational and error messages.

```python
from bls_logging.config import setup_logging, get_logger

# Setup logging (call once at application start)
setup_logging(log_level="INFO", log_dir="logs")

# Get a logger for your module
log = get_logger(__name__)

# Use the logger
log.info("Application started")
log.error("Something went wrong", exc_info=True)
```

--------------------------------

### Install API Dependencies

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Ensure all necessary dependencies for the API are installed. This is a troubleshooting step for API startup issues.

```bash
make api-install
```

--------------------------------

### Install API Dependencies

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Installs the necessary Python packages for the API using either the Makefile or pip.

```bash
# Using the Makefile
make api-install
```

```bash
# Or manually
./venv/bin/pip install fastapi "uvicorn[standard]" "psycopg[binary]" pydantic python-multipart
```

--------------------------------

### Install all dependencies with uv

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Installs all dependencies, including development and testing extras, for the bls_data project using uv. Use this for development or testing.

```bash
uv sync --all-extras
```

--------------------------------

### Install dependencies with pip

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Installs project dependencies from the requirements.txt file using pip. This is a traditional method for setting up Python project environments.

```bash
pip install -r requirements.txt
```

--------------------------------

### Start the FastAPI REST API Server

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Run this command to start the FastAPI application, which provides CRUD operations for the PostgreSQL database. The API will be accessible at `http://localhost:8000`, with interactive documentation available at `/docs`.

```bash
# Start the API server
python bls_api.py
```

--------------------------------

### Install uv package manager

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Installs the uv package manager, a fast Python package installer and resolver. Use this for managing project dependencies.

```bash
pip install uv
# or
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Start the API Server

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Starts the BLS Data RESTful API server using either the Makefile or a Python script. The API will be accessible at http://localhost:8000.

```bash
# Using the Makefile
make api-start
```

```bash
# Or manually
./venv/bin/python bls_api.py
```

--------------------------------

### Structured Logging Setup

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Configuration and usage of structured logging within the application.

```APIDOC
## Logging Configuration

### Description
Provides structured logging capabilities using `bls_logging.config`. Includes setup for rotating file handlers and console output.

### Functions
1.  **setup_logging()**
    *   **Description**: Sets up the global logging configuration. Should be called once at the application's entry point.
    *   **Parameters**:
        *   `log_level` (string) - The minimum severity level to log (e.g., "INFO", "WARNING").
        *   `log_dir` (string) - Directory to store log files. Created automatically if it doesn't exist.
        *   `max_file_size` (integer) - Maximum size of a log file in bytes before rotation (e.g., 10485760 for 10MB).
        *   `backup_count` (integer) - Number of backup log files to keep.
        *   `console_output` (boolean) - Whether to log to the console.
        *   `file_output` (boolean) - Whether to log to files.
    *   **Example Usage**:
        ```python
        from bls_logging.config import setup_logging
        setup_logging(
            log_level="INFO",
            log_dir="logs",
            max_file_size=10_485_760,
            backup_count=5,
            console_output=True,
            file_output=True,
        )
        ```

2.  **get_logger(name: str)**
    *   **Description**: Retrieves a logger instance for a given module name. Should be called in every module that needs logging.
    *   **Parameters**:
        *   `name` (string) - The name of the logger, typically `__name__`.
    *   **Example Usage**:
        ```python
        from bls_logging.config import get_logger
        log = get_logger(__name__)
        log.info("Application started")
        ```

3.  **@log_function_call**
    *   **Description**: Decorator to automatically log every function call, including arguments and return values.
    *   **Example Usage**:
        ```python
        from bls_logging.config import log_function_call
        @log_function_call
        def fetch_data(series_id: str) -> dict:
            return {}
        ```

4.  **@log_performance**
    *   **Description**: Decorator to log the execution time of a function.
    *   **Example Usage**:
        ```python
        from bls_logging.config import log_performance
        @log_performance
        def expensive_parse(raw: dict):
            pass
        ```
```

--------------------------------

### Install production dependencies with uv

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Installs the production dependencies for the bls_data project using the uv package manager. Ensures the project runs with necessary libraries.

```bash
uv sync
```

--------------------------------

### Clone bls_data repository

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Clones the bls_data repository from GitHub and navigates into the project directory. This is the first step for local setup.

```bash
git clone https://github.com/kovashikawa/bls_data.git
cd bls_data
```

--------------------------------

### Makefile Commands for Project Management

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Convenient Makefile targets for common project operations. Includes commands for installation, running scripts, testing, code formatting, linting, cleaning, and displaying project information.

```bash
make install              # Install dependencies
make run python scripts/test_cpi_extraction.py
make test                 # Run tests
make format               # Format code (ruff, black, isort)
make lint                 # Lint code (ruff, flake8, mypy)
make ruff-check           # Run ruff linting only
make ruff-format          # Run ruff formatting only
make ruff-fix             # Run ruff auto-fix
make clean                # Clean up
make info                 # Show project info
```

--------------------------------

### UV Development Commands

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

A collection of commands to manage dependencies, run scripts, and perform development tasks using `uv`. Includes commands for installing, running, formatting, linting, type checking, testing, and package management.

```bash
# Install dependencies
uv sync                    # Production dependencies
uv sync --all-extras      # All dependencies (dev, test, docs)
```

```bash
# Run scripts
uv run python scripts/test_cpi_extraction.py
uv run python setup_database.py
```

```bash
# Development commands
uv run ruff format .      # Format code with ruff
uv run ruff check .       # Lint code with ruff
uv run ruff check --fix . # Auto-fix linting issues
uv run black .            # Format code with black
uv run isort .            # Sort imports
uv run flake8 .           # Lint code with flake8
uv run mypy .             # Type checking
uv run pytest             # Run tests
uv run pytest --cov      # Run tests with coverage
```

```bash
# Package management
uv add requests           # Add a dependency
uv add --dev pytest      # Add a dev dependency
uv remove requests        # Remove a dependency
uv sync --upgrade         # Update dependencies
```

--------------------------------

### API Error Response Example

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

The API returns detailed error information in JSON format when an issue occurs. This example shows a 'Series not found' error.

```json
{
  "error": "Series not found",
  "detail": "Series with ID 'INVALID_ID' does not exist"
}
```

--------------------------------

### Run Data Extraction with Database Caching

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Use this command to extract CPI data, leveraging PostgreSQL for caching. Specify start and end years. The `--use-database` flag enables caching.

```bash
python -m data_extraction.main \
  cpi_all_items \
  --start 2020 --end 2023 \
  --use-database \
  --log DEBUG
```

--------------------------------

### Get All Series with Filtering

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Retrieves a list of series from the API, applying filters for limit, survey name, and area. Ensure the API server is running.

```bash
curl "http://localhost:8000/bls_series?limit=10&survey_name=CPI&area=U.S. City Average"
```

--------------------------------

### Fetch BLS Data by Alias with Catalog Metadata

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Use `get_bls_data` to retrieve economic series by human-readable aliases. This example includes catalog metadata and demonstrates how to print DataFrame columns and tail rows for a specific series.

```python
from data_extraction.main import get_bls_data

# Fetch three economic series by alias for 2018-2023 with catalog metadata
df = get_bls_data(
    codes_or_ids=["cpi_all_items", "ces_all_employees", "unemployment_rate"],
    start_year=2018,
    end_year=2023,
    catalog=True,          # include survey name, area, item, seasonality
    calculations=False,    # skip net/percent change columns
    annualaverage=False,   # skip M13 annual average rows
    use_database=False,    # pure API mode (no PostgreSQL required)
)

print(df.columns.tolist())
# ['series_id', 'alias', 'year', 'period', 'period_name', 'value', 'latest',
#  'seasonality', 'series_title', 'survey_name', 'measure_data_type',
#  'area', 'item', 'footnotes']

print(df[df["alias"] == "cpi_all_items"].tail(3).to_string(index=False))
# series_id          alias  year period period_name   value  ...
# CUSR0000SA0  cpi_all_items  2023    M10    October  307.671
# CUSR0000SA0  cpi_all_items  2023    M11   November  307.051
# CUSR0000SA0  cpi_all_items  2023    M12   December  306.746
```

--------------------------------

### MCP Server for AI Integration

Source: https://context7.com/kovashikawa/bls_data/llms.txt

The MCP server exposes BLS data functions as tools for AI assistants. Run `python mcp_server.py` to enable communication over stdio. Examples show tool responses for fetching series data and analyzing CPI seasonality.

```python
# mcp_server.py registers four tools via @mcp.tool():
# 1. get_series       — fetch a series by ID with optional date range
# 2. get_series_info  — fetch catalog metadata for a series
# 3. search_series    — search series titles (requires DB for full results)
# 4. analyze_cpi_seasonality — compute MoM percentile bands + base64 plot

# Example tool responses (as returned to an MCP client):

# get_series("CUUR0000SA0", start="2023", end="2023")
{
  "series_id": "CUUR0000SA0",
  "data": [
    {"series_id": "CUUR0000SA0", "year": 2023, "period": "M01", "value": 299.17, ...},
    ...
  ],
  "shape": [12, 14],
  "columns": ["series_id", "alias", "year", "period", "period_name", "value", ...]
}

# analyze_cpi_seasonality("CUUR0000SA0", start="2013", end="2023")
{
  "series_id": "CUUR0000SA0",
  "table": [
    {"month": 1, "month_name": "Jan", "p25": -0.12, "p50": 0.03, "p75": 0.21, "current": 0.51},
    ...
  ],
  "image_base64": "<base64-encoded PNG>",
  "summary_stats": {
    "analysis_period": "2013-2023",
    "historical_data_points": 120,
    "avg_historical_mom": 0.182,
    "std_historical_mom": 0.241
  }
}
```

--------------------------------

### Fetch BLS Data using Python

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Import and use the `get_bls_data` function to retrieve data by series IDs or aliases. Supports fetching catalogue metadata and specifying date ranges. Requires the `bls_data` library to be installed.

```python
from bls_data.data_extraction.main import get_bls_data

# Fetch CPI (All Items), CES (All Employees) and the Unemployment Rate
# between 2018 and 2023, including catalogue metadata
aliases = [
    "cpi_all_items",      # Consumer Price Index for All Urban Consumers
    "ces_all_employees",  # All Employees, Total Nonfarm
    "unemployment_rate"   # Civilian Unemployment Rate
]

df = get_bls_data(
    codes_or_ids=aliases,
    start_year=2018,
    end_year=2023,
    catalog=True
)

print(df.head())
```

--------------------------------

### Get Data Points for a Specific Series

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Use this endpoint to retrieve data points for a given series ID, year, and an optional limit for the number of results. Ensure the API is running on localhost:8000.

```bash
curl "http://localhost:8000/bls_data_points?series_id=CUSR0000SA0&year=2023&limit=12"
```

--------------------------------

### Interact with BLS API using cURL

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Examples of common API interactions using cURL, including health checks, listing series with filters, creating, retrieving, updating, and deleting series, and querying data points. Ensure correct URL and HTTP methods are used.

```bash
curl http://localhost:8000/health
```

```bash
curl "http://localhost:8000/bls_series?survey_name=Consumer+Price+Index&limit=5"
```

```bash
curl -X POST http://localhost:8000/bls_series \
  -H "Content-Type: application/json" \
  -d '{
    "series_id": "CUSR0000SA0",
    "series_title": "CPI All Urban Consumers - All Items SA",
    "survey_name": "Consumer Price Index",
    "area": "U.S. City Average",
    "item": "All Items",
    "seasonality": "Seasonally Adjusted",
    "data_frequency": "monthly"
  }'
```

```bash
curl http://localhost:8000/bls_series/CUSR0000SA0
```

```bash
curl "http://localhost:8000/bls_data_points?series_id=CUSR0000SA0&year=2023&limit=12"
```

```bash
curl -X PUT http://localhost:8000/bls_series/CUSR0000SA0 \
  -H "Content-Type: application/json" \
  -d '{"series_title": "Updated Title"}'
```

```bash
curl -X DELETE http://localhost:8000/bls_series/CUSR0000SA0
```

--------------------------------

### Initialize Database Tables

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Set up the necessary tables in the database. This is a troubleshooting step for data issues.

```bash
make db-setup
```

--------------------------------

### Detailed Logging Configuration

Source: https://github.com/kovashikawa/bls_data/blob/main/bls_logging/README.md

Configure logging parameters including log level, directory, file size, backup count, and output destinations. Call setup_logging with desired options.

```python
setup_logging(
    log_level="INFO",           # DEBUG, INFO, WARNING, ERROR, CRITICAL
    log_dir="logs",             # Directory for log files
    max_file_size=10*1024*1024, # 10MB file rotation
    backup_count=5,             # Number of backup files
    console_output=True,        # Output to console
    file_output=True           # Output to files
)
```

--------------------------------

### Populate Database with Sample Data

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Load sample CPI data into the database. This is a troubleshooting step for data issues.

```bash
make extract-cpi
```

--------------------------------

### Configure BLS API keys

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Sets up BLS API keys by creating a .env file in the project root. Keys must be prefixed with BLS_API_KEY_ and the library will randomly select one.

```env
BLS_API_KEY_0=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
BLS_API_KEY_1=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
```

--------------------------------

### Get Specific Series

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Retrieves details for a specific series by its ID.

```APIDOC
## GET /bls_series/{series_id}

### Description
Retrieves details for a specific series using its unique identifier.

### Method
GET

### Endpoint
/bls_series/{series_id}

#### Path Parameters
- **series_id** (string) - Required - The unique identifier of the series to retrieve.

### Response
#### Success Response (200)
- Returns the series object matching the provided ID.

#### Response Example
```json
{
  "series_id": "CUSR0000SA0",
  "series_title": "CPI All Urban Consumers - All Items SA",
  ...
}
```
```

--------------------------------

### Structured Logging Configuration

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Configure global logging with rotating file handlers and structured formatting using `setup_logging()`. Call this once at startup. Use `get_logger(__name__)` in modules for logging. Decorators `log_function_call` and `log_performance` automate logging for function calls and execution time.

```python
from bls_logging.config import setup_logging, get_logger, log_function_call, log_performance

# One-time global setup (call at application entry point)
setup_logging(
    log_level="INFO",
    log_dir="logs",          # creates logs/ directory automatically
    max_file_size=10_485_760,  # 10 MB before rotation
    backup_count=5,
    console_output=True,
    file_output=True,
)

log = get_logger(__name__)
log.info("Application started")
log.warning("API key missing, retrying...")
log.error("Database connection failed", exc_info=True)

# Decorator: auto-log every call with args/result
@log_function_call
def fetch_data(series_id: str) -> dict:
    return {}

# Decorator: log execution time
@log_performance
def expensive_parse(raw: dict):
    pass

```

--------------------------------

### API Error Handling

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Details on how the API communicates errors, including common status codes and example error responses.

```APIDOC
## Error Handling

The API returns detailed error information in JSON format.

### Example Error Response
```json
{
  "error": "Series not found",
  "detail": "Series with ID 'INVALID_ID' does not exist"
}
```

### Common HTTP Status Codes
- `200` - Success
- `201` - Created
- `400` - Bad Request
- `404` - Not Found
- `500` - Internal Server Error
- `503` - Service Unavailable (database connection issues)
```

--------------------------------

### Get data points for a specific series

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Retrieves data points for a given series ID, year, and an optional limit for the number of results.

```APIDOC
## GET /bls_data_points

### Description
Retrieves data points for a specific series, allowing filtering by year and limiting the number of results.

### Method
GET

### Endpoint
/bls_data_points

### Parameters
#### Query Parameters
- **series_id** (string) - Required - The unique identifier for the data series.
- **year** (integer) - Required - The year for which to retrieve data.
- **limit** (integer) - Optional - The maximum number of data points to return.

### Request Example
```bash
curl "http://localhost:8000/bls_data_points?series_id=CUSR0000SA0&year=2023&limit=12"
```

### Response
#### Success Response (200)
- **data** (array) - An array of data points matching the query.

#### Response Example
```json
{
  "data": [
    {
      "period": "M01",
      "value": "320.743"
    },
    {
      "period": "M02",
      "value": "321.353"
    }
  ]
}
```
```

--------------------------------

### Verify Python Environment

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Check the Python version in the virtual environment. This is a troubleshooting step for API startup issues.

```bash
./venv/bin/python --version
```

--------------------------------

### Get BLS Data using Python Library

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Import and use the `get_bls_data` function for programmatic access to BLS data within Python scripts.

```python
from data_extraction.main import get_bls_data
```

--------------------------------

### Configure PostgreSQL Database Connection

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Initialize `DatabaseConfig` to establish a SQLAlchemy engine with connection pooling. It reads database credentials from environment variables. Use `db.create_tables()` to set up the database schema.

```python
from database.config import DatabaseConfig
from database.repository import BLSDataRepository

# Connect (reads DB_USER, DB_PASS, DB_HOST, DB_PORT, DB_NAME from env)
db = DatabaseConfig()

# Create all tables (idempotent)
db.create_tables()
print(db.check_connection())  # True

# Use the context-manager session for automatic commit/rollback
with db.get_session() as session:
    repo = BLSDataRepository(session)

    # Upsert series data (e.g., after fetching from BLS API)
    series_payload = [{
        "series_id": "CUSR0000SA0",
        "series_title": "CPI for All Urban Consumers - All Items SA",
        "survey_name": "Consumer Price Index",
        "area": "U.S. City Average",
        "item": "All Items",
        "seasonality": "Seasonally Adjusted",
        "data": [
            {"year": "2023", "period": "M12", "periodName": "December", "value": "306.746"},
        ],
    }]
    inserted, updated, skipped = repo.upsert_series_data(series_payload)
    print(f"Inserted: {inserted}, Updated: {updated}")

    # Query back with metadata
    df = repo.get_series_data(
        series_ids=["CUSR0000SA0"],
        start_year=2023,
        end_year=2023,
        include_metadata=True,
    )
    print(df[["series_id", "year", "period", "value"]].head())

    # Full-text search
    results = repo.search_series("consumer price index urban", limit=5)
    for r in results:
        print(r["series_id"], r["series_title"])

    # Find series not updated in the last 24 hours
    stale = repo.get_stale_series(max_age_hours=24)
    print(f"Stale series count: {len(stale)}")
```

--------------------------------

### Test the API

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Runs the API's test suite using either the Makefile or a Python script.

```bash
# Run the test suite
make api-test
```

```bash
# Or manually
./venv/bin/python test_api.py
```

--------------------------------

### Database Configuration Environment Variables

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Configure the database connection using these environment variables. They can be set in a .env file or directly as environment variables.

```bash
DB_USER=postgres
DB_PASS=password
DB_HOST=localhost
DB_PORT=5432
DB_NAME=bls_data
```

--------------------------------

### Migrate from Print Statements to Logging

Source: https://github.com/kovashikawa/bls_data/blob/main/bls_logging/README.md

Replace `print()` statements with `log.info()` for better log management. Use string formatting for dynamic messages.

```python
# Old way
print("Processing data...")
print(f"Found {count} records")

# New way  
log.info("Processing data...")
log.info("Found %d records", count)
```

--------------------------------

### Filter BLS CPI Series IDs

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Use `get_cu_series_codes()` to retrieve BLS series IDs based on various filters like area and item codes. Call without arguments to get all series. This function requires the `cpi_series_master_list.csv` file.

```python
from cu_series.cu_series_codes import get_cu_series_codes

# U.S. city average, All items (seasonally adjusted) — typically 2 series
codes = get_cu_series_codes({"area_code": "0000", "item_code": "SA0"})
print(codes)
# ['CUSR0000SA0', 'CUUR0000SA0']

# All series for a specific metro area
nyc_codes = get_cu_series_codes({"area_code": "A103"})
print(len(nyc_codes), "series for New York-Newark-Jersey City")

# All urban Alaska CPI series
alaska_codes = get_cu_series_codes({"area_code": "S49G"})

# No filter — return every series in the master list
all_codes = get_cu_series_codes()
print(f"Total CPI series: {len(all_codes)}")  # e.g. 15000+

# Use with get_bls_data via the dynamic CU: prefix
from data_extraction.main import get_bls_data
df = get_bls_data(
    codes_or_ids=["CU:area_code=0000,item_code=SA0"],
    start_year=2022,
    end_year=2023,
)
```

--------------------------------

### Expose BLS Data as an MCP Server

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Set up an MCP server to expose BLS data retrieval and CPI seasonality analysis as tool calls for AI assistants.

```python
MCP server interface
```

--------------------------------

### Fetch BLS Data via Command Line

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Use the main script from the command line to fetch BLS data and save it to a CSV file. Supports specifying series, date ranges, and output files. Can be run with `uv` for environment management or directly with `python -m`.

```bash
# Using UV (recommended)
uv run python -m bls_data.data_extraction.main \
  cpi_all_items ces_all_employees unemployment_rate \
  --start 2018 --end 2023 \
  --catalog \
  --out data/bls_data.csv
```

```bash
# Or using traditional pip
python -m bls_data.data_extraction.main \
  cpi_all_items ces_all_employees unemployment_rate \
  --start 2018 --end 2023 \
  --catalog \
  --out data/bls_data.csv
```

--------------------------------

### CLI: Fetch Data with Options

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Command-line interface for fetching BLS data. Supports series IDs/aliases, date ranges, BLS API options (catalog, calculations, annual average), and CSV export.

```bash
# Set up .env
echo "BLS_API_KEY_0=your_key_here" > .env

# Fetch three aliases for 2020-2023, include catalog metadata, save to CSV
python -m data_extraction.main \
  cpi_all_items ces_all_employees unemployment_rate \
  --start 2020 --end 2023 \
  --catalog \
  --out output/bls_data.csv

# Include BLS-computed calculations and annual averages
python -m data_extraction.main \
  CUSR0000SA0 \
  --start 2015 --end 2023 \
  --calculations \
  --annualaverage \
  --out output/cpi_with_calcs.csv
```

--------------------------------

### Run API Tests

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Execute the API's test suite to ensure everything is functioning correctly. This command is part of the development workflow.

```bash
make api-test
```

--------------------------------

### Enable Debug Logging

Source: https://github.com/kovashikawa/bls_data/blob/main/bls_logging/README.md

Enable debug logging to see detailed information. Ensure the log level is set appropriately.

```python
setup_logging(log_level="DEBUG")
```

--------------------------------

### Using Different Log Levels

Source: https://github.com/kovashikawa/bls_data/blob/main/bls_logging/README.md

Employ appropriate log levels for different types of messages, from detailed debugging information to critical system failures. Use string formatting for dynamic values.

```python
log.debug("Variable value: %s", variable)  # Detailed debugging
log.info("Processing %d records", count)   # General information
log.warning("API rate limit approaching")  # Potential issues
log.error("Database connection failed")    # Errors
log.critical("System out of memory")       # Critical failures
```

--------------------------------

### Code Quality Checks and Formatting

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Maintain code quality by running linting and formatting checks. Use 'make ruff-fix' to automatically resolve formatting issues.

```bash
# Run linting
make ruff-check

# Run formatting
make ruff-format

# Auto-fix issues
make ruff-fix
```

--------------------------------

### Logging Exceptions with Tracebacks

Source: https://github.com/kovashikawa/bls_data/blob/main/bls_logging/README.md

Capture and log exceptions along with their full tracebacks using `exc_info=True` to aid in debugging. This should be done within an exception handler.

```python
try:
    risky_operation()
except Exception as e:
    log.error("Operation failed: %s", e, exc_info=True)
```

--------------------------------

### API Key Rotation with .env

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Loads BLS API keys from environment variables prefixed with 'BLS_API_KEY_' in a .env file for automatic rotation. The BLSClient uses this by default if no key is explicitly provided.

```python
# .env file (project root):
# BLS_API_KEY_0=aaaabbbbccccddddeeeeffffgggghhh0
# BLS_API_KEY_1=aaaabbbbccccddddeeeeffffgggghhh1

from data_extraction.api_key import get_random_bls_key

key = get_random_bls_key()
print(key)  # randomly selected from the two keys above

# Passing the key explicitly to BLSClient
from data_extraction.bls_client import BLSClient
client = BLSClient(api_key=get_random_bls_key())
```

--------------------------------

### Instantiate BLSClient for Direct API Access

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Instantiate `BLSClient` directly for lower-level control over API requests. This is useful when you need to manage session, retry logic, and request chunking manually. API key can be provided directly or loaded from `.env`.

```python
from data_extraction.bls_client import BLSClient

# Direct client usage — useful when you need to handle raw API responses
client = BLSClient(
    api_key="your_bls_api_key",   # or omit to load from .env
    series_limit=50,               # series per API call (v2 max)
    years_limit=20,                # years per API call (v2 max)
)
```

--------------------------------

### Fetch BLS Data with Database Caching

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Demonstrates using `get_bls_data` with `use_database=True` for caching. The first call fetches from the API and stores data in PostgreSQL, while subsequent calls served from the cache are significantly faster. Requires database credentials in `.env`.

```python
from data_extraction.main import get_bls_data

# First call — fetches from API and stores in PostgreSQL
# Requires .env with DB_USER, DB_PASS, DB_HOST, DB_PORT, DB_NAME
df_first = get_bls_data(
    codes_or_ids=["cpi_all_items", "unemployment_rate"],
    start_year=2020,
    end_year=2023,
    catalog=True,
    use_database=True,
    use_cache=False,     # bypass cache on first call
    force_refresh=True,
)
print(f"Fetched from API: {df_first.shape}")  # e.g. (96, 14)

# Second call — served from database cache (significantly faster)
df_cached = get_bls_data(
    codes_or_ids=["cpi_all_items", "unemployment_rate"],
    start_year=2020,
    end_year=2023,
    catalog=True,
    use_database=True,
    use_cache=True,
    force_refresh=False,
)
print(f"Fetched from cache: {df_cached.shape}")  # (96, 14)
print(f"Data identical: {df_first.equals(df_cached)}")  # True
```

--------------------------------

### AI-assistant Integration Tools

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Tools exposed via FastMCP for AI assistants to interact with BLS data.

```APIDOC
## MCP Tools

### Description
These tools are exposed via `FastMCP` for AI assistants to query BLS data, retrieve series metadata, and perform analysis.

### Tools
1.  **get_series**
    *   **Description**: Fetches a series by ID with an optional date range.
    *   **Parameters**: `series_id` (string), `start` (string, optional), `end` (string, optional)
    *   **Response Example**:
        ```json
        {
          "series_id": "CUUR0000SA0",
          "data": [
            {"series_id": "CUUR0000SA0", "year": 2023, "period": "M01", "value": 299.17, ...},
            ...
          ],
          "shape": [12, 14],
          "columns": ["series_id", "alias", "year", "period", "period_name", "value", ...]
        }
        ```

2.  **get_series_info**
    *   **Description**: Fetches catalog metadata for a series.
    *   **Parameters**: `series_id` (string)

3.  **search_series**
    *   **Description**: Searches series titles (requires DB for full results).
    *   **Parameters**: `query` (string)

4.  **analyze_cpi_seasonality**
    *   **Description**: Computes Month-over-Month percentile bands and generates a plot for CPI seasonality analysis.
    *   **Parameters**: `series_id` (string), `start` (string, optional), `end` (string, optional)
    *   **Response Example**:
        ```json
        {
          "series_id": "CUUR0000SA0",
          "table": [
            {"month": 1, "month_name": "Jan", "p25": -0.12, "p50": 0.03, "p75": 0.21, "current": 0.51},
            ...
          ],
          "image_base64": "<base64-encoded PNG>",
          "summary_stats": {
            "analysis_period": "2013-2023",
            "historical_data_points": 120,
            "avg_historical_mom": 0.182,
            "std_historical_mom": 0.241
          }
        }
        ```
```

--------------------------------

### get_bls_data() with database caching

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Demonstrates how to use `get_bls_data()` with the `use_database=True` flag for efficient data retrieval and caching via PostgreSQL.

```APIDOC
## get_bls_data() with database caching

### Description
When `use_database=True` the function routes through `BLSClient.fetch_with_database()`, which checks PostgreSQL for fresh data (default staleness threshold: 24 hours) before hitting the BLS API. Results are automatically stored back to the database. The function falls back gracefully to the pure-API path if PostgreSQL is unavailable.

### Parameters
- **codes_or_ids** (list[str]) - Required - A list of BLS series IDs or human-readable aliases.
- **start_year** (int) - Required - The starting year for the data request.
- **end_year** (int) - Required - The ending year for the data request.
- **catalog** (bool) - Optional - If True, include catalog metadata.
- **use_database** (bool) - Required - Set to `True` to enable database caching.
- **use_cache** (bool) - Optional - If True, serve data from the cache if available.
- **force_refresh** (bool) - Optional - If True, bypass the cache and force a refresh from the API.

### Request Example
```python
from data_extraction.main import get_bls_data

# First call — fetches from API and stores in PostgreSQL
# Requires .env with DB_USER, DB_PASS, DB_HOST, DB_PORT, DB_NAME
df_first = get_bls_data(
    codes_or_ids=["cpi_all_items", "unemployment_rate"],
    start_year=2020,
    end_year=2023,
    catalog=True,
    use_database=True,
    use_cache=False,     # bypass cache on first call
    force_refresh=True,
)
print(f"Fetched from API: {df_first.shape}")  # e.g. (96, 14)

# Second call — served from database cache (significantly faster)
df_cached = get_bls_data(
    codes_or_ids=["cpi_all_items", "unemployment_rate"],
    start_year=2020,
    end_year=2023,
    catalog=True,
    use_database=True,
    use_cache=True,
    force_refresh=False,
)
print(f"Fetched from cache: {df_cached.shape}")  # (96, 14)
print(f"Data identical: {df_first.equals(df_cached)}")  # True
```

### Response
#### Success Response (200)
- **DataFrame** - A pandas DataFrame containing the requested BLS data, served either from the API or the database cache.
```

--------------------------------

### Advanced Logging: Function Call and Performance

Source: https://github.com/kovashikawa/bls_data/blob/main/bls_logging/README.md

Automatically log function calls and execution times using decorators. Ensure the necessary functions are imported.

```python
from bls_logging.config import log_function_call, log_performance

# Automatically log function calls
@log_function_call
def my_function(param1, param2):
    return result

# Automatically log execution time
@log_performance  
def slow_operation():
    # Your code here
    return result
```

--------------------------------

### Download and Merge BLS CPI Metadata

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Run this script to download the necessary BLS CPI metadata files (`cu.series`, `cu.item`, `cu.area`), merge them, and create the `cpi_series_master_list.csv` file. This is a prerequisite for `get_cu_series_codes()`.

```bash
# Download and merge CPI metadata (writes cpi_series_master_list.csv)
python -m cu_series.cu_download
```

--------------------------------

### Run BLS Data as a FastAPI Microservice

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Deploy the BLS data functionality as a FastAPI microservice for integration with other services or behind an API gateway.

```python
bls_api.py
```

--------------------------------

### Run Python script with uv

Source: https://github.com/kovashikawa/bls_data/blob/main/README.md

Executes a Python script within the uv environment. This command ensures the script runs with the correct dependencies managed by uv.

```bash
uv run python scripts/test_cpi_extraction.py
```

--------------------------------

### Access BLS Data using CLI

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Utilize the command-line interface to fetch BLS data, specifying series, date ranges, and output format.

```bash
bls-data cpi_all_items --start 2020 --end 2023 --out data.csv
```

--------------------------------

### get_bls_data() - High-level data fetch function

Source: https://context7.com/kovashikawa/bls_data/llms.txt

The primary entry point for the library, `get_bls_data()` fetches BLS data, resolves aliases, parses responses into pandas DataFrames, and supports various optional flags for metadata, calculations, and database caching.

```APIDOC
## get_bls_data() - High-level data fetch function

### Description
`get_bls_data()` in `data_extraction/main.py` is the primary entry point for the library. It resolves human-readable aliases (or raw series IDs) through the mapping layer, delegates to `BLSClient.fetch()`, parses the response with `parse_results_to_df()`, and returns a tidy pandas DataFrame. Optional flags expose all BLS v2 API extras (catalog metadata, net/percent calculations, annual averages, aspects) and an optional database caching layer.

### Parameters
- **codes_or_ids** (list[str]) - Required - A list of BLS series IDs or human-readable aliases.
- **start_year** (int) - Required - The starting year for the data request.
- **end_year** (int) - Required - The ending year for the data request.
- **catalog** (bool) - Optional - If True, include catalog metadata (survey name, area, item, seasonality).
- **calculations** (bool) - Optional - If True, include net/percent change columns.
- **annualaverage** (bool) - Optional - If True, include M13 annual average rows.
- **use_database** (bool) - Optional - If True, use the PostgreSQL database caching layer.
- **use_cache** (bool) - Optional - If True, use the database cache (requires `use_database=True`).
- **force_refresh** (bool) - Optional - If True, force a refresh from the API even if data is cached.

### Request Example
```python
from data_extraction.main import get_bls_data

# Fetch three economic series by alias for 2018-2023 with catalog metadata
df = get_bls_data(
    codes_or_ids=["cpi_all_items", "ces_all_employees", "unemployment_rate"],
    start_year=2018,
    end_year=2023,
    catalog=True,
    calculations=False,
    annualaverage=False,
    use_database=False,
)

print(df.columns.tolist())
# Expected output: ['series_id', 'alias', 'year', 'period', 'period_name', 'value', 'latest',
#  'seasonality', 'series_title', 'survey_name', 'measure_data_type',
#  'area', 'item', 'footnotes']

print(df[df["alias"] == "cpi_all_items"].tail(3).to_string(index=False))
# Expected output: 
# series_id          alias  year period period_name   value  ...
# CUSR0000SA0  cpi_all_items  2023    M10    October  307.671
# CUSR0000SA0  cpi_all_items  2023    M11   November  307.051
# CUSR0000SA0  cpi_all_items  2023    M12   December  306.746
```

### Response
#### Success Response (200)
- **DataFrame** - A pandas DataFrame containing the requested BLS data with columns including 'series_id', 'alias', 'year', 'period', 'value', 'catalog' metadata, etc.
```

--------------------------------

### Create a New Series

Source: https://github.com/kovashikawa/bls_data/blob/main/API_README.md

Creates a new series entry in the BLS database via the API. Requires specifying all required fields in the JSON payload and setting the Content-Type header.

```bash
curl -X POST "http://localhost:8000/bls_series" \
  -H "Content-Type: application/json" \
  -d '{
    "series_id": "CUSR0000SA0",
    "series_title": "Consumer Price Index for All Urban Consumers",
    "survey_name": "Consumer Price Index",
    "area": "U.S. City Average",
    "item": "All Items",
    "seasonality": "Seasonally Adjusted",
    "data_frequency": "monthly"
  }'
```

--------------------------------

### BLSClient - Direct API client

Source: https://context7.com/kovashikawa/bls_data/llms.txt

The `BLSClient` class provides a lower-level interface for interacting with the BLS API, offering direct control over session management, retry logic, and request chunking.

```APIDOC
## BLSClient — Direct API client

### Description
`BLSClient` in `data_extraction/bls_client.py` is a `dataclass` that encapsulates session management, retry logic (5 attempts, backoff factor 1.2, retries on 429/500/502/503/504), automatic chunking of large series lists (≤50 per request) and wide date ranges (≤20 years per request), and optional PostgreSQL integration. Instantiate directly for lower-level control.

### Parameters
- **api_key** (str) - Optional - Your BLS API key. If omitted, it attempts to load from the `.env` file.
- **series_limit** (int) - Optional - The maximum number of series to request per API call (default is 50, the v2 API max).
- **years_limit** (int) - Optional - The maximum number of years to request per API call (default is 20, the v2 API max).

### Request Example
```python
from data_extraction.bls_client import BLSClient

# Direct client usage — useful when you need to handle raw API responses
client = BLSClient(
    api_key="your_bls_api_key",   # or omit to load from .env
    series_limit=50,               # series per API call (v2 max)
    years_limit=20,                # years per API call (v2 max)
)

# Note: The BLSClient class itself does not have a direct 'fetch' method exposed in this documentation snippet.
# Its methods like 'fetch()' and 'fetch_with_database()' are typically called by higher-level functions like get_bls_data().
```

### Response
#### Success Response (200)
- **BLSClient Instance** - An initialized `BLSClient` object ready for making API requests.
```

--------------------------------

### load_mapping() / resolve_series_ids()

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Loads alias mappings and resolves human-readable tokens or aliases to BLS series IDs.

```APIDOC
## `load_mapping()` / `resolve_series_ids()` — Alias mapping

### Description
`load_mapping()` discovers and parses a mapping file (CSV or JSON) to create an alias mapping. `resolve_series_ids()` then uses this mapping to convert human-readable tokens (aliases or raw series IDs) into BLS series ID lists. It also supports dynamic `CU:` prefixes for querying the CPI master CSV directly.

### Methods
- `load_mapping(path: str = None)`: Discovers and parses a mapping file. If `path` is not provided, it searches for conventional filenames like `code_mapping.csv`.
- `resolve_series_ids(codes_or_ids: list[str], mapping: dict)`: Converts a list of codes or IDs into a list of BLS series IDs and a reverse mapping.

### Usage
```python
from data_extraction.mapping_loader import load_mapping, resolve_series_ids

# Load mapping from default location
mapping = load_mapping()
# Or supply a custom path:
# mapping = load_mapping("my_project/series_map.json")

# Resolve mixed list of aliases and raw series IDs
series_ids, reverse_map = resolve_series_ids(
    codes_or_ids=[
        "cpi_all_items",          # alias → looks up CSV
        "LNS14000000",            # raw BLS series ID — passed through
        "CU:area_code=0000,item_code=SA0",  # dynamic CPI filter
    ],
    mapping=mapping,
)
print(series_ids)
# Example output: ['CUSR0000SA0', 'LNS14000000', 'CUSR0000SA0', 'CUSR0000AA0', ...]

print(reverse_map["CUSR0000SA0"])
# Example output: ['cpi_all_items', 'CU:area_code=0000,item_code=SA0']
```

### Mapping File Format Example (`code_mapping.csv`)
```csv
alias,series_id
cpi_all_items,CUSR0000SA0
ces_all_employees,CES0000000001
unemployment_rate,LNS14000000
```
```

--------------------------------

### Force Cache Refresh for Data Extraction

Source: https://context7.com/kovashikawa/bls_data/llms.txt

Execute this command to extract CPI data and force a refresh of the database cache. This is useful when the cache may be outdated or corrupted.

```bash
python -m data_extraction.main \
  cpi_all_items \
  --start 2020 --end 2023 \
  --use-database \
  --force-refresh
```