StatsCan Python API

https://stats-can.readthedocs.io/en/latest/
Admin
This library implements most of the functions defined by the Statistics Canada Web Data Service,...
Tokens:10,362
Snippets:83
Trust Score:9.6
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install with Pip

Source: https://stats-can.readthedocs.io/en/latest?badge=latest

Install the stats-can package using pip.

```bash
pip install stats-can

```

--------------------------------

### Install with Conda

Source: https://stats-can.readthedocs.io/en/latest?badge=latest

Install the stats-can package using conda from the conda-forge channel.

```bash
conda install -c conda-forge stats_can

```

--------------------------------

### Get Bulk Vector Data by Date Range

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Fetches bulk vector data within a specified start and end release date. It supports chunking of vectors and makes POST requests to the API.

```python
def get_bulk_vector_data_by_range(
    vectors: str | list[str], start_release_date: dt.date, end_release_date: dt.date
) -> list[VectorData]:
    """[api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a12-5)

    Parameters
    ----------
    vectors
        vector numbers to get info for
    start_release_date
        start release date for the data
    end_release_date
        end release date for the data

    Returns
    -------
    :
        List of dicts containing data for each vector
    """
    url = f"{SC_URL}getBulkVectorDataByRange"
    start_release_date_str = str(start_release_date) + "T13:00"
    end_release_date_str = str(end_release_date) + "T13:00"
    chunks = chunk_vectors(vectors)
    final_list = []
    for i, vector_ids in enumerate(chunks):
        if i > 0:
            time.sleep(_CHUNK_DELAY)
        result = _fetch_and_validate(
            url,
            schema=VectorData,
            method="POST",
            json={
                "vectorIds": vector_ids,
                "startDataPointReleaseDate": start_release_date_str,
                "endDataPointReleaseDate": end_release_date_str,
            },
        )
        final_list += result
    return final_list
```

--------------------------------

### Get Cube Metadata (Python)

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Fetches metadata for specified tables using a POST request. Accepts table IDs as a string or a list.

```python
def get_cube_metadata(tables: str | list[str]) -> list[CubeMetadata]:
    """[api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a11-1)

    Take a list of tables and return a list of dictionaries with their
    metadata

    Parameters
    ----------
    tables
        IDs of tables to get metadata for

    Returns
    -------
    :
        one for each table with its metadata
    """
    tables = parse_tables(tables)
    tables_json = [{"productId": t} for t in tables]
    url = f"{SC_URL}getCubeMetadata"
    return _fetch_and_validate(
        url, schema=CubeMetadata, method="POST", json=tables_json
    )

```

--------------------------------

### get_changed_cube_list(date)

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Gets a list of changed cubes for a specified date. Defaults to the current date.

```APIDOC
## GET /getChangedCubeList/{date}

### Description
Gets a list of changed cubes, one for each table and when it was updated.

### Method
GET

### Endpoint
/getChangedCubeList/{date}

### Parameters
#### Path Parameters
- **date** (date | None) - Date to check for table changes, defaults to current date

### Request Example
None

### Response
#### Success Response (200)
- **list[ChangedCube]** - A list of changed cubes, one for each table and when it was updated.

#### Response Example
None
```

--------------------------------

### Get Full Table Download URL

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Generates a URL to download a full table in either CSV or SDMX format. The table name is parsed before constructing the URL.

```python
def get_full_table_download(table: str, csv: bool = True) -> str:
    """Take a table name and return a url to a zipped file of that table.

    [api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a12-6)
    [api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a12-7)


    Parameters
    ----------
    table
        table name to download
    csv
        download in CSV format, if not download SDMX

    Returns
    -------
    :
        path to the file download
    """
    parsed_table = parse_tables(table)[0]
    if csv:
        url = f"{SC_URL}getFullTableDownloadCSV/{parsed_table}/en"
    else:
        url = f"{SC_URL}getFullTableDownloadSDMX/{parsed_table}"
    return _fetch_and_validate(url, schema=str)
```

--------------------------------

### Get Data (Cube PID Coord, Latest N Periods)

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Placeholder function for retrieving data using cube PID, coordinates, and the latest N periods. This function is not yet implemented.

```python
def get_data_from_cube_pid_coord_and_latest_n_periods():
    """Not implemented yet

    [api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a12-3)
    """
    pass
```

--------------------------------

### Placeholder for Get Series Info from Cube PID Coord (Python)

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

This function is a placeholder and is not yet implemented. It is intended to retrieve series information based on cube PID coordinates.

```python
def get_series_info_from_cube_pid_coord():
    """Not implemented yet

    [api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a11-2)
    """
    pass

```

--------------------------------

### Get Changed Series List (Python)

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves a list of all series updated today. Returns an empty list if no series were released.

```python
def get_changed_series_list() -> list[ChangedSeries]:
    """[api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a10-1)

    Gets all series that were updated today.

    Returns
    -------
    :
        list of changed series, one for each vector and when it was released.
        Returns an empty list if no series have been released yet today.
    """
    try:
        return _fetch_and_validate(
            url=f"{SC_URL}getChangedSeriesList",
            schema=list[ChangedSeries],
        )
    except requests.HTTPError as exc:
        # The API returns 409 when no series have been released yet today,
        # which is a normal condition, not an error.
        if exc.response is not None and exc.response.status_code == 409:
            return []
        raise

```

--------------------------------

### Get Code Sets with Caching

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves all code sets, which provide descriptive information for data. Results are cached after the first call to improve performance. Use `get_code_sets.cache_clear()` to refresh.

```python
@functools.lru_cache(maxsize=1)
def get_code_sets() -> CodeSet:
    """[api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a13-1)

    Gets all code sets which provide additional information to describe
    information and are grouped into scales, frequencies, symbols etc.

    Results are cached after the first call. Call
    ``get_code_sets.cache_clear()`` to force a refresh.

    Returns
    -------
    :
        one dictionary for each group of information
    """
    url = f"{SC_URL}getCodeSets"
    return _fetch_and_validate(url, schema=CodeSet)
```

--------------------------------

### Get Changed Series Data (Cube PID Coord)

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Placeholder function for retrieving changed series data using cube PID and coordinates. This function is not yet implemented.

```python
def get_changed_series_data_from_cube_pid_coord():
    """Not implemented yet

    [api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a12-1)
    """
    pass
```

--------------------------------

### Get Changed Series Data (Vector)

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Placeholder function for retrieving changed series data using vectors. This function is not yet implemented.

```python
def get_changed_series_data_from_vector():
    """Not implemented yet

    [api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a12-2)
    """
    pass
```

--------------------------------

### Get Series Info from Vector

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves metadata for one or more series vectors. Handles chunking of large vector lists and includes a delay between requests.

```python
def get_series_info_from_vector(vectors: str | list[str]) -> list[SeriesInfo]:
    """[api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a11-3)

    Parameters
    ----------
    vectors
        vector numbers to get info for

    Returns
    -------
    :
        List of dicts containing metadata for each v#
    """
    url = f"{SC_URL}getSeriesInfoFromVector"
    chunks = chunk_vectors(vectors)
    final_list = []
    for i, chunk in enumerate(chunks):
        if i > 0:
            time.sleep(_CHUNK_DELAY)
        vector_dict = [{"vectorId": v} for v in chunk]
        result = _fetch_and_validate(
            url, schema=SeriesInfo, method="POST", json=vector_dict
        )
        final_list += result
    return final_list
```

--------------------------------

### Get Changed Cube List (Python)

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves a list of cubes updated on a specific date, defaulting to the current date. Useful for tracking table updates.

```python
def get_changed_cube_list(date: dt.date | None = None) -> list[ChangedCube]:
    """[api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a10-2)

    Parameters
    ----------
    date
        Date to check for table changes, defaults to current date

    Returns
    -------
    :
        list of changed cubes, one for each table and when it was updated
    """
    if date is None:
        date = dt.date.today()
    return _fetch_and_validate(
        url=f"{SC_URL}getChangedCubeList/{date}", schema=list[ChangedCube]
    )

```

--------------------------------

### get_changed_series_list()

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Gets all series that were updated today. Returns an empty list if no series have been released yet today.

```APIDOC
## GET /getChangedSeriesList

### Description
Gets all series that were updated today.

### Method
GET

### Endpoint
/getChangedSeriesList

### Parameters
None

### Request Example
None

### Response
#### Success Response (200)
- **list[ChangedSeries]** - A list of changed series, one for each vector and when it was released.

#### Response Example
None

#### Error Response (409)
- An empty list is returned if no series have been released yet today.
```

--------------------------------

### Get Bulk Vector Data by Reference Period Range

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves bulk vector data for specified vector IDs within a given date range. Handles chunking of vectors and includes a delay between requests.

```python
def get_bulk_vector_data_by_reference_period_range(
    vectors: str | list[str], start_ref_date: dt.date, end_ref_date: dt.date
) -> list[VectorData]:
    """[api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a12-5a)

    Parameters
    ----------
    vectors
        vector numbers to get info for
    start_ref_date
        start reference period date for the data
    end_ref_date
        end reference period date for the data

    Returns
    -------
    :
        List of dicts containing data for each vector
    """
    url = f"{SC_URL}getDataFromVectorByReferencePeriodRange"
    chunks = chunk_vectors(vectors)
    final_list = []
    for i, vector_ids in enumerate(chunks):
        if i > 0:
            time.sleep(_CHUNK_DELAY)
        # I know the rest are .post, they changed it just for this one
        v_string = ",".join(f"{v}" for v in vector_ids)
        vector_param = f"vectorIds={v_string}"
        full_url = f"{url}?{vector_param}&startRefPeriod={start_ref_date}&endReferencePeriod={end_ref_date}"
        result = _fetch_and_validate(url=full_url, schema=VectorData)
        final_list += result
    return final_list
```

--------------------------------

### get_data_from_vectors_and_latest_n_periods

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves data for specified vectors for the latest N periods. This function is useful for getting recent data points for one or more vectors.

```APIDOC
## get_data_from_vectors_and_latest_n_periods

### Description
Retrieves data for specified vectors for the latest N periods. This function is useful for getting recent data points for one or more vectors.

### Method
POST

### Endpoint
/getDataFromVectorsAndLatestNPeriods

### Parameters
#### Request Body
- **vectors** (str | list[str]) - Required - vector numbers to get info for
- **periods** (int) - Required - number of periods (starting at latest) to retrieve data for

### Request Example
```json
{
  "vectors": ["101056", "101057"],
  "periods": 5
}
```

### Response
#### Success Response (200)
- **VectorData** (list[dict]) - List of dicts containing data for each vector
```

--------------------------------

### Get Tables for Vectors in StatsCan

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Maps vector IDs to their corresponding table IDs. It also returns a list of all unique tables used by the provided vectors.

```python
def get_tables_for_vectors(
    vectors: str | list[str],
) -> dict[str | int, str | list[str]]:
    """Get a list of dicts mapping vectors to tables.

    Parameters
    ----------
    vectors
        Vectors to find tables for

    Returns
    -------
    :
        keys for each vector number return the table, plus a key for
        'all_tables' that has a list of unique tables used by vectors
    """
    v_json = get_series_info_from_vector(vectors)
    clean_vectors = [j["vectorId"] for j in v_json]
    tables_list: dict[int | str, str | list[str]] = {
        j["vectorId"]: str(j["productId"]) for j in v_json
    }
    all_tables = {tables_list[vector] for vector in clean_vectors}
    tables_list["all_tables"] = list(all_tables)
    return tables_list
```

--------------------------------

### Get Latest N Periods of Vector Data

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves the latest N periods of data for specified vectors. Handles vector chunking and includes a delay between chunks.

```python
def get_data_from_vectors_and_latest_n_periods(
    vectors: str | list[str], periods: int
) -> list[VectorData]:
    """[api reference](https://www.statcan.gc.ca/eng/developers/wds/user-guide#a12-4)

    Parameters
    ----------
    vectors
        vector numbers to get info for
    periods
        number of periods (starting at latest) to retrieve data for

    Returns
    -------
    :
        List of dicts containing data for each vector
    """
    url = f"{SC_URL}getDataFromVectorsAndLatestNPeriods"
    chunks = chunk_vectors(vectors)
    final_list = []
    for i, chunk in enumerate(chunks):
        if i > 0:
            time.sleep(_CHUNK_DELAY)
        periods_l = [periods for i in range(len(chunk))]
        json = [{"vectorId": v, "latestN": n} for v, n in zip(chunk, periods_l)]
        result = _fetch_and_validate(url, schema=VectorData, method="POST", json=json)
        final_list += result
    return final_list
```

--------------------------------

### Get Table Subsets from Vectors in StatsCan

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Maps table IDs to a list of vectors associated with each table. This function utilizes `get_tables_for_vectors` to first determine the table-vector relationships.

```python
def table_subsets_from_vectors(vectors: str | list[str]) -> dict[str, list[str]]:
    """Get a list of dicts mapping tables to vectors.

    Parameters
    ----------
    vectors
        Vectors to find tables for

    Returns
    -------
    :
        keys for each table used by the vectors, matched to a list of vectors
    """
    start_tables_dict = get_tables_for_vectors(vectors)
    tables_dict = {t: [] for t in start_tables_dict["all_tables"]}
    vecs = list(start_tables_dict.keys())[:-1]  # all but the all_tables key
    for vec in vecs:
        tables_dict[start_tables_dict[vec]].append(vec)
    return tables_dict
```

--------------------------------

### Convert Vector Data to DataFrame

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Retrieves vector data and converts it into a pandas DataFrame. Use this function to get historical data for specified vectors over a number of periods or a date range.

```python
def vectors_to_df(
    vectors: str | list[str],
    periods: int = 1,
    start_release_date: dt.date | None = None,
    end_release_date: dt.date | None = None,
) -> pd.DataFrame:
    """Get DataFrame of vectors with n periods data or over range of release dates.

    Wrapper on get_bulk_vector_data_by_range and
    get_data_from_vectors_and_latest_n_periods function to turn the resulting
    list of JSONs into a DataFrame

    Parameters
    ----------
    vectors
        vector numbers to get info for
    periods
        number of periods to retrieve data for
    start_release_date
        start release date for the data
    end_release_date
        end release date for the data

    Returns
    -------
    :
        vectors as columns and ref_date as the index (not release date)
    """
    df = pd.DataFrame()
    if (end_release_date is None) | (start_release_date is None):
        start_list = get_data_from_vectors_and_latest_n_periods(vectors, periods)
    else:
        start_list = get_bulk_vector_data_by_range(
            vectors, start_release_date, end_release_date
        )
    for vec in start_list:
        name = "v" + str(vec["vectorId"])
        # If there's no data for the series just skip it
        if not vec["vectorDataPoint"]:
            continue
        ser = (
            pd.DataFrame(vec["vectorDataPoint"])
            .assign(
                refPer=lambda x: pd.to_datetime(
                    x["refPer"], format="%Y-%m-%d", errors="coerce"
                )
            )
            .set_index("refPer")
            .rename(columns={"value": name})
            .filter([name])
        )
        df = pd.concat([df, ser], axis=1, sort=True)
    return df

```

--------------------------------

### download_tables

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Download a JSON file and a zip of data for a list of tables to a specified path. Supports both CSV and SDMX formats.

```APIDOC
## download_tables

### Description
Download a json file and zip of data for a list of tables to path.

### Parameters
#### Path Parameters
- **tables** (str | list[str]) - Required - tables to be downloaded
- **path** (Path | None) - Optional - Where to download the table and json. Defaults to None.
- **csv** (bool) - Optional - download in CSV format, if not download SDMX. Defaults to True.

### Returns
- **list[int]** - list of tables that were downloaded
```

--------------------------------

### get_full_table_download

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Downloads a full table in either CSV or SDMX format. Users can specify the table name and the desired output format.

```APIDOC
## get_full_table_download

### Description
Downloads a full table in either CSV or SDMX format. Users can specify the table name and the desired output format.

### Method Signature
`get_full_table_download(table: str, csv: bool = True) -> str`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **table** (str) - Required - table name to download
- **csv** (bool) - Optional - download in CSV format, if not download SDMX. Defaults to True.

### Request Example
```python
# Example usage (assuming necessary imports and setup)
# table_url_csv = scwds.get_full_table_download(table='10100001')
# table_url_sdmx = scwds.get_full_table_download(table='10100001', csv=False)
```

### Response
#### Success Response (200)
- **str** - Path to the file download.

#### Response Example
```json
{
  "example": "https://www12.statcan.gc.ca/ல்களில்/data/table_10100001.zip"
}
```
```

--------------------------------

### List Available Zipped StatsCan Tables

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Lists all available StatsCan tables in JSON format from a specified directory. Defaults to the current working directory if no path is provided. Handles potential errors during JSON file reading.

```python
import pathlib
import json
import logging

logger = logging.getLogger(__name__)

def list_zipped_tables(path: pathlib.Path | None = None) -> list[str]:
    """List StatsCan tables available.

    defaults to looking in the current working directory and for zipped CSVs

    Parameters
    ----------
    path
        Where to look for zipped tables, defaults to current working directory

    Returns
    -------
    :
        list of available tables json data
    """
    # Find json files
    path = pathlib.Path(path) if path else pathlib.Path.cwd()
    jsons = path.glob("*.json")
    tables = []
    for j in jsons:
        try:
            with open(j) as json_file:
                result = json.load(json_file)
                if "productId" in result:
                    tables.append(result)
        except ValueError:
            logger.warning("failed to read json file %s", j)
    return tables
```

--------------------------------

### Download Stats Canada Tables

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Downloads specified tables from Stats Canada, either as CSV or SDMX format, and saves them to a given path. Includes progress bar for downloads.

```python
def download_tables(
    tables: str | list[str], path: pathlib.Path | None = None, csv: bool = True
) -> list[int]:
    """Download a json file and zip of data for a list of tables to path.

    Parameters
    ----------
    tables
        tables to be downloaded
    path
        Where to download the table and json
    csv
        download in CSV format, if not download SDMX

    Returns
    -------
    :
        list of tables that were downloaded
    """
    dl_path = pathlib.Path(path) if path else pathlib.Path()
    metas = get_cube_metadata(tables)
    for meta in metas:
        product_id = meta["productId"]
        zip_url = get_full_table_download(product_id, csv=csv)
        zip_file_name = f"{product_id}{'-eng' if csv else ''}.zip"
        json_file_name = f"{product_id}.json"
        zip_file = dl_path / zip_file_name
        json_file = dl_path / json_file_name

        # Thanks http://evanhahn.com/python-requests-library-useragent/
        response = _session.get(zip_url, stream=True, timeout=120)
        response.raise_for_status()

        progress_bar = tqdm(
            desc=zip_file_name,
            total=int(response.headers.get("content-length", 0)),
            unit="B",
            unit_scale=True,
        )

        # Thanks https://bit.ly/2sPYPYw
        with open(json_file, "w") as outfile:
            json.dump(meta, outfile)
        with open(zip_file, "wb") as handle:
            for chunk in response.iter_content(chunk_size=512):
                if chunk:  # filter out keep-alive new chunks
                    handle.write(chunk)
                    progress_bar.update(len(chunk)) 
        progress_bar.close()
    return [meta["productId"] for meta in metas]
```

--------------------------------

### get_code_sets

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves all code sets, which provide additional information to describe data, grouped into categories like scales, frequencies, and symbols. Results are cached after the first call.

```APIDOC
## get_code_sets

### Description
Retrieves all code sets, which provide additional information to describe data, grouped into categories like scales, frequencies, and symbols. Results are cached after the first call.

### Method Signature
`get_code_sets() -> CodeSet`

### Parameters
None

### Request Example
```python
# Example usage (assuming necessary imports and setup)
# code_sets = scwds.get_code_sets()
# To clear the cache and force a refresh:
# scwds.get_code_sets.cache_clear()
```

### Response
#### Success Response (200)
- **CodeSet** (dict) - A dictionary where each key represents a group of information (e.g., scales, frequencies, symbols) and the value contains the details for that group.

#### Response Example
```json
{
  "example": "{\"ScaleGroup\": {\"1\": \"1000s\", \"2\": \"millions\"}, \"FrequencyGroup\": {\"M\": \"Monthly\"}}"
}
```
```

--------------------------------

### Retrieve All Code Sets as DataFrame Dictionary

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Fetches all available code sets from the API and organizes them into a dictionary where keys are code set types and values are pandas DataFrames. This is useful for accessing metadata like scales, frequencies, and symbols.

```python
def code_sets_to_df_dict() -> dict[str, pd.DataFrame]:
    """Get all code sets.

    Code sets provide additional metadata to describe
    information. Code sets are grouped into scales, frequencies, symbols etc.
    and returned as dictionary of dataframes.

    Returns
    -------
    :
        dictionary of dataframes
    """
    codes = get_code_sets()
    # Packs each code group in a dataframe for better lookup via dictionary
    codes_df_lookup = {key: pd.DataFrame(codes[key]) for key in codes.keys()}
    return codes_df_lookup

```

--------------------------------

### code_sets_to_df_dict

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Retrieves all available code sets and returns them as a dictionary of pandas DataFrames.

```APIDOC
## code_sets_to_df_dict

### Description
Get all code sets. Code sets provide additional metadata to describe information. Code sets are grouped into scales, frequencies, symbols etc. and returned as dictionary of dataframes.

### Method
Not Applicable (Python function)

### Endpoint
Not Applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
None

### Request Example
```python
# Example usage:
# code_set_dict = code_sets_to_df_dict()
```

### Response
#### Success Response
- **dict[str, DataFrame]** - dictionary of dataframes, where keys are code set names (e.g., 'scales', 'frequencies') and values are pandas DataFrames containing the metadata.
```

--------------------------------

### Load StatsCan Table to DataFrame

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Reads a specified StatsCan table into a pandas DataFrame. If the table's zip file is not found locally, it will be downloaded. Ensure necessary libraries like pandas and pathlib are imported.

```python
import pandas as pd
import pathlib
import zipfile

def zip_table_to_dataframe(
    table: str, path: pathlib.Path | None = None
) -> pd.DataFrame:
    """Read a StatsCan table into a pandas DataFrame.

    If a zip file of the table does not exist in path, downloads it

    Parameters
    ----------
    table
        the table to load to dataframe from zipped csv
    path
        where to download the tables or load them, default will go to current working directory

    Returns
    -------
    :
        the table as a dataframe
    """
    path = pathlib.Path(path) if path else pathlib.Path()
    # Parse tables returns a list, can only do one table at a time here though
    table = parse_tables(table)[0]
    table_zip = table + "-eng.zip"
    table_zip = path / table_zip
    if not table_zip.is_file():
        download_tables([table], path)
    csv_file = table + ".csv"
    with zipfile.ZipFile(table_zip) as myzip:
        with myzip.open(csv_file) as myfile:
            col_names = pd.read_csv(myfile, nrows=0).columns
        # reopen the file or it misses the first row
        with myzip.open(csv_file) as myfile:
            types_dict = {"VALUE": float}
            types_dict.update({col: str for col in col_names if col not in types_dict})
            df = pd.read_csv(myfile, dtype=types_dict)

    possible_cats = [
        "GEO",
        "DGUID",
        "STATUS",
        "SYMBOL",
        "TERMINATED",
        "DECIMALS",
        "UOM",
        "UOM_ID",
        "SCALAR_FACTOR",
        "SCALAR_ID",
        "VECTOR",
        "COORDINATE",
        "Wages",
        "National Occupational Classification for Statistics (NOC-S)",
        "Supplementary unemployment rates",
        "Sex",
        "Age group",
        "Labour force characteristics",
        "Statistics",
        "Data type",
        "Job permanency",
        "Union coverage",
        "Educational attainment",
    ]
    actual_cats = [col for col in possible_cats if col in col_names]
    df[actual_cats] = df[actual_cats].astype("category")
    df["REF_DATE"] = pd.to_datetime(df["REF_DATE"], format="%Y-%m-%d", errors="coerce")
    return df
```

--------------------------------

### get_tables_for_vectors

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Retrieves a dictionary mapping vectors to their corresponding tables. It also includes a special key 'all_tables' that lists all unique tables used by the provided vectors.

```APIDOC
## `get_tables_for_vectors(vectors: str | list[str]) -> dict[str | int, str | list[str]]`

### Description
Get a list of dicts mapping vectors to tables.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Parameters
- **vectors** (str | list[str]) - Required - Vectors to find tables for

### Returns
- **dict[str | int, str | list[str]]** - keys for each vector number return the table, plus a key for 'all_tables' that has a list of unique tables used by vectors
```

--------------------------------

### list_zipped_tables

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Lists available StatsCan tables. It defaults to looking in the current working directory for zipped CSV files.

```APIDOC
## list_zipped_tables(path: pathlib.Path | None = None) -> list[str]

### Description
Lists StatsCan tables available. Defaults to looking in the current working directory and for zipped CSVs.

### Parameters
#### Path Parameters
- **path** (pathlib.Path | None) - Optional - Where to look for zipped tables, defaults to current working directory

### Returns
- **list[str]** - list of available tables json data
```

--------------------------------

### vectors_to_df

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Retrieves vector data and returns it as a pandas DataFrame. It can fetch data for a specified number of periods or within a date range.

```APIDOC
## vectors_to_df

### Description
Get DataFrame of vectors with n periods data or over range of release dates. Wrapper on get_bulk_vector_data_by_range and get_data_from_vectors_and_latest_n_periods function to turn the resulting list of JSONs into a DataFrame.

### Method
Not Applicable (Python function)

### Endpoint
Not Applicable (Python function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **vectors** (str | list[str]) - Required - vector numbers to get info for
- **periods** (int) - Optional - number of periods to retrieve data for (Default: 1)
- **start_release_date** (date | None) - Optional - start release date for the data (Default: None)
- **end_release_date** (date | None) - Optional - end release date for the data (Default: None)

### Request Example
```python
# Example usage:
# df = vectors_to_df(['10100001'], periods=5)
# df_range = vectors_to_df(['10100001'], start_release_date=date(2023, 1, 1), end_release_date=date(2023, 12, 31))
```

### Response
#### Success Response
- **DataFrame** - vectors as columns and ref_date as the index (not release date)
```

--------------------------------

### Update Stats Canada Tables Locally

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Checks local JSON metadata files against Stats Canada's remote metadata and updates any outdated tables. It compares the latest reference period to detect changes.

```python
def zip_update_tables(path: pathlib.Path | None = None, csv: bool = True) -> list[str]:
    """Check local json, update zips of outdated tables.

    Grabs the json files in path, checks them against the metadata on
    StatsCan and grabs updated tables where there have been changes
    There isn't actually a "last modified date" part to the metadata
    What I'm doing is comparing the latest reference period. Almost all
    data changes will at least include incremental releases, so this should
    capture what I want

    Parameters
    ----------
    path
        where to look for tables to update
    csv
        Downloads updates in CSV form by default, SDMX if false

    Returns
    -------
    :
        list of the tables that were updated

    """
    local_jsons = list_zipped_tables(path=path)
    tables = [j["productId"] for j in local_jsons]
    remote_jsons = get_cube_metadata(tables)
    update_table_list = [
        local["productId"]
        for local, remote in zip(local_jsons, remote_jsons)
        if local["cubeEndDate"] != remote["cubeEndDate"]
    ]

    download_tables(update_table_list, path, csv=csv)
    return update_table_list
```

--------------------------------

### zip_update_tables

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Check local JSON files, update zips of outdated tables. Compares the latest reference period of local tables against remote metadata to identify and download updates.

```APIDOC
## zip_update_tables

### Description
Check local json, update zips of outdated tables. Grabs the json files in path, checks them against the metadata on StatsCan and grabs updated tables where there have been changes. This function compares the latest reference period to detect changes.

### Parameters
#### Path Parameters
- **path** (Path | None) - Optional - where to look for tables to update. Defaults to None.
- **csv** (bool) - Optional - Downloads updates in CSV form by default, SDMX if false. Defaults to True.

### Returns
- **list[str]** - list of the tables that were updated
```

--------------------------------

### table_subsets_from_vectors

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Retrieves a dictionary mapping tables to a list of vectors that belong to each table. This is derived from the 'get_tables_for_vectors' function.

```APIDOC
## `table_subsets_from_vectors(vectors: str | list[str]) -> dict[str, list[str]]`

### Description
Get a list of dicts mapping tables to vectors.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Parameters
- **vectors** (str | list[str]) - Required - Vectors to find tables for

### Returns
- **dict[str, list[str]]** - keys for each table used by the vectors, matched to a list of vectors
```

--------------------------------

### get_cube_metadata(tables)

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves metadata for specified tables. Accepts a single table ID or a list of table IDs.

```APIDOC
## POST /getCubeMetadata

### Description
Takes a list of tables and returns a list of dictionaries with their metadata.

### Method
POST

### Endpoint
/getCubeMetadata

### Parameters
#### Request Body
- **tables** (list[dict]) - Required - A list of objects, where each object has a 'productId' field representing the table ID.
  - **productId** (str) - Required - The ID of the table.

### Request Example
```json
{
  "example": "[{\"productId\": \"11100037\"}]"
}
```

### Response
#### Success Response (200)
- **list[CubeMetadata]** - A list containing metadata for each requested table.

#### Response Example
None
```

--------------------------------

### get_bulk_vector_data_by_reference_period_range

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves bulk vector data within a specified reference period range. This function allows fetching data for multiple vectors by their unique identifiers and within a given date range.

```APIDOC
## get_bulk_vector_data_by_reference_period_range

### Description
Retrieves bulk vector data within a specified reference period range. This function allows fetching data for multiple vectors by their unique identifiers and within a given date range.

### Method Signature
`get_bulk_vector_data_by_reference_period_range(vectors: str | list[str], start_ref_date: dt.date, end_ref_date: dt.date) -> list[VectorData]`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **vectors** (str | list[str]) - Required - vector numbers to get info for
- **start_ref_date** (date) - Required - start reference period date for the data
- **end_ref_date** (date) - Required - end reference period date for the data

### Request Example
```python
# Example usage (assuming necessary imports and setup)
# from datetime import date
# data = scwds.get_bulk_vector_data_by_reference_period_range(
#     vectors=['10100001'],
#     start_ref_date=date(2023, 1, 1),
#     end_ref_date=date(2023, 12, 31)
# )
```

### Response
#### Success Response (200)
- **VectorData** (list[dict]) - List of dictionaries containing data for each vector.

#### Response Example
```json
{
  "example": "[{"vector_id": "10100001", "ref_period": "2023-01-01", "value": 100.5}, ...]"
}
```
```

--------------------------------

### zip_table_to_dataframe

Source: https://stats-can.readthedocs.io/en/latest/api/sc

Reads a StatsCan table into a pandas DataFrame. If the zipped CSV file for the table does not exist at the specified path, it will be downloaded.

```APIDOC
## zip_table_to_dataframe(table: str, path: pathlib.Path | None = None) -> pd.DataFrame

### Description
Reads a StatsCan table into a pandas DataFrame. If a zip file of the table does not exist in path, downloads it.

### Parameters
#### Path Parameters
- **table** (str) - Required - the table to load to dataframe from zipped csv
- **path** (pathlib.Path | None) - Optional - where to download the tables or load them, default will go to current working directory

### Returns
- **DataFrame** - the table as a dataframe
```

--------------------------------

### get_series_info_from_vector

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves metadata for specified vector numbers. This function allows users to fetch information about one or more series using their vector identifiers.

```APIDOC
## get_series_info_from_vector(vectors: str | list[str]) -> list[SeriesInfo]

### Description
Retrieves metadata for specified vector numbers. This function allows users to fetch information about one or more series using their vector identifiers.

### Method
POST

### Endpoint
/getSeriesInfoFromVector

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **vectors** (str | list[str]) - Required - vector numbers to get info for

### Request Example
```json
{
  "vectors": ["v123", "v456"]
}
```

### Response
#### Success Response (200)
- **SeriesInfo** (list[SeriesInfo]) - List of dicts containing metadata for each v#

#### Response Example
```json
[
  {
    "vectorId": "v123",
    "name": "Example Series 1",
    "description": "Description for series 1"
  },
  {
    "vectorId": "v456",
    "name": "Example Series 2",
    "description": "Description for series 2"
  }
]
```
```

--------------------------------

### get_bulk_vector_data_by_range

Source: https://stats-can.readthedocs.io/en/latest/api/scwds

Retrieves bulk vector data within a specified date range. This function is suitable for fetching historical data for one or more vectors over a defined period.

```APIDOC
## get_bulk_vector_data_by_range

### Description
Retrieves bulk vector data within a specified date range. This function is suitable for fetching historical data for one or more vectors over a defined period.

### Method
POST

### Endpoint
/getBulkVectorDataByRange

### Parameters
#### Request Body
- **vectors** (str | list[str]) - Required - vector numbers to get info for
- **start_release_date** (date) - Required - start release date for the data
- **end_release_date** (date) - Required - end release date for the data

### Request Example
```json
{
  "vectors": ["101056", "101057"],
  "start_release_date": "2023-01-01",
  "end_release_date": "2023-12-31"
}
```

### Response
#### Success Response (200)
- **VectorData** (list[dict]) - List of dicts containing data for each vector
```
Tokens:
Raw