### Example Usage: Search for 'cows'

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

This code snippet demonstrates how to use the `search` function to find tables related to 'cows' and display the first few rows of the resulting pandas DataFrame.

```python
tables = search("cows")
tables.head()
```

--------------------------------

### Initialize Selection Widget (Norway)

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Initializes a selection widget for Norwegian statistics using a table ID. This is a convenient way to start the data selection process for tables hosted on the Norwegian statistics API.

```python
box = select(table_id = '10714')
box
```

--------------------------------

### Define Default Data Sources in Python

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Sets up a dictionary to define default configurations for data sources, starting with Statistics Norway. This structure allows the package to be flexible for future integration with other statistical agencies by defining base URLs, language options, and API keys.

```python
# This is mainly to make the script flexible for future changes
# Makes it easier to use the same script for sources other than Statistics Norway
# The user can set a different source to be the default
# (note to self: is the extra complexity worth it? 
# more complex since functions now require defaults to be used easily)

Statistics_Norway = {'base_url' : ['http://data.ssb.no/api/v0'], 
                     'language' : ['en', 'no'], 
                     'api_key' : [None]}

# Add more sources here Statistics_Sweden = {...} etc

# set a default
default = Statistics_Norway

# make a list of the different sources
sources = ['Statistics_Norway']
```

--------------------------------

### Install and Import stats-to-pandas in Python

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

This snippet shows how to install the 'stats-to-pandas' package using pip and import its functions into a Python script. It lists the necessary dependencies for full functionality, including GUI features.

```python
# (Will work after uploading to pypi, scheduled before 1. july, 2016)
#
# Requires: requests, pyjstat, jupyter, ipywidgets, IPython
#
#pip install stats_to_pandas
#
#from stats_to_pandas import *
#import stats_to_pandas as stp
```

--------------------------------

### Call and Print Variables from SSB API

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Demonstrates how to call the `get_variables` function with a specific table ID and print the returned variable information. This is a direct usage example.

```python
variables = get_variables(table_id = '03789')
print(variables)
```

--------------------------------

### Create Interactive Variable Selection GUI

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/README.md

Generates a graphical user interface (GUI) box for selecting variables from a specific statistical table. This feature requires a Jupyter notebook environment with ipywidgets installed. The output is a widget object that captures user selections.

```python
box = stp.select(table_id = '10714')
```

--------------------------------

### Search premade tables with stats_to_pandas.search_premade

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Retrieves a pandas DataFrame of tables matching specified tags. This function searches in table tags, not headlines, and an exact match is performed. It's recommended to use phrase='*' to get a list of all tables. The 'ID' column provides a specific table ID for downloading.

```python
import stats_to_pandas

# Example 1: Search for tables with the tag 'population'
tables = stats_to_pandas.search_premade(phrase='population')

# Example 2: Get a list of all tables
tables = stats_to_pandas.search_premade()

# Example 3: Search with a specific language
tables = stats_to_pandas.search_premade(phrase='energy', language='no')

```

--------------------------------

### Get JSON from Widget Box Selection

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/README.md

Retrieves the JSON representation corresponding to the current selection made within a widget box. This function is used in conjunction with the `stp.select()` method to get the query parameters defined by the user's interaction.

```python
query = stp.get_json(box)
```

--------------------------------

### Get Variable Information

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Retrieves a list of dictionaries, where each dictionary contains detailed information about a variable within a specified table. This can include available years, categories, and textual descriptions.

```APIDOC
## GET /api/v0/{language}/table/{table_id}

### Description
Retrieves detailed information about variables associated with a specific table ID. The response is a list of objects, each describing a variable's properties such as available values, texts, and time-related information.

### Method
GET

### Endpoint
`/api/v0/{language}/table/{table_id}`

### Parameters
#### Path Parameters
- **language** (string) - Optional - Specifies the language for the table information. Defaults to 'en' (English).
- **table_id** (string) - Required - The unique identifier for the table, including leading zeros.

#### Query Parameters
- **base_url** (string) - Optional - The base URL for the API. Defaults to 'http://data.ssb.no/api/v0'.
- **full_url** (string) - Optional - If provided, this full URL will be used instead of constructing one from `base_url`, `language`, and `table_id`.

### Request Example
```python
# Example usage in Python (using the get_variables function):
variables = get_variables(table_id='03789')
print(variables)
```

### Response
#### Success Response (200)
- **variables** (list) - A list of dictionaries, each containing information about a variable.
  - **code** (string) - The code representing the variable.
  - **elimination** (boolean) - Indicates if the variable is subject to elimination.
  - **valueTexts** (list of strings) - User-friendly text representations of the variable's values.
  - **text** (string) - The descriptive name of the variable.
  - **values** (list of strings) - The actual values for the variable.
  - **time** (boolean) - Indicates if the variable represents time.

#### Response Example
```json
[
  {
    "code": "BuskapStr",
    "elimination": true,
    "valueTexts": ["Total", "1-4", "5-9", "10-14", "15-19", "20-29", "30-"],
    "text": "size of herd",
    "values": ["00", "01", "02", "03", "04", "05", "06"]
  },
  {
    "code": "ContentsCode",
    "valueTexts": ["Holdings keeping dairy cows", "Dairy cows"],
    "text": "contents",
    "values": ["Driftseiningar", "Mjolkeku"]
  },
  {
    "code": "Tid",
    "text": "year",
    "time": true,
    "valueTexts": ["1998", "1999", "2000", "2002", "2003", "2004", "2005", "2006", "2007", "2008", "2009", "2010", "2011", "2012", "2013", "2014", "2015", "2016"],
    "values": ["1998", "1999", "2000", "2002", "2003", "2004", "2005", "2006", "2007", "2008", "2009", "2010", "2011", "2012", "2013", "2014", "2015", "2016"]
  }
]
```
```

--------------------------------

### Get Full JSON Query using full_json

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Retrieves the complete JSON query for all options of a given table. This is useful for obtaining a base query that can be modified instead of building one from scratch. The output can be a dictionary or a string. If the output is a string, it can be converted back to a dictionary using the `to_dict` function.

```python
query = full_json(table_id = ‘10714’, out = ‘string’)
```

--------------------------------

### Generate Full JSON-stat Query (Python)

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Generates a JSON-stat query to retrieve all values for all options within a specified table. Useful as a starting point for custom queries when not using interactive tools. It requires the table ID and allows specifying the output format (dict or string) and language. Dependencies include `ast` for dictionary conversion.

```python
import ast

def full_json(table_id = None,
              out = 'dict',
              language = 'en',
              full_url = None):
    """
    Returns the json query for getting all the values for all options for a table.
    Useful if
        - you do not want to use the widgets/notebook to build the query
        - but want a starting point to revise and specify a the query instead of building it from the ground.
    
    Note
        - The json is a dictionary. You may want to edit the content of the dictionary directly,
        or make it a string, edit it as a string, and retransform it to a dictionary.
        
        To transform a string to a dict, use the function:
        
            query = to_dict(json_str)
        
    Example
    -------
    
    query = full_json(table_id = '10714', out = 'string')
    
    """
    
    variables = get_variables(table_id, language = language, full_url = full_url)
    nvars = len(variables)
    var_list = list(range(nvars))
    
    query_element = {}
    
    for x in var_list:
        query_element[x] ='{{"code": "{code}", "selection": {{"filter": "item", "values": {values} }}}}'.format(
                code = variables[x]['code'], 
                values = variables[x]['values'])
        query_element[x] = query_element[x].replace("\'", '"')
    all_elements = str(list(query_element.values()))
    all_elements = all_elements.replace("\'", "")
    
    query = '{{"query": {all_elements} , "response": {{"format": "json-stat" }}}}'.format(all_elements = all_elements)
    
    if out == 'dict':
        query = ast.literal_eval(query)
    
    return query

```

--------------------------------

### Get Variable Information from SSB API

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Retrieves detailed information about variables within a specified statistical table from the SSB API. It handles URL construction and uses pandas to parse the JSON response. Dependencies include pandas.

```python
import pandas as pd

def get_variables(
        table_id = None,
        source = None, 
        language = 'en',
        base_url = 'http://data.ssb.no/api/v0',
        full_url = None):
    """
        Returns a list. 
        
        Each element of the list is a dictionary that provides more 
        information about a variable. 
        
        For instance, one variable may contain information about the
        different years that are available.
        
        
        Parameters
        ----------
    
            table_id: string
                the unique table_id number, a string including leading zeros.
                                
            language: string
                default 'en' (default, English) 
                optional: 'no' (Norwegian) 
            
            base_url: string
                base url locating the table (not including table identifier)
            
            full_url: string 
                The full url to the table.
                If full_url is specified, other paramaters are ignored.
                
    """
    
    if full_url is None:
          full_url = '{base_url}/{language}/table/{table_id}'.format(
            base_url = base_url, language = language, table_id = table_id)
    
    df = pd.read_json(full_url)
    variables = [dict(values) for values in df.iloc[:,1]]
    
    return variables
```

--------------------------------

### Initialize Documentation Options (JavaScript)

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/py-modindex.html

Initializes global options for the documentation site, including root URL, version, and file suffixes. This is a standard pattern for Sphinx-generated documentation.

```javascript
var DOCUMENTATION_OPTIONS = {
    URL_ROOT: './',
    VERSION: '0.0.7',
    COLLAPSE_INDEX: false,
    FILE_SUFFIX: '.html',
    HAS_SOURCE: true
};
DOCUMENTATION_OPTIONS.COLLAPSE_INDEX = true;
```

--------------------------------

### Initialize Sphinx Documentation Options

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/modules.html

This JavaScript code initializes configuration options for Sphinx documentation, including the root URL, version, and file suffix. It is typically found in the documentation's build output.

```javascript
var DOCUMENTATION_OPTIONS = {
  URL_ROOT: './',
  VERSION: '0.0.7',
  COLLAPSE_INDEX: false,
  FILE_SUFFIX: '.html',
  HAS_SOURCE: true
};
```

--------------------------------

### Initialize Search Functionality with JavaScript

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/search.html

This snippet initializes the search functionality for the documentation website. It configures search options and loads the search index using JavaScript and jQuery.

```javascript
var DOCUMENTATION_OPTIONS = {
  URL_ROOT: './',
  VERSION: '0.0.7',
  COLLAPSE_INDEX: false,
  FILE_SUFFIX: '.html',
  HAS_SOURCE: true
};

jQuery(function() {
  Search.loadIndex("searchindex.js");
});
```

--------------------------------

### Sphinx Search Box Initialization

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/genindex.html

This JavaScript code initializes the search box functionality for the Sphinx documentation. It's part of the theme's interactive features.

```javascript
$('#searchbox').show(0);
```

--------------------------------

### Download Data from Ireland URL in Python

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Demonstrates how to download data directly from a URL provided by the Central Statistics Office of Ireland using the 'read_url' function. This bypasses the need for table IDs and GUI selections if the full URL is known.

```python
# Read data from Statistics Ireland
irl_url = 'http://www.cso.ie/StatbankServices/StatbankServices.svc/jsonservice/responseinstance/CNA31' 
df = read_url(full_url = irl_ulr)
```

--------------------------------

### Initialize Selection Widget (Sweden)

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Initializes a selection widget for Swedish statistics by providing the full URL to the data table. This is necessary because the table ID structure differs from Norway's API. The widget facilitates selecting variables for the specified Swedish statistics table.

```python
# The box select framework also works for Statistics Sweden 
# The full_url to the table has to be specified since table_id 
# does not have the same structure as in Norway
full_url = 'http://api.scb.se/OV0104/v1/doris/en/ssd/BE/BE0101/BE0101A/BefolkningNy'
box_sweden = select(full_url = full_url)
box_sweden
```

--------------------------------

### Search for Statistical Tables by Keyword in Python

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Demonstrates how to use the 'search' function to find statistical tables based on keywords. This function is part of the 'stats-to-pandas' package and helps identify relevant tables from sources like Statistics Norway.

```python
search('cows')
```

--------------------------------

### Download Full Table Data or Generate JSON Query in Python

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

This snippet covers multiple functionalities: downloading all data for a table using 'read_all', generating the full JSON query string with 'full_json', obtaining the JSON query as a dictionary, and retrieving the JSON associated with a widget box using 'get_json'.

```python
# Read all values for all variables for table with id 10714
# (Avoid having to select variables, step 2 above, but may result in large tables)
df = read_all(table_id = '10714')

# Get the json string for a full query of table number 10714
query = full_json(table_id = '10714', out = 'str')

# Get the json dict for a full query of table number 10714
query = full_json(table_id = '10714', out = 'dict')

# Get the json dict associated with the current selection from a widget box
query = get_json(box)
```

--------------------------------

### Search Premade Tables from Statistics Norway using Python

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

This Python function, `search_premade`, allows users to search for premade tables from Statistics Norway based on keywords. It queries an API endpoint and returns a Pandas DataFrame containing table titles and tags. The search is tag-based and exact, so using a wildcard '*' can retrieve all available tables. The function is useful for quickly identifying relevant datasets before downloading them with more specific parameters.

```python
def search_premade(
    phrase = '*',
    language = 'en',
    url = 'http://data.ssb.no/api/v0/dataset'):
    
    """
    Returns a pandas dataframe with the tables matching the tags specified in the search.
    
    Note that the function does not search in the headline of the tables, but in the tags.
    
    The search is exact, so it may be better to get a list of all tables 
    (use phrase = '*' or just leave it unspecified)
    
    The ID column contains a special table id for premade tables 
    (use this is when specifying the table to be downloaded)
    
    Example:
        
        tables = search_url_tables('population')
    
    
    """
    url = '{url}?lang={language}'.format(
        url = url,
        language = language)
    
    df = pd.read_html(url)
    df = df[0]
    df.index = df['ID']
    df = df.iloc[:,[0,1]]
    df = df.sort_index()
    phrase = phrase.lower()
    
    if phrase != '*':
        df = df[(df.iloc[:,0].str.lower().str.contains(phrase)) | 
                (df.iloc[:,1].str.lower().str.contains(phrase)) ]
    return df

# to do: merge the two search functions?
# make premade an bool arg in the search. 
# also allow full_url

```

```python
search_premade('birth')
```

--------------------------------

### Build Widget Container

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Constructs a vertical box widget container to hold various labels and variable selection components. It includes a title, explanatory text, variable selection area, and a URL. The container's layout is customized with a border.

```python
headline = widgets.Label(value = table_title, color = 'blue')
    
endline = widgets.Label(value = '''Select category and click on elements 
        to be included in the table (CTRL-A selects "all")''')
    
url_text = widgets.Label(value = full_url)
    
selection_container = widgets.VBox([headline, 
                                        endline, 
                                        variables_container, 
                                        url_text])
    
selection_container.layout.border = '3px grey solid'
    # may include a "click here when finished" just to make it more intuitive?
    return selection_container
```

--------------------------------

### Show Search Box with jQuery

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/index.html

This JavaScript snippet uses jQuery to display the search box element on the page. It applies a 0ms animation for immediate visibility. This is commonly found in web interfaces to dynamically show or hide elements.

```javascript
$(document).ready(function(){$('#searchbox').show(0);});
```

--------------------------------

### Select Table and Create Variable Selection Widgets

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Constructs a user interface with selection widgets for variables and their values from a specified SSB table. It relies on `get_variables` and the `ipywidgets` library. The function constructs URLs, retrieves table information, and dynamically creates select widgets.

```python
import pandas as pd
from collections import OrderedDict
from ipywidgets import widgets

def select(table_id = None, 
           language = 'en', 
           base_url = 'http://data.ssb.no/api/v0', 
           full_url = None):
    """
    Selects a table based on the table_id and returns a widget container 
    in which the user can select the set of variables and values to be 
    included in the final table.
    
    
    Example
    --------
    box = select(table_id = '10714')
    
    
    Parameters
    ----------    
    
        table_id : string 
            the id of the desired table
         
        language: string
            language for table
            'en' (default, English) 
            'no' (Norwegian): 
            language for table
        
        base_url: string.
            base url locating the table (not including table_id)
        
        full_url: string
            the full url to the table
    """
        
    # get table_id not full url was specified 
    if full_url is None:
        full_url = '{base_url}/{language}/table/{table_id}'.format(
                    base_url = base_url, 
                    language = language, 
                    table_id = table_id)
        
    table_info = pd.read_json(full_url)
    table_title = table_info.iloc[0,0]

    # get a list with dictionaries containing information about each variable
    variables = get_variables(table_id = table_id, 
                              language = language,
                              base_url = base_url,
                              full_url = full_url)
    
    # get number of variables (ok, childish approach, can be simplified!)
    nvars = len(variables)
    var_list = list(range(nvars))
    
    # a list of dictionaries of the values available for each variable
    option_list = [OrderedDict(zip(variables[var]['valueTexts'], 
                                   variables[var]['values'])) 
                   for var in var_list]
    
    # create a selection widget for each variable
    # todo: skip widget or make it invisible if there is only one option?
    # todo: make first alternative a default selection initially for all tables?
    # todo: add buttons for selecting "all", "latest" , "first" and "none"
                         
    selection_widgets = [widgets.widget_selection.SelectMultiple(
                            options = option_list[var], 
                            height = 400, 
                            width = 500) 
                         for var in var_list]
    
    # put all the widgets in a container
    variables_container = widgets.Tab(selection_widgets)

    # label each container with the variable label 
    for var in var_list:
        title = str(variables[var]['text'])
        variables_container.set_title(var, title)
    
```

--------------------------------

### Select Table Variables using GUI Box in Python

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Illustrates the use of the 'select' function to create a graphical user interface (GUI) box for choosing variables from a specific table. This simplifies the process of building a json-stat query for data download.

```python
box = select(table_id = '10714')
```

--------------------------------

### Fetch ONS Data using Python

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

This Python code snippet demonstrates how to fetch data from the UK's Office for National Statistics (ONS) API. It requires an API key and constructs a URL to retrieve specific census data. The result is loaded into a Pandas DataFrame for further analysis. Ensure you replace 'YOUR_API_KEY_HERE' with your actual ONS API key.

```python
ons_url = 'http://data.ons.gov.uk/ons/api/data/dataset/QS201EW.json?context=Census&apikey=YOUR_API_KEY_HERE&geog=2011WARDH&dm/2011WARDH=E92000001&jsontype=json-stat&totals=false'
df = read_url(ons_url)
df.head()
```

--------------------------------

### Search Premade API

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Retrieves a list of tables matching specified tags. This function searches within table tags, not headlines, and returns a DataFrame.

```APIDOC
## POST /api/dataset/search

### Description
Returns a pandas DataFrame containing tables that match the provided tags. This function searches in the tags of the tables, not in their headlines. The search is exact.

### Method
POST

### Endpoint
/api/dataset/search

### Parameters
#### Query Parameters
- **phrase** (string) - Optional - The tag(s) to search for. Defaults to '*' which returns all tables.
- **language** (string) - Optional - The language for the search. Defaults to 'en' (English).
- **url** (string) - Optional - The URL for the premade dataset search. Defaults to 'http://data.ssb.no/api/v0/dataset'.

### Request Example
```json
{
  "phrase": "population"
}
```

### Response
#### Success Response (200)
- **results** (DataFrame) - A pandas DataFrame with tables matching the specified tags. The 'ID' column contains a special table ID for premade tables.

#### Response Example
```json
{
  "results": "[Pandas DataFrame representation]"
}
```
```

--------------------------------

### Read Data from Statistics Ireland URL

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/README.md

Downloads data directly from a provided URL, specifically demonstrated with a Statistics Ireland endpoint. The function expects a full URL as input and returns the data as a pandas DataFrame. It utilizes the `requests` library internally.

```python
irl_url = 'http://www.cso.ie/StatbankServices/StatbankServices.svc/jsonservice/responseinstance/CNA31'
df = stp.read_url(full_url = irl_url)
```

--------------------------------

### Search for Statistical Tables by Keyword

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/README.md

Searches for available statistical tables based on provided keywords. Returns a pandas DataFrame containing a list of matching tables, including their IDs, which can be used for further data retrieval.

```python
stp.search('cows')
```

--------------------------------

### Select Table API

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Selects a specific table based on its ID and allows the user to interactively choose variables and values for the final dataset.

```APIDOC
## POST /api/v0/dataset/{table_id}

### Description
Selects a table based on the provided `table_id`. It returns a widget container allowing the user to select the variables and values to be included in the final table.

### Method
POST

### Endpoint
/api/v0/dataset/{table_id}

### Parameters
#### Path Parameters
- **table_id** (string) - Required - The unique identifier of the desired table.

#### Query Parameters
- **language** (string) - Optional - The language for the table. Defaults to 'en' (English). Other options include 'no' (Norwegian).
- **base_url** (string) - Optional - The base URL for locating the table (does not include `table_id`).
- **full_url** (string) - Optional - The complete URL to the table.

### Request Example
```json
{
  "table_id": "10714"
}
```

### Response
#### Success Response (200)
- **widget_container** (object) - A container object that allows interactive selection of table variables and values. The structure of this object depends on the frontend implementation.

#### Response Example
```json
{
  "widget_container": {
    "//": "Structure depends on the frontend implementation"
  }
}
```
```

--------------------------------

### Generate JSON Query from Widget with get_json

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Generates a JSON dictionary or string that represents the variables selected in a widget container. This JSON follows the JSON-Stat format and is used to fetch the selected variables. The output format can be specified as 'dict' (default) or 'str'.

```python
json_query = get_json(box)
```

--------------------------------

### Import stats-to-pandas Package

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/README.md

Imports the necessary stats-to-pandas library for use in Python scripts. This is the foundational step before utilizing any of the package's functionalities.

```python
import stats_to_pandas as stp
```

--------------------------------

### Read Premade Table into DataFrame with read_premade

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Retrieves data from a pre-defined table, identified by a `premade_id` or a `full_url`, and returns it as a pandas DataFrame. The table format can be specified, defaulting to 'json'. Note that the `premade_id` might differ from the standard table ID.

```python
df = read_premade(premade_id='some_id')
```

--------------------------------

### Import Required Libraries for Data Download in Python

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Lists the essential Python libraries required for the 'stats-to-pandas' package, including pandas for data manipulation, requests for HTTP calls, pyjstat for JSON-stat parsing, and ipywidgets/jupyter for GUI functionalities. It notes that GUI components are optional if not using the interactive interface.

```python
# The following has to be installed: 
#    pandas, requests, pyjstat, ipywidgets, jupyter, IPython
#
# The last two (jupyter and ipywidgets) are required to create a gui 
# that makes it easier to build a query. They are not needed to download 
# data if you give the full_url or specify the json query yourself.

import pandas as pd
import requests
import ast
from pyjstat import pyjstat
from collections import OrderedDict
from ipywidgets import widgets
from IPython.display import display
```

--------------------------------

### Read Premade Statistics Norway Table (Python)

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Fetches a premade table from Statistics Norway and converts it into a pandas DataFrame. Supports JSON-stat and CSV formats. Requires the 'requests', 'pyjstat', and 'pandas' libraries.

```python
import pandas as pd
import requests
import pyjstat
from collections import OrderedDict

def read_premade(premade_id = None, 
            language = 'en', 
            base_url = 'http://data.ssb.no/api/v0/dataset', 
            full_url = None, 
            table_format = 'json'):
    """
    Returns a pandas dataframe of the premade table indicated by the premade 
    table_id or the full_url.
    
    Note: The premade table id may be different from the normal table id.
    """
    
    if full_url is None:
        full_url = '{base_url}/{premade_id}.{table_format}?lang={language}'.format(
                base_url = base_url,
                premade_id = str(premade_id), 
                language = language,
                table_format = table_format)
    #print(full_url)
    
    if table_format == 'json':
        data = requests.get(full_url)
        df = pyjstat.from_json_stat(data.json(object_pairs_hook=OrderedDict))
        df = df[0]
        
    elif table_format == 'csv':
        df = pd.read_csv(full_url)
    else:
        print("""Table_format is incorrectly specified. 
              It must be 'json-stat' or 'csv'"""")
        df = None
    return df

```

```python
births = read_premade('1052')
births.head()
```

--------------------------------

### Display DataFrame Head

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Displays the first few rows of a pandas DataFrame. This is a standard operation for quickly inspecting the structure and content of the loaded data.

```python
df = read_box(box)
df.head()
```

--------------------------------

### Select table data with stats_to_pandas.select

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Selects a specific table based on its ID and returns a widget for users to choose variables and values. It supports language selection and custom base URLs. The function is designed to interactively select data before fetching it.

```python
import stats_to_pandas

# Example 1: Select table with ID '10714' in English
box = stats_to_pandas.select(table_id='10714')

# Example 2: Select table in Norwegian
box = stats_to_pandas.select(table_id='10714', language='no')

# Example 3: Select table using a full URL
full_url = 'http://data.ssb.no/api/v0/dataset/10714'
box = stats_to_pandas.select(full_url=full_url)

```

--------------------------------

### Fetch Variable Information with get_variables

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Retrieves detailed information about variables for a specified table ID. This function returns a list where each element is a dictionary containing details about a variable, such as available years. It allows specifying the table ID, language, base URL, or a full URL to the data source.

```python
get_variables(table_id = '10714', language='en')
```

--------------------------------

### Search tables with stats_to_pandas.search

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Searches for tables containing a specific phrase on Statistics Norway. It returns a pandas DataFrame with the search results. Supports multi-word phrases, truncation, and language selection. It uses a default base URL but allows for customization.

```python
import stats_to_pandas

# Example 1: Basic search
df = stats_to_pandas.search("income")

# Example 2: Multi-word phrase search
df = stats_to_pandas.search("export Norwegian parrot")

# Example 3: Truncated search
df = stats_to_pandas.search("pharma*")

# Example 4: Search in Norwegian
df = stats_to_pandas.search("inntekt", language='no')

# Example 5: Custom base URL
df = stats_to_pandas.search("population", base_url='http://api.example.com/data')

```

--------------------------------

### Read Table Data from URL into DataFrame with read_url

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Fetches data from a specified URL and loads it into a pandas DataFrame. This function is useful for accessing data hosted at a given URL, with support for different table formats, defaulting to 'json'.

```python
df = read_url(full_url='http://example.com/data.json')
```

--------------------------------

### Search API

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Searches for tables containing a specific phrase in Statistics Norway's data. Returns a pandas DataFrame with the search results.

```APIDOC
## POST /api/search

### Description
Searches for tables that contain the specified phrase in Statistics Norway. The results are returned as a pandas DataFrame.

### Method
POST

### Endpoint
/api/search

### Parameters
#### Query Parameters
- **phrase** (string) - Required - The phrase to search for. Can contain multiple words or use truncation with '*'.
- **language** (string) - Optional - The language for the search. Defaults to 'en' (English). Other options include 'no' (Norwegian).
- **base_url** (string) - Optional - The base URL for the API. Defaults to 'http://data.ssb.no/api/v0'.

### Request Example
```json
{
  "phrase": "income",
  "language": "en"
}
```

### Response
#### Success Response (200)
- **results** (DataFrame) - A pandas DataFrame containing the search results.

#### Response Example
```json
{
  "results": "[Pandas DataFrame representation]"
}
```
```

--------------------------------

### Read Data from GUI Variable Selection

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/README.md

Downloads data from Statistics Norway based on the variable selections made in a GUI box. This function processes the widget object created by `stp.select()` and returns the data as a pandas DataFrame.

```python
df = stp.read_box(box)
```

--------------------------------

### Read All Table Data into DataFrame with read_all

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Fetches all data for a given table ID and returns it as a pandas DataFrame. This function is suitable when the exact data needed is known and direct DataFrame conversion is desired without intermediate query specification. Warning: tables can be large.

```python
df = read_all(table_id = ‘10714’)
```

--------------------------------

### Read Data from Widget into DataFrame with read_box

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Converts data selected within a widget container into a pandas DataFrame. This function takes the widget container ('box') as input and returns a DataFrame containing the values for the variables chosen by the user in the widget.

```python
df = read_box(box)
```

--------------------------------

### Print JSON Query

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Prints the generated JSON query that will be used to fetch data. This is useful for debugging and understanding the structure of the request sent to the statistics API.

```python
query = get_json(box)
print(query)
```

--------------------------------

### Generate Full JSON Query Dictionary

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/README.md

Generates the complete JSON query as a Python dictionary for a given table ID. This provides a structured representation of the query, suitable for further manipulation or inspection.

```python
query = stp.full_json(table_id = '10714', out = 'dict')
```

--------------------------------

### Generate Full JSON Query String

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/README.md

Generates the complete JSON query string for a given table ID. The output format can be specified as a string. This is useful for understanding the underlying query structure or for programmatic use.

```python
query = stp.full_json(table_id = '10714', out = 'str')
```

--------------------------------

### Read Data into Pandas DataFrame

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Fetches data using a generated JSON-stat query from a specified URL and loads it into a pandas DataFrame. This function requires that all necessary variables have been selected in the input widget container. It handles the HTTP POST request and parses the JSON response.

```python
def read_box(from_box):
    """
    Takes a widget container as input (where the user has selected varables) 
    and returns a pandas dataframe with the values for the selected variables.
    
    Example
    -------
    
    df = read_box(box)
    
    """
    query = get_json(from_box)
    url = from_box.children[3].value
    data = requests.post(url, json = query)
    results = pyjstat.from_json_stat(data.json(object_pairs_hook=OrderedDict))
    return results[0]
```

--------------------------------

### Select Table Data

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Provides an interactive interface to select specific variables and values from a table. It utilizes the `get_variables` function to fetch table structure and then presents selection widgets for user interaction.

```APIDOC
## POST /api/v0/{language}/table/{table_id} (Implied by interaction)

### Description
This function, `select`, creates an interactive widget interface allowing users to choose specific variables and their corresponding values from a given table. It first retrieves table information and variable details using `get_variables` and then constructs selection widgets.

### Method
(Implicitly GET for data retrieval, interactive selection handled client-side)

### Endpoint
(Constructed based on `base_url`, `language`, and `table_id`)

### Parameters
#### Path Parameters
- **language** (string) - Optional - Specifies the language for the table. Defaults to 'en' (English).
- **table_id** (string) - Required - The unique identifier for the table.

#### Query Parameters
- **base_url** (string) - Optional - The base URL for the API. Defaults to 'http://data.ssb.no/api/v0'.
- **full_url** (string) - Optional - If provided, this full URL will be used instead of constructing one from `base_url`, `language`, and `table_id`.

### Request Example
```python
# Example usage in Python:
box = select(table_id='10714')
# Further interaction with the 'box' object would occur in an environment like Jupyter notebooks.
```

### Response
#### Success Response (200)
- **selection_widgets** (widget container) - A container object (e.g., `widgets.Tab`) holding multiple selection widgets, one for each variable in the table. Each widget allows users to select desired values for that variable.

#### Response Example
(The response is an interactive widget, not a static JSON payload. The example below illustrates the *concept* of the underlying data structure used to build the widgets.)
```json
{
  "table_title": "Example Table Title",
  "variables": [
    {
      "code": "Area",
      "text": "Region",
      "valueTexts": ["Oslo", "Bergen", "Trondheim"],
      "values": ["03", "11", "46"]
    },
    {
      "code": "Year",
      "text": "Year",
      "time": true,
      "valueTexts": ["2022", "2023"],
      "values": ["2022", "2023"]
    }
  ]
}
```
```

--------------------------------

### Read Data with Custom JSON Query using read_with_json

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/docs/_build/html/stats_to_pandas.html

Loads data into a pandas DataFrame using a specific table ID and a custom JSON query (in JSON-Stat format). This is useful when you have a precisely defined query, either created manually or modified from a generated one. The query can be provided as a dictionary.

```python
json_query = {'response': {'format': 'json-stat'}}
```

```python
df = read_with_json(table_id='10714', query=json_query)
```

--------------------------------

### Read All Data for a Statistical Table

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/README.md

Reads all available values for all variables from a specified table ID. This method bypasses the interactive variable selection and downloads the entire dataset, which may result in very large DataFrames. Requires the table ID as input.

```python
df = stp.read_all(table_id = '10714')
```

--------------------------------

### Generate JSON-stat Query

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

Generates a JSON-stat compliant query from a widget container. This function extracts selected variables and their values from the widget to create a query suitable for fetching data. It supports outputting the query as a dictionary or a string.

```python
def get_json(box=None, out = 'dict', language = 'en'):
    """
    Takes a widget container as input (where the user has selected varables) 
    and returns a json dictionary or string that will fetch these variables. 
    
    The json follows the json-stat format.
    
    Parameters
    ----------
    
    box : widget container 
        name of widget box with the selected variables
    
    out : string 
        default: 'dict', options: 'str'
        
        The json can be returned as a dictionary or a string.
        The final end query should use a dict, but some may find it useful to
        get the string and revise it before transforming it back to a dict.
    
    
    Example
    -------
    
    json_query = get_json(box)
    
    """        
    table_url = box.children[3].value
    variables = get_variables(full_url = table_url)
    nvars = len(box.children[2].children)
    var_list = list(range(nvars))
    query_element = {}
    
    # create a dict of strings, one for each variable that specifies 
    # the json-stat that selects the variables/values
    
    for x in var_list:
        value_list = str(list(box.children[2].children[x].value))
        query_element[x] = '{{"code": "{code}", "selection": {{"filter": "item", "values": {values} }}}}'.format(
            code = variables[x]['code'], 
            values = value_list)
        query_element[x] = query_element[x].replace("'", '"')
    
    all_elements = str(list(query_element.values()))
    all_elements = all_elements.replace("'", "")

    query = '{{"query": {all_elements} , "response": {{"format": "json-stat" }}}}'.format(all_elements = all_elements)
    
    if out == 'dict':
        query = ast.literal_eval(query)
    
  
    # todo: build it as a dictionary to start with (and not a string that is made into a dict as now)
    # todo: add error message if required variables are not selected
    # todo: avoid repeat downloading of same information 
    # eg. get_variables is sometimes used three times before a table is downloaded
    
    return query
```

--------------------------------

### Search Statistics Norway Tables

Source: https://github.com/hmelberg/stats-to-pandas/blob/master/notebooks/stats-to-pandas-v0.ipynb

This Python function searches the Statistics Norway API for tables containing a specific phrase. It handles URL encoding for special characters and returns a pandas DataFrame with the search results. The results are then formatted to be more readable, including splitting the title and setting the table ID as the index.

```python
import pandas as pd

def search(phrase, 
           language = 'en',
           base_url = 'http://data.ssb.no/api/v0'):
    """
        Search for tables that contain the phrase in Statistics Norway.
        Returns a pandas dataframe with the results.
            
        Example
        -------
        
            df = search("income")
        
        
        Parameters
        ----------
        
        phrase: string
            The phrase can contain several words (space separated): 
                search("export Norwegian parrot")
            
            It also supports trucation: 
                search("pharma*")
                
            Not case sensitive.
            Language sensitive (specified in the language option)
            
        language: string 
            default in Statistics Norway: 'en' (Search for English words) 
            optional in Statistics Norway: 'no' (Search for Norwegian words) 
                   
        url: string
            default in Statistics Norway: 'http://data.ssb.no/api/v0'
            different defaults can be specified
       
        """
    
    # todo: make converter part of the default specification only for statistics norway
    convert = {'æ' : '%C3%A6', 'Æ' : '%C3%86', 'ø' : '%C3%B8', 'Ø' : '%C3%98', 
             'å' : '%C3%A5', 'Å' : '%C3%85',
             '"' : '%22', '(' : '%28', ')' : '%29', ' ' : '%20'}

    search_str = '{base_url}/{language}/table/?query={phrase}'.format(
        base_url = base_url, 
        language = language, 
        phrase = phrase)
    
    for k, v in convert.items():
        search_str = search_str.replace(k, v)
    
    #print(search_str)    
    
    df = pd.read_json(search_str)
    
    if len(df) == 0:
        print("No match")
        return df
    
    # make the dataframe more readable
    # (is it worth it? increases vulnerability. formats may differ and change)
    # todo: make search and format conditional on the database being searched    
       
    # split the table name into table id and table text
  
    df['table_id'] = df['title'].str.split(':').str.get(0)
    df['table_title'] = df['title'].str.split(':').str.get(1)
    del df['title']
    
    # make table_id the index, visually more intuitive with id as first column
    df = df.set_index('table_id')
    
    # change order of columns to make it more intuitive (table_title is first) 
    cols = df.columns.tolist()
    cols.sort(reverse = True)
    df = df[cols[:-2]]

    return df

```