### Install SDMX Schemas and Validate Dataflow Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/howto.md Installs the necessary SDMX schemas and then validates a dataflow message. Ensure schemas are installed before validation. ```python import pandasdmx pandasdmx.install_schemas() ecb = pandasdmx.Request("ECB") exr = ecb.dataflow("EXR") ecb.validate(exr) # should return True ``` -------------------------------- ### Install pandasdmx from source using flit Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/install.md Install the pandasdmx package after cloning the repository, using flit. ```bash $ flit install ``` -------------------------------- ### Install pandasdmx with optional dependencies Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/install.md Install pandasdmx along with specific optional dependencies like 'cache', 'doc', or 'test'. Use a comma-separated list for multiple extras. ```bash $ pip install pandasdmx[cache] ``` ```bash $ pip install pandasdmx[cache,doc,test] ``` -------------------------------- ### Install pandaSDMX Source: https://context7.com/dr-leo/pandasdmx/llms.txt Install the pandaSDMX library using pip or conda. Ensure you use the conda-forge channel for conda installations. ```bash # Using pip pip install pandasdmx ``` ```bash # Using conda conda install pandasdmx -c conda-forge ``` -------------------------------- ### Install pandasdmx from source using pip Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/install.md Install the pandasdmx package after cloning the repository, using pip. ```bash $ pip install . ``` -------------------------------- ### Install pandasdmx using pip Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/install.md Use this command to install the core pandasdmx package with pip. ```bash $ pip install pandasdmx ``` -------------------------------- ### Install pandasdmx using conda Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/install.md Install the pandasdmx package from the conda-forge channel. ```bash $ conda install pandasdmx -c conda-forge ``` -------------------------------- ### Query Data with Key and Time Period Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Demonstrates how to query data using a specific key for the CURRENCY dimension and a start period. ```python from pandasdmx import Request # Example: Query for EUR or JPY currency data starting from 2021-01-01 # Note: This is a conceptual example, actual API calls may vary. # req = Request(source='ECB') # data = req.get(key='CURRENCY=EUR+JPY', params={'startPeriod': '2021-01-01'}) ``` -------------------------------- ### Clone pandasdmx repository Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/install.md Clone the pandaSDMX Github repository to install from source. ```bash $ git clone git@github.com:dr-leo/pandaSDMX.git ``` -------------------------------- ### Use a custom schema directory for validation Source: https://context7.com/dr-leo/pandasdmx/llms.txt Install a custom schema directory using `sdmx.install_schemas()` and then specify it during validation. This is useful for local or private schemas. ```python sdmx.install_schemas(schema_dir='/opt/sdmx/schemas') ecb.validate(exr_msg, schema_dir='/opt/sdmx/schemas') ``` -------------------------------- ### Generic Data Format Example Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Illustrates the XML structure for generic data, where dimensions and attributes are explicitly identified. ```xml ``` -------------------------------- ### get(resource_type=None, resource_id=None, tofile=None, use_cache=False, dry_run=False, **kwargs) Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Retrieve SDMX data or metadata from a specified source. This method allows flexible retrieval by resource type and ID, or by providing a resource object. It supports various options for data filtering, caching, and dry runs. ```APIDOC ## get(resource_type=None, resource_id=None, tofile=None, use_cache=False, dry_run=False, **kwargs) ### Description Retrieve SDMX data or metadata. Metadata is retrieved from the `source` of the current Request. The item(s) to retrieve can be specified in one of two ways: 1. `resource_type`, `resource_id`: These give the type (see `Resource`) and, optionally, ID of the item(s). If the `resource_id` is not given, all items of the given type are retrieved. 2. A resource object, i.e. a `MaintainableArtefact`: `resource_type` and `resource_id` are determined by the object’s class and `id` attribute, respectively. Data is retrieved with `resource_type`=’data’. In this case, the optional keyword argument `key` can be used to constrain the data that is retrieved. Examples of the formats for `key`: 1. `{'GEO': ['EL', 'ES', 'IE']}`: `dict` with dimension name(s) mapped to an iterable of allowable values. 2. `{'GEO': 'EL+ES+IE'}`: `dict` with dimension name(s) mapped to strings joining allowable values with ‘+’, the logical ‘or’ operator for SDMX web services. 3. `'....EL+ES+IE'`: `str` in which ordered dimension values (some empty, `''`) are joined with `'.'`. Using this form requires knowledge of the dimension order in the target data resource_id; in the example, dimension ‘GEO’ is the fifth of five dimensions: `'.'.join(['', '', '', '', 'EL+ES+IE'])`. For formats 1 and 2, but not 3, the `key` argument is validated against the relevant `DataStructureDefinition`, either given with the `dsd` keyword argument, or retrieved from the web service before the main query. For the optional `param` keyword argument, some useful parameters are: - ‘startperiod’, ‘endperiod’: restrict the time range of data to retrieve. - ‘references’: control which item(s) related to a metadata resource are retrieved, e.g. `references`=’parentsandsiblings’. ### Parameters #### Path Parameters None explicitly documented. #### Query Parameters - **resource_type** (string) - Optional - Type of resource to retrieve. - **resource_id** (string) - Optional - ID of the resource to retrieve. - **tofile** (string or fsspec.core.OpenFile) - Optional - File path or file-like to write SDMX data as it is received. *file-like* must be binary and writable. It may be used in a with-context (recommended when using a fsspec.core.OpenFile). - **use_cache** (boolean) - Optional - If `True`, return a previously retrieved `Message` from `cache`, or update the cache with a newly-retrieved `Message`. - **dry_run** (boolean) - Optional - If `True`, prepare and return a `requests.Request` object, but do not execute the query. The prepared URL and headers can be examined by inspecting the returned object. - **dsd** (object) - Optional - Existing object used to validate the key argument. If not provided, an additional query executed to retrieve a DSD in order to validate the key. - **force** (boolean) - Optional - If `True`, execute the query even if the `source` does not support queries for the given resource_type. Default: `False`. - **headers** (dict) - Optional - HTTP headers. Given headers will overwrite instance-wide headers passed to the constructor. Default: `None` to use the default headers of the `source` agency. - **key** (string or dict) - Optional - For queries with `resource_type`=’data’. `str` values are not validated; `dict` values are validated using `make_constraint()`. - **params** (dict) - Optional - Query parameters. The SDMX REST web service guidelines describe parameters and allowable values for different queries. `params` is not validated before the query is executed. - **provider** (string) - Optional - ID of the agency providing the data or metadata. Default: ID of the `source` agency. - **resource** (object) - Optional - Object to retrieve. If given, `resource_type` and `resource_id` are ignored. - **version** (string) - Optional - Version of a resource to retrieve. Default: the keyword ‘latest’. #### Request Body None explicitly documented. ### Request Example None provided. ### Response #### Success Response (200) - **Message** (`pandasdmx.message.Message` or `requests.Request`) - The requested SDMX message or, if `dry_run` is `True`, the prepared request object. #### Response Example None provided. ### Error Handling - **NotImplementedError** - If the `source` does not support the given `resource_type` and `force` is not `True`. ``` -------------------------------- ### Access SDMX data with a URL Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/howto.md Instantiate a Request object without a named data source and use the get() method with a URL to access SDMX data. ```python import pandasdmx as sdmx req = sdmx.Request() req.get(url='https://sdmx.example.org/path/to/webservice', ...) ``` -------------------------------- ### pandasdmx.model.ReportingYearStartDay Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Represents the starting day of the reporting year. ```APIDOC ## class pandasdmx.model.ReportingYearStartDay ### Description Represents the starting day of the reporting year. ``` -------------------------------- ### Configure HTTP Connection with Proxy Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Configure all queries made by a Request object by passing keyword arguments recognized by requests.request(). For example, a proxy server can be specified. ```python from pandasdmx import Request # Configure a proxy server for all requests proxies = { 'http': 'http://user:pass@host:port', 'https': 'http://user:pass@host:port', } ecb = Request('ECB', proxies=proxies) ``` -------------------------------- ### Structure-Specific Data Format Example Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Shows the more concise XML structure for structure-specific data, where dimensions and attributes are combined. ```xml ``` -------------------------------- ### Configure requests_cache with Session Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Pass arguments accepted by requests_cache.core.CachedSession when creating a Request to configure caching. This example sets up SQLite storage and a 10-minute cache expiration. ```python from pandasdmx import Request # Example configuration for requests_cache # This assumes requests_cache is installed # See requests_cache documentation for all options # To force requests_cache to use SQLite and expire cache entries after 10 minutes: req = Request("ECB", cache_options={"backend": "sqlite", "expire_after": 600}) # To enable the built-in dict-based cache for Message instances: req_with_msg_cache = Request("ECB", use_cache=True) ``` -------------------------------- ### Get Data Flow Definitions from Source Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Retrieve definitions for all data flows available from a specified source using the Request.get() method with resource_type='dataflow'. ```python from pandasdmx import Request # Create a Request object for the European Central Bank (ECB) req = Request("ECB") # Get all data flow definitions from the ECB flow_msg = req.get(resource_type='dataflow') # The response can be inspected: # print(flow_msg.response.url) # print(flow_msg.response.headers) ``` -------------------------------- ### Create and Access Key Values Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Demonstrates how to create a Key object with dimension values and access them directly or via the values dictionary. ```python >>> k = Key(foo=1, bar=2) >>> k.values['foo'] 1 >>> k['foo'] 1 ``` -------------------------------- ### pandasdmx.model.ItemScheme.get_hierarchical Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Get an Item by its hierarchical ID. ```APIDOC ## get_hierarchical(id: str) -> IT ### Description Get an Item by its [`hierarchical_id`](#pandasdmx.model.Item.hierarchical_id). ``` -------------------------------- ### validate Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Validate a message against installed XML schemas. ```APIDOC ## validate(msg, schema_dir=None) ### Description Validate a given message against the installed XML schemas. ### Parameters * **msg** (pandasdmx.message.Message or file-like) - Required - The XML message to validate. If a message.Message instance is provided, the file is re-downloaded. * **schema_dir** (path-like or str) - Optional - A custom directory where schemas are installed. ### Returns True on success. ### See Also LXML documentation. ``` -------------------------------- ### Query data with a key dict Source: https://context7.com/dr-leo/pandasdmx/llms.txt Download actual data by specifying a dataflow, a key dictionary, and optional parameters like startPeriod. The key is validated against the DSD. The data can be accessed from the returned Message object. ```python import pandasdmx as sdmx ecb = sdmx.Request('ECB') # --- Query data with a key dict (validated against DSD) --- # Assuming 'dsd' is already defined from previous step # dsd = exr_msg.structure['ECB_EXR1'] data_msg = ecb.get( 'data', 'EXR', key={'CURRENCY': ['USD', 'JPY'], 'FREQ': 'D'}, params={'startPeriod': '2020-01-01'}, dsd=dsd, ) print(data_msg.data[0]) # first DataSet ``` -------------------------------- ### Write response to a file using a context manager Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Use a file-like object within a `with` statement for writing the response to a file, including support for FSSPEC for cloud storage. ```python from pandasdmx import Request req = Request() with open('response.xml', 'wb') as f: msg = req.get(tofile=f) ``` -------------------------------- ### StructureMessage.objects() Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Gets a reference to the attribute for objects of a specific class. ```APIDOC ## objects(cls) ### Description Get a reference to the attribute for objects of type `cls`. For example, if `cls` is the class `DataStructureDefinition` (not an instance), return a reference to `structure`. ### Parameters - **cls**: The class of objects to retrieve a reference for. ``` -------------------------------- ### pandasdmx.model.RangePeriod Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Represents a period range with start and end points. ```APIDOC ## class pandasdmx.model.RangePeriod ### Description Represents a period range. ### Attributes - **start** (Period): The start of the period range. - **end** (Period): The end of the period range. ``` -------------------------------- ### AnnotableArtefact Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Represents an artefact that can have annotations. Provides methods to get and pop annotations. ```APIDOC ## class pandasdmx.model.AnnotableArtefact ### Description Represents an artefact that can have annotations. ### Methods #### get_annotation(**attrib) Return a [`Annotation`](#pandasdmx.model.Annotation) with given attrib, e.g. ‘id’. If more than one attrib is given, all must match a particular annotation. * **Raises:** **KeyError** – If there is no matching annotation. #### pop_annotation(**attrib) Remove and return a [`Annotation`](#pandasdmx.model.Annotation) with given attrib, e.g. ‘id’. If more than one attrib is given, all must match a particular annotation. * **Raises:** **KeyError** – If there is no matching annotation. ### Attributes #### annotations *: List[[Annotation](#pandasdmx.model.Annotation)]* [`Annotations`](#pandasdmx.model.Annotation) of the object. ``` -------------------------------- ### Add a new data source using JSON info Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Use this function to register a new data source. The info parameter must be a dictionary-like object containing JSON information about the source, including its ID, documentation URL, API endpoint, and name. Optional parameters allow specifying an ID, overriding existing sources, and setting up response handling callbacks. ```json { "id": "ESTAT", "documentation": "http://data.un.org/Host.aspx?Content=API", "url": "http://ec.europa.eu/eurostat/SDMX/diss-web/rest", "name": "Eurostat", "supported": {"codelist": false, "preview": true} } ``` -------------------------------- ### AttributeValue Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Represents an attribute value, which can be coded or uncoded, and optionally has a start date. ```APIDOC ## class pandasdmx.model.AttributeValue ### Description Represents an attribute value, which can be coded or uncoded, and optionally has a start date. ### Methods #### compare(other, strict=True) Return `True` if self is the same as other. Two AttributeValues are equal if their properties are equal. * **Parameters:** **strict** (*bool* *,* *optional*) – Passed to [`compare()`](#pandasdmx.util.compare). ### Attributes #### start_date *: date | None* #### value *: str | [Code](#pandasdmx.model.Code)* #### value_for *: [DataAttribute](#pandasdmx.model.DataAttribute) | None* ``` -------------------------------- ### Register custom SDMX data sources at runtime Source: https://context7.com/dr-leo/pandasdmx/llms.txt Use `sdmx.add_source()` to register a new SDMX data source using a JSON descriptor. `override=True` can replace an existing source definition. ```python import pandasdmx as sdmx # Add a custom source from a JSON string sdmx.add_source('{ "id": "MY_SOURCE", "url": "https://sdmx.myorg.example/rest", "name": "My Organisation SDMX Service", "documentation": "https://myorg.example/sdmx-docs", "supported": {"codelist": true, "preview": true} }') # Confirm it is registered print('MY_SOURCE' in sdmx.list_sources()) # True # Use the new source req = sdmx.Request('MY_SOURCE') flows = req.get('dataflow') # Override an existing source definition sdmx.add_source('{ "id": "ECB", "url": "https://sdw-wsrest.ecb.europa.eu/service", "name": "ECB (custom)" }', override=True) ``` -------------------------------- ### Initialize pandasdmx Request object Source: https://context7.com/dr-leo/pandasdmx/llms.txt Instantiate the `Request` class to connect to an SDMX data source. You can specify a built-in source ID, configure proxies, set timeouts, or use HTTP caching. A generic client can be created without a source ID. ```python import pandasdmx as sdmx # Connect to the European Central Bank ecb = sdmx.Request('ECB') # Configure a proxy and custom timeout ecb_proxy = sdmx.Request('ECB', proxies={'https': 'http://proxy.example.com:8080'}, timeout=60) # Use requests_cache for automatic HTTP caching (requires requests_cache package) ecb_cached = sdmx.Request( 'ECB', backend='sqlite', expire_after=600, # seconds fast_save=True, ) # Generic client — query any SDMX 2.1 endpoint directly generic = sdmx.Request() msg = generic.get(url='https://sdmx.example.org/rest/dataflow/ALL/ALL/latest') # Open source documentation in browser ecb.view_doc() # List all built-in source IDs print(sdmx.list_sources()) ``` -------------------------------- ### Request.validate() Source: https://context7.com/dr-leo/pandasdmx/llms.txt Validates an SDMX Message or raw XML file against the official SDMX 2.1 XML schemas. Requires schemas to be installed first. ```APIDOC ## `Request.validate()` — Validate XML against SDMX schemas Validates a `Message` or raw XML file against the official SDMX 2.1 XML schemas. Schemas must be downloaded once using `install_schemas()`. ### Parameters - **obj**: The `Message` object or file path to validate. ### Request Example ```python import pandasdmx as sdmx # Download and install official SDMX 2.1 XML schemas (one-time setup) sdmx.install_schemas() # Validate a retrieved message ecb = sdmx.Request('ECB') exr_msg = ecb.get('dataflow', 'EXR') result = ecb.validate(exr_msg) print(result) # True ``` ### Response Returns `True` if the object is valid against the schemas, `False` otherwise. ``` -------------------------------- ### Write response to a file Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Use the `tofile` keyword argument in `Request.get()` to write the response directly to a file. The parsed `Message` is still returned. ```python from pandasdmx import Request req = Request() msg = req.get(tofile='response.xml') ``` -------------------------------- ### Get allowed values for a dimension Source: https://context7.com/dr-leo/pandasdmx/llms.txt Retrieve the enumerated values for a specific dimension from its local representation. The `sdmx.to_pandas()` function can convert these to a pandas Series. ```python freq_codes = sdmx.to_pandas( dsd.dimensions.get('FREQ').local_representation.enumerated ) print(freq_codes) ``` -------------------------------- ### Explore series keys with Request.preview_data() Source: https://context7.com/dr-leo/pandasdmx/llms.txt Use `preview_data()` to retrieve all matching `SeriesKey` objects for a given dataflow and key without downloading the actual observations. This is efficient for counting or inspecting available series. ```python import pandasdmx as sdmx ecb = sdmx.Request('ECB') # All series keys for the EXR dataflow keys = ecb.preview_data('EXR') print(len(keys)) # number of series # Filter by key usd_keys = ecb.preview_data('EXR', key={'CURRENCY': 'USD'}) # Convert key values to DataFrame keys_df = sdmx.to_pandas(usd_keys) print(keys_df) ``` -------------------------------- ### Retrieve an object by ID from StructureMessage Source: https://context7.com/dr-leo/pandasdmx/llms.txt Use the `get()` method on a `StructureMessage` to retrieve any structural object by its ID, regardless of its type. The type of the returned object is printed. ```python obj = msg.get('ECB_EXR1') print(type(obj)) # ``` -------------------------------- ### Query Data in Generic Format Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Requests data in the generic SDMX-ML format. ```python # Example: Request data in generic format # data = req.get(key='CURRENCY=USD', format='xml') ``` -------------------------------- ### Validate SDMX XML against schemas Source: https://context7.com/dr-leo/pandasdmx/llms.txt Use `sdmx.install_schemas()` to download official SDMX 2.1 XML schemas. Then, use `Request.validate()` to validate a `Message` or XML file against these schemas. ```python import pandasdmx as sdmx # Download and install official SDMX 2.1 XML schemas (one-time setup) sdmx.install_schemas() # Validate a retrieved message ec_b = sdmx.Request('ECB') exr_msg = ec_b.get('dataflow', 'EXR') result = ec_b.validate(exr_msg) print(result) # True ``` -------------------------------- ### pandasdmx.Request Initialization Source: https://context7.com/dr-leo/pandasdmx/llms.txt Instantiate a Request object to interact with SDMX REST web services. You can specify a built-in source ID or use a generic client. Keyword arguments are forwarded to requests.Session. ```APIDOC ## pandasdmx.Request ### Description `Request` is the main entry point for querying any SDMX REST web service. Pass a built-in source ID (e.g. `'ECB'`, `'ESTAT'`, `'WB_WDI'`) or omit it for a generic client. All keyword arguments are forwarded to the underlying `requests.Session`. ### Usage ```python import pandasdmx as sdmx # Connect to the European Central Bank ecb = sdmx.Request('ECB') # Configure a proxy and custom timeout ecb_proxy = sdmx.Request('ECB', proxies={'https': 'http://proxy.example.com:8080'}, timeout=60) # Use requests_cache for automatic HTTP caching (requires requests_cache package) ecb_cached = sdmx.Request( 'ECB', backend='sqlite', expire_after=600, # seconds fast_save=True, ) # Generic client — query any SDMX 2.1 endpoint directly generic = sdmx.Request() msg = generic.get(url='https://sdmx.example.org/rest/dataflow/ALL/ALL/latest') # Open source documentation in browser ecb.view_doc() # List all built-in source IDs print(sdmx.list_sources()) # ['ABS', 'ABS_XML', 'BBK', 'BIS', 'ECB', 'ESTAT', 'ILO', 'IMF', ...] ``` ``` -------------------------------- ### SDMX Global Registry (SGR) Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/sources.md Documentation for the SDMX Global Registry source, including its class definition and a method for handling responses. ```APIDOC ## `SGR`: SDMX Global Registry SDMX-ML — [Website](https://registry.sdmx.org/ws/rest) ### *class* pandasdmx.source.sgr.Source(, id: str, api_id: str | None = None, url: HttpUrl | None = None, name: str, documentation: HttpUrl | None = None, headers: Dict[str, Any] = {}, resource_urls: Dict[str, HttpUrl] = {}, default_version: str = 'latest', data_content_type: DataContentType = DataContentType.XML, supports: Dict[str | [Resource](api.md#pandasdmx.Resource), bool] = {Resource.data: True}) #### handle_response(response, content) SGR responses do not specify content-type; set it directly. ``` -------------------------------- ### pandasdmx.source.load_package_sources Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Discovers and loads all data sources listed in the 'sources.json' file. ```APIDOC ### pandasdmx.source.load_package_sources() Discover all sources listed in `sources.json`. ``` -------------------------------- ### Get objects of a specific type from StructureMessage Source: https://context7.com/dr-leo/pandasdmx/llms.txt Filter objects within a `StructureMessage` by type using the `objects()` method, providing the desired model class (e.g., `sdmx.model.DataStructureDefinition`). ```python structures = msg.objects(sdmx.model.DataStructureDefinition) ``` -------------------------------- ### Download Eurostat Unemployment Data Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/example.md Fetches annual unemployment data for Greece, Ireland, and Spain from Eurostat using the UNE_RT_A dataflow. Specifies a key for the 'geo' dimension and a 'startPeriod' parameter. ```python from pandasdmx import Request # Data provider code for Eurostat msg = Request('EST').get(dataflow_id='UNE_RT_A', params={'startPeriod': '2020'}) resp = msg.data.get(0) ``` -------------------------------- ### Get a list of valid source IDs Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md This function returns a sorted list of all registered source IDs. These IDs can be used to instantiate Request objects for querying data from the respective sources. ```python pandasdmx.list_sources() ``` -------------------------------- ### Query Data from a Dataflow Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Initiate a data query for a specific data flow using the `Request.get()` method or its alias `Request.data()`. This is the primary function for retrieving actual data points based on specified parameters. ```python # Query data from the dataflow data_response = request.get(resource_type='data', dataflow='EXR') ``` -------------------------------- ### Preview Data with PandasDMX Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Use preview_data to get a preview of data for a given Dataflow ID. This method returns SeriesKeys without corresponding Observations. To count the number of series, use len() on the result. ```python keys = sdmx.Request('PROVIDER').preview_data('flow') len(keys) ``` -------------------------------- ### Create a Key object for a data query Source: https://context7.com/dr-leo/pandasdmx/llms.txt Construct a `Key` object using `make_key()` with the desired `SeriesKey` type and a dictionary of dimension-value pairs. This object represents a specific data point or series. ```python key = dsd.make_key(sdmx.model.SeriesKey, {'FREQ': 'D', 'CURRENCY': 'USD', 'CURRENCY_DENOM': 'EUR', 'EXR_TYPE': 'SP00', 'EXR_SUFFIX': 'A'}) print(key['CURRENCY']) # USD ``` -------------------------------- ### ComponentList.getdefault Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Retrieves or creates a component by its ID. If a new component is created, its order attribute is automatically set. ```APIDOC ## getdefault(id, cls=None, **kwargs) → CT ### Description Return or create the component with the given *id*. If the component is automatically created, its `Dimension.order` attribute is set to the value of [`auto_order`](#pandasdmx.model.ComponentList.auto_order), which is then incremented. ### Parameters * **id** (*str*) – Component ID. * **cls** ([*type*](#pandasdmx.model.Annotation.type), *optional*) – Hint for the class of a new object. * **kwargs** – Passed to the constructor of [`Component`](#pandasdmx.model.Component), or a Component subclass if [`components`](#pandasdmx.model.ComponentList.components) is overridden in a subclass of ComponentList. ``` -------------------------------- ### Configure Data Source Limitations with add_source() Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/sources.md Use `add_source()` to specify data content types or unsupported API endpoints for a data source. This prevents attempts to query unsupported features, raising a `NotImplementedError`. ```json [ { "id": "ABS", "data_content_type": "JSON" }, { "id": "UNESCO", "unsupported": ["datastructure"] }, ] ``` -------------------------------- ### pandasdmx.model.Key Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md SDMX Key class. The constructor takes an optional list of keyword arguments; the keywords are used as Dimension or Attribute IDs, and the values as KeyValues. ```APIDOC ### *class* pandasdmx.model.Key(arg: Mapping | Sequence[[KeyValue](#pandasdmx.model.KeyValue)] = None, , attrib: [DictLike](#pandasdmx.util.DictLike)[str, [AttributeValue](#pandasdmx.model.AttributeValue)] = None, described_by: [DimensionDescriptor](#pandasdmx.model.DimensionDescriptor) | None = None, values: [DictLike](#pandasdmx.util.DictLike)[str, [KeyValue](#pandasdmx.model.KeyValue)] = None) Bases: [`BaseModel`](#pandasdmx.util.BaseModel) SDMX Key class. The constructor takes an optional list of keyword arguments; the keywords are used as Dimension or Attribute IDs, and the values as KeyValues. For convience, the values of the key may be accessed directly: ```pycon >>> k = Key(foo=1, bar=2) >>> k.values['foo'] 1 >>> k['foo'] 1 ``` * **Parameters:** * **dsd** ([*DataStructureDefinition*](#pandasdmx.model.DataStructureDefinition)) – If supplied, the [`dimensions`](#pandasdmx.model.DataStructureDefinition.dimensions) and [`attributes`](#pandasdmx.model.DataStructureDefinition.attributes) are used to separate the kwargs into [`KeyValues`](#pandasdmx.model.KeyValue) and [`AttributeValues`](#pandasdmx.model.AttributeValue). The kwargs for [`described_by`](#pandasdmx.model.Key.described_by), if any, must be [`dimensions`](#pandasdmx.model.DataStructureDefinition.dimensions) or appear in [`group_dimensions`](#pandasdmx.model.DataStructureDefinition.group_dimensions). * **kwargs** – Dimension and Attribute IDs, and/or the class properties. #### attrib *: [DictLike](#pandasdmx.util.DictLike)[str, [AttributeValue](#pandasdmx.model.AttributeValue)]* #### copy(arg=None, **kwargs) Duplicate a model, optionally choose which fields to include, exclude and change. * **Parameters:** * **include** – fields to include in new model * **exclude** – fields to exclude from new model, as with values this takes precedence over include * **update** – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data * **deep** – set to True to make a deep copy of the model * **Returns:** new model instance #### described_by *: [DimensionDescriptor](#pandasdmx.model.DimensionDescriptor) | None* #### get_values() #### order(value=None) #### values *: [DictLike](#pandasdmx.util.DictLike)[str, [KeyValue](#pandasdmx.model.KeyValue)]* Individual KeyValues that describe the key. ``` -------------------------------- ### Retrieve dataflows using Request.get() Source: https://context7.com/dr-leo/pandasdmx/llms.txt Fetch all dataflow definitions from a source. The response can be inspected for specific dataflows or converted into a pandas Series for easier handling. ```python import pandasdmx as sdmx ecb = sdmx.Request('ECB') # --- Retrieve all dataflow definitions --- flow_msg = ecb.get('dataflow') # Inspect the DictLike of DataflowDefinition objects print(flow_msg.dataflow['EXR']) # Convert to pandas Series of dataflow names flows = sdmx.to_pandas(flow_msg.dataflow) print(flows.head()) ``` -------------------------------- ### Create a Key or Subclass Instance Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md make_key is used to create instances of Key, SeriesKey, or GroupKey. It takes the key class, a dictionary of values, and optional parameters for extending dimensions or specifying group IDs. ```python key = dsd.make_key(Key, {'dim1': 'val1', 'dim2': 'val2'}) ``` -------------------------------- ### Convert SDMX to Excel Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/howto.md Reads an SDMX message and converts it to an Excel file. Requires the `pandas` library. ```python msg = sdmx.read_sdmx('data.xml') sdmx.to_pandas(msg).to_excel('data.xlsx') ``` -------------------------------- ### Run pytest tests Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/install.md Execute the test suite using pytest from the package directory. ```bash $ pytest ``` -------------------------------- ### Request.preview_data() - Explore series keys Source: https://context7.com/dr-leo/pandasdmx/llms.txt Returns all SeriesKey objects matching a given key without downloading observations. This is useful for counting and inspecting available series before a full data query. ```APIDOC ## Request.preview_data() ### Description Returns all `SeriesKey` objects matching a given key, without downloading observations. Useful for counting and inspecting available series before a full query. ### Usage ```python import pandasdmx as sdmx ecb = sdmx.Request('ECB') # All series keys for the EXR dataflow keys = ecb.preview_data('EXR') print(len(keys)) # number of series # Filter by key usd_keys = ecb.preview_data('EXR', key={'CURRENCY': 'USD'}) # Convert key values to DataFrame keys_df = sdmx.to_pandas(usd_keys) print(keys_df) # FREQ CURRENCY CURRENCY_DENOM EXR_TYPE EXR_SUFFIX # 0 D USD EUR SP A # ... ``` ``` -------------------------------- ### pandasdmx.add_source() Source: https://context7.com/dr-leo/pandasdmx/llms.txt Registers a custom SDMX data source at runtime using a JSON descriptor. Existing sources can be overridden. ```APIDOC ## `pandasdmx.add_source()` — Register a custom data source Adds a new SDMX data source at runtime using a JSON descriptor. `override=True` replaces an existing source with the same ID. ### Parameters - **source_descriptor**: A JSON string or dictionary describing the data source. - **override** (bool, optional): If True, replaces an existing source with the same ID. Defaults to False. ### Request Example ```python import pandasdmx as sdmx # Add a custom source from a JSON string sdmx.add_source({ "id": "MY_SOURCE", "url": "https://sdmx.myorg.example/rest", "name": "My Organisation SDMX Service", "documentation": "https://myorg.example/sdmx-docs", "supported": {"codelist": true, "preview": true} }) # Confirm it is registered print('MY_SOURCE' in sdmx.list_sources()) # True # Use the new source req = sdmx.Request('MY_SOURCE') flows = req.get('dataflow') # Override an existing source definition sdmx.add_source({ "id": "ECB", "url": "https://sdw-wsrest.ecb.europa.eu/service", "name": "ECB (custom)" }, override=True) ``` ### Response None. This function modifies the internal state of the pandasdmx library. ``` -------------------------------- ### Request.get() - Retrieve data or metadata Source: https://context7.com/dr-leo/pandasdmx/llms.txt The universal method for fetching any SDMX resource. It supports various resource types and allows filtering and parameter customization. Returns a Message object or a requests.Request if dry_run is True. ```APIDOC ## Request.get() ### Description The universal method for fetching any SDMX resource. `resource_type` can be a string like `'dataflow'`, `'datastructure'`, `'codelist'`, `'data'`, etc., or a `pandasdmx.Resource` enum member. Returns a `Message` (or a `requests.Request` if `dry_run=True`). ### Usage ```python import pandasdmx as sdmx ecb = sdmx.Request('ECB') # --- Retrieve all dataflow definitions --- flow_msg = ecb.get('dataflow') # Inspect the DictLike of DataflowDefinition objects print(flow_msg.dataflow['EXR']) # Convert to pandas Series of dataflow names flows = sdmx.to_pandas(flow_msg.dataflow) print(flows.head()) # --- Retrieve full metadata for a single dataflow (DSD + codelists + constraints) --- exr_msg = ecb.get('dataflow', 'EXR', params={'references': 'all'}) ds d = exr_msg.structure['ECB_EXR1'] print(dsd.dimensions.components) # list of Dimension objects # --- Explore a codelist --- cl = dsd.dimensions.get('FREQ').local_representation.enumerated freq_series = sdmx.to_pandas(cl) print(freq_series) # A Annual # D Daily # M Monthly ... # --- Query data with a key dict (validated against DSD) --- data_msg = ecb.get( 'data', 'EXR', key={'CURRENCY': ['USD', 'JPY'], 'FREQ': 'D'}, params={'startPeriod': '2020-01-01'}, dsd=dsd, ) print(data_msg.data[0]) # first DataSet # --- Save response to file while parsing --- data_msg2 = ecb.get( 'data', 'EXR', key={'CURRENCY': 'USD'}, params={'startPeriod': '2023'}, tofile='ecb_exr_usd.xml', ) # --- Dry run: inspect the URL without sending the request --- req_obj = ecb.get('data', 'EXR', key={'CURRENCY': 'USD'}, dry_run=True) print(req_obj.url) ``` ``` -------------------------------- ### Validate SDMX data from a local file Source: https://context7.com/dr-leo/pandasdmx/llms.txt Use the `validate` method to check an XML file against a schema. Ensure the file is opened in binary read mode. ```python with open('ecb_exr.xml', 'rb') as f: result = ecb.validate(f) ``` -------------------------------- ### Query data with references parameter Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/howto.md Use the 'references' query parameter to include related objects in SDMX requests. The default value can be overridden. ```python response = some_agency.dataflow('SOME_ID', params={'references': 'all'}) ``` -------------------------------- ### Reverse Pandas Series using iloc Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/whatsnew.md Demonstrates how to reverse a pandas Series using the standard pandas indexing approach `s.iloc[::-1]`. This is the recommended method in v1.0+. ```python s.iloc[::-1] ``` -------------------------------- ### make_key Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Constructs a Key object (or subclass) based on provided values and key class. ```APIDOC ## make_key(key_cls, values: Mapping, extend=False, group_id=None) ### Description Make a [`Key`](#pandasdmx.model.Key) or subclass. ### Parameters * **key_cls** ([*Key*](#pandasdmx.model.Key) *or* [*SeriesKey*](#pandasdmx.model.SeriesKey) *or* [*GroupKey*](#pandasdmx.model.GroupKey)) – Class of Key to create. * **values** (*dict*) – Used to construct [`Key.values`](#pandasdmx.model.Key.values). * **extend** (*bool* *,* *optional*) – If `True`, make_key will not return `KeyError` on missing dimensions. Instead [`dimensions`](#pandasdmx.model.DataStructureDefinition.dimensions) (key_cls is Key or SeriesKey) or [`group_dimensions`](#pandasdmx.model.DataStructureDefinition.group_dimensions) (key_cls is GroupKey) will be extended by creating new Dimension objects. * **group_id** (*str* *,* *optional*) – When key_cls is :class`.GroupKey`, the ID of the [`GroupDimensionDescriptor`](#pandasdmx.model.GroupDimensionDescriptor) that structures the key. ### Returns An instance of key_cls. ### Return type [Key](#pandasdmx.model.Key) ### Raises **KeyError** – If any of the keys of values is not a Dimension or Attribute in the DSD. ``` -------------------------------- ### Instantiate pandasdmx Request Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Instantiate a pandasdmx.Request object using the string ID of a data source. This object is then ready to make queries to the specified web service. ```python from pandasdmx import Request ecbs = Request('ESTAT') ecb = Request('ECB') ``` -------------------------------- ### ComponentList.get Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Retrieves a component from the list by its ID. ```APIDOC ## get(id) → CT ### Description Return the component with the given *id*. ``` -------------------------------- ### Instantiate Request for Structure-Specific Data Sets Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/whatsnew.md To query structure-specific data sets, instantiate Request objects with agency IDs suffixed with '_S' (e.g., 'ECB_S', 'INSEE_S', 'ESTAT_S') instead of the standard agency IDs. This prompts pandaSDMX to execute queries for these optimized data sets. ```python Request(agency_id='ECB_S') ``` -------------------------------- ### pandasdmx.model.SimpleDatasource Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Represents a simple data source with a URL. ```APIDOC ## Class: pandasdmx.model.SimpleDatasource ### Description Represents a simple data source with a URL. ### Attributes - **url**: The URL of the data source. ``` -------------------------------- ### pandasdmx.remote.ResponseIO Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Wraps a requests.Response object to provide a file-like interface for reading content incrementally, with optional file output for teeing. ```APIDOC ## class pandasdmx.remote.ResponseIO(response, tee=None) ### Description Buffered wrapper for `requests.Response` with optional file output. [`ResponseIO`](#pandasdmx.remote.ResponseIO) wraps a `requests.Response` object’s ‘content’ attribute, providing a file-like object from which bytes can be `read()` incrementally. ### Parameters * **response** (`requests.Response`) – HTTP response to wrap. * **tee** (binary, writable `io.BufferedIOBase`, or `fsspec.core.OpenFile`) – or `io.PathLike`, defaults to io.BytesIO. If *tee* is an open binary file, it is used to store the received data. If *tee* is a PathLike, it is passed to `open()`. *tee* is exposed as *self.tee* and not closed, so this class may be instantiated in a with-context. The latter is also recommended if a `fsspec.core.OpenFile` is passed. ``` -------------------------------- ### InternationalString Initialization and Assignment Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Demonstrates various ways to assign values to an InternationalString field within a Pydantic BaseModel, including direct assignment, using locale-specific strings, tuples, dictionaries, and default locales. Note that only the first method preserves existing localizations. ```python class Foo(BaseModel): name: InternationalString = InternationalString() # Equivalent: no localizations f = Foo() f = Foo(name={}) # Using an explicit locale f.name['en'] = "Foo's name in English" # Using a (locale, label) tuple f.name = ('fr', "Foo's name in French") # Using a dict f.name = {'en': "Replacement English name", 'fr': "Replacement French name"} # Using a bare string, implicitly for the DEFAULT_LOCALE f.name = "Name in DEFAULT_LOCALE language" ``` -------------------------------- ### DimensionComponent Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/api.md Base class for dimension components in SDMX-IM. ```APIDOC ### *class* pandasdmx.model.DimensionComponent #### order *: int | None* ### Description Base class for dimension components in SDMX-IM. ``` -------------------------------- ### Use Custom Session with Request Source: https://github.com/dr-leo/pandasdmx/blob/master/doc/walkthrough.md Pass a pre-configured requests.Session object to the Request constructor to utilize custom adapters or alternative caching libraries. ```python from pandasdmx import Request import requests # Assume my_awesome_session is a pre-configured requests.Session object # For example, with adapters or caching libraries like CacheControl # my_awesome_session = requests.Session() # ... configure my_awesome_session ... # awesome_ecb_req = Request('ECB', session=my_awesome_session) ``` -------------------------------- ### Save response to file Source: https://context7.com/dr-leo/pandasdmx/llms.txt Download data and save the raw response directly to a file using the `tofile` parameter. This is useful for caching or offline processing. ```python import pandasdmx as sdmx ecb = sdmx.Request('ECB') # --- Save response to file while parsing --- data_msg2 = ecb.get( 'data', 'EXR', key={'CURRENCY': 'USD'}, params={'startPeriod': '2023'}, tofile='ecb_exr_usd.xml', ) ```