### Install Powerplantmatching with Pip Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/installation.md Use this command to install the package using pip or uv pip. ```bash pip install powerplantmatching ``` -------------------------------- ### Load Powerplant Data from URL Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/getting-started.md Load the pre-built dataset of power plants directly from a URL into a pandas DataFrame. This is the quickest way to get started. ```python import powerplantmatching as pm pm.powerplants(from_url=True) ``` -------------------------------- ### Download Default Configuration Source: https://context7.com/pypsa/powerplantmatching/llms.txt Download the default configuration file to use as a starting point for customization. This file controls various aspects of the powerplantmatching pipeline, including target countries, fuel types, and data sources. ```bash # Download the default config as a starting point (Linux/macOS) wget -O ~/.powerplantmatching_config.yaml \ https://raw.githubusercontent.com/PyPSA/powerplantmatching/master/powerplantmatching/package_data/config.yaml ``` -------------------------------- ### Import powerplantmatching Library Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Imports the pandas and powerplantmatching libraries. Ensure these libraries are installed before running. ```python import pandas as pd import powerplantmatching as pm ``` -------------------------------- ### Load All Powerplant Data Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/getting-started.md Load all necessary data files and combine them to get the complete dataset. This process might take a few minutes. For the matching process, Java must be installed and in the system's PATH. ```python pm.powerplants(reduced=False) ``` -------------------------------- ### Install Powerplantmatching with Conda Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/installation.md Use this command to install the package using conda or mamba from the conda-forge channel. ```bash conda install -c conda-forge powerplantmatching ``` -------------------------------- ### Load JRC Hydro-power Database Source: https://context7.com/pypsa/powerplantmatching/llms.txt Loads the JRC hydro-power database. Example demonstrates querying for 'Run-Of-River' technology and displaying Name, Capacity, and Country. ```python import powerplantmatching as pm jrc = pm.data.JRC() print(jrc.query("Technology == 'Run-Of-River'")[["Name", "Capacity", "Country"]]) ``` -------------------------------- ### Load German Marktstammdatenregister (MaStR) Data Source: https://context7.com/pypsa/powerplantmatching/llms.txt Loads the German Marktstammdatenregister (MaStR) data. Example shows how to query and sum the total German wind capacity in MW. ```python import powerplantmatching as pm mastr = pm.data.MASTR() print(mastr.query("Fueltype == 'Wind'").Capacity.sum()) # total German wind capacity in MW ``` -------------------------------- ### Load ENTSOE Capacity Statistics for Validation Source: https://context7.com/pypsa/powerplantmatching/llms.txt Loads ENTSOE capacity statistics for validation purposes. Example shows how to access country-level wind capacity totals. ```python import powerplantmatching as pm stats = pm.data.Capacity_stats() print(stats.loc["Wind"]) # country-level wind capacity totals ``` -------------------------------- ### Download Default Configuration Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/custom-config.md Use this command to download the default configuration file to your home directory for customization. ```bash wget -O ~/.powerplantmatching_config.yaml https://raw.githubusercontent.com/PyPSA/powerplantmatching/master/powerplantmatching/package_data/config.yaml ``` -------------------------------- ### Import Individual Power Plant Databases with pm.data.() Source: https://context7.com/pypsa/powerplantmatching/llms.txt Import data from specific sources using `pm.data.()`. Options include `raw=False` for cleaned data, `update=False` to use local cache, and `config=None` for default configuration. Available sources include ENTSOE, OPSD, GEO, and others. ```python import powerplantmatching as pm # Load ENTSOE data (cleaned, filtered to target countries/fueltypes) entsoe = pm.data.ENTSOE() print(entsoe[["Name", "Fueltype", "Capacity", "Country"]].head()) ``` -------------------------------- ### Load and Override Matching Configuration with pm.get_config() Source: https://context7.com/pypsa/powerplantmatching/llms.txt Use `pm.get_config()` to load the default configuration, merge custom configurations, and apply keyword overrides. The returned dictionary governs all downstream functions. A unique hash is generated for each configuration variant to manage cache files. ```python import powerplantmatching as pm # Default configuration config = pm.get_config() print(config["target_countries"]) # ['Albania', 'Austria', 'Belgium', 'Bosnia and Herzegovina', ...] print(config["matching_sources"]) # [{'ENTSOE': "not (Country == 'Germany' and Fueltype == 'Wind')"}, 'GEO', ...] # Override at call time — narrows all subsequent operations config_it = pm.get_config(target_countries=["Italy"]) print(config_it["hash"]) # unique hash for this config variant # Load a custom config file config_custom = pm.get_config(filename="/path/to/my_config.yaml") # Typical use: pass config explicitly throughout the pipeline df = pm.data.OPSD(config=config_it) print(df.Country.unique()) # ['Italy'] ``` -------------------------------- ### pm.data.() — Import individual power plant databases Source: https://context7.com/pypsa/powerplantmatching/llms.txt Each public data source is exposed as a callable in pm.data. All importers follow the same signature: `raw=False` returns the original uncleaned data, `update=False` uses a local cache, and `config=None` uses the default configuration. Available importers include ENTSOE, OPSD, GEO, GPD, JRC, MASTR, BEYONDCOAL, GEM, GCPT, GBPT, IRENASTAT, OSM, GND, EESI, GHR, and Capacity_stats. ```APIDOC ## pm.data.() ### Description Imports individual power plant databases from various public sources. Each source is available as a callable within the `pm.data` module. These functions allow for importing raw or cleaned data, utilizing local caches, and applying configurations. ### Method `pm.data.(raw=False, update=False, config=None)` ### Parameters - `` (str) - The name of the data source to import (e.g., `ENTSOE`, `OPSD`, `GEO`). - `raw` (bool) - Optional - If True, returns the original uncleaned data. Defaults to False. - `update` (bool) - Optional - If True, forces a fresh download/processing of the data, ignoring the local cache. Defaults to False. - `config` (dict) - Optional - A configuration dictionary to use for importing the data. If None, the default configuration is used. ### Request Example ```python import powerplantmatching as pm # Load cleaned ENTSOE data entsoe_data = pm.data.ENTSOE() print(entsoe_data[["Name", "Fueltype", "Capacity", "Country"]].head()) # Load raw OPSD data opsd_raw = pm.data.OPSD(raw=True) # Load GEO data with a specific configuration custom_config = pm.get_config(target_countries=["Spain"]) geo_data = pm.data.GEO(config=custom_config) # Force update for JRC data jrc_updated = pm.data.JRC(update=True) ``` ### Response #### Success Response (pandas.DataFrame) Returns a pandas DataFrame containing the imported power plant data. The structure of the DataFrame depends on the specific data source and whether the `raw` parameter was set to True. #### Response Example ``` # Example output for ENTSOE data # Name Fueltype Capacity Country # Plant A Coal 500.0 Germany # Plant B Gas 750.0 Germany # ... ``` ``` -------------------------------- ### pm.powerplants() — Load or build the full matched dataset Source: https://context7.com/pypsa/powerplantmatching/llms.txt The primary entry point of the package. Returns a cleaned, matched, and reduced pandas.DataFrame of European power plants by either downloading the pre-built CSV directly from the repository or running the full matching pipeline locally. Accepts configuration overrides, optional extension by variable renewable energy sources (VREs), and geo-position filling/filtering controls. ```APIDOC ## pm.powerplants() ### Description Loads or builds the full matched dataset of European power plants. This function can download a pre-built CSV or run the entire matching pipeline locally. It supports configuration overrides and extensions for variable renewable energy sources (VREs). ### Method `pm.powerplants()` ### Parameters - `from_url` (bool) - Optional - If True, downloads the pre-built dataset from the repository. - `update` (bool) - Optional - If True, forces a full re-run of the matching pipeline. - `config_update` (dict) - Optional - A dictionary to override specific configuration settings. - `extend_by_vres` (bool) - Optional - If True, extends the result with OPSD variable-renewables data. ### Request Example ```python import powerplantmatching as pm # Download pre-built dataset df_url = pm.powerplants(from_url=True) # Load from local cache or run matching pipeline df_local = pm.powerplants() # Force re-run df_rerun = pm.powerplants(update=True) # Override config df_override = pm.powerplants(config_update={"target_countries": ["Germany", "France"]}) # Extend with VREs df_vres = pm.powerplants(from_url=True, extend_by_vres=True) ``` ### Response #### Success Response (pandas.DataFrame) Returns a pandas DataFrame containing cleaned, matched, and standardized power plant data with columns such as Name, Fueltype, Technology, Capacity, lat, lon, DateIn, etc. #### Response Example ``` # Example DataFrame head # Name Fueltype Technology Set Country Capacity lat lon DateIn ... # Example dtypes # Name object # Fueltype object # Technology object # Set object # Country object # Capacity float64 # lat float64 # lon float64 # DateIn float64 # DateRetrofit float64 # projectID object # Example aggregated capacity by Fueltype # Fueltype # Natural Gas 250000.0 # Wind 180000.0 # Hard Coal 95000.0 # Nuclear 80000.0 # ... ``` ``` -------------------------------- ### Inspect Package Configuration Paths Source: https://context7.com/pypsa/powerplantmatching/llms.txt Prints the module-level dictionary holding runtime paths for configuration files, data directories, and bundled package data. These paths can be inspected or overridden. ```python import powerplantmatching as pm print(pm.package_config) # { # 'custom_config': '/home/user/.powerplantmatching_config.yaml', # 'data_dir': '/home/user/.local/share/powerplantmatching', # 'repo_data_dir': '/path/to/site-packages/powerplantmatching/package_data', # 'downloaders': {} # } ``` -------------------------------- ### pm.get_config() — Load and override the matching configuration Source: https://context7.com/pypsa/powerplantmatching/llms.txt Reads the default config.yaml bundled with the package, optionally merges a user-defined ~/.powerplantmatching_config.yaml, and applies any keyword overrides. Returns a dictionary that governs every downstream function: target countries, fuel types, data sources, matching logic, API tokens, and output file locations. ```APIDOC ## pm.get_config() ### Description Loads and optionally overrides the matching configuration for the powerplantmatching package. This function returns a dictionary that controls various aspects of the data processing pipeline, including target countries, fuel types, data sources, and matching logic. ### Method `pm.get_config()` ### Parameters - `filename` (str) - Optional - Path to a custom configuration YAML file to load. - `target_countries` (list) - Optional - A list of country names to filter the configuration. - `config_update` (dict) - Optional - A dictionary of key-value pairs to override settings in the loaded configuration. ### Request Example ```python import powerplantmatching as pm # Load default configuration config_default = pm.get_config() print(config_default["target_countries"]) # Load configuration and override target countries config_italy = pm.get_config(target_countries=["Italy"]) print(config_italy["hash"]) # Load configuration from a custom file config_custom = pm.get_config(filename="/path/to/my_config.yaml") # Use the configuration in other functions df_italy = pm.data.OPSD(config=config_italy) ``` ### Response #### Success Response (dict) Returns a dictionary representing the merged and overridden configuration. This dictionary includes settings for target countries, fuel types, matching sources, API tokens, and output file locations. A unique hash representing the configuration is also returned. #### Response Example ``` # Example default config output for target_countries # ['Albania', 'Austria', 'Belgium', 'Bosnia and Herzegovina', ...] # Example config output for matching_sources # [{'ENTSOE': "not (Country == 'Germany' and Fueltype == 'Wind')"}, 'GEO', ...] # Example unique hash for a specific config variant # 'a1b2c3d4e5f6...' ``` ``` -------------------------------- ### Load or Build Full Matched Dataset with pm.powerplants() Source: https://context7.com/pypsa/powerplantmatching/llms.txt Use `pm.powerplants()` to download a pre-built dataset or run the local matching pipeline. Configuration overrides and extensions for variable renewable energy sources are supported. The function returns a pandas DataFrame with standardized plant attributes. ```python import powerplantmatching as pm # Fastest: download the pre-built dataset from the repository df = pm.powerplants(from_url=True) print(df.head()) # Name Fueltype Technology Set Country Capacity lat lon DateIn ... # Load from local cache (default) or trigger a fresh matching run df = pm.powerplants() # reads from cache if already built # Force a full re-run of the matching pipeline df = pm.powerplants(update=True) # Override config: restrict to specific countries df = pm.powerplants(config_update={"target_countries": ["Germany", "France"]}) # Extend the result with OPSD variable-renewables data df = pm.powerplants(from_url=True, extend_by_vres=True) print(df.dtypes) # Name object # Fueltype object # Technology object # Set object # Country object # Capacity float64 # lat float64 # lon float64 # DateIn float64 # DateRetrofit float64 # projectID object print(df.groupby("Fueltype")["Capacity"].sum().sort_values(ascending=False)) # Fueltype # Natural Gas 250000.0 # Wind 180000.0 # Hard Coal 95000.0 # Nuclear 80000.0 # ... ``` -------------------------------- ### Create Capacity Pivot Table with `pm.utils.lookup()` Source: https://context7.com/pypsa/powerplantmatching/llms.txt Generates a rounded pivot table of total capacity grouped by Country and/or Fueltype. Supports units in MW or GW, comparison of multiple datasets, and exclusion of specific fuel types. Can be accessed via accessor. ```python import powerplantmatching as pm df = pm.powerplants(from_url=True) # Total capacity by country and fuel type (MW, default) table = pm.utils.lookup(df) print(table.head()) # Germany France Italy ... # Fueltype # Hard Coal 21000 2500 500 ... # Natural Gas 30000 15000 40000 ... ``` ```python # In GW table_gw = pm.utils.lookup(df, unit="GW") # Group only by Country by_country = pm.utils.lookup(df, by="Country") ``` ```python # Compare multiple datasets entsoe = pm.data.ENTSOE() opsd = pm.data.OPSD() comparison = pm.utils.lookup([entsoe, opsd], keys=["ENTSOE", "OPSD"], by="Country") print(comparison) # MultiIndex columns: (source, country) ``` ```python # Exclude specific fuel types no_solar = pm.utils.lookup(df, exclude=["Solar", "Wind"]) # Via accessor (equivalent) table2 = df.powerplant.lookup(by="Country, Fueltype") ``` -------------------------------- ### Cite Powerplantmatching Project (BibTeX) Source: https://github.com/pypsa/powerplantmatching/blob/master/README.md Use this BibTeX entry to cite the powerplantmatching project in academic publications. Ensure you also cite the Zenodo release for reproducibility. ```bibtex @article{gotzens_performing_2019, title = {Performing energy modelling exercises in a transparent way - {The} issue of data quality in power plant databases}, volume = {23}, issn = {2211467X}, url = {https://linkinghub.elsevier.com/retrieve/pii/S2211467X18301056}, doi = {10.1016/j.esr.2018.11.004}, language = {en}, urldate = {2018-12-03}, journal = {Energy Strategy Reviews}, author = {Gotzens, Fabian and Heinrichs, Heidi and Hörsch, Jonas and Hofmann, Fabian}, month = jan, year = {2019}, pages = {1--12} } ``` -------------------------------- ### Load Collection of Powerplants Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Load a collection of powerplants using `pm.collection.powerplants()`. Note: `matched_data` is deprecated and `powerplants` should be used instead. ```python m = pm.collection.powerplants() ``` -------------------------------- ### Custom Configuration Source: https://context7.com/pypsa/powerplantmatching/llms.txt Control the powerplantmatching pipeline using a YAML configuration file or programmatic overrides. ```APIDOC ## Custom Configuration via `~/.powerplantmatching_config.yaml` The entire pipeline is controlled by a YAML configuration file. Users can override defaults by placing a modified copy at `~/.powerplantmatching_config.yaml`. Key configurable sections include target countries, fuel types, matching and fully-included sources (with per-source pandas query filters), API credentials, and display options. ```bash # Download the default config as a starting point (Linux/macOS) wget -O ~/.powerplantmatching_config.yaml \ https://raw.githubusercontent.com/PyPSA/powerplantmatching/master/powerplantmatching/package_data/config.yaml ``` ```yaml # ~/.powerplantmatching_config.yaml # Restrict processing to a subset of countries target_countries: - Germany - France - Italy - Spain - Poland # Add ENTSOE API token for live data retrieval entsoe_token: "YOUR_TOKEN_HERE" # Add Google Maps API key for geocoding missing coordinates google_api_key: "YOUR_KEY_HERE" # Adjust matching sources — only use ENTSOE and MASTR matching_sources: - ENTSOE - MASTR # Specify sources to include fully (even without a match) fully_included_sources: - GEM - MASTR: "Fueltype != 'Nuclear'" ``` ```python import powerplantmatching as pm # The custom config is picked up automatically config = pm.get_config() print(config["target_countries"]) # ['Germany', 'France', 'Italy', 'Spain', 'Poland'] # Or load programmatically with keyword overrides (takes precedence over file) config = pm.get_config( target_countries=["Austria", "Switzerland"], display_net_caps=True, ) df = pm.powerplants(config=config, update=True) ``` ``` -------------------------------- ### Build Matched Collection with collect() Source: https://context7.com/pypsa/powerplantmatching/llms.txt Runs the full multi-source matching pipeline, cleaning and aggregating each source before matching. Results are cached. Use `reduced=False` to return the full multi-indexed matched form. ```python import powerplantmatching as pm from powerplantmatching.collection import collect # Combine ENTSOE, OPSD, and GEO into a matched dataset matched = collect(["ENTSOE", "OPSD", "GEO"], update=True) print(f"Matched plants: {len(matched)}") print(matched[["Name", "Fueltype", "Capacity", "Country"]].head()) # Return the full multi-indexed matched form (not reduced) matched_full = collect(["ENTSOE", "OPSD"], reduced=False) print(matched_full.columns) # MultiIndex: (attribute, source) # Use cached result (no re-computation) matched_cached = collect(["ENTSOE", "OPSD", "GEO"]) # Pass a custom config to restrict to a single country config = pm.get_config(target_countries=["Spain"]) matched_es = collect(["ENTSOE", "OPSD"], config=config, update=True) ``` -------------------------------- ### Customize Powerplantmatching Configuration Source: https://context7.com/pypsa/powerplantmatching/llms.txt Users can override default settings by placing a modified `~/.powerplantmatching_config.yaml` file in their home directory. The configuration controls target countries, API credentials, data sources, and display options. Configurations can also be loaded programmatically with keyword overrides. ```yaml # ~/.powerplantmatching_config.yaml # Restrict processing to a subset of countries target_countries: - Germany - France - Italy - Spain - Poland # Add ENTSOE API token for live data retrieval entsoe_token: "YOUR_TOKEN_HERE" # Add Google Maps API key for geocoding missing coordinates google_api_key: "YOUR_KEY_HERE" # Adjust matching sources — only use ENTSOE and MASTR matching_sources: - ENTSOE - MASTR # Specify sources to include fully (even without a match) fully_included_sources: - GEM - MASTR: "Fueltype != 'Nuclear'" ``` ```python import powerplantmatching as pm # The custom config is picked up automatically config = pm.get_config() print(config["target_countries"]) # ['Germany', 'France', 'Italy', 'Spain', 'Poland'] # Or load programmatically with keyword overrides (takes precedence over file) config = pm.get_config( target_countries=["Austria", "Switzerland"], display_net_caps=True, ) df = pm.powerplants(config=config, update=True) ``` -------------------------------- ### Find Cached Data File Locations Source: https://context7.com/pypsa/powerplantmatching/llms.txt Constructs paths to raw input and processed output data directories based on the package configuration's data directory. Requires the `os` module. ```python import os import powerplantmatching as pm data_in = os.path.join(pm.package_config["data_dir"], "data", "in") data_out = os.path.join(pm.package_config["data_dir"], "data", "out") print("Raw inputs:", os.listdir(data_in)) print("Processed outputs:", os.listdir(data_out)) ``` -------------------------------- ### collect Source: https://context7.com/pypsa/powerplantmatching/llms.txt Runs the full multi-source matching pipeline for a specified list of data sources. Each source is individually cleaned and aggregated, then all pairs are matched via Duke, and a reduced (best-estimate) DataFrame is returned. Results are cached to disk and reused unless `update=True` is passed. ```APIDOC ## collect() ### Description Runs the full multi-source matching pipeline for a specified list of data sources. Each source is individually cleaned and aggregated, then all pairs are matched via Duke, and a reduced (best-estimate) DataFrame is returned. Results are cached to disk and reused unless `update=True` is passed. ### Usage ```python import powerplantmatching as pm from powerplantmatching.collection import collect # Combine ENTSOE, OPSD, and GEO into a matched dataset matched = collect(["ENTSOE", "OPSD", "GEO"], update=True) print(f"Matched plants: {len(matched)}") print(matched[["Name", "Fueltype", "Capacity", "Country"]].head()) # Return the full multi-indexed matched form (not reduced) matched_full = collect(["ENTSOE", "OPSD"], reduced=False) print(matched_full.columns) # MultiIndex: (attribute, source) # Use cached result (no re-computation) matched_cached = collect(["ENTSOE", "OPSD", "GEO"]) # Pass a custom config to restrict to a single country config = pm.get_config(target_countries=["Spain"]) matched_es = collect(["ENTSOE", "OPSD"], config=config, update=True) ``` ``` -------------------------------- ### Load IRENA Capacity Statistics Source: https://context7.com/pypsa/powerplantmatching/llms.txt Loads capacity statistics from the IRENA dataset. ```python import powerplantmatching as pm irena = pm.data.IRENASTAT() ``` -------------------------------- ### Calculate Total Capacity and Unique Technologies Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Calculates the total capacity and lists the unique technology types present in the power plant dataset. Use this to understand the energy sources and generation capacities. ```python print(f"Total capacity of GEO is: \n {geo.Capacity.sum()} MW \n") print(f"The technology types are: \n {geo.Technology.unique()} ") ``` -------------------------------- ### Rescale Capacities to Country Totals with `rescale_capacities_to_country_totals` Source: https://context7.com/pypsa/powerplantmatching/llms.txt Scales aggregated capacity in the dataset to match ENTSOE national statistics for each country and fuel type. Useful for aligning bottom-up data with top-down national totals. Can scale all fuel types or specific ones. ```python import powerplantmatching as pm from powerplantmatching.heuristics import rescale_capacities_to_country_totals df = pm.powerplants(from_url=True) # Scale all fuel types df_scaled = rescale_capacities_to_country_totals(df) print(df_scaled[["Country", "Fueltype", "Capacity", "Scaled Capacity"]].head(10)) ``` ```python # Scale only specific fuel types df_gas_scaled = rescale_capacities_to_country_totals(df, fueltypes=["Natural Gas"]) de_gas = df_gas_scaled.query("Country == 'Germany' and Fueltype == 'Natural Gas'") print(f"Raw total: {de_gas.Capacity.sum():.0f} MW") print(f"Scaled total: {de_gas['Scaled Capacity'].sum():.0f} MW") ``` ```python # Or use the accessor df_scaled2 = df.powerplant.rescale_capacities_to_country_totals( fueltypes=["Hard Coal", "Lignite"] ) ``` -------------------------------- ### Cross-match Two Datasets with compare_two_datasets() Source: https://context7.com/pypsa/powerplantmatching/llms.txt Cross-matches entries in two power plant databases using the Duke record-linkage engine. Comparisons are made country-by-country by default. Disable `country_wise` for global matching on smaller datasets. ```python import powerplantmatching as pm from powerplantmatching.matching import compare_two_datasets from powerplantmatching.cleaning import aggregate_units entsoe = aggregate_units(pm.data.ENTSOE(), dataset_name="ENTSOE") opsd = aggregate_units(pm.data.OPSD(), dataset_name="OPSD") # Match ENTSOE vs OPSD — country-wise, one-to-one links matched = compare_two_datasets([entsoe, opsd], labels=["ENTSOE", "OPSD"]) print(matched.columns.tolist()) # Show the top matched pairs with highest similarity score print(matched.nlargest(10, "scores")骢["ENTSOE", "OPSD", "scores"]) # Disable country-wise matching (slower, for small datasets) matched_global = compare_two_datasets([entsoe, opsd], labels=["ENTSOE", "OPSD"], country_wise=False) ``` -------------------------------- ### Aggregate Power Plant Units and Combine Datasets Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Aggregates power plant units within each dataset and then combines them into a single dataset, handling intersections. This ensures data consistency before merging. ```python dfs = [geo.powerplant.aggregate_units(), entsoe.powerplant.aggregate_units()] intersection = pm.matching.combine_multiple_datasets(dfs) ``` -------------------------------- ### Match Powerplant Data with ENTSOE Source: https://context7.com/pypsa/powerplantmatching/llms.txt Matches the current powerplant DataFrame against the ENTSOE dataset using the `.powerplant.match_with()` method. `reduced=True` provides a reduced output. ```python entsoe = pm.data.ENTSOE() matched = df.powerplant.match_with(entsoe, reduced=True) ``` -------------------------------- ### Display DataFrame Head Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Displays the first few rows of a pandas DataFrame. Useful for a quick overview of the data. ```python m.head() ``` -------------------------------- ### Inspect Power Plant Data Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Displays the first few rows of the power plant dataset after filling missing commissioning years. This is useful for a quick overview of the data. ```python geo.powerplant.fill_missing_commissioning_years().head() ``` -------------------------------- ### Export to PyPSA format Source: https://context7.com/pypsa/powerplantmatching/llms.txt Rename DataFrame columns to match PyPSA conventions and map power plants to network buses. ```APIDOC ## `pm.export.to_pypsa_names()` / `map_bus()` — Export to PyPSA format `to_pypsa_names` renames DataFrame columns to match PyPSA's component naming conventions (`Fueltype → carrier`, `Capacity → p_nom`, `Duration → max_hours`, `Set → component`). `map_bus` and `map_country_bus` assign each power plant to its nearest bus in a network topology using a KD-tree spatial lookup. Together they prepare powerplant data for direct ingestion into a PyPSA network. ```python import powerplantmatching as pm from powerplantmatching.export import to_pypsa_names, map_bus, map_country_bus import pandas as pd df = pm.powerplants(from_url=True) # Rename to PyPSA column conventions df_pypsa = to_pypsa_names(df) print(df_pypsa[["carrier", "p_nom", "component"]].head()) # carrier p_nom component # natural gas 800.0 PP # Suppose you have a PyPSA bus DataFrame with columns x (lon), y (lat) buses = pd.DataFrame({ "x": [13.4, 2.35, 12.48], "y": [52.5, 48.85, 41.90] }, index=["Berlin", "Paris", "Rome"]) df_with_bus = map_bus(df, buses) print(df_with_bus[["Name", "bus"]].head()) # Country-aware bus assignment (buses also have a 'country' column) buses_c = buses.assign(country=["Germany", "France", "Italy"]) df_country_bus = map_country_bus(df, buses_c) # Full export to a pypsa.Network object # import pypsa # network = pypsa.Network() # network.import_from_csv_folder("my_network/") # df.powerplant.to_pypsa_names().pipe( # lambda d: pm.export.to_pypsa_network(d, network) # ) ``` ``` -------------------------------- ### Load Raw ENTSOE Data Source: https://context7.com/pypsa/powerplantmatching/llms.txt Loads raw ENTSOE data without any cleaning. Use `update=True` to force a re-download from the source URL. ```python import powerplantmatching as pm entsoe_raw = pm.data.ENTSOE(raw=True) entsoe_fresh = pm.data.ENTSOE(update=True) ``` -------------------------------- ### Load Beyond Coal/Fossil Fuels Data Source: https://context7.com/pypsa/powerplantmatching/llms.txt Loads coal plant data from the Beyond Coal/Fossil Fuels dataset. ```python import powerplantmatching as pm beyond = pm.data.BEYONDCOAL() ``` -------------------------------- ### BibTeX Entry for powerplantmatching Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/citing.md Use this BibTeX entry to cite the powerplantmatching project in academic papers. It provides all the necessary metadata for referencing the associated publication. ```bibtex @article{gotzens_performing_2019, title = {Performing energy modelling exercises in a transparent way - {The} issue of data quality in power plant databases}, volume = {23}, issn = {2211467X}, url = {https://linkinghub.elsevier.com/retrieve/pii/S2211467X18301056}, doi = {10.1016/j.esr.2018.11.004}, language = {en}, urldate = {2018-12-03}, journal = {Energy Strategy Reviews}, author = {Gotzens, Fabian and Heinrichs, Heidi and Hörsch, Jonas and Hofmann, Fabian}, month = jan, year = {2019}, pages = {1--12} } ``` -------------------------------- ### compare_two_datasets Source: https://context7.com/pypsa/powerplantmatching/llms.txt Uses the Duke probabilistic record-linkage engine to identify entries in two power plant databases that refer to the same physical plant. Comparisons are made country-by-country on plant name, fuel type, technology, country, capacity, and geo-coordinates. Returns a multi-indexed DataFrame of matched pairs with their similarity scores. ```APIDOC ## compare_two_datasets() ### Description Uses the Duke probabilistic record-linkage engine to identify entries in two power plant databases that refer to the same physical plant. Comparisons are made country-by-country on plant name, fuel type, technology, country, capacity, and geo-coordinates. Returns a multi-indexed DataFrame of matched pairs with their similarity scores. ### Usage ```python import powerplantmatching as pm from powerplantmatching.matching import compare_two_datasets from powerplantmatching.cleaning import aggregate_units entsoe = aggregate_units(pm.data.ENTSOE(), dataset_name="ENTSOE") opsd = aggregate_units(pm.data.OPSD(), dataset_name="OPSD") # Match ENTSOE vs OPSD — country-wise, one-to-one links matched = compare_two_datasets([entsoe, opsd], labels=["ENTSOE", "OPSD"]) print(matched.columns.tolist()) # [('Name', 'ENTSOE'), ('Name', 'OPSD'), ('Fueltype', 'ENTSOE'), ..., 'scores'] # Show the top matched pairs with highest similarity score print(matched.nlargest(10, "scores")骢["ENTSOE", "OPSD", "scores"]) # Disable country-wise matching (slower, for small datasets) matched_global = compare_two_datasets([entsoe, opsd], labels=["ENTSOE", "OPSD"], country_wise=False) ``` ``` -------------------------------- ### Load Global Energy Observatory (GEO) Data Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Loads the open-source power plant data from the Global Energy Observatory (GEO) into a pandas DataFrame. This data is standardized for use with powerplantmatching. ```python geo = pm.data.GEO() geo.head() ``` -------------------------------- ### Gather Specifications from Raw Data Source: https://context7.com/pypsa/powerplantmatching/llms.txt Parses and gathers specifications from raw DataFrame. Ensure target and parse columns are correctly specified. ```python import pandas as pd import powerplantmatching as pm raw = pd.DataFrame({ "Name": ["GuD Berlin", "Pumpspeicher Lac Noir", "Solar Park Freiburg"], "Fueltype": ["gas", "hydro", "solar"], "Technology": ["combined cycle", "pump storage", None], "Set": [None, None, None], }) config = pm.get_config() result = pm.gather_specifications( raw, target_columns=["Fueltype", "Technology", "Set"], parse_columns=["Name", "Fueltype", "Technology", "Set"], config=config, ) print(result[["Name", "Fueltype", "Technology", "Set"]]) ``` -------------------------------- ### Aggregate Plant Units with aggregate_units() Source: https://context7.com/pypsa/powerplantmatching/llms.txt Aggregates rows corresponding to individual generating units into single plant entries. Use `threads` for faster processing and `pre_clean_name` to clean names before aggregation. ```python import powerplantmatching as pm from powerplantmatching.cleaning import aggregate_units # Load raw MASTR data (unit-level) mastr_raw = pm.data.MASTR(raw=False) # already cleaned but still unit-level print(f"Before aggregation: {len(mastr_raw)} units") # Aggregate units belonging to the same plant mastr_agg = aggregate_units(mastr_raw, dataset_name="MASTR") print(f"After aggregation: {len(mastr_agg)} plants") # Use multiple threads for faster processing mastr_agg_fast = aggregate_units(mastr_raw, dataset_name="MASTR", threads=4) # Pre-clean names before aggregation mastr_agg_clean = aggregate_units(mastr_raw, dataset_name="MASTR", pre_clean_name=True) print(mastr_agg_clean[["Name", "Fueltype", "Capacity", "Country"]].head()) ``` -------------------------------- ### Load Powerplant Data Source: https://context7.com/pypsa/powerplantmatching/llms.txt Loads powerplant data from a URL. This is a prerequisite for using the `.powerplant` accessor methods. ```python import powerplantmatching as pm df = pm.powerplants(from_url=True) ``` -------------------------------- ### Load ENTSOE Data Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Loads power plant data from the ENTSOE transparency platform. This data is also standardized to match the GEO format. ```python entsoe = pm.data.ENTSOE() entsoe.head() ``` -------------------------------- ### Plot Aggregated Capacity Overview Source: https://context7.com/pypsa/powerplantmatching/llms.txt Generates a bar chart overview of aggregated capacity grouped by country and fuel type using `.powerplant.plot_aggregated()`. Saves the figure to a PNG file. ```python fig, ax = df.powerplant.plot_aggregated(by=["Country", "Fueltype"], figsize=(14, 22)) fig.savefig("capacity_by_country_fueltype.png", bbox_inches="tight") ``` -------------------------------- ### Inspect Combined Dataset Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Displays the first few rows of the combined dataset after aggregating units and handling intersections. This is useful for verifying the merging process. ```python intersection.head() ``` -------------------------------- ### Lookup and Aggregate Power Plant Capacity by Country and Fueltype Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Utilizes the 'powerplant' accessor to perform a lookup and aggregate the total capacity of power plants by country and fuel type. Displays the first 20 results. ```python geo.powerplant.lookup().head(20).to_frame() ``` -------------------------------- ### Extend Dataset with Non-Matched Plants using `extend_by_non_matched` Source: https://context7.com/pypsa/powerplantmatching/llms.txt Appends unmatched entries from a specified data source to a matched DataFrame. Useful for ensuring complete coverage. Can optionally filter entries before appending and control aggregation of added data. ```python import powerplantmatching as pm from powerplantmatching.heuristics import extend_by_non_matched df_matched = pm.collection.collect(["ENTSOE", "OPSD", "GEO"]) # Extend with all GEM entries not already matched df_extended = extend_by_non_matched(df_matched, "GEM") print(f"Before extension: {len(df_matched)} | After: {len(df_extended)}") ``` ```python # Extend with a filtered subset of MASTR df_extended2 = extend_by_non_matched( df_matched, "MASTR", query="Fueltype != 'Nuclear'", # exclude nuclear from extension ) ``` ```python # Extend without re-aggregating the added data df_raw_ext = extend_by_non_matched(df_matched, "BEYONDCOAL", aggregate_added_data=False) ``` -------------------------------- ### Compare Factors Between Datasets Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Compare factors between the processed data and statistics data using `factor_comparison`. This function helps in analyzing differences in various factors. ```python pm.plot.factor_comparison([m, stats], keys=["Processed", "Statistics"]) ``` -------------------------------- ### Clean Plant Names with Custom Configuration Source: https://context7.com/pypsa/powerplantmatching/llms.txt Uses `clean_name` with a custom configuration to preserve block numbers for specific fuel types, such as 'Nuclear'. ```python import powerplantmatching as pm from powerplantmatching.cleaning import clean_name opsd = pm.data.OPSD(raw=True) config = pm.get_config() config["clean_name"]["fueltypes_with_blocks"] = ["Nuclear"] cleaned_nuclear = clean_name(opsd, config=config) ``` -------------------------------- ### Fill Missing Commissioning Years Source: https://context7.com/pypsa/powerplantmatching/llms.txt Fills missing commissioning years in the DataFrame by using fuel and country averages via the `.powerplant.fill_missing_commissioning_years()` method. ```python df_dates = df.powerplant.fill_missing_commissioning_years() ``` -------------------------------- ### Load Capacity Statistics Data Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Loads capacity statistics data. This is a prerequisite for further analysis and comparison with other power plant datasets. ```python stats = pm.data.Capacity_stats() ``` -------------------------------- ### Export Powerplant Data to PyPSA Format Source: https://context7.com/pypsa/powerplantmatching/llms.txt Use `to_pypsa_names` to rename DataFrame columns to PyPSA conventions. `map_bus` and `map_country_bus` assign power plants to the nearest network bus using spatial lookups. This prepares data for direct ingestion into a PyPSA network. ```python import powerplantmatching as pm from powerplantmatching.export import to_pypsa_names, map_bus, map_country_bus import pandas as pd df = pm.powerplants(from_url=True) # Rename to PyPSA column conventions df_pypsa = to_pypsa_names(df) print(df_pypsa[["carrier", "p_nom", "component"]].head()) # carrier p_nom component # natural gas 800.0 PP # Suppose you have a PyPSA bus DataFrame with columns x (lon), y (lat) buses = pd.DataFrame({ "x": [13.4, 2.35, 12.48], "y": [52.5, 48.85, 41.90] }, index=["Berlin", "Paris", "Rome"]) df_with_bus = map_bus(df, buses) print(df_with_bus[["Name", "bus"]].head()) # Country-aware bus assignment (buses also have a 'country' column) buses_c = buses.assign(country=["Germany", "France", "Italy"]) df_country_bus = map_country_bus(df, buses_c) # Full export to a pypsa.Network object # import pypsa # network = pypsa.Network() # network.import_from_csv_folder("my_network/") # df.powerplant.to_pypsa_names().pipe( # lambda d: pm.export.to_pypsa_network(d, network) # ) ``` -------------------------------- ### Geographic visualization of power plants Source: https://context7.com/pypsa/powerplantmatching/llms.txt Renders a geographic scatter plot of power plants, optionally with Cartopy projections. ```APIDOC ## `pm.plot.powerplant_map()` — Geographic visualization of power plants Renders a geographic scatter plot of all power plants sized by capacity and colored by fuel type. Supports optional Cartopy projections for proper cartographic rendering. Also available via the `.powerplant.plot_map()` accessor method. ```python import powerplantmatching as pm import matplotlib.pyplot as plt df = pm.powerplants(from_url=True) # Full map of European power plants (requires cartopy for projection) fig, ax = pm.plot.powerplant_map(df, scale=20, alpha=0.7, figsize=(14, 10)) fig.savefig("european_powerplants.png", dpi=150, bbox_inches="tight") # Fuel-type capacity pie chart pm.plot.fueltype_stats(df) plt.savefig("fueltype_distribution.png") plt.close() # Filter to a single country before plotting de = df.query("Country == 'Germany'") fig, ax = pm.plot.powerplant_map(de, european_bounds=False, figsize=(8, 8)) # Quick aggregated bar chart via accessor fig, ax = df.powerplant.plot_aggregated(by=["Country", "Fueltype"]) fig.tight_layout() plt.show() ``` ``` -------------------------------- ### Visualize Power Plants Geographically Source: https://context7.com/pypsa/powerplantmatching/llms.txt Use `powerplant_map` to render a geographic scatter plot of power plants, sized by capacity and colored by fuel type. Supports Cartopy projections for accurate cartographic rendering. `fueltype_stats` creates a pie chart of fuel type distribution. ```python import powerplantmatching as pm import matplotlib.pyplot as plt df = pm.powerplants(from_url=True) # Full map of European power plants (requires cartopy for projection) fig, ax = pm.plot.powerplant_map(df, scale=20, alpha=0.7, figsize=(14, 10)) fig.savefig("european_powerplants.png", dpi=150, bbox_inches="tight") # Fuel-type capacity pie chart pm.plot.fueltype_stats(df) plt.savefig("fueltype_distribution.png") plt.close() # Filter to a single country before plotting de = df.query("Country == 'Germany'") fig, ax = pm.plot.powerplant_map(de, european_bounds=False, figsize=(8, 8)) # Quick aggregated bar chart via accessor fig, ax = df.powerplant.plot_aggregated(by=["Country", "Fueltype"]) fig.tight_layout() plt.show() ``` -------------------------------- ### Count Power Plants by Fuel Type and Date Existence Source: https://github.com/pypsa/powerplantmatching/blob/master/docs/examples/example.ipynb Counts power plants based on their fuel type, differentiating between those with and without an existing 'DateIn' value. This helps in understanding data completeness for different fuel types. ```python pd.concat( [ m[m.DateIn.notnull()].groupby("Fueltype").DateIn.count(), m[m.DateIn.isna()].fillna(1).groupby("Fueltype").DateIn.count(), ], keys=["DateIn existent", "DateIn missing"], axis=1, ) ``` -------------------------------- ### extend_by_non_matched Source: https://context7.com/pypsa/powerplantmatching/llms.txt After the matching process, some entries from a "fully included" data source may not have matched anything in the core dataset. This function identifies those unmatched entries, optionally aggregates them, and appends them to the matched DataFrame, ensuring complete coverage from that source. ```APIDOC ## `pm.heuristics.extend_by_non_matched()` — Append non-matched plants from a source After the matching process, some entries from a "fully included" data source may not have matched anything in the core dataset. This function identifies those unmatched entries, optionally aggregates them, and appends them to the matched DataFrame, ensuring complete coverage from that source. ```python import powerplantmatching as pm from powerplantmatching.heuristics import extend_by_non_matched df_matched = pm.collection.collect(["ENTSOE", "OPSD", "GEO"]) # Extend with all GEM entries not already matched df_extended = extend_by_non_matched(df_matched, "GEM") print(f"Before extension: {len(df_matched)} | After: {len(df_extended)}") # Extend with a filtered subset of MASTR df_extended2 = extend_by_non_matched( df_matched, "MASTR", query="Fueltype != 'Nuclear'", # exclude nuclear from extension ) # Extend without re-aggregating the added data df_raw_ext = extend_by_non_matched(df_matched, "BEYONDCOAL", aggregate_added_data=False) ``` ``` -------------------------------- ### lookup Source: https://context7.com/pypsa/powerplantmatching/llms.txt Returns a rounded pivot table of total capacity (in MW or GW) grouped by any combination of `Country` and/or `Fueltype`. Accepts a single DataFrame or a list of DataFrames (with optional keys for column labels in a MultiIndex result), making it easy to compare multiple datasets side by side. ```APIDOC ## `pm.utils.lookup()` — Capacity pivot table by country and/or fuel type Returns a rounded pivot table of total capacity (in MW or GW) grouped by any combination of `Country` and `Fueltype`. Accepts a single DataFrame or a list of DataFrames (with optional keys for column labels in a MultiIndex result), making it easy to compare multiple datasets side by side. ```python import powerplantmatching as pm df = pm.powerplants(from_url=True) # Total capacity by country and fuel type (MW, default) table = pm.utils.lookup(df) print(table.head()) # Germany France Italy ... # Fueltype # Hard Coal 21000 2500 500 ... # Natural Gas 30000 15000 40000 ... # In GW table_gw = pm.utils.lookup(df, unit="GW") # Group only by Country by_country = pm.utils.lookup(df, by="Country") # Compare multiple datasets entsoe = pm.data.ENTSOE() opsd = pm.data.OPSD() comparison = pm.utils.lookup([entsoe, opsd], keys=["ENTSOE", "OPSD"], by="Country") print(comparison) # MultiIndex columns: (source, country) # Exclude specific fuel types no_solar = pm.utils.lookup(df, exclude=["Solar", "Wind"]) # Via accessor (equivalent) table2 = df.powerplant.lookup(by="Country, Fueltype") ``` ```