### Install Project Dependencies with Poetry Source: https://github.com/vengroff/censusdis/blob/main/DEVELOPER.md Use this command to install all necessary dependencies for local development. Ensure Poetry is installed first. ```shell poetry install ``` -------------------------------- ### Install Project with Explore Extra Source: https://github.com/vengroff/censusdis/blob/main/DEVELOPER.md Install the project with the 'explore' extra dependencies, required for using geopandas.GeoDataFrame.explore in notebooks. ```shell poetry install -E explore ``` -------------------------------- ### Install censusdis Package Source: https://github.com/vengroff/censusdis/blob/main/README.md Install the censusdis package using pip. This is the first step before using any of its functionalities. ```shell pip install censusdis ``` -------------------------------- ### Download Census Data Example Source: https://github.com/vengroff/censusdis/blob/main/README.md Example of downloading U.S. Census data for median household income for all counties in New Jersey for the year 2022. Requires specifying the dataset, vintage, variables, and geography. ```python import censusdis.data as ced from censusdis.datasets import ACS5 from censusdis import states df_median_income = ced.download( # Data set: American Community Survey 5-Year dataset=ACS5, # Vintage: 2022 vintage=2022, # Variable: median household income download_variables=['NAME', 'B19013_001E'], # Geography: All counties in New Jersey. state=states.NJ, county='*' ) ``` -------------------------------- ### Install CensusDis with Explore Extra Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Explore Interactively.ipynb Install censusdis with the 'explore' extra to enable interactive map features. This installs necessary dependencies for geopandas' .explore() method. ```shell pip install censusdis[explore] ``` -------------------------------- ### Activate Poetry Shell Source: https://github.com/vengroff/censusdis/blob/main/DEVELOPER.md Start a shell within a virtual environment managed by Poetry. This ensures all project dependencies are available. ```shell poetry shell ``` -------------------------------- ### Import Libraries Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Block Groups in CBSAs.ipynb Imports necessary libraries for data manipulation, mapping, and census data access. Ensure these are installed. ```python import censusdis.data as ced import censusdis.maps as cem import censusdis.states as states import numpy as np import pandas as pd import geopandas as gpd ``` -------------------------------- ### Initialize CensusDis Data Module Source: https://github.com/vengroff/censusdis/blob/main/docs/intro.rst Import the censusdis.data module and define constants for the dataset, year, and variables to be queried. This setup is necessary before making any data requests. ```python import censusdis.data as ced # American Community Survey 5-Year Data # https://www.census.gov/data/developers/data-sets/acs-5year.html DATASET = "acs/acs5" # The year we want data for. YEAR = 2020 # This are the census variables for total population and median household income. # For more details, see # # https://api.census.gov/data/2020/acs/acs5/variables.html, # https://api.census.gov/data/2020/acs/acs5/variables/B01003_001E.html, and # https://api.census.gov/data/2020/acs/acs5/variables/B19013_001E.html. # TOTAL_POPULATION_VARIABLE = "B01003_001E" MEDIAN_HOUSEHOLD_INCOME_VARIABLE = "B19013_001E" # The variables we are going to query. VARIABLES = ["NAME", TOTAL_POPULATION_VARIABLE, MEDIAN_HOUSEHOLD_INCOME_VARIABLE] ``` -------------------------------- ### Download Data with Specific Certificates Source: https://github.com/vengroff/censusdis/blob/main/notebooks/SSL Debugging.ipynb Use specific certificate files for data and map services. This example demonstrates downloading data with geometry, ensuring the correct certificates are applied. ```python data_cert = "/path/to/certs/api-census.gov.pem" map_cert = "/path/to/certs/www-census.gov.pem" try: with ced.certificates.use(data_cert=data_cert, map_cert=map_cert): gdf = ced.download( dataset=ACS5, vintage=2022, download_variables=["NAME", VARIABLE_TOTAL_POPULATION], state=NJ, county="*", with_geometry=True, ) except OSError as e: print(e) ``` -------------------------------- ### Download Data Variables Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Querying Available Data Sets.ipynb Downloads specified variables for a given dataset and year. Use state="*" to get data for all states. ```python df_download_variables = ced.download(dataset, YEAR, leaves, state="*") ``` ```python df_download_leaves_of_group = ced.download( dataset, YEAR, leaves_of_group=group, state="*" ) ``` -------------------------------- ### Get Variables to Download Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_def1223f610d3855_yamlspec_py.html Returns a list of variables required for download, including any specified denominator variable. Ensures the denominator is included if it's not already in the main list. ```python if ( isinstance(self.denominator, str) and self.denominator not in self._variables ): # We specified a specific denominator that was not already # one of the variables, so get it. return self._variables + [self.denominator] else: # We don't need to fetch an extra variable for the denominator. return self._variables ``` -------------------------------- ### Get All Variables for a Dataset Source: https://github.com/vengroff/censusdis/blob/main/notebooks/datasets/Current Population Survey Basic Monthly.ipynb Retrieves a list of all available variables for a specified dataset and year. Use this to understand the full scope of data available. ```python variables = ced.variables.group_variables(DATASET, YEAR, None) str(variables) ``` -------------------------------- ### Get Path Spec by Number Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_5a366ca900b21d9c_geography_py.html Retrieves the path specification for a given U.S. Census numerical geography code. For example, '050' represents a state and county specification. ```python @classmethod def by_number(cls, dataset: str, year: int, num: str): """ Get the path spec for a given U.S. Census numerical geography code. For example, the code '050' represents a state and county specification. """ return cls.get_path_specs(dataset, year).get(num, None) ``` -------------------------------- ### Python File Coverage: __init__.py (resources) Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/index.html Coverage report for the __init__.py file in the resources directory. Shows lines executed, missed, and total lines. ```text 0 0 0 100% ``` -------------------------------- ### Python File Coverage: __init__.py (utils) Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/index.html Coverage report for the __init__.py file in the utils directory. Shows lines executed, missed, and total lines. ```text 1 0 0 100% ``` -------------------------------- ### Get Current Datetime Source: https://github.com/vengroff/censusdis/blob/main/notebooks/2023 ACS1.ipynb Imports the datetime module and gets the current date and time. ```python from datetime import datetime datetime.now() ``` -------------------------------- ### Prepare Group Keys and Selectors Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_5a366ca900b21d9c_data_py.html Initializes lists and dictionaries for grouping keys and selectors based on provided geography bindings. ```python group_keys = [] selectors = {} for geo, binding in geo_bindings.items(): group_keys.append(f"{geo.upper()}") if binding != "*": selectors[f"{geo.upper()}"] = binding ``` -------------------------------- ### Requests GET with Verification Skipped Source: https://github.com/vengroff/censusdis/blob/main/notebooks/SSL Debugging.ipynb Make a GET request to the Census API while explicitly skipping SSL certificate verification. This can help diagnose issues related to certificate validation. ```python # Skip verification response = requests.get("https://api.census.gov", verify=False) response.status_code ``` -------------------------------- ### Requests GET with Specific Certificate Source: https://github.com/vengroff/censusdis/blob/main/notebooks/SSL Debugging.ipynb Use a specific certificate file when making a GET request to the Census API. This method is helpful when you have a custom certificate or are troubleshooting certificate path issues. ```python # Use a certificate file try: response = requests.get("https://api.census.gov", cert=data_cert) print(response.status_code) except OSError as e: print(e) ``` -------------------------------- ### Basic Requests GET without Verification Source: https://github.com/vengroff/censusdis/blob/main/notebooks/SSL Debugging.ipynb Perform a simple GET request to the Census API without certificate verification. This is useful for initial debugging to isolate network or certificate issues. ```python import requests # Do not ignore or replace the cert. response = requests.get("https://api.census.gov") response.status_code ``` -------------------------------- ### Download Strategy Metrics Initialization Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_5a366ca900b21d9c_data_py.html Initializes counters for tracking the usage of different strategies ('merge' and 'concat') when handling wide tables during data downloads. ```Python __dw_strategy_metrics = {"merge": 0, "concat": 0} """ Counters for how often we use each strategy for wide tables. """ ``` -------------------------------- ### Display Help for ced.download Function Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Error Messages for Bad Arguments.ipynb Use the built-in help() function to display the docstring and signature of the ced.download function. This is useful for understanding all available arguments and their types. ```python help(ced.download) ``` -------------------------------- ### Display First Few Rows of Downloaded Data Source: https://github.com/vengroff/censusdis/blob/main/notebooks/datasets/Current Population Survey Basic Monthly.ipynb After downloading the data, use the `.head()` method to display the first five rows of the DataFrame. This is useful for a quick inspection of the data structure and content. ```python df_data.head() ``` -------------------------------- ### Get state name from FIPS code Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_5a366ca900b21d9c_states_py.html Import the states module, get a state's FIPS code using its abbreviation, and then use the NAMES_FROM_IDS dictionary to find the human-friendly state name. This is useful for displaying state names based on their FIPS codes. ```python from censusdis import states state = states.CA print( f"The name of the state with ID '{state}' " f"is '{states.NAMES_FROM_IDS[state]}'." ) ``` -------------------------------- ### Set up Project Path Source: https://github.com/vengroff/censusdis/blob/main/notebooks/datasets/Current Population Survey Basic Monthly.ipynb Ensures the censusdis package can be imported by adding its parent directory to the system path. This is useful when running scripts from within the project. ```python # So we can run from within the censusdis project and find the packages we need. import os import sys sys.path.append( os.path.join( os.path.abspath(os.path.join(os.path.curdir, os.path.pardir, os.path.pardir)) ) ) ``` -------------------------------- ### Get Number of Children in GroupTreeNode Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Returns the number of children a GroupTreeNode has. ```python return len(self._children) ``` -------------------------------- ### Get Cache Size Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Returns the total number of elements currently stored in the cache. ```python def __len__(self): """Return he number of elements in the cache.""" ``` -------------------------------- ### Get Child from GroupTreeNode Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Retrieves a child GroupTreeNode using its corresponding path component. ```python return self._children[component] ``` -------------------------------- ### Define Data Parameters Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Remove Water.ipynb Sets up constants for the year, dataset, and the specific variable to be used in data downloads. ```python YEAR = 2020 DATASET = "acs/acs5" VARIABLE = "B19013_001E" ``` -------------------------------- ### Set up project path for censusdis Source: https://github.com/vengroff/censusdis/blob/main/notebooks/datasets/SF1.ipynb Add the censusdis project directory to the system path to allow imports. This is useful when running scripts from within the project. ```python import os import sys sys.path.append( os.path.join( os.path.abspath(os.path.join(os.path.curdir, os.path.pardir, os.path.pardir)) ) ) ``` -------------------------------- ### PathSpec.by_number Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_5a366ca900b21d9c_geography_py.html Get the path spec for a given U.S. Census numerical geography code. ```APIDOC ## PathSpec.by_number ### Description Get the path spec for a given U.S. Census numerical geography code. For example, the code '050' represents a state and county specification. ### Method classmethod ### Signature by_number(cls, dataset: str, year: int, num: str) ### Parameters * **dataset** (str) - The dataset to search within. * **year** (int) - The year of the dataset. * **num** (str) - The numerical geography code. ### Returns The PathSpec object for the given code, or None if not found. ``` -------------------------------- ### Download Renter Costs for All States Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Renter Costs.ipynb Use this snippet to download renter cost data for all states. Specify '*' for the state parameter to include all states. Requesting the 'NAME' variable provides human-readable state names. ```python df_all_states = ced.download( ACS1, YEAR, download_variables="NAME", group=GROUP, state="*" ) ``` -------------------------------- ### Get All Groups Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Retrieves a DataFrame containing metadata on all groups within a specified dataset and year. ```APIDOC ## all_groups ### Description Get descriptions of all the groups in the data set. ### Parameters #### Path Parameters - **dataset** (str) - Required - The data set. - **year** (int) - Required - The year. ### Returns - Metadata on all the groups in the data set. ``` -------------------------------- ### Using Dataset Constants Source: https://context7.com/vengroff/censusdis/llms.txt Utilizes symbolic names for common datasets to avoid typos and improve IDE auto-completion. Download data using these constants. ```python from censusdis.datasets import ( ACS1, # "acs/acs1" — 1-Year ACS ACS5, # "acs/acs5" — 5-Year ACS ACS5_PROFILE, # "acs/acs5/profile" ACS5_SUBJECT, # "acs/acs5/subject" DECENNIAL_PL, # "dec/pl" — Redistricting Data DECENNIAL_DHC, # "dec/dhc" — Demographic and Housing Characteristics CBP, # "cbp" — County Business Patterns ) import censusdis.data as ced from censusdis import states # Use any constant as the dataset argument df = ced.download(ACS5_PROFILE, 2022, ["NAME", "DP03_0062E"], state=states.FL, county="*") # DP03_0062E = Median household income (profile table) print(df.head(3)) ``` -------------------------------- ### Get Path Components Keys Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_5a366ca900b21d9c_geography_py.html Retrieves a list of keys that identify the components of a geography path. ```python def keys(self) -> List[str]: """Get the keys identifying the path components.""" return list(self._path) ``` -------------------------------- ### Get Group Name Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Querying Available Data Sets.ipynb Retrieves the group name from the DataFrame. This is a prerequisite for querying variables within that group. ```python group = df_groups.iloc[-5]["GROUP"] group ``` -------------------------------- ### Set up dataset and year variables Source: https://github.com/vengroff/censusdis/blob/main/notebooks/DHC-A.ipynb Define the dataset identifier and the year for which data will be fetched. Ensure these match the available data in the censusdis package. ```python CENSUS_API_KEY = None YEAR = 2020 DATASET = "dec/ddhca" ``` -------------------------------- ### Configure Logging Source: https://github.com/vengroff/censusdis/blob/main/notebooks/All ZCTAs Many Fields.ipynb Sets up basic logging to display debug messages. This is useful for monitoring the execution and troubleshooting API requests. ```python logging.basicConfig(level=logging.DEBUG, handlers=[logging.StreamHandler()]) ``` -------------------------------- ### Select State for Analysis Source: https://github.com/vengroff/censusdis/blob/main/notebooks/ACS Demo.ipynb Specifies the state for data analysis. NJ (New Jersey) is used as an example. ```python # Feel free to try other states. STATE = NJ ``` -------------------------------- ### Initialize ShapeReader Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Map Geographies.ipynb Creates an instance of the ShapeReader, specifying the root directory for caching and the year for the shapefiles. ```python reader = ShapeReader(SHAPEFILE_ROOT, YEAR) ``` -------------------------------- ### Get Cache Values Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Returns an iterable of values (variable descriptions) from the cache. Each value corresponds to a key in the cache. ```python def values(self) -> Iterable[dict]: """Values, i.e. the descriptions of variables, in the cache.""" for _, value in self.items(): yield value ``` -------------------------------- ### Configure Shapefile Cache Directory Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Map Geographies.ipynb Sets up a local directory to cache downloaded shapefiles. This is optional; if not provided, a default location will be used. The directory is created if it doesn't exist. ```python # We are going to provide a path for our shape file # reader to cache the shapefiles it downloads locally. # This is optional, and if we don't provide this when # we construct the reader, a default location will be # selected for us. SHAPEFILE_ROOT = os.path.join(os.environ["HOME"], "data", "shapefiles") # Make sure it is there. os.makedirs(SHAPEFILE_ROOT, exist_ok=True) ``` -------------------------------- ### Get Cache Keys Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Returns an iterable of keys from the cache. Each key is a tuple representing (dataset, year, name). ```python def keys(self) -> Iterable[Tuple[str, int, str]]: """Keys, i.e. the names of variables, in the cache.""" for key, _ in self.items(): yield key ``` -------------------------------- ### Import Libraries and Configuration Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Nationwide Diversity and Integration.ipynb Imports necessary libraries from censusdis and divintseg. Configure the CENSUS_API_KEY if needed for higher volume queries. ```python import censusdis.data as ced import censusdis.maps as cem from censusdis.states import ALL_STATES_AND_DC import divintseg as dis ``` ```python # Fill in your own key here, or leave as is # if you are doing low volume queries that # will not hit the census server usage limits. CENSUS_API_KEY = None ``` -------------------------------- ### Get the name of the minimum leaf Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Returns the name of the first leaf node encountered when iterating through leaf variables. ```python @property def min_leaf_name(self) -> str: """The name of the first leaf.""" return min(self.leaf_variables()) ``` -------------------------------- ### Get All Variables for a Dataset Source: https://github.com/vengroff/censusdis/blob/main/notebooks/datasets/Small Area Income and Poverty (SAIPE) School Districts.ipynb Retrieves a list of all available variables for a given dataset and timeseries. The result is a pandas DataFrame. ```python df_variables = ced.variables.all_variables(DATASET, "timeseries", None) ``` -------------------------------- ### Run symbolic.py utility Source: https://github.com/vengroff/censusdis/blob/main/DEVELOPER.md Execute the symbolic.py utility from the root directory to update datasets.py. Commit any resulting changes before submitting a pull request. ```shell poetry run python utils/symbolic.py datasets.py ``` -------------------------------- ### Display Downloaded Time Series Data (All Levels) Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Time Series Export.ipynb Displays the first few rows of the downloaded time series DataFrame, showing data for all commodity levels. ```python df_ts_all_levels ``` -------------------------------- ### Import censusdis libraries Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Seeing White.ipynb Imports necessary modules from the censusdis package for data manipulation and mapping. Ensure these libraries are installed. ```python import censusdis.data as ced import censusdis.maps as cdm from censusdis.states import ALL_STATES_AND_DC ``` -------------------------------- ### Import necessary libraries and modules Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Geographies Contained within Geographies.ipynb Imports the required modules from the censusdis library and matplotlib for plotting. Ensure these libraries are installed. ```python import censusdis.data as ced import censusdis.maps as cem from censusdis.states import NJ, KS, MO, IL from censusdis.places.new_jersey import NEWARK_CITY from censusdis.msa_msa import KANSAS_CITY_MO_KS_METRO_AREA from censusdis.datasets import ACS5 from matplotlib.ticker import StrMethodFormatter ``` -------------------------------- ### Initialize DataSpec Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_def1223f610d3855_yamlspec_py.html Initializes a DataSpec object to define data download parameters. Use this to specify the dataset, vintage, variables, and geography for your data request. ```python self._dataset = getattr(censusdis.datasets, dataset, dataset) ``` ```python self._vintage = vintage ``` ```python self._variable_spec = ( specs if isinstance(specs, VariableSpec) else VariableSpecCollection(specs) ) ``` ```python self._geography = self.map_state_and_county_names(geography) ``` ```python if contained_within is None: self._contained_within = None else: contained_within = self.map_state_and_county_names(contained_within) self._contained_within = ced.ContainedWithin( area_threshold, **contained_within ) ``` ```python self._with_geometry = with_geometry ``` ```python self._remove_water = remove_water ``` -------------------------------- ### Import Libraries for CensusDis Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Data With Geometry.ipynb Imports necessary libraries for working with census data, geographical information, and plotting. Ensure these are installed before running. ```python import os import geopandas as gpd import matplotlib.pyplot as plt from typing import Optional import censusdis.data as ced import censusdis.maps as cem import censusdis.values as cev from censusdis import states ``` -------------------------------- ### Global Certificate Verification Settings Source: https://github.com/vengroff/censusdis/blob/main/notebooks/SSL Debugging.ipynb Configure SSL certificate verification globally for the `ced` library. This example shows how to change and revert the `data_verify` setting and demonstrates its behavior within a context manager. ```python print(f"Initially, ced.certificates.data_verify is {ced.certificates.data_verify}.") # Global change. ced.certificates.use(data_verify=False) print(f"Now, globally, ced.certificates.data_verify is {ced.certificates.data_verify}.") # Change it back globally. ced.certificates.use(data_verify=True) print( f"Now, globally, ced.certificates.data_verify is back to {ced.certificates.data_verify}." ) # We can change it temporarily in a context manager. with ced.certificates.use(data_verify=False): print( f" Within the context, ced.certificates.data_verify is {ced.certificates.data_verify}." ) # And back outside the context the globally set value is back. print( f"Finally, globally, ced.certificates.data_verify is {ced.certificates.data_verify}." ) ``` -------------------------------- ### Initialize Shapefile Reader Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_us_census_shapefiles_py.html Initializes and returns a shapefile reader for a given year. If a reader for the year does not exist, it creates one using the specified root path and year. ```python reader = __shapefile_readers.get(year, None) if reader is None: reader = cmap.ShapeReader( __shapefile_root.shapefile_root, year, ) __shapefile_readers[year] = reader return reader ``` -------------------------------- ### Import necessary libraries Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Map Demo.ipynb Imports required modules for data manipulation, plotting, and census data access. Ensure these libraries are installed. ```python from collections import OrderedDict import matplotlib.pyplot as plt from censusdis import states from censusdis.maps import ShapeReader ``` -------------------------------- ### Import censusdis Libraries Source: https://github.com/vengroff/censusdis/blob/main/notebooks/2023 ACS5.ipynb Imports necessary modules from the censusdis library for data access, mapping, and dataset definitions. Ensure these are installed. ```python import censusdis.data as ced import censusdis.maps as cem from censusdis.datasets import ACS5 from censusdis.states import ALL_STATES_AND_DC ``` -------------------------------- ### Symbolic class initialization Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_c810615cce0f7acb_symbolic_py.html Initializes the symbolic class, setting up an empty dictionary to store symbolic names and defining a module message template with a dynamic copyright year. ```python class symbolic: """ A generator of datasets' symbolic names file. This creates symbolic names for datasets based on dataset names. The symbolic names are stored as dictionary keys with values of the dataset names and url. Users will use this to generate most up to date dataset documentation file. """ def __init__(self): self.dictionary = {} self.module_message = ( f"# Copyright (c) {datetime.now().year} Darren Erik Vengroff\n" ``` -------------------------------- ### Import Libraries for ACS Data Source: https://github.com/vengroff/censusdis/blob/main/notebooks/ACS Comparison Profile.ipynb Imports necessary libraries for accessing Census data, mapping, and state information. Ensure these are installed. ```python import censusdis.data as ced import censusdis.maps as cem from censusdis.states import ALL_STATES_AND_DC from matplotlib.ticker import FuncFormatter ``` -------------------------------- ### Display Help for Variable Search Function Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Variable Search.ipynb View the documentation for the `ced.variables.search` method to understand its parameters and usage. ```python help(ced.variables.search) ``` -------------------------------- ### Import Libraries for ACS Data Source: https://github.com/vengroff/censusdis/blob/main/notebooks/ACS Data Profile.ipynb Imports necessary libraries for interacting with Census data and creating maps. Ensure these are installed before running. ```python import censusdis.data as ced import censusdis.maps as cem from matplotlib.ticker import FuncFormatter ``` -------------------------------- ### Download County Data for a Specific State Source: https://github.com/vengroff/censusdis/blob/main/docs/intro.rst Use this snippet to download aggregated data for a specific state at the county level. Ensure the necessary variables and dataset are defined. ```python from censusdis import states df_counties = ced.download( DATASET, YEAR, VARIABLES, state=states.NJ, county="*", ) ``` -------------------------------- ### Clear Entire Cache Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Resets the entire cache to an empty state. Subsequent calls to get() will require fetching data from the source. ```python def clear(self): """ Clear the entire cache. This just means that further calls to :py:meth:`~get` will have to make a call to the source behind the cache. """ self._variable_cache = defaultdict(lambda: defaultdict(dict)) ``` -------------------------------- ### Get Variables from a Census Group Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Retrieves all variables within a specified census group. Supports filtering annotations and subgroup variables. ```python def group_variables( self, dataset: str, year: int, group_name: str, *, skip_annotations: bool = True, skip_subgroup_variables: bool = True, ) -> List[str]: """ Find the variables of a given group. Parameters ---------- dataset The census dataset. year The year group_name The name of the group. skip_annotations If `True` try to filter out variables that are annotations rather than actual values, by skipping those with labels that begin with "Annotation" or "Margin of Error". skip_subgroup_variables If this is `True`, then we will ignore variables from alphabetical subgroups. These are relatively common in the ACS, where there are groups like `B01001` that have subgroups `B01001A`, `B01001B` and so on. The underlying census API sometimes reports variables like `B01001A_001E` from these as members of `B01001` and other times as members of `B01001A`. Setting this `True`, which is the default, does not report `B01001A_001E` when the group name `name='B01001' Returns ------- A list of the variables in the group. """ tree = self.get_group( dataset, year, group_name, skip_subgroup_variables=skip_subgroup_variables ) if skip_annotations: group_variables = [ k for k, v in tree.items() if (not v["label"].startswith("Annotation")) and (not v["label"].startswith("Margin of Error")) ] else: group_variables = list(tree.keys()) return sorted(group_variables) ``` -------------------------------- ### Display the head of the dataframe Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Column Labels.ipynb Shows the first 5 rows of the downloaded dataframe to inspect the data and column names. ```python df.head() ``` -------------------------------- ### Python File Coverage: utah.py Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/index.html Coverage report for the utah.py file. Shows lines executed, missed, and total lines. ```text 334 0 0 100% ``` -------------------------------- ### Get Shapefile Root Directory Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_us_census_shapefiles_py.html Returns the root directory path used for caching downloaded shapefiles. This is typically used internally. ```python return __shapefile_root.shapefile_root ``` -------------------------------- ### Fetch All Available Datasets Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Querying Available Data Sets.ipynb Retrieves a DataFrame containing all available datasets across all years. This is a foundational step for further analysis. ```python df_all_datasets = ced.variables.all_data_sets() df_all_datasets ``` -------------------------------- ### Get All Variables in a Group Source: https://github.com/vengroff/censusdis/blob/main/notebooks/datasets/SF1.ipynb Retrieves a list of all variables associated with a given dataset, year, and group. Use `str()` to view the list. ```python variables = ced.variables.group_variables(DATASET, YEAR, group) str(variables) ``` -------------------------------- ### Download State Geometries Source: https://github.com/vengroff/censusdis/blob/main/notebooks/DHC-A.ipynb Downloads state geometries for plotting background layers. Specify the dataset, year, desired columns, states, and API key. `with_geometry=True` is essential. ```python gdf_states = ced.download( DATASET, YEAR, ["NAME"], state=states, with_geometry=True, api_key=CENSUS_API_KEY, ) ``` -------------------------------- ### Get all groups in SF1 dataset Source: https://github.com/vengroff/censusdis/blob/main/notebooks/datasets/SF1.ipynb Retrieve all available groups within the specified SF1 dataset and year. The result is a pandas DataFrame. ```python df_groups = ced.variables.all_groups(DATASET, YEAR) df_groups.head() ``` -------------------------------- ### Get Dataset Name Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Querying Available Data Sets.ipynb Extracts the dataset name from a specific row in the `df_datasets_for_year` DataFrame. This is a preparatory step for querying dataset groups. ```python dataset = df_datasets_for_year.iloc[6]["DATASET"] dataset ``` -------------------------------- ### Python File Coverage: vermont.py Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/index.html Coverage report for the vermont.py file. Shows lines executed, missed, and total lines. ```text 180 0 0 100% ``` -------------------------------- ### Define Dataset and Year Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Data With Geometry.ipynb Specifies the census dataset and year to be used for data retrieval. This is a common setup step before downloading data. ```python DATASET = "acs/acs5" YEAR = 2020 ``` -------------------------------- ### Construct URL and Parameters Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_5a366ca900b21d9c_geography_py.html Builds the URL and parameters for a geography data request. Handles optional parameters like 'for', 'in', and API key. ```python url = "/".join([self._BASE_URL, self.dataset]) params = { "get": ",".join(self.variables), } if self.bound_path.bindings: params["for"] = self.for_component if query_filter is not None: params.update(query_filter) in_components = self.in_components if in_components is not None: params["in"] = in_components if self.api_key is not None: params["key"] = self.api_key return url, params ``` -------------------------------- ### Python File Coverage: washington.py Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/index.html Coverage report for the washington.py file. Shows lines executed, missed, and total lines. ```text 640 0 0 100% ``` -------------------------------- ### Get Supported Geographies Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_5a366ca900b21d9c_data_py.html Determines the supported geographies for a given dataset and vintage. This utility lists geography keywords usable with the download function. ```python return list(cgeo.geo_path_snake_specs(dataset, vintage).values()) ``` -------------------------------- ### Get Group Name from Variable List Source: https://github.com/vengroff/censusdis/blob/main/notebooks/datasets/American Community Survey (ACS) 5-Year Data.ipynb This snippet shows how to extract the group name from a variable list, useful for identifying data categories. ```python group ``` -------------------------------- ### Python File Coverage: values.py Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/index.html Coverage report for the values.py file. Shows lines executed, missed, and total lines. ```text 14 0 0 100% ``` -------------------------------- ### Download All State Data Source: https://github.com/vengroff/censusdis/blob/main/notebooks/Getting Started Examples.ipynb Downloads all available data for all states for a given year and variables. Requires specifying the dataset, year, and variables. ```python df_states = ced.download( DATASET, YEAR, VARIABLES, state="*", ) ``` ```python print(df_states) ``` -------------------------------- ### Get child node by path component Source: https://github.com/vengroff/censusdis/blob/main/reports/coverage/d_1ef9b076751c2c3b_varcache_py.html Retrieves a child node using a path component. Returns a default value if the component is not found. ```python def get( self, component, default: Optional["VariableCache.GroupTreeNode"] = None ): """ Get the child at the given path component below us. Parameters ---------- component The next component of the path below us. default The default value to return if there is no node at the path. Returns ------- The node below us or `default` if it is not there, """ return self._children.get(component, default) ```