### Run PopulationSim Example (Single Process)

Source: https://github.com/activitysim/populationsim/blob/master/docs/getting_started.rst

This command executes the PopulationSim script for the 'example_calm' setup. It assumes the 'popsim' conda environment is activated and the command is run from the example's directory. The output will be generated in the 'output' folder.

```bash
activate popsim
python run_populationsim.py
```

--------------------------------

### Define PopulationSim Configuration

Source: https://context7.com/activitysim/populationsim/llms.txt

Example of a settings.yaml file defining algorithm parameters and the geographic hierarchy for the synthesis process.

```yaml
INTEGERIZE_WITH_BACKSTOPPED_CONTROLS: True
SUB_BALANCE_WITH_FLOAT_SEED_WEIGHTS: False
GROUP_BY_INCIDENCE_SIGNATURE: True
USE_NUMBA: True
NUMBA_PRECISION: 'float32'
USE_SIMUL_INTEGERIZER: True
USE_CVXPY: False
INTEGERIZER_TIMEOUT: 300
max_expansion_factor: 30
geographies: [REGION, PUMA, TRACT, TAZ]
seed_geography: PUMA
```

--------------------------------

### Run PopulationSim Example (Multiprocessed)

Source: https://github.com/activitysim/populationsim/blob/master/docs/getting_started.rst

This command runs the multiprocessed version of PopulationSim ('example_calm_mp'). It requires specifying configuration directories using '-c'. The 'num_processes' should be set in 'configs_mp/setting.yaml' prior to execution. Output is saved in the 'output' folder.

```bash
activate popsim
python run_populationsim.py -c configs_mp -c configs
```

--------------------------------

### Install PopulationSim Package

Source: https://github.com/activitysim/populationsim/blob/master/docs/getting_started.rst

This command installs the PopulationSim package using pip after ensuring pytables is installed via conda. Pytables is recommended for consistency with the ActivitySim package. This should be run in the activated conda environment.

```bash
# best to use the conda version of pytables for consistency with activitysim
conda install pytables

pip install populationsim
```

--------------------------------

### Setup Data Structures for Population Simulation (Python)

Source: https://context7.com/activitysim/populationsim/llms.txt

This step builds essential data structures for population simulation, including geographic crosswalks, control tables, and incidence tables. It processes control specifications to map households to control targets.

```python
# setup_data_structures creates these pipeline tables:
# - crosswalk: geographic correspondence table
# - control_spec: parsed control specification
# - incidence_table: household x control incidence matrix
# - <geography>_controls: control totals for each geography level
# - household_groups: (if GROUP_BY_INCIDENCE_SIGNATURE is True)

# Example control_totals_taz.csv format:
# TAZ,HHBASE,HHSIZE1,HHSIZE2,HHSIZE3,HHSIZE4,HHAGE1,HHAGE2,HHAGE3,HHAGE4,...
# 100,1500,400,350,300,450,100,600,400,400,...
# 101,2000,500,450,400,650,150,800,500,550,...

# Example geo_cross_walk.csv format:
# REGION,PUMA,TRACT,TAZ
# 1,600,10100,100
# 1,600,10100,101
# 1,600,10200,102
# 1,601,10300,103
```

--------------------------------

### Execute PopulationSim via CLI

Source: https://github.com/activitysim/populationsim/blob/master/README.md

This command runs the PopulationSim tool by specifying the paths for configuration files, input data, and output destination. It requires the population simulation environment to be correctly installed and configured.

```bash
populationsim -c /path/to/configs -d /path/to/data -o /path/to/output
```

--------------------------------

### Initial Seed Balancing for Population Simulation (Python)

Source: https://context7.com/activitysim/populationsim/llms.txt

This step balances household weights at the seed geography level (e.g., PUMA) using iterative proportional fitting. It generates preliminary balanced weights as starting points for finer geographic balancing.

```python
# initial_seed_balancing produces a weight table with structure:
# <seed_geography>_weights (e.g., PUMA_weights)
#
# | hh_id | PUMA | preliminary_balanced_weight | sample_weight |
# |-------|------|----------------------------|---------------|
# | 0     | 600  | 0.313555                   | 1.0           |
# | 1     | 600  | 0.627110                   | 2.0           |
# | 2     | 601  | 0.313555                   | 1.0           |

# Key settings that affect balancing:
# settings.yaml
total_hh_control: num_hh          # Column name for total household control
max_expansion_factor: 30          # Maximum weight multiplier
min_expansion_factor: null        # Minimum weight multiplier (optional)
absolute_upper_bound: null        # Hard upper bound on weights (optional)
absolute_lower_bound: null        # Hard lower bound on weights (optional)
USE_HARD_CONSTRAINTS: False       # Use hard vs soft constraints
USE_NUMBA: True                   # Use Numba for performance
NUMBA_PRECISION: 'float32'        # Numba floating point precision
```

--------------------------------

### Manage Simulation Pipeline State and Tables

Source: https://context7.com/activitysim/populationsim/llms.txt

Provides core functions for initializing, running, and closing the simulation pipeline. Supports managing simulation state through checkpoints and performing operations on pipeline tables like getting, replacing, extending, and dropping.

```python
from populationsim.core import pipeline, inject, config

# Open/close pipeline
pipeline.open_pipeline(resume_after=None)  # Start new or resume
pipeline.close_pipeline()                   # Close and save state
pipeline.is_open()                          # Check if pipeline is open

# Run models
pipeline.run(
    models=['input_pre_processor', 'setup_data_structures'],
    resume_after=None
)

# Table operations
df = pipeline.get_table('households')                    # Get current table
df = pipeline.get_table('households', 'checkpoint_name') # Get from checkpoint
pipeline.replace_table('households', modified_df)        # Replace table
pipeline.extend_table('households', new_rows_df)         # Add rows
pipeline.drop_table('temp_table')                        # Remove table

# Checkpoint operations
pipeline.add_checkpoint('my_checkpoint')  # Save current state
checkpoints_df = pipeline.get_checkpoints()  # List all checkpoints
```

--------------------------------

### Run Population Simulation Script

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Checks if the population simulation script needs to be run and executes it using subprocess if any summary files are missing. This ensures the simulation starts with the necessary prerequisites.

```python
missing = [f for f in summary_files if not os.path.isfile(os.path.join(popsim_dir, f))]

if len(missing) > 0:
    import subprocess
    subprocess.run(['python3', f'{popsim_dir}/run_populationsim.py'])
```

--------------------------------

### Execute PopulationSim via Command Line Interface

Source: https://context7.com/activitysim/populationsim/llms.txt

Demonstrates various CLI commands for running population synthesis, including specifying directories, enabling multiprocessing, resuming from checkpoints, and limiting sample sizes.

```bash
populationsim -w /path/to/project
populationsim -c /path/to/configs -d /path/to/data -o /path/to/output
populationsim -w /path/to/project -m
populationsim -w /path/to/project -m 4
populationsim -w /path/to/project -r final_seed_balancing
populationsim -w /path/to/project -s custom_settings.yaml
populationsim -w /path/to/project --households_sample_size 1000
populationsim -w /path/to/project -e ./extensions
```

--------------------------------

### Import Required Python Packages

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Imports essential Python libraries for data manipulation, configuration parsing, and plotting. Includes a fallback mechanism to install missing packages using pip if they are not found.

```python
import os
import sys

try:
    import yaml
    import pandas as pd
    import numpy as np
    import matplotlib.pyplot as plt
except ImportError:
    packages = 'pyyaml pandas numpy matplotlib'
    ret = os.system(f'uv pip install {packages}')
    if ret != 0:
        os.system(f'{sys.executable} -m pip install {packages}')
    import yaml
    import pandas as pd
    import numpy as np
    import matplotlib.pyplot as plt
```

--------------------------------

### Display Configuration Parameters

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Prints the current state of the parameters dictionary in a formatted YAML string.

```python
print(yaml.dump(parameters))
```

--------------------------------

### Accessing Configuration and Injecting Data

Source: https://context7.com/activitysim/populationsim/llms.txt

Demonstrates how to retrieve and override configuration settings, resolve file paths, and inject custom tables or values into the PopulationSim environment using the config and inject modules.

```python
value = config.setting('max_expansion_factor')
config.override_setting('max_expansion_factor', 50)
file_path = config.data_file_path('seed_households.csv')
file_path = config.output_file_path('results.csv')

inject.add_table('my_table', my_dataframe)
inject.add_injectable('my_value', 42)
value = inject.get_injectable('my_value')
```

--------------------------------

### Initialize Parameters Dictionary

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Initializes an empty dictionary to store input parameters read from configuration files or command line arguments. This dictionary will be populated as the script progresses.

```python
parameters = {}
```

--------------------------------

### Execute PopulationSim via Python API

Source: https://context7.com/activitysim/populationsim/llms.txt

Shows how to programmatically invoke the run() function using argparse to manage configuration arguments and execute the synthesis process.

```python
import argparse
from populationsim import run, add_run_args

parser = argparse.ArgumentParser()
add_run_args(parser)

args = parser.parse_args(['-w', '/path/to/example_calm', '-c', 'configs', '-d', 'data', '-o', 'output'])

args = argparse.Namespace(
	working_dir='/path/to/example_calm',
	config=['configs'],
	data=['data'],
	output='output',
	resume=None,
	pipeline=None,
	settings_file=None,
	households_sample_size=None,
	fast=False,
	ext=None,
	multiprocess=False
)

exit_code = run(args)
print(f"PopulationSim completed with exit code: {exit_code}")
```

--------------------------------

### Population Simulation Data Loading and Processing

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Demonstrates the initialization and data loading process for the population simulation. It configures logging, reads various control files and seed data (households, persons), and performs initial data validation and processing steps like dropping empty control rows and filtering households.

```python
INFO:populationsim:Configured logging using basicConfig
INFO - Read logging configuration from: configs/logging.yaml
INFO - SETTING configs_dir: configs
INFO - SETTING settings_file_name: settings.yaml
INFO - SETTING data_dir: data
INFO - SETTING output_dir: output
INFO - SETTING multiprocess: None
INFO - SETTING num_processes: None
INFO - SETTING resume_after: None
INFO - SETTING GROUP_BY_INCIDENCE_SIGNATURE: True
INFO - SETTING INTEGERIZE_WITH_BACKSTOPPED_CONTROLS: True
INFO - SETTING SUB_BALANCE_WITH_FLOAT_SEED_WEIGHTS: False
INFO - SETTING meta_control_data: None
INFO - SETTING control_file_name: controls.csv
INFO - SETTING USE_CVXPY: False
INFO - SETTING USE_SIMUL_INTEGERIZER: True
INFO - ENV MKL_NUM_THREADS: None
INFO - ENV OMP_NUM_THREADS: None
INFO - ENV OPENBLAS_NUM_THREADS: None
INFO - ENV NUMBA_NUM_THREADS: None
INFO - NumPy build info:
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /usr/local/include
    lib directory: /usr/local/lib
    name: openblas64
    openblas configuration: USE_64BITINT=1 DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS= 
      NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= HASWELL MAX_THREADS=2
    pc file directory: /usr/local/lib/pkgconfig
    version: 0.3.23.dev
  lapack:
    detection method: internal
    found: true
    include directory: unknown
    lib directory: unknown
    name: dep140551260102944
    openblas configuration: unknown
    pc file directory: unknown
    version: 1.26.4
Compilers:
  c:
    args: -fno-strict-aliasing
    commands: cc
    linker: ld.bfd
    linker args: -Wl,--strip-debug, -fno-strict-aliasing
    name: gcc
    version: 10.2.1
  c++:
    commands: c++
    linker: ld.bfd
    linker args: -Wl,--strip-debug
    name: gcc
    version: 10.2.1
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.8
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
Python Information:
  path: /opt/python/cp312-cp312/bin/python
  version: '3.12'
SIMD Extensions:
  baseline:
  - SSE
  - SSE2
  - SSE3
  found:
  - SSSE3
  - SSE41
  - POPCNT
  - SSE42
  - AVX
  - F16C
  - FMA3
  - AVX2
  not found:
  - AVX512F
  - AVX512CD
  - AVX512_KNL
  - AVX512_KNM
  - AVX512_SKX
  - AVX512_CLX
  - AVX512_CNL
  - AVX512_ICL


INFO - run single process simulation
INFO - Time to execute open_pipeline : 0.02 seconds (0.0 minutes)
INFO - #run_model running step input_pre_processor
Running step 'input_pre_processor'
INFO - Using table list: [{'tablename': 'households', 'filename': 'seed_households.csv', 'index_col': 'hh_id', 'rename_columns': {'hhnum': 'hh_id'}}, {'tablename': 'persons', 'filename': 'seed_persons.csv', 'rename_columns': {'hhnum': 'hh_id', 'SPORDER': 'per_num'}, 'drop_columns': ['indp02', 'naicsp02', 'occp02', 'socp00', 'occp10', 'socp10', 'indp07', 'naicsp07']}, {'tablename': 'geo_cross_walk', 'filename': 'geo_cross_walk.csv', 'rename_columns': {'TRACTCE': 'TRACT'}}, {'tablename': 'TAZ_control_data', 'filename': 'control_totals_taz.csv'}, {'tablename': 'TRACT_control_data', 'filename': 'control_totals_tract.csv'}, {'tablename': 'REGION_control_data', 'filename': 'scaled_control_totals_meta.csv'}]
INFO - Reading CSV file data/seed_households.csv
INFO - registering table households
INFO - Reading CSV file data/seed_persons.csv
WARNING - /home/nick/clonux/populationsim/populationsim/core/input.py:231: DtypeWarning: Columns (219,221,225) have mixed types. Specify dtype option on import or set low_memory=False.
  df = pd.read_csv(filepath, comment="#", dtype=dtypes)

INFO - registering table persons
INFO - Reading CSV file data/geo_cross_walk.csv
INFO - registering table geo_cross_walk
INFO - Reading CSV file data/control_totals_taz.csv
INFO - registering table TAZ_control_data
INFO - Reading CSV file data/control_totals_tract.csv
INFO - registering table TRACT_control_data
INFO - Reading CSV file data/scaled_control_totals_meta.csv
INFO - registering table REGION_control_data
Time to execute step 'input_pre_processor': 0.45 s
Total time to execute iteration 1 with iteration value None: 0.45 s
INFO - #run_model running step setup_data_structures
Running step 'setup_data_structures'
INFO - Reading control file configs/controls.csv
INFO - dropping 149 TAZ control rows with empty total_hh_control
INFO - dropped 0 households not in seed zones
INFO - kept 4839 households in seed zones
INFO - control target num_hh
INFO - control target hh_size_1
INFO - control target hh_size_2
INFO - control target hh_size_3
INFO - control target hh_size_4_plus
```

--------------------------------

### Define PopulationSim Control Expressions

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

Examples of Python/Pandas expressions used to filter seed household or person data for control targets. These expressions utilize standard logical operators to identify subsets of the population based on attributes like age, income, or household size.

```python
# Household size filter
households.NP >= 4

# Age range filter
(households.AGEHOH > 15) & (households.AGEHOH <= 24)

# Income range filter
(households.HHINCADJ > -999999999) & (households.HHINCADJ <= 21297)

# Categorical flag filter
persons.OSUTAG == 1

# Infinity range filter
(households.WGTP > 0) & (households.WGTP < np.inf)
```

--------------------------------

### Executing PopulationSim with Multiprocessing

Source: https://context7.com/activitysim/populationsim/llms.txt

Shows various command-line interface patterns to execute PopulationSim with multiprocessing enabled, including flag-based configuration loading and process count overrides.

```bash
cd example_calm
python run_populationsim.py -m
python run_populationsim.py -c configs_mp -c configs -m 4
populationsim -w . -m 8
```

--------------------------------

### Load PopulationSim Configuration from Files

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

This snippet demonstrates how to load configuration parameters from a YAML file and optionally override them with a CSV file. It includes logic to detect if the code is running as a script or within a Jupyter notebook environment.

```python
import sys
import os
import yaml
import pandas as pd

is_script = 'ipykernel' not in sys.modules
if is_script and len(sys.argv) > 1:
    if os.path.isfile(sys.argv[1]):
        region_yaml = sys.argv[1]

with open(region_yaml) as f:
    parameters.update(yaml.safe_load(f))

if is_script and len(sys.argv) > 2:
    if os.path.isfile(sys.argv[2]):
        parameters_csv = sys.argv[2]
        parameters.update(pd.read_csv(parameters_csv, header=None, index_col=0, comment='#').to_dict()[1])
```

--------------------------------

### Run Validation Script from Command Line

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Demonstrates how to execute the validation script from the command line, providing the region YAML file and an optional parameters CSV file as arguments. This is useful for batch processing or when running the script outside of a notebook environment.

```bash
python validation.py calm_verification.yaml [parameters.csv]
```

--------------------------------

### Extract PopulationSim Configuration Parameters

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Retrieves simulation settings such as geography, summary files, and household data paths from a parameters dictionary. This is essential for initializing the simulation environment with user-defined inputs.

```python
use_geographies = parameters.get('group_geographies')
summary_files   = parameters.get('summaries', [])
aggregate_list  = parameters.get('aggregate_summaries', [])
scenario        = parameters.get('scenario')
region          = parameters.get('region')
exp_hh_file     = parameters.get('expanded_hhid')
exp_hh_id_col   = parameters.get('expanded_hhid_col')
seed_hh_file    = parameters.get('seed_households')
seed_hh_cols    = parameters.get('seed_cols')
```

--------------------------------

### Repop Mode Run Steps Configuration

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

Defines the sequence of steps for PopulationSim's repop mode, which modifies an existing synthetic population. It includes options for expanding households ('append' or 'replace') and specifies the resume point.

```yaml
run_list:
  steps:
    - input_pre_processor.repop
    - repop_setup_data_structures
    - initial_seed_balancing.final=true
    - integerize_final_seed_weights.repop
    - repop_balancing
    # expand_households options are append or replace
    - expand_households.repop;replace
    - summarize.repop
    - write_synthetic_population.repop
    - write_tables.repop

  resume_after: summarize
```

--------------------------------

### Configure Input Pre-processor

Source: https://context7.com/activitysim/populationsim/llms.txt

Details the configuration for the input_pre_processor pipeline step, which handles loading, renaming, and filtering of seed data tables from CSV files.

```yaml
input_table_list:
  - tablename: households
    filename: seed_households.csv
    index_col: hh_id
    rename_columns:
      hhnum: hh_id
      SERIALNO: serial_no
    drop_columns:
      - unused_column1
      - unused_column2
```

--------------------------------

### Calculate Uniformity Metrics and Export

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Aggregates expansion factors by geography, computes RMSE, and exports the resulting uniformity statistics to a CSV file.

```python
weight_mask = seed_hh_df[geog_weight] > 0
weight = expanded_hhids[exp_hh_id_col].value_counts()[weight_mask]
expansion_factor = (weight/seed_hh_df[geog_weight]).fillna(0)

df = pd.DataFrame({
    geog: seed_hh_df[geog],
    geog_weight: seed_hh_df[geog_weight],
    'weight': weight,
    'ef': expansion_factor,
})

geog_group = df.groupby(geog)
geog_final_weight = geog_group.sum()['weight']
expansion = geog_final_weight/geog_group.sum()[geog_weight]

expansion.name = 'avg_expansion'
df = df.join(expansion, on=geog)
df['diff_sq'] = (df['avg_expansion'] - df['ef']) ** 2
rmse = df.groupby(geog).mean()['diff_sq'] ** 0.5

uniformity_df = pd.DataFrame({
    'W': geog_group.sum()[geog_weight],
    'Z': geog_group.sum()['weight'],
    'N': geog_group.count()[geog_weight],
    'EXP': expansion,
    'EXP_MIN': geog_group.min()['ef'],
    'EXP_MAX': geog_group.max()['ef'],
    'RMSE': rmse, 
})

uniformity_df.to_csv(os.path.join(validation_dir, 'uniformity.csv'))
```

--------------------------------

### Extract Parameters from Configuration Dictionary

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Retrieves specific configuration keys from the parameters dictionary, providing fallback options for different naming conventions.

```python
popsim_dir = parameters.get('POPSIMDIR') or parameters.get('popsim_dir')
validation_dir = parameters.get('VALID_DIR') or parameters.get('validation_dir')
geography_file = parameters.get('geographies')
```

--------------------------------

### Perform List Balancing with do_balancing

Source: https://context7.com/activitysim/populationsim/llms.txt

Implements iterative proportional fitting to adjust household weights to match control totals. Supports importance weights and optional bound constraints. Requires incidence, control specification, control totals, and initial weights.

```python
from populationsim.balancing import do_balancing
import pandas as pd
import numpy as np

# Prepare incidence dataframe (households x controls)
incidence_df = pd.DataFrame({
    'num_hh': [1, 1, 1, 1],
    'hh_size_1': [1, 0, 0, 0],
    'hh_size_2': [0, 1, 0, 0],
    'hh_size_3': [0, 0, 1, 0],
    'hh_size_4_plus': [0, 0, 0, 1],
}, index=[0, 1, 2, 3])

# Prepare control specification
control_spec = pd.DataFrame({
    'target': ['num_hh', 'hh_size_1', 'hh_size_2', 'hh_size_3', 'hh_size_4_plus'],
    'importance': [1000000, 5000, 5000, 5000, 5000]
})

# Control totals for this zone
control_totals = pd.Series({
    'num_hh': 100,
    'hh_size_1': 30,
    'hh_size_2': 25,
    'hh_size_3': 25,
    'hh_size_4_plus': 20
})

# Initial sample weights
initial_weights = pd.Series([1.0, 2.0, 1.5, 0.5], index=[0, 1, 2, 3])

# Run balancing
status, weights_df, controls_df = do_balancing(
    control_spec=control_spec,
    total_hh_control_col='num_hh',
    max_expansion_factor=30,
    min_expansion_factor=None,
    absolute_upper_bound=None,
    absolute_lower_bound=None,
    incidence_df=incidence_df,
    control_totals=control_totals,
    initial_weights=initial_weights,
    use_hard_constraints=False,
    use_numba=True,
    numba_precision='float64'
)

print(f"Converged: {status['converged']}")
print(f"Iterations: {status['iter']}")
print(f"Final weights:
{weights_df['final']}")
```

--------------------------------

### Define PopulationSim Run Steps and Resumption

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

Lists the sequence of sub-modules or steps for the PopulationSim orchestrator. Allows resuming a run from a specific point using 'resume_after'. Supports repeated steps for sub-geographies.

```yaml
run_list:
  steps:
    - input_pre_processor
    - setup_data_structures
    - initial_seed_balancing
    - meta_control_factoring
    - final_seed_balancing
    - integerize_final_seed_weights
    - sub_balancing.geography=TRACT
    - sub_balancing.geography=TAZ
    - expand_households
    - write_results
    - summarize

  #resume_after: integerize_final_seed_weights
```

--------------------------------

### Configuring Repopulation Mode

Source: https://context7.com/activitysim/populationsim/llms.txt

Sets up the YAML configuration for repopulation mode, specifying control files and the model pipeline steps required to update specific geographic zones.

```yaml
inherit_settings: True
repop_control_file_name: repop_controls.csv
repop_input_table_list:
  - tablename: TAZ_control_data
    filename: repop_control_totals_taz.csv
models:
  - input_pre_processor.table_list=repop_input_table_list
  - repop_setup_data_structures
  - initial_seed_balancing.repop=True
  - sub_balancing.geography=TAZ
  - expand_households.replace=True
  - write_tables
  - write_synthetic_population
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/activitysim/populationsim/blob/master/docs/getting_started.rst

These commands demonstrate how to create a new conda environment named 'popsim' with Python 3.8 and then activate it. This isolates project dependencies. Activation commands differ slightly between Windows and macOS/Linux.

```bash
conda create -n popsim python=3.8

# Windows
activate popsim

# Mac
conda activate popsim
```

--------------------------------

### Create Project Runner Script

Source: https://context7.com/activitysim/populationsim/llms.txt

A template for a custom runner script that uses the inject module to define logging settings and execute the PopulationSim pipeline.

```python
import os
import sys
import argparse
from populationsim.core import inject
import populationsim

@inject.injectable()
def log_settings():
	return [
		"multiprocess", "num_processes", "resume_after",
		"GROUP_BY_INCIDENCE_SIGNATURE", "INTEGERIZE_WITH_BACKSTOPPED_CONTROLS",
		"SUB_BALANCE_WITH_FLOAT_SEED_WEIGHTS", "meta_control_data",
		"control_file_name", "USE_CVXPY", "USE_SIMUL_INTEGERIZER"
	]

if __name__ == "__main__":
	parser = argparse.ArgumentParser()
	populationsim.add_run_args(parser)
	args = parser.parse_args()
	args.working_dir = os.path.dirname(__file__)
	sys.exit(populationsim.run(args))
```

--------------------------------

### Write Synthetic Population Tables (Python)

Source: https://context7.com/activitysim/populationsim/llms.txt

Outputs the final synthetic households and persons tables as CSV files. This step merges expanded household IDs with seed data attributes and assigns unique synthetic household IDs.

```python

```

--------------------------------

### Executing Repopulation Workflow

Source: https://context7.com/activitysim/populationsim/llms.txt

Provides the shell commands to perform a base synthesis, migrate the pipeline state, and execute the repopulation process for specific zones.

```bash
cd example_calm
python run_populationsim.py
cp output/pipeline.h5 ../example_calm_repop/output/
cd ../example_calm_repop
python run_populationsim.py
```

--------------------------------

### Expand Households to Synthetic Population (Python)

Source: https://context7.com/activitysim/populationsim/llms.txt

Creates the final synthetic population by expanding integerized weights into individual household records. Households with integer_weight > 0 are replicated according to their weight.

```python
# expand_households creates the expanded_household_ids table:
# | PUMA | TRACT | TAZ | hh_id |
# |------|-------|-----|-------|
# | 600  | 10100 | 100 | 0     |
# | 600  | 10100 | 100 | 0     |  # Same hh_id appears twice (integer_weight=2)
# | 600  | 10100 | 100 | 1     |
# | 600  | 10100 | 101 | 5     |

# When GROUP_BY_INCIDENCE_SIGNATURE is True, households are grouped
# by their incidence pattern, and expansion randomly selects from
# group members based on their sample weights

# For repop mode, use append or replace:
# expand_households.append=True   # Add to existing population
# expand_households.replace=True  # Replace zones in repop crosswalk
```

--------------------------------

### Configure PopulationSim Settings

Source: https://context7.com/activitysim/populationsim/llms.txt

Defines the core configuration for PopulationSim, including data directories, input table specifications, output table settings, and the model pipeline execution order.

```yaml
trace_geography:
  TAZ: 100
  TRACT: 10200

data_dir: data

input_table_list:
  - tablename: households
    filename: seed_households.csv
    index_col: hh_id
    rename_columns:
      hhnum: hh_id

models:
  - input_pre_processor
  - setup_data_structures
  - initial_seed_balancing
  - summarize
  - write_tables
```

--------------------------------

### Load and Process Household Weights

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Reads seed household data and expanded household IDs from CSV files, then calculates the expansion factors per geography.

```python
seed_cols = (seed_hh_cols.values())
geog = seed_hh_cols.get('geog')
geog_weight = seed_hh_cols.get('geog_weight')
hh_id = seed_hh_cols.get('hh_id')

expanded_hhids = pd.read_csv(os.path.join(popsim_dir, exp_hh_file), usecols=[exp_hh_id_col])
seed_hh_df = pd.read_csv(os.path.join(popsim_dir, seed_hh_file), usecols=seed_cols, index_col=hh_id)
```

--------------------------------

### Configure Multiprocessing in PopulationSim Settings

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

This YAML configuration defines settings for running PopulationSim with multiprocessing. It specifies the number of processes, whether to clean up intermediate pipelines, and how to slice geographic data for parallel processing. The `multiprocess_steps` section details which model steps run in single or multiple processes and how data should be sliced or coalesced.

```yaml
inherit_settings: True
multiprocess: True
num_processes: 2
cleanup_pipeline_after_run: True
slice_geography: TRACT

multiprocess_steps:
  - name: mp_seed_balancing
    begin: input_pre_processor
  - name: mp_sub_balancing_TAZ
    begin: sub_balancing.geography=TAZ
    num_processes: 2
    slice:
      tables:
        - slice_crosswalk
        - crosswalk
      # don't slice any tables not explicitly listed above in slice.tables
      except: True
      # the following tables are added by sub_balancer and should be coalesced
      coalesce:
        - TAZ_weights
        - TAZ_weights_sparse
        - trace_TAZ_weights
  - name: mp_summarize
    begin: expand_households
```

--------------------------------

### Specify Region YAML Configuration

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Defines the path to the YAML configuration file for specifying PopulationSim model run details and desired summaries. This allows for regional customization without altering the main script.

```python
region_yaml = 'calm_verification.yaml'  # default
```

--------------------------------

### Configure Repop Mode Input Tables

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

This YAML configuration specifies the input data tables for PopulationSim's 'repop' mode. The `repop_input_table_list` defines the control data files, including their filenames and the specific table names within those files, used for repopulating a subset of geographies.

```yaml
repop_input_table_list:
  - taz_control_data:
    filename : repop_control_totals_taz.csv
    tablename: TAZ_control_data
```

--------------------------------

### Define PopulationSim Input Tables

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

Configures the input data pipeline by mapping CSV files to internal table names, defining index columns, and specifying column renames or drops.

```yaml
input_table_list:
- tablename: households
	filename : seed_households.csv
	index_col: hh_id
	rename_columns:
	hhnum: hh_id
- tablename: persons
	filename : seed_persons.csv
	rename_columns:
	hhnum: hh_id
	SPORDER: per_num
	drop_columns:
	- indp02
	- naicsp02
	- occp02
	- socp00
	- occp10
	- socp10
	- indp07
	- naicsp07
- tablename: geo_cross_walk
	filename : geo_cross_walk.csv
	rename_columns:
	TRACTCE: TRACT
- tablename: TAZ_control_data
	filename : control_totals_taz.csv
- tablename: TRACT_control_data
	filename : control_totals_tract.csv
- tablename: REGION_control_data
	filename : scaled_control_totals_meta.csv
```

--------------------------------

### Create Validation Directory

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Checks for the existence of a validation directory and creates it if it does not exist. This ensures that output files have a valid destination path.

```python
if not os.path.isdir(validation_dir):
    os.mkdir(validation_dir)
```

--------------------------------

### Configure Output Tables in PopulationSim

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

Controls which output tables are written to disk using 'include' or 'skip' actions. The HDF5 pipeline and summary files are always written.

```yaml
output_tables:
  action: include
  tables:
    - expanded_household_ids
```

--------------------------------

### Configure Table Writing Options

Source: https://context7.com/activitysim/populationsim/llms.txt

Configures the write_tables pipeline step to include or skip specific tables, specify output format (CSV/HDF5), and add prefixes to output filenames. This is useful for debugging and production data export.

```yaml
# Include specific tables only
output_tables:
  action: include
  tables:
    - summary_TAZ
    - summary_TRACT
    - expanded_household_ids
    - households
    - persons
```

```yaml
# Skip specific tables (output all others)
output_tables:
  action: skip
  tables:
    - households
    - persons
```

```yaml
# Output to HDF5 instead of CSV
output_tables:
  h5_store: True
  action: include
  tables:
    - summary_TAZ
    - expanded_household_ids
```

```yaml
# Add prefix to output files (default: "final_")
output_tables:
  prefix: "output_"
  action: include
  tables:
    - summary_TAZ
```

--------------------------------

### Configure Repop Mode Control File Name

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

This YAML setting specifies the filename for the control data used in PopulationSim's 'repop' mode. This file contains the control totals for the geographic units that are to be repopulated.

```yaml
repop_control_file_name: repop_controls.csv
```

--------------------------------

### Visualize and Export Validation Metrics

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Iterates through a list of parameters to generate frequency histograms for control vs. result differences and exports the aggregated statistics to a CSV file.

```python
fig_w = 3
fig_l = int(len(aggregate_list) / fig_w) + 1
summary_fig, axes = plt.subplots(fig_l, fig_w, figsize=(fig_w * 5,fig_l*5))
stats = []
for params, ax in zip(aggregate_list, axes.ravel()):
    s, f, diff = process_control(summary_df, **params)
    stats.append(s)
    ax.set_title(f"{params['geography']} - {params['name']}")
    ax.set_ylabel('Frequency'); ax.set_xlabel('Difference: control vs. result')
    ax.hist(diff, bins=10, range=(-5,5), alpha=0.5)
summary_fig.savefig(os.path.join(validation_dir, 'frequencies.pdf'))
stats_df = pd.DataFrame(stats)
stats_df.to_csv(os.path.join(validation_dir, f'{scenario}_{region}_popsim_stats.csv'), index=False)
```

--------------------------------

### Visualize Expansion Factor Distribution

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Generates histograms for each geographic region to visualize the distribution of expansion factors.

```python
geogs = df[geog].unique()
geog_fig = plt.figure(figsize=(10*len(geogs), 10))

for i, g in enumerate(geogs):
    geog_df = df[df[geog] == g]
    counts, bins = np.histogram(geog_df['ef'])
    ax = geog_fig.add_subplot(1, len(geogs), i+1)
    ax.set_title(f'{geog} {g}')
    ax.set_ylabel('Percentage'); ax.set_xlabel('Expansion Factor Range')
    ax.hist(bins[:-1], bins, weights=counts*100/len(geog_df), alpha=0.6)
```

--------------------------------

### Configure Matplotlib Plotting Styles

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Sets up the plotting environment for Matplotlib, including enabling inline plots in Jupyter notebooks, defining a color cycle for plots, and configuring figure size and layout. These settings ensure consistent and aesthetically pleasing visualizations.

```python
%matplotlib inline
from matplotlib import cycler
plt.style.use('ggplot')
plt.rcParams['lines.linewidth'] = 1.5
plt.rcParams['axes.prop_cycle'] = cycler(color=['b', 'g', 'r', 'y'])
plt.rcParams['savefig.dpi'] = 200
plt.rcParams['figure.constrained_layout.use'] = True
plt.rcParams['figure.constrained_layout.h_pad'] = 0.5
plt.rcParams['figure.figsize'] = (8,15)
plt.rcParams['hist.bins'] = 25
```

--------------------------------

### Survey Weighting Mode: Output Tables Configuration

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

Specifies that the 'seed_geography_weights' table should be included in the output for survey weighting mode, which contains detailed weight information.

```yaml
output_tables:
  action: include
  tables:
    - seed_geography_weights
    ...
```

--------------------------------

### Configuring Multiprocessing

Source: https://context7.com/activitysim/populationsim/llms.txt

Defines the YAML configuration structure for enabling parallel processing in PopulationSim, including inheritance, process count, and memory management settings.

```yaml
inherit_settings: True
multiprocess: True
num_processes: 4
slice_geography: PUMA
fail_fast: True
mem_tick: 30
```

--------------------------------

### Specify Synthetic Population Output in PopulationSim

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

Defines the output filenames and columns for synthetic households and persons. Requires specifying the household ID column and desired columns from the seed sample.

```yaml
output_synthetic_population:
  household_id: household_id
  households:
    filename: synthetic_households.csv
    columns:
      - NP
      - AGEHOH
      - HHINCADJ
      - NWESR
  persons:
    filename: synthetic_persons.csv
    columns:
      - per_num
      - AGEP
      - OSUTAG
      - OCCP
```

--------------------------------

### Configure PopulationSim Seed Columns

Source: https://github.com/activitysim/populationsim/blob/master/scripts/validation.ipynb

Defines the mapping for geographic and household identifier columns used in the simulation process.

```yaml
seed_cols:
    geog: PUMA
    geog_weight: WGTP
    hh_id: hh_id
```

--------------------------------

### Set PopulationSim Data Directory

Source: https://github.com/activitysim/populationsim/blob/master/docs/application_configuration.rst

Specifies the relative path to the data directory within the working directory.

```yaml
data_dir: data
```