### CRDS Test Setup and Sync

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Sets up the CRDS environment for testing, including specifying a test server URL and syncing specific CRDS contexts.

```bash
export CRDS_SERVER_URL=https://roman-crds-test.stsci.edu
export CRDS_PATH=$HOME/roman-crds-test-cache
crds sync --contexts roman-edit
```

--------------------------------

### Install Additional Development Tools

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs common utility packages like IPython and pytest-xdist for enhanced development workflows.

```bash
pip install ipython pytest-xdist
```

--------------------------------

### Install Development Version from GitHub

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/pipeline_installation.rst

Installs the latest development version of romancal directly from its GitHub repository.

```bash
conda create -n <env_name> python
conda activate <env_name>
pip install git+https://github.com/spacetelescope/romancal
```

--------------------------------

### Install Developer Dependencies

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/pipeline_installation.rst

Installs additional dependencies for testing and documentation building when installing romancal for development.

```bash
pip install -e '.[test]'
pip install -e '.[docs]'
pip install -e '.[test,docs]'
```

--------------------------------

### Install Development Dependencies from Requirements Files

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs STScI-specific and third-party development dependencies listed in separate requirement files.

```bash
pip install -r requirements-dev-st.txt
pip install -r requirements-dev-thirdparty.txt
```

--------------------------------

### Build Documentation Locally

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs necessary dependencies and builds the project's HTML documentation from the source code.

```bash
pip install -e ".[docs]"
cd docs
make html
```

--------------------------------

### Install Development Version from GitHub

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs the latest code directly from the romancal GitHub repository's main branch. This version may be less stable.

```bash
conda create -n <env_name> python
conda activate <env_name>
pip install git+https://github.com/spacetelescope/romancal
```

--------------------------------

### Install for Developers (Editable)

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/pipeline_installation.rst

Installs the romancal package in editable mode from a local clone of the repository, suitable for development.

```bash
conda create -n <env_name> python
conda activate <env_name>
cd <where you want to put the repo>
git clone https://github.com/spacetelescope/romancal
cd romancal
pip install -e .
```

--------------------------------

### Install romancal with Full Development Dependencies

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs the romancal package with all development dependencies, including testing, documentation, linting, and pre-commit tools.

```bash
pip install -e ".[dev]"
```

--------------------------------

### Run Unit Tests

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs testing dependencies and executes the unit test suite using pytest.

```bash
pip install -e ".[test]"
pytest
```

--------------------------------

### Preparing and uploading to test PyPI

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Installs necessary build tools, cleans the directory, builds the package, and uploads it to the test PyPI server.

```shell
pip install twine build
git checkout a.b.c
git clean -xdf
umask 0022
chmod -R a+Xr .
python -m build .
twine check --strict dist/*
twine upload --repository testpypi dist/*
```

--------------------------------

### Install romancal with Test and Docs Dependencies

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs the romancal package along with dependencies required for running unit/regression tests and building documentation.

```bash
pip install -e ".[test]"
pip install -e ".[docs]"
pip install -e ".[test,docs]"
```

--------------------------------

### Developer Install romancal from Local Clone

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs the romancal package in an editable mode from a local copy of the repository, allowing for direct code modifications and testing.

```bash
cd <where you want to put the repo>
git clone https://github.com/spacetelescope/romancal
cd romancal
pip install -e .
```

--------------------------------

### Testing installed package from test PyPI

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Installs the package from test PyPI and runs automated tests using pytest.

```shell
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple romancal[test]==a.b.c
pip install pytest-xdist
pushd /
pytest -n auto --pyargs romancal
popd
```

--------------------------------

### CRDS Setup for External Use

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Configures the CRDS environment variables to point to the correct cache path and server URL for running the Roman pipeline outside the STScI network.

```bash
export CRDS_PATH=$HOME/crds_cache
export CRDS_SERVER_URL=https://roman-crds.stsci.edu
```

--------------------------------

### Parallelize Unit Tests

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs the pytest-xdist plugin to enable parallel execution of unit tests across multiple CPU cores.

```bash
pip install pytest-xdist
pytest -n 8
```

--------------------------------

### Install Developer Dependencies

Source: https://github.com/spacetelescope/romancal/wiki/Workflow-for-Contributors

Installs the necessary development dependencies for the romancal project using pip and a requirements file.

```Shell
cd romancal
pip install -r requirements-dev-st.txt
pip install .[test]
```

--------------------------------

### Install Specific Stable Release

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/pipeline_installation.rst

Installs a specific version of the romancal package using pip, allowing for version control.

```bash
conda create -n <env_name> python
conda activate <env_name>
pip install romancal==0.5.0
```

--------------------------------

### Build Changelog Entry

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Installs towncrier and builds the next changelog entry from fragments in the 'changes/' directory.

```shell
pip install towncrier
towncrier build --version 0.16.1
```

--------------------------------

### Install Specific romancal Version

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs a particular version of the romancal package, useful for reproducibility or testing older versions.

```bash
conda create -n <env_name> python
conda activate <env_name>
pip install romancal==0.10.0
```

--------------------------------

### Install Latest romancal Release

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs the most recent stable version of the romancal package into a specified conda environment.

```bash
conda create -n <env_name> python
conda activate <env_name>
pip install romancal
```

--------------------------------

### Setup Upstream Remote

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Adds the central romancal repository as an upstream remote for fetching updates.

```shell
git remote add upstream https://github.com/spacetelescope/romancal.git
```

--------------------------------

### Custom Source Catalog Example (Bash)

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/tweakreg/tweakreg_examples.rst

Provides an example of a custom source catalog in CSV format, used for astrometric calibration when custom catalogs are enabled.

```Bash
$ cat ref_catalog_1

x,y
846.1321662446178,945.839358133909
664.7073537074112,1028.4613139252003
1036.160742774408,642.3379043578552
935.8827367579428,594.1745467413945
1491.9672737821606,1037.4723609624757
735.1256651803337,1410.2791591559157
1358.2876707625007,651.7112260833995
526.4715950130742,751.745104066621
1545.082698426152,703.601696337681
374.9609365496525,972.6561578187437
1110.3498547121228,1644.2214966576498
341.18333252240654,891.4733849441861
820.0520846885105,312.0088351823117
567.7054174813052,386.8883078361564
1447.356249085851,1620.3390168916592
1400.4271386280673,1674.3765672924937
1681.9744852889235,571.6748779060324
959.7317254404431,197.8757865066898
1806.3360866990297,769.0603031839573
487.1560001146406,257.30706691141086
1048.7910126076483,85.36675265982751
1075.508595999755,29.085099663125334
```

--------------------------------

### Custom Catalog Mapping File Example (Bash)

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/tweakreg/tweakreg_examples.rst

Illustrates the format of a mapping file (`catfile`) that links Roman datamodel files to their corresponding custom source catalogs.

```Bash
$ cat /path/to/catfile/catfilename

img1 ref_catalog_1
img2 ref_catalog_2
img3 ref_catalog_3
```

--------------------------------

### RomanCal DQInitStep Example

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/pipeline_naming_conventions.rst

Demonstrates the execution of the DQInitStep in RomanCal, showing how the output file name is generated based on the input file name and the step name.

```bash
$ strun romancal.dq_init.DQInitStep r0008308002010007027_0019_wfi01_uncal.asdf

produces an output file named
r0008308002010007027_0019_wfi01_dq_init.asdf.
```

--------------------------------

### ASDF Parameter File Example

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/user_step.rst

An example of an ASDF parameter file used to configure a `stpipe.cleanup` step. It specifies the class, an optional name, and parameters like threshold and scale.

```ASDF
#ASDF 1.0.0
#ASDF_STANDARD 1.3.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
---
!core/asdf-1.1.0
class: stpipe.cleanup
name: MyCleanup
parameters:
  threshold: 42.0
  scale: 0.01
...
```

--------------------------------

### Run romancal Regression Tests Locally

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Installs test dependencies, sets the TEST_BIGDATA environment variable (for STSci staff), and runs pytest with the --bigdata flag for romancal regression tests.

```bash
pip install -e ".[test]"
export TEST_BIGDATA=https://bytesalad.stsci.edu/artifactory
pytest --bigdata romancal/regtest
```

--------------------------------

### Creating a GitHub Release

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Instructions for creating a new release on GitHub by drafting a new release and selecting an existing tag.

```shell
Go to https://github.com/spacetelescope/romancal/releases
Click Draft a new release
Select the tag and title the release
```

--------------------------------

### ExposurePipeline Configuration (YAML)

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/user_pipeline.rst

Example ASDF parameter file for configuring the ExposurePipeline, specifying parameters for individual steps like DQInitStep and SaturationStep.

```yaml
#ASDF 1.0.0
#ASDF_STANDARD 1.5.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
---
!core/asdf-1.1.0
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf', name: asdf, version: 2.13.0}
history:
extensions:
- !core/extension_metadata-1.0.0
  extension_class: asdf.extension.BuiltinExtension
  software: !core/software-1.0.0 {name: asdf, version: 2.13.0}
class: romancal.pipeline.exposure_pipeline.ExposurePipeline
meta:
  author: <SPECIFY>
  date: '2022-09-15T13:59:54'
  description: Parameters for calibration step romancal.pipeline.exposure_pipeline.ExposurePipeline
  instrument: {name: <SPECIFY>}
  origin: <SPECIFY>
  pedigree: <SPECIFY>
  reftype: <SPECIFY>
  telescope: <SPECIFY>
  useafter: <SPECIFY>
name: ExposurePipeline
parameters:
  input_dir: ''
  output_dir: null
  output_ext: .asdf
  output_file: null
  output_use_index: true
  output_use_model: false
  post_hooks: []
  pre_hooks: []
  save_results: true
  search_output_file: true
  skip: false
  suffix: null
  steps:
  - class: romancal.dq_init.dq_init_step.DQInitStep
    name: dq_init
    parameters:
      input_dir: ''
      output_dir: null
      output_ext: .asdf
      output_file: null
      output_use_index: true
      output_use_model: false
      post_hooks: []
      pre_hooks: []
    save_results: false
    search_output_file: true
    skip: false
    suffix: null
  - class: romancal.saturation.saturation_step.SaturationStep
...
```

--------------------------------

### Run Specific Regression Test

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Demonstrates how to execute a specific regression test by using the -k option with pytest, allowing filtering by test name or ID.

```bash
pytest --bigdata romancal/regtest -k test_flat
```

--------------------------------

### Run TweakReg with Default Parameters (Python)

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/tweakreg/tweakreg_examples.rst

Demonstrates how to initialize and run the TweakRegStep with default parameters using a Roman datamodel or ASDF file.

```Python
from romancal.tweakreg.tweakreg_step import TweakRegStep
step = TweakRegStep()
step.run([img])
```

--------------------------------

### ASDF File Structure Example

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/config_asdf.rst

Provides an example of the content of an ASDF parameter file, showcasing the preamble, YAML structure, and key-value pairs for pipeline configuration.

```yaml
#ASDF 1.0.0
#ASDF_STANDARD 1.5.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
---
!core/asdf-1.1.0
asdf_library: !core/software-1.0.0 {author: The ASDF Developers, homepage: 'http://github.com/asdf-format/asdf', name: asdf, version: 3.3.0}
history:
extensions:
- !core/extension_metadata-1.0.0
  extension_class: asdf.extension._manifest.ManifestExtension
  extension_uri: asdf://asdf-format.org/core/extensions/core-1.5.0
  manifest_software: !core/software-1.0.0 {name: asdf_standard, version: 1.1.1}
  software: !core/software-1.0.0 {name: asdf, version: 3.3.0}
class: romancal.pipeline.exposure_pipeline.ExposurePipeline
meta:
  author: <SPECIFY>
  date: '2024-11-20T19:41:22'
  description: Parameters for calibration step romancal.pipeline.exposure_pipeline.ExposurePipeline
  instrument: {name: <SPECIFY>}
  origin: <SPECIFY>
  pedigree: <SPECIFY>
  reftype: <SPECIFY>
  telescope: <SPECIFY>
  useafter: <SPECIFY>
name: ExposurePipeline
parameters:
  input_dir: rsim
  output_dir: null
  output_ext: .asdf
  output_file: null
  output_use_index: true
  output_use_model: false
  post_hooks: []
  pre_hooks: []
  save_results: true
  search_output_file: true
  skip: false
  suffix: cal
steps:
- class: romancal.source_catalog.source_catalog_step.SourceCatalogStep
  name: source_catalog
  parameters:
    aperture_ee1: 30
    aperture_ee2: 50
    aperture_ee3: 70
    bkg_boxsize: 1000
    ci1_star_threshold: 2.0
    ci2_star_threshold: 1.8
    deblend: false
    fit_psf: true
    input_dir: rsim
    kernel_fwhm: 2.0
    npixels: 25
    output_dir: null
    output_ext: .asdf
    output_file: null
    output_use_index: true
    output_use_model: false
    post_hooks: []
    pre_hooks: []
    save_results: false
    search_output_file: true
    skip: false
    snr_threshold: 3.0
    suffix: cat
    ...

```

--------------------------------

### Example: Save ExposurePipeline Parameters

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/parameter_files.rst

An example demonstrating how to save the parameters for the ExposurePipeline to a file named 'my_exppars.asdf'. This file can then be edited to customize parameter values.

```bash
$ strun roman_elp r0000101001001001001_0001_wfi01_uncal.asdf --save-parameters my_exppars.asdf
```

--------------------------------

### Logging Configuration Example

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/user_logging.rst

An example of a logging configuration file in INI format. This configuration sets the logging level to INFO and directs all log messages to a file named 'myrun.log'.

```INI
[*]
level = INFO
handler = file:myrun.log
```

--------------------------------

### Tagging a release

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Creates an annotated git tag for a release and pushes it to the upstream repository.

```shell
git tag -a a.b.c -m "Romancal DMS Build 23Q4_B11"
git push upstream a.b.c
```

--------------------------------

### Install romancal with SDP extra

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/pipeline_static_preview.rst

Installs the romancal package with the 'sdp' extra, which includes the requirements for the static preview script.

```shell
pip install "romancal[sdp]"
```

--------------------------------

### SourceCatalogStep Configuration (YAML)

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/user_pipeline.rst

Example YAML snippet showing how to configure parameters for the SourceCatalogStep within a pipeline, including aperture and deblending parameters.

```yaml
- class: romancal.source_catalog.source_catalog_step.SourceCatalogStep
  name: SourceCatalogStep
  parameters:
    aperture_ee1: 30
    aperture_ee2: 50
    aperture_ee3: 70
    bkg_boxsize: 1000
    ci1_star_threshold: 2.0
    ci2_star_threshold: 1.8
    deblend: false
    fit_psf: true
```

--------------------------------

### Viewing Pipeline/Step Parameters

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/pipeline_parameters.rst

Illustrates how to view available parameters for a pipeline or step using the '-h' option with 'strun'. Examples are provided for different pipelines and steps.

```bash
$ strun roman_elp -h
```

```bash
$ strun romancal.dq_init.DQInitStep -h
```

```bash
$ strun roman_mos -h
```

```bash
$ strun romancal.skymatch.SkyMatchStep -h
```

--------------------------------

### Clone and Set Up Upstream Remote

Source: https://github.com/spacetelescope/romancal/wiki/Workflow-for-Contributors

Clones the forked romancal repository and sets the upstream remote to the main repository for tracking changes.

```Shell
git clone https://github.com/<your-user-name>/romancal.git
cd romancal
git remote add upstream https://github.com/spacetelescope/romancal.git
```

--------------------------------

### Push Feature Branch and Create Pull Request

Source: https://github.com/spacetelescope/romancal/wiki/Workflow-for-Contributors

Pushes the local feature branch to the forked repository and provides instructions on how to create a pull request on GitHub.

```Shell
git push origin feature1
```

--------------------------------

### Commit Metadata Updates

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Stages and commits changes to the changelog and README files, preparing for a release.

```shell
git add CHANGES.rst README.md 
git commit -m "metadata updates for release 0.3.0"
```

--------------------------------

### Exporting Configuration for SourceCatalogStep

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/config_asdf.rst

Provides a Python code example using the `Step.export_config` method to create an ASDF parameter file for the `SourceCatalogStep`.

```Python
from romancal..source_catalog import SourceCatalogStep
step = SourceCatalogStep()
step.export_config('source_catalog_step.asdf')
```

--------------------------------

### CRDS Setup for External Use

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/pipeline_installation.rst

Configures the Calibration References Data System (CRDS) by setting environment variables for accessing reference files outside the STScI network.

```bash
export CRDS_PATH=$HOME/crds_cache
export CRDS_SERVER_URL=https://roman-crds.stsci.edu
```

--------------------------------

### strun Command-line Usage for Step Help

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/user_step.rst

Demonstrates how to use the `strun` command to display help and accepted parameters for a specific Step class, such as `romancal.dq_init.DQInitStep`.

```Shell
$ strun -h romancal.dq_init.DQInitStep
```

--------------------------------

### Install Latest Stable Release

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/pipeline_installation.rst

Installs the most recent stable version of the romancal package using pip within a conda environment.

```bash
conda create -n <env_name> python
conda activate <env_name>
pip install romancal
```

--------------------------------

### Install Romancal Package

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Confirms the installation of a specific version of the romancal package using pip.

```shell
pip install romancal==a.b.c
```

--------------------------------

### Setup.cfg Update for Warnings and Errors

Source: https://github.com/spacetelescope/romancal/blob/main/CHANGES.rst

The `setup.cfg` file has been updated to match JWST warnings and error lists, and includes an initial pass for code fixes.

```Python
# Update setup.cfg to match JWST warnings & error list and initial pass for code fixes. (#188)
```

--------------------------------

### strun Command-line Arguments

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/user_step.rst

Lists the selected optional arguments available when using the `strun` command to run a step, including help, logging configuration, verbosity, debugging, and parameter saving.

```APIDOC
usage: strun [-h] [--logcfg LOGCFG] [--verbose] [--debug] [--save-parameters SAVE_PARAMETERS] [--disable-crds-steppars]
             [--pre_hooks] [--post_hooks] [--output_file] [--output_dir] [--output_ext] [--output_use_model] [--output_use_index]
             [--save_results] [--skip] [--suffix] [--search_output_file] [--input_dir] [--override_mask]
             cfg_file_or_class [args ...]

(selected) optional arguments:
  -h, --help       show this help message and exit
  --logcfg LOGCFG  The logging configuration file to load
  --verbose, -v    Turn on all logging messages
  --debug            When an exception occurs, invoke the Python debugger, pdb
  --save-parameters SAVE_PARAMETERS Save step parameters to specified file.
  --disable-crds-steppars Disable retrieval of step parameter references files from CRDS
  --output_file   File to save the output to
```

--------------------------------

### Get CRDS Operational Context

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Sets the CRDS server URL to the test server and lists the current operational context.

```shell
export CRDS_SERVER_URL=https://roman-crds-test.stsci.edu
crds list --operational-context
```

--------------------------------

### Accessing Metadata Example

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/datamodels/metadata.rst

Demonstrates accessing metadata, specifically the observation start time, using the 'meta' member of a data model.

```python
print(model.meta.observation.start_time)
```

--------------------------------

### Example RomanCal Step Implementation

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/devel_step.rst

Demonstrates the structure of a custom step class inheriting from RomanStep. It includes the 'process' method for core logic, parameter access, input model handling, reference file usage, and returning results. The 'spec' member defines configurable parameters.

```Python
from romancal.stpipe import RomanStep

from roman_datamodels.datamodels import ImageModel
from my_awesome_astronomy_library import combine

class ExampleStep(RomanStep):
    """
    Every step should include a docstring.  This docstring will be
    displayed by the `strun --help`.
    """

    # 1.
    def process(self, image1, image2):
        # 2.
        threshold = self.threshold

        # 3.
        with ImageModel(image1) as image1, ImageModel(image2) as image2:
            # 4.
            with self.get_reference_file_model(image1, "flat_field") as flat:
                new_image = combine(image1, image2, flat, threshold)

        # 5.
        return new_image

   # 6.
   spec = """
   # This is the configspec file for ExampleStep

   threshold = float(default=1.0)  # maximum flux
   """

   # 7.
   reference_file_types = ['flat_field']
```

--------------------------------

### Run TweakReg with Custom Source Catalogs (Python)

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/tweakreg/tweakreg_examples.rst

Demonstrates how to configure and run TweakReg to use custom source catalogs, specifying the catalog format and the path to the mapping file.

```Python
from romancal.tweakreg.tweakreg_step import TweakRegStep
step = TweakRegStep()
step.use_custom_catalogs = True
step.catalog_format = "ascii.ecsv"
step.catfile = '/path/to/catfile/catfilename'
step.run([img])
```

--------------------------------

### Control Test Output Directory

Source: https://github.com/spacetelescope/romancal/blob/main/README.md

Specifies how to control the base temporary directory used by pytest for test results, warning that this directory will be wiped clean for each test session.

```bash
pytest --bigdata romancal/regtest --basetemp=<PATH>
```

--------------------------------

### Example Pipeline Definition - Python

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/devel_pipeline.rst

Demonstrates defining a pipeline with multiple steps and custom parameters. The pipeline inherits from `stpipe.Pipeline` and uses `step_defs` to map step names to classes. The `process` method outlines the data flow, and the `spec` member defines pipeline-level parameters.

```Python
from romancal.stpipe import Pipeline

from roman_datamodels.datamodels import ImageModel

# Some locally-defined steps
from . import FlatField, Combine

class ExamplePipeline(Pipeline):
    """
    This example pipeline demonstrates how to combine steps
    using Python code, in some way that it not necessarily
    a linear progression.
    """

    step_defs = {
        'flat_field': FlatField,
        'combine': Combine,
        }

    def process(self, input):
        with ImageModel(input) as science:

            flattened = self.flat_field(science, self.multiplier)

            combined = self.combine(flattened)

        return combined

    spec = """
    multiplier = float()     # A multiplier constant
    """

```

--------------------------------

### Run TweakReg with Association File (Python)

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/tweakreg/tweakreg_examples.rst

Shows how to run the TweakRegStep using an association file, which typically contains metadata for processing multiple related data products.

```Python
from romancal.tweakreg.tweakreg_step import TweakRegStep
step = TweakRegStep()
step.run("asn_file.json")
```

--------------------------------

### Step Execution Methods in Python

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/user_step.rst

Demonstrates different ways to execute ROMANCAL steps programmatically within a Python environment, catering to various levels of control and familiarity with command-line tools.

```Python
from romancal.pipeline import ExposurePipeline

# Equivalent to: $ strun romancal.pipeline.ExposurePipeline r0000101001001001001_0001_wfi01_uncal.asdf \
#         --steps.ramp_fit.override_gain=roman_wfi_gain_0033.asdf
ExposurePipeline.from_cmdline([
    'r0000101001001001001_0001_wfi01_uncal.asdf',
    'steps.ramp_fit.override_gain',
    'roman_wfi_gain_0033.asdf'
])
```

```Python
from romancal.jump import JumpStep

# Basic execution
output = JumpStep.call('r0000101001001001001_0001_wfi01_uncal.asdf')

# With a specific parameter value
output = JumpStep.call('r0000101001001001001_0001_wfi01_uncal.asdf',
                       rejection_threshold=42.0)

# With a parameter configuration file
output = JumpStep.call('r0000101001001001001_0001_wfi01_uncal.asdf',
                       config_file='my_jumpstep_config.asdf')
```

```Python
from romancal.flatfield import FlatFieldStep

# Using instance method with parameter set on instance
mystep = FlatFieldStep()
mystep.override_sflat = 'sflat.asdf'
output = mystep.run(input)

# Using instance method with parameter during instantiation
mystep = FlatFieldStep(override_sflat='sflat.asdf')
output = mystep.run(input)
```

--------------------------------

### Scripts: MosaicModel and Schema Installation

Source: https://github.com/spacetelescope/romancal/blob/main/CHANGES.rst

Updates scripts by removing the installation of 'schema_editor' and 'schemadoc' and allowing `MosaicModel` in `roman_static_preview`.

```Python
Scripts
- Remove install of missing scripts "schema_editor" and "schemadoc". (#1538)
- allow `MosaicModel` in `roman_static_preview` (#1613)
```

--------------------------------

### Execute ExposurePipeline Directly

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/call_via_run.rst

Demonstrates how to instantiate and run the ExposurePipeline class directly, manually setting parameters like 'skip', 'override_gain', 'save_result', and 'output_dir' before execution.

```python
from romancal.pipeline import ExposurePipeline

# Instantiate the class. Do not provide a parameter file.
pipe = ExposurePipeline()

# Manually set any desired non-default parameter values
pipe.assign_wcs.skip = True
pipe.ramp_fit.override_gain = 'my_gain_file.asdf'
pipe.save_result = True
pipe.output_dir = '/my/data/pipeline_outputs'

# Run the pipeline
result = pipe.run('r0000101001001001001_0001_wfi01_uncal.asdf')
```

--------------------------------

### Execute Pipeline Step with call() and config_file

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/call_via_call.rst

Demonstrates executing a pipeline step with `call()`, providing custom parameters via a configuration file.

```Python
# Calling a step
result = LinearityStep.call('r0000101001001001001_01101_0001_WFI01_dqinit.asdf', config_file = 'linearity_pars.asdf')
```

--------------------------------

### DQ Step Flags Guide Window Data

Source: https://github.com/spacetelescope/romancal/blob/main/CHANGES.rst

The DQ (Data Quality) step has been updated to flag science data affected by guide window reads. This improves the identification of data affected by instrumental artifacts.

```Python
# DQ step flags science data affected by guide window read
```

--------------------------------

### RomanCal Sky Matching Examples

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/skymatch/description.rst

Provides example data tables illustrating the results of 'Local', 'Global', 'Match', and 'Global+Match' sky level computations for two different sets of images, demonstrating the impact of mosaic configuration and overlap.

```English
+-------+-------+--------+-------+--------------+
| Sky   | Local | Global | Match | Global+Match |
+=======+=======+========+=======+==============+
| 100   |  100  |  100   |    0  |        100   |
+-------+-------+--------+-------+--------------+
| 120   |  120  |  100   |   20  |        120   |
+-------+-------+--------+-------+--------------+
| 105   |  105  |  100   |    5  |        105   |
+-------+-------+--------+-------+--------------+
| 110   |  110  |  100   |   10  |        110   |
+-------+-------+--------+-------+--------------+
| 105   |  105  |  100   |    5  |        105   |
+-------+-------+--------+-------+--------------+
| 115   |  115  |  100   |   15  |        115   |
+-------+-------+--------+-------+--------------+

+-------+-------+--------+-------+--------------+
| Sky   | Local | Global | Match | Global+Match |
+=======+=======+========+=======+==============+
| 100   |  100  |   90   |     0 |    86.25     |
+-------+-------+--------+-------+--------------+
| 120   |  120  |   90   |    20 |   106.25     |
+-------+-------+--------+-------+--------------+
| 105   |  105  |   90   |     5 |    91.25     |
+-------+-------+--------+-------+--------------+
| 110   |  110  |   90   |    10 |    96.25     |
+-------+-------+--------+-------+--------------+
|  95   |   95  |   90   |  8.75 |     95       |
+-------+-------+--------+-------+--------------+
|  90   |   90  |   90   |  3.75 |     90       |
+-------+-------+--------+-------+--------------+
| 100   |  100  |   90   | 13.75 |    100       |
+-------+-------+--------+-------+--------------+
```

--------------------------------

### RomanCAL DQ Flag Specification Examples

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/references_general/references_general.rst

Demonstrates various ways to specify acceptable data quality (DQ) flags for computations in RomanCAL. This includes using integer sums, comma-separated lists, '+' separated lists, and Roman mnemonics, along with examples of reversing flag meanings.

```APIDOC
Integer Sum:
  "6"  (equivalent to 2 + 4)

Comma-separated List:
  "4,8"

Plus-separated List:
  "4+8"

Mnemonic Representation:
  "JUMP_DET, DROPOUT"

Equivalent Specifications:
  "12" == "4+8" == "4, 8" == "JUMP_DET, DROPOUT"

Default Value:
  0 (all non-zero DQ pixels considered 'bad')

Disable DQ Usage:
  None

Reverse Flag Meaning (exclude specific flags):
  "~4+8" or "~4,8" (exclude pixels with DQ flags 4 and 8)
  "~0" (equivalent to None, considers all pixels 'good')

```

--------------------------------

### PEP8 Checking Update

Source: https://github.com/spacetelescope/romancal/blob/main/CHANGES.rst

The setup has been updated for more strict PEP8 checking.

```Python
# Update setup for more strict PEP8 checking [#176]
```

--------------------------------

### Push Release Branch Updates

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Pushes the updated release branch to the upstream remote.

```shell
git push upstream HEAD
```

--------------------------------

### DeepDiff Development Version

Source: https://github.com/spacetelescope/romancal/blob/main/requirements-dev-thirdparty.txt

Installs the development version of the DeepDiff library from its GitHub repository.

```git
git+https://github.com/seperman/deepdiff.git@dev
```

--------------------------------

### Push Release Metadata

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Pushes the metadata updates to a specific branch on the user's fork.

```shell
git push <UserName> release_metadata
```

--------------------------------

### Add user documentation for roman_static_preview script

Source: https://github.com/spacetelescope/romancal/blob/main/CHANGES.rst

Provides user-facing documentation for the 'roman_static_preview' script.

```Python
# added user documentation for ``roman_static_preview`` script [#1046]
```

--------------------------------

### Push New Release Branch

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Pushes the newly created release branch to the upstream remote.

```shell
git push -u upstream HEAD
```

--------------------------------

### Documentation for PHOTOM and Area Reference Files

Source: https://github.com/spacetelescope/romancal/blob/main/CHANGES.rst

Documentation for PHOTOM and Area reference files has been added, including placeholder documentation for the photom step. An improper object in dark documentation was also fixed.

```Python
# Added documentation for PHOTOM and Area reference files, which required placeholder documentation for the photom step. In addition, I fixed an improper object in dark documentation. [#452]
```

--------------------------------

### Pull All Changes into Release Branch

Source: https://github.com/spacetelescope/romancal/wiki/Release-Procedures

Fetches all changes from the upstream 'master' branch and merges them into the current release branch.

```shell
git fetch upstream
git pull upstream master
```

--------------------------------

### Configspec Type Declarations

Source: https://github.com/spacetelescope/romancal/blob/main/docs/roman/stpipe/devel_step.rst

Demonstrates how to declare parameter types and their constraints in the configspec format. Includes examples for integers, floats, and strings with optional bounds and default values.

```Python
n_iterations = integer(1, 100)  # The number of iterations to run
factor = float()                # A multiplication factor
author = string()               # The author of the file
name = string(default="John Doe")
```