### Install Development Dependencies for rad Source: https://github.com/spacetelescope/rad/blob/main/README.md Installs additional packages required for development, including testing utilities and documentation build tools. ```Bash pip install -e ".[test]" pip install -e ".[docs]" pip install -e ".[test,docs]" ``` -------------------------------- ### Install Additional Development Tools Source: https://github.com/spacetelescope/rad/blob/main/README.md Installs useful tools for interactive development and parallel testing within the rad project environment. ```Bash pip install ipython pytest-xdist ``` -------------------------------- ### Install Development Version from GitHub Source: https://github.com/spacetelescope/rad/blob/main/README.md Installs the latest development version of rad directly from its GitHub repository. This version may be less stable but includes the newest features. ```Bash conda create -n python conda activate pip install git+https://github.com/spacetelescope/rad ``` -------------------------------- ### Install Latest rad Release Source: https://github.com/spacetelescope/rad/blob/main/README.md Installs the most recent stable version of the rad package into a new or existing conda environment. ```Bash conda create -n python conda activate pip install rad ``` -------------------------------- ### Install Specific rad Version Source: https://github.com/spacetelescope/rad/blob/main/README.md Installs a particular version of the rad package, useful for maintaining compatibility or testing specific releases. ```Bash conda create -n python conda activate pip install rad==0.5.0 ``` -------------------------------- ### Install rad for Development (Editable Mode) Source: https://github.com/spacetelescope/rad/blob/main/README.md Sets up the rad package in an editable mode, allowing developers to modify the source code and see changes reflected immediately without reinstallation. ```Bash cd git clone https://github.com/spacetelescope/rad cd rad pip install -e . ``` -------------------------------- ### RAD Schema URI Examples Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Examples of schema_uri and tag_uri formats for RAD schemas, illustrating the structure and prefix conventions. ```text schema_uri: asdf://stsci.edu/datamodels/roman/schemas/reference_files/dark-1.0.0 tag_uri: asdf://stsci.edu/datamodels/roman/tags/reference_files/dark-1.0.0 ``` -------------------------------- ### Initial Guide Window Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces the initial version of the Guide Window schema, defining its structure and fields. ```APIDOC Initial Guide Window Schema [#120] ``` -------------------------------- ### Add L1 Guide Window Detector Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces the schema for the L1 Guide Window Detector, enhancing data processing capabilities for guide window information. ```APIDOC Schema Addition: - L1 Guide Window Detector Schema - L1 Average FACE Guide Window File Schema ``` -------------------------------- ### Workflow for Updating a RAD Resource Version Source: https://github.com/spacetelescope/rad/blob/main/docs/versioning.rst Provides a step-by-step guide for updating the version of a RAD resource, adhering to Semantic Versioning. It covers determining the new version, updating symlinks, copying files, and modifying version identifiers in the 'latest' directory. ```APIDOC Workflow for Updating a Resource Version: 1. Determine New Version: - Follow Semantic Versioning guidelines (e.g., `a.b.c`). 2. Update Symlink Name: - In `src/rad/resources`, update the symlink name from the current version to the new version. - The symlink's target (the file in `latest`) should **never** be modified at this stage. 3. Copy Resource File: - Copy the file from the `latest` directory to `src/rad/resources`. - Modify the copied file's name to include the new version suffix (e.g., `-a.b.c.yaml`). - This copy should be done **before** modifying the content to preserve the exact previous version's content. 4. Update Version in 'latest': - In the `latest` directory, update the version number for the URI (`id:` keyword) to reflect the new version. ``` -------------------------------- ### Update Guidewindow/Guidestar Variable Types Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adjusts the variable types for x/y start, stop, and size in guidewindow and guidestar schemas for improved data accuracy. ```APIDOC Updated the variable type of x/y start/stop/size in guidewindow and guidestar schemas. [#205] ``` -------------------------------- ### Install Rad Package with Specific Version Source: https://github.com/spacetelescope/rad/wiki/Release-Process This command installs a specific version of the 'rad' package using pip. Replace 'a.b.c' with the desired version number. ```shell pip install rad==a.b.c ``` -------------------------------- ### Update Guide Star ID, Catalog Version, and Science Filename Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Enhances the guide star information by updating the guide star ID, adding the catalog version, and including the science file name. ```APIDOC Update guide star id, add catalog version, and add science file name [#258] ``` -------------------------------- ### Add GuideWindow DB Table to Basic Scalars Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds the 'GuideWindow' database table to the Basic tagged scalars. ```text - Added GuideWindow db table to Basic tagged scalars. [#327] ``` -------------------------------- ### Clarify Database Tables for Guidewindows and Guidestar Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Provides clarification on the database tables used for guidewindows and guidestar, improving database schema understanding. ```APIDOC Clarifying database tables for guidewindows and guidestar." [#250] ``` -------------------------------- ### Add Documentation for Creating Schemas Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Provides documentation on the process of creating new schemas. ```text - Add documentation on how to create a new schema. [#375] ``` -------------------------------- ### Setuptools_scm Workaround Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst A workaround implemented to address issues with setuptools_scm in recent versions of pip. ```APIDOC Workaround for setuptools_scm issues with recent versions of pip. ``` -------------------------------- ### RAD Resource Creation Source: https://github.com/spacetelescope/rad/blob/main/docs/helper.rst Describes the process of creating resources in RAD. The 'Create' button initiates resource creation with specified options, while the 'Return' button cancels the operation. Resource URIs must follow a semantic versioning suffix. ```APIDOC Resource Creation: - 'Create' button (green): Creates the resource with given options. - 'Return' button (red): Closes the current screen without creating a resource. URI Suffix Requirements: - Must end with a '-' suffix (e.g., '1.0.0', '2.3.4'). - Final URI is checked for conflicts with existing resource URIs in RAD. - Input field border turns blue if valid, red otherwise. 'Create' Button Enable Condition: - Enabled only if all options are configured or entered correctly. - Grayed out if one or more options need updating. Resource Types: 1. Untagged Schema: - Requires: Title, Schema URI prefix, URI suffix, Tag switch off. 2. Tagged Schema: - Requires: Title, Description, Schema URI prefix, URI suffix, Tag switch on. 3. Manifest: - Requires: Title, Description, Manifest URI prefix, URI suffix, Tag switch disabled (off). Tagged Schema with Locked Datamodels Manifest: - Clicking 'Create' opens a bump dialog to update the datamodels manifest version before schema creation. ``` -------------------------------- ### Run Rad Helper Script Source: https://github.com/spacetelescope/rad/blob/main/docs/helper.rst This command executes the Rad helper script from the root of the RAD repository. The script automates the creation, editing, and version bumping of RAD schemas and manifests. ```bash python scripts/rad.py ``` -------------------------------- ### Move Gw Time Entries from GuideStar to GuideWindow Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Relocates gw_function_start_time, gw_function_end_time, and gw_acq_exec_stat from GuideStar to GuideWindow schemas and removes duplicates. ```APIDOC Moved gw_function_start_time, gw_function_end_time, and gw_acq_exec_stat from GuideStar to GuideWindow. Removed duplicate gw time entries. [#154] ``` -------------------------------- ### Update Guidewindow Titles and Descriptions Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Refines the titles and descriptions for guidewindow schemas to improve clarity and user understanding. ```APIDOC Update guidwindow titles and descriptions. [#193] ``` -------------------------------- ### RAD Project Helper Script Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces a helper script, scripts/rad.py, for managing RAD resources. Features include listing latest versions, creating new resources, bumping versions, and editing existing resources, with automatic manifest version bumping and cascade handling. ```Python scripts/rad.py Features include: - Listing the latest versions of all the RAD resources. - includes indications of which resources are "frozen" (not changeable without a new version) at the time of script execution. - Creating (and integrating) a new RAD resource. - Includes bumping the manifest version if necessary - Bumping existing RAD resource versions. - Includes solving the cascade of version bumps that maybe required. - Editing existing RAD resources. - Includes bumping the resource (and the cascade of other resources) if the edits necessitate a version bump. ``` -------------------------------- ### Pseudo Inheritance with JSON Schema Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Demonstrates how to achieve pseudo-inheritance in JSON schemas using the 'allOf' combiner and the '$ref' keyword to combine existing schema structures with additional properties. ```YAML allOf: - $ref: - type: object properties: ``` -------------------------------- ### Add pre-commit Support Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Integrates 'pre-commit' hooks to automate code quality checks and formatting before commits, improving development workflow. ```APIDOC Add ``pre-commit`` support. [#199] ``` -------------------------------- ### Schema Reference and URI Handling Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Refactors schema references to use 'allOf' combiners and updates '$ref' URIs to be absolute, improving schema structure and maintainability. ```APIDOC Schema Refactoring: - Moved schema references under 'allOf' combiners when the schema contains other items. - Updated all '$ref' to use absolute URIs rather than relative URIs. ``` -------------------------------- ### Tagging Main/Master for Development Source: https://github.com/spacetelescope/rad/wiki/Release-Process This Git command sequence fetches upstream changes, checks out the master branch, creates a development tag (e.g., '1.2.2.dev'), and pushes the tag to the upstream repository. This is crucial for `setuptools-scm` to correctly identify development versions after a release. ```git git fetch upstream git checkout upstream/master git tag -a a.b.d.dev -m "development tag after divergence" git push upstream a.b.d.dev ``` -------------------------------- ### Create Flux Step Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Creates a new schema for the flux step. ```text - Create the flux step schema. [#395] ``` -------------------------------- ### Initial Schemas for Roman Calibration Pipeline Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst The initial release (0.1.0) included the foundational schemas for the Roman Calibration Pipeline and SDP file generation. ```APIDOC Initial Schemas for Roman Calibration Pipeline and SDP file generation. ``` -------------------------------- ### Move Metadata to pyproject.toml Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Migrates project metadata to 'pyproject.toml' in compliance with PEP621 standards, centralizing project configuration. ```APIDOC move metadata to ``pyproject.toml`` in accordance with PEP621 [#196] ``` -------------------------------- ### Create Docs for RTD Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Generates documentation specifically for Read the Docs (RTD), improving project documentation accessibility. ```APIDOC Create docs for RTD. [#151] ``` -------------------------------- ### Fetch and Checkout Upstream Main Source: https://github.com/spacetelescope/rad/wiki/Release-Process Fetches all branches and tags from the upstream repository and checks out the main branch. ```shell git fetch --all --tags git checkout -t upstream/main ``` -------------------------------- ### Remove Reference File and Guidewindow DB Tables Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Removes reference file and guidewindow database tables from cal_step schemas. ```text - This PR removes reference file and guidewindow db tables from cal_step schemas. [#420] ``` -------------------------------- ### RAD Schema Boilerplate Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Provides the essential YAML structure for a new RAD schema file, including schema identification, title, and description. ```yaml YAML 1.1 --- $schema: asdf://stsci.edu/datamodels/roman/schemas/rad_schema-1.0.0 id: asdf://stsci.edu/datamodels/roman/schemas/ # No .yaml title: description: | <A long description of the schema> ... ``` -------------------------------- ### Cal_step and Meta.pedigree Schema Additions Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates to the `cal_step` schema to include more steps and the addition of an enumeration for `meta.pedigree`. ```APIDOC Schema Additions: - Added more steps to the `cal_step` schema. - Added enumeration for `meta.pedigree`. ``` -------------------------------- ### Implement Changelog CI Workflow Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces a Continuous Integration (CI) workflow for managing the changelog, automating changelog updates and consistency. ```APIDOC Changelog CI workflow has been added. [#240] ``` -------------------------------- ### Add Source Catalog and Segmentation Map Schemas Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds 'source_catalog' and 'segmentation_map' schemas for Level 2 and Level 3 files. ```text - Add source_catalog and segmentation_map schemas for Level 2 and Level 3 files. [#393] ``` -------------------------------- ### Add Streamlined Level 3 Mosaic Metadata Schemas Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces streamlined Level 3 Mosaic metadata schemas. ```text - Added streamlined Level 3 Mosaic metadata schemas. [#334] ``` -------------------------------- ### Add CRDS and Reference Steps Required Lists Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Inclusion of CRDS and reference steps required lists. ```text - Added CRDS and reference steps required lists. (`#506 <https://github.com/spacetelescope/rad/issues/506>`_) ``` -------------------------------- ### Create Annotated Tag Source: https://github.com/spacetelescope/rad/wiki/Release-Process Creates an annotated Git tag for the release, including a message. ```shell git tag -a a.b.c -m "a.b.c on release/a.b.x" ``` -------------------------------- ### Support for New Reference File Types Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst This section covers the addition of support for several new types of reference files, including saturation, super-bias, and pixel area reference files, expanding the calibration capabilities. ```APIDOC New Reference File Support: - Added nonlinearity support. - Added saturation reference file support. - Added support for super-bias reference files. - Added pixel area reference file support. ``` -------------------------------- ### Switch Linting from flake8 to ruff Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Migrates the project's linting tool from 'flake8' to 'ruff' for improved performance and features. ```APIDOC Switch linting from ``flake8`` to ``ruff``. [#201] ``` -------------------------------- ### Support Distortion Reference Files and Tweak WFI Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds support for Distortion reference files and refines the schema for the WFI detector list. ```APIDOC Added support for Distortion reference files. Tweaked schema for WFI detector list. [#122] ``` -------------------------------- ### RampFitOutput and WFIimgphotom Model Updates Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates to the `rampfitoutput` model and `WFIimgphotom` models, including renaming `rampfitoutput` to `ramp_fit_output`. ```APIDOC Model Updates: - Updated `rampfitoutput` model and `WFIimgphotom` models. - Renamed `rampfitoutput` to `ramp_fit_output`. ``` -------------------------------- ### Add Attributes to WfiMosaic.meta Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds attributes under the 'basic' schema to 'WfiMosaic.meta'. ```text - Add attributes under the ``basic`` schema to ``WfiMosaic.meta``. [#390] ``` -------------------------------- ### Create outlier_detection Schema and Add Bit Mask Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Creates the 'outlier_detection' schema and adds a bit mask field to it and the 'resample' schema. ```text - Create ``outlier_detection`` schema and add bit mask field to both it and ``resample``. [#401] ``` -------------------------------- ### Add FGS Modes to Guidestar Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Incorporates Fine Guidance System (FGS) modes into the guidestar schema, providing more detailed guidance information. ```APIDOC Add FGS (Fine Guidance System) modes to guidestar schema. [#103] ``` -------------------------------- ### Add New Schemas to Documentation Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds new schemas to the project's documentation. ```text - Add new schemas to documentation. [#386] ``` -------------------------------- ### Documentation and Schema Enhancements Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Improves project documentation and introduces new schemas for specific data types, aiding in better understanding and utilization of the RAD system. ```APIDOC Documentation Updates: - Added section on versioning and old file support. - Updated RAD documentation to match current schemas and fix broken links. - Used 'towncrier' for managing changelog entries. - Clarified schema documentation headings and added links to roman_datamodels and ASDF. New Features: - Added schema for skycell reference file. - Added ePSF, ABVegaOffset, and ApCorr schemas. - Added 'refpix' entry in 'ref_file'. ``` -------------------------------- ### Reference File Schemas Source: https://github.com/spacetelescope/rad/blob/main/docs/reference_files.rst Lists the schemas for various reference files used in the Nancy Grace Roman Space Telescope. These schemas define the structure and content of the reference data. ```APIDOC abvegaoffset-1.0.0 apcorr-1.0.0 dark-1.0.0 distortion-1.0.0 epsf-1.0.0 flat-1.0.0 gain-1.0.0 inverselinearity-1.0.0 ipc-1.0.0 linearity-1.0.0 mask-1.0.0 matable-1.0.0 pixelarea-1.0.0 readnoise-1.0.0 ref_common-1.0.0 ref_exposure_type-1.0.0 ref_optical_element-1.0.0 refpix-1.0.0 saturation-1.0.0 skycells-1.0.0 superbias-1.0.0 wfi_img_photom-1.0.0 ``` -------------------------------- ### Update cal_step Schema Steps Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates the steps listed in the 'cal_step' schema to reflect currently implemented steps, including 'outlier_detection', 'refpix', 'sky_match', and 'tweak_reg'. ```text - Update the steps listed in ``cal_step`` to reflect the currently implemented steps. The new additions are ``outlier_detection``, ``refpix``, ``sky_match``, and ``tweak_reg``. [#282] ``` -------------------------------- ### Required and Ordering Information Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Specifies required fields and their order within a schema, placed at the same indentation level as 'properties'. ```yaml required: [<required field 1>, <required field 2>, <required field 3>] order: [<field 1>, <field 2>, <field 3>] ``` -------------------------------- ### Add msos_stack-1.0.0 Schema for SSC Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds the 'msos_stack-1.0.0' schema as a level 3 schema for the SSC (Space Science Telescope), expanding the project's schema capabilities. ```APIDOC Add schema ``msos_stack-1.0.0`` as a level 3 schema for SSC. [#276] ``` -------------------------------- ### Duplicate Keywords for Exposure and Guidestar Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Duplicates keywords from 'base_exposure' to 'exposure' and similarly for 'base_guidestar' and 'guidestar'. ```text - Duplicated the keywords from base_exposure to exposure and similarly for base_guidestar and guidestar. [#406] ``` -------------------------------- ### Python Version and Dependency Management Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Ensures compatibility with the latest Python versions and manages project dependencies, including removing upper pins for libraries. ```APIDOC Python Version Support: - Tested with the latest supported version of Python. - Bumped minimum Python version to 3.11 per SPEC 0. - Added Python 3.13 support. Dependency Management: - Removed the upper pin for ASDF. ``` -------------------------------- ### Add photom Keyword Attribute to cal_step Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Includes the 'photom' keyword attribute in the 'cal_step' schema, likely for photometric calibration steps. ```APIDOC Added photom keyword attribute to cal_step schema. [#132] ``` -------------------------------- ### Set all calsteps to Required Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Ensures that all calibration steps ('calsteps') are marked as required in the relevant schemas. ```APIDOC Set all calsteps to required. [#102] ``` -------------------------------- ### Add Keyword for Datamodel Description Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces a keyword to specify if and which datamodel a schema describes, enhancing schema organization and clarity. ```APIDOC Add keyword to indicate if and which datamodel the schema describes. [#278] ``` -------------------------------- ### Create New Release Branch Source: https://github.com/spacetelescope/rad/wiki/Release-Process Creates a new release branch based on the main branch, following the pattern 'release/major.minor.x'. ```shell git checkout -b release/a.b.x ``` -------------------------------- ### Top-Level Schema Definition Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Defines the structure for top-level schemas in RAD, including 'datamodel_name' for mapping to Python DataModel objects and 'archive_meta' as a placeholder for future metadata. ```YAML datamodel_name: <name of the datamodel in Python> archive_meta: None ``` -------------------------------- ### Add FPS and TVAC Schemas Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces new schemas for FPS (Focal Plane System) and TVAC (Thermal Vacuum). ```text - Add ``FPS`` and ``TVAC`` schemas. [#364] ``` -------------------------------- ### Remove Deprecated pytest-openfiles Plugin Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Replaces the deprecated 'pytest-openfiles' plugin with ResourceWarning catching to manage resource warnings more effectively. ```APIDOC Remove use of deprecated ``pytest-openfiles`` ``pytest`` plugin. This has been replaced by catching ``ResourceWarning`` s. [#231] ``` -------------------------------- ### Update Product Table Names Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates the names of product tables. ```text - Updated product table names. [#382] ``` -------------------------------- ### RAD Resource Directory Structure and Naming Conventions Source: https://github.com/spacetelescope/rad/blob/main/docs/versioning.rst Explains the organization and naming conventions for RAD resources (schemas and manifests) within the ASDF extension context. Resources are located in 'src/rad/resources', organized into 'schemas' and 'manifests' subdirectories, with symlinks pointing to the 'latest' directory for the most current versions. ```APIDOC Resource Location: - RAD resources (schemas and manifests) are located in `src/rad/resources`. - This directory is organized into two subdirectories: - `schemas`: Contains ASDF schemas or symlinks to schemas. - `manifests`: Contains ASDF manifests or symlinks to manifests. Latest Version Location: - The *latest* version of any resource is located within the `latest` directory in the top level of the RAD repository. - Files in `src/rad/resources` are symlinks targeting files within `latest`. Naming Conventions: 1. Files in `src/rad/resources`: - Named with a version suffix (e.g., `-a.b.c.yaml`). - The latest version is a symlink targeting a file within `latest`. 2. Files in `latest` directory: - End only with the normal `.yaml` suffix (no version number). - These files are the target of the version indicated by the symlink filename and the `id:` keyword within the file. ``` -------------------------------- ### Change input_unit and output_unit Keyword Types Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates the types, titles, and tests for the 'input_unit' and 'output_unit' keywords to improve unit handling. ```APIDOC Changed input_unit and output_unit keyword types, titles, and tests. [#126] ``` -------------------------------- ### Add refpix-1.0.0 Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds the 'refpix-1.0.0' schema, which serves as a schema for the reference pixel correction's reference file. ```text - Add schema ``refpix-1.0.0`` as a schema for the reference pixel correction's reference file. [#270] ``` -------------------------------- ### Format Code with isort and black Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Applies code formatting using 'isort' and 'black' to ensure consistent code style across the project. ```APIDOC Format the code with ``isort`` and ``black``. [#200] ``` -------------------------------- ### Keyword and Schema Management Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Manages keyword requirements and updates schema definitions for various components, ensuring adherence to data standards and improving schema accuracy. ```APIDOC Schema Keyword Requirements: - Require 'archive_catalog' and 'sdf' marked keywords in the 'required' list for their containing object. - Added maxLength keywords to schemas matching nvarchar archive_catalog datatypes. Schema Updates: - Removed WFICommon db from the guidestar schema. - Updated SDP origin names for various attributes. - Adjusted RTB directed L1 & L2 metadata schema changes. - Removed units from the reference file schemas. - Renamed 'source_detection' to 'source_catalog' to match step name in romancal. - Updated all '$ref' to use absolute URIs. - Removed 'var_flat' from list of required mosaic attributes. - Removed units from Guidewindow schema. - Updated 'cal_step' and 'cal_log' schema information. - Updated 'coordinate' schema descriptions. - Updated 'ephemeris' schema descriptions. - Updated 'guide_star', 'instrument', and 'photometry' schemas. - Added 'rcs' and removed 'aperture', 'target' schemas. - Updated and added keywords and descriptions in basic-1.0.0 and statistics schemas. - Updated the pointing schema. - Updated velocity_aberration descriptions and keywords. - Updated wcs keywords and attribute information. ``` -------------------------------- ### Add Schema for Source Detection Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces a new schema specifically designed for source detection, likely used in astronomical data analysis. ```APIDOC Add schema for source detection. [#215] ``` -------------------------------- ### Tagged Scalar Schema (Non-Primitive Type) Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Illustrates how to tag a scalar of a non-primitive type (like time) using an 'allOf' combiner with a '$ref' to the schema_uri. ```yaml allOf: - $ref: <schema_uri> ``` -------------------------------- ### New Schema Additions (Photometry, Dark, DQ, etc.) Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst This section lists the addition of several new schemas, including photometry, dark, DQ, readnoise, mask, gain, ramp fitting, variance objects, and updates to existing schemas like aperture and exposure. ```APIDOC New Schema Additions: - Added `wfi photom` schema and tests. - Added Dark schema and updated Flat schema. - Added dq schema. - Added readnoise, mask, and gain schemas. - Added support for ramp fitting schemas. - Added support for variance object schemas. - Updated aperture, basic, ephemeris, exposure, guidestar, observation, pixelarea, and visit schemas. ``` -------------------------------- ### RAD Version Bumping Source: https://github.com/spacetelescope/rad/blob/main/docs/helper.rst Details the version bumping process in RAD. The 'Bump' button is enabled for locked resources and opens a dialog to update version numbers. Input validation ensures semantic versioning compliance. ```APIDOC Bump Button Functionality: - Enabled when a locked resource (requiring a version bump) is selected. - Opens a dialog screen for version bumping. Bump Dialog Screen: - 'Bump' button (green): Bumps versions of all needed resources. Enabled when version numbers are correctly filled. - 'Return' button (red): Closes the screen without bumping resources. - Scrollable list of resources needing version bumps, showing current URI and an input field for the new version number. Cascade Bumping: - Resources referencing bumped resources may also need bumping. - Resources are listed in order: selected resource, then resources directly referencing it, and so on. Version Number Validation: 1. Semantic Version String: Must be a valid semantic version (e.g., '1.0.0', '2.3.4'). 2. Strictly Greater: Must be semantically greater than the current version. - Input field border turns blue if valid, red otherwise. ``` -------------------------------- ### Add Sky Background Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduced a new schema for sky background. ```text - Added sky background schema. [#432] ``` -------------------------------- ### Add Restrictions to patternProperties in wfi_img_photom Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Implements additional restrictions on the 'patternProperties' keywords within the 'wfi_img_photom' schema for better data validation. ```APIDOC Add further restrictions to the ``patternProperties`` keywords in the ``wfi_img_photom`` schema. [#254] ``` -------------------------------- ### RAD Resource Editing Source: https://github.com/spacetelescope/rad/blob/main/docs/helper.rst Explains how to edit resources in RAD. The 'Edit' button opens the selected resource file in the configured text editor. Changes are applied after saving and closing the editor. Locked resources may require version bumping. ```APIDOC Edit Button Functionality: - Enabled when any resource is selected in the directory tree. - Opens the selected resource file in the configured text editor ($EDITOR). - Edits a temporary file copy to prevent data loss on premature exit. - Automatically attempts to apply changes after saving and closing the editor. Default Editor: - If $EDITOR is not set, 'vim' is used. Editing Locked Resources: - If changes require a version bump, a bump dialog screen opens before applying changes. - All relevant resources, including the edited one, are bumped. - Changes made to locked resources that are not applied due to exiting the bump dialog are saved to '<rad_repository_directory>/unsaved_changes/'. ``` -------------------------------- ### Dictionary-like Schema Definition Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Defines a dictionary-like type in a schema, allowing nested fields. Each field can have a title and a detailed description. ```yaml type: object properties: <first keyword>: title: <Title of the field> description: | <A long description of the field> ``` -------------------------------- ### Add Statistics Schemas to FPS and TVAC Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds statistics schemas to both FPS and TVAC. ```text - Added statistics schemas to both FPS and TVAC. [#423] ``` -------------------------------- ### Add Distortion Keyword Option to Reference Files Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Includes the 'distortion' keyword as an option for reference files, allowing the 'distortion' reference file to be accepted by the 'ref_file-1.0.0' schema. ```APIDOC Add ``distortion`` keyword option to the list of reference files, so that the ``distortion`` reference file can be properly allowed in by the ``ref_file-1.0.0`` schema. [#237] ``` -------------------------------- ### Add Pull Request Template Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Implements a Pull Request (PR) template to standardize the process of contributing code and ensure necessary information is provided. ```APIDOC Added PR Template. [#221] ``` -------------------------------- ### Push Tag to Upstream Source: https://github.com/spacetelescope/rad/wiki/Release-Process Pushes the newly created Git tag to the upstream remote repository. ```shell git push upstream a.b.c ``` -------------------------------- ### Update Reference File Schemas with Quantities Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Enhances reference file schemas by utilizing quantities for all relevant arrays, improving data representation. ```APIDOC Update Reference file schemas to utilize quantities for all relevant arrays. [#198] ``` -------------------------------- ### Schema Updates and Fixes Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Addresses inconsistencies in schemas and applies workarounds for external library bugs, ensuring data integrity and compatibility. ```APIDOC Bug Fixes: - Corrected inconsistencies in 'msos_step' and 'sky_background' schemas. - Reordered 'anyOf' items in 'apcorr' schema to resolve ASDF bug. ``` -------------------------------- ### Update Exposure and Program Attributes Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates to exposure and program attributes information, including schema descriptions and metadata. ```text - Update exposure and program attributes information (`#478 <https://github.com/spacetelescope/rad/issues/478>`_) ``` -------------------------------- ### Update RTD with New Data Products Documentation Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates the Read the Docs (RTD) with documentation for newly added data products. ```text - Updated RTD with documentation for new data products. [#419] ``` -------------------------------- ### Update Reference File Descriptions Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates to the descriptions of reference files within the rad schema. ```text - Update ref_file descriptions (`#479 <https://github.com/spacetelescope/rad/issues/479>`_) ``` -------------------------------- ### Move Detector List to New File Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Relocates the detector list to a separate file for easier importing into both data and reference schemas. ```APIDOC Moved detector list to new file for importing to both data and reference schemas. [#119] ``` -------------------------------- ### Add Missing Unit Enforcements Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds missing unit enforcements to various schemas. ```text - Add missing unit enforcements to various schemas. [#300] ``` -------------------------------- ### SDF External Metadata Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Describes the 'sdf' external metadata fields used for data processed by SDF software prior to the Romancal pipeline. Includes 'special_processing' and 'source'. ```YAML special_processing: <string describing special processing> source: <string describing data source> ``` -------------------------------- ### Convert Tag Keywords to Wildcards Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Converts tag keywords to wildcards for external tags. ```text - Convert tag keywords to wildcards for external tags. [#370] ``` -------------------------------- ### Update Exposure Schema with Descriptions Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Enhances the exposure schema by including descriptions for its fields, improving clarity. ```APIDOC exposure schema update in include descriptions [#139] ``` -------------------------------- ### Pin asdf Version Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Pins the 'asdf' package to a version less than 4.0. ```text - pin `asdf<4.0` (`#509 <https://github.com/spacetelescope/rad/issues/509>`_) ``` -------------------------------- ### Schema Updates for Reference Files and Data Products Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst This section details schema modifications for various Roman Calibration Pipeline files, including ramp, level 1, and level 2 data. It covers updates to reference pixel representation, addition of uncertainty attributes, and changes to required properties in common schemas. ```APIDOC Schema Updates: - Exposure Group: - Added `p_exptype` for reference files (dark & readnoise) to enable automatic rmap generation. - Added `level0_compressed` boolean attribute keyword to indicate if level 0 data was compressed. - Ramp, Level 1, and Level 2 Files: - Updated schemas for accurate representation of reference pixels. - Level 1 file: Contains arrays for science and border reference pixels, and a separate array for amp33 reference pixels. - Ramp models: Contain arrays for science data and border reference pixels, and separate arrays for original border reference pixels (copied during dq_init) and their DQ. - Level 2 file: Data array contains only science pixels (border pixels trimmed during ramp fit), with separate arrays for original border pixels, their DQ arrays, and amp33 reference pixels. - Photometry Reference File Schema: - Added `uncertainty` attributes to `photometry` and `pixelareasr`. - Common Schema: - Removed `Photometry` from required properties. ``` -------------------------------- ### Delete source_type_apt from target-1.0.0.yaml Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Removes the 'source_type_apt' field from 'target-1.0.0.yaml', simplifying the target schema. ```APIDOC Deleted source_type_apt from target-1.0.0.yaml [#206] ``` -------------------------------- ### Update Required Elements for Release Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates the list of required elements for the release. ```text - Update required elements for release. [#337] ``` -------------------------------- ### Add Support for Level 3 Mosaic Model Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces support for the level 3 mosaic model, expanding the project's data modeling capabilities. ```APIDOC Added support for level 3 mosaic model. [#241] ``` -------------------------------- ### Move Archive Info in Basic Schema to Tagged Object Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Restructures the 'basic' schema by moving archive-related information into a tagged object for easier retrieval by ASDF. ```APIDOC Moved archive related information in the ``basic`` schema directly into a tagged object for easier retrieval by ASDF. [#153, #158, #159] ``` -------------------------------- ### Update cal_step Schema with resample Step Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates the 'cal_step' schema to include the 'resample' step. ```text - Update the steps listed in ``cal_step`` with the ``resample`` step. [#295] ``` -------------------------------- ### Dark and Readnoise Schema Enhancements Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates to the dark and readnoise schemas include the addition of group keywords from exposure and a new mode keyword, enhancing the metadata associated with these reference files. ```APIDOC Dark and Readnoise Schema Updates: - Dark Schema: - Updated to include group keywords from exposure. - Added `ma_table_name` attribute. - Readnoise Schema: - Added `mode` keyword. - Dark Schema (additional): - Added `mode` keyword. ``` -------------------------------- ### Require New cal_step Steps Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Enforces that all new 'cal_step' steps must be present in the 'cal_step' schema. ```text - Require all the new ``cal_step`` steps to be present in the ``cal_step`` schema. [#301] ``` -------------------------------- ### Update ePSF Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates to the ePSF schema. ```text - Updated the ePSF schema. (`#507 <https://github.com/spacetelescope/rad/issues/507>`_) ``` -------------------------------- ### Add IPC Reference Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces a new reference schema for IPC (Inter-Process Communication), likely for managing communication between different processes. ```APIDOC Add IPC reference schema. [#203] ``` -------------------------------- ### Add Members Keyword to Resample Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Adds the 'Members' keyword to the 'Resample' schema. ```text - Add Members Keyword to Resample Schema. [#396] ``` -------------------------------- ### Update Titles and Descriptions Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates titles and descriptions to align with specifications provided by INS (Instrument Science). ```text - Update titles and descriptions to those provided by INS. [#361] ``` -------------------------------- ### FPS Ground Test Schemas Source: https://github.com/spacetelescope/rad/blob/main/docs/ground_tests.rst Schemas related to the FPS (Focal Plane System) ground tests. Includes basic, calibration, common, exposure, exposure type, ground test, guidestar, and reference file schemas. ```APIDOC fps-1.0.0 fps/basic-1.0.0 fps/cal_step-1.0.0 fps/common-1.0.0 fps/exposure-1.0.0 fps/exposure_type-1.0.0 fps/groundtest-1.0.0 fps/guidestar-1.0.0 fps/ref_file-1.0.0 ``` -------------------------------- ### Push Release Branch to Upstream Source: https://github.com/spacetelescope/rad/wiki/Release-Process Pushes the newly created release branch to the upstream remote repository. ```shell git push -u upstream HEAD ``` -------------------------------- ### Add Alternate WFI Aperture Names Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Added alternate WFI aperture names to align with both SIAF and legacy naming conventions. ```text - Added alternate WFI aperture names to match both the SIAF and legacy names. (`#498 <https://github.com/spacetelescope/rad/issues/498>`_) ``` -------------------------------- ### Fix URIs for inverselinearity and Add Consistency Checks Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Fixes the URIs for 'inverselinearity' and adds consistency checks for names and URIs. ```text - Fix the URIs for ``inverselinearity`` and add consistency checks for names/uris. [#296] ``` -------------------------------- ### Create Inverse Non-Linearity Schema Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Introduces a new schema for inverse non-linearity, likely for calibration or detector characterization. ```APIDOC Created inverse non-linearity schema. [#213] ``` -------------------------------- ### Exposure Time and Guidestar Variable Updates Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Changes related to exposure time data types and units for guidestar variables, including reverting exposure time types and adding units where appropriate. ```APIDOC Exposure Time and Guidestar Variable Updates: - Reverted exposure time types from string back to astropy Time. - Changed certain exposure time types to string. - Added units to guidestar variables where appropriate. - Removed references to RGS in guidestar. - Added examples of observation numbers. - Changed `gs` to `gw` in guidestar to reflect that they are all windows. ``` -------------------------------- ### Duplicate Keywords for TVAC Groundtest Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Duplicates keywords from 'groundtest' to 'tvac_groundtest'. ```text - Duplicated the keywords from groundtest to tvac_groundtest. [#409] ``` -------------------------------- ### Update Changelog Source: https://github.com/spacetelescope/rad/wiki/Release-Process Modifies the CHANGES.rst file to reflect the new release version and date, and prepares for the next unreleased version. ```diff +0.16.1 (unreleased) =================== -0.16.0 (unreleased) +0.16.0 (2020-05-04) =================== ``` -------------------------------- ### Tagged Scalar Schema (Primitive Type) Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Shows the structure for a RAD schema that tags a scalar of a primitive type, specifying the type directly. ```yaml type: <primitive type> ``` -------------------------------- ### Cherry-pick Commits Source: https://github.com/spacetelescope/rad/wiki/Release-Process Applies specific commits from another branch (e.g., main) to the current release branch. ```shell git cherry-pick ... ``` -------------------------------- ### Add Database Team to Code Owners Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Assigns the database team ownership for specific parts of the codebase by adding them to the CODEOWNERS file. ```APIDOC Add database team to Code Owners file [#227] ``` -------------------------------- ### Update Minimum Python Version Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Updates the minimum required Python version to 3.9. ```text - Updated minimum python version to 3.9. [#325] ``` -------------------------------- ### Split cal_step into L2 and L3 Versions Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Splits the 'cal_step' into Level 2 (L2) and Level 3 (L3) versions. ```text - Split cal_step into L2 and L3 versions. [#397] ``` -------------------------------- ### Fix Schemas with $ref at Root Level Source: https://github.com/spacetelescope/rad/blob/main/CHANGES.rst Resolves an issue where schemas with '$ref' at the root level were not functioning correctly. ```APIDOC Fix schemas with $ref at root level. [#222] ``` -------------------------------- ### Multiple Possibilities Schema Definition Source: https://github.com/spacetelescope/rad/blob/main/docs/creating.rst Defines a field that can accept multiple different types using the 'oneOf' combiner. The 'null' type can be included for optional fields. ```yaml oneOf: - type: <type1> - type: <type2> - type: <type3> ```