### Create and inspect a target_validations_collection Source: https://hubverse-org.github.io/hubValidations/reference/new_target_validations_collection.html Complete example showing how to create a `target_validations_collection` from two `target_validations` objects, print it, get element names, access a specific file's validations, and access a specific check. The output shows that empty collections print as `Empty `, and that checks with hive-partitioned paths are skipped with an info message. ```r new_target_validations_collection() #> Empty # Create validations for two different files file_path_1 <- "time-series.csv" validations_1 <- new_target_validations( target_file_name = check_target_file_name(file_path_1), target_file_ext_valid = check_target_file_ext_valid(file_path_1) ) file_path_2 <- "other-data.csv" validations_2 <- new_target_validations( target_file_name = check_target_file_name(file_path_2), target_file_ext_valid = check_target_file_ext_valid(file_path_2) ) # Combine into a collection collection <- new_target_validations_collection(validations_1, validations_2) # Print the collection collection #> #> ── time-series.csv ──── #> #> ℹ [target_file_name]: Target file path not hive-partitioned. Check skipped. #> ✔ [target_file_ext_valid]: Target data file extension is valid. #> #> ── other-data.csv ──── #> #> ℹ [target_file_name]: Target file path not hive-partitioned. Check skipped. #> ✔ [target_file_ext_valid]: Target data file extension is valid. # Get file paths (element names) names(collection) #> [1] "time-series.csv" "other-data.csv" # Access validations for a specific file collection[[file_path_1]] #> #> ── time-series.csv ──── #> #> ℹ [target_file_name]: Target file path not hive-partitioned. Check skipped. #> ✔ [target_file_ext_valid]: Target data file extension is valid. # Access a specific check within a file's validations collection[["time-series.csv"]]$target_file_name #> #> Message: #> Target file path not hive-partitioned. Check skipped. ``` -------------------------------- ### Install Air CLI on Windows (PowerShell) Source: https://hubverse-org.github.io/hubValidations/CONTRIBUTING.html Installs the Air CLI on Windows using PowerShell. ```powershell powershell -ExecutionPolicy Bypass -c "irm https://github.com/posit-dev/air/releases/latest/download/air-installer.ps1 | iex" ``` -------------------------------- ### Example usage with withr::with_tempdir() Source: https://hubverse-org.github.io/hubValidations/reference/create_custom_check.html Example usage of create_custom_check() within a temporary directory, as shown in the documentation examples. The code is incomplete in the source; only the opening line is displayed. ```r withr::with_tempdir({ ``` -------------------------------- ### Install Air CLI on macOS/Linux Source: https://hubverse-org.github.io/hubValidations/CONTRIBUTING.html Installs the Air CLI on macOS or Linux using the official installer script. ```bash curl -LsSf https://github.com/posit-dev/air/releases/latest/download/air-installer.sh | sh ``` -------------------------------- ### Install Air CLI on macOS (Homebrew) Source: https://hubverse-org.github.io/hubValidations/CONTRIBUTING.html Installs the Air CLI on macOS using Homebrew. ```bash brew install air ``` -------------------------------- ### Install arrow from Apache R Universe Source: https://hubverse-org.github.io/hubValidations/index.html Alternative installation for the arrow package from the Apache R Universe repository. Use this if you encounter arrow installation problems, as hubValidations depends on arrow. ```r install.packages("arrow", repos = c("https://apache.r-universe.dev", "https://cran.r-project.org")) ``` -------------------------------- ### Example: opt_check_tbl_counts_lt_popn usage Source: https://hubverse-org.github.io/hubValidations/reference/opt_check_tbl_counts_lt_popn.html Example of using `opt_check_tbl_counts_lt_popn` with a test hub. It reads a model output file, filters to a specific target, and runs the check, expecting a success message. ```r hub_path <- system.file("testhubs/flusight", package = "hubValidations") file_path <- "hub-ensemble/2023-05-08-hub-ensemble.parquet" tbl <- hubValidations::read_model_out_file(file_path, hub_path) # Single target key list targets <- list("target" = "wk ahead inc flu hosp") opt_check_tbl_counts_lt_popn(tbl, file_path, hub_path, targets = targets) #> #> Message: #> Target counts are less than location population size. ``` -------------------------------- ### Example: validate_model_metadata with output Source: https://hubverse-org.github.io/hubValidations/reference/validate_model_metadata.html Example call to validate_model_metadata on a test hub, showing the validation output including checks for schema, file existence, extension, location, and schema matching. ```r hub_path <- system.file("testhubs/simple", package = "hubValidations") validate_model_metadata(hub_path, file_path = "hub-baseline.yml" ) #> #> ── hub-baseline.yml ──── #> #> ✔ [metadata_schema_exists]: Model metadata schema file exists at path #> hub-config/model-metadata-schema.json. #> ✔ [metadata_file_exists]: File exists at path #> model-metadata/team1-goodmodel.yaml. #> ✔ [metadata_file_ext]: Metadata file extension is "yml" or "yaml". #> ✔ [metadata_file_location]: Metadata file directory name matches #> "model-metadata". #> ✔ [metadata_matches_schema]: Metadata file contents are consistent with schema #> specifications. #> ✔ [metadata_file_name]: Metadata file name matches the `model_id` specified #> within the metadata file. validate_model_metadata(hub_path, file_path = "team1-goodmodel.yaml" ) #> #> ── team1-goodmodel.yaml ──── #> #> ✔ [metadata_schema_exists]: Model metadata schema file exists at path #> hub-config/model-metadata-schema.json. #> ✔ [metadata_file_exists]: File exists at path #> model-metadata/team1-goodmodel.yaml. #> ✔ [metadata_file_ext]: Metadata file extension is "yml" or "yaml". #> ✔ [metadata_file_location]: Metadata file directory name matches #> "model-metadata". #> ⓧ [metadata_matches_schema]: Metadata file contents must be consistent with #> schema specifications. - must have required property 'model_details' . - #> must NOT have additional properties; saw unexpected property #> 'models_details'. - must NOT have additional properties; saw unexpected #> property 'ensemble_of_hub_models"'. - /include_ensemble must be boolean . ``` -------------------------------- ### Example: Create and access a hub_validations_collection Source: https://hubverse-org.github.io/hubValidations/reference/new_hub_validations_collection.html Example from the documentation. It creates an empty collection, then builds two hub_validations objects for different files, combines them into a collection, prints it, extracts element names, and accesses validations by file path and by check. The output shows a success and a failure check, and demonstrates that collection elements are named by their 'where' attribute. ```r new_hub_validations_collection() #> Empty hub_path <- system.file("testhubs/simple", package = "hubValidations") # Create validations for two different files file_path_1 <- "team1-goodmodel/2022-10-08-team1-goodmodel.csv" validations_1 <- new_hub_validations( file_exists = check_file_exists(file_path_1, hub_path), file_name = check_file_name(file_path_1) ) file_path_2 <- "team1-goodmodel/2022-10-15-team1-goodmodel.csv" validations_2 <- new_hub_validations( file_exists = check_file_exists(file_path_2, hub_path), file_name = check_file_name(file_path_2) ) # Combine into a collection collection <- new_hub_validations_collection(validations_1, validations_2) # Print the collection collection #> #> ── team1-goodmodel/2022-10-08-team1-goodmodel.csv ──── #> #> ✔ [file_exists]: File exists at path #> model-output/team1-goodmodel/2022-10-08-team1-goodmodel.csv. #> ✔ [file_name]: File name "2022-10-08-team1-goodmodel.csv" is valid. #> #> ── team1-goodmodel/2022-10-15-team1-goodmodel.csv ──── #> #> ⓧ [file_exists]: File does not exist at path #> model-output/team1-goodmodel/2022-10-15-team1-goodmodel.csv. #> ✔ [file_name]: File name "2022-10-15-team1-goodmodel.csv" is valid. # Get file paths (element names) names(collection) #> [1] "team1-goodmodel/2022-10-08-team1-goodmodel.csv" #> [2] "team1-goodmodel/2022-10-15-team1-goodmodel.csv" # Access validations for a specific file collection[[file_path_1]] #> #> ── team1-goodmodel/2022-10-08-team1-goodmodel.csv ──── #> #> ✔ [file_exists]: File exists at path #> model-output/team1-goodmodel/2022-10-08-team1-goodmodel.csv. #> ✔ [file_name]: File name "2022-10-08-team1-goodmodel.csv" is valid. # Access validations for a specific file and check collection$`team1-goodmodel/2022-10-08-team1-goodmodel.csv`$file_exists #> #> Message: #> File exists at path #> model-output/team1-goodmodel/2022-10-08-team1-goodmodel.csv. ``` -------------------------------- ### Example usage of get_config_derived_task_ids Source: https://hubverse-org.github.io/hubValidations/reference/get_config_derived_task_ids.html Example usage of get_config_derived_task_ids with a hub path and optional round_id, showing the returned character vector. ```r hub_path <- system.file("testhubs/v4/flusight", package = "hubUtils") config_tasks <- read_config(hub_path) get_config_derived_task_ids(config_tasks) #> [1] "target_date" get_config_derived_task_ids(config_tasks, round_id = "2023-05-08") #> [1] "target_date" ``` -------------------------------- ### validate_model_file() example with missing file Source: https://hubverse-org.github.io/hubValidations/reference/validate_model_file.html Example validating a model output file that does not exist. The output shows a check_error for file_exists and the function returns early. ```r validate_model_file(hub_path, file_path = "team1-goodmodel/2022-10-15-team1-goodmodel.csv" ) #> #> ── team1-goodmodel/2022-10-15-team1-goodmodel.csv ──── #> #> ⓧ [file_exists]: File does not exist at path #> model-output/team1-goodmodel/2022-10-15-team1-goodmodel.csv. ``` -------------------------------- ### Example: get_tbl_compound_taskid_set with sample hub Source: https://hubverse-org.github.io/hubValidations/reference/get_tbl_compound_taskid_set.html Example usage of get_tbl_compound_taskid_set with a sample hub. It reads a model output file as character, loads the tasks config, and shows the default compact output (only modeling task 2) and the non-compact output (including NULL for task 1). ```r hub_path <- system.file("testhubs/samples", package = "hubValidations") file_path <- "flu-base/2022-10-22-flu-base.csv" round_id <- "2022-10-22" tbl <- read_model_out_file( file_path = file_path, hub_path = hub_path, coerce_types = "chr" ) config_tasks <- read_config(hub_path, "tasks") get_tbl_compound_taskid_set(tbl, config_tasks, round_id) #> $`2` #> [1] "reference_date" "location" get_tbl_compound_taskid_set(tbl, config_tasks, round_id, compact = FALSE ) #> $`1` #> NULL #> #> $`2` #> [1] "reference_date" "location" ``` -------------------------------- ### expand_model_out_grid with sample output type and compound task ID structure Source: https://hubverse-org.github.io/hubValidations/reference/expand_model_out_grid.html These examples demonstrate how to use expand_model_out_grid() with a config file that has a sample output type and a compound task ID structure. The first example uses the default compound task ID set, the second overrides it with a coarser set for the first modeling task, and the third sets both compound task ID sets to NULL. All examples use round_id = "2022-12-26" and include_sample_ids = TRUE, and each returns a tibble with 42 rows. ```r config_tasks <- read_config_file( system.file("config", "tasks-comp-tid.json", package = "hubValidations") ) expand_model_out_grid(config_tasks, round_id = "2022-12-26", include_sample_ids = TRUE ) #> # A tibble: 42 × 6 #> forecast_date target horizon location output_type output_type_id #> #> 1 2022-12-26 wk ahead inc flu h… 2 US mean NA #> 2 2022-12-26 wk ahead inc flu h… 1 US mean NA #> 3 2022-12-26 wk ahead inc flu h… 2 01 mean NA #> 4 2022-12-26 wk ahead inc flu h… 1 01 mean NA #> 5 2022-12-26 wk ahead inc flu h… 2 02 mean NA #> 6 2022-12-26 wk ahead inc flu h… 1 02 mean NA #> 7 2022-12-26 wk ahead inc flu h… 2 US sample 1 #> 8 2022-12-26 wk ahead inc flu h… 2 01 sample 1 #> 9 2022-12-26 wk ahead inc flu h… 2 02 sample 1 #> 10 2022-12-26 wk ahead inc flu h… 1 US sample 2 #> # ℹ 32 more rows ``` ```r # Override config compound task ID set # Create coarser compound task ID set for the first modeling task which contains # samples expand_model_out_grid(config_tasks, round_id = "2022-12-26", include_sample_ids = TRUE, compound_taskid_set = list( c("forecast_date", "target"), NULL ) ) #> # A tibble: 42 × 6 #> forecast_date target horizon location output_type output_type_id #> #> 1 2022-12-26 wk ahead inc flu h… 2 US mean NA #> 2 2022-12-26 wk ahead inc flu h… 1 US mean NA #> 3 2022-12-26 wk ahead inc flu h… 2 01 mean NA #> 4 2022-12-26 wk ahead inc flu h… 1 01 mean NA #> 5 2022-12-26 wk ahead inc flu h… 2 02 mean NA #> 6 2022-12-26 wk ahead inc flu h… 1 02 mean NA #> 7 2022-12-26 wk ahead inc flu h… 2 US sample 1 #> 8 2022-12-26 wk ahead inc flu h… 1 US sample 1 #> 9 2022-12-26 wk ahead inc flu h… 2 01 sample 1 #> 10 2022-12-26 wk ahead inc flu h… 1 01 sample 1 #> # ℹ 32 more rows ``` ```r expand_model_out_grid(config_tasks, round_id = "2022-12-26", include_sample_ids = TRUE, compound_taskid_set = list( NULL, NULL ) ) #> # A tibble: 42 × 6 #> forecast_date target horizon location output_type output_type_id #> #> 1 2022-12-26 wk ahead inc flu h… 2 US mean NA #> 2 2022-12-26 wk ahead inc flu h… 1 US mean NA #> 3 2022-12-26 wk ahead inc flu h… 2 01 mean NA #> 4 2022-12-26 wk ahead inc flu h… 1 01 mean NA #> 5 2022-12-26 wk ahead inc flu h… 2 02 mean NA #> 6 2022-12-26 wk ahead inc flu h… 1 02 mean NA #> 7 2022-12-26 wk ahead inc flu h… 2 US sample 1 #> 8 2022-12-26 wk ahead inc flu h… 1 US sample 2 #> 9 2022-12-26 wk ahead inc flu h… 2 01 sample 3 #> 10 2022-12-26 wk ahead inc flu h… 1 01 sample 4 #> # ℹ 32 more rows ``` -------------------------------- ### Install development version from GitHub Source: https://hubverse-org.github.io/hubValidations/index.html Installs the development version of hubValidations from GitHub. Use this to test unreleased features. ```r remotes::install_github("hubverse-org/hubValidations") ``` -------------------------------- ### validate_model_file() example with successful validation Source: https://hubverse-org.github.io/hubValidations/reference/validate_model_file.html Example validating a model output file that exists and passes all checks. The output shows check results for file_exists, file_name, file_location, round_id_valid, file_format, file_n, and metadata_exists. ```r hub_path <- system.file("testhubs/simple", package = "hubValidations") validate_model_file(hub_path, file_path = "team1-goodmodel/2022-10-08-team1-goodmodel.csv" ) #> #> ── team1-goodmodel/2022-10-08-team1-goodmodel.csv ──── #> #> ✔ [file_exists]: File exists at path #> model-output/team1-goodmodel/2022-10-08-team1-goodmodel.csv. #> ✔ [file_name]: File name "2022-10-08-team1-goodmodel.csv" is valid. #> ✔ [file_location]: File directory name matches `model_id` metadata in file #> name. #> ✔ [round_id_valid]: `round_id` is valid. #> ✔ [file_format]: File is accepted hub format. #> ✔ [file_n]: Number of accepted model output files per round met. #> ✔ [metadata_exists]: Metadata file exists at path #> model-metadata/team1-goodmodel.yaml. ``` -------------------------------- ### Install local package in GitHub Action Source: https://hubverse-org.github.io/hubValidations/articles/writing-custom-fns.html Add this step to the validate-submission.yaml workflow to install the local 'validations' package and its dependencies. This ensures the package is available during validation. ```yaml - uses: r-lib/actions/setup-r-dependencies@v2 with: packages: | any::hubValidations any::sessioninfo local::./src/validations ``` -------------------------------- ### Expand model output grid with default settings Source: https://hubverse-org.github.io/hubValidations/reference/expand_model_out_grid.html Connects to the example hub 'flusight' and expands the model output grid for round '2023-01-02' using the default settings, returning a tibble with 3,132 rows. ```r hub_con <- hubData::connect_hub( system.file("testhubs/flusight", package = "hubUtils") ) config_tasks <- attr(hub_con, "config_tasks") expand_model_out_grid(config_tasks, round_id = "2023-01-02") #> # A tibble: 3,132 × 6 #> forecast_date target horizon location output_type output_type_id #> #> 1 2023-01-02 wk flu hosp rate c… 2 US pmf large_decrease #> 2 2023-01-02 wk flu hosp rate c… 1 US pmf large_decrease #> 3 2023-01-02 wk flu hosp rate c… 2 01 pmf large_decrease #> 4 2023-01-02 wk flu hosp rate c… 1 01 pmf large_decrease #> 5 2023-01-02 wk flu hosp rate c… 2 02 pmf large_decrease #> 6 2023-01-02 wk flu hosp rate c… 1 02 pmf large_decrease #> 7 2023-01-02 wk flu hosp rate c… 2 04 pmf large_decrease #> 8 2023-01-02 wk flu hosp rate c… 1 04 pmf large_decrease #> 9 2023-01-02 wk flu hosp rate c… 2 05 pmf large_decrease #> 10 2023-01-02 wk flu hosp rate c… 1 05 pmf large_decrease #> # ℹ 3,122 more rows ``` -------------------------------- ### Example of validate_submission_time with output Source: https://hubverse-org.github.io/hubValidations/reference/validate_submission_time.html Example usage of validate_submission_time with a test hub and file path. The output shows a failed check because the current time is outside the accepted submission window. ```r hub_path <- system.file("testhubs/simple", package = "hubValidations") file_path <- "team1-goodmodel/2022-10-08-team1-goodmodel.csv" validate_submission_time(hub_path, file_path) #> #> ── team1-goodmodel/2022-10-08-team1-goodmodel.csv ──── #> #> ✖ [submission_time]: Submission time must be within accepted submission window #> for round. Current time "2026-07-15 15:03:59 UTC" is outside window #> 2022-10-02 EDT--2022-10-09 23:59:59 EDT. ``` -------------------------------- ### capture_check_cnd examples: success, failure, and error Source: https://hubverse-org.github.io/hubValidations/reference/capture_check_cnd.html Three examples demonstrating success, conditional failure, and error outputs. The first returns a ``; the second a ``; the third a `` when `error = TRUE`. Messages use `cli` formatting such as `{.var round_id}`. ```r capture_check_cnd( check = TRUE, file_path = "test/file.csv", msg_subject = "{.var round_id}", msg_attribute = "valid.", error = FALSE ) #> #> Message: #> `round_id` is valid. capture_check_cnd( check = FALSE, file_path = "test/file.csv", msg_subject = "{.var round_id}", msg_attribute = "valid.", error = FALSE, details = "Must be one of 'A' or 'B', not 'C'" ) #> #> Error: #> ! `round_id` must be valid. Must be one of 'A' or 'B', not 'C' capture_check_cnd( check = FALSE, file_path = "test/file.csv", msg_subject = "{.var round_id}", msg_attribute = "valid.", error = TRUE, details = "Must be one of {.val {c('A', 'B')}}, not {.val C}" ) #> #> Error: #> ! `round_id` must be valid. Must be one of "A" and "B", not "C" ``` -------------------------------- ### Create custom check with extra_args = TRUE (create_custom_check) Source: https://hubverse-org.github.io/hubValidations/articles/writing-custom-fns.html Creates a custom check function template with an extra example argument by setting extra_args = TRUE. The output shows the file created and instructions to edit the template. ```R create_custom_check("cstm_check_tbl_args", hub_path = hub_path, extra_args = TRUE ) #> ✔ Custom validation check template function file "cstm_check_tbl_args.R" created. #> → Edit the function template to add your custom check logic. #> ℹ See the Writing custom check functions article for more information. #> () ``` -------------------------------- ### parse_file_name() examples with output Source: https://hubverse-org.github.io/hubValidations/reference/parse_file_name.html Two example calls to parse_file_name() with their console output. The first parses a CSV file name and returns round_id, team_abbr, model_abbr, model_id, and ext. The second parses a gzip-compressed parquet file and additionally returns compression_ext as "gzip". ```r parse_file_name("hub-baseline/2022-10-15-hub-baseline.csv") #> $round_id #> [1] "2022-10-15" #> #> $team_abbr #> [1] "hub" #> #> $model_abbr #> [1] "baseline" #> #> $model_id #> [1] "hub-baseline" #> #> $ext #> [1] "csv" #> parse_file_name("hub-baseline/2022-10-15-hub-baseline.gzip.parquet") #> $round_id #> [1] "2022-10-15" #> #> $team_abbr #> [1] "hub" #> #> $model_abbr #> [1] "baseline" #> #> $model_id #> [1] "hub-baseline" #> #> $ext #> [1] "parquet" #> #> $compression_ext #> [1] "gzip" #> ``` -------------------------------- ### Configuration for custom function with source argument Source: https://hubverse-org.github.io/hubValidations/articles/deploying-custom-functions Example validations.yml configuration for deploying a custom function stored in src/validations/R/ and referenced via the source argument. The function is called cstm_check_tbl_example and is sourced from the specified file. ```yaml default: validate_model_data: custom_check: fn: "cstm_check_tbl_example" source: "src/validations/R/cstm_check_tbl_example.R" ``` -------------------------------- ### Validate submission with validate_submission() and inspect hub_validations_collection Source: https://hubverse-org.github.io/hubValidations/articles/hub-validations-class.html Example of using validate_submission() to create a hub_validations_collection object. Shows the class, names, subsetting to a hub_validations object, and accessing a specific check (file_exists) within a file's validations. ```r v_collection <- validate_submission(hub_path, file_path = "team1-goodmodel/2022-10-08-team1-goodmodel.csv" ) # Class of the returned object class(v_collection) #> [1] "hub_validations_collection" "list" # Names show what was validated names(v_collection) #> [1] "hub-config" #> [2] "team1-goodmodel/2022-10-08-team1-goodmodel.csv" # Subset to access a specific hub_validations object v_collection[["hub-config"]] #> #> ── hub-config ──── #> #> ✔ [valid_config]: All hub config files are valid. # Access a specific check within a file's validations v_collection[["team1-goodmodel/2022-10-08-team1-goodmodel.csv"]]$file_exists #> #> Message: #> File exists at path #> model-output/team1-goodmodel/2022-10-08-team1-goodmodel.csv. ``` -------------------------------- ### Configure custom check function in validation.yml Source: https://hubverse-org.github.io/hubValidations/articles/deploying-custom-functions Use the pkg configuration property in validation.yml to specify the package namespace for custom functions. The example sources the cstm_check_tbl_example function from the installed validations package. ```yaml default: validate_model_data: custom_check: fn: "cstm_check_tbl_example" pkg: "validations" ``` -------------------------------- ### Format entire project with Air Source: https://hubverse-org.github.io/hubValidations/CONTRIBUTING.html Formats the entire project using the Air CLI. ```bash air format . ``` -------------------------------- ### Expand model out grid with required_vals_only and force_output_types Source: https://hubverse-org.github.io/hubValidations/reference/expand_model_out_grid.html Both examples use the same hub_path and config_tasks setup. The first call with required_vals_only = TRUE returns only required output types and output_type_ids, producing a tibble with 23 rows. The second call adds force_output_types = TRUE to also include the 'mean' output type with NA output_type_id, resulting in 24 rows. ```r hub_path <- system.file("testhubs", "v4", "simple", package = "hubUtils") config_tasks <- read_config(hub_path) # Return required output types and output_types_ids only expand_model_out_grid( config_tasks = config_tasks, round_id = "2022-10-22", required_vals_only = TRUE ) #> # A tibble: 23 × 6 #> origin_date target horizon age_group output_type output_type_id #> #> 1 2022-10-22 wk inc flu hosp 1 65+ quantile 0.01 #> 2 2022-10-22 wk inc flu hosp 1 65+ quantile 0.025 #> 3 2022-10-22 wk inc flu hosp 1 65+ quantile 0.05 #> 4 2022-10-22 wk inc flu hosp 1 65+ quantile 0.1 #> 5 2022-10-22 wk inc flu hosp 1 65+ quantile 0.15 #> 6 2022-10-22 wk inc flu hosp 1 65+ quantile 0.2 #> 7 2022-10-22 wk inc flu hosp 1 65+ quantile 0.25 #> 8 2022-10-22 wk inc flu hosp 1 65+ quantile 0.3 #> 9 2022-10-22 wk inc flu hosp 1 65+ quantile 0.35 #> 10 2022-10-22 wk inc flu hosp 1 65+ quantile 0.4 #> # ℹ 13 more rows ``` ```r # Force all output types to be required expand_model_out_grid( config_tasks = config_tasks, round_id = "2022-10-22", required_vals_only = TRUE, force_output_types = TRUE ) #> # A tibble: 24 × 6 #> origin_date target horizon age_group output_type output_type_id #> #> 1 2022-10-22 wk inc flu hosp 1 65+ mean NA #> 2 2022-10-22 wk inc flu hosp 1 65+ quantile 0.01 #> 3 2022-10-22 wk inc flu hosp 1 65+ quantile 0.025 #> 4 2022-10-22 wk inc flu hosp 1 65+ quantile 0.05 #> 5 2022-10-22 wk inc flu hosp 1 65+ quantile 0.1 #> 6 2022-10-22 wk inc flu hosp 1 65+ quantile 0.15 #> 7 2022-10-22 wk inc flu hosp 1 65+ quantile 0.2 #> 8 2022-10-22 wk inc flu hosp 1 65+ quantile 0.25 #> 9 2022-10-22 wk inc flu hosp 1 65+ quantile 0.3 #> 10 2022-10-22 wk inc flu hosp 1 65+ quantile 0.35 #> # ℹ 14 more rows ``` -------------------------------- ### Install hubValidations from R-universe Source: https://hubverse-org.github.io/hubValidations/index.html Installs the latest release of hubValidations from the R-universe repository. Use this when you want the current stable version. ```r install.packages("hubValidations", repos = c("https://hubverse-org.r-universe.dev", "https://cloud.r-project.org")) ``` -------------------------------- ### Add additional dependency to setup-r-dependencies step Source: https://hubverse-org.github.io/hubValidations/articles/deploying-custom-functions Edit the setup-r-dependencies step in the validate-submission.yaml workflow to add additional packages to the packages field list. This ensures the dependency is available during validation on GitHub but not locally for hub administrators or submitting teams. ```yaml - uses: r-lib/actions/setup-r-dependencies@v2 with: packages: | any::hubValidations any::sessioninfo any::additionalPackage ``` -------------------------------- ### Create and convert hub_validations objects Source: https://hubverse-org.github.io/hubValidations/reference/new_hub_validations.html Create a new empty hub_validations object, then create one with two checks (file_exists and file_name) and convert a list of checks to a hub_validations object. The output shows the validation summary for the file path. ```r new_hub_validations() #> Empty hub_path <- system.file("testhubs/simple", package = "hubValidations") file_path <- "team1-goodmodel/2022-10-08-team1-goodmodel.csv" new_hub_validations( file_exists = check_file_exists(file_path, hub_path), file_name = check_file_name(file_path) ) #> #> ── team1-goodmodel/2022-10-08-team1-goodmodel.csv ──── #> #> ✔ [file_exists]: File exists at path #> model-output/team1-goodmodel/2022-10-08-team1-goodmodel.csv. #> ✔ [file_name]: File name "2022-10-08-team1-goodmodel.csv" is valid. x <- list( file_exists = check_file_exists(file_path, hub_path), file_name = check_file_name(file_path) ) as_hub_validations(x) #> #> ── team1-goodmodel/2022-10-08-team1-goodmodel.csv ──── #> #> ✔ [file_exists]: File exists at path #> model-output/team1-goodmodel/2022-10-08-team1-goodmodel.csv. #> ✔ [file_name]: File name "2022-10-08-team1-goodmodel.csv" is valid. ``` -------------------------------- ### Example check function with skip logic Source: https://hubverse-org.github.io/hubValidations/articles/writing-custom-fns.html Example of a check function that skips when no cdf or quantile output types are present. It uses capture_check_info() to return a message and capture_check_cnd() for the main check. ```R check_tbl_value_col_ascending function (tbl, file_path, hub_path, round_id, derived_task_ids = get_hub_derived_task_ids(hub_path)) { check_output_types <- intersect(c("cdf", "quantile"), unique(tbl[["output_type"]])) if (length(check_output_types) == 0L) { return(capture_check_info(file_path, "No quantile or cdf output types to check for non-descending values.\n Check skipped.")) } config_tasks <- hubUtils::read_config(hub_path, "tasks") if (!is.null(derived_task_ids)) { tbl[derived_task_ids] <- NA_character_ } error_tbl <- purrr::list_rbind(purrr::map(check_output_types, function(.x) { check_values_ascending_by_output_type(.x, tbl, config_tasks, round_id, derived_task_ids) })) check <- nrow(error_tbl) == 0L if (check) { details <- NULL error_tbl <- NULL } else { details <- cli::format_inline("See {.var error_tbl} attribute for details.") } capture_check_cnd(check = check, file_path = file_path, msg_subject = "Quantile or cdf {.var value} values", msg_verbs = c("increase", "do not all increase"), msg_attribute = "when ordered by {.var output_type_id}.", details = details, error_tbl = error_tbl) } ``` -------------------------------- ### Read target file with default types Source: https://hubverse-org.github.io/hubValidations/reference/read_target_file.html Reads a time-series target file from a hub using read_target_file. The example first sets the hub path to the test hub's target_file directory, then reads 'time-series.csv'. The output is a tibble with 66 rows and 4 columns: target_end_date (date), target, location, and observation (numeric). ```R # download example hub hub_path <- system.file("testhubs/v5/target_file", package = "hubUtils" ) # read in time-series file read_target_file("time-series.csv", hub_path) #> # A tibble: 66 × 4 #> target_end_date target location observation #> #> 1 2022-10-22 flu_hosp_inc 02 3 #> 2 2022-10-22 flu_hosp_inc 01 141 #> 3 2022-10-22 flu_hosp_inc US 2380 #> 4 2022-10-29 flu_hosp_inc 02 14 #> 5 2022-10-29 flu_hosp_inc 01 262 #> 6 2022-10-29 flu_hosp_inc US 4353 #> 7 2022-11-05 flu_hosp_inc 02 10 #> 8 2022-11-05 flu_hosp_inc 01 360 #> 9 2022-11-05 flu_hosp_inc US 6571 #> 10 2022-11-12 flu_hosp_inc 02 20 #> # ℹ 56 more rows ``` -------------------------------- ### validate_pr() example Source: https://hubverse-org.github.io/hubValidations/reference/validate_pr.html Example call to validate_pr() that checks a pull request against a hub repository. The code is wrapped in \dontrun{} to prevent execution during package checks. It requires a valid hub path, GitHub repository, and pull request number. ```r if (FALSE) { # \dontrun{ validate_pr( hub_path = ".", gh_repo = "hubverse-org/ci-testhub-simple", pr_number = 3 ) } # } ``` -------------------------------- ### Create submission template for a specific output type Source: https://hubverse-org.github.io/hubValidations/reference/submission_tmpl.html Create a submission template for a specific output type ("sample") by passing a hub path to submission_tmpl(). The output is a tibble with 640 rows and 8 columns, including reference_date, target, horizon, location, target_end_date, output_type, output_type_id, and value. ```r hub_path <- system.file("testhubs", "samples", package = "hubValidations") submission_tmpl( hub_path, round_id = "2022-12-17", output_types = "sample" ) #> # A tibble: 640 × 8 #> reference_date target horizon location target_end_date output_type #> #> 1 2022-12-17 wk inc flu hosp 0 US 2022-10-22 sample #> 2 2022-12-17 wk inc flu hosp 1 US 2022-10-22 sample #> 3 2022-12-17 wk inc flu hosp 2 US 2022-10-22 sample #> 4 2022-12-17 wk inc flu hosp 3 US 2022-10-22 sample #> 5 2022-12-17 wk inc flu hosp 0 US 2022-10-29 sample #> 6 2022-12-17 wk inc flu hosp 1 US 2022-10-29 sample #> 7 2022-12-17 wk inc flu hosp 2 US 2022-10-29 sample #> 8 2022-12-17 wk inc flu hosp 3 US 2022-10-29 sample #> 9 2022-12-17 wk inc flu hosp 0 US 2022-11-05 sample #> 10 2022-12-17 wk inc flu hosp 1 US 2022-11-05 sample #> # ℹ 630 more rows #> # ℹ 2 more variables: output_type_id , value ``` -------------------------------- ### Capture validation warning with and without cli formatting Source: https://hubverse-org.github.io/hubValidations/reference/capture_validation_warning.html The first example shows a simple warning without location or extra fields. The second example attempts to use cli inline formatting with a variable, but fails because the variable is not in scope when the message is evaluated. ```r capture_validation_warning( msg = "Configuration files were modified" ) ``` ```r #> #> Warning: #> Configuration files were modified ``` ```r config_files <- c("tasks.json", "admin.json") capture_validation_warning( msg = "Config files modified: {.path {config_files}}", where = "hub-config", config_files = config_files ) ``` ```r #> Error in "fun(..., .envir = .envir)": ! Could not evaluate cli `{}` expression: `config_files`. #> Caused by error in `eval(expr, envir = envir)`: #> ! object 'config_files' not found ``` -------------------------------- ### Create submission template from S3 bucket hub Source: https://hubverse-org.github.io/hubValidations/reference/submission_tmpl.html Creates a submission template file using a config file from an AWS S3 bucket hub. The path to the hub's root directory is created with `s3_bucket()`, then passed to `submission_tmpl()` with a round ID and output type. The output is a tibble with 4,968 rows and 7 columns, and a message about updating a superseded URL. ```r s3_hub_path <- arrow::s3_bucket("hubverse/hubutils/testhubs/simple/") submission_tmpl( path = s3_hub_path, round_id = "2022-10-01", output_types = "quantile" ) ```