### Install pbkrtest from CRAN and GitHub Source: https://github.com/hojsgaard/pbkrtest/blob/master/README.Rmd Install the stable version from CRAN with install.packages, or the development version from GitHub using remotes::install_github with build_vignettes = TRUE. ```R ## Install from CRAN: install.packages('pbkrtest') ## Install from Github: Use the remotes package: remotes::install_github("hojsgaard/pbkrtest", build_vignettes = TRUE) ``` -------------------------------- ### Basic PBrefdist example Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb-refdist.Rd This example loads the beets data, fits a linear mixed model with lmer, updates it to remove the harvest term, and computes the parametric bootstrap reference distribution with nsim=20 and cl=1. The result is stored in rd and printed. ```R data(beets) head(beets) beet0 <- lmer(sugpct ~ block + sow + harvest + (1|block:harvest), data=beets, REML=FALSE) beet_no.harv <- update(beet0, . ~ . -harvest) rd <- PBrefdist(beet0, beet_no.harv, nsim=20, cl=1) rd ``` -------------------------------- ### Parametric bootstrap model comparison example Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb__modcomp.Rd This example demonstrates how to use PBmodcomp to compare nested linear mixed models fitted with lmer. It fits three models with increasing complexity and sets the number of simulations to 50 (default is 1000). The example is wrapped in a dontrun block, indicating it is not run during package checks. ```r \dontrun{ (fm0 <- lmer(Reaction ~ (Days|Subject), sleepstudy)) (fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)) (fm2 <- lmer(Reaction ~ Days + I(Days^2) + (Days|Subject), sleepstudy)) NSIM <- 50 ## Simulations in parametric bootstrap; default is 1000. ``` -------------------------------- ### Convert PBmodcomp result and summary to data frames Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/as.data.frame.PBmodcomp.Rd Run a parametric bootstrap model comparison with pb2_modcomp, then convert the result and its summary to data frames. The example uses the sleepstudy dataset and requires lme4. Note that the example is wrapped in \dontrun{} and will not run automatically. ```R sleepstudy$Days2 <- sleepstudy$Days^2 lmer_fit1 <- lme4::lmer(Reaction ~ Days + Days2 + (1 | Subject), data = sleepstudy, REML=FALSE) lmer_fit0 <- update(lmer_fit1, . ~ . - Days2) res <- pb2_modcomp(lmer_fit1, lmer_fit0, nsim = 200) df <- as.data.frame(res) head(df) s <- summary(res) as.data.frame(s) ``` -------------------------------- ### Create example data for linear models Source: https://github.com/hojsgaard/pbkrtest/blob/master/vignettes/a02-coercion.rmd Creates a small data frame with N=4 rows, an intercept column, a predictor x, and a response y generated from a normal distribution. This data is used to fit the example models. ```r N <- 4 dat <- data.frame(int=rep(1, N), x=1:N, y=rnorm(N)) ``` -------------------------------- ### Parallel computation with forking (non-Windows) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb-refdist.Rd This example shows how to use multiple cores on operating systems that fork processes (not Windows). It sets the number of cores via options('mc.cores') for all calls, or via the cl argument for a single call. Note that many more simulations are needed in practice. ```R if (Sys.info()["sysname"] != "Windows"){ N <- 2 ## Or N <- parallel::detectCores() # N cores used in all calls to function in a session options("mc.cores"=N) rd <- PBrefdist(beet0, beet_no.harv, nsim=20) # N cores used just in one specific call (when cl is set, # options("mc.cores") is ignored): rd <- PBrefdist(beet0, beet_no.harv, nsim=20, cl=N) } ``` -------------------------------- ### Parallel computation with clusters (all platforms) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb-refdist.Rd This example demonstrates using a cluster created with makeCluster on all platforms, including Windows. It sets the cluster via options('pb.cl') for all calls, or via the cl argument for a single call, and stops the cluster afterwards. ```R library(parallel) N <- 2 ## Or N <- detectCores() clus <- makeCluster(rep("localhost", N)) # In all calls in a session options("pb.cl"=clus) rd <- PBrefdist(beet0, beet_no.harv, nsim=20) # Just once: rd <- PBrefdist(beet0, beet_no.harv, nsim=20, cl=clus) stopCluster(clus) ``` -------------------------------- ### R example: Fit models, compare with anova and KRmodcomp, and compute ddf with get_Lb_ddf Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/get_ddf_Lb.Rd This example fits two linear mixed models to the sleepstudy data, compares them with anova and KRmodcomp, and uses get_Lb_ddf to obtain the denominator degrees of freedom for a specific contrast. It also shows how to obtain the restriction matrix L using model2restriction_matrix. The output comments indicate that both KRmodcomp and get_Lb_ddf yield 17 denominator degrees of freedom. ```R (fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)) (fm0 <- lmer(Reaction ~ 1 + (Days|Subject), sleepstudy)) anova(fm1, fm0) KRmodcomp(fm1, fm0) ## 17 denominator df's get_Lb_ddf(fm1, c(0, 1)) ## 17 denominator df's # Notice: The restriction matrix L corresponding to the test above # can be found with L <- model2restriction_matrix(fm1, fm0) L ``` -------------------------------- ### getLRT with lme models Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/getLRT.Rd Example for linear mixed-effects models (lme) using the nlme package. Requires nlme to be installed. Fits full and nested models with method = "ML". ```R ## lme if (requireNamespace("nlme", quietly = TRUE)) { library(nlme) fit1 <- lme(distance ~ age + Sex, random = ~1 | Subject, data = Orthodont, method = "ML") fit0 <- lme(distance ~ age, random = ~1 | Subject, data = Orthodont, method = "ML") getLRT(fit1, fit0) } ``` -------------------------------- ### getLRT with gls models Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/getLRT.Rd Example for generalized least squares models (gls) using the nlme package. Requires nlme to be installed. Fits full and nested models with method = "ML". ```R ## gls if (requireNamespace("nlme", quietly = TRUE)) { library(nlme) fit1 <- gls(mpg ~ wt + hp, data = mtcars, method = "ML") fit0 <- gls(mpg ~ wt, data = mtcars, method = "ML") getLRT(fit1, fit0) } ``` -------------------------------- ### Satterthwaite F-test examples with sleepstudy Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/sat__modcomp.Rd These examples demonstrate SATmodcomp for testing fixed effects in linear mixed models. They show how to test for the effect of Days, Days-squared, and combinations using model formulas, restriction matrices, and model objects. Note that some restriction matrix calls are commented out as FIXME. ```R (fm0 <- lmer(Reaction ~ (Days|Subject), sleepstudy)) (fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)) (fm2 <- lmer(Reaction ~ Days + I(Days^2) + (Days|Subject), sleepstudy)) ## Test for no effect of Days in fm1, i.e. test fm0 under fm1 SATmodcomp(fm1, "Days") SATmodcomp(fm1, ~.-Days) L1 <- cbind(0, 1) ## SATmodcomp(fm1, L1) ## FIXME SATmodcomp(fm1, fm0) anova(fm1, fm0) ## Test for no effect of Days and Days-squared in fm2, i.e. test fm0 under fm2 SATmodcomp(fm2, "(Days+I(Days^2))") SATmodcomp(fm2, ~. - Days - I(Days^2)) L2 <- rbind(c(0, 1, 0), c(0, 0, 1)) ## SATmodcomp(fm2, L2) ## FIXME SATmodcomp(fm2, fm0) anova(fm2, fm0) ## Test for no effect of Days-squared in fm2, i.e. test fm1 under fm2 SATmodcomp(fm2, "I(Days^2)") SATmodcomp(fm2, ~. - I(Days^2)) L3 <- rbind(c(0, 0, 1)) ## SATmodcomp(fm2, L3) ## FIXME SATmodcomp(fm2, fm1) anova(fm2, fm1) ``` -------------------------------- ### getLRT with glmerMod models Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/getLRT.Rd Example for generalized linear mixed models (glmerMod) using the lme4 package. Requires lme4 to be installed. Fits full and nested models with binomial family. ```R ## glmerMod if (requireNamespace("lme4", quietly = TRUE)) { library(lme4) data(cbpp) fit1 <- glmer(cbind(incidence, size - incidence) ~ period + (1 | herd), data = cbpp, family = binomial) fit0 <- glmer(cbind(incidence, size - incidence) ~ 1 + (1 | herd), data = cbpp, family = binomial) getLRT(fit1, fit0) } ``` -------------------------------- ### getLRT with lm models Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/getLRT.Rd Example for linear models (lm). Fits a full model with wt and hp, and a nested model with only wt, then calls getLRT. ```R ## lm fit1 <- lm(mpg ~ wt + hp, data = mtcars) fit0 <- lm(mpg ~ wt, data = mtcars) getLRT(fit1, fit0) ``` -------------------------------- ### getLRT with lmerMod models Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/getLRT.Rd Example for linear mixed models (lmerMod) using the lme4 package. Requires lme4 to be installed. Fits full and nested models with REML = FALSE. ```R ## lmerMod if (requireNamespace("lme4", quietly = TRUE)) { library(lme4) fit1 <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy, REML = FALSE) fit0 <- lmer(Reaction ~ 1 + (Days | Subject), sleepstudy, REML = FALSE) getLRT(fit1, fit0) } ``` -------------------------------- ### PBmodcomp for Generalized Linear Mixed Model (glmer) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb__modcomp.Rd Fits a binomial GLMM with glmer and compares models using PBmodcomp with nsim=NSIM. The example is wrapped in \dontrun{} because fitting takes a while. It shows both formula-based and model-object-based comparisons, plus anova. ```r (gm1 <- glmer(cbind(incidence, size - incidence) ~ period + (1 | herd), data = cbpp, family = binomial)) (gm2 <- update(gm1, .~.-period)) PBmodcomp(gm1, "period", nsim=NSIM) PBmodcomp(gm1, ~. -period, nsim=NSIM) PBmodcomp(gm1, gm2, nsim=NSIM) anova(gm1, gm2) ``` -------------------------------- ### Simulate responses for GLS, LME, lmerMod, and lm models Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/simulate0.Rd Example usage of simulate0 with four model types on the Orthodont dataset. Requires nlme, lme4, and MASS packages. Sets NSIM to 6 and seed to 123 for reproducibility, then prints the first rows of each simulated data frame. ```R if (require(nlme) && require(lme4) && require(MASS)) { data(Orthodont, package = "nlme") NSIM <- 6 # GLS model fit_gls <- gls(distance ~ age, data = Orthodont) sim_gls <- simulate0(fit_gls, nsim = NSIM, seed = 123) head(sim_gls) # LME model fit_lme <- lme(distance ~ age, random = ~1 | Subject, data = Orthodont) sim_lme <- simulate0(fit_lme, nsim = NSIM, seed = 123) head(sim_lme) # lmerMod model fit_lmer <- lme4::lmer(distance ~ age + (1 | Subject), data = Orthodont) sim_lmer <- simulate0(fit_lmer, nsim = NSIM, seed = 123) head(sim_lmer) # lm fit_lm <- lm(distance ~ age, data = Orthodont) sim_lm <- simulate0(fit_lm, nsim = NSIM, seed = 123) head(sim_lm) } ``` -------------------------------- ### Refit lm, gls, and lme models with new response Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/refit.Rd Fits lm, gls, and lme models to the Orthodont dataset, simulates a new response vector, and refits each model using the refit generic. Requires the nlme and lme4 packages. The example sets a seed for reproducibility and uses rnorm to generate new responses. ```R if (require(nlme) && require(lme4)) { data(Orthodont, package = "nlme") # Fit models fit_lm <- lm(distance ~ age, data = Orthodont) fit_gls <- gls(distance ~ age, data = Orthodont) fit_lme <- lme(distance ~ age, random = ~ 1 | Subject, data = Orthodont) # Simulate new response vectors set.seed(123) new_y <- rnorm(nrow(Orthodont), mean = mean(Orthodont$distance), sd = sd(Orthodont$distance)) # Refit models with new response refit(fit_lm, newresp = new_y) refit(fit_gls, newresp = new_y) refit(fit_lme, newresp = new_y) } ``` -------------------------------- ### PBrefdist usage and methods Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb-refdist.Rd The usage section shows the generic function and its methods for lm, merMod, and gls objects. All share the same arguments: largeModel, smallModel, nsim, seed, cl, and details. ```R PBrefdist( largeModel, smallModel, nsim = 1000, seed = NULL, cl = NULL, details = 0 ) \method{PBrefdist}{lm}( largeModel, smallModel, nsim = 1000, seed = NULL, cl = NULL, details = 0 ) \method{PBrefdist}{merMod}( largeModel, smallModel, nsim = 1000, seed = NULL, cl = NULL, details = 0 ) \method{PBrefdist}{gls}( largeModel, smallModel, nsim = 1000, seed = NULL, cl = NULL, details = 0 ) ``` -------------------------------- ### X2modcomp usage and default method Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/x2__modcomp.Rd Shows the usage signature for X2modcomp and its default method. The function compares a large model against a small model or restriction matrix, with optional betaH and details arguments. ```r X2modcomp(largeModel, smallModel, betaH = 0, details = 0, ...) \method{X2modcomp}{default}(largeModel, smallModel, betaH = 0, details = 0, ...) ``` -------------------------------- ### pb_refdist_sequential(fit1, fit0, h = 20, nsim = 1000, batch_size = 50, engine = c("serial", "parallel", "future"), nworkers = 2, verbose = FALSE) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb_refdist.Rd Runs batches of parametric bootstrap simulations until a target number of extreme hits is reached or a maximum number of simulations is hit, useful for efficient estimation of small p-values. ```APIDOC ## pb_refdist_sequential(fit1, fit0, h = 20, nsim = 1000, batch_size = 50, engine = c("serial", "parallel", "future"), nworkers = 2, verbose = FALSE) ### Description Runs batches of parametric bootstrap simulations until a target number of extreme hits is reached (or a maximum number of simulations is hit), useful for efficient estimation of small p-values. ### Usage pb_refdist_sequential(fit1, fit0, h = 20, nsim = 1000, batch_size = 50, engine = c("serial", "parallel", "future"), nworkers = 2, verbose = FALSE) ### Parameters - **fit1** (model object) - Required - The larger (alternative) model, e.g. lm, gls, lme, lmer. - **fit0** (model object) - Required - The smaller (null) model, nested in fit1. - **h** (numeric) - Optional - Number of extreme hits to target. Default is 20. - **nsim** (numeric) - Optional - Maximum number of simulations. Default is 1000. - **batch_size** (numeric) - Optional - Number of simulations per batch. Default is 50. - **engine** (character) - Optional - Computation engine: "serial", "parallel", or "future". Default is "serial". - **nworkers** (numeric) - Optional - Number of workers for parallel/future backend. Default is 2. - **verbose** (logical) - Optional - If TRUE, print progress messages. Default is FALSE. ### Return Value An object of class "PBrefdist" containing the observed statistic, the bootstrap replicates, degrees of freedom, asymptotic p-value, and optionally the number of hits and standard error. ### Details Computes a parametric bootstrap reference distribution for a likelihood ratio statistic, comparing a large (alternative) and a small (null) model. The sequential version is useful when one wants to control Monte Carlo error by targeting a fixed number of extreme values exceeding the observed test statistic. ### Notes - Best Practice: Always fit models with the data= argument to ensure all covariates are stored with the model object, enabling reliable simulation and refitting for bootstrap analysis, including on parallel workers. - The returned object can be passed to summary(), plot(), and as.data.frame(). - The function automatically ensures that the models have their required data embedded, guaranteeing that parametric bootstrap simulations can be run in parallel workers without errors about missing variables. ``` -------------------------------- ### pb_refdist(fit1, fit0, nsim = 1000, engine = c("serial", "parallel", "future"), nworkers = 2, verbose = FALSE) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb_refdist.Rd Simulates a fixed number of parametric bootstrap replicates to compute a reference distribution for a likelihood ratio statistic comparing a larger model against a nested smaller model. ```APIDOC ## pb_refdist(fit1, fit0, nsim = 1000, engine = c("serial", "parallel", "future"), nworkers = 2, verbose = FALSE) ### Description Simulates a fixed number of parametric bootstrap replicates to compute a reference distribution for a likelihood ratio statistic comparing a larger (alternative) model against a nested smaller (null) model. ### Usage pb_refdist(fit1, fit0, nsim = 1000, engine = c("serial", "parallel", "future"), nworkers = 2, verbose = FALSE) ### Parameters - **fit1** (model object) - Required - The larger (alternative) model, e.g. lm, gls, lme, lmer. - **fit0** (model object) - Required - The smaller (null) model, nested in fit1. - **nsim** (numeric) - Optional - Number of simulations. Default is 1000. - **engine** (character) - Optional - Computation engine: "serial", "parallel", or "future". Default is "serial". - **nworkers** (numeric) - Optional - Number of workers for parallel/future backend. Default is 2. - **verbose** (logical) - Optional - If TRUE, print progress messages. Default is FALSE. ### Return Value An object of class "PBrefdist" containing the observed statistic, the bootstrap replicates, degrees of freedom, asymptotic p-value, and optionally the number of hits and standard error. ### Details Computes a parametric bootstrap reference distribution for a likelihood ratio statistic, comparing a large (alternative) and a small (null) model. The distribution can be used to estimate a bootstrap p-value. ### Notes - Best Practice: Always fit models with the data= argument to ensure all covariates are stored with the model object, enabling reliable simulation and refitting for bootstrap analysis, including on parallel workers. - The returned object can be passed to summary(), plot(), and as.data.frame(). - The function automatically ensures that the models have their required data embedded, guaranteeing that parametric bootstrap simulations can be run in parallel workers without errors about missing variables. ``` -------------------------------- ### Compute covariance matrix for lmerMod model Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/cov_matrix.Rd Shows how to compute the marginal covariance matrix for an lmerMod model from the lme4 package using the sleepstudy dataset. The example requires the lme4 and Matrix packages and prints the resulting matrix. ```R if (require(lme4) && require(Matrix)) { ## lmerMod example fit_lmer <- lme4::lmer(Reaction ~ Days + (Days | Subject), sleepstudy) V_lmer <- cov_matrix(fit_lmer) print(V_lmer) } ``` -------------------------------- ### Compare models with anova, PBmodcomp, KRmodcomp, and SATmodcomp Source: https://github.com/hojsgaard/pbkrtest/blob/master/README.html Tests whether there is an effect of harvest time by comparing the full model (fm0) with the reduced model (fm1) using four different methods: anova, PBmodcomp (parametric bootstrap), KRmodcomp (Kenward-Roger), and SATmodcomp (Satterthwaite). The tidy() output for each test is shown. ```r ## Is there an effect of harvest time? an <- anova(fm0, fm1) pb <- PBmodcomp(fm0, fm1) kr <- KRmodcomp(fm0, fm1) sa <- SATmodcomp(fm0, fm1) tidy(an) #> # A tibble: 2 × 9 #> term npar AIC BIC logLik deviance statistic df p.value #> #> 1 fm1 9 -69.1 -56.5 43.5 -87.1 NA NA NA #> 2 fm0 10 -80.0 -66.0 50.0 -100. 12.9 1 0.000326 tidy(pb) #> # A tibble: 2 × 4 #> type stat df p.value #> #> 1 LRT 12.9 1 0.000326 #> 2 PBtest 12.9 NA 0.0300 tidy(kr) #> # A tibble: 1 × 6 #> type stat ndf ddf F.scaling p.value #> #> 1 Ftest 15.2 1 2.00 1 0.0599 tidy(sa) #> # A tibble: 1 × 5 #> type statistic ndf ddf p.value #> #> 1 Ftest 15.2 1 2.00 0.0599 ``` -------------------------------- ### Details on cl argument Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb-refdist.Rd The details section includes two code blocks that explain how the 'cl' argument controls parallel computations. The first block describes the general behavior, and the second block provides the recommended way to set it via options(). ```R The argument 'cl' (originally short for 'cluster') is used for controlling parallel computations. 'cl' can be NULL (default), positive integer or a list of clusters. ``` ```R The recommended way of controlling cl is to specify the component \code{pbcl} in options() with e.g. \code{options("pbcl"=4)}. If cl is NULL, the function will look at if the pbcl has been set in the options list with \code{getOption("pbcl")} If cl=N then N cores will be used in the computations. If cl is NULL then the function will look for ``` -------------------------------- ### Load broom and shoes data Source: https://github.com/hojsgaard/pbkrtest/blob/master/vignettes/a01-pbkrtest.rmd Loads the broom package and the shoes dataset from MASS, then displays the data. ```r library(broom) ``` ```r data(shoes, package="MASS") shoes ``` -------------------------------- ### Compute covariance matrix for gls and lme models Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/cov_matrix.Rd Shows how to compute the marginal covariance matrix for gls and lme models from the nlme package using the Orthodont dataset. The example requires the nlme and Matrix packages and prints the resulting matrices. ```R if (require(nlme) && require(Matrix)) { ## gls example fit_gls <- gls(distance ~ age, data = Orthodont) V_gls <- cov_matrix(fit_gls) print(V_gls) ## lme example fit_lme <- lme(distance ~ age, random = ~ 1 | Subject, data = Orthodont) V_lme <- cov_matrix(fit_lme) print(V_lme) } ``` -------------------------------- ### PBrefdist Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb__modcomp.Rd Simulates the reference distribution for the parametric bootstrap test. This function is used to generate the bootstrap distribution separately, which can then be used in PBmodcomp. ```APIDOC ## PBrefdist ### Description Simulates the reference distribution for the parametric bootstrap test. This function is used to generate the bootstrap distribution separately, which can then be used in PBmodcomp. ### Usage PBrefdist(largeModel, smallModel, nsim = NULL, cl = NULL) ### Arguments - **largeModel** (model object) - Required - The full model. - **smallModel** (formula, character string, or model object) - Required - The reduced model specification. - **nsim** (numeric) - Optional - Number of bootstrap simulations. - **cl** (cluster object) - Optional - A cluster for parallel computation. ### Details This function simulates the reference distribution for the parametric bootstrap test. It can be used to pre-compute the distribution and then pass it to PBmodcomp. ### Examples ```r refdist <- PBrefdist(sug, sug.h, nsim=1000, cl=1) refdist <- PBrefdist(sug, "harvest", nsim=1000, cl=1) refdist <- PBrefdist(sug, ~.-harvest, nsim=1000, cl=1) ``` ### See Also PBmodcomp, KRmodcomp ### Author Søren Højsgaard ``` -------------------------------- ### Extract covariance parameters from GLS, LME, lmerMod, and lm models Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/cov_par.Rd Example usage of cov_par on the Orthodont dataset for four model classes: gls, lme, lmerMod, and lm. Requires the nlme and lme4 packages. Each model is fitted and cov_par is called to extract the variance and correlation parameters. ```r if (require(nlme) && require(lme4)) { data(Orthodont, package = "nlme") # GLS model fit_gls <- gls(distance ~ age, data = Orthodont) cov_par(fit_gls) # LME model fit_lme <- lme(distance ~ age, random = ~ 1 | Subject, data = Orthodont) cov_par(fit_lme) # lmerMod model fit_lmer <- lme4::lmer(distance ~ age + (1 | Subject), data = Orthodont) cov_par(fit_lmer) # lm model fit_lm <- lm(distance ~ age, data = Orthodont) cov_par(fit_lm) } ``` -------------------------------- ### Read budworm data with SAS Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/data-budworm.Rd SAS data step to read the budworm data from an external file 'budworm.txt'. The INFILE statement uses FIRSTOBS=2 to skip the header line, and the INPUT statement reads the variables sex, dose, ndead, and ntotal. This example is wrapped in \dontrun{} in the R documentation, indicating it is not run automatically. ```sas * SAS example; data budworm; infile 'budworm.txt' firstobs=2; input sex dose ndead ntotal; run; ``` -------------------------------- ### Load and analyze beets data with ANOVA Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/data-beets.Rd Loads the beets dataset, creates an interaction term for block and harvest, then fits ANOVA models for yield and sugar percentage with block, sow, harvest, and the Error(bh) term. ```R data(beets) beets$bh <- with(beets, interaction(block, harvest)) summary(aov(yield ~ block + sow + harvest + Error(bh), beets)) summary(aov(sugpct ~ block + sow + harvest + Error(bh), beets)) ``` -------------------------------- ### Convert models to restriction matrices and back with pbkrtest Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/model-coerce.Rd Loads the pbkrtest package and the beets dataset, fits a linear model, creates reduced models, converts them to restriction matrices, converts back to models, and checks that fitted values and log-likelihoods match. ```R library(pbkrtest) data("beets", package = "pbkrtest") sug <- lm(sugpct ~ block + sow + harvest, data=beets) sug.h <- update(sug, .~. - harvest) sug.s <- update(sug, .~. - sow) ## Construct restriction matrices from models L.h <- model2restriction_matrix(sug, sug.h); L.h L.s <- model2restriction_matrix(sug, sug.s); L.s ## Construct submodels from restriction matrices mod.h <- restriction_matrix2model(sug, L.h); mod.h mod.s <- restriction_matrix2model(sug, L.s); mod.s ## Sanity check: The models have the same fitted values and log likelihood max(abs(fitted(mod.h) - fitted(sug.h))) max(abs(fitted(mod.s) - fitted(sug.s))) logLik(mod.h) logLik(sug.h) logLik(mod.s) logLik(sug.s) ``` -------------------------------- ### PBmodcomp for Linear Model (lm) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb__modcomp.Rd Fits a linear model with lm and compares it to reduced models using PBmodcomp with cl=2. Shows how to test removal of the 'speed' and 'I(speed^2)' terms. ```r lm1 <- lm(dist~speed+I(speed^2), data=cars) PBmodcomp(lm1, .~.-speed, cl=2) PBmodcomp(lm1, .~.-I(speed^2), cl=2) ``` -------------------------------- ### PBmodcomp for Linear Mixed Effects Model (lmer) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/pb__modcomp.Rd Fits a linear mixed model with lmer and compares models using PBmodcomp and anova. Shows how to test removal of fixed effects 'harvest' and 'sow'. Also demonstrates simulating reference distributions with PBrefdist and parallel computation using clusters. ```r sug <- lmer(sugpct ~ block + sow + harvest + (1|block:harvest), data=beets, REML=FALSE) sug.h <- update(sug, .~. -harvest) sug.s <- update(sug, .~. -sow) anova(sug, sug.h) PBmodcomp(sug, sug.h, nsim=NSIM, cl=1) PBmodcomp(sug, "harvest", nsim=NSIM, cl=1) anova(sug, sug.s) PBmodcomp(sug, sug.s, nsim=NSIM, cl=1) PBmodcomp(sug, "sow", nsim=NSIM, cl=1) ## Simulate reference distribution separately: refdist <- PBrefdist(sug, sug.h, nsim=1000, cl=1) refdist <- PBrefdist(sug, "harvest", nsim=1000, cl=1) refdist <- PBrefdist(sug, ~.-harvest, nsim=1000, cl=1) ## Do computations with multiple processors: ## Number of cores: (nc <- detectCores()) ## Create clusters cl <- makeCluster(rep("localhost", nc)) ## Then do: refdist <- PBrefdist(sug, sug.h, nsim=1000, cl=cl) ## It is recommended to stop the clusters before quitting R: stopCluster(cl) ``` -------------------------------- ### Load pbkrtest and plot beets data Source: https://github.com/hojsgaard/pbkrtest/blob/master/README.html Loads the pbkrtest and ggplot2 packages, then creates a scatter plot of sugar content versus sowing date, colored by harvest time, using the beets dataset. ```R library(pbkrtest) library(ggplot2) ## Sugar beets: Does suger content depend on harvest time? beets |> ggplot(aes(x=sow, y=sugpct, group=harvest)) + geom_jitter(aes(color=harvest), width=0) ``` -------------------------------- ### summarize_pb(LRTstat, ref, conf.level = 0.95) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/summarize_pb.Rd Calculates asymptotic p-value, bootstrap p-value with confidence interval, Bartlett corrections (mean and median-based), and F-approximations (mean and median-based) from observed and simulated test statistics. ```APIDOC ## summarize_pb(LRTstat, ref, conf.level = 0.95) ### Description Calculates asymptotic p-value, bootstrap p-value with CI, Bartlett corrections (mean and median-based) and F-approximations (mean and median-based). ### Usage ```r summarize_pb(LRTstat, ref, conf.level = 0.95) ``` ### Arguments - **LRTstat** (numeric vector) - Required - Numeric vector: c(tobs, df) = observed test statistic and df. - **ref** (numeric vector) - Required - Numeric vector of simulated bootstrap test statistics. - **conf.level** (numeric) - Optional - Confidence level for bootstrap p CI (default 0.95). ### Value List with results table, moments, samples info, CI, SE, etc. ``` -------------------------------- ### Parametric bootstrap for balanced data Source: https://github.com/hojsgaard/pbkrtest/blob/master/vignettes/a01-pbkrtest.rmd Performs a parametric bootstrap test with 500 simulations and 2 cores, then displays tidy results. ```r pb.bal <- PBmodcomp(lmm1.bal, lmm0.bal, nsim=500, cl=2) pb.bal |> tidy() summary(pb.bal) |> tidy() ``` -------------------------------- ### Compare column spaces with compare_column_space Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/compare_column_space.Rd Creates a 3x2 matrix A1 and a single-column matrix A2 (the first column of A1), then calls compare_column_space to compare their column spaces. The first call returns 0 (C(A1) contained in C(A2)), the second returns 1 (C(A2) contained in C(A1)), and the third returns -1 (equal spaces). ```r A1 <- matrix(c(1,1,1,1,2,3), nrow=3) A2 <- A1[, 1, drop=FALSE] compare_column_space(A1, A2) compare_column_space(A2, A1) compare_column_space(A1, A1) ``` -------------------------------- ### print.anovax(x, ...) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/anovax.Rd Prints an anovax object. ```APIDOC ## print.anovax(x, ...) ### Description Prints an anovax object. ### Usage ```r \method{print}{anovax}(x, ...) ``` ### Arguments - **x** (anovax object) - Required - An anovax object. - **...** - Optional - Further arguments. ``` -------------------------------- ### Compare column spaces of model matrices Source: https://github.com/hojsgaard/pbkrtest/blob/master/vignettes/a02-coercion.rmd Demonstrates compare_column_space() to check whether one column space contains another. The first call checks if Xlg's column space contains Xsm's, the second checks the reverse, and the third checks if Xlg's column space is identical to itself. ```r ## The first column space contains the second compare_column_space(Xlg, Xsm) ## The second column space contains the first compare_column_space(Xsm, Xlg) ## The two column spaces are identical compare_column_space(Xlg, Xlg) ``` -------------------------------- ### SATmodcomp(largeModel, smallModel, betaH = 0, details = 0, eps = sqrt(.Machine$double.eps)) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/sat__modcomp.Rd Performs an approximate F-test based on the Satterthwaite approximation to compare a large model against a small model or a restriction matrix. The small model can be specified as an lmer model object, a formula, or a restriction matrix. ```APIDOC ## SATmodcomp(largeModel, smallModel, betaH = 0, details = 0, eps = sqrt(.Machine$double.eps)) ### Description Performs an approximate F-test based on the Satterthwaite approach for comparing nested linear mixed models. ### Usage ```r SATmodcomp( largeModel, smallModel, betaH = 0, details = 0, eps = sqrt(.Machine$double.eps) ) ``` ### Arguments - **largeModel** (lmer model) - Required - An `lmer` model representing the larger model. - **smallModel** (lmer model or restriction matrix) - Required - An `lmer` model or a restriction matrix representing the smaller model or hypothesis. - **betaH** (numeric) - Optional - A number or a vector of the beta of the hypothesis, e.g. L beta = L betaH. If `smallModel` is a model object then betaH = 0. - **details** (numeric) - Optional - If larger than 0 some timing details are printed. - **eps** (numeric) - Optional - A small number used for numerical tolerance. ### Details Notice: It cannot be guaranteed that the results agree with other implementations of the Satterthwaite approach! ### Value An object containing the F-test results, including the test statistic and degrees of freedom. ### Examples ```r (fm0 <- lmer(Reaction ~ (Days|Subject), sleepstudy)) (fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)) (fm2 <- lmer(Reaction ~ Days + I(Days^2) + (Days|Subject), sleepstudy)) ## Test for no effect of Days in fm1, i.e. test fm0 under fm1 SATmodcomp(fm1, "Days") SATmodcomp(fm1, ~.-Days) L1 <- cbind(0, 1) ## SATmodcomp(fm1, L1) ## FIXME SATmodcomp(fm1, fm0) anova(fm1, fm0) ## Test for no effect of Days and Days-squared in fm2, i.e. test fm0 under fm2 SATmodcomp(fm2, "(Days+I(Days^2))") SATmodcomp(fm2, ~. - Days - I(Days^2)) L2 <- rbind(c(0, 1, 0), c(0, 0, 1)) ## SATmodcomp(fm2, L2) ## FIXME SATmodcomp(fm2, fm0) anova(fm2, fm0) ## Test for no effect of Days-squared in fm2, i.e. test fm1 under fm2 SATmodcomp(fm2, "I(Days^2)") SATmodcomp(fm2, ~. - I(Days^2)) L3 <- rbind(c(0, 0, 1)) ## SATmodcomp(fm2, L3) ## FIXME SATmodcomp(fm2, fm1) anova(fm2, fm1) ``` ### References Ulrich Halekoh, Søren Højsgaard (2014)., A Kenward-Roger Approximation and Parametric Bootstrap Methods for Tests in Linear Mixed Models - The R Package pbkrtest., Journal of Statistical Software, 58(10), 1-30., https://www.jstatsoft.org/v59/i09/ ### See Also `getKR`, `lmer`, `vcovAdj`, `PBmodcomp`, `KRmodcomp` ### Author Søren Højsgaard, sorenh@math.aau.dk ``` -------------------------------- ### X2modcomp(largeModel, smallModel, betaH = 0, details = 0, ...) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/x2__modcomp.Rd Performs a chi-square test comparing a large model against a small model or a restriction matrix. The smallModel can be a model object or a restriction matrix, and betaH specifies the hypothesis values. ```APIDOC ## X2modcomp(largeModel, smallModel, betaH = 0, details = 0, ...) ### Description Performs a chi-square test for comparing a large model against a small model or a restriction matrix. ### Usage X2modcomp(largeModel, smallModel, betaH = 0, details = 0, ...) ### Arguments - **largeModel** (lmer model) - Required - An `lmer` model. - **smallModel** (lmer model or restriction matrix) - Required - An `lmer` model or a restriction matrix. - **betaH** (number or vector) - Optional - A number or a vector of the beta of the hypothesis, e.g. L beta = L betaH. If `smallModel` is a model object then betaH = 0. - **details** (numeric) - Optional - If larger than 0 some timing details are printed. - **...** - Optional - Additional arguments, currently not used. ### Examples ```r (fm0 <- lmer(Reaction ~ (Days|Subject), sleepstudy)) (fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)) (fm2 <- lmer(Reaction ~ Days + I(Days^2) + (Days|Subject), sleepstudy)) # Test for no effect of Days in fm1, i.e. test fm0 under fm1 X2modcomp(fm1, "Days") X2modcomp(fm1, ~.-Days) X2modcomp(fm1, fm0) anova(fm1, fm0) L1 <- cbind(0, 1) X2modcomp(fm1, L1) ## FIXME ``` ``` -------------------------------- ### as.data.frame.summary_PBmodcomp(x, ...) Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/as.data.frame.PBmodcomp.Rd Converts a summary_PBmodcomp object into a data frame containing the test results with attributes preserved. ```APIDOC ## as.data.frame.summary_PBmodcomp(x, ...) ### Description Converts a summary_PBmodcomp object into a data frame containing the test results with attributes preserved. ### Method as.data.frame.summary_PBmodcomp ### Signature as.data.frame.summary_PBmodcomp(x, ...) ### Parameters - **x** (summary_PBmodcomp) - Required - An object of class summary_PBmodcomp. - **...** - Optional - Further arguments (currently ignored). ### Return Value A data.frame containing the test results with attributes preserved. ### Details This method extracts the internal test component (a data frame of test statistics and p-values) from the object and returns it as a standard data frame. The original object's attributes are preserved as attributes on the returned data frame. ### Example ```r \dontrun{ sleepstudy$Days2 <- sleepstudy$Days^2 lmer_fit1 <- lme4::lmer(Reaction ~ Days + Days2 + (1 | Subject), data = sleepstudy, REML=FALSE) lmer_fit0 <- update(lmer_fit1, . ~ . - Days2) res <- pb2_modcomp(lmer_fit1, lmer_fit0, nsim = 200) s <- summary(res) as.data.frame(s) } ``` ``` -------------------------------- ### Extract ddf from KRmodcomp using getKR Source: https://github.com/hojsgaard/pbkrtest/blob/master/man/get_modcomp.Rd Creates a KRmodcomp object comparing fm1 and fm0, then extracts the denominator degrees of freedom using getKR. Also shows pipe-based extraction for three model comparisons. ```R x10 <- KRmodcomp(fm1, fm0) getKR(x10, "ddf") ``` ```R KRmodcomp(fm1, fm0) |> getKR("ddf") KRmodcomp(fm2, fm0) |> getKR("ddf") KRmodcomp(fm2, fm1) |> getKR("ddf") ```