### Install from CRAN Source: https://ohdsi.github.io/EmpiricalCalibration/index.html Use this command to install the latest stable version of the package from CRAN. ```R install.packages("EmpiricalCalibration") ``` -------------------------------- ### Install from GitHub Source: https://ohdsi.github.io/EmpiricalCalibration/index.html Use these commands to install the latest development version directly from the OHDSI GitHub repository. ```R install.packages("remotes") library(remotes) install_github("ohdsi/EmpiricalCalibration", ref = "develop") ``` -------------------------------- ### Example usage of computeTraditionalCi Source: https://ohdsi.github.io/EmpiricalCalibration/reference/computeTraditionalCi.html An example demonstrating how to use computeTraditionalCi with the sccs dataset. ```R data(sccs) positive <- sccs[sccs$groundTruth == 1, ] computeTraditionalCi(positive$logRr, positive$seLogRr) #> rr lb ub #> 1 2.080532 1.800637 2.403934 ``` -------------------------------- ### plotMcmcTrace example usage Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotMcmcTrace.html An example showing how to fit an MCMC null distribution and plot the trace. ```R if (FALSE) { # \dontrun{ data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitMcmcNull(negatives$logRr, negatives$seLogRr) plotMcmcTrace(null) } # } ``` -------------------------------- ### calibrateP usage example Source: https://ohdsi.github.io/EmpiricalCalibration/reference/calibrateP.html Example demonstrating how to fit a null distribution and calibrate p-values for positive control estimates. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNull(negatives$logRr, negatives$seLogRr) positive <- sccs[sccs$groundTruth == 1, ] calibrateP(null, positive$logRr, positive$seLogRr) #> [1] 0.8389142 ``` -------------------------------- ### fitNull example usage Source: https://ohdsi.github.io/EmpiricalCalibration/reference/fitNull.html Example showing how to filter negative controls from a dataset and fit the null distribution. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNull(negatives$logRr, negatives$seLogRr) null #> Estimated null distribution #> #> Estimate #> Mean 0.7922 #> SD 0.2834 ``` -------------------------------- ### Example usage of computeTraditionalP Source: https://ohdsi.github.io/EmpiricalCalibration/reference/computeTraditionalP.html An example demonstrating how to compute the p-value using the sccs dataset. ```R data(sccs) positive <- sccs[sccs$groundTruth == 1, ] computeTraditionalP(positive$logRr, positive$seLogRr) #> [1] 0 ``` -------------------------------- ### Example usage of plotCiCoverage Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotCiCoverage.html A basic example demonstrating how to simulate controls and generate a confidence interval coverage plot. ```R if (FALSE) { # \dontrun{ data <- simulateControls(n = 50 * 3, mean = 0.25, sd = 0.25, trueLogRr = log(c(1, 2, 4))) plotCiCoverage(data$logRr, data$seLogRr, data$trueLogRr) } # } ``` -------------------------------- ### Comparing EASE of two methods Source: https://ohdsi.github.io/EmpiricalCalibration/reference/compareEase.html Example demonstrating how to simulate two sets of negative control results and compare their EASE using compareEase. ```R # Simulate results of first method: ncs1 <- simulateControls(n = 50) # Simulate second method to be more biased: ncs2 <- ncs1 ncs2$logRr <- ncs2$logRr + rnorm(nrow(ncs2), mean = 0.1, sd = 0.1) delta <- compareEase( logRr1 = ncs1$logRr, seLogRr1 = ncs1$seLogRr, logRr2 = ncs2$logRr, seLogRr2 = ncs2$seLogRr ) delta #> delta ciLb ciUb p #> 1 -0.09135734 -0.1215314 -0.06221161 0 attr(delta, "ease1") #> ease ciLb ciUb #> 1 0.06262257 0.04643558 0.0785099 attr(delta, "ease2") #> ease ciLb ciUb #> 1 0.1539799 0.1208099 0.1869629 ``` -------------------------------- ### calibrateLlr example usage Source: https://ohdsi.github.io/EmpiricalCalibration/reference/calibrateLlr.html Example demonstrating how to fit a null distribution using negative controls and then calibrate the log likelihood ratio for a positive control. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNull(negatives$logRr, negatives$seLogRr) positive <- sccs[sccs$groundTruth == 1, ] calibrateLlr(null, positive) #> Detected data following normal distribution #> [1] 0 ``` -------------------------------- ### plotForest usage example Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotForest.html Example showing how to generate a forest plot using data from the sccs dataset. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] plotForest(negatives$logRr, negatives$seLogRr, negatives$drugName) ``` -------------------------------- ### fitSystematicErrorModel example usage Source: https://ohdsi.github.io/EmpiricalCalibration/reference/fitSystematicErrorModel.html Example showing how to simulate controls and fit a systematic error model. ```R controls <- simulateControls(n = 50 * 3, mean = 0.25, sd = 0.25, trueLogRr = log(c(1, 2, 4))) model <- fitSystematicErrorModel(controls$logRr, controls$seLogRr, controls$trueLogRr) model #> meanIntercept meanSlope sdIntercept sdSlope #> 0.24339542 1.01498772 0.18418950 0.04939429 #> attr(,"class") #> [1] "systematicErrorModel" ``` -------------------------------- ### fitMcmcNull usage example Source: https://ohdsi.github.io/EmpiricalCalibration/reference/fitMcmcNull.html Example demonstrating how to fit the null distribution using negative control data and calibrate p-values for positive controls. ```R if (FALSE) { # \dontrun{ data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitMcmcNull(negatives$logRr, negatives$seLogRr) null plotMcmcTrace(null) positive <- sccs[sccs$groundTruth == 1, ] calibrateP(null, positive$logRr, positive$seLogRr) } # } ``` -------------------------------- ### evaluateCiCalibration usage example Source: https://ohdsi.github.io/EmpiricalCalibration/reference/evaluateCiCalibration.html An example demonstrating how to simulate controls and evaluate confidence interval calibration using the evaluateCiCalibration function. ```R if (FALSE) { # \dontrun{ data <- simulateControls(n = 50 * 3, mean = 0.25, sd = 0.25, trueLogRr = log(c(1, 2, 4))) eval <- evaluateCiCalibration(data$logRr, data$seLogRr, data$trueLogRr) } # } ``` -------------------------------- ### Example usage of plotCiCalibration Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotCiCalibration.html An example demonstrating how to simulate control data and generate a confidence interval calibration plot. ```R if (FALSE) { # \dontrun{ data <- simulateControls(n = 50 * 3, mean = 0.25, sd = 0.25, trueLogRr = log(c(1, 2, 4))) plotCiCalibration(data$logRr, data$seLogRr, data$trueLogRr) } # } ``` -------------------------------- ### calibrateConfidenceInterval example usage Source: https://ohdsi.github.io/EmpiricalCalibration/reference/calibrateConfidenceInterval.html Example demonstrating the simulation of control data, fitting a systematic error model, and calibrating confidence intervals for new data. ```R data <- simulateControls(n = 50 * 3, mean = 0.25, sd = 0.25, trueLogRr = log(c(1, 2, 4))) model <- fitSystematicErrorModel(data$logRr, data$seLogRr, data$trueLogRr) newData <- simulateControls(n = 15, mean = 0.25, sd = 0.25, trueLogRr = log(c(1, 2, 4))) result <- calibrateConfidenceInterval(newData$logRr, newData$seLogRr, model) result #> logRr logLb95Rr logUb95Rr seLogRr #> 1 0.3633442 -0.2215560 0.9160445 0.2902095 #> 2 0.7721447 0.1459751 1.3463734 0.3062297 #> 3 1.1476296 0.5005040 1.7445525 0.3173651 #> 4 -0.0978154 -0.6475848 0.4608360 0.2827656 #> 5 1.1071717 0.4268373 1.7371203 0.3342620 #> 6 1.4525291 0.9184392 1.9378269 0.2600527 #> 7 -0.1621166 -0.7149772 0.4053362 0.2857995 #> 8 0.3809725 -0.2428391 0.9731956 0.3102186 #> 9 1.1543678 0.5655266 1.6930417 0.2876367 #> 10 0.1361149 -0.4309147 0.6909088 0.2861847 #> 11 0.6254329 -0.0715306 1.2756716 0.3436803 #> 12 1.3846512 0.7810774 1.9391390 0.2954293 #> 13 0.1129226 -0.4440026 0.6595497 0.2815236 #> 14 0.3980328 -0.2251682 0.9882119 0.3095414 #> 15 1.4982276 0.8546360 2.0932464 0.3159778 ``` -------------------------------- ### Perform Empirical Calibration Analysis Source: https://ohdsi.github.io/EmpiricalCalibration/index.html This example demonstrates loading a dataset, fitting a null distribution using negative controls, plotting the calibration effect, and computing a calibrated p-value. ```R data(sccs) #Load one of the included data sets negatives <- sccs[sccs$groundTruth == 0,] #Select the negative controls null <- fitNull(logRr = negatives$logRr, seLogRr = negatives$seLogRr) #Fit the null distribution positive <- sccs[sccs$groundTruth == 1,] #Select the positive control #Create the plot above: plotCalibrationEffect(logRrNegatives = negatives$logRr, seLogRrNegatives = negatives$seLogRr, logRrPositives = positive$logRr, seLogRrPositives = positive$seLogRr, null = null) #Compute the calibrated p-value: calibrateP(null = null, logRr = positive$logRr, seLogRr = positive$seLogRr) #Compute calibrated p-value [1] 0.8390598 ``` -------------------------------- ### Simulate MaxSPRT data example Source: https://ohdsi.github.io/EmpiricalCalibration/reference/simulateMaxSprtData.html An example of generating simulated data with 1000 subjects and displaying the first few rows of the resulting data frame. ```R data <- simulateMaxSprtData(n = 1000) head(data) #> time outcome exposure lookTime outcomeId #> 5 8.661996 0 TRUE 10 1 #> 11 9.386998 0 TRUE 10 1 #> 20 0.498716 0 FALSE 10 1 #> 22 7.061753 0 TRUE 10 1 #> 85 3.030036 0 TRUE 10 1 #> 98 2.959496 0 TRUE 10 1 ``` -------------------------------- ### Example usage of fitNullNonNormalLl Source: https://ohdsi.github.io/EmpiricalCalibration/reference/fitNullNonNormalLl.html Example demonstrating how to fit the null distribution using negative control data from the sccs dataset. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNullNonNormalLl(negatives) #> Detected data following normal distribution null #> Estimated null distribution #> #> Estimate #> Mean 0.7922 #> SD 0.2834 ``` -------------------------------- ### computeCvPoisson usage example Source: https://ohdsi.github.io/EmpiricalCalibration/reference/computeCvPoisson.html An example demonstrating how to call computeCvPoisson with a vector of group sizes and the resulting output including the selected alpha and critical value. ```R groupSizes <- rep(1, 10) computeCvPoisson(groupSizes) #> Selected alpha: 0.044 (least conservative value below 0.05) #> [1] 2.545177 #> attr(,"alpha") #> [1] 0.044113 ``` -------------------------------- ### computeCvPoissonRegression usage example Source: https://ohdsi.github.io/EmpiricalCalibration/reference/computeCvPoissonRegression.html An example demonstrating how to compute the critical value for a sequence of 10 groups with a control-to-risk time ratio of 4. ```R groupSizes <- rep(1, 10) computeCvPoissonRegression(groupSizes, z = 4) #> Selected alpha: 0.049 (least conservative value below 0.05) #> [1] 2.677723 #> attr(,"alpha") #> [1] 0.049444 ``` -------------------------------- ### Example usage of plotCalibration Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotCalibration.html A basic example demonstrating how to generate a calibration plot using negative control data from the sccs dataset. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] plotCalibration(negatives$logRr, negatives$seLogRr) ``` -------------------------------- ### Example usage of plotCiCalibrationEffect Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotCiCalibrationEffect.html An example demonstrating how to simulate control data and generate a CI calibration effect plot. ```R data <- simulateControls(n = 50 * 3, mean = 0.25, sd = 0.25, trueLogRr = log(c(1, 2, 4))) plotCiCalibrationEffect(data$logRr, data$seLogRr, data$trueLogRr) ``` -------------------------------- ### Example usage of plotErrorModel Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotErrorModel.html An example demonstrating how to simulate control data and visualize the systematic error model using plotErrorModel. ```R data <- simulateControls(n = 50 * 3, mean = 0.25, sd = 0.25, trueLogRr = log(c(1, 2, 4))) plotErrorModel(data$logRr, data$seLogRr, data$trueLogRr) ``` -------------------------------- ### plotTrueAndObserved example usage Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotTrueAndObserved.html An example demonstrating how to simulate control data and plot the true versus observed effect sizes. ```R data <- simulateControls(n = 50 * 3, mean = 0.25, sd = 0.25, trueLogRr = log(c(1, 2, 4))) plotTrueAndObserved(data$logRr, data$seLogRr, data$trueLogRr) ``` -------------------------------- ### Example usage with sccs data Source: https://ohdsi.github.io/EmpiricalCalibration/reference/computeExpectedAbsoluteSystematicError.html Shows how to fit a null distribution using the sccs dataset and compute the expected absolute systematic error. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNull(negatives$logRr, negatives$seLogRr) computeExpectedAbsoluteSystematicError(null) #> [1] 0.792597 ``` -------------------------------- ### computeCvBinomial usage example Source: https://ohdsi.github.io/EmpiricalCalibration/reference/computeCvBinomial.html An example demonstrating how to call computeCvBinomial with a vector of group sizes and a specified z value, including the expected output. ```R groupSizes <- rep(1, 10) computeCvBinomial(groupSizes, z = 4) #> Selected alpha: 0.049 (least conservative value below 0.05) #> [1] 3.064954 #> attr(,"alpha") #> [1] 0.049321 ``` -------------------------------- ### Example of calibrating confidence intervals Source: https://ohdsi.github.io/EmpiricalCalibration/reference/convertNullToErrorModel.html A complete workflow demonstrating fitting a null distribution, converting it to a systematic error model, and using that model to calibrate confidence intervals for positive controls. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNull(negatives$logRr, negatives$seLogRr) model <- convertNullToErrorModel(null) positive <- sccs[sccs$groundTruth == 1, ] calibrateConfidenceInterval(positive$logRr, positive$seLogRr, model) #> logRr logLb95Rr logUb95Rr seLogRr #> 1 -0.05953451 -0.6335409 0.5144719 0.2928658 ``` -------------------------------- ### Example usage of plotExpectedType1Error Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotExpectedType1Error.html A basic example demonstrating how to prepare negative and positive control data from the sccs dataset and generate the expected type 1 error plot. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] positive <- sccs[sccs$groundTruth == 1, ] plotExpectedType1Error(negatives$logRr, negatives$seLogRr, positive$seLogRr) ``` -------------------------------- ### compareEase(logRr1, seLogRr1, logRr2, seLogRr2, alpha = 0.05, sampleSize = 1000) Source: https://ohdsi.github.io/EmpiricalCalibration/reference/compareEase.html Compares the expected absolute systematic error (EASE) of two sets of effect estimates for the same set of negative controls. Returns a data frame with the difference in EASE, confidence interval, and p-value, along with attributes containing EASE estimates for each set. ```APIDOC ## compareEase(logRr1, seLogRr1, logRr2, seLogRr2, alpha = 0.05, sampleSize = 1000) ### Description Compares the expected absolute systematic error (EASE) of two sets of estimates for the same set of negative controls. The two sets of estimates must be in identical order. ### Usage ```r compareEase( logRr1, seLogRr1, logRr2, seLogRr2, alpha = 0.05, sampleSize = 1000 ) ``` ### Arguments - **logRr1** (numeric vector) - Required - A numeric vector of effect estimates generated using the first method on the log scale. - **seLogRr1** (numeric vector) - Required - The standard error of the log of the effect estimates generated using the first method. - **logRr2** (numeric vector) - Required - A numeric vector of effect estimates generated using the second method on the log scale. - **seLogRr2** (numeric vector) - Required - The standard error of the log of the effect estimates generated using the second method. - **alpha** (numeric) - Optional - The expected type I error for computing confidence intervals and p-values. Default is 0.05. - **sampleSize** (integer) - Optional - The number of samples in the bootstraps. Default is 1000. ### Value A data frame with 4 columns: the point estimate, confidence interval lower bound, and upper bound for the difference between EASE in the two sets of negative controls, and a p value against the null hypothesis that the EASE is the same for the two sets. The data frame has two attributes: ease1 and ease2, providing the EASE estimates (and confidence intervals) for the two sets, computed using bootstrapping. ### Details Compare the expected absolute systematic error (EASE) of two sets of estimates for the same set of negative controls. Important: the two sets of estimates (logRr1 + seLogRr1 and logRr2 + seLogRr2) should be in identical order, so that for example the first item in each vector corresponds to the same negative control. ### Example ```r # Simulate results of first method: ncs1 <- simulateControls(n = 50) # Simulate second method to be more biased: ncs2 <- ncs1 ncs2$logRr <- ncs2$logRr + rnorm(nrow(ncs2), mean = 0.1, sd = 0.1) delta <- compareEase( logRr1 = ncs1$logRr, seLogRr1 = ncs1$seLogRr, logRr2 = ncs2$logRr, seLogRr2 = ncs2$seLogRr ) delta #> delta ciLb ciUb p #> 1 -0.09135734 -0.1215314 -0.06221161 0 attr(delta, "ease1") #> ease ciLb ciUb #> 1 0.06262257 0.04643558 0.0785099 attr(delta, "ease2") #> ease ciLb ciUb #> 1 0.1539799 0.1208099 0.1869629 ``` ``` -------------------------------- ### compareEase() Source: https://ohdsi.github.io/EmpiricalCalibration/reference/index.html Compares the EASE (Expected Absolute Systematic Error) of correlated sets of estimates. ```APIDOC ## compareEase() ### Description Compares EASE of correlated sets of estimates. ### Usage compareEase(logRr1, seLogRr1, logRr2, seLogRr2) ### Arguments - **logRr1** (numeric) - Required - Log effect estimates for the first set. - **seLogRr1** (numeric) - Required - Standard errors for the first set. - **logRr2** (numeric) - Required - Log effect estimates for the second set. - **seLogRr2** (numeric) - Required - Standard errors for the second set. ### Value A data frame with the EASE comparison. ``` -------------------------------- ### Example: plotCalibrationEffect with sccs data Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotCalibrationEffect.html Example usage of plotCalibrationEffect with negative and positive controls from the sccs dataset. The output includes a warning about a removed row containing missing values. ```R data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] positive <- sccs[sccs$groundTruth == 1, ] plotCalibrationEffect(negatives$logRr, negatives$seLogRr, positive$logRr, positive$seLogRr) #> Warning: Removed 1 row containing missing values or values outside the scale range #> (`geom_vline()`). ``` -------------------------------- ### Basic usage of computeExpectedAbsoluteSystematicError Source: https://ohdsi.github.io/EmpiricalCalibration/reference/computeExpectedAbsoluteSystematicError.html Demonstrates the basic syntax for calling the function with a null object and an optional alpha parameter. ```R computeExpectedAbsoluteSystematicError(null, alpha = 0.05) ``` -------------------------------- ### Load and inspect SCCS study data Source: https://ohdsi.github.io/EmpiricalCalibration/articles/EmpiricalPCalibrationVignette.html Loads the example SCCS study results from the EmpiricalCalibration package and filters for the drug of interest. ```R data(sccs) drugOfInterest <- sccs[sccs$groundTruth == 1, ] drugOfInterest ``` -------------------------------- ### Load cohortMethod dataset Source: https://ohdsi.github.io/EmpiricalCalibration/reference/cohortMethod.html Load the cohortMethod dataset into the R environment. ```R data(cohortMethod) ``` -------------------------------- ### simulateControls() Source: https://ohdsi.github.io/EmpiricalCalibration/reference/index.html Simulates (negative) controls for testing and demonstration purposes. ```APIDOC ## simulateControls() ### Description Simulates (negative) controls. ### Usage simulateControls(n = 100) ### Arguments - **n** (numeric) - Optional - The number of controls to simulate. Default is 100. ### Value A data frame with simulated controls. ``` -------------------------------- ### Retrieve package citation Source: https://ohdsi.github.io/EmpiricalCalibration/articles/EmpiricalCiCalibrationVignette.html Displays the citation information for the EmpiricalCalibration package. ```R citation("EmpiricalCalibration") ``` ```R ## To cite EmpiricalCalibration in publications use: ## ## Schuemie MJ, Ryan PB, DuMouchel W, Suchard MA, Madigan D (2013). ## "Interpreting observational studies: why empirical calibration is ## needed to correct p-values." _Statistics in Medicine_, *33*(2), ## 209-218. . ## ## Schuemie MJ, Hripcsak G, Ryan PB, Madigan D, Suchard MA (2018). ## "Empirical confidence interval calibration for population-level ## effect estimation studies in observational healthcare data." _Proc. ## Natl. Acad. Sci. U.S.A._, *115*(11), 2571-2577. ## . ## ## To see these entries in BibTeX format, use 'print(, ## bibtex=TRUE)', 'toBibtex(.)', or set ## 'options(citation.bibtex.max=999)'. ``` -------------------------------- ### Load caseControl dataset Source: https://ohdsi.github.io/EmpiricalCalibration/reference/caseControl.html Loads the caseControl dataset into the R environment. ```R data(caseControl) ``` -------------------------------- ### simulateMaxSprtData() Source: https://ohdsi.github.io/EmpiricalCalibration/reference/index.html Simulates survival data for MaxSPRT computation. ```APIDOC ## simulateMaxSprtData() ### Description Simulates survival data for MaxSPRT computation. ### Usage simulateMaxSprtData(n = 100) ### Arguments - **n** (numeric) - Optional - The number of observations to simulate. Default is 100. ### Value A data frame with simulated survival data. ``` -------------------------------- ### calibrateP function signatures Source: https://ohdsi.github.io/EmpiricalCalibration/reference/calibrateP.html Function signatures for calibrateP, including S3 methods for null and mcmcNull classes. ```R calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, ...) # S3 method for class 'null' calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, ...) # S3 method for class 'mcmcNull' calibrateP( null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, pValueOnly, ... ) ``` -------------------------------- ### fitNull(logRr, seLogRr) Source: https://ohdsi.github.io/EmpiricalCalibration/reference/fitNull.html Fits the null distribution to a set of negative controls. Returns an object containing the parameters of the null distribution. ```APIDOC ## fitNull(logRr, seLogRr) ### Description Fits a Gaussian function to the negative control estimates as described in Schuemie et al (2014). Returns an object containing the parameters of the null distribution. ### Usage fitNull(logRr, seLogRr) ### Arguments - **logRr** (numeric vector) - Required - A numeric vector of effect estimates on the log scale. - **seLogRr** (numeric vector) - Required - The standard error of the log of the effect estimates. Hint: often the standard error = (log() - log())/qnorm(0.025). ### Value An object containing the parameters of the null distribution. ### Details This function fits a Gaussian function to the negative control estimates as described in Schuemie et al (2014). ### References Schuemie MJ, Ryan PB, Dumouchel W, Suchard MA, Madigan D. Interpreting observational studies: why empirical calibration is needed to correct p-values. Statistics in Medicine 33(2):209-18,2014 ### Examples ```r data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNull(negatives$logRr, negatives$seLogRr) null #> Estimated null distribution #> #> Estimate #> Mean 0.7922 #> SD 0.2834 ``` ``` -------------------------------- ### calibrateLlr() Source: https://ohdsi.github.io/EmpiricalCalibration/reference/index.html Calibrates the log likelihood ratio (LLR) for MaxSPRT-based sequential testing. ```APIDOC ## calibrateLlr() ### Description Calibrates the log likelihood ratio. ### Usage calibrateLlr(llr, seLogRr, model) ### Arguments - **llr** (numeric) - Required - The log likelihood ratio. - **seLogRr** (numeric) - Required - The standard error of the log effect estimate. - **model** (systematicErrorModel) - Required - An object of class 'systematicErrorModel'. ### Value The calibrated log likelihood ratio. ``` -------------------------------- ### Function signature for computeTraditionalCi Source: https://ohdsi.github.io/EmpiricalCalibration/reference/computeTraditionalCi.html The basic syntax for calling the computeTraditionalCi function. ```R computeTraditionalCi(logRr, seLogRr, ciWidth = 0.95) ``` -------------------------------- ### calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, ...) Source: https://ohdsi.github.io/EmpiricalCalibration/reference/calibrateP.html Computes calibrated p-values using the fitted null distribution. This is the main function signature for the S3 generic. ```APIDOC ## calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, ...) ### Description Computes calibrated p-values using the fitted null distribution. ### Usage ```r calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, ...) ``` ### Arguments - **null** - An object of class `null` created using the `fitNull` function or an object of class `mcmcNull` created using the `fitMcmcNull` function. - **logRr** - A numeric vector of one or more effect estimates on the log scale. - **seLogRr** - The standard error of the log of the effect estimates. Hint: often the standard error = (log() - log())/qnorm(0.025). - **twoSided** - Compute two-sided (TRUE) or one-sided (FALSE) p-value? Default is TRUE. - **upper** - If one-sided: compute p-value for upper (TRUE) or lower (FALSE) bound? Default is TRUE. - **...** - Any additional parameters (currently none). ### Value The calibrated p-value. ### Details This function computes a calibrated two-sided p-value as described in Schuemie et al (2014). ### Methods (by class) - `calibrateP(null)`: Computes the calibrated P-value using asymptotic assumptions. - `calibrateP(mcmcNull)`: Computes the calibrated P-value and 95 percent credible interval using Markov Chain Monte Carlo (MCMC). ### References Schuemie MJ, Ryan PB, Dumouchel W, Suchard MA, Madigan D. Interpreting observational studies: why empirical calibration is needed to correct p-values. Statistics in Medicine 33(2):209-18,2014 ### Examples ```r data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNull(negatives$logRr, negatives$seLogRr) positive <- sccs[sccs$groundTruth == 1, ] calibrateP(null, positive$logRr, positive$seLogRr) #> [1] 0.8389142 ``` ``` -------------------------------- ### calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, ...) S3 method for class 'null' Source: https://ohdsi.github.io/EmpiricalCalibration/reference/calibrateP.html S3 method for class 'null' that computes the calibrated P-value using asymptotic assumptions. ```APIDOC ## calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, ...) S3 method for class 'null' ### Description Computes the calibrated P-value using asymptotic assumptions. ### Usage ```r calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, ...) ``` ### Arguments - **null** - An object of class `null` created using the `fitNull` function. - **logRr** - A numeric vector of one or more effect estimates on the log scale. - **seLogRr** - The standard error of the log of the effect estimates. Hint: often the standard error = (log() - log())/qnorm(0.025). - **twoSided** - Compute two-sided (TRUE) or one-sided (FALSE) p-value? Default is TRUE. - **upper** - If one-sided: compute p-value for upper (TRUE) or lower (FALSE) bound? Default is TRUE. - **...** - Any additional parameters (currently none). ### Value The calibrated p-value. ### Details This method computes the calibrated P-value using asymptotic assumptions. ### References Schuemie MJ, Ryan PB, Dumouchel W, Suchard MA, Madigan D. Interpreting observational studies: why empirical calibration is needed to correct p-values. Statistics in Medicine 33(2):209-18,2014 ### Examples ```r data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNull(negatives$logRr, negatives$seLogRr) positive <- sccs[sccs$groundTruth == 1, ] calibrateP(null, positive$logRr, positive$seLogRr) #> [1] 0.8389142 ``` ``` -------------------------------- ### fitMcmcNull() Source: https://ohdsi.github.io/EmpiricalCalibration/reference/index.html Fits the null distribution using Markov Chain Monte Carlo (MCMC) methods. This is an alternative to fitNull() that may be more robust in certain scenarios. ```APIDOC ## fitMcmcNull() ### Description Fits the null distribution using MCMC. ### Usage fitMcmcNull(logRr, seLogRr) ### Arguments - **logRr** (numeric) - Required - A numeric vector of log effect estimates. - **seLogRr** (numeric) - Required - The standard error of the log effect estimates. ### Value An object of class 'mcmcNull' containing the MCMC samples and fitted null distribution. ``` -------------------------------- ### plotCalibration() Source: https://ohdsi.github.io/EmpiricalCalibration/reference/index.html Creates a calibration plot showing the relationship between traditional and calibrated p-values. ```APIDOC ## plotCalibration() ### Description Creates a calibration plot. ### Usage plotCalibration(null, logRr, seLogRr) ### Arguments - **null** (null) - Required - An object of class 'null'. - **logRr** (numeric) - Required - A numeric vector of log effect estimates. - **seLogRr** (numeric) - Required - The standard error of the log effect estimates. ### Value A ggplot object. ``` -------------------------------- ### plotForest(logRr, seLogRr, names, xLabel, title, fileName) Source: https://ohdsi.github.io/EmpiricalCalibration/reference/plotForest.html Creates a forest plot of effect size estimates. Estimates significantly different from 1 (alpha = 0.05) are marked in orange, others in blue. ```APIDOC ## plotForest(logRr, seLogRr, names, xLabel, title, fileName) ### Description Creates a forest plot of effect size estimates (ratios). Estimates that are significantly different from 1 (alpha = 0.05) are marked in orange, others are marked in blue. ### Usage ```r plotForest( logRr, seLogRr, names, xLabel = "Relative risk", title, fileName = NULL ) ``` ### Arguments - **logRr** (numeric vector) - Required - A numeric vector of effect estimates on the log scale. - **seLogRr** (numeric vector) - Required - The standard error of the log of the effect estimates. Hint: often the standard error = (log() - log())/qnorm(0.025). - **names** (character vector) - Required - A vector containing the names of the drugs or outcomes. - **xLabel** (character) - Optional - The label on the x-axis: the name of the effect estimate. Default is "Relative risk". - **title** (character) - Optional - The main title for the plot. - **fileName** (character) - Optional - Name of the file where the plot should be saved, for example 'plot.png'. See the function `ggsave` in the ggplot2 package for supported file formats. ### Value A Ggplot object. Use the `ggsave` function to save to file. ### Examples ```r data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] plotForest(negatives$logRr, negatives$seLogRr, negatives$drugName) ``` ``` -------------------------------- ### plotCiCalibration() Source: https://ohdsi.github.io/EmpiricalCalibration/reference/index.html Creates a confidence interval calibration plot. ```APIDOC ## plotCiCalibration() ### Description Creates a confidence interval calibration plot. ### Usage plotCiCalibration(logRr, seLogRr, trueLogRr) ### Arguments - **logRr** (numeric) - Required - A numeric vector of log effect estimates. - **seLogRr** (numeric) - Required - The standard error of the log effect estimates. - **trueLogRr** (numeric) - Required - The true log effect estimates. ### Value A ggplot object. ``` -------------------------------- ### calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, pValueOnly, ...) S3 method for class 'mcmcNull' Source: https://ohdsi.github.io/EmpiricalCalibration/reference/calibrateP.html S3 method for class 'mcmcNull' that computes the calibrated P-value and 95 percent credible interval using Markov Chain Monte Carlo (MCMC). ```APIDOC ## calibrateP(null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, pValueOnly, ...) S3 method for class 'mcmcNull' ### Description Computes the calibrated P-value and 95 percent credible interval using Markov Chain Monte Carlo (MCMC). ### Usage ```r calibrateP( null, logRr, seLogRr, twoSided = TRUE, upper = TRUE, pValueOnly, ... ) ``` ### Arguments - **null** - An object of class `mcmcNull` created using the `fitMcmcNull` function. - **logRr** - A numeric vector of one or more effect estimates on the log scale. - **seLogRr** - The standard error of the log of the effect estimates. Hint: often the standard error = (log() - log())/qnorm(0.025). - **twoSided** - Compute two-sided (TRUE) or one-sided (FALSE) p-value? Default is TRUE. - **upper** - If one-sided: compute p-value for upper (TRUE) or lower (FALSE) bound? Default is TRUE. - **pValueOnly** - If true, will return only the calibrated P-value itself, not the credible interval. - **...** - Any additional parameters (currently none). ### Value The calibrated p-value. ### Details This method computes the calibrated P-value and 95 percent credible interval using Markov Chain Monte Carlo (MCMC). ### References Schuemie MJ, Ryan PB, Dumouchel W, Suchard MA, Madigan D. Interpreting observational studies: why empirical calibration is needed to correct p-values. Statistics in Medicine 33(2):209-18,2014 ``` -------------------------------- ### fitMcmcNull(logRr, seLogRr, iter = 1e+05) Source: https://ohdsi.github.io/EmpiricalCalibration/reference/fitMcmcNull.html Fits the null distribution to a set of negative controls using Markov Chain Monte Carlo (MCMC). This is an experimental function for computing the 95 percent credible interval of a calibrated p-value. ```APIDOC ## fitMcmcNull(logRr, seLogRr, iter = 1e+05) ### Description Fits the null distribution to a set of negative controls using Markov Chain Monte Carlo (MCMC). This is an experimental function for computing the 95 percent credible interval of a calibrated p-value. ### Usage fitMcmcNull(logRr, seLogRr, iter = 1e+05) ### Arguments - **logRr** (numeric vector) - Required - A numeric vector of effect estimates on the log scale. - **seLogRr** (numeric vector) - Required - The standard error of the log of the effect estimates. Hint: often the standard error = (log() - log())/qnorm(0.025). - **iter** (numeric) - Optional - Number of iterations of the MCMC. Default is 1e+05. ### Value An object of type `mcmcNull` containing the mean and standard deviation (both on the log scale) of the null distribution, as well as the MCMC trace. ### Details This is an experimental function for computing the 95 percent credible interval of a calibrated p-value using Markov-Chain Monte Carlo (MCMC). ### Examples ```r if (FALSE) { # \dontrun{ data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitMcmcNull(negatives$logRr, negatives$seLogRr) null plotMcmcTrace(null) positive <- sccs[sccs$groundTruth == 1, ] calibrateP(null, positive$logRr, positive$seLogRr) } # } ``` ``` -------------------------------- ### calibrateLlr(null, likelihoodApproximation, twoSided = FALSE, upper = TRUE) Source: https://ohdsi.github.io/EmpiricalCalibration/reference/calibrateLlr.html Computes the calibrated log likelihood ratio using the fitted null distribution. The function accepts a null object and likelihood approximations, with options for one- or two-sided p-values. ```APIDOC ## calibrateLlr(null, likelihoodApproximation, twoSided = FALSE, upper = TRUE) ### Description Computes calibrated log likelihood ratio using the fitted null distribution. ### Usage calibrateLlr(null, likelihoodApproximation, twoSided = FALSE, upper = TRUE) ### Arguments - **null** - An object of class `null` created using the `fitNull` function or an object of class `mcmcNull` created using the `fitMcmcNull` function. - **likelihoodApproximation** - Either a data frame containing normal, skew-normal, or custom parametric likelihood approximations, or a list of (adaptive) grid likelihood profiles. - **twoSided** - Compute two-sided (TRUE) or one-sided (FALSE) p-value? - **upper** - If one-sided: compute p-value for upper (TRUE) or lower (FALSE) bound? ### Value The calibrated log likelihood ratio. ### Examples ```r data(sccs) negatives <- sccs[sccs$groundTruth == 0, ] null <- fitNull(negatives$logRr, negatives$seLogRr) positive <- sccs[sccs$groundTruth == 1, ] calibrateLlr(null, positive) #> Detected data following normal distribution #> [1] 0 ``` ``` -------------------------------- ### Plot MCMC Trace Source: https://ohdsi.github.io/EmpiricalCalibration/articles/EmpiricalPCalibrationVignette.html Visualizes the MCMC trace to ensure the null distribution estimation is reliable. The trace should resemble random noise without significant auto-correlation. ```R plotMcmcTrace(null) ```