bestNormalize

https://petersonr.github.io/bestNormalize/
Admin
bestNormalize is an R package that finds the optimal normalizing transformation for a vector of data...
Tokens:26,394
Snippets:266
Trust Score:9.7
Update:5 months ago
Show doc for...
Tokens:
Raw
### Example: Apply and bake orderNorm transformation

Source: https://petersonr.github.io/bestNormalize/reference/step_orderNorm.html

Demonstrates how to create a recipe, apply the `step_orderNorm` transformation to numeric variables, prepare the recipe with training data, and then bake the transformed data. This example also includes plotting the density before and after transformation.

```R
library(recipes)
rec <- recipe(~ ., data = as.data.frame(iris))

orq_trans <- step_orderNorm(rec, all_numeric())

orq_estimates <- prep(orq_trans, training = as.data.frame(iris))

orq_data <- bake(orq_estimates, as.data.frame(iris))

plot(density(iris[, "Petal.Length"]), main = "before")

plot(density(orq_data$Petal.Length), main = "after")
```

--------------------------------

### Install bestNormalize from GitHub

Source: https://petersonr.github.io/bestNormalize/index.html

Install the development version of the bestNormalize package from GitHub using the devtools package.

```r
# install.packages("devtools")
devtools::install_github("petersonR/bestNormalize")
```

--------------------------------

### Example Usage of step_best_normalize

Source: https://petersonr.github.io/bestNormalize/reference/step_best_normalize.html

Demonstrates how to use step_best_normalize to apply normalization to numeric variables in a recipe, including preparation and baking the data.

```R
library(recipes)
#> Loading required package: dplyr
#> 
#> Attaching package: ‘dplyr’
#> The following objects are masked from ‘package:stats’:
#> 
#>     filter, lag
#> The following objects are masked from ‘package:base’:
#> 
#>     intersect, setdiff, setequal, union
#> 
#> Attaching package: ‘recipes’
#> The following object is masked from ‘package:stats’:
#> 
#>     step
rec <- recipe(~ ., data = as.data.frame(iris))

bn_trans <- step_best_normalize(rec, all_numeric())

bn_estimates <- prep(bn_trans, training = as.data.frame(iris))

bn_data <- bake(bn_estimates, as.data.frame(iris))

plot(density(iris[, "Petal.Length"])), main = "before")

plot(density(bn_data$Petal.Length), main = "after")
```

--------------------------------

### Example Usage

Source: https://petersonr.github.io/bestNormalize/reference/lambert.html

Demonstrates how to use the lambert function and its predict method for transformation and inverse transformation.

```APIDOC
## Example
```R
if (FALSE) {
x <- rgamma(100, 1, 1)

lambert_obj <- lambert(x)
lambert_obj
p <- predict(lambert_obj)
x2 <- predict(lambert_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
}
```
```

--------------------------------

### Example Usage

Source: https://petersonr.github.io/bestNormalize/reference/double_reverse_log.html

Demonstrates how to use the double_reverse_log function and its predict method for transformation and inverse transformation.

```APIDOC
## Example

```R
x <- rgamma(100, 1, 1)

double_reverse_log_obj <- double_reverse_log(x)
double_reverse_log_obj
#> Standardized double reversed Log_b(x + a) Transformation with 100 nonmissing obs.:
#>  Relevant statistics:
#>  - a = 
#>  - b = 10 
#>  - max(x) = 4.303169 ; min(x) = -0.3902701 
#>  - mean (before standardization) = 0.2146823 
#>  - sd (before standardization) = 0.199244 
p <- predict(double_reverse_log_obj)
x2 <- predict(double_reverse_log_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```
```

--------------------------------

### Install bestNormalize from CRAN

Source: https://petersonr.github.io/bestNormalize/index.html

Install the bestNormalize package from CRAN.

```r
install.packages("bestNormalize")
```

--------------------------------

### Example: Applying and Inverting Identity Transformation

Source: https://petersonr.github.io/bestNormalize/reference/no_transform.html

This example demonstrates creating an identity transformation object using `no_transform`, applying it to new data with `predict`, and then inverting the transformation to verify the original data is recovered.

```R
x <- rgamma(100, 1, 1)

no_transform_obj <- no_transform(x)
no_transform_obj
#> I(x) Transformation with 100 nonmissing obs.
p <- predict(no_transform_obj)
x2 <- predict(no_transform_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Example Usage of sqrt_x and predict

Source: https://petersonr.github.io/bestNormalize/reference/sqrt_x.html

Demonstrates how to use sqrt_x to transform data and then use the predict method for both forward and inverse transformations. Verifies the inverse transformation.

```R
x <- rgamma(100, 1, 1)

sqrt_x_obj <- sqrt_x(x)
sqrt_x_obj
#> Standardized sqrt(x + a) Transformation with 100 nonmissing obs.:
#>  Relevant statistics:
#>  - a = 0 
#>  - mean (before standardization) = 0.9442748 
#>  - sd (before standardization) = 0.451784 
p <- predict(sqrt_x_obj)
x2 <- predict(sqrt_x_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Example: Applying and Inverting Log_x Transformation

Source: https://petersonr.github.io/bestNormalize/reference/log_x.html

Demonstrates how to use log_x to transform data and then use predict to invert the transformation. Verifies the inverse transformation matches the original data.

```R
x <- rgamma(100, 1, 1)

log_x_obj <- log_x(x)
log_x_obj
#> Standardized Log_b(x + a) Transformation with 100 nonmissing obs.:
#>  Relevant statistics:
#>  - a = 0 
#>  - b = 10 
#>  - mean (before standardization) = -0.2010094 
#>  - sd (before standardization) = 0.4766709 
p <- predict(log_x_obj)
x2 <- predict(log_x_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Example of exp_x Transformation and Prediction

Source: https://petersonr.github.io/bestNormalize/reference/exp_x.html

Demonstrates how to apply the exp_x transformation, predict transformed values, and perform an inverse transformation. Ensure the original data is available for comparison.

```R
x <- rgamma(100, 1, 1)

exp_x_obj <- exp_x(x)
exp_x_obj
#> Standardized exp(x) Transformation with 100 nonmissing obs.:
#>  Relevant statistics:
#>  - mean (before standardization) = 10.70445 
#>  - sd (before standardization) = 41.39896 
p <- predict(exp_x_obj)
x2 <- predict(exp_x_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Box-Cox Transformation Example

Source: https://petersonr.github.io/bestNormalize/reference/boxcox.html

Demonstrates how to apply the Box-Cox transformation to a gamma-distributed dataset, predict transformed values, and perform the inverse transformation to verify.

```R
x <- rgamma(100, 1, 1)

bc_obj <- boxcox(x)
bc_obj
#> Standardized Box Cox Transformation with 100 nonmissing obs.:
#>  Estimated statistics:
#>  - lambda = 0.2240439 
#>  - mean (before standardization) = -0.2826479 
#>  - sd (before standardization) = 0.9754222 
p <- predict(bc_obj)
x2 <- predict(bc_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Binarize Transformation Example

Source: https://petersonr.github.io/bestNormalize/reference/binarize.html

Demonstrates how to use the binarize function and its predict method for both forward and inverse transformations. The inverse transformation returns a factor indicating where the original data was cut.

```R
x <- rgamma(100, 1, 1)
binarize_obj <- binarize(x)
(p <- predict(binarize_obj))
```

```R
predict(binarize_obj, newdata = p, inverse = TRUE)
```

--------------------------------

### Example of Double Reverse Log Transformation

Source: https://petersonr.github.io/bestNormalize/reference/double_reverse_log.html

Demonstrates how to use the double_reverse_log function, predict, and inverse predict. Ensures the inverse transformation returns the original data.

```R
x <- rgamma(100, 1, 1)

double_reverse_log_obj <- double_reverse_log(x)
double_reverse_log_obj
#> Standardized double reversed Log_b(x + a) Transformation with 100 nonmissing obs.:
#>  Relevant statistics:
#>  - a = 
#>  - b = 10 
#>  - max(x) = 4.303169 ; min(x) = -0.3902701 
#>  - mean (before standardization) = 0.2146823 
#>  - sd (before standardization) = 0.199244 
p <- predict(double_reverse_log_obj)
x2 <- predict(double_reverse_log_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Example: Yeo-Johnson Transformation and Prediction

Source: https://petersonr.github.io/bestNormalize/reference/yeojohnson.html

Demonstrates how to apply the Yeo-Johnson transformation to a gamma-distributed vector, predict transformed values, and then reverse the transformation to verify the original data.

```R
x <- rgamma(100, 1, 1)

yeojohnson_obj <- yeojohnson(x)
yeojohnson_obj
#> Standardized Yeo-Johnson Transformation with 100 nonmissing obs.:
#>  Estimated statistics:
#>  - lambda = -0.6285334 
#>  - mean (before standardization) = 0.4860509 
#>  - sd (before standardization) = 0.2433941 
p <- predict(yeojohnson_obj)
x2 <- predict(yeojohnson_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Example of Lambert W x F Transformation and Inverse

Source: https://petersonr.github.io/bestNormalize/reference/lambert.html

Demonstrates how to apply the Lambert W x F transformation to sample data, predict the transformed values, and then perform an inverse transformation to verify the process.

```R
if (FALSE) {
x <- rgamma(100, 1, 1)

lambert_obj <- lambert(x)
lambert_obj
p <- predict(lambert_obj)
x2 <- predict(lambert_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
}
```

--------------------------------

### Example of arcsinh_x Transformation and Prediction

Source: https://petersonr.github.io/bestNormalize/reference/arcsinh_x.html

Demonstrates how to apply the arcsinh_x transformation, predict transformed values, and perform an inverse transformation. Ensures the inverse transformation recovers the original data.

```R
x <- rgamma(100, 1, 1)

arcsinh_x_obj <- arcsinh_x(x)
arcsinh_x_obj
#> Standardized asinh(x) Transformation with 100 nonmissing obs.:
#>  Relevant statistics:
#>  - mean (before standardization) = 0.7198804 
#>  - sd (before standardization) = 0.5443389 
p <- predict(arcsinh_x_obj)
x2 <- predict(arcsinh_x_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Generate Sample Data for Normalization

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Generates a gamma-distributed random variable for demonstrating normalization techniques. Ensure the `MASS` package is installed for `truehist`.

```r
set.seed(100)
x <- rgamma(250, 1, 1)
MASS::truehist(x, nbins = 12)
```

--------------------------------

### Example Usage of orderNorm and predict

Source: https://petersonr.github.io/bestNormalize/reference/orderNorm.html

Demonstrates how to use orderNorm to transform data and predict to apply the transformation to new data and then invert it. Verifies that the inverted transformation returns the original data.

```R
x <- rgamma(100, 1, 1)

orderNorm_obj <- orderNorm(x)
orderNorm_obj
#> orderNorm Transformation with 100 nonmissing obs and no ties 
#>  - Original quantiles:
#>    0%   25%   50%   75%  100% 
#> 0.001 0.313 0.808 1.424 5.744 
p <- predict(orderNorm_obj)
x2 <- predict(orderNorm_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Apply bestNormalize to Autotrader Price Data

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Applies the bestNormalize function to the 'price' column of the autotrader dataset. This example demonstrates normalization for a real-world dataset and may produce warnings related to ties.

```R
data("autotrader")
autotrader$yearsold <- 2017 - autotrader$Year
### Using best-normalize
(priceBN <- bestNormalize(autotrader$price, r = 1, k = 5))
```

--------------------------------

### Apply bestNormalize to Autotrader Mileage Data

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Applies the bestNormalize function to the 'mileage' column of the autotrader dataset. This example shows the effectiveness of orderNorm for mileage data, even with ties.

```R
(mileageBN <- bestNormalize(autotrader$mileage, r = 1, k = 5))
```

--------------------------------

### Estimate Best Normalizing Transformation Without CV

Source: https://petersonr.github.io/bestNormalize/reference/bestNormalize.html

Estimates the optimal normalizing transformation without using cross-validation, which is faster for in-sample estimation. The `predict` function can be used to apply the transformation and invert it. This example demonstrates overriding `allow_orderNorm` and `out_of_sample`.

```r
x <- rgamma(100, 1, 1)

# Without CV
BN_obj <- bestNormalize(x, allow_orderNorm = FALSE, out_of_sample = FALSE)
BN_obj
#> Best Normalizing transformation with 100 Observations
#>  Estimated Normality Statistics (Pearson P / df, lower => more normal):
#>  - arcsinh(x): 3.624
#>  - Box-Cox: 0.712
#>  - Center+scale: 7.68
#>  - Double Reversed Log_b(x+a): 15.376
#>  - Exp(x): 34.33
#>  - Log_b(x+a): 0.582
#>  - sqrt(x + a): 1.544
#>  - Yeo-Johnson: 1.544
#> Estimation method: In-sample
#>  
#> Based off these, bestNormalize chose:
#> Standardized Log_b(x + a) Transformation with 100 nonmissing obs.:
#>  Relevant statistics:
#>  - a = 0 
#>  - b = 10 
#>  - mean (before standardization) = -0.2058717 
#>  - sd (before standardization) = 0.5173331 
#>  
p <- predict(BN_obj)
x2 <- predict(BN_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
#> [1] TRUE
```

--------------------------------

### Tidy Output of Prepared step_best_normalize

Source: https://petersonr.github.io/bestNormalize/reference/step_best_normalize.html

Displays the tidy output after the step_best_normalize has been prepared, showing the chosen transformation for each variable.

```R
tidy(bn_estimates, number = 1)
#> # A tibble: 4 × 4
#>   terms        chosen_transform cv_info          id                  
#>   <chr>        <chr>            <named list>     <chr>               
#> 1 Sepal.Length sqrt_x           <tibble [9 × 4]> best_normalize_Wm728
#> 2 Sepal.Width  center_scale     <tibble [9 × 4]> best_normalize_Wm728
#> 3 Petal.Length orderNorm        <tibble [9 × 4]> best_normalize_Wm728
#> 4 Petal.Width  orderNorm        <tibble [9 × 4]> best_normalize_Wm728
```

--------------------------------

### Load bestNormalize library

Source: https://petersonr.github.io/bestNormalize/index.html

Load the bestNormalize library into the R session.

```r
library(bestNormalize)
```

--------------------------------

### axe_env Method for step_best_normalize

Source: https://petersonr.github.io/bestNormalize/reference/step_best_normalize.html

Provides an axe_env method for the step_best_normalize object, likely for environment management.

```R
axe_env(x, ...)
```

--------------------------------

### Implement Optional Print Method for Custom Function

Source: https://petersonr.github.io/bestNormalize/articles/customization.html

Define a `print.cuberoot_x` method to display relevant statistics of the custom transformation when the object is printed. This provides a summary of the transformation parameters.

```R
print.cuberoot_x <- function(x, ...) {
  cat(ifelse(x$standardize, "Standardized", "Non-Standardized"),
      'cuberoot(x + a) Transformation with', x$n, 'nonmissing obs.:\n', 
      'Relevant statistics:\n', 
      '- a =', x$a, '\n',
      '- mean (before standardization) =', x$mean, '\n',
      '- sd (before standardization) =', x$sd, '\n')
}
```

--------------------------------

### print Method for log_x

Source: https://petersonr.github.io/bestNormalize/reference/log_x.html

Prints a summary of the log_x transformation object.

```APIDOC
## print.log_x

### Description
Prints a summary of the log_x transformation object.

### Arguments
- **x** (log_x) - An object of class 'log_x'.
- **...** - Additional arguments.

### Value
Prints a summary of the transformation to the console.
```

--------------------------------

### Visualizing In-Sample Transformations

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Displays histograms of the in-sample transformed data for several normalization methods. Helps assess the normality achieved by each transformation.

```r
par(mfrow = c(2,2))
MASS::truehist(arcsinh_obj$x.t, main = "Arcsinh transformation", nbins = 12)
MASS::truehist(boxcox_obj$x.t, main = "Box Cox transformation", nbins = 12)
MASS::truehist(yeojohnson_obj$x.t, main = "Yeo-Johnson transformation", nbins = 12)
MASS::truehist(orderNorm_obj$x.t, main = "orderNorm transformation", nbins = 12)
```

--------------------------------

### print Method for sqrt_x

Source: https://petersonr.github.io/bestNormalize/reference/sqrt_x.html

Prints a summary of the sqrt_x transformation object.

```APIDOC
## print(x, ...)

### Description
Prints the 'sqrt_x' object.

### Arguments
- **x** (object of class 'sqrt_x') - The object to print.
- **...** - Additional arguments.

### Value
Prints a summary of the transformation.
```

--------------------------------

### Tidy Output of step_best_normalize

Source: https://petersonr.github.io/bestNormalize/reference/step_best_normalize.html

Shows the output of the tidy method applied to a step_best_normalize object before preparation, indicating the selected variables.

```R
tidy(bn_trans, number = 1)
#> # A tibble: 1 × 4
#>   terms         chosen cv_info id                  
#>   <chr>          <dbl>   <dbl> <chr>               
#> 1 all_numeric()     NA      NA best_normalize_Wm728
```

--------------------------------

### print Method

Source: https://petersonr.github.io/bestNormalize/reference/bestNormalize.html

Prints the results of a bestNormalize object.

```APIDOC
## print.bestNormalize

### Description
Prints the results of a `bestNormalize` object.

### Parameters

- **x** (bestNormalize) - An object of class 'bestNormalize'.
- **...** (any) - Not used.
```

--------------------------------

### Verify Custom Transformation Output

Source: https://petersonr.github.io/bestNormalize/articles/customization.html

Perform a sanity check to confirm that the `bestNormalize` function correctly applied the custom cube-root transformation by comparing the transformed data with direct calculation.

```R
all.equal(x^(1/3), b$chosen_transform$x.t)
```

--------------------------------

### Visualize Transformations with ggplot2 and scales

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Demonstrates how to use the scales package with ggplot2 to visualize data that has been transformed by bestNormalize. Requires loading the ggplot2 library and creating a bestNormalize object.

```R
library(ggplot2)

x <- rgamma(1000, 1, .1)
bn <- bestNormalize(x)

bn
```

```R
# say y is related linearly to the transformed x
y <- bn$x.t * 1 + rnorm(1000)

# A log transformation does OK...
ggplot(data.frame(x=x,y=y), aes(x, y)) +
  geom_point() +
  scale_x_continuous(trans = "log", breaks = scales::log_breaks())
```

```R
# Create bestNormalize scale for use in ggplot (using bestNormalize object)
bn_trans <- scales::trans_new(
  name = "bn_trans",
  trans = function(x) predict(bn, newdata = x),
  inverse = function(x) predict(bn, newdata = x, inverse = TRUE)
)


ggplot(data.frame(x=x,y=y), aes(x, y)) +
  geom_point() +
  scale_x_continuous(trans = bn_trans)
```

--------------------------------

### Define orderNorm recipe step

Source: https://petersonr.github.io/bestNormalize/reference/step_orderNorm.html

Defines a recipe step for ORQ normalization. Use this to specify which variables should be transformed. The `transform_options` argument can pass arguments to the underlying `orderNorm` function.

```R
step_orderNorm(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  transform_info = NULL,
  transform_options = list(),
  num_unique = 5,
  skip = FALSE,
  id = rand_id("orderNorm")
)
```

--------------------------------

### Tidy estimates after baking

Source: https://petersonr.github.io/bestNormalize/reference/step_orderNorm.html

Extracts the transformation details after the `step_orderNorm` has been prepared (trained) using `prep()`. The `terms` column shows the variables that were transformed, and `value` contains the specific `orderNorm` objects for each variable.

```R
tidy(orq_estimates, number = 1)
```

--------------------------------

### Tidy estimates before baking

Source: https://petersonr.github.io/bestNormalize/reference/step_orderNorm.html

Extracts the transformation details from a `step_orderNorm` object before it has been trained using `prep()`. The `terms` column indicates the variables selected for transformation, and `value` is NA as the transformation has not yet been computed.

```R
tidy(orq_trans, number = 1)
```

--------------------------------

### print Method for exp_x

Source: https://petersonr.github.io/bestNormalize/reference/exp_x.html

Prints the results of an exp_x transformation object.

```APIDOC
## Method: print.exp_x

### Description
Prints the results of an exp_x transformation object.

### Arguments
- **x** (exp_x) - An object of class 'exp_x'.
- **...** - Additional arguments.

### Value
Prints a summary of the transformation, including standardization status and relevant statistics.
```

--------------------------------

### print Method for lambert

Source: https://petersonr.github.io/bestNormalize/reference/lambert.html

Prints an object of class 'lambert'.

```APIDOC
## print(x, ...)

### Description
Prints an object of class 'lambert'.

### Arguments
- **x** - An object of class 'lambert'.
- **...** - Additional arguments.
```

--------------------------------

### Print Method for Binarize

Source: https://petersonr.github.io/bestNormalize/reference/binarize.html

Prints the binarize object.

```APIDOC
## print(x, ...)

### Description
Prints the binarize object.

### Arguments
- **x** (binarize) - An object of class 'binarize'.
- **...** - Additional arguments.
```

--------------------------------

### Reversing Custom Statistic for Minimization

Source: https://petersonr.github.io/bestNormalize/articles/customization.html

To correctly use a statistic like a p-value with `bestNormalize`'s default minimization behavior, reverse the statistic (e.g., by subtracting from 1) so that the best transformation yields the smallest value.

```r
(do_this <- bestNormalize(x, norm_stat_fn = function(x) 1-nortest::lillie.test(x)$p))
```

--------------------------------

### Create Predict Method for Custom Transformation

Source: https://petersonr.github.io/bestNormalize/articles/customization.html

Implement the `predict.cuberoot_x` method to apply the cube-root transformation to new data or reverse it. This method handles both forward and inverse transformations, including standardization.

```R
predict.cuberoot_x <- function(object, newdata = NULL, inverse = FALSE, ...) {
  
  # If no data supplied and not inverse
  if (is.null(newdata) & !inverse)
    newdata <- object$x
  
  # If no data supplied and inverse
  if (is.null(newdata) & inverse)
    newdata <- object$x.t
  
  # Actually performing transformations
  
  # Perform inverse transformation as estimated
  if (inverse) {
    
    # Reverse-standardize
    if (object$standardize) 
      newdata <- newdata * object$sd + object$mean
    
    # Reverse-cube-root (cube)
    newdata <-  newdata^3 - object$a
    
    
    # Otherwise, perform transformation as estimated
  } else if (!inverse) {
    # Take cube root
    newdata <- (newdata + object$a)^(1/3)
    
    # Standardize to mean 0, sd 1
    if (object$standardize) 
      newdata <- (newdata - object$mean) / object$sd
  }
  
  # Return transformed data
  unname(newdata)
}
```

--------------------------------

### print Method for orderNorm

Source: https://petersonr.github.io/bestNormalize/reference/orderNorm.html

Prints the results of the orderNorm transformation.

```APIDOC
## print(x, ...)

### Description
Prints the results of the orderNorm transformation.

### Arguments
- **x** (orderNorm) - An object of class 'orderNorm'.
- **...** - Additional arguments.

### Value
Prints a summary of the orderNorm object.
```

--------------------------------

### Integrate Custom Function with bestNormalize

Source: https://petersonr.github.io/bestNormalize/articles/customization.html

Integrate custom transformation functions into the `bestNormalize` workflow by passing them in a list to the `new_transforms` argument. This allows `bestNormalize` to evaluate and select the custom transformation.

```R
# Store custom functions into list
custom_transform <- list(
  cuberoot_x = cuberoot_x,
  predict.cuberoot_x = predict.cuberoot_x,
  print.cuberoot_x = print.cuberoot_x
)

set.seed(123129)
x <- rgamma(100, 1, 1)
(b <- bestNormalize(x = x, new_transforms = custom_transform, standardize = FALSE))
```

--------------------------------

### axe_env Method for step_orderNorm

Source: https://petersonr.github.io/bestNormalize/reference/step_orderNorm.html

The `axe_env` method for `step_orderNorm` is used for managing the environment of the step object, typically for serialization or package management.

```APIDOC
## axe_env(x, ...)

### Description
Removes the environment from a `step_orderNorm` object, which can be useful for serialization or reducing package size.

### Arguments

* `x`: A `step_orderNorm` object.
* `...`: Not currently used.

### Value
An updated `step_orderNorm` object with its environment removed.
```

--------------------------------

### print Method for center_scale

Source: https://petersonr.github.io/bestNormalize/reference/no_transform.html

Prints a summary of the center_scale object.

```APIDOC
## print.center_scale

### Description
Prints a summary of the center_scale object.

### Arguments
- **x** (center_scale) - An object of class 'center_scale'.
- **...** - Not used.
```

--------------------------------

### print Method for boxcox

Source: https://petersonr.github.io/bestNormalize/reference/boxcox.html

Prints a summary of the Box-Cox transformation object.

```APIDOC
## print(x, ...)

### Description
Prints a summary of the 'boxcox' object, including estimated lambda, mean, and standard deviation.

### Arguments
- **x** (boxcox) - An object of class 'boxcox'.
- **...** - Additional arguments.
```

--------------------------------

### print Method for double_reverse_log

Source: https://petersonr.github.io/bestNormalize/reference/double_reverse_log.html

Prints a summary of the double reverse log transformation object.

```APIDOC
## print.double_reverse_log

### Description
Prints a summary of the double reverse log transformation object.

### Arguments

* **x** (double_reverse_log) - An object of class 'double_reverse_log'.
* **...** - Additional arguments.
```

--------------------------------

### Define Custom Cube-Root Transformation Function

Source: https://petersonr.github.io/bestNormalize/articles/customization.html

Define a custom transformation function `cuberoot_x` that handles negative numbers by adding a constant `a` and optionally standardizes the output. Assign a class to the returned object for compatibility with bestNormalize.

```R
cuberoot_x <- function(x, a = NULL, standardize = TRUE, ...) {
  stopifnot(is.numeric(x))
  
  min_a <- max(0, -(min(x, na.rm = TRUE)))
  if(!length(a)) 
    a <- min_a
  if(a < min_a) {
    warning("Setting a <  max(0, -(min(x))) can lead to transformation issues",
            "Standardize set to FALSE")
    standardize <- FALSE
  }
  
  
  x.t <- (x + a)^(1/3)
  mu <- mean(x.t, na.rm = TRUE)
  sigma <- sd(x.t, na.rm = TRUE)
  if (standardize) x.t <- (x.t - mu) / sigma
  
  # Get in-sample normality statistic results
  ptest <- nortest::pearson.test(x.t)
  
  val <- list(
    x.t = x.t,
    x = x,
    mean = mu,
    sd = sigma,
    a = a,
    n = length(x.t) - sum(is.na(x)),
    norm_stat = unname(ptest$statistic / ptest$df),
    standardize = standardize
  )
  
  # Assign class
  class(val) <- c('cuberoot_x', class(val))
  val
}
```

--------------------------------

### step_orderNorm Function

Source: https://petersonr.github.io/bestNormalize/reference/step_orderNorm.html

This function creates a specification for a recipe step that applies the ORQ (orderNorm) transformation to selected variables. It approximates the 'true' normalizing transformation and is computationally efficient.

```APIDOC
## step_orderNorm()

### Description
Creates a specification of a recipe step that will transform data using the ORQ (orderNorm) transformation, which approximates the "true" normalizing transformation if one exists. This is considerably faster than `step_bestNormalize`.

### Arguments

* `recipe`: A formula or recipe object.
* `...`: One or more selector functions to choose which variables are affected by the step. See `selections()` for more details.
* `role`: Not used by this step since no new variables are created.
* `trained`: For recipes functionality.
* `transform_info`: A numeric vector of transformation values. This is `NULL` until computed by `prep.recipe()`.
* `transform_options`: Options to be passed to `orderNorm`.
* `num_unique`: An integer where data that have less possible values will not be evaluated for a transformation.
* `skip`: For recipes functionality.
* `id`: For recipes functionality.

### Value
An updated version of `recipe` with the new step added to the sequence of existing steps (if any).

### Details
The orderNorm transformation can be used to rescale a variable to be more similar to a normal distribution. See `?orderNorm` for more information; `step_orderNorm` is the implementation of `orderNorm` in the `recipes` context.

### References
Ryan A. Peterson (2019). Ordered quantile normalization: a semiparametric transformation built for the cross-validation era. Journal of Applied Statistics, 1-16.

### See also
`orderNorm`, `bestNormalize`, `recipe()`, `prep.recipe()`, `bake.recipe()`

### Examples
```R
library(recipes)
rec <- recipe(~ ., data = as.data.frame(iris))
orq_trans <- step_orderNorm(rec, all_numeric())
orq_estimates <- prep(orq_trans, training = as.data.frame(iris))
orq_data <- bake(orq_estimates, as.data.frame(iris))

plot(density(iris[, "Petal.Length"]), main = "before")
plot(density(orq_data$Petal.Length), main = "after")
```
```

--------------------------------

### Visualizing Best Transformation

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Plots a histogram of the best overall transformation and a line plot of the transformation function itself. This helps to visualize the final chosen normalization.

```r
par(mfrow = c(1,2))
MASS::truehist(BNobject$x.t, 
               main = paste("Best Transformation:", 
                            class(BNobject$chosen_transform)[1]), nbins = 12)
plot(xx, predict(BNobject, newdata = xx), type = "l", col = 1, 
     main = "Best Normalizing transformation", ylab = "g(x)", xlab = "x")
```

--------------------------------

### print Method for arcsinh_x

Source: https://petersonr.github.io/bestNormalize/reference/arcsinh_x.html

Prints an object of class 'arcsinh_x'. This method is part of the S3 generic `print` for objects of class 'arcsinh_x'.

```APIDOC
## Method: print.arcsinh_x

### Description
Prints an object of class 'arcsinh_x'. This method is part of the S3 generic `print` for objects of class 'arcsinh_x'.

### Arguments
- **x** (arcsinh_x) - An object of class 'arcsinh_x'.
- **...** (any) - Additional arguments.
```

--------------------------------

### Print Box-Cox Object

Source: https://petersonr.github.io/bestNormalize/reference/boxcox.html

Prints a summary of the Box-Cox transformation object, including estimated lambda, mean, and standard deviation.

```R
print(x, ...)
```

--------------------------------

### Fast Normalization without Out-of-Sample Estimation

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Performs normalization by disabling out-of-sample estimation to speed up computation. Note that this often results in the orderNorm transformation being chosen. Setting allow_orderNorm = FALSE is recommended in this case.

```r
bestNormalize(x, allow_orderNorm = FALSE, out_of_sample = FALSE)
```

--------------------------------

### print Function for Yeo-Johnson Objects

Source: https://petersonr.github.io/bestNormalize/reference/yeojohnson.html

Prints a summary of the Yeo-Johnson transformation object.

```APIDOC
## print(x, ...)

### Description
Prints a summary of the Yeo-Johnson transformation object, including estimated statistics like lambda, mean, and standard deviation.

### Arguments
- **x** (yeojohnson) - An object of class 'yeojohnson'.
- **...** - Additional arguments.

### Example
```R
x <- rgamma(100, 1, 1)
yeojohnson_obj <- yeojohnson(x)
print(yeojohnson_obj)
```
```

--------------------------------

### Binarize Transformation

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Applies a binarization transformation to the input data. This is often used as a last resort when other transformations are not suitable.

```r
(binarize_obj <- binarize(x))
```

--------------------------------

### bestNormalize Function Signature

Source: https://petersonr.github.io/bestNormalize/reference/step_best_normalize.html

Defines the signature for the step_best_normalize function, including its arguments and their types.

```R
step_best_normalize(
  recipe,
  ...,
  role = NA,
  trained = FALSE,
  transform_info = NULL,
  transform_options = list(),
  num_unique = 5,
  skip = FALSE,
  id = rand_id("best_normalize")
)
```

--------------------------------

### Create Center/Scale Transformation Object

Source: https://petersonr.github.io/bestNormalize/reference/no_transform.html

The `center_scale` function performs centering and scaling, which can be passed to `bestNormalize` when `standardize = TRUE`. It provides a comparable normalization statistic for standardized data.

```R
center_scale(x, warn = TRUE, ...)
```

--------------------------------

### Visualize Normalization Transformation

Source: https://petersonr.github.io/bestNormalize/index.html

Plot the results of the best normalization transformation, with the legend positioned at the bottom right.

```r
plot(BN_obj, leg_loc = "bottomright")
```

--------------------------------

### predict Method for sqrt_x

Source: https://petersonr.github.io/bestNormalize/reference/sqrt_x.html

Applies the trained sqrt_x transformation to new data, or performs the inverse transformation.

```APIDOC
## predict(object, newdata = NULL, inverse = FALSE, ...)

### Description
Applies the transformation to new data, or performs the inverse transformation.

### Arguments
- **object** (object of class 'sqrt_x') - The trained sqrt_x object.
- **newdata** (vector, optional) - A vector of data to be transformed. If NULL, uses the original data from the object.
- **inverse** (boolean) - If TRUE, performs the reverse transformation. Defaults to FALSE.
- **...** - Additional arguments.

### Value
The numeric value of the transformation performed on new data, or the reverse transformation.
```

--------------------------------

### predict Method for exp_x

Source: https://petersonr.github.io/bestNormalize/reference/exp_x.html

Applies the exp_x transformation to new data or performs the inverse transformation.

```APIDOC
## Method: predict.exp_x

### Description
Applies the exp_x transformation to new data or performs the inverse transformation.

### Arguments
- **object** (exp_x) - An object of class 'exp_x'.
- **newdata** (vector) - A vector of data to be (potentially reverse) transformed. Defaults to NULL.
- **inverse** (boolean) - If TRUE, performs the reverse transformation. Defaults to FALSE.
- **...** - Additional arguments.

### Value
The numeric value of the transformation performed on new data, or the result of the inverse transformation.
```

--------------------------------

### print Method for no_transform

Source: https://petersonr.github.io/bestNormalize/reference/no_transform.html

Prints a summary of the no_transform object.

```APIDOC
## print.no_transform

### Description
Prints a summary of the no_transform object.

### Arguments
- **x** (no_transform) - An object of class 'no_transform'.
- **...** - Not used.
```

--------------------------------

### Manually Selecting Best Transformation from Custom Statistic

Source: https://petersonr.github.io/bestNormalize/articles/customization.html

After obtaining results from a custom `norm_stat_fn`, you can manually select the best transformation by identifying the maximum value (if using p-values) from the `norm_stats` and accessing the corresponding transformation.

```r
best_transform <- names(which.max(dont_do_this$norm_stats))
(do_this <- dont_do_this$other_transforms[[best_transform]])
```

--------------------------------

### predict Method for lambert

Source: https://petersonr.github.io/bestNormalize/reference/lambert.html

Applies the Lambert W x F transformation to new data or performs the inverse transformation.

```APIDOC
## predict(object, newdata = NULL, inverse = FALSE, ...)

### Description
Applies the Lambert W x F transformation to new data or performs the inverse transformation.

### Arguments
- **object** - An object of class 'lambert'.
- **newdata** (vector) - A vector of data to be (reverse) transformed.
- **inverse** (logical) - If TRUE, performs reverse transformation.
- **...** - Additional arguments.

### Value
The numeric value of the transformation performed on new data, and allows for the inverse transformation as well.
```

--------------------------------

### Predict Method for Binarize

Source: https://petersonr.github.io/bestNormalize/reference/binarize.html

Applies the binarize transformation to new data or performs the inverse transformation.

```APIDOC
## predict(object, newdata = NULL, inverse = FALSE, ...)

### Description
Applies the binarize transformation to new data or performs the inverse transformation.

### Arguments
- **object** (binarize) - An object of class 'binarize'.
- **newdata** (vector) - A vector of data to be (reverse) transformed. Defaults to the original data.
- **inverse** (boolean) - If TRUE, performs the reverse transformation. Defaults to FALSE.
- **...** - Additional arguments.

### Value
- If `inverse = FALSE`, returns the numeric value (0 or 1) of the transformation on `newdata`.
- If `inverse = TRUE`, returns a factor indicating where the original data was cut, as the transform is not one-to-one.
```

--------------------------------

### Plotting Multiple Transformations

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Visualizes the effect of different normalization transformations (arcsinh, Box Cox, Yeo-Johnson, OrderNorm) on a range of input values. Useful for comparing transformation behaviors.

```r
xx <- seq(min(x), max(x), length = 100)
plot(xx, predict(arcsinh_obj, newdata = xx), type = "l", col = 1, ylim = c(-4, 4),
     xlab = 'x', ylab = "g(x)")
lines(xx, predict(boxcox_obj, newdata = xx), col = 2)
lines(xx, predict(yeojohnson_obj, newdata = xx), col = 3)
lines(xx, predict(orderNorm_obj, newdata = xx), col = 4)

legend("bottomright", legend = c("arcsinh", "Box Cox", "Yeo-Johnson", "OrderNorm"), 
       col = 1:4, lty = 1, bty = 'n')
```

--------------------------------

### Estimate Best Normalizing Transformation with Leave-One-Out CV

Source: https://petersonr.github.io/bestNormalize/reference/bestNormalize.html

Estimates the optimal normalizing transformation using leave-one-out cross-validation. This method provides the most accurate estimate of normalization efficacy but can be very time-consuming for large datasets. Arguments `k` and `r` are ignored when `loo = TRUE`.

```r
x <- rgamma(100, 1, 1)

if (FALSE) {
# With leave-one-out CV
BN_obj <- bestNormalize(x, loo = TRUE)
BN_obj
p <- predict(BN_obj)
x2 <- predict(BN_obj, newdata = p, inverse = TRUE)

all.equal(x2, x)
}
```

--------------------------------

### Plotting Best Normalizing Transformations

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Visualizes the best normalizing transformation functions for 'price', 'mileage', and 'yearsold'. This plot shows the relationship between the original variable and its transformed value.

```R
par(mfrow = c(2, 2))
price.xx <- seq(min(autotrader$price), max(autotrader$price), length = 100)
mileage.xx <- seq(min(autotrader$mileage), max(autotrader$mileage), length = 100)
yearsold.xx <- seq(min(autotrader$yearsold), max(autotrader$yearsold), length = 100)

plot(price.xx, predict(priceBN, newdata = price.xx), type = "l", 
     main = "Price bestNormalizing transformation", 
     xlab = "Price ($)", ylab = "g(price)")
plot(mileage.xx, predict(mileageBN, newdata = mileage.xx), type = "l", 
     main = "Mileage bestNormalizing transformation", 
     xlab = "Mileage", ylab = "g(Mileage)")
plot(yearsold.xx, predict(yearsoldBN, newdata = yearsold.xx), type = "l", 
     main = "Years-old bestNormalizing transformation", 
     xlab = "Years-old", ylab = "g(Years-old)")
```

--------------------------------

### Automatically Select Best Normalization

Source: https://petersonr.github.io/bestNormalize/articles/bestNormalize.html

Automatically selects the best normalization transformation for the given data using the `bestNormalize` function. This function evaluates multiple transformations and chooses the one that best fits a normal distribution based on the Pearson P statistic.

```r
# Pick the best one automatically
(BNobject <- bestNormalize(x))
```

--------------------------------

### log_x Function

Source: https://petersonr.github.io/bestNormalize/reference/log_x.html

Performs a log_b(x + a) normalization transformation on a given vector. It estimates the parameter 'a' by default and allows customization of the base 'b' and standardization.

```APIDOC
## log_x

### Description
Performs a log_b(x + a) normalization transformation.

### Arguments
- **x** (vector) - A vector to normalize.
- **a** (numeric, optional) - The constant to add to x. Defaults to max(0, -min(x) + eps).
- **b** (numeric, optional) - The base of the log. Defaults to 10.
- **standardize** (logical, optional) - If TRUE, the transformed values are centered and scaled to attempt a standard normal distribution.
- **eps** (numeric, optional) - The allowed error in the expression for the selected a.
- **warn** (logical, optional) - Should a warning result from infinite values?
- **...** - Additional arguments.

### Value
A list of class 'log_x' containing transformation details:
- **x.t** (numeric) - Transformed original data.
- **x** (numeric) - Original data.
- **mean** (numeric) - Mean after transformation but prior to standardization.
- **sd** (numeric) - Standard deviation after transformation but prior to standardization.
- **a** (numeric) - Estimated 'a' value.
- **b** (numeric) - Estimated base 'b' value.
- **n** (integer) - Number of non-missing observations.
- **norm_stat** (numeric) - Pearson's P / degrees of freedom.
- **standardize** (logical) - Indicates if standardization was applied.

### Details
The parameter 'a' is estimated by default using the training set, while the base 'b' must be specified. This transformation is designed to be estimated and applied to new data using the `predict` function.
```
Tokens:
Raw