### Setup Development Environment

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/running_tests_locally.md

Activate the project environment and include necessary files for local development and testing. This is a crucial step for the recommended workflow.

```julia
using Pkg; Pkg.activate("."); TestEnv.activate(); include("test/front_matter.jl");
```

--------------------------------

### Julia Basic Block Structure Example

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/ir_representation.md

Illustrates the basic block structure of Julia IR, showing sequential statements and control flow jumps.

```julia
  1 ─
  │  
  └──
  2 ─
  3 ─
  └──
```

--------------------------------

### Example Function with Return Value

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/rule_system.md

Illustrates how communication between function calls occurs via return values.

```julia
function foo(x)
    y = bar(x)
    z = bar(y)
    return z
end
```

--------------------------------

### Julia Function for AD Example

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/algorithmic_differentiation.md

Defines a Julia function with multiple arguments, including a tuple, for demonstrating AD.

```julia
f(x::Float64, y::Tuple{Float64, Float64}) = x + y[1] * y[2]
```

--------------------------------

### Instantiate and call rrule!! with CoDual objects

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/rule_system.md

Demonstrate how to use the rrule!! function by creating CoDual objects for the function and its input, then calling rrule!! to get the output and the pullback function.

```jldoctest
julia> codual_foo = CoDual(foo, NoFData())

julia> codual_x = CoDual((5.0, [1.0, 2.0]), (NoFData(), [0.0, 0.0]))

julia> out, pb!! = rrule!!(codual_foo, codual_x)

julia> out
CoDual{Float64, NoFData}(8.0, NoFData())

julia> pb!! isa Function
true
```

--------------------------------

### Example Function with Global Mutable State (Outlawed)

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/rule_system.md

This example demonstrates a pattern that Mooncake.jl explicitly does not support due to the use of global mutable state.

```julia
const a = randn(10)

function bar(x)
    a .+= x
    return nothing
end

function foo(x)
    bar(x)
    return a
end
```

--------------------------------

### Forwards-Rule Interface Example

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/forwards_mode_design.md

Demonstrates the expected interface for a forwards-rule callable. It must accept Dual numbers for inputs and return a Dual number for the output, preserving primal state.

```julia
z_dz = rule_for_f(Dual(f, df), Dual(x, dx), Dual(y, dy))::Dual
```

--------------------------------

### Example Function Modifying Arguments

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/rule_system.md

Demonstrates how functions can communicate implicitly by modifying their arguments, even without returning a value.

```julia
function bar(x::Vector{Float64})
    x .*= 2
    return nothing
end

function foo(x::Vector{Float64})
    bar(x)
    bar(x)
    return x
end
```

--------------------------------

### Example Function with Closed-Over Values (Supported)

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/rule_system.md

Illustrates a supported pattern using closed-over values within nested functions, contrasting with the outlawed global mutable state.

```julia
function foo(x)
    function bar(y)
        x .+= y
        return nothing
    end
    return bar(x)
end
```

--------------------------------

### Handling Mutation and Reshape on GPU

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/HISTORY.md

Provides examples of differentiating operations involving mutation and reshaping on GPU arrays, including `reshape`, CPU-to-GPU transfers (`cu`), and GPU-to-CPU transfers (`Array`).

```julia
f = x -> sum(reshape(x, 4, 2))     # reshape on GPU
f = x -> sum(sin.(cu(x)))           # CPU → GPU (gradient flows back to CPU)
f = x -> sum(Array(x).^2)           # GPU → CPU
```

--------------------------------

### Construct BBCode from IRCode

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/ir_representation.md

Demonstrates the conversion of an existing IRCode object to Mooncake's BBCode representation. This is a common starting point for using BBCode.

```jldoctest
julia> using Mooncake: BBCode

julia> bb_ir = BBCode(ir);

julia> bb_ir isa BBCode
true

julia> Core.Compiler.IRCode(bb_ir)
  1 ─      nothing::Nothing
4 2 ┄ %2 = φ (#1 => 1, #3 => %7)::Int64
  │   %3 = φ (#1 => 0, #3 => %6)::Int64
  │   %4 = intrinsic Base.slt_int(%3, _2)::Bool
  └──      goto #4 if not %4
5 3 ─ %6 = intrinsic Base.add_int(%3, 1)::Int64
6 │   %7 = intrinsic Base.mul_int(%2, %6)::Int64
7 └──      goto #2
8 4 ─      return %2
```

--------------------------------

### Julia Intrinsic Function Example

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/ir_representation.md

Shows an example of a Julia intrinsic function, which has special handling in the compiler.

```jldoctest
julia> Base.lt_float
lt_float (intrinsic function #33)
```

--------------------------------

### Example of an incorrect rrule implementation

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/utilities/debug_mode.md

This example demonstrates a rule that incorrectly returns an `rdata` type (`Float64`) that does not match the primal type of its argument (`Float32`). This can lead to silent failures or hard-to-debug errors.

```julia
function rrule!!(::CoDual{typeof(+)}, x::CoDual{<:Real}, y::CoDual{<:Real})
    plus_reverse_pass(dz::Real) = NoRData(), dz, dz
    return zero_fcodual(primal(x) + primal(y))
end
```

--------------------------------

### Example of _new_ primitive usage

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/custom_tangent_type.md

Illustrates how a constructor call is lowered to the `_new_` primitive in Mooncake's IR. This primitive serves as a dispatchable wrapper around the `:new` IR node for object construction.

```julia
_new_(A{Float64}, 1.0, nothing)
```

--------------------------------

### Derived Rule for a Julia Function (Manual Example)

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/forwards_mode_design.md

Illustrates the structure of a derived forwards-mode rule for a simple Julia function `f(x)` composed of `g(x)` and `h(x, y)`. It shows how primal calls are replaced with calls to their respective rules and arguments are wrapped in `Dual`.

```julia
function rule_for_f(::Dual{typeof(f)}, x::Dual)
    y = rule_for_g(zero_dual(g), x)
    z = rule_for_h(zero_dual(h), x, y)
    return z
end
```

--------------------------------

### Enabling Debug Mode with AutoMooncake

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/utilities/debug_mode.md

This example shows how to enable debug mode when configuring the `AutoMooncake` backend in ADTypes.jl. Setting `debug_mode=true` ensures that type checks are performed during the differentiation process.

```julia
backend = AutoMooncake(; config=Mooncake.Config(; debug_mode=true));
```

--------------------------------

### Demonstrating Primitives and Overlays Conflicts

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/known_limitations.md

This example illustrates two failure modes: an overlay changing the return type of a primitive, and an overlay within a primitive's body that never runs. It checks if Mooncake's inferred return type for callers matches the original type 'A' rather than the overlaid type 'B'.

```jldoctest
julia> struct A end

julia> struct B end

julia> direct_primitive(::A) = A()
direct_primitive (generic function with 1 method)

julia> Mooncake.@mooncake_overlay direct_primitive(::A) = B()

julia> Mooncake.@is_primitive Mooncake.DefaultCtx Mooncake.ReverseMode Tuple{typeof(direct_primitive), A}

julia> helper(::A) = A()
helper (generic function with 1 method)

julia> Mooncake.@mooncake_overlay helper(::A) = B()

julia> indirect_primitive(x::A) = helper(x)
indirect_primitive (generic function with 1 method)

julia> Mooncake.@is_primitive Mooncake.DefaultCtx Mooncake.ReverseMode Tuple{typeof(indirect_primitive), A}

julia> caller_direct(x::A) = direct_primitive(x)
caller_direct (generic function with 1 method)

julia> caller_indirect(x::A) = indirect_primitive(x)
caller_indirect (generic function with 1 method)

julia> interp = Mooncake.MooncakeInterpreter(Mooncake.DefaultCtx, Mooncake.ReverseMode);

julia> Base.code_ircode_by_type(Tuple{typeof(caller_direct), A}; interp)[1][2] === A
true

julia> Base.code_ircode_by_type(Tuple{typeof(caller_indirect), A}; interp)[1][2] === A
true
```

--------------------------------

### Differentiate Function with Pointer Argument

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/known_limitations.md

Demonstrates differentiating a function that uses a pointer derived from a Vector. This example shows how to build a rule and compute the value and gradient, illustrating a scenario where pointers are used within a function.

```jldoctest
function foo(x::Vector{Float64})
    p = pointer(x, 2)
    return unsafe_load(p)
end

rule = build_rrule(Tuple{typeof(foo), Vector{Float64}})
Mooncake.value_and_gradient!!(rule, foo, [5.0, 4.0])

# output
(4.0, (NoTangent(), [0.0, 1.0]))
```

--------------------------------

### Run Extension and Integration Tests

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/running_tests_locally.md

Include the main test file for extension and integration tests. This activates the specific environment for these tests.

```julia
include("test/ext/ExampleExtension.jl")
```

--------------------------------

### Example Tangent Type Representation

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/custom_tangent_type.md

Illustrates how Mooncake.jl represents tangent types for a given primal type, including the split into forward and reverse data components. This example shows the tangent structure for a tuple containing a Float64, a Vector{Float64}, and an Int.

```julia
Tuple{Float64, Vector{Float64}, NoTangent}
```

```julia
Tuple{NoFData, Vector{Float64}, NoFData}
```

```julia
Tuple{Float64, NoRData, NoRData}
```

--------------------------------

### Compute Gradient In-Place with Prepared Input

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

For maximum efficiency, compute the gradient in-place by providing storage space using `DI.gradient!`.

```julia
grad = similar(x)
DI.gradient!(f, grad, prep, backend, x)
```

--------------------------------

### Prepare and Compute Jacobian with DI

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

Demonstrates how to prepare for and compute a Jacobian matrix for a function using the DifferentiationInterface.jl API.

```julia
h(x) = cos.(x) .* sin.(reverse(x))
prep = DI.prepare_jacobian(h, backend, x)
DI.jacobian(h, prep, backend, x)
```

--------------------------------

### Cache Preparation Utilities

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/interface.md

Utilities for preparing cache objects for different differentiation modes.

```APIDOC
## Mooncake.prepare_derivative_cache

### Description
Prepares a cache object suitable for derivative calculations.

### Usage
```julia
derivative_cache = Mooncake.prepare_derivative_cache()
```
```

```APIDOC
## Mooncake.prepare_gradient_cache

### Description
Prepares a cache object suitable for gradient calculations.

### Usage
```julia
gradient_cache = Mooncake.prepare_gradient_cache()
```
```

```APIDOC
## Mooncake.prepare_pullback_cache

### Description
Prepares a cache object suitable for pullback calculations.

### Usage
```julia
pullback_cache = Mooncake.prepare_pullback_cache()
```
```

```APIDOC
## Mooncake.prepare_hvp_cache

### Description
Prepares a cache object suitable for Hessian-vector product calculations.

### Usage
```julia
hvp_cache = Mooncake.prepare_hvp_cache()
```
```

```APIDOC
## Mooncake.prepare_hessian_cache

### Description
Prepares a cache object suitable for Hessian calculations.

### Usage
```julia
hessian_cache = Mooncake.prepare_hessian_cache()
```
```

--------------------------------

### Get closure field types

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/rule_system.md

Inspect the types of variables closed over by a closure. This demonstrates how closures are essentially callable structs.

```jldoctest
function foo(x)
    function bar(y)
        x .+= y
        return nothing
    end
    return bar
end
bar = foo(randn(5))
fieldtypes(typeof(bar))

# output
(Vector{Float64},)
```

--------------------------------

### Create Mooncake.jl Backend for DI

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

Instantiate the Mooncake.jl backend for use with DifferentiationInterface.jl. The config parameter can be set to nothing for default settings.

```julia
backend = DI.AutoMooncake(; config=nothing)
```

--------------------------------

### Manipulate IRCode Statements

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/reverse_mode_design.md

Provides examples of how to perform line-by-line transformations on the IRCode statements, which is crucial for forward mode AD modifications.

```julia
Mooncake.make_ad_stmts!
```

--------------------------------

### Bad Generated Function Implementation for tangent_type

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/misc_internals_notes.md

An example of a flawed generated function implementation for tangent_type that relies on its own definition, leading to world age issues.

```julia
bad_tangent_type(::Type{Float64}) = Float64
@generated function bad_tangent_type(::Type{P}) where {P<:Tuple}
    return Tuple{map(bad_tangent_type, fieldtypes(P))...}
end
bad_tangent_type(::Type{Float32}) = Float32
```

--------------------------------

### Compute Gradient with Tangent Zeroing Configuration

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/interface.md

Demonstrates gradient computation with `friendly_tangents = true` and the `args_to_zero` option. This allows specifying which arguments' tangents should be skipped during computation, useful for constant arguments.

```julia
cache = MC.prepare_gradient_cache(g, x_eval; config=MC.Config(friendly_tangents=true))
val, grad = MC.value_and_gradient!!(
    cache,
    g,
    x_eval;
    args_to_zero = (false, true),
)
```

--------------------------------

### Get Tangent Type of a Pointer

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/known_limitations.md

Retrieves the tangent type for a given pointer type. This is useful for understanding how tangents are handled for pointer-based data structures.

```jldoctest
julia> Mooncake.tangent_type(Ptr{Float64})
Ptr{Float64}
```

--------------------------------

### Run Mooncake.jl Performance Benchmarks

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/bench/README.md

Execute the Mooncake.jl performance benchmarks locally. This will produce a DataFrame with detailed results on primal time, Mooncake AD time, and their ratio.

```julia
include("bench/run_benchmarks.jl")
df = DataFrame(benchmark_derived_rrules!!(Xoshiro))
```

--------------------------------

### Define Function and Input for Differentiation

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

Define a sample function `f(x)` and an input vector `x` for demonstrating gradient computation.

```julia
f(x) = sum(abs2, x)
x = float.(1:3)
```

--------------------------------

### Import Mooncake.jl and DifferentiationInterface.jl

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

Import the necessary libraries for using Mooncake.jl with the DifferentiationInterface.jl API.

```julia
import DifferentiationInterface as DI
import Mooncake
```

--------------------------------

### Julia function for differentiation

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/algorithmic_differentiation.md

Defines a simple Julia function `f(x)` that takes a Float64 and returns its sine. This serves as a basic example for demonstrating AD.

```julia
f(x::Float64) = sin(x)
```

--------------------------------

### AD With Mutable Data Example

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/rule_system.md

This function modifies its input vector in-place and returns the sum. The mathematical model represents this as updating the input vector and returning the sum.

```julia
function f!(x::Vector{Float64})
    x .*= x
    return sum(x)
end
```

--------------------------------

### Get Tangent Type for a Primitive Type

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/rule_system.md

Retrieves the specific tangent type associated with a given primitive type `P`. This is fundamental for understanding how gradients are stored and managed.

```julia
```julia
Mooncake.tangent_type(P)
```
```

--------------------------------

### Import Mooncake.jl for Native API

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

Import the Mooncake.jl library to use its native API for differentiation.

```julia
import Mooncake
```

--------------------------------

### Insert New Instruction with IRCode

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/ir_representation.md

Demonstrates inserting a new multiplication instruction before an existing add_int instruction using IRCode. It shows how `insert_node!` adds the instruction and returns its new SSAValue, and how `compact!` re-numbers subsequent instructions.

```jldoctest
julia> ni = CC.NewInstruction(Expr(:call, Base.mul_int, SSAValue(3), 2), Int)
Compiler.NewInstruction(:((Core.Intrinsics.mul_int)(%3, 2)), Int64, Compiler.NoCallInfo(), nothing, nothing)

julia> new_ssa = CC.insert_node!(new_ir, SSAValue(6), ni)
:(%10)

julia> new_ir
  1 ─      nothing::Nothing
4 2 ┄ %2 = φ (#1 => 1, #3 => %7)::Int64
  │   %3 = φ (#1 => 0, #3 => %6)::Int64
  │   %4 = intrinsic Base.slt_int(%3, _2)::Bool
  └──      goto #4 if not %4
5 3 ─      intrinsic (Core.Intrinsics.mul_int)(%3, 2)::Int64
  │   %6 = intrinsic Base.add_int(%3, 1)::Int64
6 │   %7 = intrinsic (Core.Intrinsics.add_int)(%2, %6)::Int64
7 └──      goto #2
8 4 ─      return %2
```

```jldoctest
julia> stmt = CC.getindex(CC.getindex(new_ir, SSAValue(6)), :stmt)
:(Base.add_int(%3, 1))

julia> stmt.args[2] = new_ssa;

julia> new_ir
  1 ─       nothing::Nothing
4 2 ┄ %2  = φ (#1 => 1, #3 => %7)::Int64
  │   %3  = φ (#1 => 0, #3 => %6)::Int64
  │   %4  = intrinsic Base.slt_int(%3, _2)::Bool
  └──       goto #4 if not %4
5 3 ─ %10 = intrinsic (Core.Intrinsics.mul_int)(%3, 2)::Int64
  │   %6  = intrinsic Base.add_int(%10, 1)::Int64
6 │   %7  = intrinsic (Core.Intrinsics.add_int)(%2, %6)::Int64
7 └──       goto #2
8 4 ─       return %2

julia> new_ir = CC.compact!(new_ir)
  1 ─      nothing::Nothing
4 2 ┄ %2 = φ (#1 => 1, #3 => %8)::Int64
  │   %3 = φ (#1 => 0, #3 => %7)::Int64
  │   %4 = intrinsic Base.slt_int(%3, _2)::Bool
  └──      goto #4 if not %4
5 3 ─ %6 = intrinsic (Core.Intrinsics.mul_int)(%3, 2)::Int64
  │   %7 = intrinsic Base.add_int(%6, 1)::Int64
6 │   %8 = intrinsic (Core.Intrinsics.add_int)(%2, %7)::Int64
7 └──      goto #2
8 4 ─      return %2
```

--------------------------------

### Define Custom Structure and Function for Differentiation

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/interface.md

Defines a custom struct `SimplePair` and a function `g` that operates on it. This setup is used to demonstrate gradient calculation for user-defined types.

```julia
import Mooncake as MC

struct SimplePair
    x1::Float64
    x2::Float64
end

# Define a simple function
g(x::SimplePair) = x.x1^2 + x.x2^2

# Where to evaluate the derivative
x_eval = SimplePair(1.0, 2.0)
```

--------------------------------

### Mutating Global Variables

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/known_limitations.md

This example demonstrates how mutating a global variable within a function can lead to incorrect gradient calculations. Avoid differentiating functions that modify global state.

```julia
const x = Ref(1.0);

function foo(y::Float64)
           x[] = y
           return x[]
       end
```

```julia
rule = Mooncake.build_rrule(foo, 2.0);
```

```julia
Mooncake.value_and_gradient!!(rule, foo, 2.0)
```

--------------------------------

### Gradient and Hessian Evaluation with Mooncake.jl

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/README.md

Demonstrates how to use Mooncake.jl for reverse mode gradient, forward mode gradient, and Hessian calculations. Prepare caches for repeated evaluations to improve performance.

```julia
import Mooncake as MC

f(x) = (1 - x[1])^2 + 100 * (x[2] - x[1]^2)^2  # Rosenbrock
x = [1.2, 1.2]

# Reverse mode
grad_cache = MC.prepare_gradient_cache(f, x);
val, grad = MC.value_and_gradient!!(grad_cache, f, x)

# Forward mode
fwd_cache = MC.prepare_derivative_cache(f, x);
val_fwd, grad_fwd = MC.value_and_gradient!!(fwd_cache, f, x)

# Hessian
hess_cache = MC.prepare_hessian_cache(f, x);
val, grad, H = MC.value_gradient_and_hessian!!(hess_cache, f, x)
# val  : f(x)
# grad : ∇f(x)  (length-n vector)
# H    : ∇²f(x) (n×n matrix)
```

--------------------------------

### Prepare Gradient Computation with DI

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

Prepare for fast gradient computations by calling `DI.prepare_gradient` with a typical input. This captures necessary differentiation rules.

```julia
typical_x = rand(3)
prep = DI.prepare_gradient(f, backend, typical_x)
```

--------------------------------

### Configure Mooncake to Empty Cache Before Rebuilding Rules

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/HISTORY.md

Instantiate Mooncake's Config with `empty_cache=true` to ensure internal caches are freed before new rules are built. This is useful for scenarios requiring a clean slate for gradient caching.

```julia
config = Mooncake.Config(empty_cache=true)
cache = Mooncake.prepare_gradient_cache(sin, 1.0; config)
```

--------------------------------

### Get Tangent Field Value

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/custom_tangent_type.md

Defines the `get_tangent_field` function for `TangentForA`. This function allows accessing specific fields (like `:x` or `:a`) of a tangent object. It throws an error for unhandled fields.

```julia
@inline function Mooncake.get_tangent_field(t::TangentForA, f)
    if f === :x
        return t.x
    elseif f === :a
        return t.a
    else
        throw(error("Unhandled field $f"))
    end
end
```

--------------------------------

### Prepare Gradient for Multi-Argument Function

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

Prepare for gradient computation of a multi-argument function, wrapping non-differentiated arguments with `DI.Constant`.

```julia
typical_a, typical_b = 1.0, 1.0
prep = DI.prepare_gradient(g, backend, typical_x, DI.Constant(typical_a), DI.Constant(typical_b))
```

--------------------------------

### CUDA Linear Algebra Operations

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/HISTORY.md

Shows examples of differentiating standard Julia/CUDA linear algebra operations like matrix multiplication, dot products, and norms on CUDA arrays. Scalar indexing is not supported.

```julia
# matrix multiply
f = (A, B) -> sum(A * B)
A, B = CUDA.randn(Float32, 4, 4), CUDA.randn(Float32, 4, 4)
cache = prepare_gradient_cache(f, A, B)
_, (_, ∂A, ∂B) = value_and_gradient!!(cache, f, A, B)

# matrix-vector multiply
f = (A, x) -> sum(A * x)
A, x = CUDA.randn(Float32, 4, 4), CUDA.randn(Float32, 4)
cache = prepare_gradient_cache(f, A, x)
_, (_, ∂A, ∂x) = value_and_gradient!!(cache, f, A, x)

# norm², dot, mean — same pattern
f = x -> norm(x)^2
f = (x, y) -> dot(x, y)
f = x -> mapreduce(abs2, +, x) / length(x)

# complex inputs work too
f = A -> real(sum(A * adjoint(A)))
```

--------------------------------

### Get FData Type for a Primitive Type

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/understanding_mooncake/rule_system.md

Determines the `fdata_type` for a given type `T`. This type is used internally by Mooncake.jl to propagate data forwards and backwards during automatic differentiation, distinguishing between memory-identified and value-identified data.

```julia
```julia
Mooncake.fdata_type(T)
```
```

--------------------------------

### Differentiate a Simple Function with Custom Tangent

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/custom_tangent_type.md

Demonstrates differentiating the `f1` function after implementing the necessary custom tangent rules. It shows the preparation of the gradient cache and the calculation of value and gradient.

```julia
a = A(1.0)
cache = Mooncake.prepare_gradient_cache(f1, a)
val, (_, grad) = Mooncake.value_and_gradient!!(cache, f1, a)
```

--------------------------------

### Example of a Self-Referential Type Declaration

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/known_limitations.md

This Julia code defines a mutable struct 'A' that refers to itself directly in its definition, illustrating a scenario that can cause stack overflow errors with Mooncake.jl's default tangent type implementation.

```julia
mutable struct A
    x::Float64
    a::A
    function A(x::Float64)
        a = new(x)
        a.a = a
        return a
    end
end
```

--------------------------------

### Test a Rule with Mooncake.jl

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/utilities/debugging_and_mwes.md

Use `test_rule` to check if automatic differentiation runs and produces correct, performant answers without manual tangent generation. This example demonstrates an error case due to unsafe casting.

```julia
f(x) = Core.bitcast(Float64, x)
Mooncake.TestUtils.test_rule(Random.Xoshiro(123), f, 3; is_primitive=false)
```

--------------------------------

### Differentiable Type Parameters

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/known_limitations.md

This example illustrates a silent correctness issue where differentiating a function that uses differentiable type parameters can result in a zero tangent. This is considered a pathological use case and is unlikely to cause practical issues.

```julia
struct T{x} end

@noinline getparam(::T{x}) where {x} = x

mysquare(x) = getparam(T{x}())^2
```

```julia
cache = Mooncake.prepare_derivative_cache(mysquare, 3.0);
```

```julia
Mooncake.value_and_derivative!!(cache, Mooncake.zero_dual(mysquare), Mooncake.Dual(3.0, 1.0))
```

--------------------------------

### Run GitHub Actions Locally with act

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/running_tests_locally.md

Execute GitHub Actions workflows locally using the `act` tool. Replace `{workflow}` with the specific workflow file name.

```bash
act -W .github/workflows/{workflow}.yml
```

--------------------------------

### Compute Value and Gradient with Prepared Input

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

Obtain both the function's value and its gradient simultaneously using `DI.value_and_gradient` with a prepared object.

```julia
DI.value_and_gradient(f, prep, backend, x)
```

--------------------------------

### Get Julia SSA-form IR by Type

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/ir_representation.md

Use `Base.code_ircode_by_type` to retrieve the SSA-form IR for a given function signature. The output includes the IR statements, where each statement is associated with a unique SSA number (e.g., %1, %2). Arguments are represented as `_n` and type information is available for statements and uses.

```jldoctest
julia> function foo(x)
           y = sin(x)
           z = cos(y)
           return z
       end
foo (generic function with 1 method)

julia> signature = Tuple{typeof(foo), Float64}
Tuple{typeof(foo), Float64}

julia> Base.code_ircode_by_type(signature)[1][1]
2 1 ─ %1 =    invoke sin(_2::Float64)::Float64
3 │   %2 =    invoke cos(%1::Float64)::Float64
4 └──      return %2
```

--------------------------------

### Implement rrule!! or build_primitive_rrule

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/utilities/defining_rules.md

Implement a method for either `Mooncake.rrule!!` or `Mooncake.build_primitive_rrule` after defining `is_primitive`. These are core functions for defining custom differentiation rules.

```julia
Mooncake.rrule!!
```

```julia
Mooncake.build_primitive_rrule
```

--------------------------------

### Inspect Forward Mode IR and World Age Info

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/advanced_debugging.md

Capture IR for forward mode AD using `inspect_ir` with `mode=:forward`. `show_world_info` can then be used to inspect the world age at which each stage was compiled, which is crucial for debugging stale code.

```julia
# Forward mode
ins = inspect_ir(demo_fn, 1.0; mode=:forward)

# World age info (useful for debugging stale code)
show_world_info(ins)
```

--------------------------------

### Calculating Value and Gradient

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/interface.md

Demonstrates how to compute the value and gradient of a function with custom types. It shows the default behavior with `friendly_tangents = false` and the more readable output when `friendly_tangents = true`.

```APIDOC
## `value_and_gradient!!` with `friendly_tangents = false`

### Description
Computes the primal value and gradient of a function using Mooncake's native API. When `friendly_tangents` is false (default), gradients for custom structures are represented using `Mooncake.Tangent` types.

### Method
`value_and_gradient!!(cache, g, x_eval)`

### Parameters
- `cache`: The pre-computed gradient cache.
- `g`: The function for which to compute the gradient.
- `x_eval`: The point at which to evaluate the function and its gradient.

### Request Example
```julia
import Mooncake as MC

struct SimplePair
    x1::Float64
    x2::Float64
end

g(x::SimplePair) = x.x1^2 + x.x2^2
x_eval = SimplePair(1.0, 2.0)

cache = MC.prepare_gradient_cache(g, x_eval)
val, grad = MC.value_and_gradient!!(cache, g, x_eval)
```

### Response
- `val`: The computed value of the function `g` at `x_eval`.
- `grad`: The computed gradient. For custom types, this is typically wrapped in a `Tangent` object.

## `value_and_gradient!!` with `friendly_tangents = true`

### Description
Computes the primal value and gradient, returning gradients in a more readable format when `friendly_tangents` is set to true.

### Method
`value_and_gradient!!(cache, g, x_eval; config=MC.Config(friendly_tangents=true))

### Parameters
- `cache`: The pre-computed gradient cache.
- `g`: The function for which to compute the gradient.
- `x_eval`: The point at which to evaluate the function and its gradient.
- `config`: Configuration object, with `friendly_tangents=true` enabling readable gradients.

### Request Example
```julia
import Mooncake as MC

struct SimplePair
    x1::Float64
    x2::Float64
end

g(x::SimplePair) = x.x1^2 + x.x2^2
x_eval = SimplePair(1.0, 2.0)

cache = MC.prepare_gradient_cache(g, x_eval; config=MC.Config(friendly_tangents=true))
val, grad = MC.value_and_gradient!!(cache, g, x_eval)
```

### Response
- `val`: The computed value of the function `g` at `x_eval`.
- `grad`: The computed gradient, returned as a more user-friendly structure (e.g., a NamedTuple for `SimplePair`).

## `value_and_gradient!!` with `args_to_zero`

### Description
Allows specifying which arguments' tangents should be zeroed during gradient computation, useful for optimizing performance when certain arguments are constant.

### Method
`value_and_gradient!!(cache, g, x_eval; args_to_zero=(false, true))

### Parameters
- `cache`: The pre-computed gradient cache.
- `g`: The function for which to compute the gradient.
- `x_eval`: The point at which to evaluate the function and its gradient.
- `args_to_zero`: A tuple of booleans indicating whether to zero the tangent for each argument.

### Request Example
```julia
import Mooncake as MC

struct SimplePair
    x1::Float64
    x2::Float64
end

g(x::SimplePair) = x.x1^2 + x.x2^2
x_eval = SimplePair(1.0, 2.0)

cache = MC.prepare_gradient_cache(g, x_eval; config=MC.Config(friendly_tangents=true))
val, grad = MC.value_and_gradient!!(
    cache,
    g,
    x_eval;
    args_to_zero = (false, true),
)
```
```

--------------------------------

### Build Reverse Rule with Interpreter and Signature

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/reverse_mode_design.md

The core method for building reverse rules, operating on an interpreter and either a signature or a method instance. It handles custom rules or generates IR for differentiation.

```julia
build_rrule(interp::MooncakeInterpreter{C}, sig_or_mi; debug_mode=false)
```

--------------------------------

### Build Reverse Rule Function Signature

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/reverse_mode_design.md

This function is the entry point for building reverse rules. It accepts arguments and optionally a debug mode flag, extracting types to call the main rule building method.

```julia
build_rrule(args...; debug_mode=false)
```

--------------------------------

### Access Instructions within a BBlock

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/ir_representation.md

Illustrates how to retrieve the instructions associated with a specific basic block. Instructions are stored in the 'insts' field of a BBlock as Core.Compiler.NewInstruction objects.

```jldoctest
julia> using Mooncake.BasicBlockCode: ID # to improve printing

julia> bb_ir.blocks[3].insts[1]
Compiler.NewInstruction(:(Base.add_int(ID(2), 1)), Int64, Compiler.NoCallInfo(), (0, 0, 0), 0x00002478)
```

--------------------------------

### Compute Gradient Efficiently with Prepared Input

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/tutorial.md

Calculate the gradient of function `f` with respect to `x` using the prepared object for optimal speed.

```julia
DI.gradient(f, prep, backend, x)
```

--------------------------------

### Comprehensive Tangent Testing Utility

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/tangents.md

This utility function runs all individual tangent interface tests at once. It provides a convenient way to verify the complete tangent type implementation.

```julia
Mooncake.TestUtils.test_data
```

--------------------------------

### Mooncake Configuration

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/interface.md

Access and manage configuration settings for Mooncake.

```APIDOC
## Mooncake.Config

### Description
Provides access to the configuration object for the Mooncake library.

### Usage
```julia
config = Mooncake.Config
```
```

--------------------------------

### Include Test File for Iterative Testing

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/running_tests_locally.md

Include a specific test file to run its tests. This is part of the recommended development workflow for iterative testing.

```julia
include("test/rules/new.jl")
```

--------------------------------

### Testing Method Addition with Generated Function

Source: https://github.com/chalk-lab/mooncake.jl/blob/main/docs/src/developer_documentation/misc_internals_notes.md

Demonstrates that the generated function correctly handles additions to its methods, as the recursion is not in the generated function's body.

```julia
good_tangent_type(Tuple{Float32})
```