### Install Zygote.jl

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/index.md

Install Zygote.jl using the Julia package manager.

```julia
]
add Zygote

```

--------------------------------

### Zygote Pullback Example

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Demonstrates how to use Zygote.pullback to get the function's result and its corresponding pullback function for gradient computation.

```jldoctest
julia> using Zygote

julia> y, back = Zygote.pullback(sin, 0.5);

julia> y
0.479425538604203
```

--------------------------------

### Gradient with Control Flow (Loops)

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/index.md

Zygote fully supports control flow, including loops. This example demonstrates calculating the gradient of a function with a `for` loop.

```jldoctest
julia> function pow(x, n)
         r = 1
         for i = 1:n
           r *= x
         end
         return r
       end
pow (generic function with 1 method)

gradient(x -> pow(x, 3), 5)
(75.0,)
```

--------------------------------

### Iterate Through Gradients in Zygote

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/utils.md

Provides an example of iterating through a gradient dictionary using `pairs` to access both the parameter and its corresponding gradient.

```julia
for (p, g) in pairs(gs)
  # do something with parameter `p` and corresponding gradient `g`
end
```

--------------------------------

### Differentiate External Libraries (Colors)

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/index.md

Zygote can compute gradients for functions from external Julia libraries, even if they were not designed for automatic differentiation. This example uses the `Colors` package.

```julia
julia> using Colors

colordiff(RGB(1, 0, 0), RGB(0, 1, 0))
86.60823557376344

gradient(colordiff, RGB(1, 0, 0), RGB(0, 1, 0))
((r = 0.4590887719632896, g = -9.598786801605689, b = 14.181383399012862), (r = -1.7697549557037275, g = 28.88472330558805, b = -0.044793892637761346))
```

--------------------------------

### Gradient with Control Flow (Recursion)

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/index.md

Zygote supports recursion, allowing gradients to be computed for recursive functions. This example shows differentiation of a recursive power function.

```jldoctest
julia> pow2(x, n) = n <= 0 ? 1 : x*pow2(x, n-1)
pow2 (generic function with 1 method)

gradient(x -> pow2(x, 3), 5)
(75.0,)
```

--------------------------------

### Explicit Parameter Handling in Models

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/index.md

When differentiating models, parameters can be explicitly passed as arguments. This example shows a `linear` function where parameters `θ` are an argument.

```julia
julia> linear(θ, x) = θ[:W] * x .+ θ[:b]
linear (generic function with 1 method)

julia> x = rand(5);

julia> θ = Dict(:W => rand(2, 5), :b => rand(2))
Dict{Any,Any} with 2 entries:
  :b => [0.0430585, 0.530201]
  :W => [0.923268 … 0.589691]

# Alternatively, use a named tuple or struct rather than a dict.

```

--------------------------------

### Zygote _pullback with Closures

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Illustrates how `_pullback` handles closures. When the function being differentiated captures variables (like `a` in the example), the pullback returns gradients for these captured variables along with the input gradients.

```julia
f = let a = 3; x -> x*a; end
y, back = Zygote._pullback(f, 2)
back(1)
# ((a = 2,), 3)
```

--------------------------------

### Define Power Function with Loop

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Implements a power function `pow(x, n)` using a loop for repeated multiplication. This serves as an example for deriving pullbacks with control flow.

```julia
function pow(x, n) # x^n
  r = 1
  for _ = 1:n
    r *= x
  end
  return r
end
```

--------------------------------

### Gradient Algebra with IdDict in Zygote

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/utils.md

Shows how to use IdDict for gradient algebra, which is necessary for GPU computations, and how to add new gradients to an existing gradient dictionary.

```julia
# gradients and IdDict interact nicely
# note that an IdDict must be used for gradient algebra on the GPU
gs .+= IdDict(p => randn(size(p)) for p in keys(gs))
```

--------------------------------

### Calculate Gradient Contribution in Zygote.jl

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Use `gradient` to compute the contribution of a specific term to the total gradient. This example shows the gradient contribution from the first `a` in `a * a`.

```julia
# Note: shows gradient contribution, not total gradient
gradient(2) do a
    @showgrad(a) * a  # gradient contribution from first a is 2
end
# Prints: ∂(a) = 2
# Output: (4,)
```

--------------------------------

### Clip Gradients In-Place with Zygote

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/utils.md

Illustrates how to clip gradient values in-place using `clamp!` and `foreach` for efficient gradient manipulation.

```julia
# clip gradients in-place
foreach(x -> clamp!(x, -0.1, 0.1), gs)
```

--------------------------------

### Manual Gradient Calculation with Pullback

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Illustrates how to manually compute the gradient of a function using the pullback function obtained from Zygote.pullback, by providing a derivative of 1.

```julia
dsin(x) = (sin(x), ȳ -> (ȳ * cos(x),))
```

```julia
function gradsin(x)
         _, back = dsin(x)
         back(1)
       end
```

```jldoctest
julia> gradsin(0.5)
(0.8775825618903728,)

julia> cos(0.5)
0.8775825618903728
```

--------------------------------

### Debugging with `_pullback`

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Demonstrates how to use `Zygote._pullback` for debugging by isolating function calls and inspecting intermediate results and gradients.

```julia
bad(x) = x

Zygote.@adjoint bad(x) = x, _ -> error("bad")

foo(x) = bad(sin(x))

gradient(foo, 1) # error!

julia> y, back = Zygote._pullback(foo, 1);

julia> back(1) # just make up a value here, it just needs to look similar to `y`
ERROR: bad

# Ok, so we try functions that foo calls

julia> y, back = Zygote._pullback(sin, 1);

julia> back(1)
(nothing, 0.5403023058681398)
```

--------------------------------

### Define Composite Function for Differentiation

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

A simple composite function `foo` that calls `sin` and `cos` sequentially. This serves as an example for deriving pullbacks of composed functions.

```julia
function foo(x)
  a = sin(x)
  b = cos(a)
  return b
end
```

--------------------------------

### Accumulate Gradients with Zygote

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/utils.md

Demonstrates how to accumulate gradients from multiple computations using Zygote's gradient function and the '+' operator for Params objects.

```julia
using Zygote, Test

w, x1, x2, b = rand(2), rand(2), rand(2), rand(2)

gs1 = gradient(() -> sum(tanh.(w .* x1 .+ b)), Params([w, b]))
gs2 = gradient(() -> sum(tanh.(w .* x2 .+ b)), Params([w, b]))

# accumulate gradients
gs = gs1 .+ gs2
@test gs[w] ≈ gs1[w] + gs2[w]
@test gs[b] ≈ gs1[b] + gs2[b]
```

--------------------------------

### Gradient with Dictionary of Functions

Source: https://github.com/fluxml/zygote.jl/blob/master/README.md

Illustrates how Zygote can handle gradients for functions selected from a dictionary based on user input. Requires interactive input for `readline()`.

```julia
fs = Dict("sin" => sin, "cos" => cos, "tan" => tan);

gradient(x -> fs[readline()](x), 1)
```

--------------------------------

### Hook for Gradient Inspection/Modification in Zygote.jl

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Use `Zygote.hook` within a `gradient` call to intercept, inspect, or modify gradients during the backward pass. This example prints the gradient of `a` before it's used in further calculations.

```julia
# hook for gradient inspection/modification
gradient(2, 3) do a, b
    Zygote.hook(a) do ā
        println("Gradient of a: ", ā)
        ā  # Return unchanged
    end * b
end
```

--------------------------------

### Params Set Operations

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Demonstrates combining and copying parameter sets for Zygote.jl. Ensure parameters are initialized before use.

```julia
ps1 = Params([W, b])
ps2 = Params([W])
union(ps1, ps2)      # Combine parameter sets
intersect(ps1, ps2)  # Common parameters

# Copy parameters to/from vectors
x = zeros(length(W) + length(b))
copy!(x, Params([W, b]))  # Flatten params to vector
copy!(Params([W, b]), x)  # Copy vector back to params
```

--------------------------------

### Zygote _pullback for Standard Functions

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Demonstrates the usage of Zygote's lower-level `_pullback` function for a standard function like `sin`. It returns the function's output and a pullback function.

```julia
y, back = Zygote._pullback(sin, 0.5)
back(1)
# (nothing, 0.8775825618903728)
```

--------------------------------

### Clip Gradients with Zygote

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/utils.md

Shows how to clip gradient values using `map` and `clamp` to limit the magnitude of gradients.

```julia
# clip gradients
map(x -> clamp.(x, -0.1, 0.1), gs)
```

--------------------------------

### Gradient Operations in Zygote.jl

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Shows arithmetic, mapping, and iteration over gradients using Zygote.jl. Gradients are returned as dictionary-like containers.

```julia
using Zygote

w, b = rand(2), rand(2)
x1, x2 = rand(2), rand(2)

gs1 = gradient(() -> sum(tanh.(w .* x1 .+ b)), Params([w, b]))
gs2 = gradient(() -> sum(tanh.(w .* x2 .+ b)), Params([w, b]))

# Arithmetic operations on gradients
gs = gs1 .+ gs2          # Add gradients
gs[w]                    # Access combined gradient

gs_diff = gs1 .- gs2     # Subtract gradients
gs_scaled = 2 .* gs1     # Scale gradients
gs_div = gs1 ./ 2        # Divide gradients

# In-place accumulation
gs1 .+= gs2

# Map operations for gradient clipping
clipped = map(x -> clamp.(x, -0.1, 0.1), gs1)

# In-place clipping
foreach(x -> clamp!(x, -0.1, 0.1), gs1)

# Iterate over parameter-gradient pairs
for (p, g) in pairs(gs1)
    println("Gradient for parameter: ", g)
end

# Dictionary interface
keys(gs1)    # Returns Params
values(gs1)  # Returns gradient values
length(gs1)  # Number of parameters

# Merge gradients from different computations
gs_merged = merge!(copy(gs1), gs2)
```

--------------------------------

### Differentiating Safe Square Root with Try-Catch

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/limitations.md

Demonstrates a safe square root function using try-catch. Zygote can differentiate when no error occurs, but fails if the catch block is executed.

```julia
julia> function safe_sqrt(x)
           try
               sqrt(x)
           catch
               0.
           end
       end
safe_sqrt (generic function with 1 method)

julia> gradient(safe_sqrt, 4.)
(0.25,)

julia> val, pull = pullback(safe_sqrt, -1.)
(0.0, Zygote.var"#76#77"{Zygote.Pullback{Tuple{typeof(safe_sqrt), Float64}, Any}}(∂(safe_sqrt)))

julia> pull(1.)
ERROR: Can't differentiate function execution in catch block at #= REPL[2]:3 =#.
Stacktrace:

```

--------------------------------

### Basic Function Differentiation

Source: https://github.com/fluxml/zygote.jl/blob/master/README.md

Demonstrates basic function differentiation using Zygote.jl. Ensure Zygote is added and imported before use.

```julia
using Zygote

f(x) = 5x + 3

f(10), f'(10)
```

--------------------------------

### Debugging Gradients with `@showgrad`

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Introduces the `@showgrad` macro for displaying gradient values at specific points within a computation graph. Useful for debugging and understanding gradient flow.

```julia
using Zygote

# @showgrad - display gradient at a point in computation
gradient(2, 3) do a, b
    @showgrad(a) * b
end
```

--------------------------------

### Differentiate Function with Macro

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Annotate functions with `@differentiable` to make them compatible with Zygote's differentiation.

```julia
@differentiable foo(x) = sin(cos(x))
```

--------------------------------

### LLVM IR for Gradient Function

Source: https://github.com/fluxml/zygote.jl/blob/master/README.md

Shows the Low-Level Virtual Machine (LLVM) Intermediate Representation (IR) generated for the gradient of a simple function. This is useful for understanding the compiler's output.

```julia
@code_llvm f'(10)
```

--------------------------------

### Demonstrate Nesting Levels with grad Function

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Shows how the `nestlevel` function reports different nesting depths during forward and backward passes of differentiation. A helper `grad` function is defined for convenience.

```julia
function f(x)
         println(nestlevel(), " levels of nesting")
         return x
       end

grad(f, x) = gradient(f, x)[1]

f(1);

grad(f, 1);

grad(x -> x*grad(f, x), 1);
```

--------------------------------

### Differentiate with Checkpointing

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Calculates the gradient of the sine function using the `checkpoint` mechanism. This demonstrates how checkpointing can be used for memory optimization during differentiation.

```jldoctest
gradient(x -> checkpoint(sin, x), 1)
```

--------------------------------

### Execute and Type-inspect Zygote's pullback

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/profiling.md

After obtaining the pullback function using `_pullback`, you can execute it and use `code_typed` to inspect its generated code. This is useful for understanding how gradients are computed.

```julia
julia> y, back = _pullback(Context(), add, 1, 2)
y, back
(3, ∂(add))

julia> @code_typed back(1)
CodeInfo(
1 ─ %1 = (Base.mul_int)(Δ, 1)::Int64
│   %2 = (Base.mul_int)(Δ, 1)::Int64
│   %3 = (Zygote.tuple)(nothing, %1, %2)::PartialTuple(Tuple{Nothing,Int64,Int64}, Any[Const(nothing, false), Int64, Int64])
└──      return %3
) => Tuple{Nothing,Int64,Int64}
```

--------------------------------

### Complex Number Differentiation (Holomorphic)

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Demonstrates Zygote's handling of complex numbers, focusing on holomorphic functions where derivatives are computed via the real part. Verifies with Cauchy-Riemann equations.

```julia
using Zygote

# Gradient for real-valued functions of complex arguments
gradient(c -> abs2(c), 1+2im)
```

```julia
using Zygote

# Holomorphic function derivatives via real part
gradient(x -> real(log(x)), 1+2im)[1] |> conj
```

```julia
using Zygote

# Verify holomorphicity via Cauchy-Riemann equations
-im * gradient(x -> imag(log(x)), 1+2im)[1] |> conj
```

--------------------------------

### Forward-Mode Differentiation with `forwarddiff`

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Instructs Zygote to use forward-mode differentiation for specific computations using `forwarddiff`. Beneficial when input dimensions are small relative to output, or for performance.

```julia
using Zygote

# Forward mode for scalar operations
function pow(x, n)
    r = one(x)
    for i = 1:n
        r *= x
    end
    return r
end

gradient(5) do x
    forwarddiff(x) do x
        pow(x, 2)
    end
end
```

```julia
using Zygote

# Note: closed-over values don't get gradients in forwarddiff
gradient(2, 3) do a, b
    forwarddiff(a) do a
        a * b  # b is closed over, gets nothing
    end
end
```

```julia
using Zygote

# Pass all values explicitly to get all gradients
gradient(2, 3) do a, b
    forwarddiff([a, b]) do (a, b)
        a * b
    end
end
```

```julia
using Zygote

# Control chunk size for performance tuning
forwarddiff(x; chunk_threshold=8) do x
    sum(x.^2)
end
```

--------------------------------

### Gradient Hook for Debugging

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Demonstrates using a gradient hook to inspect gradient values during computation. The `@show` macro within the hook prints the gradient when it's computed.

```julia
hook(f, x) = x
@adjoint hook(f, x) = x, x̄ -> (nothing, f(x̄))

# Use hook for debugging
gradient((a, b) -> hook(ā -> (@show ā; ā), a) * b, 2, 3)
```

--------------------------------

### Complex Number Differentiation (Non-Holomorphic)

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Explains and demonstrates Wirtinger derivatives for non-holomorphic functions involving complex numbers. Shows how to compute both the Wirtinger and complex derivatives.

```julia
using Zygote

# For non-holomorphic functions, use Wirtinger derivatives
function wirtinger(f, x)
    y, back = Zygote.pullback(f, x)
    du = back(1)[1]
    dv = back(im)[1]
    (du' + im*dv')/2, (du + im*dv)/2
end

wirtinger(x -> 3x^2 + 2x + 1, 1+2im)
```

```julia
using Zygote

# For non-holomorphic functions, use Wirtinger derivatives
function wirtinger(f, x)
    y, back = Zygote.pullback(f, x)
    du = back(1)[1]
    dv = back(im)[1]
    (du' + im*dv')/2, (du + im*dv)/2
end

wirtinger(x -> abs2(x), 1+2im)
```

--------------------------------

### Differentiate with Gradient Hook and Show Gradient

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Illustrates using a gradient hook to print the gradient value during backpropagation. The `@show` macro within the hook's pullback function captures and displays the gradient.

```jldoctest
gradient((a, b) -> hook(ā -> @show(ā), a)*b, 2, 3)
```

--------------------------------

### Gradient Accumulation Error Handling in Zygote

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/utils.md

Demonstrates an error case where gradient accumulation fails due to mismatched parameter key sets between gradient objects.

```julia
# note that gradients must be w.r.t. to the same parameter key set
gs3 = gradient(() -> sum(tanh.(w .* x2)), Params([w]))
# gs3 does not have the key b
@test_throws ArgumentError gs1 .+ gs3
```

--------------------------------

### Params and Implicit Parameters

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Container for implicit parameters enabling gradient computation for a zero-argument function with respect to external variables. Useful for large models with many parameters.

```APIDOC
## Params and Implicit Parameters

### Description
Container for implicit parameters enabling gradient computation for a zero-argument function with respect to external variables. Useful for large models with many parameters.

### Method
`Params`

### Parameters
- `parameters` (Array of Arrays/Tensors) - A collection of parameters (e.g., weight matrices, bias vectors) for which gradients are to be computed.

### Usage
`Params` is typically used with the `gradient` function when the function to be differentiated does not explicitly take the parameters as arguments but relies on them being in the outer scope.

### Request Example
```julia
using Zygote

# Define model parameters
W = rand(2, 5)
b = rand(2)

# Define model using external parameters
linear(x) = W * x .+ b

# Compute gradients with respect to implicit parameters
x = rand(5)
grads = gradient(() -> sum(linear(x)), Params([W, b]))
# grads is of type Grads(...)

# Access gradients using arrays as keys
grads[W]  # Returns 2×5 gradient matrix
grads[b]  # Returns 2-element gradient vector

# Check if parameter has gradient
haskey(grads, W)  # true

# Multiple parameters in computation
x_param = [1 2 3; 4 5 6]
y_param = [7, 8]
z_val = [1, 10, 100]

g = gradient(Params([x_param, y_param])) do
    sum(x_param .* y_param .* z_val')
end
g[x_param]  # 2×3 Matrix
g[y_param]  # Vector
```
```

--------------------------------

### Inspect Intermediate Representation (IR)

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Use Zygote's `@code_ir` macro to see the code represented in Julia's SSA form.

```julia
julia> Zygote.@code_ir foo(1)
1 1 ─ %1 = (Main.bar)(_2)::Any
  │   %2 = (Main.baz)(%1)::Any
  └──      return %2
```

--------------------------------

### Differentiating Implicitly Used Parameters

Source: https://github.com/fluxml/zygote.jl/blob/master/README.md

Demonstrates how Zygote can compute gradients with respect to parameters that are implicitly used within a model, not just direct function arguments. Requires parameters to be wrapped in `Params`.

```julia
W, b = rand(2, 3), rand(2);

predict(x) = W*x .+ b;

g = gradient(Params([W, b])) do
         sum(predict([1,2,3]))
       end

g[W], g[b]
```

--------------------------------

### Buffer for Mutable Array Operations

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Shows how to use `Zygote.Buffer` to create and mutate arrays within differentiated code. This is essential for incrementally building arrays during gradient computation.

```julia
using Zygote

# Function that builds an array with mutation
function vstack(xs)
    buf = Buffer(xs, length(xs), 5)  # Create buffer similar to xs
    for i = 1:5
        buf[:, i] = xs  # Mutate buffer
    end
    return copy(buf)  # Convert back to array
end

vstack([1, 2, 3])
```

```julia
using Zygote

# Gradients work through Buffer operations
gradient(x -> sum(vstack(x)), [1, 2, 3])
```

```julia
using Zygote

# Buffer from existing array
function accumulate_buffer(x)
    buf = Zygote.bufferfrom(x)
    for i in 2:length(buf)
        buf[i] += buf[i-1]
    end
    return copy(buf)
end
```

```julia
using Zygote

# Push to buffer
function collect_results(f, xs)
    buf = Buffer(Float64[], length(xs))
    for x in xs
        push!(buf, f(x))
    end
    return copy(buf)
end
```

--------------------------------

### Differentiate with Checkpointing and Side Effects

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Demonstrates checkpointing with a function that has side effects (printing). The output shows that the function is executed twice, once during the forward pass and once during the re-computed pullback.

```jldoctest
foo(x) = (println(x); sin(x))

gradient(x -> checkpoint(foo, x), 1)
```

--------------------------------

### Debugging Zygote Pullbacks

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Use Zygote._pullback to manually inspect gradient calculations. This is useful for narrowing down test cases when Zygote is not behaving as expected.

```julia
julia> y, back = Zygote._pullback(bad, 1);

julia> back(1) # ok, here's our issue. Lather, rinse, repeat.
ERROR: bad
```

--------------------------------

### Default Adjoint for getfield on Point

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Demonstrates Zygote's default adjoint behavior for accessing fields of a custom type using dot notation. Shows that only the differentiated field receives a gradient.

```julia
gradient(a -> a.x, Point(1, 2))
```

--------------------------------

### Wirtinger Derivatives for a Polynomial

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/complex.md

Applies the `wirtinger` function to a polynomial `3x^2 + 2x + 1` with a complex input `1+2im`. Demonstrates the Wirtinger derivatives for a holomorphic function.

```jldoctest
julia> wirtinger(x -> 3x^2 + 2x + 1, 1+2im)
(8.0 + 12.0im, 0.0 + 0.0im)
```

--------------------------------

### Inspect Adjoint Code Generation

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Use Zygote's `@code_adjoint` macro to inspect the generated code for the reverse-mode differentiation pass.

```julia
julia> Zygote.@code_adjoint foo(1)
1 1 ─ %1  = (Zygote._pullback)(_2, Zygote.unwrap, Main.bar)::Any
  │   %2  = (Base.getindex)(%1, 1)::Any
  │         (Base.getindex)(%1, 2)::Any
  │   %4  = (Zygote._pullback)(_2, %2, _4)::Any
  │   %5  = (Base.getindex)(%4, 1)::Any
  │         (Base.getindex)(%4, 2)::Any
  │   %7  = (Zygote._pullback)(_2, Zygote.unwrap, Main.baz)::Any
  │   %8  = (Base.getindex)(%7, 1)::Any
  │         (Base.getindex)(%7, 2)::Any
  │   %10 = (Zygote._pullback)(_2, %8, %5)::Any
  │   %11 = (Base.getindex)(%10, 1)::Any
  │         (Base.getindex)(%10, 2)::Any
  └──       return %11
  1 ─ %1  = Δ()::Any
1 │   %2  = (@12)(%1)::Any
  │   %3  = (Zygote.gradindex)(%2, 1)::Any
  │   %4  = (Zygote.gradindex)(%2, 2)::Any
  │         (@9)(%3)::Any
  │   %6  = (@6)(%4)::Any
  │   %7  = (Zygote.gradindex)(%6, 1)::Any
  │   %8  = (Zygote.gradindex)(%6, 2)::Any
  │         (@3)(%7)::Any
  │   %10 = (Zygote.tuple)(nothing, %8)::Any
  └──       return %10
, [1])
```

--------------------------------

### Custom Adjoint for Point Type

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Illustrates defining custom adjoints for user-defined types (Point) and their associated functions (width, height). Ensures gradients are correctly propagated through custom structures.

```julia
struct Point
    x::Float64
    y::Float64
end

width(p::Point) = p.x
height(p::Point) = p.y

@adjoint width(p::Point) = p.x, x̄ -> (Point(x̄, 0),)
@adjoint height(p::Point) = p.y, ȳ -> (Point(0, ȳ),)
```

--------------------------------

### Defining Custom Gradients with ChainRulesCore

Source: https://github.com/fluxml/zygote.jl/blob/master/README.md

Shows how to define custom gradient rules for a function by extending `ChainRulesCore.jl`'s `rrule`. This allows for specialized differentiation behavior.

```julia
using ChainRulesCore

add(a, b) = a + b

function ChainRulesCore.rrule(::typeof(add), a, b)
           add_pb(dy) = (NoTangent(), dy, dy)
           return add(a, b), add_pb
       end
```

--------------------------------

### Handle Implicit Parameters with Zygote

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Use `Params` to manage external variables as implicit parameters for gradient computation, especially useful for large models. Gradients are returned as a `Grads` object.

```julia
using Zygote

# Define model parameters
W = rand(2, 5)
b = rand(2)

# Define model using external parameters
linear(x) = W * x .+ b

# Compute gradients with respect to implicit parameters
x = rand(5)
grads = gradient(() -> sum(linear(x)), Params([W, b]))
# Grads(...)

# Access gradients using arrays as keys
grads[W]  # Returns 2×5 gradient matrix
grads[b]  # Returns 2-element gradient vector

# Check if parameter has gradient
haskey(grads, W)  # true

# Multiple parameters in computation
x = [1 2 3; 4 5 6]
y = [7, 8]
z = [1, 10, 100]

g = gradient(Params([x, y])) do
    sum(x .* y .* z')
end
g[x]  # 2×3 Matrix
g[y]  # Vector
```

--------------------------------

### Gradient with Array Arguments

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/index.md

Zygote supports differentiating functions where arguments are arrays. Ensure the function returns a scalar value for optimization purposes.

```julia
julia> W = rand(2, 3); x = rand(3);

gradient(W -> sum(W*x), W)[1]
2×3 Array{Float64,2}:
 0.0462002  0.817608  0.979036
 0.0462002  0.817608  0.979036

gradient(x -> 3x^2 + 2x + 1, 1//4)
(7//2,)
```

--------------------------------

### Gradient of Real Part of Logarithm of Conjugate (Non-Holomorphic)

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/complex.md

Demonstrates a non-holomorphic case using the real part of the logarithm of the complex conjugate. The resulting gradients differ from the holomorphic case.

```jldoctest
julia> gradient(x -> real(log(x')), 1+2im)[1] |> conj
0.2 - 0.4im
```

--------------------------------

### Calculate Gradient with Explicit Parameters (Dict)

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/index.md

Calculates the gradient of a sum of a linear function with respect to its parameters, which are provided as a dictionary.

```julia
julia> θ̄ = gradient(θ -> sum(linear(θ, x)), θ)[1]
Dict{Any,Any} with 2 entries:
  :b => [1.0, 1.0]
  :W => [0.628998 … 0.433006]
```

--------------------------------

### Gradient Hook for Gradient Clipping

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Implements a gradient hook to apply a function (e.g., `clamp`) to gradients during backpropagation. Useful for stabilizing training by limiting gradient magnitudes.

```julia
hook(f, x) = x
@adjoint hook(f, x) = x, x̄ -> (nothing, f(x̄))

# Use hook for gradient clipping
gradient((a, b) -> hook(x -> clamp(x, -1, 1), a) * b, 10, 3)
```

--------------------------------

### Legacy @adjoint Macro

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Zygote's native macro for defining custom adjoints (legacy).

```APIDOC
## @adjoint Macro (Legacy)

### Description
Zygote's native macro for defining custom adjoints. While ChainRulesCore is preferred, @adjoint provides direct control over Zygote's behavior.

### Method
`@adjoint`

### Endpoint
N/A

### Parameters
N/A

### Request Example
```julia
using Zygote
using Zygote: @adjoint

# Example usage of @adjoint would go here, but is not provided in the source text.
```

### Response
Custom adjoint definitions.
```

--------------------------------

### Parameter Set Operations

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Operations for managing parameter sets, including union, intersection, and copying to/from vectors.

```APIDOC
## Params Set Operations

### Description
Operations for combining and manipulating parameter sets.

### Method
N/A (Function calls)

### Endpoint
N/A

### Parameters
N/A

### Request Example
```julia
using Zygote

W = rand(2, 2)
b = rand(2)

ps1 = Params([W, b])
ps2 = Params([W])

union(ps1, ps2)      # Combine parameter sets
intersect(ps1, ps2)  # Common parameters

# Copy parameters to/from vectors
x = zeros(length(W) + length(b))
copy!(x, Params([W, b]))  # Flatten params to vector
copy!(Params([W, b]), x)  # Copy vector back to params
```

### Response
N/A
```

--------------------------------

### Differentiate with Gradient Hook

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Demonstrates using the `hook` function with a custom adjoint to apply a function to the gradient during backpropagation. The pullback applies `f` to the gradient of `x`.

```jldoctest
gradient((a, b) -> hook(-, a)*b, 2, 3)
```

--------------------------------

### Summation with Variable Re-binding in Julia

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/limitations.md

Demonstrates accumulating values in a loop by re-binding the same variable name. Be aware that Zygote may have issues if the type of the re-bound value changes.

```julia
function mysum(x::Real, n::Int)
  tot = 0.0
  for i in 1:n
    tot += x^n  # binds symbol `tot` to new value
  end
  return tot
end
```

--------------------------------

### Gradient Checkpointing for Memory Efficiency

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Utilizes `Zygote.checkpointed` to trade computation for memory by recomputing forward passes during the backward pass. Ideal for large models where memory is a constraint.

```julia
using Zygote

# Define an expensive computation
function expensive_layer(x)
    # Normally intermediate values are stored for backward pass
    y = x.^2
    z = sin.(y)
    return sum(z)
end

# With checkpointing - forward pass runs twice, but uses less memory
gradient(x -> Zygote.checkpointed(expensive_layer, x), rand(1000))
```

```julia
using Zygote

# Chain multiple checkpointed operations
function deep_network(x, layers)
    h = x
    for layer in layers
        h = Zygote.checkpointed(layer, h)
    end
    return sum(h)
end
```

```julia
using Zygote

# Verify checkpointing with side effects (runs twice)
function show_execution(x)
    println("Forward pass executed")
    return sin(x)
end

gradient(x -> Zygote.checkpointed(show_execution, x), 1.0)
```

--------------------------------

### Calculate Gradient with Implicit Parameters (Params)

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/index.md

Calculates gradients using implicit parameters by providing a Params object to the gradient function. This method is useful when the loss function does not explicitly take the parameters as arguments.

```julia
julia> W = rand(2, 5); b = rand(2);

julia> linear(x) = W * x .+ b
linear (generic function with 2 methods)

julia> grads = gradient(() -> sum(linear(x)), Params([W, b]))
Grads(...)

julia> grads[W], grads[b] # access gradients using arrays as keys
([0.652543 … 0.683588], [1.0, 1.0])
```

--------------------------------

### Define Explicit Pullbacks for Simple Functions

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Manually define the pullback function `J` for basic mathematical operations like sin, cos, and multiplication. This serves as a foundational step before automating the process.

```julia
J(::typeof(sin), x) = sin(x), ȳ -> ȳ*cos(x)
J(::typeof(cos), x) = cos(x), ȳ -> -ȳ*sin(x)
J(::typeof(*), a, b) = a*b, c̄ -> (b*c̄, a*c̄)
```

--------------------------------

### Overload Width Adjoint for Point

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Customizes the adjoint for the `width` function on `Point` to return a `Point` gradient. Requires Zygote.refresh() after definition.

```julia
@adjoint width(p::Point) = p.x, x̄ -> (Point(x̄, 0),)
```

--------------------------------

### Compute Hessian Matrix with Zygote.jl

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Calculates the Hessian matrix (second derivatives) for scalar-valued functions. Uses forward-over-reverse mode by default. For efficiency, consider using `diaghessian`.

```julia
using Zygote

# Hessian of a function of 2 variables
hessian(x -> x[1] * x[2], randn(2))
# 2×2 Matrix:
#  0.0  1.0
#  1.0  0.0

# Hessian with matrix input (uses linear indexing)
hessian(x -> sum(x.^3), [1 2; 3 4])
# 4×4 diagonal Matrix:
#   6   0   0   0
#   0  18   0   0
#   0   0  12   0
#   0   0   0  24

# Scalar input
hessian(sin, pi/2)
# Output: -1.0

# Reverse-over-reverse mode (usually slower, for comparison)
hessian_reverse(x -> x[1] * x[2], randn(2))
# Same result but computed differently

# Diagonal of Hessian only (more efficient)
diaghessian(x -> sum(x.^3), [1 2; 3 4])[1]
# 2×2 Matrix:
#   6  12
#  18  24

# Multiple arguments
diaghessian((x, y) -> sum(x .* y .* y'), [1 22; 333 4], [0.5, 0.666])
# Returns tuple of diagonal Hessians for each argument

# Two scalar arguments
diaghessian(atan, 1, 2)
# Output: (-0.16, 0.16)
```

--------------------------------

### Overload Point Constructor Adjoint

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/adjoints.md

Customizes the adjoint for the `Point` constructor to handle gradient inputs as `Point` objects. This is recommended when defining custom field access gradients.

```julia
@adjoint Point(a, b) = Point(a, b), p̄ -> (p̄.x, p̄.y)
```

--------------------------------

### Differentiate Custom Structs

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/index.md

Zygote allows differentiation through custom Julia types. Define necessary operations (like addition, subtraction) and the function to be differentiated.

```julia
import Base: +, -

struct Point
  x::Float64
  y::Float64
end

a::Point + b::Point = Point(a.x + b.x, a.y + b.y)
a::Point - b::Point = Point(a.x - b.x, a.y - b.y)
dist(p::Point) = sqrt(p.x^2 + p.y^2)

julia> a = Point(1, 2)
Point(1.0, 2.0)

julia> b = Point(3, 4)
Point(3.0, 4.0)

julia> dist(a + b)
7.211102550927978

julia> gradient(a -> dist(a + b), a)[1]
(x = 0.5547001962252291, y = 0.8320502943378437)
```

--------------------------------

### Type-inspect Zygote's _pullback

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/profiling.md

Use `_pullback` with `Context()` and `code_typed` to inspect the generated adjoint code for a function. This shows the derived adjoint code or custom adjoint if available.

```julia
julia> using Zygote: Context, _pullback

julia> add(a, b) = a+b

julia> @code_typed _pullback(Context(), add, 1, 2)
CodeInfo(
1 ─ %1 = (Base.getfield)(args, 1)::Int64
│   %2 = (Base.getfield)(args, 2)::Int64
│   %3 = (Base.add_int)(%1, %2)::Int64
│   %4 = (Base.tuple)(%3, $(QuoteNode(∂(add))))::PartialTuple(Tuple{Int64,typeof(∂(add))}, Any[Int64, Const(∂(add), false)])
└──      return %4
) => Tuple{Int64,typeof(∂(add))}
```

--------------------------------

### Hessian Computation

Source: https://context7.com/fluxml/zygote.jl/llms.txt

Computes the Hessian matrix (second derivatives) for scalar-valued functions.

```APIDOC
## hessian

### Description
Computes the Hessian matrix (second derivatives) for scalar-valued functions. Uses forward-over-reverse mode (ForwardDiff over Zygote) by default.

### Method
`hessian`, `diaghessian`

### Endpoint
N/A

### Parameters
N/A

### Request Example
```julia
using Zygote

# Hessian of a function of 2 variables
hessian(x -> x[1] * x[2], randn(2))
# 2×2 Matrix:
#  0.0  1.0
#  1.0  0.0

# Hessian with matrix input (uses linear indexing)
hessian(x -> sum(x.^3), [1 2; 3 4])
# 4×4 diagonal Matrix:
#   6   0   0   0
#   0  18   0   0
#   0   0  12   0
#   0   0   0  24

# Scalar input
hessian(sin, pi/2)
# Output: -1.0

# Reverse-over-reverse mode (usually slower, for comparison)
hessian_reverse(x -> x[1] * x[2], randn(2))
# Same result but computed differently

# Diagonal of Hessian only (more efficient)
diaghessian(x -> sum(x.^3), [1 2; 3 4])[1]
# 2×2 Matrix:
#   6  12
#  18  24

# Multiple arguments
diaghessian((x, y) -> sum(x .* y .* y'), [1 22; 333 4], [0.5, 0.666])
# Returns tuple of diagonal Hessians for each argument

# Two scalar arguments
diaghessian(atan, 1, 2)
# Output: (-0.16, 0.16)
```

### Response
Hessian matrix or diagonal Hessian.
```

--------------------------------

### Define Pullback with Generic Struct

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Illustrates how Zygote uses a generic struct `Pullback` to handle closures in lowered code, storing necessary data for the reverse pass.

```julia
struct Pullback{F}
  data
end

function J(::typeof(foo), x)
  a, da = J(sin, x)
  b, db = J(cos, a)
  return b, Pullback{typeof(foo)}((da, db))
end

function (p::Pullback{typeof(foo)})(b̄)
  da, db = p.data[1], p.data[2]
  ā = db(b̄)
  x̄ = da(ā)
  return x̄
end
```

--------------------------------

### Define Nested Mutating and Non-Mutating Functions

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/limitations.md

Illustrates a scenario where an outer function appears non-mutating but calls an inner mutating function. Zygote will still encounter the mutation limitation.

```julia
function g_inner!(x, y)
  for i in eachindex(x, y)
    x[i] = 2 * y[i]
  end
  return x
end

function g_outer(y)
  z = similar(y)
  g_inner!(z, y)
  return z
end
```

--------------------------------

### Wirtinger Derivatives Calculation Function

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/complex.md

Implements the calculation of Wirtinger derivatives using the Jacobian. This provides the two complex derivatives \(\frac{\partial f}{\partial z}\) and \(\frac{\partial f}{\partial z'}\).

```julia
function wirtinger(f, x)
  du, dv = jacobi(f, x)
  (du' + im*dv')/2, (du + im*dv)/2
end
```

--------------------------------

### Derive Pullback for a Composite Function

Source: https://github.com/fluxml/zygote.jl/blob/master/docs/src/internals.md

Manually defines the pullback for the composite function `foo` by composing the pullbacks of its constituent functions (`sin` and `cos`). This demonstrates how to chain gradients.

```julia
function J(::typeof(foo), x)
  a, da = J(sin, x)
  b, db = J(cos, a)
  return b, function(b̄)
    ā, = db(b̄)
    x̄, = da(ā)
    return x̄
  end
end
gradient(foo, 1) # (-0.403,)
```