### Worker Pool Examples

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/worker-pools.md

Demonstrates creating and using WorkerPool and CachingPool for remote function calls and parallel mapping.

```julia
# Create pool from vector
pool = WorkerPool([2, 3, 4])

# Or caching pool for function caching
pool = CachingPool([2, 3, 4])

# Use with remotecall
f = remotecall(sqrt, pool, 4.0)
result = fetch(f)
```

--------------------------------

### Example: Using check_same_host for Shared Memory Allocation

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/internals.md

Demonstrates how to use check_same_host to decide between using SharedArray (if on the same host) or distributed arrays.

```julia
# Check if processes share memory
if check_same_host([2, 3, 4])
    # Can use shared memory
    shared_array = SharedArray(Array, (100, 100); pids=[2, 3, 4])
else
    # Use distributed arrays
end
```

--------------------------------

### clear! Usage Example

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/worker-pools.md

Shows how to use clear! after using a CachingPool with pmap to free up memory or ensure function updates are transmitted.

```julia
cp = CachingPool([2, 3, 4])

# Use pool...
pmap(sqrt, cp, 1.0:100.0)

# Clear cache to free memory
clear!(cp)

# Or clear when done with pool
clear!(cp)
```

--------------------------------

### WorkerPool Usage Examples

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/worker-pools.md

Illustrates using WorkerPool with remotecall, pmap, dynamically adding workers, and retrieving pool information.

```julia
# Use with remotecall
f = remotecall(sqrt, wp, 4.0)

# Use with pmap
results = pmap(sqrt, wp, [1.0, 4.0, 9.0, 16.0])

# Add workers dynamically
push!(wp, 5)

# Get workers from pool
workers_list = workers(wp)

# Number of workers
n = length(wp)
```

--------------------------------

### Using Built-in and Custom Cluster Managers

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-managers.md

Examples demonstrating how to add worker processes using the default LocalManager, SSHManager, and a custom manager from the ClusterManagers.jl package.

```julia
# Use built-in LocalManager
addprocs(4)  # Uses LocalManager by default

# Use SSHManager
addprocs(["user@host1", "user@host2"])

# Custom cluster manager from ClusterManagers.jl
using ClusterManagers
addprocs_slurm(15)  # Uses SLURM cluster manager
```

--------------------------------

### WorkerConfig Examples

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-managers.md

Demonstrates creating a default WorkerConfig and customizing it for SSH launches, including setting host, port, user, tunnel, and executable flags. Also shows how to assign custom user data.

```julia
# Create default config
config = WorkerConfig()

# Customize for SSH launch
config.host = "remote.example.com"
config.port = 2222
config.user = "alice"
config.tunnel = true
config.exeflags = `-O3 --threads=4`

# Use custom user data
config.userdata = Dict(:gpu_id => 0, :numa_node => 1)
```

--------------------------------

### Implement `launch` Method for ClusterManager

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-managers.md

Provides an example implementation of the `launch` method for a custom cluster manager. This method is responsible for creating and configuring worker processes, creating `WorkerConfig` objects, and adding them to the `launched` array.

```julia
function launch(manager::MyManager, params::Dict, launched::Array,
                launch_ntfy::Condition)
    for i in 1:params[:np]
        # Configure worker
        config = WorkerConfig()
        config.host = "worker$(i).example.com"
        config.port = 9000 + i
        config.count = 1
        
        # Push and notify
        push!(launched, config)
        notify(launch_ntfy)
    end
end
```

--------------------------------

### pmap with All Options Combined

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/pmap.md

An example showcasing the combination of multiple pmap options including a specific worker pool, batching, error handling, and retry delays.

```julia
# Complex with all options
pool = CachingPool(workers())
large_data = rand(10^7)
let large_data = large_data
    results = pmap(x -> process_with_context(x, large_data), 
                   pool, input_data;
                   batch_size=10,
                   on_error = ex -> missing,
                   retry_delays = [0.1, 0.2])
end
```

--------------------------------

### Handle RemoteException Example

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/internals.md

Demonstrates how to catch and inspect a RemoteException, which occurs when a computation on a remote worker fails. It shows how to access the worker ID, original exception, and backtrace.

```julia
f = remotecall(sqrt, 2, -4.0)
try
    fetch(f)
catch e
    if isa(e, RemoteException)
        println("Error on worker $(e.pid)")
        println("Original exception: $(e.captured.ex)")
        println("Backtrace:")
        println(e.captured.bt)
    end
end
```

--------------------------------

### Add SLURM Processes with Specific Partition and Threads

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/configuration.md

Launch processes on a SLURM cluster using ClusterManagers.jl. This example specifies the number of processes, thread count, and the SLURM partition to use.

```julia
# Using ClusterManagers.jl for SLURM
using ClusterManagers
addprocs_slurm(
    num_processes=128,
    exeflags=`--threads=auto`,
    partition="gpu"
)
```

--------------------------------

### Launch Local Workers with LocalManager

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-managers.md

Examples of launching local Julia worker processes using the `addprocs` function, which defaults to `LocalManager`. Demonstrates launching a specific number of workers and with custom flags.

```julia
# Launch 4 local workers
addprocs(4)

# Launch 8 workers
addprocs(8)

# Launch workers with custom flags
addprocs(4; exeflags=`--threads=2 -O3`)

# Launch with project environment
addprocs(4; exeflags=`--project=@.`)
```

--------------------------------

### Implement `manage` Method for ClusterManager

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-managers.md

An example implementation of the `manage` method for a custom cluster manager. This method handles lifecycle operations for workers, such as establishing connections, sending interrupts, or deregistering workers.

```julia
function manage(manager::MyManager, id::Integer, config::WorkerConfig, op::Symbol)
    if op === :connect
        # Connect to worker
        stream = connect(config.host, config.port)
        return stream
    elseif op === :interrupt
        # Send interrupt signal
        kill(config.process, SIGINT)
    elseif op === :deregister
        # Cleanup
    end
end
```

--------------------------------

### Check Running Tasks Before Worker Removal

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/errors.md

Provides an example of checking running tasks on workers before attempting to remove them, to diagnose shutdown issues.

```julia
@everywhere begin
    tasks = all_tasks()
    println("Running: $tasks")
end
rmprocs(workers())
```

--------------------------------

### Worker Shutdown Timeout Example

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/errors.md

Illustrates an ErrorException when workers do not terminate within the specified `waitfor` timeout during `rmprocs`.

```julia
rmprocs([2, 3]; waitfor=5)  # Timeout after 5 seconds
# ERROR: workers 2,3 not terminated after 5 seconds
```

--------------------------------

### Get Default AddProcs Parameters

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/internals.md

Provides default parameters for the addprocs function. It can return general defaults or defaults specific to a given ClusterManager.

```julia
default_addprocs_params()
default_addprocs_params(mgr::ClusterManager)
```

--------------------------------

### Remote SSH Cluster Setup

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/INDEX.md

Shows how to add worker processes to a distributed Julia session by connecting to remote machines via SSH. It specifies the number of workers per host.

```julia
addprocs([
    "user@host1",
    ("user@host2", 4),  # 4 workers on host2
])

# Now use cluster normally
results = pmap(expensive_computation, data)

```

--------------------------------

### nprocs

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Get the total number of available processes in the cluster, including the master process.

```APIDOC
## nprocs

### Description

Get the total number of available processes in the cluster.

### Return Value

Integer count including the master process.

### Description

Returns the total number of Julia processes in the cluster. This includes process 1 (master) plus all worker processes. Does not count processes being added or removed.

### Examples

```julia
# Single process (master only)
nprocs()  # Returns 1

addprocs(2)

# With workers
nprocs()  # Returns 3 (1 master + 2 workers)
```

### Source

`src/cluster.jl:877`
```

--------------------------------

### Add Local Workers

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Launches a specified number of Julia worker processes locally. This is the simplest way to start parallel processing.

```julia
addprocs(4)
```

--------------------------------

### Launch Remote Workers with SSHManager

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-managers.md

Examples of launching Julia workers on remote machines via SSH using `SSHManager`. Shows configurations for single workers, multiple workers per host, specific users/ports, SSH tunneling, and custom SSH options.

```julia
# Single worker per host
addprocs(["host1", "host2", "host3"])

# Specific users and ports
addprocs([
    "user@host1",
    "alice@host2:2222",
    "bob@host3"
])

# Multiple workers per host
addprocs([
    ("host1", 2),
    ("host2", 4),
    ("host3", :auto)  # Use all CPU cores
])

# With SSH tunneling
addprocs(["host1", "host2"]; tunnel=true)

# Custom SSH options
addprocs(["host1"]; sshflags=`-i ~/.ssh/id_rsa -o ConnectTimeout=60`)

# Windows C shell
addprocs(["winhost"]; shell=:wincmd)

# Specify Julia executable
addprocs(["host1"]; exename="/opt/julia-1.8/bin/julia")

# With environment variables
addprocs(["host1"]; env=["JULIA_DEPOT_PATH"=>"/depot", "JULIA_NUM_THREADS"=>"4"])
```

--------------------------------

### Error Handling in pmap with on_error

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/INDEX.md

Shows how to handle errors during parallel map operations using the `on_error` keyword argument in `pmap`. This example logs a warning and returns `missing` for failed tasks, allowing the operation to continue.

```julia
# Continue on errors
results = pmap(risky_func, data;
    on_error = e -> @warn("Error: $e"); missing)

```

--------------------------------

### Add Workers with Verbose Debugging Flags

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/configuration.md

Starts worker processes with additional flags like `--depwarn` and `--inline=no` to enable verbose output and disable inlining for debugging purposes.

```julia
# Add debugging flags
addprocs(4; exeflags=`--depwarn --inline=no`)
```

--------------------------------

### Get Total Number of Processes

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Use `nprocs()` to get the total count of Julia processes in the cluster, including the master process (ID 1) and all worker processes. This count is static and does not reflect processes being added or removed.

```julia
# Single process (master only)
nprocs()  # Returns 1

addprocs(2)

# With workers
nprocs()  # Returns 3 (1 master + 2 workers)
```

--------------------------------

### workers

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Get the process IDs of all worker processes, excluding the master process.

```APIDOC
## workers

### Description

Get the process IDs of all worker processes.

```julia
workers() -> Vector{Int}
```

### Return Value

Vector of process IDs excluding the master process.

### Description

Returns array of worker process IDs (those with ID > 1). When `nprocs() == 1` (only master), returns `[1]`.

### Examples

```julia
addprocs(3)

workers()    # Returns [2, 3, 4]
procs()      # Returns [1, 2, 3, 4]
```

### Source

`src/cluster.jl:1006`
```

--------------------------------

### nworkers

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Get the number of available worker processes, excluding the master process.

```APIDOC
## nworkers

### Description

Get the number of available worker processes.

### Return Value

Integer count of worker processes only (excludes master).

### Description

Returns the number of worker processes (those with ID > 1). Equivalent to `nprocs() - 1` except when `nprocs() == 1`, in which case returns 1 (the single master process serves as a worker).

### Examples

```julia
addprocs(3)

nworkers()   # Returns 3
nprocs()     # Returns 4 (includes master)
```

### Source

`src/cluster.jl:908`
```

--------------------------------

### procs

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Get the process IDs of all processes in the cluster, optionally filtered by connection to a specific process.

```APIDOC
## procs

### Description

Get the process IDs of all processes in the cluster.

```julia
procs() -> Vector{Int}
procs(pid::Integer) -> Vector{Int}
```

### Parameters

#### Path Parameters

* **pid** (Integer) - Optional - return workers connected to this process

### Return Value

Vector of process IDs.

### Description

Returns array of process IDs in the cluster. If `pid` is specified, returns processes connected to that specific process (useful for custom topologies).

### Examples

```julia
addprocs(3)

procs()      # Returns [1, 2, 3]
procs(1)     # Returns [1, 2, 3] (all connected to master)

# With master_worker topology
procs()      # Returns [1, 2, 3]
procs(2)     # Returns [1, 2] (worker 2 connected only to master)
```

### Source

`src/cluster.jl:945`
```

--------------------------------

### Create and Use a RemoteChannel

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/futures-and-channels.md

Illustrates creating a RemoteChannel on a remote process for passing any values, then putting data into it and taking it out. Also shows creating a channel with a specific type and size, and using it in a producer-consumer pattern.

```julia
# Create channel for passing any values
rc = RemoteChannel(2)
put!(rc, "data")
value = take!(rc)

# Create channel with specific type and size
rc = RemoteChannel(()->Channel{Int}(10), 2)

# Use for producer-consumer pattern
rc = RemoteChannel(2)
@async begin
    for i in 1:5
        put!(rc, i)
    end
    close(rc)
end

for value in rc
    println(value)
end
```

--------------------------------

### pgenerate with Multiple Input Collections

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/pmap.md

Shows how to use pgenerate with multiple input collections, applying a function that processes corresponding elements from each collection.

```julia
# Multiple inputs
for (a, b) in pgenerate((x, y) -> (x, y*2), xs, ys)
    println("$a -> $b")
end
```

--------------------------------

### Create and Fetch a Future

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/futures-and-channels.md

Demonstrates creating a Future on another process using `remotecall` and retrieving its result with `fetch`. Also shows creating an empty Future and filling it later, checking readiness with `isready`, and waiting with `wait`.

```julia
# Create a future on another process
f = remotecall(sqrt, 2, 4.0)
result = fetch(f)  # Block and get result (2.0)

# Create empty future and fill later
f = Future(2)
# ... remotely: put!(f, some_value) ...
result = fetch(f)

# Check if ready without blocking
isready(f)  # Returns true/false

# Wait for completion without getting value
wait(f)
```

--------------------------------

### Get Remote Reference ID

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/futures-and-channels.md

Retrieves the unique Remote Reference ID (RRID) for a Future or RemoteChannel. The RRID contains 'whence' and 'id' fields.

```julia
remoteref_id(r::AbstractRemoteRef) -> RRID
```

--------------------------------

### CachingPool Usage with pmap and let

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/worker-pools.md

Demonstrates using CachingPool with pmap for efficient function transmission, especially with large data. Uses a let block to capture data values.

```julia
# Use with pmap (transmits function once per worker)
large_data = rand(10^7)
results = pmap(x -> compute(x, large_data), cp, input_data)
# Instead of: pmap(x -> compute(x, large_data), cp, input_data)
# which would send large_data each time

# Better pattern with let
cp = CachingPool(workers())
let large_data = large_data
    results = pmap(x -> compute(x, large_data), cp, input_data)
end
```

--------------------------------

### Asynchronous Remote Call with Future

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/remote-calls.md

Execute a function on a remote worker asynchronously and get a Future to retrieve the result later. Supports positional and keyword arguments.

```julia
future = remotecall(sqrt, 2, 4.0)
result = fetch(future)  # Returns 2.0
```

```julia
# With keyword arguments
f = remotecall(parse, 3, Int, "42"; base=10)
fetch(f)  # Returns 42
```

```julia
# Chaining operations
future = remotecall(myfunction, 2, x, y)
other_future = remotecall(anotherfunction, 3, fetch(future))
```

--------------------------------

### Basic Distributed Julia Usage

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/INDEX.md

Demonstrates adding worker processes, executing functions remotely using `remotecall` and `fetch`, and performing a parallel map operation with `pmap`. Includes cleanup of worker processes.

```julia
using Distributed

# Add 4 worker processes
addprocs(4)

# Execute function remotely
f = remotecall(sqrt, 2, 4.0)
result = fetch(f)  # Returns 2.0

# Or directly get result
result = remotecall_fetch(sqrt, 2, 4.0)

# Execute on all workers
@everywhere begin
    function my_func(x)
        return x^2
    end
end

# Parallel map
results = pmap(my_func, 1:100)

# Cleanup
rmprocs(workers())

```

--------------------------------

### pmap with Multiple Input Collections

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/pmap.md

Demonstrates using pmap with multiple input collections, applying a function that takes multiple arguments to corresponding elements from each collection.

```julia
# Multiple input collections
results = pmap((x, y) -> x + y, [1, 2, 3], [10, 20, 30])
```

--------------------------------

### Get Current Process ID

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Returns the unique identifier for the current Julia process. Process ID 1 is the master process, and subsequent IDs are assigned to workers.

```julia
myid()
```

--------------------------------

### init_worker

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Initialize a newly launched worker process with a cookie and cluster manager.

```APIDOC
## init_worker

### Description

Initialize a newly launched worker process.

```julia
init_worker(cookie::AbstractString, manager::ClusterManager=DefaultClusterManager()) -> Nothing
```

### Parameters

#### Path Parameters

* **cookie** (AbstractString) - Cluster authentication cookie
* **manager** (ClusterManager) - Default: `DefaultClusterManager()` - Cluster manager for custom transports

### Description

Called by custom cluster managers to initialize a newly launched Julia process as a worker. The process is configured with the given authentication cookie and cluster manager. This function is automatically called when using `--worker` command line argument.

For custom cluster transports, managers should call this function instead of using the default TCP/IP transport.

### Throws

* `AssertionError` - If RemoteChannels, Futures, or addprocs have already been called

### Examples

```julia
# Custom cluster manager implementation
function launch(manager::MyManager, params::Dict, launched::Array, launch_ntfy::Condition)
    # ... launch worker process ...
    # Worker process calls:
    init_worker(params[:cookie], manager)
end
```

### Source

`src/cluster.jl:380`
```

--------------------------------

### Custom Serialization with Worker ID Optimization

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/internals.md

Example of using worker_id_from_socket within a custom serialization function to optimize data transfer based on the destination worker's ID.

```julia
# In custom serialization
function serialize(s::ClusterSerializer, obj::MyType)
    pid = worker_id_from_socket(s.io)
    if pid > 0
        # Optimize for specific worker
        serialize_optimized(s, obj, pid)
    else
        # Local or unknown
        serialize_generic(s, obj)
    end
end
```

--------------------------------

### Synchronization with Futures and RemoteChannels

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/INDEX.md

Illustrates synchronization mechanisms in Distributed.jl. It shows how to wait for a remote call to complete using `fetch` and how to use `RemoteChannel` for passing multiple values between processes asynchronously.

```julia
# Wait for result
f = remotecall(long_function, 2, args)
result = fetch(f)  # Blocks until ready

# Or wait without getting result
wait(f)

# Channel for multiple values
rc = RemoteChannel(2)
@async for i in 1:5
    put!(rc, i)
end

for value in rc
    println(value)
end

```

--------------------------------

### Fetch from Future and RemoteChannel with Error Handling

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/futures-and-channels.md

Demonstrates fetching values from both Futures and RemoteChannels. For Futures, it shows caching behavior. For RemoteChannels, it illustrates fetching without removing the value. Includes error handling for remote exceptions.

```julia
# Fetch from Future
f = remotecall(sqrt, 2, 4.0)
result = fetch(f)  # Blocks until available, returns 2.0
result2 = fetch(f)  # Returns cached value immediately

# Fetch from RemoteChannel
rc = RemoteChannel(2)
put!(rc, 42)
value = fetch(rc)  # Returns 42 without removing it
peek_value = fetch(rc)  # Still 42

# Error handling
f = remotecall(sqrt, 2, -4.0)
try
    fetch(f)
catch e
    if isa(e, RemoteException)
        println("Error on worker: $(e.captured.ex)")
    end
end
```

--------------------------------

### myid

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Returns the cluster-unique ID of the current Julia process. Process ID 1 is reserved for the master process, while worker processes are assigned sequential IDs starting from 2.

```APIDOC
## myid

### Description

Returns the cluster-unique ID of the current Julia process. Process ID 1 is reserved for the master process. Worker processes have IDs 2, 3, 4, etc. assigned sequentially as they are added.

### Method

`myid() -> Int`

### Return Value

`Int` - Integer process identifier. Process 1 is the master/driver process.

### Examples

```julia
# On master process
myid()  # Returns 1
```
```

--------------------------------

### Catching RemoteException

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/errors.md

Demonstrates how to catch and handle RemoteException when fetching results from a remote computation that failed.

```julia
using Distributed

f = remotecall(sqrt, 2, -4.0)  # sqrt of negative number

try
    result = fetch(f)
catch e
    if isa(e, RemoteException)
        println("Error occurred on worker $(e.pid)")
        println("Original error: $(e.captured.ex)")
        println("Backtrace: $(e.captured.bt)")
    else
        rethrow()
    end
end
```

--------------------------------

### Get and Set Cluster Cookie

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/internals.md

Manages the authentication cookie for cluster communication. The cookie must be an ASCII string of at most 16 bytes. Setting the cookie affects newly launched workers.

```julia
# Get current cookie
cookie = cluster_cookie()

# Set new cookie for security
cluster_cookie("my_secret_key_123")

# Both workers and master must have same cookie
```

--------------------------------

### Get Worker ID from Socket

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/internals.md

Retrieves the process ID associated with an IO connection. Returns -1 if the connection is not to a worker process. Useful for optimizing data transfer in custom serialization.

```julia
worker_id_from_socket(s)
```

--------------------------------

### Execute and fetch result immediately with @fetch

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/macros.md

Combines @spawnat :any and fetch() to execute an expression on an auto-selected worker and block until the result is available. More concise than separate calls.

```julia
result = @fetch sqrt(4)  # Returns 2.0
```

```julia
value = @fetch 1 + 2  # Returns 3
```

```julia
data = [1, 2, 3, 4, 5]
total = @fetch sum(data)
```

--------------------------------

### Catching LaunchWorkerError

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/errors.md

Demonstrates catching LaunchWorkerError during `addprocs` if a worker fails to launch or initialize.

```julia
try
    addprocs(10; exename="/path/to/julia")
catch e
    if isa(e, LaunchWorkerError)
        println("Failed to launch workers: $(e.msg)")
        # Try alternative configuration
        addprocs(4)  # Fewer workers
    else
        rethrow()
    end
end
```

--------------------------------

### cluster_cookie

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/internals.md

Manages the cluster authentication cookie. This function can be used to get the current cookie or set a new one for authenticating inter-process communication. The cookie must be an ASCII string of at most 16 bytes.

```APIDOC
## cluster_cookie

### Description

Get or set the cluster authentication cookie. Gets or sets the authentication cookie used for cluster communication. The cookie is used to authenticate connections between processes. Must be an ASCII string of at most 16 bytes. Setting the cookie configures how newly launched workers authenticate.

### Parameters

- `cookie` (String) - New cookie value

### Return Value

The current cluster cookie (padded to 16 bytes).

### Examples

```julia
# Get current cookie
cookie = cluster_cookie()

# Set new cookie for security
cluster_cookie("my_secret_key_123")

# Both workers and master must have same cookie
```
```

--------------------------------

### Add Workers with a Custom Project Environment

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/configuration.md

Launch worker processes that use a specific Julia project environment. This is achieved by passing the project path via `exeflags`.

```julia
# Add workers with specific Julia project
addprocs(4; exeflags=`--project=/path/to/project`)
```

--------------------------------

### Distributed Reduction with @distributed

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/INDEX.md

Demonstrates how to perform a distributed reduction operation using the `@distributed` macro. This allows summing or other reduction operations on results computed in parallel across worker processes.

```julia
# Sum computed in parallel
result = @distributed (+) for i in 1:1000
    compute(i)
end

```

--------------------------------

### Add Workers with Consistent Lazy Setting

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/errors.md

Shows how to successfully add workers by using a consistent 'lazy' setting across `addprocs` calls.

```julia
addprocs(2; lazy=false)
addprocs(2; lazy=false)  # OK
```

--------------------------------

### @fetchfrom

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/macros.md

Macro combining `@spawnat` and `fetch()`. Executes expression on specified worker and blocks until result is available. Equivalent to `fetch(@spawnat p expr)`.

```APIDOC
## @fetchfrom

### Description
Execute expression on specific worker and return result immediately.

### Method
Macro

### Parameters
#### Path Parameters
- **p** (Int) - Required - Process ID of target worker
- **expr** (Expr) - Required - Julia expression to execute remotely

### Return Value
The return value of the expression.

### Examples
```julia
# Specific worker
result = @fetchfrom 2 sqrt(4)  # Returns 2.0

# Multiple workers
results = [@fetchfrom 2 compute1(), @fetchfrom 3 compute2()]

# With complex expressions
data = [1, 2, 3]
sum_result = @fetchfrom 2 sum(data)  # Returns 6
```
```

--------------------------------

### List and Inspect Workers

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/configuration.md

Provides Julia code to list all available worker PIDs using `workers()` and the total number of workers using `nworkers()`. It also demonstrates fetching `versioninfo()` from each worker for inspection.

```julia
# List all workers
@show workers()
@show nworkers()

# Check specific worker
@eval Main import InteractiveUtils
for w in workers()
    println("Worker $w:")
    remotecall_fetch(() -> versioninfo(), w)
end
```

--------------------------------

### Get Worker Process IDs

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

The `workers()` function returns a vector containing the process IDs of all worker processes (those with IDs greater than 1). If only the master process is present (`nprocs() == 1`), it returns `[1]`.

```julia
addprocs(3)

workers()    # Returns [2, 3, 4]
procs()      # Returns [1, 2, 3, 4]
```

--------------------------------

### @spawnat

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/macros.md

Asynchronously executes an expression on the specified worker process and returns a `Future`. If `p` is the quoted literal `:any`, the system automatically selects a worker. Captures local variables in a closure.

```APIDOC
## @spawnat

### Description
Run an expression on a specific worker process or auto-selected worker.

### Method
Macro

### Parameters
#### Path Parameters
- **p** (Int or :any) - Required - Process ID (2, 3, etc.) or :any for auto-selection
- **expr** (Expr) - Required - Julia expression to execute remotely

### Return Value
`Future` referencing the result of the expression.

### Examples
```julia
# Specific worker
f = @spawnat 2 sqrt(4)
fetch(f)  # Returns 2.0

# Auto-selected worker
f = @spawnat :any sqrt(4)
fetch(f)  # Returns 2.0

# Capture variables
x = [1, 2, 3]
f = @spawnat 3 sum(x)
fetch(f)  # Returns 6

# With synchronization
@sync begin
    for i in 1:10
        @spawnat :any process_item(data[i])
    end
end
```
```

--------------------------------

### Get Number of Worker Processes

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

Use `nworkers()` to retrieve the count of active worker processes (those with IDs greater than 1). This is equivalent to `nprocs() - 1`, except when only the master process is running, in which case it returns 1.

```julia
addprocs(3)

nworkers()   # Returns 3
nprocs()     # Returns 4 (includes master)
```

--------------------------------

### launch

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-managers.md

Launches worker processes. This method is called by the cluster system and requires implementations to create, configure, and register new worker configurations.

```APIDOC
## launch

Launch worker processes (manager method).

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `manager` | `ClusterManager` | Manager instance |
| `params` | `Dict{Symbol,Any}` | Launch parameters from `addprocs` |
| `launched` | `Array{WorkerConfig}` | Append WorkerConfig for each launched worker |
| `launch_ntfy` | `Condition` | Notify when workers available |

### Description

Called by cluster system to launch workers. Implementation must:
1. Create/configure workers according to parameters
2. Create `WorkerConfig` object for each worker
3. Push each config to `launched` array
4. Notify `launch_ntfy` condition when configs available

Should handle SSH connections, startup scripts, environment setup, etc.

### Examples

```julia
function launch(manager::MyManager, params::Dict, launched::Array, launch_ntfy::Condition)
    for i in 1:params[:np]
        # Configure worker
        config = WorkerConfig()
        config.host = "worker$(i).example.com"
        config.port = 9000 + i
        config.count = 1
        
        # Push and notify
        push!(launched, config)
        notify(launch_ntfy)
    end
end
```
```

--------------------------------

### Get Current Process ID

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

On a worker process, `myid()` returns its unique process ID (e.g., 2, 3, 4). From the master process, `remotecall_fetch` can be used to query a specific worker's ID.

```julia
# On worker process (after addprocs)
myid()  # Returns 2, 3, 4, etc.

# From master, query worker process ID
remotecall_fetch(() -> myid(), 2)  # Returns 2
```

--------------------------------

### @fetch

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/macros.md

Macro combining `@spawnat :any` and `fetch()`. Executes expression on an automatically-selected worker and blocks until the result is available. More concise than separate `@spawn` and `fetch` calls.

```APIDOC
## @fetch

### Description
Execute expression on auto-selected worker and return result immediately.

### Method
Macro

### Parameters
#### Path Parameters
- **expr** (Expr) - Required - Julia expression to execute remotely

### Return Value
The return value of the expression.

### Examples
```julia
# Simple computation
result = @fetch sqrt(4)  # Returns 2.0

# Gets result directly
value = @fetch 1 + 2  # Returns 3

# Works with complex expressions
data = [1, 2, 3, 4, 5]
total = @fetch sum(data)
```
```

--------------------------------

### Get or Set Default Worker Pool

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/worker-pools.md

Manages the default worker pool used by `remote()` and `pmap()` when no pool is explicitly specified. On the master process, this is a `WorkerPool` containing all workers. On worker processes, it retrieves the pool from the master.

```julia
# Get current default pool
pool = default_worker_pool()

# Set new default pool
cp = CachingPool(workers())
default_worker_pool!(cp)

# Now pmap uses the caching pool by default
results = pmap(sqrt, input_data)

# Reset to standard pool
wp = WorkerPool(workers())
default_worker_pool!(wp)
```

--------------------------------

### Julia pgenerate Function Signatures

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/pmap.md

Presents the function signatures for pgenerate, highlighting its ability to work with a specified worker pool or default settings.

```julia
pgenerate(f, c...) -> iterator
pgenerate(pool::AbstractWorkerPool, f, c...) -> iterator
```

--------------------------------

### Execute and fetch result immediately from specific worker with @fetchfrom

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/macros.md

Combines @spawnat and fetch() to execute an expression on a specified worker and block until the result is available.

```julia
result = @fetchfrom 2 sqrt(4)  # Returns 2.0
```

```julia
results = [@fetchfrom 2 compute1(), @fetchfrom 3 compute2()]
```

```julia
data = [1, 2, 3]
sum_result = @fetchfrom 2 sum(data)  # Returns 6
```

--------------------------------

### channel_from_id

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/futures-and-channels.md

Gets the backing channel for a remote reference ID. This low-level API returns the actual channel object backing a remote reference ID and is valid only on the process where the reference was created. It is used internally for garbage collection and cleanup.

```APIDOC
## channel_from_id

Get the backing channel for a remote reference ID.

```julia
channel_from_id(id) -> AbstractChannel
```

### Parameters

| Parameter | Type | Description |
|-----------|------|-------------|
| `id` | RRID | Remote reference ID from `remoteref_id()` |

### Return Value

The backing `AbstractChannel` for the reference.

### Description

Low-level API that returns the actual channel object backing a remote reference ID. Valid only on the process where the reference was created. Used internally for garbage collection and cleanup.

### Throws

- `ErrorException` - If reference not found on this process

### Examples

```julia
rc = RemoteChannel(2)
rrid = remoteref_id(rc)

# On process 2:
channel = remotecall_fetch(channel_from_id, 2, rrid)
```
```

--------------------------------

### Get Remote Reference ID

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/futures-and-channels.md

Retrieves the unique Remote Reference ID (RRID) for a future object. The RRID contains the process ID where the reference was created and a unique ID within that process. This can be useful for custom serialization logic.

```julia
f = remotecall(compute, 2, data)
rrid = remoteref_id(f)
println("Created on process $(rrid.whence) with id $(rrid.id)")

# Use in custom serialization
function serialize_custom(s::ClusterSerializer, obj::MyType)
    target_pid = worker_id_from_socket(s.io)
    # Optimize serialization based on target
    ...
end
```

--------------------------------

### Add Local Processes with Maximum Performance Flags

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/configuration.md

Launch local Julia processes optimized for maximum performance using `-O3` and auto-detected threads. This is ideal for maximizing local computation power.

```julia
# Maximum performance on local machine
addprocs(Sys.CPU_THREADS; exeflags=`-O3 --threads=auto`)
```

--------------------------------

### Configure Custom Master-Worker Topology

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/configuration.md

Set up distributed processes using a master-worker topology where only the master connects to the workers. This can simplify communication patterns in certain scenarios.

```julia
# Only master connects to workers, workers don't connect to each other
addprocs(4; topology=:master_worker)
```

--------------------------------

### Get All Process IDs

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

The `procs()` function returns a vector of all process IDs currently in the cluster. An optional `pid` argument can filter the results to show only processes connected to the specified process ID, useful for custom cluster topologies.

```julia
addprocs(3)

procs()      # Returns [1, 2, 3]
procs(1)     # Returns [1, 2, 3] (all connected to master)

# With master_worker topology
procs()      # Returns [1, 2, 3]
procs(2)     # Returns [1, 2] (worker 2 connected only to master)
```

--------------------------------

### Get Backing Channel from Remote Reference ID

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/futures-and-channels.md

Retrieves the `AbstractChannel` associated with a given Remote Reference ID (RRID). This function is intended for use on the process where the reference was originally created and is primarily used internally for garbage collection. An `ErrorException` is thrown if the reference is not found on the current process.

```julia
rc = RemoteChannel(2)
rrid = remoteref_id(rc)

# On process 2:
channel = remotecall_fetch(channel_from_id, 2, rrid)
```

--------------------------------

### take!

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/futures-and-channels.md

Removes and returns the next value from a RemoteChannel, blocking if the channel is empty. Not supported on Futures.

```APIDOC
## take!

Remove and return a value from a RemoteChannel.

### Method

`take!(r::RemoteChannel, args...)

### Parameters

- **r** (`RemoteChannel`) - Required - Remote channel to take from
- **args** (`Varargs`) - Optional - Additional arguments passed to underlying channel

### Return Value

Value removed from the channel.

### Description

Removes and returns the next value from a RemoteChannel. Blocks if the channel is empty until a value becomes available.

Note: `take!` is not supported on `Future` objects (which are read-only).

### Throws

- `InvalidStateException` - If channel is closed and empty
- Error if remote process dies or connection fails

### Examples

```julia
# Producer-consumer pattern
rc = RemoteChannel(2)

# Producer (on main process)
@async for i in 1:5
    put!(rc, i)
end

# Consumer
for i in 1:5
    value = take!(rc)  # Blocks until available
    println(value)
end
```
```

--------------------------------

### Add Workers with All-to-All Topology

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/configuration.md

Use when arbitrary inter-worker communication is needed and all workers must connect to each other. This offers flexibility but incurs higher overhead due to more connections.

```julia
addprocs(4; topology=:all_to_all)
```

--------------------------------

### pgenerate with a Specified Worker Pool

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/pmap.md

Demonstrates using pgenerate with an explicitly defined worker pool for distributing the parallel generation of results.

```julia
# With worker pool
pool = CachingPool(workers())
for result in pgenerate(pool, compute, data)
    println(result)
end
```

--------------------------------

### Adding Local Workers

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/INDEX.md

Demonstrates the basic method for adding multiple local worker processes to the current Julia session using `addprocs`.

```julia
# Local workers
addprocs(4)  # 4 processes on same machine

```

--------------------------------

### Configure SSH Cluster with Tunneling and Threaded BLAS

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/configuration.md

Set up a secure remote cluster using SSH with tunneling and multiplexing enabled. It also configures SSH flags, worker threads, and enables threaded BLAS for performance.

```julia
# Secure remote cluster with SSH tunneling
addprocs(
    [("alice@host1", 4), ("bob@host2", 4)];
    tunnel=true,
    multiplex=true,
    sshflags=`-i ~/.ssh/id_rsa`,
    exeflags=`--threads=4`,
    enable_threaded_blas=true
)
```

--------------------------------

### Initialize Worker Process

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/cluster-management.md

The `init_worker` function is used by custom cluster managers to initialize a newly launched worker process. It configures the worker with an authentication cookie and a specified cluster manager, essential for custom transport mechanisms.

```julia
# Custom cluster manager implementation
function launch(manager::MyManager, params::Dict, launched::Array, launch_ntfy::Condition)
    # ... launch worker process ...
    # Worker process calls:
    init_worker(params[:cookie], manager)
end
```

--------------------------------

### Add Workers with Consistent Topology

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/errors.md

Shows successful worker addition by maintaining a consistent topology setting.

```julia
addprocs(2; topology=:master_worker)
addprocs(2; topology=:master_worker)  # OK
```

--------------------------------

### pgenerate

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/pmap.md

Generates results in parallel as they become available, returning an iterator. This is more efficient than `pmap` when results can be processed incrementally and order does not matter.

```APIDOC
## pgenerate

### Description

Generates results in parallel as they become available, returning an iterator. This is more efficient than `pmap` when results can be processed incrementally and order does not matter.

### Method Signatures

```julia
pgenerate(f, c...) -> iterator
pgenerate(pool::AbstractWorkerPool, f, c...) -> iterator
```

### Parameters

- **`pool`** (AbstractWorkerPool) - Optional - Worker pool for distribution.
- **`f`** (Function) - Required - Function to apply to each element.
- **`c`** (Collection or multiple) - Required - Input collection(s).

### Return Value

Iterator yielding results as they become available (not necessarily in input order).

### Examples

```julia
# Results as available, not in input order
for result in pgenerate(sqrt, 1.0:1000.0)
    process_result(result)
end

# With worker pool
pool = CachingPool(workers())
for result in pgenerate(pool, compute, data)
    println(result)
end

# Multiple inputs
for (a, b) in pgenerate((x, y) -> (x, y*2), xs, ys)
    println("$a -> $b")
end
```
```

--------------------------------

### @spawn

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/macros.md

Asynchronously executes an expression on a randomly-selected worker and returns a `Future`. This macro is deprecated; `@spawnat :any` should be used instead. It captures local variables in a closure.

```APIDOC
## @spawn

### Description
Run an expression on an automatically-chosen worker process.

### Method
Macro

### Parameters
#### Path Parameters
- **expr** (Expr) - Required - Julia expression to execute remotely

### Return Value
`Future` referencing the result of the expression.

### Deprecation
This macro is deprecated as of Julia 1.3. Use `@spawnat :any expr` instead for equivalent functionality.

### Examples
```julia
f = @spawn sqrt(4)
result = fetch(f)  # Returns 2.0

# Captures local variables
x = 42
f = @spawn x + 1
fetch(f)  # Returns 43

# Chain with sync
@sync begin
    f1 = @spawn compute1()
    f2 = @spawn compute2()
end
results = [fetch(f1), fetch(f2)]
```
```

--------------------------------

### pmap with Batching for Efficiency

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/pmap.md

Utilizes the batch_size parameter to send multiple elements to each worker at once, which can improve performance for certain operations.

```julia
# Batching for efficiency
results = pmap(compute_batch, input_data; batch_size=100)
```

--------------------------------

### isready

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/api-reference/futures-and-channels.md

Non-blocking check to determine if a Future or RemoteChannel has a value available. Can have race conditions for RemoteChannels.

```APIDOC
## isready

Check if a Future or RemoteChannel has a value available.

### Method

`isready(r::Future)`
`isready(r::RemoteChannel, args...)

### Parameters

- **r** (`Future` or `RemoteChannel`) - Required - Remote reference to check
- **args** (`Varargs`) - Optional - Additional arguments passed to underlying channel

### Return Value

`true` if value(s) available, `false` otherwise.

### Description

Non-blocking check for whether value is available. For `Future`, safe to use as Futures are assigned only once. For `RemoteChannel`, can have race conditions since state may change after the call returns.

### Examples

```julia
# Check if Future is ready
f = remotecall(compute, 2, data)
if isready(f)
    result = fetch(f)
else
    # Do something else while waiting
end

# Use in wait pattern
while !isready(f)
    sleep(0.1)
end
result = fetch(f)
```
```

--------------------------------

### Asynchronous Computation with Future

Source: https://github.com/julialang/distributed.jl/blob/master/_autodocs/INDEX.md

Initiates a remote computation and returns a Future object. Use fetch() to retrieve the result once the computation is complete. Suitable for single asynchronous tasks.

```julia
f = remotecall(compute, 2, args)  # Returns Future
result = fetch(f)  # Wait and get result
```