### Clone and install Wool from GitHub

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Clones the Wool repository from GitHub and installs it locally. Run these commands in order to set up a development environment.

```sh
git clone https://github.com/wool-labs/wool.git
cd wool
pip install ./wool
```

--------------------------------

### Install Wool pre-release with pip --pre

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Installs the latest release candidate of Wool. Use this flag to get pre-release versions for major and minor releases.

```sh
pip install --pre wool
```

--------------------------------

### Install Wool with pip

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Installs the latest stable release of Wool from PyPI.

```sh
pip install wool
```

--------------------------------

### Quick start: define and call a Wool routine

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Demonstrates the core usage of Wool: decorate an async function with @wool.routine, create a WorkerPool with 4 workers, and call the routine as a normal async function. The output is 3.

```python
import asyncio
import wool


@wool.routine
async def add(x: int, y: int) -> int:
    return x + y


async def main():
    async with wool.WorkerPool(spawn=4):
        result = await add(1, 2)
        print(result)  # 3


asyncio.run(main())
```

--------------------------------

### WorkerPool with lazy=False for eager startup

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Sets lazy=False to start WorkerProxy eagerly, establishing connections before the first dispatch. Default is lazy=True, which defers startup and avoids discovery subscription and worker-sentinel setup for tasks that never invoke nested @wool.routine calls.

```python
# Eager proxy startup — connections established before first dispatch
async with wool.WorkerPool(spawn=4, lazy=False):
    result = await my_routine()
```

--------------------------------

### Advertise worker identity with WorkerPool

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Creates a provider from credentials with a fixed set of peer names, then starts a WorkerPool with the given identity and spawn count. The identity must be a name in the worker's certificate; a mismatch surfaces at the first handshake as a HandshakeError.

```python
provider = credentials.as_provider(peers={"api.wool.svc", "batch.wool.svc"})

async with wool.WorkerPool(
    spawn=4,
    identity="api.wool.svc",
    credentials=provider,
):
    result = await my_routine()
```

--------------------------------

### Start a durable LocalWorker with wildcard bind and publish via LanDiscovery

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Use for a durable worker that must be reachable off-host. Binds to all interfaces so the publisher can auto-resolve a routable address; leaving the default loopback bind triggers a LoopbackAdvertisementWarning.

```python
worker = LocalWorker(host="0.0.0.0")
await worker.start()
async with LanDiscovery("my-pool").publisher as publisher:
    await publisher.publish("worker-added", worker.metadata)
    ...
```

--------------------------------

### WorkerPool lease: cap discovered workers

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Two examples of the lease parameter in one block. First, with spawn and lease set, the pool accepts up to `spawn + lease` total workers. Second, a durable pool with no spawn accepts up to `lease` discovered workers. The lease caps admission from discovery, not reservations.

```python
import wool

# Spawn 4 local workers, accept up to 4 more from discovery (8 total)
async with wool.WorkerPool(spawn=4, lease=4, discovery=wool.LanDiscovery()):
    result = await my_routine()

# Durable pool capped at 10 discovered workers
async with wool.WorkerPool(discovery=wool.LanDiscovery(), lease=10):
    result = await my_routine()
```

--------------------------------

### Redis-backed custom discovery implementation

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/discovery/README.md

A sketch of a custom discovery backend using Redis pub/sub. Includes an async context manager factory for lifecycle management.

```python
import wool
from contextlib import asynccontextmanager


class RedisDiscovery:
    """Implements DiscoveryLike via Redis pub/sub."""

    def __init__(self, redis_url: str):
        self._url = redis_url
        self._publisher = RedisPublisher(redis_url)
        self._subscriber = RedisSubscriber(redis_url)

    @property
    def publisher(self) -> wool.DiscoveryPublisherLike:
        return self._publisher

    @property
    def subscriber(self) -> wool.DiscoverySubscriberLike:
        return self._subscriber

    def subscribe(self, filter=None) -> wool.DiscoverySubscriberLike:
        return RedisSubscriber(self._url, filter=filter)


# Wrap in an async context manager factory for lifecycle management
@asynccontextmanager
async def redis_discovery():
    svc = RedisDiscovery("redis://localhost:6379")
    await svc.connect()
    try:
        yield svc
    finally:
        await svc.close()
```

--------------------------------

### Accept peer names by predicate with as_provider

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Configures the provider to accept any peer name ending in ".wool.svc" via a predicate. This is the unconfigured admission state upgrade path: roll out workers with identity set first, then configure peers on clients.

```python
provider = credentials.as_provider(peers=lambda name: name.endswith(".wool.svc"))
```

--------------------------------

### WorkerOptions / ChannelOptions

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Documentation for WorkerOptions and ChannelOptions configuration classes.

```APIDOC
## WorkerOptions

### Description
Composes a `ChannelOptions` instance with server-only settings that are not communicated to clients. Pass to `LocalWorker` or `WorkerProcess` to customize transport behavior.

### Parameters
- **channel** (ChannelOptions) - Required - Channel options advertised via `WorkerMetadata`.
- **http2_min_recv_ping_interval_without_data_ms** (int) - Optional - Minimum allowed client ping interval.
- **max_ping_strikes** (int) - Optional - Ping violations before GOAWAY.
- **max_connection_idle_ms** (int) - Optional - Connection lifecycle limit.
- **max_connection_age_ms** (int) - Optional - Connection lifecycle limit.
- **max_connection_age_grace_ms** (int) - Optional - Connection lifecycle limit.

### ChannelOptions Fields
- **max_receive_message_length** (int) - Optional - Maximum receive message size.
- **max_send_message_length** (int) - Optional - Maximum send message size.
- **keepalive_time_ms** (int) - Optional - Keepalive time.
- **keepalive_timeout_ms** (int) - Optional - Keepalive timeout.
- **keepalive_permit_without_calls** (bool) - Optional - Permit keepalive without calls.
- **max_pings_without_data** (int) - Optional - Maximum pings without data.
- **max_concurrent_streams** (int) - Optional - Maximum concurrent streams.
- **compression** (str) - Optional - Compression algorithm.

### Example
```python
from wool.runtime.worker.base import ChannelOptions, WorkerOptions
from wool.runtime.worker.local import LocalWorker

options = WorkerOptions(
    channel=ChannelOptions(
        keepalive_time_ms=10_000,
        keepalive_timeout_ms=5_000,
        max_concurrent_streams=50,
    ),
    max_connection_idle_ms=300_000,
)

async with wool.WorkerPool(
    spawn=4,
    worker=lambda *tags, credentials=None: LocalWorker(
        *tags,
        credentials=credentials,
        options=options,
    ),
):
    result = await my_routine()
```
```

--------------------------------

### DiscoveryPublisherLike interface

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/discovery/README.md

Shows the required attributes and methods for a discovery publisher: a bind_host attribute and a publish method.

```python
bind_host: str


async def publish(self, type: DiscoveryEventType, metadata: WorkerMetadata) -> None: ...
```

--------------------------------

### Launch subprocess with asyncio.create_subprocess_exec and start_new_session=True

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Shows the only subprocess launch path available on macOS for asyncio subprocesses: a real fork() from a worker thread while gRPC threads are live. Use this when you need a new session or a bare command name, which disqualifies the posix_spawn fast path on every platform.

```python
# Always the forking path: new session, and a bare name resolved through PATH
await asyncio.create_subprocess_exec("tabix", *args, start_new_session=True)
```

--------------------------------

### wool.ContextVar and wool.Token

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/context/README.md

Documentation for wool.ContextVar and wool.Token, covering their methods, parameters, and behavior.

```APIDOC
## wool.ContextVar

### Description
`ContextVar` mirrors `contextvars.ContextVar` at the surface — `get()`, `set()`, `reset()` — with an optional `namespace`. Its identity model is structural, not object-based. Every `ContextVar` has a `(namespace, name)` key; two `ContextVar(name="foo", namespace="bar")` calls in the same process resolve to the *same* singleton via the process-wide var registry, and collisions on the same key with mismatched declarations raise `ContextVarCollision`. The namespace is inferred from the top-level package of the calling frame when not passed explicitly, which keeps wire keys stable when modules are refactored deeper within the same package.

### Methods

#### get()
Returns the value bound in the active context, falling back to the supplied default, then the constructor default, then raising `LookupError`.

#### set(value)
Rebuilds the context with the new binding and returns a `Token`.

#### reset(token)
Defers to native validation — stdlib raises `ValueError` for a token spent twice, minted by another variable, or minted in a different `contextvars.Context`. Wool layers only its chain bookkeeping on top: a reset that rewinds to no prior value records the pending reset signal so the drop propagates over the wire.

### Parameters
- **namespace** (string) - Optional - The namespace for the variable. Inferred from the top-level package of the calling frame when not passed explicitly.
- **name** (string) - Required - The name of the variable.

### Notes
- Pickling is gated through Wool's pickler; vanilla `pickle.dumps` is rejected via `__reduce_ex__` to keep the registry-bound semantics from leaking into ad-hoc serialization.
- The `(namespace, name)` identity is what survives pickle round-trips. A `wool.ContextVar` arriving on the wire from a worker reduces by `(namespace, name)` and re-resolves to the same instance on the receiver.

## wool.Token

### Description
`wool.Token` *is* `contextvars.Token` — re-exported, not wrapped. `set()` returns the native token, `reset(token)` spends it natively, and `Token.MISSING` is stdlib's sentinel for no-prior-binding. Tokens are process-local handles and are not serializable: they never ride the wire, and cross-process token transport is deferred (see [#231](https://github.com/wool-labs/wool/issues/231)).

### Methods

#### reset(token)
Defers to native validation — stdlib raises `ValueError` for a token spent twice, minted by another variable, or minted in a different `contextvars.Context`. Wool layers only its chain bookkeeping on top: a reset that rewinds to no prior value records the pending reset signal so the drop propagates over the wire.

### Notes
- Tokens are process-local handles and are not serializable.
- `Token.MISSING` is stdlib's sentinel for no-prior-binding.
```

--------------------------------

### Plugging ResilientWorker into WorkerPool with partial

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Shows how to use the ResilientWorker with a WorkerPool by passing a partial factory that sets check_interval to 10.0. The pool spawns 4 workers and runs a routine.

```python
import wool
from functools import partial

async with wool.WorkerPool(
    spawn=4,
    worker=partial(ResilientWorker, check_interval=10.0),
):
    result = await my_routine()

```

--------------------------------

### Hybrid WorkerPool: spawn and discovery

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Spawns 4 local workers and discovers remote ones through the same protocol. The pool's total capacity is spawn plus the lease cap if lease is set.

```python
import wool

async with wool.WorkerPool(spawn=4, discovery=wool.LanDiscovery()):
    result = await my_routine()
```

--------------------------------

### Create WorkerCredentials and use with WorkerPool

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Loads PEM certificates and creates gRPC credentials with mutual TLS enabled. The credentials are passed to a WorkerPool with spawn=4, and the pool is used as an async context manager to run a routine.

```python
import wool

creds = wool.WorkerCredentials.from_files(
    ca_path="certs/ca-cert.pem",
    key_path="certs/worker-key.pem",
    cert_path="certs/worker-cert.pem",
    mutual=True,
)

async with wool.WorkerPool(spawn=4, credentials=creds):
    result = await my_routine()
```

--------------------------------

### Create a fixed credential provider with WorkerCredentials.as_provider

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Use this to supply fixed, non-rotating credentials to a WorkerPool. The provider is created from a WorkerCredentials object and names the workers this client will accept via peers. The pool's identity must match the peers policy, otherwise construction is refused or a warning is issued.

```python
import wool

credentials = wool.WorkerCredentials.from_files(
    ca_path="certs/ca-cert.pem",
    key_path="certs/worker-key.pem",
    cert_path="certs/worker-cert.pem",
)
provider = credentials.as_provider(peers="wool-worker.svc")

async with wool.WorkerPool(
    spawn=4,
    identity="wool-worker.svc",
    credentials=provider,
):
    result = await my_routine()
```

--------------------------------

### Hybrid pool with LanDiscovery

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/discovery/README.md

Spawns local workers and publishes them to the LAN, admitting workers from other processes.

```python
import wool

# LAN
async with wool.WorkerPool(spawn=4, discovery=wool.LanDiscovery()):
    result = await my_routine()
```

--------------------------------

### Enable strict mode for wool.SerializationWarning

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/context/README.md

Use this to opt into strict mode, promoting per-entry decode/encode warnings to errors. After this, any wool.SerializationWarning becomes an exception, and failures aggregate into a single wool.ChainSerializationError raised after the loop. Strict mode is fatal to the whole frame, discarding all entries.

```python
import warnings
import wool

warnings.filterwarnings("error", category=wool.SerializationWarning)
```

--------------------------------

### Create a reloadable credential provider with WorkerCredentialsProvider

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Use this to supply rotating credentials to a WorkerPool, LocalWorker, or WorkerProxy. The factory is called to obtain current material, so rotated certificates are adopted without restart. The factory must be safe to call concurrently and must survive serialization into the worker subprocess.

```python
import functools

# Re-read the PEM files each time the factory runs. For an expensive source,
# or to survive a torn read mid-rotation, add caching / last-good fallback here.
factory = functools.partial(
    wool.WorkerCredentials.from_files,
    "certs/ca-cert.pem",
    "certs/worker-key.pem",
    "certs/worker-cert.pem",
)
provider = wool.WorkerCredentialsProvider(
    factory, peers="wool-worker.svc", reloadable=True
)
```

--------------------------------

### Create WorkerCredentials from files and run a secure WorkerPool

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Creates fixed TLS credentials from PEM files and uses them to spawn a secure worker pool. The `mutual=True` flag enables mutual TLS, requiring client authentication. The pool runs `my_routine()` with four workers.

```python
creds = wool.WorkerCredentials.from_files(
    ca_path="certs/ca-cert.pem",
    key_path="certs/worker-key.pem",
    cert_path="certs/worker-cert.pem",
    mutual=True,
)

async with wool.WorkerPool(spawn=4, credentials=creds):
    result = await my_routine()
```

--------------------------------

### WorkerPool

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Documentation for the WorkerPool class, the main entry point for running routines in the wool library.

```APIDOC
## WorkerPool

### Description
`WorkerPool` is the main entry point for running routines. It orchestrates worker subprocess lifecycles, discovery, and load-balanced dispatch. The pool supports four configurations depending on which arguments are provided.

### Constructor
`wool.WorkerPool(spawn=None, discovery=None, lease=None, lazy=True)`

### Parameters
- **spawn** (int) - Optional - Number of local workers to spawn. If omitted, defaults to `cpu_count` local workers with internal `LocalDiscovery`.
- **discovery** (Discovery) - Optional - Discovery mechanism for connecting to existing workers. If omitted, uses internal `LocalDiscovery`.
- **lease** (int) - Optional - Maximum number of additionally discovered workers the pool will admit. Defaults to `None` (unbounded).
- **lazy** (bool) - Optional - Whether the pool's internal `WorkerProxy` defers startup until the first task is dispatched. Defaults to `True`.

### Usage
Use as an async context manager. Inside the context, call routines directly.

### Examples

**Default** — no arguments needed:

```python
async with wool.WorkerPool():
    result = await my_routine()
```

**Ephemeral** — spawn a fixed number of local workers, optionally with tags:

```python
async with wool.WorkerPool("gpu-capable", spawn=4):
    result = await gpu_task()
```

**Durable** — connect to workers already running on the network:

```python
async with wool.WorkerPool(discovery=wool.LanDiscovery()):
    result = await my_routine()
```

**Hybrid** — spawn local workers and discover remote ones:

```python
async with wool.WorkerPool(spawn=4, discovery=wool.LanDiscovery()):
    result = await my_routine()
```

**Lease cap** — spawn 4 local workers, accept up to 4 more from discovery (8 total):

```python
async with wool.WorkerPool(spawn=4, lease=4, discovery=wool.LanDiscovery()):
    result = await my_routine()
```

**Durable pool capped at 10 discovered workers**:

```python
async with wool.WorkerPool(discovery=wool.LanDiscovery(), lease=10):
    result = await my_routine()
```

**Eager proxy startup** — connections established before first dispatch:

```python
async with wool.WorkerPool(spawn=4, lazy=False):
    result = await my_routine()
```

### Notes
- `spawn` controls how many workers the pool starts — it does not cap the total number of workers available. In Hybrid mode, additional workers may join via discovery beyond the initial `spawn`.
- `lease` caps how many additionally discovered workers the pool will admit. The total pool capacity is `spawn + lease` when both are set, or just `lease` for discovery-only pools. Defaults to `None` (unbounded). The lease count is a cap on admission, not a reservation — discovered workers may serve multiple pools simultaneously, and there is no guarantee that a leased slot will remain filled for the life of the pool.
- With `lazy=True`, worker subprocesses that never invoke nested `@wool.routine` calls avoid the cost of discovery subscription and worker-sentinel task setup entirely. Set `lazy=False` to start proxies eagerly — useful when you want connections established before the first dispatch.
```

--------------------------------

### Default WorkerPool: no arguments

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Creates a pool with no arguments, which spawns cpu_count local workers using internal LocalDiscovery. Use for the simplest local execution.

```python
import wool

async with wool.WorkerPool():
    result = await my_routine()
```

--------------------------------

### DiscoveryLike protocol definition

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/discovery/README.md

Defines the required interface for custom discovery protocols. Implement this protocol to integrate a custom discovery backend.

```python
class DiscoveryLike(Protocol):
    @property
    def publisher(self) -> DiscoveryPublisherLike: ...

    @property
    def subscriber(self) -> DiscoverySubscriberLike: ...

    def subscribe(
        self, filter: PredicateFunction | None = None
    ) -> DiscoverySubscriberLike: ...
```

--------------------------------

### Inspect mode: capture wool.SerializationWarning with warnings.catch_warnings

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Use this pattern to enable Inspect mode for wool's decode failures. It captures wool.SerializationWarning instances into a list while still returning the routine's primary signal, providing a best-effort audit trail.

```python
import warnings
import wool

with warnings.catch_warnings(record=True) as captured:
    warnings.simplefilter("always", category=wool.SerializationWarning)
    result = await some_routine()  # always returns
    decode_failures = [w for w in captured if issubclass(w.category, wool.SerializationWarning)]
if decode_failures:
    log.warning("context propagation degraded for %d frame(s)", len(decode_failures))
```

--------------------------------

### Define a Fibonacci async generator with @wool.routine

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Decorates an async generator with @wool.routine for streaming results. The generator yields values one at a time; the worker advances it per client command. All yielded values must be cloudpickle-serializable.

```python
@wool.routine
async def fib(n: int):
    a, b = 0, 1
    for _ in range(n):
        yield a
        a, b = b, a + b
```

--------------------------------

### Configure LocalWorker with ChannelOptions and WorkerOptions

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Configure transport options for a local worker by composing ChannelOptions and WorkerOptions. Pass the WorkerOptions instance to LocalWorker inside a WorkerPool; all options default to gRPC's own defaults unless overridden.

```python
from wool.runtime.worker.base import ChannelOptions, WorkerOptions
from wool.runtime.worker.local import LocalWorker

options = WorkerOptions(
    channel=ChannelOptions(
        keepalive_time_ms=10_000,
        keepalive_timeout_ms=5_000,
        max_concurrent_streams=50,
    ),
    max_connection_idle_ms=300_000,
)

async with wool.WorkerPool(
    spawn=4,
    worker=lambda *tags, credentials=None: LocalWorker(
        *tags,
        credentials=credentials,
        options=options,
    ),
):
    result = await my_routine()
```

--------------------------------

### Register an iteration event handler for streaming

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/routine/README.md

Registers a handler for iteration events on streaming dispatches. Prints the event type, step index, and iteration kind.

```python
@wool.IterationEvent.handler("task-iteration-started", "task-iteration-completed")
def on_iteration(event: wool.IterationEvent, timestamp: int, context=None) -> None:
    print(f"[{event.type}] step={event.step} kind={event.kind}")
```

--------------------------------

### Use WorkerPool as a single-use async context manager

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Shows the correct single-use pattern for WorkerPool as an async context manager. Each instance must be entered and exited exactly once; create a new instance for a second pool. Attempting to re-enter the same instance raises RuntimeError.

```python
# Correct — one instance per context
async with wool.WorkerPool(spawn=4):
    await my_routine()

# Need another pool? Create a new instance.
async with wool.WorkerPool(spawn=4):
    await my_routine()
```

--------------------------------

### Durable pool with LanDiscovery

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/discovery/README.md

Connects to workers already running on the LAN. Uses a bare subscriber and spawns no workers.

```python
import wool

# LAN
async with wool.WorkerPool(discovery=wool.LanDiscovery()):
    result = await my_routine()
```

--------------------------------

### Programmatic worker-side strict mode with os.environ

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Programmatically enable worker-side strict mode before constructing the pool. Workers spawned after this setting will promote wool.SerializationWarning to an exception, which is shipped back through the routine-exception channel.

```python
import os
os.environ["PYTHONWARNINGS"] = "error::wool.SerializationWarning"

import wool

async with wool.WorkerPool():
    ...   # workers spawned now promote the warning to an exception
```

--------------------------------

### Default WorkerPool configuration

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Spawns cpu_count local workers with internal LocalDiscovery. Use when no arguments are needed.

```python
async with wool.WorkerPool():
    result = await my_routine()
```

--------------------------------

### Propagate a wool.ContextVar across a dispatched routine

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Shows how to define a wool.ContextVar with a default, set it in the caller's context, and read it inside a routine dispatched to a worker. The value set with set() is propagated to the worker and printed as "acme-corp"; reset() restores the prior value. Requires a running WorkerPool with spawn=2.

```python
import asyncio

import wool

tenant_id: wool.ContextVar[str] = wool.ContextVar("tenant_id", default="unknown")


@wool.routine
async def handle_request() -> str:
    return tenant_id.get()


async def main():
    async with wool.WorkerPool(spawn=2):
        token = tenant_id.set("acme-corp")
        try:
            result = await handle_request()
            print(result)  # "acme-corp" — propagated to the worker
        finally:
            tenant_id.reset(token)


asyncio.run(main())
```

--------------------------------

### StickyTagBalancer: consistent-hashing load balancer

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/loadbalancer/README.md

Implements a custom load balancer that routes tasks by a stable hash of the task's tag, providing consistent-hashing/affinity routing. The delegate method yields worker UIDs and handles the generator protocol (asend/athrow/anext) to skip failed or departed workers.

```python
import wool


class StickyTagBalancer:
    """Route each task to a worker chosen by hashing its tag."""

    async def delegate(
        self,
        task: wool.Task,
        *,
        context: wool.LoadBalancerContextView,
    ):
        workers = list(context.workers)
        if not workers:
            return

        # Consistent-hashing key: prefer the caller-supplied tag so
        # tasks sharing a tag route together, falling back to the task
        # id so untagged tasks still route deterministically.
        key = task.tag or str(task.id)
        start = hash(key) % len(workers)

        # Probe the hashed worker first, then walk the ring so a single
        # unhealthy worker does not strand the task.
        for offset in range(len(workers)):
            uid = workers[(start + offset) % len(workers)]
            try:
                sent = yield uid
            except Exception:
                # Dispatch failed (reported via athrow). Advance to the
                # next worker on the ring. Catch Exception (not
                # BaseException) so GeneratorExit and CancelledError
                # propagate correctly.
                continue
            if sent is not None:
                # Success echo (asend). The contract requires the
                # generator to terminate after a success.
                return
            # The proxy resumed us with None (anext): the worker left
            # the pool before dispatch, so nothing was tried. Walk on to
            # the next worker on the ring.
```

--------------------------------

### Default load balancer with WorkerPool

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/loadbalancer/README.md

Use when no loadbalancer argument is provided; WorkerPool defaults to RoundRobinLoadBalancer.

```python
import wool

async with wool.WorkerPool(spawn=4):
    result = await my_routine()
```

--------------------------------

### Ephemeral WorkerPool with spawn=4

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Spawns 4 local workers, optionally with tags. Use for ephemeral pools that do not connect to remote workers.

```python
async with wool.WorkerPool("gpu-capable", spawn=4):
    result = await gpu_task()
```

--------------------------------

### Hybrid pool with LocalDiscovery owner

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/discovery/README.md

Spawns local workers and owns the namespace for the life of the block. Other processes can borrow it.

```python
import wool

# Local: owns "my-namespace" for the life of the block
async with wool.WorkerPool(spawn=4, discovery=wool.LocalDiscovery("my-namespace")):
    result = await my_routine()
```

--------------------------------

### Define a recursive Fibonacci coroutine with @wool.routine

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Decorates an async function with @wool.routine. The function is serialized and dispatched to a worker when called. Uses asyncio.TaskGroup to spawn recursive tasks. The function, arguments, return values, and exceptions must be cloudpickle-serializable.

```python
@wool.routine
async def fib(n: int) -> int:
    if n <= 1:
        return n
    async with asyncio.TaskGroup() as tg:
        a = tg.create_task(fib(n - 1))
        b = tg.create_task(fib(n - 2))
    return a.result() + b.result()
```

--------------------------------

### DiscoverySubscriberLike async iterable

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/discovery/README.md

Defines the async iterator protocol for discovery subscribers, yielding DiscoveryEvent objects.

```python
def __aiter__(self) -> AsyncIterator[DiscoveryEvent]: ...
```

--------------------------------

### Custom load balancer with WorkerPool

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/loadbalancer/README.md

Pass any LoadBalancerLike instance (or factory) to WorkerPool to use a custom load balancer.

```python
import wool

balancer = LeastLoadedBalancer()

async with wool.WorkerPool(spawn=4, loadbalancer=balancer):
    result = await my_routine()
```

--------------------------------

### Ephemeral WorkerPool: spawn with tags

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Spawns 4 local workers with a tag 'gpu-capable' and internal LocalDiscovery. The pool does not cap total workers from discovery in this mode.

```python
import wool

async with wool.WorkerPool("gpu-capable", spawn=4):
    result = await gpu_task()
```

--------------------------------

### LeastLoadedBalancer: least-loaded worker selection

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/loadbalancer/README.md

Implements a worker-keyed load balancer that ranks workers by in-flight task count, yielding the least-loaded worker first. Tracks reservations in a dictionary and handles the generator protocol, decrementing counts on failure or when a worker leaves the pool.

```python
import uuid

import wool


class LeastLoadedBalancer:
    """Yield workers in ascending order of in-flight task count."""

    def __init__(self):
        self._in_flight: dict[uuid.UUID, int] = {}

    async def delegate(
        self,
        task: wool.Task,
        *,
        context: wool.LoadBalancerContextView,
    ):
        # Rank workers by current in-flight count (ascending).
        ranked = sorted(
            context.workers,
            key=lambda uid: self._in_flight.get(uid, 0),
        )

        for uid in ranked:
            self._in_flight[uid] = self._in_flight.get(uid, 0) + 1
            try:
                sent = yield uid
            except Exception:
                # Dispatch failed. Decrement and try the next candidate.
                # Catch Exception (not BaseException) so GeneratorExit
                # and CancelledError propagate correctly.
                self._in_flight[uid] -= 1
                continue
            if sent is not None:
                # Success echo (asend); the contract requires the
                # generator to terminate here. The in-flight count stays
                # incremented for the life of this dispatch; the matching
                # decrement happens out-of-band when the stream drains (not
                # shown — requires a lifecycle hook beyond this protocol).
                return
            # The proxy resumed us with None (anext): the worker left the
            # pool before dispatch. Nothing was tried, so release the
            # reservation and rank on.
            self._in_flight[uid] -= 1
```

--------------------------------

### Task exceptions

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/routine/README.md

When a task raises during execution, the exception is captured as a TaskException dataclass with type and traceback fields.

```APIDOC
## Task exceptions

### Description
When a task raises during execution, the exception is captured as a `TaskException` dataclass with two fields:

- **type** — qualified name of the exception class (e.g., `"ValueError"`)
- **traceback** — list of formatted traceback lines

This is set on `task.exception`.
```

--------------------------------

### Durable pool with LocalDiscovery subscriber

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/discovery/README.md

Connects to workers already running in a namespace owned by another process. Uses a bare subscriber and spawns no workers.

```python
import wool

# Local: borrow the registry of whichever instance owns "my-namespace"
async with wool.WorkerPool(discovery=wool.LocalDiscovery.Subscriber("my-namespace")):
    result = await my_routine()
```

--------------------------------

### WorkerConnection.idle

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Documentation for the WorkerConnection.idle method, which polls worker idle duration.

```APIDOC
## WorkerConnection.idle

### Description
Polls how long the remote worker has been continuously idle, returning the duration in seconds. It wraps the `rpc idle (Void) returns (Idle)` on the worker's gRPC service.

### Method
RPC

### Endpoint
`rpc idle (Void) returns (Idle)`

### Parameters
- **timeout** (optional) - The gRPC deadline for the poll itself. Must be positive. `None` (default) applies no deadline.

### Response
- **seconds** (float) - The duration in seconds the worker has been continuously idle.

### Errors
- `IdleUnavailable` - Raised when the worker predates the idle capability and answers with gRPC `UNIMPLEMENTED`. Descends from `WoolError`, not `RpcError`.
- `TransientRpcError` - Raised for transient gRPC codes (e.g., `UNAVAILABLE`, `DEADLINE_EXCEEDED`, `RESOURCE_EXHAUSTED`).
- `RpcError` - Raised for all other gRPC failures.
```

--------------------------------

### Task lifecycle events

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/routine/README.md

Provides lifecycle hooks via event handlers. Handlers are executed on a dedicated thread. Register handlers with the decorator.

```APIDOC
## Task lifecycle events

### Description
Wool provides task lifecycle hooks in the form of events that can be intercepted with event handlers. Handlers are executed on a dedicated thread to avoid blocking the main event loop. `TaskEvent` is an `Event` subclass that carries a reference to the associated `Task`.

### Registering handlers
```python
@wool.TaskEvent.handler("task-created", "task-completed")
def on_task(event: wool.TaskEvent, timestamp: int, context=None) -> None:
    ...
```

### Event types
`TaskEventType` defines the valid event type literals:

| Event type                    | When emitted                                               |
| ----------------------------- | ---------------------------------------------------------- |
| `task-created`                | Task dataclass is instantiated.                            |
| `task-scheduled`              | Worker service begins executing the task.                  |
| `task-started`                | Asyncio handle enters the task's context.                  |
| `task-stopped`                | Asyncio handle exits the task's context.                   |
| `task-completed`              | Asyncio task finishes (success or failure).                |
| `task-iteration-initiated`    | Client is about to write an iteration request to the gRPC stream.  |
| `task-iteration-started`      | Server forwards the iteration command to the worker loop.  |
| `task-iteration-completed`    | Server receives the iteration result from the worker loop. |

`TaskEventHandler` is the protocol for handler callables.
```

--------------------------------

### Durable WorkerPool: discovery only

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Connects to workers already running via discovery without spawning local workers. Use when workers are externally managed.

```python
import wool

async with wool.WorkerPool(discovery=wool.LanDiscovery()):
    result = await my_routine()
```

--------------------------------

### Register a task lifecycle event handler

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/routine/README.md

Registers a handler for task lifecycle events. Handlers run on a dedicated thread to avoid blocking the main event loop. The handler receives the event, a timestamp, and an optional context.

```python
@wool.TaskEvent.handler("task-created", "task-completed")
def on_task(event: wool.TaskEvent, timestamp: int, context=None) -> None:
    ...
```

--------------------------------

### Worker-side strict mode via PYTHONWARNINGS environment variable

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Enable worker-side strict mode by setting the PYTHONWARNINGS environment variable before launching the application. multiprocessing propagates this to spawned worker subprocesses by default, promoting decode failures to exceptions.

```bash
export PYTHONWARNINGS="error::wool.SerializationWarning"
python my_app.py
```

--------------------------------

### Offload work with wool.to_thread()

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Use wool.to_thread() instead of asyncio.to_thread() when offloading work that needs the caller's bindings. It forks a fresh, detached chain for the worker thread, avoiding wool.ChainContention.

```python
result = await wool.to_thread(cpu_bound, payload)
```

--------------------------------

### ResilientWorker class extending LocalWorker with crash recovery

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/worker/README.md

Defines a ResilientWorker that extends LocalWorker to automatically restart the worker process if it crashes. It overrides _start, _stop, and adds a background monitor task that checks WorkerProcess.is_alive() periodically and restarts the subprocess using the original port.

```python
import asyncio
import logging

from wool.runtime.worker.local import LocalWorker
from wool.runtime.worker.process import WorkerProcess

logger = logging.getLogger(__name__)


class ResilientWorker(LocalWorker):
    """LocalWorker that automatically restarts on crash."""

    def __init__(self, *tags, check_interval: float = 5.0, **kwargs):
        super().__init__(*tags, **kwargs)
        self._check_interval = check_interval
        self._monitor_task = None

    async def _start(self, timeout=None):
        await super()._start(timeout=timeout)
        self._monitor_task = asyncio.create_task(self._monitor())

    async def _stop(self, grace=None):
        if self._monitor_task:
            self._monitor_task.cancel()
            try:
                await self._monitor_task
            except asyncio.CancelledError:
                pass
        await super()._stop(grace=grace)

    async def _restart(self):
        """Replace the dead process, reusing the original port."""
        self._worker_process = WorkerProcess(
            host=self._worker_process.host,
            port=self._worker_process.port,
            credentials=self._provider,
        )
        loop = asyncio.get_running_loop()
        await loop.run_in_executor(
            None, lambda: self._worker_process.start(timeout=None)
        )
        logger.info("Worker %s restarted at %s", self.uid, self.address)

    async def _monitor(self):
        """Periodically check if the worker process is alive."""
        while True:
            await asyncio.sleep(self._check_interval)
            if not self._worker_process.is_alive():
                logger.warning("Worker %s crashed, restarting", self.uid)
                await self._restart()

```

--------------------------------

### Task dataclass

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/routine/README.md

Represents a unit of work in the Wool runtime. Each routine call produces a Task with fields for identification, callable, arguments, proxy, caller, and timeout.

```APIDOC
## Task dataclass

### Description
`Task` is the unit of work in the Wool runtime. Each Wool routine call produces a `Task` containing:

- **id** — unique UUID
- **callable** — the function to execute
- **args / kwargs** — serialized arguments
- **proxy** — the `WorkerProxyLike` that dispatches it, enables nested routines
- **caller** — parent task UUID for nested task tracking, `None` for root tasks
- **timeout** — execution timeout in seconds

Tasks are serialized to protobuf for gRPC transmission (`to_protobuf` / `from_protobuf`) using `cloudpickle` for the callable and its arguments.
```

--------------------------------

### @wool.routine decorator

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/routine/README.md

Decorator that converts an async function or async generator into a distributable task. When called, the function is serialized and dispatched to a worker pool, with results streamed back.

```APIDOC
## @wool.routine decorator

### Description
Wraps an async function so that calls are intercepted and routed to the worker pool. Supports async generators for streaming results.

### Usage
```python
@wool.routine
async def fib(n: int) -> int:
    if n <= 1:
        return n
    async with asyncio.TaskGroup() as tg:
        a = tg.create_task(fib(n - 1))
        b = tg.create_task(fib(n - 2))
    return a.result() + b.result()
```

Async generator example:
```python
@wool.routine
async def fib(n: int):
    a, b = 0, 1
    for _ in range(n):
        yield a
        a, b = b, a + b
```

### Notes
- The decorated function, its arguments, returned or yielded values, and exceptions must be serializable via `cloudpickle`.
- Avoid unpicklable objects like open file handles, database connections, and locks.
- Instance, class, and static methods are all supported. Decorator order does not matter.
```

--------------------------------

### Iteration events

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/routine/README.md

Track individual iterations of a streaming (async generator) dispatch. Carried by IterationEvent, a TaskEvent subclass with kind and step fields.

```APIDOC
## Iteration events

### Description
The three `task-iteration-*` event types track individual iterations of a streaming (async generator) dispatch. They are carried by `IterationEvent`, a `TaskEvent` subclass with two additional fields:

- **kind** (`IterationEventKind`) — the operation that drove the iteration: `"next"`, `"send"`, or `"throw"`.
- **step** (`int`) — zero-based iteration index within the stream.

Together, the three event types enable two latency measurements per iteration: network dispatch latency (`initiated` → `started`) and worker execution latency (`started` → `completed`).

### Example handler
```python
@wool.IterationEvent.handler("task-iteration-started", "task-iteration-completed")
def on_iteration(event: wool.IterationEvent, timestamp: int, context=None) -> None:
    print(f"[{event.type}] step={event.step} kind={event.kind}")
```
```

--------------------------------

### Set dispatch_timeout with wool.RuntimeContext

Source: https://github.com/wool-labs/wool/blob/main/wool/src/wool/runtime/context/README.md

Sets a block-scoped dispatch_timeout override for the enclosed routine. The timeout is auto-captured on every Task at construction time and rides the wire on every dispatch, so the worker restores the caller's effective timeout before running the routine.

```python
import wool

with wool.RuntimeContext(dispatch_timeout=30):
    result = await my_routine()
```

--------------------------------

### Catch routine exception with except ValueError

Source: https://github.com/wool-labs/wool/blob/main/wool/README.md

Shows the caller-side exception handling contract: wool re-raises the original routine exception (type and traceback preserved) without wrapping it in an ExceptionGroup or wool-specific wrapper, so existing except clauses like this keep matching. Ancillary decode errors are chained via __cause__ and remain visible in the traceback.

```python
try:
    result = await my_routine()
except ValueError as e:
    print(f"Task failed: {e}")
```