### get_backend_adapter_install_hints Source: https://github.com/curvedinf/wove/blob/main/docs/reference/api/wove.integrations.md Gets installation hints for backend adapters. ```APIDOC ## wove.integrations.get_backend_adapter_install_hints() ### Description Gets installation hints for backend adapters. ### Return type: Dict[str, str] ``` -------------------------------- ### Install Wove with uv Source: https://github.com/curvedinf/wove/blob/main/README.md Use 'uv add wove' to install the Wove library. ```bash uv add wove ``` -------------------------------- ### Install Wove with pip Source: https://github.com/curvedinf/wove/blob/main/README.md Use 'pip install wove' to install the Wove library. ```bash pip install wove ``` -------------------------------- ### start Source: https://github.com/curvedinf/wove/blob/main/_autodocs/backend-adapter.md Initializes the adapter. This method is called once at the start of the executor runtime. The default implementation is a no-operation. ```APIDOC ## Async Method: start Initialize the adapter. Called once at the start of executor runtime. **Default Implementation:** No-op (returns `None`). **Example Override:** ```python class CustomAdapter(BackendAdapter): async def start(self) -> None: # Connect to backend service self.client = await connect_to_backend(self.config) ``` ``` -------------------------------- ### CeleryAdapter Installation Hint Source: https://github.com/curvedinf/wove/blob/main/_autodocs/backend-adapter.md Provides an installation instruction string for the CeleryAdapter when its dependencies are missing. This helps users install the necessary packages. ```python class CeleryAdapter(BackendAdapter): install_hint = 'wove[dev-executors]' # or specific package ``` -------------------------------- ### Install Dispatch Support and RQ Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/rq.md Install the necessary Wove dispatch support and RQ package in both the submitting process and the RQ workers. ```bash pip install "wove[dispatch]" rq ``` -------------------------------- ### Install Wove with Dispatch Source: https://github.com/curvedinf/wove/blob/main/_autodocs/README.md Install Wove with the dispatch extra for background or forking capabilities. ```bash pip install "wove[dispatch]" ``` -------------------------------- ### Async Start Method Override (Python) Source: https://github.com/curvedinf/wove/blob/main/_autodocs/backend-adapter.md Override this async method to perform initialization tasks when the adapter starts, such as connecting to a backend service. ```python class CustomAdapter(BackendAdapter): async def start(self) -> None: # Connect to backend service self.client = await connect_to_backend(self.config) ``` -------------------------------- ### Install Dispatch and Temporal Support Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/temporal.md Install the necessary packages for Wove dispatch and Temporal integration in both the submitting process and Temporal workers. ```bash pip install "wove[dispatch]" temporalio ``` -------------------------------- ### Install Dispatch Support and ARQ Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/arq.md Install the necessary packages for Wove dispatch support and ARQ in both the submitting process and ARQ workers. ```bash pip install "wove[dispatch]" arq ``` -------------------------------- ### Install Dispatch Support and Kubernetes Client Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/kubernetes-jobs.md Install the necessary Python packages for Wove dispatch support and the Kubernetes client. This is required in the submitting process. ```bash pip install "wove[dispatch]" kubernetes ``` -------------------------------- ### Install Dispatch Support and Boto3 Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/aws-batch.md Install the necessary Wove dispatch support and the boto3 library in the submitting process. The worker image must also include Wove with dispatch support. ```bash pip install "wove[dispatch]" boto3 ``` -------------------------------- ### Install Taskiq Dispatch Support Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/taskiq.md Install the necessary packages for Wove dispatch support and Taskiq. This is required for both the submitting process and Taskiq workers. ```bash pip install "wove[dispatch]" taskiq ``` -------------------------------- ### Run Task Command Frame Example Source: https://github.com/curvedinf/wove/blob/main/docs/reference/executors/index.md An example of the 'run_task' command frame, used to request the execution of a specific task with given arguments and delivery configurations. ```python def greet(name): return f"Hello, {name}" { "type": "run_task", "run_id": "task_name:uuid", "task_id": "greet", "callable": greet, "args": {"name": "Ada"}, "delivery": { "delivery_timeout": 30.0, "delivery_cancel_mode": "best_effort", "delivery_idempotency_key": "job:123", }, } ``` -------------------------------- ### Install All Optional Wove Features Source: https://github.com/curvedinf/wove/blob/main/_autodocs/README.md Install Wove with the dev-executors extra to include all optional features, such as dispatch and network executors. ```bash pip install "wove[dev-executors]" ``` -------------------------------- ### Install Wove Dispatch and Celery Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/celery.md Install the necessary packages for Wove dispatch support and Celery in your project and worker environments. ```bash pip install "wove[dispatch]" celery ``` -------------------------------- ### Install Ray and Wove Dispatch Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/ray.md Install the necessary packages for both dispatching and Ray execution. This command should be run in both the submitting process and on Ray workers. ```bash pip install "wove[dispatch]" ray ``` -------------------------------- ### Start an RQ Worker Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/rq.md Start a standard RQ worker process that listens to the specified 'wove' queue. ```bash rq worker wove ``` -------------------------------- ### Configuration Precedence Example Source: https://github.com/curvedinf/wove/blob/main/_autodocs/configuration.md Demonstrates how Wove resolves configuration settings, prioritizing task-level options over weave-level, environment-specific, and global defaults. ```python from wove import config, weave # Global default config.configure(retries=1) # Environment override config.configure(environments={ "fast": {"retries": 0} }) # Weave override with weave(environment="fast", retries=2) as w: # Task uses retries=3 @w.do(retries=3) def task(): return 1 # Resolution: retries=3 (task > weave > environment > global) ``` -------------------------------- ### Install gRPC Executor Dependencies Source: https://github.com/curvedinf/wove/blob/main/docs/reference/executors/grpc-executor.md Install the necessary packages for the gRPC executor, including Wove's dispatch serialization and the Python gRPC runtime. ```bash pip install "wove[dispatch]" grpcio ``` -------------------------------- ### Project Configuration File Example Source: https://github.com/curvedinf/wove/blob/main/docs/reference/environments/index.md Defines project-specific Wove settings, including default environment and environment-specific configurations like executors, timeouts, and broker URLs. This file is optional and loaded from the current working directory or its parents. ```python # wove_config.py WOVE_CONFIG = { "default_environment": "default", "environments": { "default": {"executor": "local"}, "reports": { "executor": "celery", "executor_config": { "broker_url": "redis://redis:6379/0", "task_name": "myapp.wove_task", }, "timeout": 120.0, "delivery_timeout": 30.0, }, }, } ``` -------------------------------- ### Install Dask and Wove Dispatch Dependencies Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/dask.md Install the necessary packages for Wove dispatch and Dask distributed support in both the submitting process and on Dask workers. ```bash pip install "wove[dispatch]" "dask[distributed]" ``` -------------------------------- ### Install WebSocket Executor Dependency Source: https://github.com/curvedinf/wove/blob/main/docs/reference/executors/websocket-executor.md Install the necessary packages for the WebSocket executor, including Wove with dispatch serialization and the Python `websockets` library. ```bash pip install "wove[dispatch]" websockets ``` -------------------------------- ### Wove Config File Example Source: https://github.com/curvedinf/wove/blob/main/_autodocs/runtime-config.md Define runtime configurations such as default environment, worker limits, and environment-specific settings in a Python file. ```python # wove_config.py default_environment = "production" max_workers = 128 background = False fork = False environments = { "default": { "executor": "local", }, "production": { "executor": "http", "executor_config": { "url": "https://worker.example.com", }, }, "celery": { "executor": "backend_adapter", "executor_config": { "backend_adapter": "celery", "broker": "redis://localhost:6379", }, }, } ``` -------------------------------- ### Configure WebSocket Executor Source: https://github.com/curvedinf/wove/blob/main/_autodocs/environment-executors.md Example configuration for a WebSocket executor, specifying the worker URL and authentication token. ```python environments = { "ws_worker": { "executor": "websocket", "executor_config": { "url": "wss://worker.example.com/wove", "security": { "type": "bearer", "token": "auth-token", }, }, }, } ``` -------------------------------- ### Wove Class Data Pipeline Example Source: https://github.com/curvedinf/wove/blob/main/_autodocs/weave-class.md Demonstrates defining a data pipeline using a Wove class with multiple task methods, including dependency mapping and aggregation. Shows how to instantiate and run the pipeline. ```python from wove import weave, Weave class DataPipeline(Weave): @Weave.do def fetch_data(self): """Fetch raw data from a source.""" return [1, 2, 3, 4, 5] @Weave.do(retries=3, timeout=10.0) def process_item(self, item): """Process a single item with retries and timeout.""" return item * 2 @Weave.do("process_item") def map_processing(self, fetch_data): """Map process_item over fetch_data items.""" from wove import merge return merge(self.process_item, fetch_data) @Weave.do def aggregate(self, map_processing): """Combine processed results.""" return sum(map_processing) # Use the class workflow async with weave(DataPipeline) as w: @w.do def final(aggregate): return aggregate + 100 print(w.result.final) # Result: 130 ``` -------------------------------- ### Applying Task Quality of Life Options Source: https://github.com/curvedinf/wove/blob/main/docs/how-to/task-quality-of-life.md This example demonstrates how to apply various task quality-of-life options to `@w.do` tasks. It shows how to configure concurrency limits (`workers`), rate limiting (`limit_per_minute`), retries (`retries`), and timeouts (`timeout`) for a task that fetches customer profiles. ```python from myapp.customers import load_customer_ids, profile_service from wove import weave with weave(raw_customer_ids=load_customer_ids(), max_pending=10_000) as w: @w.do def customer_ids(raw_customer_ids): return raw_customer_ids @w.do("customer_ids", workers=25, limit_per_minute=600, retries=2, timeout=10.0) async def profile(item): return await profile_service.fetch(item) @w.do def by_customer_id(profile): return {row["id"]: row for row in profile} ``` -------------------------------- ### get_backend_adapter_dependencies Source: https://github.com/curvedinf/wove/blob/main/docs/reference/api/wove.integrations.md Gets the required dependencies for all backend adapters. ```APIDOC ## wove.integrations.get_backend_adapter_dependencies() ### Description Gets the required dependencies for all backend adapters. ### Return type: Dict[str, tuple] ``` -------------------------------- ### Configure Backend Environments Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/index.md Configure different execution environments, specifying the executor and its configuration. This example sets up a local default environment and a 'reports' environment using Celery with Redis. ```python import wove from myapp.reports import build_report, load_account from wove import weave wove.config( default_environment="default", environments={ "default": {"executor": "local"}, "reports": { "executor": "celery", "executor_config": { "broker_url": "redis://redis:6379/0", "task_name": "myapp.wove_task", }, "delivery_timeout": 30.0, }, }, ) with weave() as w: @w.do def account(): return load_account() @w.do(environment="reports") def report(account): return build_report(account) ``` -------------------------------- ### Global Wove Configuration and Weave Usage Source: https://github.com/curvedinf/wove/blob/main/_autodocs/runtime-config.md Illustrates a comprehensive global configuration setup for Wove, including defining multiple environments and using the 'weave' context manager to apply configurations to tasks. ```python from wove import config, weave # Configure globally once at startup config.configure( default_environment="local", max_workers=32, retries=3, timeout=30.0, environments={ "local": { "executor": "local", }, "remote": { "executor": "http", "executor_config": { "url": "https://worker.example.com", "security": { "type": "signed", "secret_env": "WORKER_SECRET", } }, }, }, ) # All weaves use the configuration with weave() as w: @w.do(environment="remote") def slow_task(): return "result from remote worker" @w.do # Uses default "local" environment def fast_task(): return "local result" print(w.result.slow_task) print(w.result.fast_task) ``` -------------------------------- ### Run Wove Performance Benchmarks Source: https://github.com/curvedinf/wove/blob/main/README.md Execute the benchmark script to compare Wove's performance against threading and asyncio. Ensure the script is run from the project root to access the examples directory. ```bash $ python examples/benchmark.py Starting performance benchmarks... Number of tasks: 200 CPU load iterations per task: 100000 I/O sleep duration per task: 0.1s =================================== --- Running Threading Benchmark --- Threading total time: 0.6978 seconds ----------------------------------- --- Running Asyncio Benchmark --- Asyncio total time: 0.6831 seconds ----------------------------------- --- Running Wove Benchmark --- Wove timing details: - data: 0.5908s - planning: 0.0001s - tier_1_execution: 0.6902s - tier_1_post_execution: 0.0000s - tier_1_pre_execution: 0.0004s - wove_task: 0.6882s Wove total time: 0.6937 seconds ----------------------------------- --- Running Wove Async Benchmark --- Wove Async timing details: - data: 0.5515s - planning: 0.0000s - tier_1_execution: 0.6550s - tier_1_post_execution: 0.0000s - tier_1_pre_execution: 0.0004s - wove_async_task: 0.6534s Wove Async total time: 0.6571 seconds ----------------------------------- Benchmarks finished. ``` -------------------------------- ### HttpEnvironmentExecutor Configuration Source: https://github.com/curvedinf/wove/blob/main/_autodocs/environment-executors.md Example configuration for the HttpEnvironmentExecutor, specifying the executor type, URL, and security settings. ```python environments = { "remote": { "executor": "http", "executor_config": { "url": "https://worker.example.com", "security": { "type": "signed", "secret": "shared-secret", }, }, }, } ``` -------------------------------- ### Example Usage of LocalEnvironmentExecutor Source: https://github.com/curvedinf/wove/blob/main/_autodocs/environment-executors.md Demonstrates how to configure and use the LocalEnvironmentExecutor for local task execution within a Wove context. Tasks decorated with @w.do will run in the current process. ```python from wove import weave, config config.configure( environments={ "local": {"executor": "local"} } ) with weave(environment="local") as w: @w.do def local_task(): return "runs in this process" print(w.result.final) ``` -------------------------------- ### Resolve Environment Name with Defaults Source: https://github.com/curvedinf/wove/blob/main/_autodocs/runtime-config.md Demonstrates how to get the effective environment name. If no name is provided, it defaults to the configured default environment. ```python from wove import config config.configure(default_environment="my_env") name = config.resolve_environment_name(None) # Returns "my_env" name = config.resolve_environment_name("prod") # Returns "prod" ``` -------------------------------- ### Implement Custom Backend Adapter Source: https://github.com/curvedinf/wove/blob/main/_autodocs/backend-adapter.md Create a custom backend adapter by extending Wove's BackendAdapter class. Implement start, submit, cancel, and close methods for custom task handling. ```python from wove.integrations.base import BackendAdapter from typing import Dict, Any import httpx class MyCustomAdapter(BackendAdapter): required_modules = ("requests",) install_hint = "pip install requests" async def start(self) -> None: self.http_client = httpx.AsyncClient() async def submit(self, payload: Dict[str, Any], frame: Dict[str, Any]) -> str: import cloudpickle task_data = { "callable": cloudpickle.dumps(frame["callable"]), "args": cloudpickle.dumps(frame["args"]), "callback_url": payload["callback_url"], "run_id": frame["run_id"], "task_id": frame["task_id"], } response = await self.http_client.post( f"{self.config['backend_url']}/submit", json=task_data, ) return response.json()["job_id"] async def cancel(self, run_id: str, submission: Any, frame: Dict[str, Any]) -> None: await self.http_client.post( f"{self.config['backend_url']}/cancel/{submission}", ) async def close(self) -> None: await self.http_client.aclose() # Register the adapter from wove.integrations import register_backend_adapter register_backend_adapter("my_backend", MyCustomAdapter) # Use it from wove import config, weave config.configure( environments={ "my_backend": { "executor": "backend_adapter", "executor_config": { "backend_adapter": "my_backend", "backend_url": "https://job-queue.example.com", }, }, } ) with weave(environment="my_backend") as w: @w.do def long_running_task(): return "computed on my backend" ``` -------------------------------- ### Example Usage of BackendAdapterEnvironmentExecutor with Celery Source: https://github.com/curvedinf/wove/blob/main/_autodocs/environment-executors.md Shows how to configure Wove to use the BackendAdapterEnvironmentExecutor with Celery. Tasks decorated with @w.do will be submitted to Celery for distributed execution. ```python from wove import weave, config config.configure( environments={ "celery": { "executor": "backend_adapter", "executor_config": { "backend_adapter": "celery", "broker": "redis://localhost:6379", "backend": "redis://localhost:6379", }, }, } ) with weave(environment="celery") as w: @w.do def long_running(): # Submitted to Celery instead of running locally return "executed by Celery" print(w.result.final) ``` -------------------------------- ### Configure Global Runtime Instance Source: https://github.com/curvedinf/wove/blob/main/_autodocs/runtime-config.md Demonstrates how to configure the global Wove runtime instance with specific parameters like max_workers. This is useful for setting up the global configuration once at the start of your application. ```python from wove.runtime import runtime runtime.configure(max_workers=64) ``` -------------------------------- ### GrpcEnvironmentExecutor Configuration Source: https://github.com/curvedinf/wove/blob/main/_autodocs/environment-executors.md Example configuration for the GrpcEnvironmentExecutor, specifying the executor type, gRPC target, method, and security settings. ```python environments = { "grpc_worker": { "executor": "grpc", "executor_config": { "target": "worker.example.com:50051", "method": "/wove.network_executor.WorkerService/Send", "security": { "type": "signed", "secret": "shared-secret", }, }, }, } ``` -------------------------------- ### HttpEnvironmentExecutor Usage Example Source: https://github.com/curvedinf/wove/blob/main/_autodocs/environment-executors.md Demonstrates configuring and using the HttpEnvironmentExecutor to run a task on a remote worker via HTTP/HTTPS with bearer token authentication. ```python from wove import weave, config config.configure( environments={ "remote": { "executor": "http", "executor_config": { "url": "https://worker.example.com/wove", "security": { "type": "bearer", "token_env": "WORKER_TOKEN", }, }, }, } ) with weave(environment="remote") as w: @w.do def remote_task(): # Sent to remote worker return "computed remotely" print(w.result.final) ``` -------------------------------- ### Minimal AcmeQueueAdapter Implementation Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/custom-backend-adapters.md This Python code defines a custom backend adapter for a fictional queue client. It demonstrates how to implement the `start`, `submit`, `cancel`, and `close` methods to integrate with an external queue system, ensuring the Wove payload is enqueued unchanged. ```python from wove import BackendAdapter class AcmeQueueAdapter(BackendAdapter): required_modules = ("acme_queue",) install_hint = "acme-queue" async def start(self): import acme_queue self.client = self.config.get("client") if self.client is None: self.client = acme_queue.Client(self.config["url"]) self.queue = self.config.get("queue", "default") async def submit(self, payload, frame): delivery = frame.get("delivery") or {} return await self.client.enqueue( self.queue, payload, job_id=delivery.get("delivery_idempotency_key") or frame["run_id"], ) async def cancel(self, run_id, submission, frame): if submission is not None: await self.client.cancel(submission) async def close(self): await self.client.close() ``` -------------------------------- ### Submit Tasks to a Remote Worker Service Source: https://github.com/curvedinf/wove/blob/main/docs/how-to/patterns-for-production.md In the submitting process, configure Wove to use a specific environment (e.g., 'workers') for certain tasks. This example shows how to load accounts, build reports using a remote worker, and then process the report index locally. ```python # submitting process import wove from myapp.reports import build_report, load_accounts from wove import weave wove.config() with weave() as w: @w.do def accounts(): return load_accounts() @w.do("accounts", workers=8, environment="workers") def report(item): return build_report(item) @w.do def report_index(report): return {row["account_id"]: row for row in report} ``` -------------------------------- ### Raise MissingDispatchFeatureError Example Source: https://github.com/curvedinf/wove/blob/main/_autodocs/errors.md This code snippet demonstrates how to trigger a MissingDispatchFeatureError. It will be raised if the 'cloudpickle' library is not installed when attempting to use background execution with forking. ```python from wove import weave # This raises MissingDispatchFeatureError if cloudpickle is not installed with weave(background=True, fork=True) as w: @w.do def task(): return 1 ``` -------------------------------- ### EnvironmentExecutor.start Source: https://github.com/curvedinf/wove/blob/main/_autodocs/environment-executors.md Initializes the executor for a specific environment, setting up the connection and configuration for task execution. ```APIDOC ## EnvironmentExecutor.start ### Description Initialize the executor for a named environment. ### Method `start` ### Parameters #### Path Parameters - **environment_name** (str) - Required - Name of the environment being initialized. - **environment_config** (Dict[str, Any]) - Required - Configuration for the environment (includes `executor`, `executor_config`). - **run_config** (Dict[str, Any]) - Required - Run-time settings (e.g., `error_mode`, `max_pending`). ``` -------------------------------- ### Cancel Task Command Frame Example Source: https://github.com/curvedinf/wove/blob/main/docs/reference/executors/index.md An example of the 'cancel_task' command frame, used to request the cancellation of a previously submitted task run. ```python { "type": "cancel_task", "run_id": "task_name:uuid", "task_id": "task_name", "delivery_cancel_mode": "best_effort", "delivery_orphan_policy": "requeue", } ``` -------------------------------- ### Initialize Security with None Configuration Source: https://github.com/curvedinf/wove/blob/main/_autodocs/security.md Use when no security is required. The security mode defaults to 'none'. ```python config = None security = NetworkExecutorSecurity.from_config(config) # mode="none" ``` -------------------------------- ### Raise UnresolvedSignatureError Example Source: https://github.com/curvedinf/wove/blob/main/_autodocs/errors.md This example shows how an UnresolvedSignatureError can occur. It is raised when a task parameter cannot be resolved because no matching task name is found upstream within the weave context. ```python from wove import weave with weave() as w: @w.do def task_a(nonexistent_task): # No task named 'nonexistent_task' return 1 # Raises UnresolvedSignatureError when the weave block exits ``` -------------------------------- ### Taskiq Worker Task Example Source: https://github.com/curvedinf/wove/blob/main/docs/reference/backend-adapters/taskiq.md Define a Taskiq worker task that uses Wove's `arun` function to process payloads. This example uses an InMemoryBroker for demonstration purposes; use your production broker in real deployments. ```python from taskiq import InMemoryBroker from wove.integrations.worker import arun broker = InMemoryBroker() @broker.task(task_name="myapp.wove_task") async def run_wove_payload(payload): return await arun(payload) ``` -------------------------------- ### Set WOVE_TOKEN Environment Variable Source: https://github.com/curvedinf/wove/blob/main/_autodocs/configuration.md Example of how to export the WOVE_TOKEN environment variable in a bash shell. ```bash export WOVE_TOKEN="my-api-token" ``` -------------------------------- ### Backend Adapter Environment Configuration (Celery, Temporal, Ray) Source: https://github.com/curvedinf/wove/blob/main/_autodocs/configuration.md Configure environments using backend adapters for Celery, Temporal, or Ray, specifying connection details and queues. ```json { "celery_queue": { "executor": "backend_adapter", "executor_config": { "backend_adapter": "celery", "broker": "redis://localhost:6379", "backend": "redis://localhost:6379", "queue": "wove_tasks" } }, "temporal": { "executor": "backend_adapter", "executor_config": { "backend_adapter": "temporal", "target": "localhost:7233", "namespace": "default" } }, "ray_cluster": { "executor": "backend_adapter", "executor_config": { "backend_adapter": "ray", "address": "ray://localhost:10001" } } } ``` -------------------------------- ### Set WOVE_SECRET Environment Variable Source: https://github.com/curvedinf/wove/blob/main/_autodocs/configuration.md Example of how to export the WOVE_SECRET environment variable in a bash shell. ```bash export WOVE_SECRET="my-shared-secret" ``` -------------------------------- ### Basic Task Execution with Wove Source: https://github.com/curvedinf/wove/blob/main/_autodocs/INDEX.md Demonstrates defining and running simple tasks synchronously. Tasks are decorated with `@w.do` and results are accessed via `w.result.final`. ```python from wove import weave with weave() as w: @w.do def task_a(): return 1 @w.do def task_b(): return 2 @w.do def combined(task_a, task_b): return task_a + task_b print(w.result.final) # 3 ``` -------------------------------- ### CeleryAdapter Required Modules Source: https://github.com/curvedinf/wove/blob/main/_autodocs/backend-adapter.md Specifies the Python modules that must be installed for the CeleryAdapter to function. This is used for dependency checking. ```python class CeleryAdapter(BackendAdapter): required_modules = ("celery", "kombu") ``` -------------------------------- ### Initialize Signed Security with Inline Secret Source: https://github.com/curvedinf/wove/blob/main/_autodocs/security.md Configures signed security using an inline secret and key ID. Specifies clock skew tolerance in seconds. ```python config = { "type": "signed", "secret": "inline-secret", "key": "key-id", "clock_skew_seconds": 600.0, } security = NetworkExecutorSecurity.from_config(config) ``` -------------------------------- ### EnvironmentExecutor Abstract Base Class Source: https://github.com/curvedinf/wove/blob/main/_autodocs/environment-executors.md Defines the abstract contract for executor transport, including methods for starting, sending, receiving, and stopping. ```python class EnvironmentExecutor(abc.ABC): @abc.abstractmethod async def start( self, *, environment_name: str, environment_config: Dict[str, Any], run_config: Dict[str, Any], ) -> None @abc.abstractmethod async def send(self, frame: Dict[str, Any]) -> None @abc.abstractmethod async def recv(self) -> Dict[str, Any] @abc.abstractmethod async def stop(self) -> None ``` -------------------------------- ### Wove Execution with Initial Seed Values Source: https://github.com/curvedinf/wove/blob/main/_autodocs/weave-context-manager.md Shows how to initialize the Wove context with seed values like `user_id` and `api_key`. These values can be passed to tasks as arguments, as demonstrated in `fetch_user` and `authenticate`. ```python with weave(user_id=123, api_key="secret") as w: @w.do def fetch_user(data): return {"id": data["user_id"], "name": "Alice"} @w.do def authenticate(data): return data["api_key"] == "secret" ``` -------------------------------- ### Configure Environments and Handle Delivery Timeouts Source: https://github.com/curvedinf/wove/blob/main/_autodocs/errors.md Configure custom environments with specific executor settings and set a delivery_timeout to catch backend task delivery failures. ```python from wove import weave, config config.configure( environments={ "slow_backend": { "executor": "http", "executor_config": { "url": "https://worker.example.com", }, }, } ) with weave(environment="slow_backend", delivery_timeout=1.0) as w: @w.do def task(): return 1 # Raises DeliveryTimeoutError if delivery takes > 1 second ``` -------------------------------- ### BackendCallbackServer Source: https://github.com/curvedinf/wove/blob/main/docs/reference/api/wove.backend.md A small HTTP receiver that turns backend worker callbacks into executor frames. It allows for receiving, starting, and stopping callback events. ```APIDOC ## class wove.backend.BackendCallbackServer(, host='127.0.0.1', port=0, public_url=None, token=None) ### Description Small HTTP receiver that turns backend worker callbacks into executor frames. ### Parameters: * **host** (str) * **port** (int) * **public_url** (str | None) * **token** (str | None) #### async recv() * **Return type:** Dict[str, Any] #### async start() * **Return type:** None #### async stop() * **Return type:** None ``` -------------------------------- ### Override close Method Source: https://github.com/curvedinf/wove/blob/main/_autodocs/backend-adapter.md Override the `close` method to clean up resources when the executor stops. This example demonstrates closing an asynchronous client connection. ```python async def close(self) -> None: if hasattr(self, "client"): await self.client.close() ``` -------------------------------- ### Initialize Signed Security with Environment Variable Secret Source: https://github.com/curvedinf/wove/blob/main/_autodocs/security.md Configures signed security using a secret from an environment variable and a specified key ID. Ensure the environment variable is set. ```python config = { "type": "signed", "secret_env": "WORKER_SECRET", "key": "key-id", } security = NetworkExecutorSecurity.from_config(config) ``` -------------------------------- ### Getting the Number of Completed Tasks Source: https://github.com/curvedinf/wove/blob/main/_autodocs/wove-result.md Returns the count of successfully completed tasks. This is useful for verifying the number of tasks that finished without errors. ```python with weave() as w: @w.do def task1(): return 1 @w.do def task2(): return 2 print(len(w.result)) # 2 ``` -------------------------------- ### Override cancel Method Source: https://github.com/curvedinf/wove/blob/main/_autodocs/backend-adapter.md Override the `cancel` method to implement task cancellation logic for a specific backend. This example shows how to revoke a Celery task. ```python async def cancel(self, run_id: str, submission: Any, frame: Dict[str, Any]) -> None: # submission is the task ID from celery celery_app.control.revoke(submission, terminate=True) ``` -------------------------------- ### Resolve Environment Settings with Overrides Source: https://github.com/curvedinf/wove/blob/main/_autodocs/runtime-config.md Shows how to retrieve merged environment settings. It prioritizes environment-specific overrides over global defaults. ```python from wove import config config.configure( max_workers=64, environments={ "fast": { "max_workers": 256, } } ) settings = config.resolve_environment_settings("fast") print(settings["max_workers"]) # 256 (environment override) settings = config.resolve_environment_settings("default") print(settings["max_workers"]) # 64 (global default) ``` -------------------------------- ### Initialize WebSocketEnvironmentExecutor Source: https://github.com/curvedinf/wove/blob/main/_autodocs/environment-executors.md Defines the constructor for the WebSocketEnvironmentExecutor class. Requires the 'websockets' package. ```python class WebSocketEnvironmentExecutor(EnvironmentExecutor): def __init__(self) -> None ``` -------------------------------- ### BackendAdapter Class Definition Source: https://github.com/curvedinf/wove/blob/main/_autodocs/backend-adapter.md Defines the interface for backend adapters. Subclasses must implement methods for starting, submitting tasks, canceling runs, and closing connections. ```python class BackendAdapter: required_modules: Tuple[str, ...] = () install_hint: Optional[str] = None def __init__( self, *, name: str, config: Dict[str, Any], callback_url: str, run_config: Dict[str, Any], ) -> None @classmethod def missing_dependencies(cls) -> Tuple[str, ...] async def start(self) -> None async def submit(self, payload: Dict[str, Any], frame: Dict[str, Any]) -> Any async def cancel(self, run_id: str, submission: Any, frame: Dict[str, Any]) -> None async def close(self) -> None ``` -------------------------------- ### Worker Service Response Format Source: https://github.com/curvedinf/wove/blob/main/docs/reference/executors/http-executor.md Example of the JSON response format expected from a worker service handling Wove command frames. It should contain a list of event frames. ```json { "events": [ {"type": "task_started", "run_id": "...", "task_id": "render_report"}, {"type": "task_result", "run_id": "...", "task_id": "render_report", "result_pickle": "..."} ] } ``` -------------------------------- ### Handle Invalid Configuration with ValueError Source: https://github.com/curvedinf/wove/blob/main/_autodocs/errors.md Demonstrates how Wove raises a ValueError for invalid configuration settings. This example shows an invalid 'error_mode' being set, which would trigger the error. ```python from wove import weave # Raises ValueError with weave(error_mode="invalid") as w: pass ``` -------------------------------- ### Global Configuration Source: https://github.com/curvedinf/wove/blob/main/_autodocs/configuration.md Configure Wove globally once at startup using `wove.config()`. All subsequent weaves inherit these settings. ```python from wove import config config.configure( default_environment="production", max_workers=64, retries=3, timeout=30.0, environments={ "local": {"executor": "local"}, "production": {"executor": "http", "executor_config": {...}}, }, ) ``` -------------------------------- ### Map Over Task Results Source: https://github.com/curvedinf/wove/blob/main/docs/how-to/task-mapping.md Map over a data dependency name (string) when the iterable is produced by another task. Wove waits for the upstream task to complete before starting mapped instances. ```python import asyncio from wove import weave async def main(): step = 10 async with weave(min=10, max=40) as w: # Generates the data we want to map over. @w.do async def numbers(min, max): # This scope can read local variables outside the `weave` block, but # passing them in as initialization data is cleaner. return range(min, max, step) # Map each item produced by `numbers` to the `squares` function. # Each item's instance of `squares` will run concurrently, and then # be collected as a list after all have completed. @w.do("numbers") async def squares(item): return item * item # Collects the results. # You can mix `async def` and `def` tasks. @w.do def summary(squares): return f"Sum of squares: {sum(squares)}" print(w.result.final) asyncio.run(main()) ``` -------------------------------- ### Initialize Security with Environment Variable Secret Source: https://github.com/curvedinf/wove/blob/main/_autodocs/security.md Reads the signing secret from a specified environment variable. Ensure the environment variable is set before execution. ```python config = "env:WORKER_SECRET" security = NetworkExecutorSecurity.from_config(config) ```