### Simple command loop pattern Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Setup a serving loop for synchronous function calls. ```python import os ``` -------------------------------- ### Install execnet via pip Source: https://execnet.readthedocs.io/en/latest/install.html Use this command to install the execnet package from PyPI. Ensure you have pip installed. ```bash pip install execnet ``` -------------------------------- ### Start remote socket server Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Command to start the standalone socketserver script for remote bootstrapping. ```bash python socketserver.py :8888 # bind to all IPs, port 8888 ``` -------------------------------- ### Example execnet trace output Source: https://execnet.readthedocs.io/en/latest/example/test_debug.html Sample output showing PID-prefixed trace entries generated when debugging is enabled. ```text [2326] gw0 starting to receive [2326] gw0 sent [2327] creating workergateway on [2327] gw0-worker starting to receive [2327] gw0-worker received [2327] gw0-worker execution starts[1]: '42' [2327] gw0-worker execution finished [2327] gw0-worker sent [2327] gw0-worker 1 sent channel close message [2326] gw0 received [2326] gw0 1 channel.__del__ [2326] === atexit cleanup === [2326] gw0 gateway.exit() called [2326] gw0 --> sending GATEWAY_TERMINATE [2326] gw0 sent [2326] gw0 joining receiver thread [2327] gw0-worker received [2327] gw0-worker putting None to execqueue [2327] gw0-worker io.close_read() [2327] gw0-worker leaving [2327] gw0-worker 1 channel.__del__ [2327] gw0-worker io.close_write() [2327] gw0-worker workergateway.serve finished [2327] gw0-worker gateway.join() called while receiverthread already finished [2326] gw0 leaving ``` -------------------------------- ### Install execnet via pip Source: https://execnet.readthedocs.io/en/latest/index.html Use this command to install the latest version of execnet from the Python Package Index. ```bash pip install -U execnet ``` -------------------------------- ### Configure and Test Threading Models Source: https://execnet.readthedocs.io/en/latest/basics.html Sets the execution model for local and remote sides before creating a gateway. Requires eventlet to be installed in the environment. ```python # content of threadmodel.py import execnet # locally use "eventlet", remotely use "thread" model execnet.set_execmodel("eventlet", "thread") gw = execnet.makegateway() print (gw) print (gw.remote_status()) print (gw.remote_exec("channel.send(1)").receive()) ``` ```bash $ python threadmodel.py 1 ``` -------------------------------- ### Saturate multiple hosts/CPUs with tasks Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html A comprehensive example of distributing tasks across multiple gateways (simulating CPUs/hosts) and managing their execution and termination. This pattern is suitable for high-throughput parallel processing. ```python from __future__ import annotations import execnet group = execnet.Group() for i in range(4): # 4 CPUs group.makegateway() def process_item(channel): # task processor, sits on each CPU import time import random channel.send("ready") for x in channel: if x is None: # we can shutdown break # sleep random time, send result time.sleep(random.randrange(3)) channel.send(x * 10) # execute taskprocessor everywhere mch = group.remote_exec(process_item) # get a queue that gives us results q = mch.make_receive_queue(endmarker=-1) tasks: list[int] | None = list(range(10)) # a list of tasks, here just integers terminated = 0 while 1: channel, item = q.get() if item == -1: terminated += 1 print("terminated %s" % channel.gateway.id) if terminated == len(mch): print("got all results, terminating") break continue if item != "ready": print(f"other side {channel.gateway.id} returned {item!r}") if not tasks and tasks is not None: print("no tasks remain, sending termination request to all") mch.send_each(None) tasks = None if tasks: task = tasks.pop() channel.send(task) print(f"sent task {task!r} to {channel.gateway.id}") group.terminate() ``` -------------------------------- ### Connect to Python 2 and Numpy from Python 3 Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html Demonstrates remote execution on a Python 2.7 interpreter with Numpy installed, sending data to be processed and receiving the result. ```python import execnet gw = execnet.makegateway("popen//python=python2.7") channel = gw.remote_exec(""" import numpy array = numpy.array([1,2,3]) while 1: x = channel.receive() if x is None: break array = numpy.append(array, x) channel.send(repr(array)) """) for x in range(10): channel.send(x) channel.send(None) print (channel.receive()) ``` ```text array([1, 2, 3, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) ``` -------------------------------- ### Get Gateway IDs Before Instantiation Source: https://execnet.readthedocs.io/en/latest/example/test_group.html Determine a gateway's ID before it's created by using execnet.XSpec and group.allocate_id. This is useful for pre-planning gateway identification. ```python import execnet group = execnet.Group() spec = execnet.XSpec("popen") group.allocate_id(spec) allocated_id = spec.id gw = group.makegateway(spec) assert gw.id == allocated_id ``` -------------------------------- ### Get information from remote SSH account Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Obtain environment information from a remote SSH host. ```python >>> import execnet, os >>> gw = execnet.makegateway("ssh=codespeak.net") >>> channel = gw.remote_exec(""" ... import sys, os ... channel.send((sys.platform, tuple(sys.version_info), os.getpid())) ... """) >>> platform, version_info, remote_pid = channel.receive() >>> platform 'linux2' >>> version_info (2, 6, 6, 'final', 0) ``` -------------------------------- ### Create a proxied gateway using execnet Source: https://execnet.readthedocs.io/en/latest/example/test_proxy.html Demonstrates initializing a master gateway and using it to control worker gateways via the via=master specification. ```python >>> import execnet >>> group = execnet.Group() >>> group.defaultspec = 'popen//via=master' >>> master = group.makegateway('popen//id=master') >>> master >>> worker = group.makegateway() >>> worker >>> group ``` -------------------------------- ### Receive results with a Queue from MultiChannel Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html Shows how to create a receive queue from a MultiChannel to process results sequentially. Useful for ordered retrieval of data from multiple channels. ```python >>> ch1 = execnet.makegateway().remote_exec("channel.send(1)") >>> ch2 = execnet.makegateway().remote_exec("channel.send(2)") >>> mch = execnet.MultiChannel([ch1, ch2]) >>> queue = mch.make_receive_queue() >>> chan1, res1 = queue.get() >>> chan2, res2 = queue.get(timeout=3) >>> res1 + res2 3 ``` -------------------------------- ### Create and use MultiChannel Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html Demonstrates creating a MultiChannel from individual channels and performing operations like checking length and membership. Use when managing multiple concurrent communication streams. ```python import execnet >>> ch1 = execnet.makegateway().remote_exec("channel.send(1)") >>> ch2 = execnet.makegateway().remote_exec("channel.send(2)") >>> mch = execnet.MultiChannel([ch1, ch2]) >>> len(mch) 2 >>> mch[0] is ch1 and mch[1] is ch2 True >>> ch1 in mch and ch2 in mch True >>> sum(mch.receive_each()) 3 ``` -------------------------------- ### Instantiate a Default Gateway Source: https://execnet.readthedocs.io/en/latest/basics.html Creates a simple Python subprocess gateway using default settings. ```python >>> gateway = execnet.makegateway() ``` -------------------------------- ### Asynchronous channel communication with callbacks Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html Illustrates setting up a callback function to process incoming data asynchronously without blocking execution. Ideal for event-driven processing of channel messages. ```python >>> import execnet >>> gw = execnet.makegateway() >>> ch = gw.remote_exec("channel.receive() ; channel.send(42)") >>> l = [] >>> ch.setcallback(l.append) >>> ch.send(1) >>> ch.waitclose() >>> assert l == [42] ``` -------------------------------- ### Execute remote commands locally Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Demonstrates how to trigger functions defined in a remote module via a gateway channel. ```python >>> import execnet, remotecmd >>> gw = execnet.makegateway() >>> ch = gw.remote_exec(remotecmd) >>> ch.send('simple(10)') # execute func-call remotely >>> ch.receive() 11 ``` -------------------------------- ### Configure Gateway Options Source: https://execnet.readthedocs.io/en/latest/basics.html Available configuration keys for gateway specifications. ```text id= specifies the gateway id python= specifies which python interpreter to execute execmodel=model 'thread', 'main_thread_only', 'eventlet', 'gevent' execution model chdir= specifies to which directory to change nice= specifies process priority of new process env:NAME=value specifies a remote environment variable setting. ``` -------------------------------- ### Debugging Configuration Source: https://execnet.readthedocs.io/en/latest/basics.html Environment variable configuration for tracing execnet execution. ```APIDOC ## Debugging ### Environment Variable: EXECNET_DEBUG - **EXECNET_DEBUG=1**: Write per-process trace-files to `execnet-debug-PID`. - **EXECNET_DEBUG=2**: Perform tracing to stderr. ``` -------------------------------- ### Basic RSync Usage Source: https://execnet.readthedocs.io/en/latest/basics.html Use this to synchronize a local directory to a remote destination via an execnet gateway. Ensure the source directory exists locally and the destination directory is accessible on the remote gateway. ```python rsync = execnet.RSync('/tmp/source') gw = execnet.makegateway() rsync.add_target(gw, '/tmp/dest') rsync.send() ``` -------------------------------- ### Execute source code in subprocess Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Instantiate a subprocess gateway to execute code and communicate bidirectionally. ```python >>> import execnet >>> gw = execnet.makegateway() >>> channel = gw.remote_exec("channel.send(channel.receive()+1)") >>> channel.send(1) >>> channel.receive() 2 ``` -------------------------------- ### execnet.makegateway Source: https://execnet.readthedocs.io/en/latest/basics.html Creates and configures a gateway to a Python interpreter based on a specification string. ```APIDOC ## execnet.makegateway(spec) ### Description Create and configure a gateway to a Python interpreter. The spec string encodes the target gateway type and configuration information. ### Parameters #### Request Body - **spec** (string) - Required - The specification string (e.g., 'popen', 'ssh=hostname', 'socket=host:port'). Supports configurations like id, python, execmodel, chdir, nice, and env:NAME. ### Request Example ``` >>> gateway = execnet.makegateway('popen//python=python3.3') ``` ``` -------------------------------- ### Compare current working directories Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Verify the working directory of a local subprocess gateway. ```python >>> import execnet, os >>> gw = execnet.makegateway() >>> ch = gw.remote_exec("import os; channel.send(os.getcwd())") >>> res = ch.receive() >>> assert res == os.getcwd() ``` -------------------------------- ### Manage Gateways with execnet.Group Source: https://execnet.readthedocs.io/en/latest/example/test_group.html Use execnet.Group to manage a collection of gateways. It handles membership, lifetime, and provides convenient access to individual gateways. ```python import execnet group = execnet.Group(['popen'] * 2) print(len(group)) print(group) print(list(group)) print('gw0' in group and 'gw1' in group) print(group['gw0'] == group[0]) print(group['gw1'] == group[1]) group.terminate() # exit all member gateways print(group) ``` -------------------------------- ### Connect to remote socket gateway Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Instantiate a gateway by connecting to a remote socket endpoint. ```python import execnet gw = execnet.makegateway("socket=TARGET-IP:8888") ``` -------------------------------- ### Python Client to Retrieve Remote File Contents Source: https://execnet.readthedocs.io/en/latest/example/test_ssh_fileserver.html This Python code demonstrates how to connect to a remote server using execnet, execute the `servefiles` function, and retrieve file contents. It requires the `servefiles.py` script to be available on the remote host. ```python import execnet import servefiles gw = execnet.makegateway("ssh=codespeak.net") channel = gw.remote_exec(servefiles) for fn in ('/etc/passwd', '/etc/group'): channel.send(fn) content = channel.receive() print(fn) print(content) ``` -------------------------------- ### execnet Gateway Methods Source: https://execnet.readthedocs.io/en/latest/genindex.html Details on methods available for interacting with and managing execnet gateways. ```APIDOC ## Gateway Management ### Description Methods for controlling and querying the state of an execnet gateway. ### Methods - `reconfigure()` - `remote_exec()` - `remote_status()` ### Endpoint N/A (Methods on a Gateway object) ### Parameters - **remote_exec(source, **kwargs)**: `source` (str) - Python code to execute remotely. - **remote_status()**: No parameters. ### Request Example ```python # Assuming 'gateway' is an active execnet gateway object result = gateway.remote_exec("print('Hello from remote!')") status = gateway.remote_status() ``` ### Response - **remote_exec**: Returns an object representing the remote execution context. - **remote_status**: Returns information about the gateway's status. ``` -------------------------------- ### Enable execnet debugging Source: https://execnet.readthedocs.io/en/latest/example/test_debug.html Set the EXECNET_DEBUG environment variable to 2 to output trace information to stderr. ```bash EXECNET_DEBUG=2 # or "set EXECNET_DEBUG=2" on windows python -c 'import execnet ; execnet.makegateway().remote_exec("42")' ``` -------------------------------- ### Work with C# objects from CPython Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html Connect to an IronPython interpreter to instantiate and manipulate C# classes via the CLR. ```python import execnet gw = execnet.makegateway("popen//python=ipy") channel = gw.remote_exec(""" import clr clr.AddReference("System") from System import Array array = Array[float]([1,2]) channel.send(str(array)) """) print (channel.receive()) ``` ```text System.Double[](1.0, 2.0) ``` -------------------------------- ### Robustly receive results and termination notification Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html Demonstrates using an endmarker with `make_receive_queue` to identify channel closures or gateway terminations. Essential for reliable cleanup and status monitoring in distributed tasks. ```python >>> group = execnet.Group(['popen'] * 3) # create three gateways >>> mch = group.remote_exec("channel.send(channel.receive()+1)") >>> queue = mch.make_receive_queue(endmarker=42) >>> mch[0].send(1) >>> chan1, res1 = queue.get() >>> res1 2 >>> group.terminate(timeout=1) # kill processes waiting on receive >>> for i in range(3): ... chan1, res1 = queue.get() ... assert res1 == 42 >>> group ``` -------------------------------- ### Enable execnet Debug Tracing Source: https://execnet.readthedocs.io/en/latest/basics.html Set the EXECNET_DEBUG environment variable to control the level of tracing. Level 1 writes per-process trace files, while level 2 sends tracing to stderr. ```bash EXECNET_DEBUG=1: ``` ```bash EXECNET_DEBUG=2: ``` -------------------------------- ### execnet Channel Methods Source: https://execnet.readthedocs.io/en/latest/genindex.html Documentation for methods related to channel operations within execnet, used for communication between gateways. ```APIDOC ## Channel Operations ### Description Methods for managing communication channels between Python interpreters. ### Methods - `close()` - `makefile()` - `receive()` - `send()` - `setcallback()` - `waitclose()` ### Endpoint N/A (Methods on a Channel object) ### Parameters - **send(data)**: `data` (any) - The data to send over the channel. - **receive()**: No parameters. - **setcallback(callback)**: `callback` (function) - The function to call when a message is received. - Other methods may have specific parameters not detailed here. ### Request Example ```python # Assuming 'channel' is an active execnet channel object channel.send({'message': 'hello'}) response = channel.receive() ``` ### Response - **receive**: Returns the received message. - **send**: Typically returns None or status information. - **close**: Closes the channel. - **waitclose**: Blocks until the channel is closed. ``` -------------------------------- ### Use a callback for channel communication Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Set a callback to react to incoming data. The callback executes in the receiver thread and should not block. ```python >>> import execnet >>> gw = execnet.makegateway() >>> channel = gw.remote_exec("for i in range(10): channel.send(i)") >>> l = [] >>> channel.setcallback(l.append, endmarker=None) >>> channel.waitclose() # waits for closing, i.e. remote exec finish >>> l [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, None] ``` -------------------------------- ### execnet RSync Source: https://execnet.readthedocs.io/en/latest/genindex.html Information about the RSync class and its associated methods for synchronizing files. ```APIDOC ## RSync Class ### Description Provides functionality for synchronizing files and directories. ### Methods - `add_target()` - `send()` ### Endpoint N/A (Methods on an RSync object) ### Parameters - **add_target(target)**: `target` (str or object) - Specifies a target for synchronization. - **send()**: No parameters documented. ### Request Example ```python # Assuming 'rsync' is an RSync object rsync.add_target('/path/to/remote/dir') rsync.send() ``` ### Response - **send**: Initiates the file synchronization process. ``` -------------------------------- ### execnet Group Methods Source: https://execnet.readthedocs.io/en/latest/genindex.html Details on methods for managing groups of gateways. ```APIDOC ## Group Management ### Description Methods for managing a group of execnet gateways. ### Method - `terminate()` ### Endpoint N/A (Method on a Group object) ### Parameters - **terminate()**: No parameters. ### Request Example ```python # Assuming 'group' is an execnet group object group.terminate() ``` ### Response - **terminate**: Terminates all gateways within the group. ``` -------------------------------- ### Serialize data across Python versions Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html Use execnet.dumps and execnet.loads to safely transfer builtin data structures between Python 2 and Python 3. ```python # using python2 import execnet with open("data.py23", "wb") as f: f.write(execnet.dumps(["hello", "world"])) # using Python3 import execnet with open("data.py23", "rb") as f: val = execnet.loads(f.read(), py2str_as_py3str=True) assert val == ["hello", "world"] ``` -------------------------------- ### execnet Core Functions Source: https://execnet.readthedocs.io/en/latest/genindex.html This section details the primary functions available in the execnet module for establishing distributed Python environments. ```APIDOC ## makegateway() ### Description Creates a new gateway to a remote Python interpreter. ### Method `makegateway()` ### Endpoint N/A (Function call) ### Parameters None explicitly documented in the provided text. ### Request Example ```python import execnet gateway = execnet.makegateway() ``` ### Response - **gateway** (object) - A gateway object representing the connection to the remote interpreter. ``` ```APIDOC ## dumps() and loads() ### Description Functions for serializing and deserializing Python objects. ### Method `dumps(obj)` and `loads(string)` ### Endpoint N/A (Function calls) ### Parameters - **obj** (any) - The Python object to serialize. - **string** (str) - The serialized string to deserialize. ### Request Example ```python import execnet data = {'key': 'value'} serialized_data = execnet.dumps(data) deserialized_data = execnet.loads(serialized_data) ``` ### Response - **dumps**: Returns a string representation of the object. - **loads**: Returns the deserialized Python object. ``` -------------------------------- ### Gateway.remote_exec Source: https://execnet.readthedocs.io/en/latest/basics.html Executes source code in the instantiated subprocess-interpreter. ```APIDOC ## Gateway.remote_exec(source) ### Description Return a channel object and connect it to a remote execution thread where the given source executes. ### Parameters #### Request Body - **source** (string/function/module) - Required - The source code to execute remotely. If a string, it is executed with a 'channel' in the global namespace. If a function, it is serialized and called with 'channel' as a keyword argument. ### Response - **channel** (object) - Returns a channel object whose symmetric counterpart is available to the remotely executing source. ``` -------------------------------- ### Work with Java objects from CPython Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html Connect to a Jython interpreter to interact with Java classes like Vector. ```python import execnet gw = execnet.makegateway("popen//python=jython") channel = gw.remote_exec(""" from java.util import Vector v = Vector() v.add('aaa') v.add('bbb') for val in v: channel.send(val) """) for item in channel: print (item) ``` ```text aaa bbb ``` -------------------------------- ### Use Default Group for Gateways Source: https://execnet.readthedocs.io/en/latest/example/test_group.html Gateways created with execnet.makegateway() are automatically added to the execnet.default_group. The default specification for empty calls is 'popen'. ```python import execnet gw = execnet.makegateway() print(gw in execnet.default_group) print(execnet.default_group.defaultspec) ``` -------------------------------- ### execnet Exception Types Source: https://execnet.readthedocs.io/en/latest/genindex.html Lists the custom exception types provided by the execnet library. ```APIDOC ## Exception Types ### Description Custom exceptions raised by execnet for error handling. ### Exceptions - `RemoteError` (Attribute of `execnet.gateway_base.Channel`) - `TimeoutError` (Attribute of `execnet.gateway_base.Channel`) ### Usage These exceptions can be caught using standard Python `try...except` blocks to handle specific error conditions during distributed operations. ``` -------------------------------- ### Gateway.reconfigure Source: https://execnet.readthedocs.io/en/latest/basics.html Reconfigures the string-coercion behavior of the gateway. ```APIDOC ## Gateway.reconfigure(py2str_as_py3str=True, py3str_as_py2str=False) ### Description Updates how the gateway handles string coercion between Python versions. ### Parameters #### Request Body - **py2str_as_py3str** (boolean) - Optional - Whether to treat Python 2 strings as Python 3 strings. - **py3str_as_py2str** (boolean) - Optional - Whether to treat Python 3 strings as Python 2 strings. ``` -------------------------------- ### Configure Default Gateway Specification for a Group Source: https://execnet.readthedocs.io/en/latest/example/test_group.html Set group.defaultspec to define the default gateway specification used by group.makegateway(). This allows creating gateways without explicitly specifying connection details. ```python import execnet group = execnet.Group() group.defaultspec = "ssh=localhost//chdir=mytmp//nice=20" gw = group.makegateway() ch = gw.remote_exec(""" import os.path basename = os.path.basename(os.getcwd()) channel.send(basename) """) print(ch.receive()) ``` -------------------------------- ### Reconfigure string coercion between Python 2 and 3 Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html Adjust how strings are coerced between Python 2 and 3 using gw.reconfigure or channel.reconfigure. ```python >>> import execnet >>> execnet.makegateway("popen//python=python3.2") >>> gw=execnet.makegateway("popen//python=python3.2") >>> gw.remote_exec("channel.send('hello')").receive() u'hello' >>> gw.reconfigure(py3str_as_py2str=True) >>> gw.remote_exec("channel.send('hello')").receive() 'hello' >>> ch = gw.remote_exec('channel.send(type(channel.receive()).__name__)') >>> ch.send('a') >>> ch.receive() 'str' >>> ch = gw.remote_exec('channel.send(type(channel.receive()).__name__)') >>> ch.reconfigure(py2str_as_py3str=False) >>> ch.send('a') >>> ch.receive() u'bytes' ``` -------------------------------- ### Robustly Terminate Gateways with Timeout Source: https://execnet.readthedocs.io/en/latest/example/test_group.html Ensure all member gateways are terminated and local subprocesses are cleaned up using group.terminate(timeout). A timeout can be specified to enforce process killing. ```python import execnet group = execnet.Group() gw = group.makegateway("popen//id=sleeper") ch = gw.remote_exec("import time ; time.sleep(2.0)") print(group) group.terminate(timeout=1.0) print(group) ``` -------------------------------- ### Python Server to Serve File Contents Source: https://execnet.readthedocs.io/en/latest/example/test_ssh_fileserver.html This Python code defines a server function that reads and sends the contents of specified files over a channel. It's designed to be executed remotely. ```python # content of servefiles.py def servefiles(channel): for fn in channel: f = open(fn, "rb") channel.send(f.read()) f.close() if __name__ == "__channelexec__": servefiles(channel) # type: ignore[name-defined] ``` -------------------------------- ### Define Gateway Specification Format Source: https://execnet.readthedocs.io/en/latest/basics.html The specification string uses a key-value pair format separated by double slashes. ```text key1=value1//key2=value2//... ``` -------------------------------- ### Define remote file server Source: https://execnet.readthedocs.io/en/latest/example/test_info.html A script to serve file contents over a channel. ```python # content of servefiles.py def servefiles(channel): for fn in channel: f = open(fn, "rb") channel.send(f.read()) f.close() if __name__ == "__channelexec__": servefiles(channel) # type: ignore[name-defined] ``` -------------------------------- ### Gateway and Group Management Source: https://execnet.readthedocs.io/en/latest/basics.html Methods for managing gateway groups and obtaining remote execution status. ```APIDOC ## Group.terminate(timeout=None) ### Description Triggers exit of member gateways and waits for termination of associated subprocesses. ### Parameters - **timeout** (float) - Optional - Seconds to wait before attempting to kill local subprocesses. ## Gateway.remote_status() ### Description Obtains low-level execution information from the remote side, such as queued tasks and active channels. ``` -------------------------------- ### RSync Class API Source: https://execnet.readthedocs.io/en/latest/basics.html The RSync class allows for recursive directory synchronization to remote filesystems via gateways. ```APIDOC ## RSync Class ### Description Allows sending a directory structure recursively to one or multiple remote filesystems. ### Constructor `execnet.RSync(sourcedir, callback=None, verbose=True)` - **sourcedir** (str) - The source directory path. - **callback** (Callable) - Optional callback function. - **verbose** (bool) - Enable verbose logging. ### Methods #### add_target `add_target(gateway, destdir, finishedcallback=None, **options)` - **gateway** (Gateway) - The remote gateway. - **destdir** (str|PathLike) - Remote destination directory. - **finishedcallback** (Callable) - Optional callback when sync finishes. #### send `send(raises=True)` - **raises** (bool) - Whether to raise an error if no targets are defined. ``` -------------------------------- ### Remote-exec a function Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Execute parametrized pure functions remotely. Note that interactively defined functions cannot be used due to inspect.getsource limitations. ```python import execnet def multiplier(channel, factor): while not channel.isclosed(): param = channel.receive() channel.send(param * factor) gw = execnet.makegateway() channel = gw.remote_exec(multiplier, factor=10) for i in range(5): channel.send(i) result = channel.receive() assert result == i * 10 gw.exit() ``` -------------------------------- ### Assign Custom Gateway IDs Source: https://execnet.readthedocs.io/en/latest/example/test_group.html Assign a specific ID to a gateway when creating it using group.makegateway by appending '//id=MYNAME' to the gateway specification. ```python import execnet group = execnet.Group() gw = group.makegateway("popen//id=sub1") assert gw.id == "sub1" print(group['sub1']) ``` -------------------------------- ### Channel Operations Source: https://execnet.readthedocs.io/en/latest/basics.html Methods for sending, receiving, and managing data channels between remote programs. ```APIDOC ## Channel.send(item) ### Description Sends an item to the other side of the channel. The item must be a simple Python type and is copied by value. ### Parameters - **item** (any) - Required - The data item to send. ## Channel.receive(timeout=None) ### Description Receives a data item from the other side. Blocks until an item is received or the timeout expires. ### Parameters - **timeout** (float) - Optional - Seconds to wait before raising channel.TimeoutError. ## Channel.setcallback(callback, endmarker=NOENDMARKER) ### Description Sets a callback function for receiving items. Once set, calls to receive() will raise an error. ## Channel.makefile(mode='w', proxyclose=False) ### Description Returns a file-like object for the channel. ### Parameters - **mode** (string) - Optional - 'r' for readable or 'w' for writeable. - **proxyclose** (bool) - Optional - If true, file.close() also closes the channel. ## Channel.close(error=None) ### Description Closes the channel with an optional error message. ## Channel.waitclose(timeout=None) ### Description Waits until the channel is closed by the remote side. ``` -------------------------------- ### Sending channels over channels Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Transfer newly created channels over an existing channel to facilitate communication. ```python >>> import execnet >>> gw = execnet.makegateway() >>> channel = gw.remote_exec(""" ... ch1, ch2 = channel.receive() ... ch2.send("world") ... ch1.send("hello") ... """) >>> c1 = gw.newchannel() # create new channel >>> c2 = gw.newchannel() # create another channel >>> channel.send((c1, c2)) # send them over >>> c1.receive() 'hello' >>> c2.receive() 'world' ``` -------------------------------- ### Cross-interpreter Serialization Source: https://execnet.readthedocs.io/en/latest/basics.html Functions for serializing and deserializing Python objects across different interpreter versions. ```APIDOC ## Serialization Functions ### execnet.dumps(spec) - **Description**: Serialize a builtin Python object to a bytestring. - **Parameters**: `spec` (object) - The object to serialize. ### execnet.loads(spec, py2str_as_py3str=False, py3str_as_py2str=False) - **Description**: Deserialize a bytestring to a Python object. - **Parameters**: - **spec** (bytes) - The bytestring to deserialize. - **py2str_as_py3str** (bool) - Convert Python2 strings to Python3 text objects. - **py3str_as_py2str** (bool) - Convert Python3 strings to Python2 strings. ``` -------------------------------- ### Define remote command module Source: https://execnet.readthedocs.io/en/latest/example/test_info.html A module designed to be executed remotely, evaluating incoming channel items. ```python def simple(arg): return arg + 1 def listdir(path): return os.listdir(path) if __name__ == "__channelexec__": for item in channel: # type: ignore[name-defined] channel.send(eval(item)) # type: ignore[name-defined] ``` -------------------------------- ### Retrieve remote file contents Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Client code to request and receive file contents from a remote SSH host. ```python import execnet import servefiles gw = execnet.makegateway("ssh=codespeak.net") channel = gw.remote_exec(servefiles) for fn in ('/etc/passwd', '/etc/group'): channel.send(fn) content = channel.receive() print(fn) print(content) ``` -------------------------------- ### Cross-Interpreter Object Serialization Source: https://execnet.readthedocs.io/en/latest/basics.html Serialize Python objects to a bytestring using `dumps` and deserialize them back using `loads`. This is useful for transferring data between different Python interpreters. Ensure objects are of builtin Python types. ```python import execnet dump = execnet.dumps([1,2,3]) execnet.loads(dump) ``` -------------------------------- ### Remote-exec a module Source: https://execnet.readthedocs.io/en/latest/example/test_info.html Pass a module object to remote_exec to transfer its source code. The module can detect remote execution via the __channelexec__ attribute. ```python # content of a module remote1.py if __name__ == "__channelexec__": channel.send("initialization complete") # type: ignore[name-defined] ``` ```python >>> import execnet, remote1 >>> gw = execnet.makegateway() >>> ch = gw.remote_exec(remote1) >>> print (ch.receive()) initialization complete ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.