execnet

https://execnet.readthedocs.io/en/latest/
Admin
execnet is a Python library providing a share-nothing model with channel-based communication for...
Tokens:10,766
Snippets:143
Trust Score:9.6
Update:6 months ago
Show doc for...
Tokens:
Raw
### Simple command loop pattern

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Setup a serving loop for synchronous function calls.

```python
import os
```

--------------------------------

### Install execnet via pip

Source: https://execnet.readthedocs.io/en/latest/install.html

Use this command to install the execnet package from PyPI. Ensure you have pip installed.

```bash
pip install execnet

```

--------------------------------

### Start remote socket server

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Command to start the standalone socketserver script for remote bootstrapping.

```bash
python socketserver.py :8888   # bind to all IPs, port 8888
```

--------------------------------

### Example execnet trace output

Source: https://execnet.readthedocs.io/en/latest/example/test_debug.html

Sample output showing PID-prefixed trace entries generated when debugging is enabled.

```text
[2326] gw0 starting to receive
[2326] gw0 sent <Message.CHANNEL_EXEC channelid=1 '42'>
[2327] creating workergateway on <execnet.gateway_base.Popen2IO instance at 0x9f1c20c>
[2327] gw0-worker starting to receive
[2327] gw0-worker received <Message.CHANNEL_EXEC channelid=1 '42'>
[2327] gw0-worker execution starts[1]: '42'
[2327] gw0-worker execution finished
[2327] gw0-worker sent <Message.CHANNEL_CLOSE channelid=1 ''>
[2327] gw0-worker 1 sent channel close message
[2326] gw0 received <Message.CHANNEL_CLOSE channelid=1 ''>
[2326] gw0 1 channel.__del__
[2326] === atexit cleanup <Group ['gw0']> ===
[2326] gw0 gateway.exit() called
[2326] gw0 --> sending GATEWAY_TERMINATE
[2326] gw0 sent <Message.GATEWAY_TERMINATE channelid=0 ''>
[2326] gw0 joining receiver thread
[2327] gw0-worker received <Message.GATEWAY_TERMINATE channelid=0 ''>
[2327] gw0-worker putting None to execqueue
[2327] gw0-worker io.close_read()
[2327] gw0-worker leaving <Thread(receiver, started daemon -1220277392)>
[2327] gw0-worker 1 channel.__del__
[2327] gw0-worker io.close_write()
[2327] gw0-worker workergateway.serve finished
[2327] gw0-worker gateway.join() called while receiverthread already finished
[2326] gw0 leaving <Thread(receiver, started daemon -1221223568)>
```

--------------------------------

### Install execnet via pip

Source: https://execnet.readthedocs.io/en/latest/index.html

Use this command to install the latest version of execnet from the Python Package Index.

```bash
pip install -U execnet
```

--------------------------------

### Configure and Test Threading Models

Source: https://execnet.readthedocs.io/en/latest/basics.html

Sets the execution model for local and remote sides before creating a gateway. Requires eventlet to be installed in the environment.

```python
# content of threadmodel.py
import execnet
# locally use "eventlet", remotely use "thread" model
execnet.set_execmodel("eventlet", "thread")
gw = execnet.makegateway()
print (gw)
print (gw.remote_status())
print (gw.remote_exec("channel.send(1)").receive())
```

```bash
$ python threadmodel.py
<Gateway id='gw0' receive-live, eventlet model, 0 active channels>
<RInfo 'numchannels=0, numexecuting=0, execmodel=thread'>
1
```

--------------------------------

### Saturate multiple hosts/CPUs with tasks

Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html

A comprehensive example of distributing tasks across multiple gateways (simulating CPUs/hosts) and managing their execution and termination. This pattern is suitable for high-throughput parallel processing.

```python
from __future__ import annotations

import execnet

group = execnet.Group()
for i in range(4):  # 4 CPUs
    group.makegateway()


def process_item(channel):
    # task processor, sits on each CPU
    import time
    import random

    channel.send("ready")
    for x in channel:
        if x is None:  # we can shutdown
            break
        # sleep random time, send result
        time.sleep(random.randrange(3))
        channel.send(x * 10)


# execute taskprocessor everywhere
mch = group.remote_exec(process_item)

# get a queue that gives us results
q = mch.make_receive_queue(endmarker=-1)
tasks: list[int] | None = list(range(10))  # a list of tasks, here just integers
terminated = 0
while 1:
    channel, item = q.get()
    if item == -1:
        terminated += 1
        print("terminated %s" % channel.gateway.id)
        if terminated == len(mch):
            print("got all results, terminating")
            break
        continue
    if item != "ready":
        print(f"other side {channel.gateway.id} returned {item!r}")
    if not tasks and tasks is not None:
        print("no tasks remain, sending termination request to all")
        mch.send_each(None)
        tasks = None
    if tasks:
        task = tasks.pop()
        channel.send(task)
        print(f"sent task {task!r} to {channel.gateway.id}")

group.terminate()
```

--------------------------------

### Connect to Python 2 and Numpy from Python 3

Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html

Demonstrates remote execution on a Python 2.7 interpreter with Numpy installed, sending data to be processed and receiving the result.

```python
import execnet
gw = execnet.makegateway("popen//python=python2.7")
channel = gw.remote_exec("""
    import numpy
    array = numpy.array([1,2,3])
    while 1:
        x = channel.receive()
        if x is None:
            break
        array = numpy.append(array, x)
    channel.send(repr(array))
""")
for x in range(10):
    channel.send(x)
channel.send(None)
print (channel.receive())
```

```text
array([1, 2, 3, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
```

--------------------------------

### Get Gateway IDs Before Instantiation

Source: https://execnet.readthedocs.io/en/latest/example/test_group.html

Determine a gateway's ID before it's created by using execnet.XSpec and group.allocate_id. This is useful for pre-planning gateway identification.

```python
import execnet
group = execnet.Group()
spec = execnet.XSpec("popen")
group.allocate_id(spec)
allocated_id = spec.id
gw = group.makegateway(spec)
assert gw.id == allocated_id
```

--------------------------------

### Get information from remote SSH account

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Obtain environment information from a remote SSH host.

```python
>>> import execnet, os
>>> gw = execnet.makegateway("ssh=codespeak.net")
>>> channel = gw.remote_exec("""
...     import sys, os
...     channel.send((sys.platform, tuple(sys.version_info), os.getpid()))
... """)
>>> platform, version_info, remote_pid = channel.receive()
>>> platform
'linux2'
>>> version_info
(2, 6, 6, 'final', 0)
```

--------------------------------

### Create a proxied gateway using execnet

Source: https://execnet.readthedocs.io/en/latest/example/test_proxy.html

Demonstrates initializing a master gateway and using it to control worker gateways via the via=master specification.

```python
>>> import execnet
>>> group = execnet.Group()
>>> group.defaultspec = 'popen//via=master'
>>> master = group.makegateway('popen//id=master')
>>> master
<Gateway id='master' receive-live, thread model, 0 active channels>
>>> worker = group.makegateway()
>>> worker
<Gateway id='gw0' receive-live, thread model, 0 active channels>
>>> group
<Group ['master', 'gw0']>
```

--------------------------------

### Receive results with a Queue from MultiChannel

Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html

Shows how to create a receive queue from a MultiChannel to process results sequentially. Useful for ordered retrieval of data from multiple channels.

```python
>>> ch1 = execnet.makegateway().remote_exec("channel.send(1)")
>>> ch2 = execnet.makegateway().remote_exec("channel.send(2)")
>>> mch = execnet.MultiChannel([ch1, ch2])
>>> queue = mch.make_receive_queue()
>>> chan1, res1 = queue.get()
>>> chan2, res2 = queue.get(timeout=3)
>>> res1 + res2
3
```

--------------------------------

### Create and use MultiChannel

Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html

Demonstrates creating a MultiChannel from individual channels and performing operations like checking length and membership. Use when managing multiple concurrent communication streams.

```python
import execnet
>>> ch1 = execnet.makegateway().remote_exec("channel.send(1)")
>>> ch2 = execnet.makegateway().remote_exec("channel.send(2)")
>>> mch = execnet.MultiChannel([ch1, ch2])
>>> len(mch)
2
>>> mch[0] is ch1 and mch[1] is ch2
True
>>> ch1 in mch and ch2 in mch
True
>>> sum(mch.receive_each())
3
```

--------------------------------

### Instantiate a Default Gateway

Source: https://execnet.readthedocs.io/en/latest/basics.html

Creates a simple Python subprocess gateway using default settings.

```python
>>> gateway = execnet.makegateway()
```

--------------------------------

### Asynchronous channel communication with callbacks

Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html

Illustrates setting up a callback function to process incoming data asynchronously without blocking execution. Ideal for event-driven processing of channel messages.

```python
>>> import execnet
>>> gw = execnet.makegateway()
>>> ch = gw.remote_exec("channel.receive() ; channel.send(42)")
>>> l = []
>>> ch.setcallback(l.append)
>>> ch.send(1)
>>> ch.waitclose()
>>> assert l == [42]
```

--------------------------------

### Execute remote commands locally

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Demonstrates how to trigger functions defined in a remote module via a gateway channel.

```python
>>> import execnet, remotecmd
>>> gw = execnet.makegateway()
>>> ch = gw.remote_exec(remotecmd)
>>> ch.send('simple(10)') # execute func-call remotely
>>> ch.receive()
11
```

--------------------------------

### Configure Gateway Options

Source: https://execnet.readthedocs.io/en/latest/basics.html

Available configuration keys for gateway specifications.

```text
id=<string>     specifies the gateway id
python=<path>   specifies which python interpreter to execute
execmodel=model 'thread', 'main_thread_only', 'eventlet', 'gevent' execution model
chdir=<path>    specifies to which directory to change
nice=<path>     specifies process priority of new process
env:NAME=value  specifies a remote environment variable setting.
```

--------------------------------

### Debugging Configuration

Source: https://execnet.readthedocs.io/en/latest/basics.html

Environment variable configuration for tracing execnet execution.

```APIDOC
## Debugging

### Environment Variable: EXECNET_DEBUG
- **EXECNET_DEBUG=1**: Write per-process trace-files to `execnet-debug-PID`.
- **EXECNET_DEBUG=2**: Perform tracing to stderr.
```

--------------------------------

### Basic RSync Usage

Source: https://execnet.readthedocs.io/en/latest/basics.html

Use this to synchronize a local directory to a remote destination via an execnet gateway. Ensure the source directory exists locally and the destination directory is accessible on the remote gateway.

```python
rsync = execnet.RSync('/tmp/source')
gw = execnet.makegateway()
rsync.add_target(gw, '/tmp/dest')
rsync.send()
```

--------------------------------

### Execute source code in subprocess

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Instantiate a subprocess gateway to execute code and communicate bidirectionally.

```python
>>> import execnet
>>> gw = execnet.makegateway()
>>> channel = gw.remote_exec("channel.send(channel.receive()+1)")
>>> channel.send(1)
>>> channel.receive()
2
```

--------------------------------

### execnet.makegateway

Source: https://execnet.readthedocs.io/en/latest/basics.html

Creates and configures a gateway to a Python interpreter based on a specification string.

```APIDOC
## execnet.makegateway(spec)

### Description
Create and configure a gateway to a Python interpreter. The spec string encodes the target gateway type and configuration information.

### Parameters
#### Request Body
- **spec** (string) - Required - The specification string (e.g., 'popen', 'ssh=hostname', 'socket=host:port'). Supports configurations like id, python, execmodel, chdir, nice, and env:NAME.

### Request Example
```
>>> gateway = execnet.makegateway('popen//python=python3.3')
```
```

--------------------------------

### Compare current working directories

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Verify the working directory of a local subprocess gateway.

```python
>>> import execnet, os
>>> gw = execnet.makegateway()
>>> ch = gw.remote_exec("import os; channel.send(os.getcwd())")
>>> res = ch.receive()
>>> assert res == os.getcwd()
```

--------------------------------

### Manage Gateways with execnet.Group

Source: https://execnet.readthedocs.io/en/latest/example/test_group.html

Use execnet.Group to manage a collection of gateways. It handles membership, lifetime, and provides convenient access to individual gateways.

```python
import execnet
group = execnet.Group(['popen'] * 2)
print(len(group))
print(group)
print(list(group))
print('gw0' in group and 'gw1' in group)
print(group['gw0'] == group[0])
print(group['gw1'] == group[1])
group.terminate() # exit all member gateways
print(group)
```

--------------------------------

### Connect to remote socket gateway

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Instantiate a gateway by connecting to a remote socket endpoint.

```python
import execnet
gw = execnet.makegateway("socket=TARGET-IP:8888")
```

--------------------------------

### Python Client to Retrieve Remote File Contents

Source: https://execnet.readthedocs.io/en/latest/example/test_ssh_fileserver.html

This Python code demonstrates how to connect to a remote server using execnet, execute the `servefiles` function, and retrieve file contents. It requires the `servefiles.py` script to be available on the remote host.

```python
import execnet
import servefiles
gw = execnet.makegateway("ssh=codespeak.net")
channel = gw.remote_exec(servefiles)

for fn in ('/etc/passwd', '/etc/group'):
    channel.send(fn)
    content = channel.receive()
    print(fn)
    print(content)

```

--------------------------------

### execnet Gateway Methods

Source: https://execnet.readthedocs.io/en/latest/genindex.html

Details on methods available for interacting with and managing execnet gateways.

```APIDOC
## Gateway Management

### Description
Methods for controlling and querying the state of an execnet gateway.

### Methods
- `reconfigure()`
- `remote_exec()`
- `remote_status()`

### Endpoint
N/A (Methods on a Gateway object)

### Parameters
- **remote_exec(source, **kwargs)**: `source` (str) - Python code to execute remotely.
- **remote_status()**: No parameters.

### Request Example
```python
# Assuming 'gateway' is an active execnet gateway object
result = gateway.remote_exec("print('Hello from remote!')")
status = gateway.remote_status()
```

### Response
- **remote_exec**: Returns an object representing the remote execution context.
- **remote_status**: Returns information about the gateway's status.
```

--------------------------------

### Enable execnet debugging

Source: https://execnet.readthedocs.io/en/latest/example/test_debug.html

Set the EXECNET_DEBUG environment variable to 2 to output trace information to stderr.

```bash
EXECNET_DEBUG=2  # or "set EXECNET_DEBUG=2" on windows

python -c 'import execnet ; execnet.makegateway().remote_exec("42")'
```

--------------------------------

### Work with C# objects from CPython

Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html

Connect to an IronPython interpreter to instantiate and manipulate C# classes via the CLR.

```python
import execnet
gw = execnet.makegateway("popen//python=ipy")

channel = gw.remote_exec("""
    import clr
    clr.AddReference("System")
    from System import Array
    array = Array[float]([1,2])
    channel.send(str(array))
""")
print (channel.receive())
```

```text
System.Double[](1.0, 2.0)
```

--------------------------------

### Robustly receive results and termination notification

Source: https://execnet.readthedocs.io/en/latest/example/test_multi.html

Demonstrates using an endmarker with `make_receive_queue` to identify channel closures or gateway terminations. Essential for reliable cleanup and status monitoring in distributed tasks.

```python
>>> group = execnet.Group(['popen'] * 3) # create three gateways
>>> mch = group.remote_exec("channel.send(channel.receive()+1)")
>>> queue = mch.make_receive_queue(endmarker=42)
>>> mch[0].send(1)
>>> chan1, res1 = queue.get()
>>> res1
2
>>> group.terminate(timeout=1) # kill processes waiting on receive
>>> for i in range(3):
...    chan1, res1 = queue.get()
...    assert res1 == 42
>>> group
<Group []>
```

--------------------------------

### Enable execnet Debug Tracing

Source: https://execnet.readthedocs.io/en/latest/basics.html

Set the EXECNET_DEBUG environment variable to control the level of tracing. Level 1 writes per-process trace files, while level 2 sends tracing to stderr.

```bash
EXECNET_DEBUG=1:
```

```bash
EXECNET_DEBUG=2:
```

--------------------------------

### execnet Channel Methods

Source: https://execnet.readthedocs.io/en/latest/genindex.html

Documentation for methods related to channel operations within execnet, used for communication between gateways.

```APIDOC
## Channel Operations

### Description
Methods for managing communication channels between Python interpreters.

### Methods
- `close()`
- `makefile()`
- `receive()`
- `send()`
- `setcallback()`
- `waitclose()`

### Endpoint
N/A (Methods on a Channel object)

### Parameters
- **send(data)**: `data` (any) - The data to send over the channel.
- **receive()**: No parameters.
- **setcallback(callback)**: `callback` (function) - The function to call when a message is received.
- Other methods may have specific parameters not detailed here.

### Request Example
```python
# Assuming 'channel' is an active execnet channel object
channel.send({'message': 'hello'})
response = channel.receive()
```

### Response
- **receive**: Returns the received message.
- **send**: Typically returns None or status information.
- **close**: Closes the channel.
- **waitclose**: Blocks until the channel is closed.
```

--------------------------------

### Use a callback for channel communication

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Set a callback to react to incoming data. The callback executes in the receiver thread and should not block.

```python
>>> import execnet
>>> gw = execnet.makegateway()
>>> channel = gw.remote_exec("for i in range(10): channel.send(i)")
>>> l = []
>>> channel.setcallback(l.append, endmarker=None)
>>> channel.waitclose() # waits for closing, i.e. remote exec finish
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, None]
```

--------------------------------

### execnet RSync

Source: https://execnet.readthedocs.io/en/latest/genindex.html

Information about the RSync class and its associated methods for synchronizing files.

```APIDOC
## RSync Class

### Description
Provides functionality for synchronizing files and directories.

### Methods
- `add_target()`
- `send()`

### Endpoint
N/A (Methods on an RSync object)

### Parameters
- **add_target(target)**: `target` (str or object) - Specifies a target for synchronization.
- **send()**: No parameters documented.

### Request Example
```python
# Assuming 'rsync' is an RSync object
rsync.add_target('/path/to/remote/dir')
rsync.send()
```

### Response
- **send**: Initiates the file synchronization process.
```

--------------------------------

### execnet Group Methods

Source: https://execnet.readthedocs.io/en/latest/genindex.html

Details on methods for managing groups of gateways.

```APIDOC
## Group Management

### Description
Methods for managing a group of execnet gateways.

### Method
- `terminate()`

### Endpoint
N/A (Method on a Group object)

### Parameters
- **terminate()**: No parameters.

### Request Example
```python
# Assuming 'group' is an execnet group object
group.terminate()
```

### Response
- **terminate**: Terminates all gateways within the group.
```

--------------------------------

### Serialize data across Python versions

Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html

Use execnet.dumps and execnet.loads to safely transfer builtin data structures between Python 2 and Python 3.

```python
# using python2
import execnet
with open("data.py23", "wb") as f:
    f.write(execnet.dumps(["hello", "world"]))

# using Python3
import execnet
with open("data.py23", "rb") as f:
    val = execnet.loads(f.read(), py2str_as_py3str=True)
assert val == ["hello", "world"]
```

--------------------------------

### execnet Core Functions

Source: https://execnet.readthedocs.io/en/latest/genindex.html

This section details the primary functions available in the execnet module for establishing distributed Python environments.

```APIDOC
## makegateway()

### Description
Creates a new gateway to a remote Python interpreter.

### Method
`makegateway()`

### Endpoint
N/A (Function call)

### Parameters
None explicitly documented in the provided text.

### Request Example
```python
import execnet
gateway = execnet.makegateway()
```

### Response
- **gateway** (object) - A gateway object representing the connection to the remote interpreter.
```

```APIDOC
## dumps() and loads()

### Description
Functions for serializing and deserializing Python objects.

### Method
`dumps(obj)` and `loads(string)`

### Endpoint
N/A (Function calls)

### Parameters
- **obj** (any) - The Python object to serialize.
- **string** (str) - The serialized string to deserialize.

### Request Example
```python
import execnet
data = {'key': 'value'}
serialized_data = execnet.dumps(data)
deserialized_data = execnet.loads(serialized_data)
```

### Response
- **dumps**: Returns a string representation of the object.
- **loads**: Returns the deserialized Python object.
```

--------------------------------

### Gateway.remote_exec

Source: https://execnet.readthedocs.io/en/latest/basics.html

Executes source code in the instantiated subprocess-interpreter.

```APIDOC
## Gateway.remote_exec(source)

### Description
Return a channel object and connect it to a remote execution thread where the given source executes.

### Parameters
#### Request Body
- **source** (string/function/module) - Required - The source code to execute remotely. If a string, it is executed with a 'channel' in the global namespace. If a function, it is serialized and called with 'channel' as a keyword argument.

### Response
- **channel** (object) - Returns a channel object whose symmetric counterpart is available to the remotely executing source.
```

--------------------------------

### Work with Java objects from CPython

Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html

Connect to a Jython interpreter to interact with Java classes like Vector.

```python
import execnet
gw = execnet.makegateway("popen//python=jython")
channel = gw.remote_exec("""
    from java.util import Vector
    v = Vector()
    v.add('aaa')
    v.add('bbb')
    for val in v:
        channel.send(val)
""")

for item in channel:
    print (item)
```

```text
aaa
bbb
```

--------------------------------

### Use Default Group for Gateways

Source: https://execnet.readthedocs.io/en/latest/example/test_group.html

Gateways created with execnet.makegateway() are automatically added to the execnet.default_group. The default specification for empty calls is 'popen'.

```python
import execnet
gw = execnet.makegateway()
print(gw in execnet.default_group)
print(execnet.default_group.defaultspec)
```

--------------------------------

### execnet Exception Types

Source: https://execnet.readthedocs.io/en/latest/genindex.html

Lists the custom exception types provided by the execnet library.

```APIDOC
## Exception Types

### Description
Custom exceptions raised by execnet for error handling.

### Exceptions
- `RemoteError` (Attribute of `execnet.gateway_base.Channel`)
- `TimeoutError` (Attribute of `execnet.gateway_base.Channel`)

### Usage
These exceptions can be caught using standard Python `try...except` blocks to handle specific error conditions during distributed operations.
```

--------------------------------

### Gateway.reconfigure

Source: https://execnet.readthedocs.io/en/latest/basics.html

Reconfigures the string-coercion behavior of the gateway.

```APIDOC
## Gateway.reconfigure(py2str_as_py3str=True, py3str_as_py2str=False)

### Description
Updates how the gateway handles string coercion between Python versions.

### Parameters
#### Request Body
- **py2str_as_py3str** (boolean) - Optional - Whether to treat Python 2 strings as Python 3 strings.
- **py3str_as_py2str** (boolean) - Optional - Whether to treat Python 3 strings as Python 2 strings.
```

--------------------------------

### Configure Default Gateway Specification for a Group

Source: https://execnet.readthedocs.io/en/latest/example/test_group.html

Set group.defaultspec to define the default gateway specification used by group.makegateway(). This allows creating gateways without explicitly specifying connection details.

```python
import execnet
group = execnet.Group()
group.defaultspec = "ssh=localhost//chdir=mytmp//nice=20"
gw = group.makegateway()
ch = gw.remote_exec("""
     import os.path
     basename = os.path.basename(os.getcwd())
     channel.send(basename)
""")
print(ch.receive())
```

--------------------------------

### Reconfigure string coercion between Python 2 and 3

Source: https://execnet.readthedocs.io/en/latest/example/hybridpython.html

Adjust how strings are coerced between Python 2 and 3 using gw.reconfigure or channel.reconfigure.

```python
>>> import execnet
>>> execnet.makegateway("popen//python=python3.2")
<Gateway id='gw0' receive-live, 0 active channels>
>>> gw=execnet.makegateway("popen//python=python3.2")
>>> gw.remote_exec("channel.send('hello')").receive()
u'hello'
>>> gw.reconfigure(py3str_as_py2str=True)
>>> gw.remote_exec("channel.send('hello')").receive()
'hello'
>>> ch = gw.remote_exec('channel.send(type(channel.receive()).__name__)')
>>> ch.send('a')
>>> ch.receive()
'str'
>>> ch = gw.remote_exec('channel.send(type(channel.receive()).__name__)')
>>> ch.reconfigure(py2str_as_py3str=False)
>>> ch.send('a')
>>> ch.receive()
u'bytes'
```

--------------------------------

### Robustly Terminate Gateways with Timeout

Source: https://execnet.readthedocs.io/en/latest/example/test_group.html

Ensure all member gateways are terminated and local subprocesses are cleaned up using group.terminate(timeout). A timeout can be specified to enforce process killing.

```python
import execnet
group = execnet.Group()
gw = group.makegateway("popen//id=sleeper")
ch = gw.remote_exec("import time ; time.sleep(2.0)")
print(group)
group.terminate(timeout=1.0)
print(group)
```

--------------------------------

### Python Server to Serve File Contents

Source: https://execnet.readthedocs.io/en/latest/example/test_ssh_fileserver.html

This Python code defines a server function that reads and sends the contents of specified files over a channel. It's designed to be executed remotely.

```python
# content of servefiles.py

def servefiles(channel):
    for fn in channel:
        f = open(fn, "rb")
        channel.send(f.read())
        f.close()


if __name__ == "__channelexec__":
    servefiles(channel)  # type: ignore[name-defined]

```

--------------------------------

### Define Gateway Specification Format

Source: https://execnet.readthedocs.io/en/latest/basics.html

The specification string uses a key-value pair format separated by double slashes.

```text
key1=value1//key2=value2//...
```

--------------------------------

### Define remote file server

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

A script to serve file contents over a channel.

```python
# content of servefiles.py


def servefiles(channel):
    for fn in channel:
        f = open(fn, "rb")
        channel.send(f.read())
        f.close()


if __name__ == "__channelexec__":
    servefiles(channel)  # type: ignore[name-defined]
```

--------------------------------

### Gateway and Group Management

Source: https://execnet.readthedocs.io/en/latest/basics.html

Methods for managing gateway groups and obtaining remote execution status.

```APIDOC
## Group.terminate(timeout=None)

### Description
Triggers exit of member gateways and waits for termination of associated subprocesses.

### Parameters
- **timeout** (float) - Optional - Seconds to wait before attempting to kill local subprocesses.

## Gateway.remote_status()

### Description
Obtains low-level execution information from the remote side, such as queued tasks and active channels.
```

--------------------------------

### RSync Class API

Source: https://execnet.readthedocs.io/en/latest/basics.html

The RSync class allows for recursive directory synchronization to remote filesystems via gateways.

```APIDOC
## RSync Class

### Description
Allows sending a directory structure recursively to one or multiple remote filesystems.

### Constructor
`execnet.RSync(sourcedir, callback=None, verbose=True)`
- **sourcedir** (str) - The source directory path.
- **callback** (Callable) - Optional callback function.
- **verbose** (bool) - Enable verbose logging.

### Methods
#### add_target
`add_target(gateway, destdir, finishedcallback=None, **options)`
- **gateway** (Gateway) - The remote gateway.
- **destdir** (str|PathLike) - Remote destination directory.
- **finishedcallback** (Callable) - Optional callback when sync finishes.

#### send
`send(raises=True)`
- **raises** (bool) - Whether to raise an error if no targets are defined.
```

--------------------------------

### Remote-exec a function

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Execute parametrized pure functions remotely. Note that interactively defined functions cannot be used due to inspect.getsource limitations.

```python
import execnet


def multiplier(channel, factor):
    while not channel.isclosed():
        param = channel.receive()
        channel.send(param * factor)


gw = execnet.makegateway()
channel = gw.remote_exec(multiplier, factor=10)

for i in range(5):
    channel.send(i)
    result = channel.receive()
    assert result == i * 10

gw.exit()
```

--------------------------------

### Assign Custom Gateway IDs

Source: https://execnet.readthedocs.io/en/latest/example/test_group.html

Assign a specific ID to a gateway when creating it using group.makegateway by appending '//id=MYNAME' to the gateway specification.

```python
import execnet
group = execnet.Group()
gw = group.makegateway("popen//id=sub1")
assert gw.id == "sub1"
print(group['sub1'])
```

--------------------------------

### Channel Operations

Source: https://execnet.readthedocs.io/en/latest/basics.html

Methods for sending, receiving, and managing data channels between remote programs.

```APIDOC
## Channel.send(item)

### Description
Sends an item to the other side of the channel. The item must be a simple Python type and is copied by value.

### Parameters
- **item** (any) - Required - The data item to send.

## Channel.receive(timeout=None)

### Description
Receives a data item from the other side. Blocks until an item is received or the timeout expires.

### Parameters
- **timeout** (float) - Optional - Seconds to wait before raising channel.TimeoutError.

## Channel.setcallback(callback, endmarker=NOENDMARKER)

### Description
Sets a callback function for receiving items. Once set, calls to receive() will raise an error.

## Channel.makefile(mode='w', proxyclose=False)

### Description
Returns a file-like object for the channel.

### Parameters
- **mode** (string) - Optional - 'r' for readable or 'w' for writeable.
- **proxyclose** (bool) - Optional - If true, file.close() also closes the channel.

## Channel.close(error=None)

### Description
Closes the channel with an optional error message.

## Channel.waitclose(timeout=None)

### Description
Waits until the channel is closed by the remote side.
```

--------------------------------

### Sending channels over channels

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Transfer newly created channels over an existing channel to facilitate communication.

```python
>>> import execnet
>>> gw = execnet.makegateway()
>>> channel = gw.remote_exec("""
...        ch1, ch2 = channel.receive()
...        ch2.send("world")
...        ch1.send("hello")
... """)
>>> c1 = gw.newchannel()    # create new channel
>>> c2 = gw.newchannel()    # create another channel
>>> channel.send((c1, c2))  # send them over
>>> c1.receive()
'hello'
>>> c2.receive()
'world'
```

--------------------------------

### Cross-interpreter Serialization

Source: https://execnet.readthedocs.io/en/latest/basics.html

Functions for serializing and deserializing Python objects across different interpreter versions.

```APIDOC
## Serialization Functions

### execnet.dumps(spec)
- **Description**: Serialize a builtin Python object to a bytestring.
- **Parameters**: `spec` (object) - The object to serialize.

### execnet.loads(spec, py2str_as_py3str=False, py3str_as_py2str=False)
- **Description**: Deserialize a bytestring to a Python object.
- **Parameters**:
  - **spec** (bytes) - The bytestring to deserialize.
  - **py2str_as_py3str** (bool) - Convert Python2 strings to Python3 text objects.
  - **py3str_as_py2str** (bool) - Convert Python3 strings to Python2 strings.
```

--------------------------------

### Define remote command module

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

A module designed to be executed remotely, evaluating incoming channel items.

```python
def simple(arg):
    return arg + 1


def listdir(path):
    return os.listdir(path)


if __name__ == "__channelexec__":
    for item in channel:  # type: ignore[name-defined]
        channel.send(eval(item))  # type: ignore[name-defined]
```

--------------------------------

### Retrieve remote file contents

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Client code to request and receive file contents from a remote SSH host.

```python
import execnet
import servefiles
gw = execnet.makegateway("ssh=codespeak.net")
channel = gw.remote_exec(servefiles)

for fn in ('/etc/passwd', '/etc/group'):
    channel.send(fn)
    content = channel.receive()
    print(fn)
    print(content)
```

--------------------------------

### Cross-Interpreter Object Serialization

Source: https://execnet.readthedocs.io/en/latest/basics.html

Serialize Python objects to a bytestring using `dumps` and deserialize them back using `loads`. This is useful for transferring data between different Python interpreters. Ensure objects are of builtin Python types.

```python
import execnet
dump = execnet.dumps([1,2,3])
execnet.loads(dump)
```

--------------------------------

### Remote-exec a module

Source: https://execnet.readthedocs.io/en/latest/example/test_info.html

Pass a module object to remote_exec to transfer its source code. The module can detect remote execution via the __channelexec__ attribute.

```python
# content of a module remote1.py

if __name__ == "__channelexec__":
    channel.send("initialization complete")  # type: ignore[name-defined]
```

```python
>>> import execnet, remote1
>>> gw = execnet.makegateway()
>>> ch = gw.remote_exec(remote1)
>>> print (ch.receive())
initialization complete
```
Tokens:
Raw