### Basic FLAME Integration Setup Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/integration-guide.md Install the Pythonx environment and paths, then start a FLAME runner. Evaluate Python code on the FLAME runner. ```elixir # Parent node (after Pythonx init) env = Pythonx.install_env() paths = Pythonx.install_paths() # Start FLAME runner {:ok, runner} = FLAME.start_link( env: env, pool_per_node: 1 ) # Evaluate on FLAME runner {result, _} = Pythonx.eval("numpy.array([1,2,3])", %{}) ``` -------------------------------- ### Example Usage of install_env/0 Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Demonstrates how to obtain and use the environment variables returned by `install_env/0` when starting a FLAME runner or a remote node. ```elixir env = Pythonx.install_env() # Use env when starting FLAME runner or remote node ``` -------------------------------- ### Application Setup for Pythonx Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Shows how to integrate Pythonx into an Elixir application by adding it to dependencies and configuring initialization in `config/config.exs`. Pythonx initializes automatically when the application starts. ```elixir # mix.exs def deps do [ {:pythonx, "~> 0.4.0"} ] end # config/config.exs import Config config :pythonx, :uv_init, pyproject_toml: """ [project] name = "myapp" version = "0.1.0" requires-python = "==3.13.*" dependencies = ["numpy==2.2.2"] """ # Application starts, Pythonx initializes automatically # Now evaluate Python code in your application def my_function do {result, _} = Pythonx.eval("1 + 1", %{}) Pythonx.decode(result) end ``` -------------------------------- ### Initialize Python with uv Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Installs Python and dependencies using uv. Use the `force` option to clear the cache and re-install. ```elixir Pythonx.uv_init("\n[project]\nname = \"project\"\nversion = \"0.0.0\"\nrequires-python = \"==3.13.*\"\ndependencies = [\n \"numpy==2.2.2\"\n]") ``` ```elixir Pythonx.uv_init("\n[project]\nname = \"myapp\"\nversion = \"1.0.0\"\nrequires-python = \"==3.13.*\"\n", force: true, uv_version: "0.7.21") ``` -------------------------------- ### Elixir Manual Application Start for Pythonx Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Demonstrates how to manually start the :pythonx application if it doesn't auto-start. ```elixir Application.start(:pythonx) ``` -------------------------------- ### Initialize Pythonx in Script (Dynamic) Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Demonstrates how to install and initialize Pythonx dynamically within a script using Mix.install. This is useful for quick testing and development. ```elixir Mix.install([{:pythonx, "~> 0.4.0"}]) Pythonx.uv_init(""" [project] name = "project" version = "0.0.0" requires-python = "==3.13.*" """) # Now you can use Pythonx {result, _} = Pythonx.eval("2 + 2", %{}) Pythonx.decode(result) ``` -------------------------------- ### Initialize Python Environment with Requests Dependency Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Configures a Python environment with the Requests library, enabling HTTP requests to be made. This example shows the setup for making a GET request. ```elixir Pythonx.uv_init("\n[project]\nname = \"http_example\"\nversion = \"0.0.0\"\nrequires-python = \"==3.13.*\"\ndependencies = [\"requests>=2.31.0\"]\n") {result, _} = Pythonx.eval("\nimport requests\nresponse = requests.get('https://api.example.com/data')\ndata = response.json()\n", %{}) ``` -------------------------------- ### Initialize uv with Full pyproject.toml Configuration Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Example of initializing uv with a comprehensive pyproject.toml configuration string in Elixir. ```elixir Pythonx.uv_init(""" [project] name = "data-pipeline" version = "2.1.0" requires-python = ">=3.12,<4" dependencies = [ "numpy==2.0.0", "pandas==2.1.4", "polars==0.19.0", "requests>=2.31.0", "python-dateutil>=2.8.2", ] [build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta" """) ``` -------------------------------- ### uv_init/2 Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Installs Python and dependencies using the uv package manager and initializes the interpreter. It takes the content of a `pyproject.toml` file and optional configuration keywords. ```APIDOC ## Function: `uv_init/2` ### Description Installs Python and dependencies using the uv package manager and initializes the interpreter. ### Parameters #### Path Parameters - `pyproject_toml` (string) - Required - Content of a `pyproject.toml` file with Python configuration. Must include `project.name` and `project.version` fields. - `opts` (keyword list) - Optional - Optional configuration keywords. #### Options - `:force` (boolean) - Optional - `false` - If true, runs with empty project cache and forces re-installation. - `:uv_version` (string) - Optional - `"0.8.5"` - Version of the uv package manager to use. - `:native_tls` (boolean) - Optional - `false` - Use system's native TLS instead of vendored rustls. Useful in corporate environments. - `:python` (string) - Optional - `nil` - Specifies the Python version or variant to install (e.g., `"3.14t"` for free-threaded). ### Returns - `:ok` on success. ### Raises - `RuntimeError` - If Python and dependencies installation fails. ### Examples ```elixir Pythonx.uv_init(""" [project] name = "project" version = "0.0.0" requires-python = "==3.13.*" dependencies = [ "numpy==2.2.2" ]""") Pythonx.uv_init(""" [project] name = "myapp" version = "1.0.0" requires-python = "==3.13.*""", force: true, uv_version: "0.7.21") ``` ``` -------------------------------- ### FLAME Distributed Deployment Setup for Pythonx Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Set up Pythonx for distributed deployment across remote nodes using FLAME. This involves installing the environment on the origin node and starting a FLAME runner with the necessary environment and private directory. ```elixir # On origin node, after Pythonx initialization env = Pythonx.install_env() paths = Pythonx.install_paths() # Start FLAME runner with environment runner = FLAME.start_link!( env: env, priv_dir: paths # Ensure remote has the priv files ) ``` -------------------------------- ### Example of Free-Threaded Python Integration Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/INDEX.md Demonstrates how to leverage free-threaded Python (available in Python 3.14+) for improved concurrency. This requires specific setup and configuration. ```elixir # Ensure Python is compiled with --enable-free-threaded # Configuration might involve setting specific environment variables or uv_init options # Pythonx.uv_init(python_executable: "/path/to/free_threaded_python") # Code that benefits from concurrency without GIL contention ``` -------------------------------- ### Initialize Python Environment with NumPy Dependency Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Initializes a Python environment using uv_init and installs the NumPy library. This setup is required before using NumPy functions. ```elixir Pythonx.uv_init("\n[project]\nname = \"numpy_example\"\nversion = \"0.0.0\"\nrequires-python = \"==3.13.*\"\ndependencies = [\"numpy==2.2.2\"]\n") # Matrix multiplication {result, _} = Pythonx.eval("\nimport numpy as np\na = np.array([[1, 2], [3, 4]])\nb = np.array([[5, 6], [7, 8]])\nresult = a @ b\n", %{}) Pythonx.decode(result) #=> [[19, 22], [43, 50]] ``` -------------------------------- ### Get File Paths for Pythonx Initialization Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Retrieves the list of file paths that the `install_env/0` function depends on for Pythonx initialization. This is necessary for Pythonx to function correctly on remote nodes. ```elixir @spec install_paths() :: list(String.t()) def install_paths() ``` -------------------------------- ### RuntimeError: Missing Python Installation Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/errors.md This error occurs when PythonX fails to download or install Python. Check network connectivity, ensure write permissions for PYTHONX_CACHE_DIR, and review uv errors. ```text RuntimeError: fetching Python and dependencies failed, see standard output for details ``` -------------------------------- ### Compile-Time Pythonx Configuration Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Configure Pythonx at compile time by setting application configuration in `config/config.exs`. This method installs Python and dependencies during compilation and initializes the interpreter automatically on application start. ```elixir import Config config :pythonx, :uv_init, pyproject_toml: """ [project] name = "project" version = "0.0.0" requires-python = "==3.13.*" dependencies = [ "numpy==2.2.2" ] """, uv_version: "0.8.5", python: nil, native_tls: false ``` -------------------------------- ### Initialize Python Environment with Pandas Dependency Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Sets up a Python environment with the Pandas library installed. This allows for the creation and manipulation of DataFrames. ```elixir Pythonx.uv_init("\n[project]\nname = \"pandas_example\"\nversion = \"0.0.0\"\nrequires-python = \"==3.13.*\"\ndependencies = [\"pandas==2.1.4\"]\n") {_result, globals} = Pythonx.eval("\nimport pandas as pd\ndf = pd.DataFrame({ 'name': ['Alice', 'Bob', 'Charlie'], 'age': [25, 30, 35], 'city': ['NYC', 'LA', 'Chicago'] }) ", %{}) # Now df is in globals for further operations ``` -------------------------------- ### Get Environment for Distribution Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/INDEX.md Retrieves the current Python environment configuration, including installed packages and paths. This is essential for replicating the environment on other nodes or for deployment. ```elixir Pythonx.install_env() ``` -------------------------------- ### Example of Handling Remote Objects Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/INDEX.md Illustrates how to work with Python objects that reside on a remote Elixir node. This pattern is crucial for distributed applications. ```elixir # Assume 'remote_obj_ref' is a reference to a Python object on a remote node # Copy the object to the local node local_obj = Pythonx.copy_remote_object(remote_obj_ref) # Now you can interact with 'local_obj' as if it were created locally Pythonx.eval("print(obj)", obj: local_obj) ``` -------------------------------- ### Sigil PY Basic Arithmetic Example Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Demonstrates basic arithmetic operations using the `~PY` sigil, showing how Elixir variables are implicitly used as Python globals and the result is captured. ```elixir import Pythonx # Basic arithmetic with implicit globals x = 5 ~PY""" result = x * 2 """ result # => #Pythonx.Object<10> ``` -------------------------------- ### Handle ArgumentError for Nil Decoding Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/errors.md This example demonstrates catching an `ArgumentError` when `Pythonx.decode/1` is called with `nil` instead of a valid `%Pythonx.Object{}`. ```elixir try do Pythonx.decode(nil) rescue error in ArgumentError -> IO.puts(error.message) # => "Pythonx.decode/1 expects a %Pythonx.Object{}, but got nil..." end ``` -------------------------------- ### Handle RuntimeError in Pythonx Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/errors.md Demonstrates how to catch and handle RuntimeError exceptions from Pythonx.eval/3. It includes logic to start the application or initialize the interpreter based on the error message. ```elixir try do Pythonx.eval("code", %{}) rescue error in RuntimeError -> case error.message do "the :pythonx application needs to be started" <> _ -> # Start the application Application.start(:pythonx) "before calling" <> _ -> # Initialize first Pythonx.uv_init(pyproject_content) _ -> reraise(error, __STACKTRACE__()) end end ``` -------------------------------- ### Example of State Management Pattern Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/INDEX.md Shows a pattern for managing state when interacting with Python from Elixir. This often involves passing state explicitly or using shared Python objects carefully. ```elixir # Example: Passing state explicitly state = %{counter: 0} # Increment counter in Python new_state = Pythonx.eval("state['counter'] += 1; state", state: state) # Use 'new_state' for subsequent operations ``` -------------------------------- ### Get Paths for Distribution Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/INDEX.md Retrieves the necessary paths for distributing the Python environment. Use this information to ensure consistency across different deployment targets. ```elixir Pythonx.install_paths() ``` -------------------------------- ### Pythonx Decoding Flow Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/modules-overview.md Outlines the steps for decoding Python objects back into Elixir terms, starting from Pythonx.decode and involving NIF calls and recursive Elixir processing. ```text Pythonx.decode(object) ↓ Pythonx.NIF.decode_once(object) ↓ [C++ NIF] Python object inspection ↓ {type, intermediate_data} ↓ Recursive Elixir processing ↓ Elixir term ``` -------------------------------- ### Elixir Custom Encoder Example Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/types.md Demonstrates how to define and use a custom Elixir encoder function to encode Elixir tuples as Python lists. Shows implicit and explicit usage with Pythonx.encode!. ```elixir custom_encoder = fn tuple, encoder when is_tuple(tuple) -> Pythonx.Encoder.encode(Tuple.to_list(tuple), encoder) other, encoder -> Pythonx.Encoder.encode(other, encoder) end Pythonx.encode!({1, 2, 3}, custom_encoder) #=> #Pythonx.Object<[1, 2, 3]> ``` ```elixir # With encode! in eval context {result, _} = Pythonx.eval("x", %{"x" => Pythonx.encode!(data, custom_encoder)}) ``` -------------------------------- ### Get Environment Variables for Pythonx Initialization Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Use this function to retrieve environment variables required for initializing Pythonx on remote nodes, such as FLAME nodes. It returns a map preserving the initialization state. ```elixir @spec install_env() :: map() def install_env() ``` -------------------------------- ### Example Implementation of Pythonx.Encoder for Complex Type Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Demonstrates how to implement the Encoder protocol for a custom Elixir struct `Complex`. This involves using `Pythonx.eval` to create a Python complex number. ```elixir defmodule Complex do defstruct [:re, :im] end defimpl Pythonx.Encoder, for: Complex do def encode(complex, _encoder) do {result, %{}} = Pythonx.eval( """ complex(re, im) """, %{"re" => complex.re, "im" => complex.im} ) result end end Pythonx.encode!(%Complex{re: 1, im: -1}) #=> #Pythonx.Object<(1-1j)> ``` -------------------------------- ### Example of Custom Data Type Encoding Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/INDEX.md Demonstrates how to define a custom encoder for a specific Elixir data type to be used with Pythonx. This allows for seamless conversion of complex or application-specific types. ```elixir # Define a custom encoder module defmodule MyEncoder do @behaviour Pythonx.Encoder def encode(data, _target) do # Custom encoding logic here {:ok, encoded_data} end def decode(data, _source) do # Custom decoding logic here {:ok, decoded_data} end end # Use the custom encoder Pythonx.encode!(my_custom_data, :python, encoder: MyEncoder) ``` -------------------------------- ### Sigil PY Multiple Variables and Decoding Example Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Illustrates using the `~PY` sigil with multiple Elixir variables and performing a calculation involving imported Python modules. The result is then decoded back into an Elixir-compatible format. ```elixir import Pythonx # Multiple variables a, b = 3, 4 hypotenuse = ~PY""" import math math.sqrt(a ** 2 + b ** 2) """ Pythonx.decode(hypotenuse) # => 5.0 ``` -------------------------------- ### Elixir Implementing Custom Encoders with Protocol Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/types.md Shows how to implement custom encoders for specific Elixir structs by defining a module and using the `Pythonx.Encoder` protocol. This example converts a MyStruct to a map before re-encoding. ```elixir defmodule MyStruct do defstruct [:field] end defimpl Pythonx.Encoder, for: MyStruct do def encode(my_struct, encoder) do # Convert to map and re-encode Pythonx.Encoder.encode(%{"field" => my_struct.field}, encoder) end end ``` -------------------------------- ### State Management with Agent: Eval and Get Global Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Manage Python interpreter state using Elixir's Agent. The `eval` function updates the state, and `get_global` retrieves a variable. ```elixir defmodule PythonInterpreter do def start_link(_opts) do Agent.start_link(fn -> %{} end, name: __MODULE__) end def eval(code) do Agent.get_and_update(__MODULE__, fn globals -> {result, new_globals} = Pythonx.eval(code, globals) {result, new_globals} end) end def get_global(name) do Agent.get(__MODULE__, fn globals -> Map.get(globals, name) end) end end # Usage PythonInterpreter.start_link([]) PythonInterpreter.eval("x = 42") obj = PythonInterpreter.get_global("x") Pythonx.decode(obj) #=> 42 ``` -------------------------------- ### Pythonx.uv_init/2 Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Initializes a Python project using the uv package manager. It takes the pyproject.toml content and optional configuration settings. ```APIDOC ## Function: `Pythonx.uv_init/2` ### Description Initializes a Python project using the uv package manager. It takes the `pyproject.toml` file content as a string and optional configuration settings. ### Parameters - `pyproject_toml` (string): Required. The content of the `pyproject.toml` file. Must include `project.name` and `project.version` fields. - `opts` (keyword): Optional. Configuration options for initialization. Defaults to `[]`. ### Options - `:force` (boolean): Optional. If `true`, clears the project cache and Python installation, then re-fetches everything. Defaults to `false`. - `:uv_version` (string): Optional. Specifies the version of the uv package manager to use. Defaults to `"0.8.5"`. - `:native_tls` (boolean): Optional. If `true`, uses the system's native TLS implementation. Defaults to `false`. - `:python` (string): Optional. Specifies the Python version or variant to use. Takes precedence over `requires-python` in `pyproject.toml`. Examples: `"3.13"`, `"3.14t"` (free-threaded), `"3.12.1"`. ### Examples ```elixir # Basic initialization with default options Pythonx.uv_init(""" [project] name = \"myapp\" version = \"1.0.0\" requires-python = \"==3.13.*\" dependencies = [\"numpy>=2.0\"] """) # Custom uv version Pythonx.uv_init(""" [project] name = \"myapp\" version = \"1.0.0\" requires-python = \"==3.13.*\" """, uv_version: \"0.7.21\") # Force reinstallation Pythonx.uv_init(pyproject_content, force: true) # Use free-threaded Python Pythonx.uv_init(""" [project] name = \"myapp\" version = \"1.0.0\" requires-python = \"==3.14.*\" """, python: \"3.14t\") # Corporate environment with native TLS Pythonx.uv_init(""" [project] name = \"myapp\" version = \"1.0.0\" requires-python = \"==3.13.*\" """, native_tls: true) ``` ``` -------------------------------- ### Basic Pythonx uv_init Initialization Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Initializes a Python project using uv with default options. Ensure the pyproject.toml content includes project name, version, and Python requirements. ```elixir Pythonx.uv_init(""" [project] name = "myapp" version = "1.0.0" requires-python = "==3.13.*" dependencies = ["numpy>=2.0"] """) ``` -------------------------------- ### install_paths/0 Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Provides the file paths required for the `install_env/0` function to initialize Pythonx. These paths are essential for Pythonx to operate correctly on remote nodes. ```APIDOC ## Function: `install_paths/0` ### Description Returns file paths that `install_env/0` initialization depends on. ### Returns List of directory paths needed for Pythonx to function on remote nodes. ### Raises * `RuntimeError`: If Pythonx has not been initialized yet. ### Examples ```elixir paths = Pythonx.install_paths() # Use paths when configuring Pythonx initialization ``` ``` -------------------------------- ### Initialize Pythonx at Runtime Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Initialize Pythonx with a Python project string at runtime. Useful for dynamic configurations. ```elixir Pythonx.uv_init(""" [project] name = "myapp" version = "1.0.0" requires-python = "==3.13.*" """) ``` -------------------------------- ### Configure Pythonx UV Initialization Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/integration-guide.md Optionally configure Pythonx with a `pyproject.toml` content for compile-time initialization. This specifies project details and Python dependencies. ```elixir import Config config :pythonx, :uv_init, pyproject_toml: """ [project] name = "project" version = "0.0.0" requires-python = "==3.13.*" dependencies = [ "numpy==2.2.2" ] """ ``` -------------------------------- ### Pythonx uv_init with Force Reinstallation Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Forces reinstallation of project dependencies by clearing the cache and re-fetching everything. Use this option when encountering issues or to ensure a clean state. ```elixir Pythonx.uv_init(pyproject_content, force: true) ``` -------------------------------- ### Pythonx uv_init for Corporate Environments with Native TLS Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Initializes a Python project using the system's native TLS implementation, suitable for corporate environments with strict certificate store requirements. Set `native_tls` to `true`. ```elixir Pythonx.uv_init(""" [project] name = "myapp" version = "1.0.0" requires-python = "==3.13.*" """, native_tls: true) ``` -------------------------------- ### Initialize Python with Packages Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/INDEX.md Initializes the Python interpreter with a specified list of packages. Ensure all required packages are listed here before evaluating any Python code. ```elixir Pythonx.uv_init(packages: ["numpy", "pandas"]) ``` -------------------------------- ### Initialize Pythonx at Compile-time Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Configure Pythonx with a pyproject.toml file during compilation. This is the recommended approach for applications. ```elixir config :pythonx, :uv_init, pyproject_toml: """ [project] name = "myapp" version = "1.0.0" requires-python = "==3.13.*" dependencies = ["numpy==2.2.2"] """ ``` -------------------------------- ### install_env/0 Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Retrieves environment variables necessary for initializing Pythonx on remote nodes. This is crucial for setting up the Python execution environment across distributed systems like FLAME. ```APIDOC ## Function: `install_env/0` ### Description Returns environment variables needed to initialize Pythonx on remote nodes (e.g., FLAME nodes). ### Returns A map of environment variables that preserve Pythonx initialization state. ### Raises * `RuntimeError`: If Pythonx has not been initialized yet. ### Examples ```elixir env = Pythonx.install_env() # Use env when starting FLAME runner or remote node ``` ``` -------------------------------- ### Pythonx uv_init with Custom uv Version Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Initializes a Python project using a specific version of the uv package manager. This is useful for compatibility or testing with different uv releases. ```elixir Pythonx.uv_init(""" [project] name = "myapp" version = "1.0.0" requires-python = "==3.13.*" """, uv_version: "0.7.21") ``` -------------------------------- ### Configure Python Interpreter (Application) Source: https://github.com/livebook-dev/pythonx/blob/main/README.md Configure Pythonx initialization, including pyproject.toml content and optionally uv_version, in `config/config.exs` for application-wide use. ```elixir import Config config :pythonx, :uv_init, pyproject_toml: """ [project] name = "project" version = "0.0.0" requires-python = "==3.13.* " dependencies = [ "numpy==2.2.2" ] """ config :pythonx, :uv_init, ..., uv_version: "0.7.21" ``` -------------------------------- ### RuntimeError: Unsupported Architecture Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/errors.md Indicates that the uv package manager does not have a binary for your system's architecture. Use a supported platform or consider building uv from source. ```text RuntimeError: uv is not available for architecture: ``` -------------------------------- ### Initialize and Evaluate Python Code Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/integration-guide.md Initialize Pythonx with a `pyproject.toml` string and evaluate Python code. The `eval/2` function returns a result and global state, which can then be decoded. ```elixir Pythonx.uv_init(""" [project] name = "project" version = "0.0.0" requires-python = "==3.13.*" """) # Evaluate Python code {result, globals} = Pythonx.eval("1 + 1", %{}) Pythonx.decode(result) # => 2 ``` -------------------------------- ### Pythonx uv_init with Free-Threaded Python Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Initializes a Python project using a free-threaded Python variant. This is specified using the 't' suffix for the Python version, e.g., '3.14t'. ```elixir Pythonx.uv_init(""" [project] name = "myapp" version = "1.0.0" requires-python = "==3.14.*" """, python: "3.14t") ``` -------------------------------- ### Usage of Pythonx.Object Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/types.md Demonstrates how to create and use Pythonx.Object instances for encoding, evaluation, and decoding Python objects. ```elixir # Objects are created through encoding obj = Pythonx.encode!([1, 2, 3]) # Or from evaluation results {obj, _globals} = Pythonx.eval("x + 1", %{"x" => 10}) # Decode to Elixir terms value = Pythonx.decode(obj) # Pass to other evaluations {_result, _} = Pythonx.eval("len(lst)", %{"lst" => obj}) ``` -------------------------------- ### Runtime Pythonx Initialization Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Initialize Pythonx at runtime by calling `Pythonx.uv_init` with the content of `pyproject.toml`. This is used when compile-time configuration is not employed. ```elixir Pythonx.uv_init(""" [project] name = "project" version = "0.0.0" requires-python = "==3.13.*" dependencies = ["numpy==2.2.2"] """) ``` -------------------------------- ### Initialize Python Interpreter (Script) Source: https://github.com/livebook-dev/pythonx/blob/main/README.md Initialize the Python interpreter, specifying the Python version and project dependencies via a pyproject.toml string. ```elixir Pythonx.uv_init(""" [project] name = "project" version = "0.0.0" requires-python = "==3.13.* " dependencies = [ "numpy==2.2.2" ] """) ``` -------------------------------- ### Simple Calculation with ~PY Macro Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Demonstrates a basic arithmetic calculation using the ~PY macro in an interactive Elixir environment. The result is automatically decoded. ```elixir import Pythonx # Simple calculation x = 10 y = ~PY""" x * 2 """ y #=> #Pythonx.Object<20> ``` -------------------------------- ### Elixir Python Interpreter Initialization with uv_init Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Initializes the Python interpreter using `uv_init/2`, specifying project details and Python version requirements. ```elixir Pythonx.uv_init(""" [project] name = "project" version = "0.0.0" requires-python = "==3.13.*" """) ``` -------------------------------- ### Initialize Python Interpreter Source: https://github.com/livebook-dev/pythonx/blob/main/README.md Initializes the Python interpreter with a specified Python version and dependencies defined in a pyproject.toml string. ```APIDOC ## Initialize Python Interpreter ### Description Initializes the Python interpreter, specifying the desired Python version and dependencies. ### Method `Pythonx.uv_init/1` ### Parameters #### Request Body - **pyproject_toml** (string) - Required - A string containing the pyproject.toml content defining project metadata, Python version, and dependencies. ### Request Example ```elixir Pythonx.uv_init(""" [project] name = "project" version = "0.0.0" requires-python = "==3.13.*" dependencies = [ "numpy==2.2.2" ] """) ``` ``` -------------------------------- ### Elixir Mix.exs Dependency Configuration for Pythonx Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Shows how to add the Pythonx library as a dependency in your Elixir project's mix.exs file. ```elixir def deps do [{:pythonx, "~> 0.4.0"}] end ``` -------------------------------- ### Elixir Python Session Context Manager Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Demonstrates a basic Elixir context manager for evaluating Python code within a session. Ensures proper handling of session state across evaluations. ```elixir defmodule PythonSession do def new, do: %{} def eval(session, code) do {result, new_session} = Pythonx.eval(code, session) {result, new_session} end end session = PythonSession.new() {_r, session} = PythonSession.eval(session, "x = 1") {result, _session} = PythonSession.eval(session, "x + 1") ``` -------------------------------- ### Add Pythonx Dependency (Mix) Source: https://github.com/livebook-dev/pythonx/blob/main/README.md Add Pythonx to your project's dependencies using Mix.install for script-based usage. ```elixir Mix.install([ {:pythonx, "~> 0.4.0"} ]) ``` -------------------------------- ### Complex Code Execution with ~PY Macro Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Illustrates executing more complex Python code, including string formatting with f-strings, using the ~PY macro. The result is decoded using Pythonx.decode. ```elixir name = "World" greeting = ~PY""" greeting = f"Hello, {name}!" """ Pythonx.decode(greeting) #=> "Hello, World!" ``` -------------------------------- ### Add Pythonx Dependency (Application) Source: https://github.com/livebook-dev/pythonx/blob/main/README.md Declare Pythonx as a project dependency in your application's `mix.exs` file. ```elixir def deps do [ {:pythonx, "~> 0.4.0"} ] end ``` -------------------------------- ### Pythonx (Main Module) Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/modules-overview.md The main module for Pythonx functionality, handling initialization, evaluation, encoding/decoding, and remote object management. ```APIDOC ## Pythonx (Main Module) ### Description Public API for all Pythonx functionality. Entry point for initialization, evaluation, encoding/decoding, and remote object handling. ### Key Responsibilities: - Python initialization via uv package manager - Code evaluation with variable passing and result retrieval - Automatic encoding of Elixir terms to Python objects - Decoding of Python objects to Elixir terms - Remote evaluation and object lifecycle management - Distributed serialization via pickle ### Main Functions: - `uv_init/2` - Initialize interpreter with Python packages - `eval/3` - Evaluate Python code with globals - `encode!/2` - Convert Elixir terms to Python objects - `decode/1` - Convert Python objects to Elixir terms - `remote_eval/4` - Evaluate on remote nodes - `copy_remote_object/1` - Copy remote objects locally - `install_env/0` - Get environment for FLAME/distributed - `install_paths/0` - Get file paths for distribution - `sigil_PY/2` - Macro for convenient Python evaluation ### Public Type: `Pythonx.encoder` ``` -------------------------------- ### RuntimeError: NIF Load Error Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/errors.md The libpythonx NIF library failed to load. This can be due to missing native dependencies, incompatible Erlang/OTP versions, or missing shared libraries. ```text RuntimeError: failed to load NIF library, reason: ``` -------------------------------- ### Handling Python ZeroDivisionError Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Demonstrates how to catch and handle Python's ZeroDivisionError when using Pythonx.eval by rescuing Pythonx.Error in Elixir. ```elixir try do Pythonx.eval("1 / 0", %{}) rescue error in Pythonx.Error -> IO.puts(Exception.message(error)) # Output: # Python exception raised # ZeroDivisionError: division by zero end ``` -------------------------------- ### Defining Project Dependencies in pyproject.toml Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md List project dependencies using standard PEP 508 specifications. ```toml [project] dependencies = [ "numpy==2.2.2", "pandas>=1.5.0", "requests[security]", ] ``` -------------------------------- ### Set custom cache directory via export Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Use a custom directory for Pythonx caching by exporting the PYTHONX_CACHE_DIR environment variable before running commands. ```bash # Use custom cache directory export PYTHONX_CACHE_DIR=/custom/cache/path ``` -------------------------------- ### Basic Remote Evaluation Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Performs a basic remote evaluation on a different node using Pythonx. Ensure Pythonx is initialized on both nodes before attempting remote evaluation. ```elixir # Ensure Pythonx is initialized on both nodes # On local node {result, _globals} = Pythonx.remote_eval( :remote_node, """ import numpy numpy.array([1, 2, 3]) """, %{} ) # Result is a remote object reference result #=> #Pythonx.Object<...>[node: :remote_node] ``` -------------------------------- ### Managing Globals for State Persistence Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Shows how to maintain state across multiple Python evaluations by passing updated globals. This is crucial for sequential operations. ```elixir {_result, g1} = Pythonx.eval("x = 1", %{}) {_result, g2} = Pythonx.eval("x = x + 1", g1) {result, _g3} = Pythonx.eval("x", g2) Pythonx.decode(result) # => 2 ``` -------------------------------- ### Elixir Compile-Time Pythonx Configuration Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Configure Pythonx to fetch Python dependencies at compile time for Elixir releases. This ensures all necessary files are included in the release package. ```elixir import Config config :pythonx, :uv_init, pyproject_toml: """ [project] name = "myapp" version = "1.0.0" requires-python = "==3.13.*" dependencies = ["numpy==2.2.2"] """ # Build the release # mix release # On target, Pythonx is automatically initialized on application startup ``` -------------------------------- ### Encode Elixir Data Types to Python Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Illustrates how to convert various Elixir data types, including atoms, numbers, strings, binaries, collections, and PIDs, into their Python equivalents using `Pythonx.encode!/1`. ```elixir # Atoms Pythonx.encode!(nil) #=> None Pythonx.encode!(true) #=> True Pythonx.encode!(:hello) #=> 'hello' # Numbers Pythonx.encode!(42) #=> 42 Pythonx.encode!(3.14) #=> 3.14 Pythonx.encode!(2 ** 100) #=> 1267650600228229401496703205376 # Strings and binaries Pythonx.encode!("hello") #=> b'hello' Pythonx.encode!(<<65, 66>>) #=> b'AB' # Collections Pythonx.encode!({1, 2, 3}) #=> (1, 2, 3) Pythonx.encode!([1, 2, 3]) #=> [1, 2, 3] Pythonx.encode!(%{"a" => 1}) #=> {b'a': 1} Pythonx.encode!(MapSet.new([1, 2])) #=> {1, 2} # PIDs (for message passing) Pythonx.encode!(self()) #=> ``` -------------------------------- ### Handle Python Exceptions in Elixir Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/api-reference.md Illustrates how to rescue and handle Python exceptions raised during the evaluation of Python code using Pythonx.eval. It shows how to access and print the formatted error message. ```elixir try do Pythonx.eval("1 / 0", %{}) rescue error in Pythonx.Error -> IO.puts(Exception.message(error)) # Outputs: Python exception raised # ZeroDivisionError: division by zero end ``` -------------------------------- ### Reusing Global State in Pythonx Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/integration-guide.md Improve performance by importing Python packages once and reusing the global state across multiple evaluations, rather than reimporting each time. ```elixir # ✗ SLOW: Reimport packages each time Pythonx.eval("import numpy; numpy.array([1,2,3])", %{}) Pythonx.eval("import numpy; numpy.array([4,5,6])", %{}) # ✓ FAST: Import once, reuse {_result, globals} = Pythonx.eval("import numpy", %{}) {arr1, globals} = Pythonx.eval("numpy.array([1,2,3])", globals) {arr2, globals} = Pythonx.eval("numpy.array([4,5,6])", globals) ``` -------------------------------- ### Pythonx Evaluation Flow Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/modules-overview.md Illustrates the data flow during code evaluation, from Elixir's Pythonx.eval to the Python interpreter and back. ```text Pythonx.eval(code, globals) ↓ encode!(globals values) ↓ [Pythonx.Encoder protocol] Pythonx.NIF.eval(code, encoded_globals) ↓ [C++ NIF] Python interpreter evaluates code ↓ Results + Python exceptions captured ↓ Pythonx.Janitor routes output ↓ {result_object, globals_map} ``` -------------------------------- ### FLAME Integration with Remote Evaluation Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/integration-guide.md Pythonx integrates with FLAME.Trackable, keeping objects alive while referenced. Use FLAME.call for remote computations. ```elixir # Pythonx automatically integrates with FLAME.Trackable # Objects are tracked and kept alive while referenced def expensive_computation do FLAME.call(runner, fn -> Pythonx.eval(""" import numpy as np result = np.random.randn(10000, 10000) result @ result.T """, %{}) end) end ``` -------------------------------- ### Elixir Copying Remote Python Objects Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Shows how to retrieve an object from a remote Python process and copy it to the local environment for use in local evaluations. ```elixir remote_obj = Pythonx.remote_eval(:node1, "[1,2,3]", %{}) local_obj = Pythonx.copy_remote_object(remote_obj) {_result, _} = Pythonx.eval("len(lst)", %{"lst" => local_obj}) ``` -------------------------------- ### Evaluate Python Code with Input Variables Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Execute Python code, passing Elixir variables as input. The result and global state are returned. ```elixir # Evaluate code with input variables {result, globals} = Pythonx.eval( """ output = x + y """, %{"x" => 10, "y" => 20} ) # Decode result to Elixir term Pythonx.decode(result) # => 30 # Carry state forward {_result, globals2} = Pythonx.eval("z = x + y", globals) ``` -------------------------------- ### Set custom cache directory via Elixir Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Configure a custom cache directory for Pythonx within an Elixir application before initializing uv. This ensures dependencies are cached in the specified location. ```elixir # Or in Elixir before initializing System.put_env("PYTHONX_CACHE_DIR", "/custom/cache/path") Pythonx.uv_init(pyproject_content) ``` -------------------------------- ### Copy Remote Object to Local Node Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Demonstrates copying a remote object from a remote node to the local node. The copied object can then be used in local Pythonx evaluations. ```elixir # Evaluate on remote node {remote_list, _} = Pythonx.remote_eval(:node1, "[1, 2, 3]", %{}) # Copy to local node local_list = Pythonx.copy_remote_object(remote_list) # Now can use in local evaluation {result, _} = Pythonx.eval("sum(lst)", %{"lst" => local_list}) Pythonx.decode(result) #=> 6 ``` -------------------------------- ### Concurrency Patterns for Python Evaluation Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/README.md Highlights recommended and discouraged concurrency patterns when using Pythonx. Due to Python's GIL, concurrent evaluations from multiple processes serialize, impacting performance. ```elixir # ✓ GOOD: Single process handles Python {:ok, _} = Agent.start_link(fn -> %{} end, name: :python_state) # ✗ BAD: Multiple processes calling eval (they serialize) Task.start(fn -> Pythonx.eval(code1, g1) end) Task.start(fn -> Pythonx.eval(code2, g2) end) ``` -------------------------------- ### Catch Pythonx.Error in Elixir Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/errors.md Demonstrates how to use a try/rescue block in Elixir to catch Pythonx.Error exceptions raised during Python code evaluation. ```elixir try do Pythonx.eval("1 / 0", %{}) rescue error in Pythonx.Error -> IO.puts(Exception.message(error)) end ``` -------------------------------- ### Required Fields in pyproject.toml Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md The 'name' and 'version' fields are mandatory for a Python project configuration. ```toml [project] name = "project" version = "0.0.0" ``` -------------------------------- ### Evaluate Python Code (Script) Source: https://github.com/livebook-dev/pythonx/blob/main/README.md Evaluate Python code with initial global variables and retrieve the result and updated globals. Use Pythonx.decode/1 to convert Python objects to Elixir. ```elixir {result, globals} = Pythonx.eval( """ y = 10 x + y """, %{"x" => 1} ) Pythonx.decode(result) #=> 11 globals #=> %{ #=> "x" => #Pythonx.Object< #=> 1 #=> >, #=> "y" => #Pythonx.Object< #=> 10 #=> > #=> } ``` -------------------------------- ### Bulk Operations in Pythonx Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/integration-guide.md Use bulk operations for better performance by evaluating multiple operations in a single Pythonx.eval call instead of many small ones. ```elixir # ✗ SLOW: Many small evaluations results = Enum.map(data, fn item -> {result, _} = Pythonx.eval("process(x)", %{"x" => item}) Pythonx.decode(result) end) # ✓ FAST: Bulk operation in single evaluation {result, _} = Pythonx.eval(""" results = [process(x) for x in data] """, %{"data" => data}) results = Pythonx.decode(result) ``` -------------------------------- ### Benchmark Elixir eval Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/integration-guide.md Benchmarks the time taken to evaluate 1000 simple arithmetic expressions using Pythonx.eval. This snippet is useful for understanding the overhead of basic evaluations. ```elixir defmodule PythonxBench do def bench_eval do {time, _result} = :timer.tc(fn -> Enum.each(1..1000, fn _ -> Pythonx.eval("1 + 1", %{}) end) end) IO.puts("1000 evaluations took #{time / 1000}ms (#{time / 1000 / 1000}ms per eval)") end def bench_numpy do {time, _result} = :timer.tc(fn -> Enum.each(1..100, fn _ -> Pythonx.eval(""" import numpy result = numpy.array([1,2,3,4,5]) result @ result """, %{}) end) end) IO.puts("100 numpy operations took #{time / 1000}ms") end end ``` -------------------------------- ### Defining Optional Dependencies in pyproject.toml Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/configuration.md Specify optional dependency groups, such as 'dev', for development purposes. ```toml [project] optional-dependencies = { dev = [ "pytest==8.0.0", "black==24.1.0", ] } ``` -------------------------------- ### Utilize Pre-computed Python Objects in Parallel Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/integration-guide.md A good practice for concurrency is to compute Python objects once (e.g., using NumPy arrays) and then use these pre-computed objects in multiple Elixir processes. This leverages CPU-intensive operations that release the GIL. ```elixir # ✓ GOOD: Sequential processing, then parallel operations {numpy_obj, _} = Pythonx.eval("import numpy; numpy.array([1,2,3])", %{}) # Now use numpy_obj in multiple processes (it's already computed) ``` -------------------------------- ### Inefficient Bulk Operation: Many Small Evals Source: https://github.com/livebook-dev/pythonx/blob/main/_autodocs/examples.md Avoid this pattern for performance. It results in thousands of NIF calls for a simple operation. ```elixir # ✗ SLOW - Thousands of NIF calls results = Enum.map(1..1000, fn i -> {result, _} = Pythonx.eval("x * 2", %{"x" => i}) Pythonx.decode(result) end) ```