### Install L2F and Dependencies Source: https://github.com/rl-tools/raptor/blob/master/README.md Installs the necessary packages for L2F, ui-server, and the foundation policy. Ensure you have pip installed. ```bash pip install l2f==2.0.18 ui-server==0.0.13 foundation-policy==1.0.1 ``` -------------------------------- ### Install Foundation Policy Source: https://github.com/rl-tools/raptor/blob/master/README.md This snippet shows how to install the foundation policy package using pip. It specifies the package name and version, making it easy to integrate into a project. ```bash pip install foundation-policy==1.0.1 ``` -------------------------------- ### Run UI Server Source: https://github.com/rl-tools/raptor/blob/master/README.md Starts the ui-server, which provides a web interface for visualizing the simulation. Access it at http://localhost:13337. ```bash ui-server ``` -------------------------------- ### Configure and Build Raptor Project with MKL Backend Source: https://github.com/rl-tools/raptor/blob/master/README.md Configures the raptor project using CMake with the MKL backend enabled and then builds specific targets. This step prepares the project for execution on systems with Intel MKL installed. ```bash mkdir build MKL_ROOT=/opt/intel/oneapi/mkl/latest cmake -S /rl-tools -B /build -DCMAKE_BUILD_TYPE=Release -DRL_TOOLS_BACKEND_ENABLE_MKL=ON -DRL_TOOLS_ENABLE_TARGETS=ON -DRL_TOOLS_EXPERIMENTAL=ON -DRL_TOOLS_ENABLE_HDF5=ON -DRL_TOOLS_ENABLE_JSON=ON -DRL_TOOLS_ENABLE_TENSORBOARD=ON cmake --build /build --target foundation_policy_pre_training_sample_dynamics_parameters --target foundation_policy_pre_training --target foundation_policy_post_training -j$(nproc) ``` -------------------------------- ### Raptor Policy Usage in Python Source: https://github.com/rl-tools/raptor/blob/master/README.md This Python code demonstrates how to use the Raptor policy for quadrotor control. It initializes the policy, resets it, and then iterates through simulation steps, evaluating observations to get actions for the simulation. It highlights the expected observation format and how to apply the resulting action. ```python from foundation_policy import Raptor policy = Raptor() policy.reset() for simulation_step in range(1000): observation = np.array([[*sim.position, *R(sim.orientation).flatten(), *sim.linear_velocity, *sim.angular_velocity, *sim.action]]) action = policy.evaluate_step(observation)[0] # the policy works on batches by default simulation.step(action) # simulation dt=10 ms ``` -------------------------------- ### Clone and Initialize Raptor Project and Submodules Source: https://github.com/rl-tools/raptor/blob/master/README.md Clones the raptor repository and initializes its git submodules, including external libraries required for the project. This ensures all necessary components are downloaded and ready for use. ```bash git clone https://github.com/rl-tools/raptor.git cd raptor git submodule update --init rl-tools cd rl-tools git submodule update --init --recursive -- external/highfive external/json external/tensorboard cd .. ``` -------------------------------- ### L2F Simulation with Foundation Policy in Python Source: https://github.com/rl-tools/raptor/blob/master/README.md This Python script demonstrates a full simulation loop using L2F and a foundation policy. It connects to the ui-server, initializes the environment, and runs the policy for a set number of steps, rendering the state at each step. It requires numpy, asyncio, websockets, json, l2f, and foundation_policy. ```python from copy import copy import numpy as np import asyncio, websockets, json import l2f from l2f import vector8 as vector from foundation_policy import Raptor policy = Raptor() device = l2f.Device() rng = vector.VectorRng() env = vector.VectorEnvironment() ui = l2f.UI() params = vector.VectorParameters() state = vector.VectorState() observation = np.zeros((env.N_ENVIRONMENTS, env.OBSERVATION_DIM), dtype=np.float32) next_state = vector.VectorState() vector.initialize_rng(device, rng, 0) vector.initialize_environment(device, env) vector.sample_initial_parameters(device, env, params, rng) vector.sample_initial_state(device, env, params, state, rng) def configure_3d_model(parameters_message): parameters_message = json.loads(parameters_message) for d in parameters_message["data"]: d["ui"] = { "model": "95d22881d444145176db6027d44ebd3a15e9699a", "name": "x500" } return json.dumps(parameters_message) async def render(websocket, state, action): ui_state = copy(state) for i, s in enumerate(ui_state.states): s.position[0] += i * 0.1 # Spacing for visualization state_action_message = vector.set_state_action_message(device, env, params, ui, ui_state, action) await websocket.send(state_action_message) async def main(): uri = "ws://localhost:13337/backend" # connection to the UI server async with websockets.connect(uri) as websocket: handshake = json.loads(await websocket.recv(uri)) assert(handshake["channel"] == "handshake") namespace = handshake["data"]["namespace"] ui.ns = namespace ui_message = vector.set_ui_message(device, env, ui) parameters_message = vector.set_parameters_message(device, env, params, ui) # parameters_message = configure_3d_model(parameters_message) # use this for a more realistic 3d model await websocket.send(ui_message) await websocket.send(parameters_message) await asyncio.sleep(1) await render(websocket, state, np.zeros((8, 4))) await asyncio.sleep(2) policy.reset() for _ in range(500): vector.observe(device, env, params, state, observation, rng) action = policy.evaluate_step(observation[:, :22]) dts = vector.step(device, env, params, state, action, next_state, rng) state.assign(next_state) await render(websocket, state, action) await asyncio.sleep(dts[-1]) if __name__ == "__main__": asyncio.run(main()) ``` -------------------------------- ### Configure and Build Raptor Project with Accelerate Backend (macOS) Source: https://github.com/rl-tools/raptor/blob/master/README.md Configures the raptor project using CMake with the Accelerate framework backend enabled for macOS, and then builds specific targets. This is an alternative to MKL for Apple systems. ```bash cd rl-tools mkdir build cd build cmake .. -DCMAKE_BUILD_TYPE=Release -DRL_TOOLS_BACKEND_ENABLE_ACCELERATE=ON -DRL_TOOLS_ENABLE_TARGETS=ON -DRL_TOOLS_EXPERIMENTAL=ON -DRL_TOOLS_ENABLE_HDF5=ON -DRL_TOOLS_ENABLE_JSON=ON -DRL_TOOLS_ENABLE_TENSORBOARD=ON cmake --build . --target foundation_policy_pre_training_sample_dynamics_parameters --target foundation_policy_pre_training --target foundation_policy_post_training cd .. ./build/src/foundation_policy/foundation_policy_post_training ``` -------------------------------- ### C++ RL Tools Adapter for Flight Controllers Source: https://github.com/rl-tools/raptor/blob/master/README.md This C++ code snippet demonstrates the adapter implementation for integrating rl-tools policies into flight controller firmware. It includes necessary headers for operations, neural network layers, and inference executors. The configuration struct `RL_TOOLS_INFERENCE_APPLICATIONS_L2F_CONFIG` sets up device specifications, policy types, and timing parameters. Ensure the control interval is configured correctly for training and inference frequencies. ```c++ #include #include #include #include #include #include #include #include #include "blob/policy.h" namespace rlt = rl_tools; namespace other{ using DEV_SPEC = rlt::devices::DefaultARMSpecification; using DEVICE = rlt::devices::arm::OPT; } struct RL_TOOLS_INFERENCE_APPLICATIONS_L2F_CONFIG{ using DEVICE = other::DEVICE; using TI = typename other::DEVICE::index_t; using RNG = other::DEVICE::SPEC::RANDOM::ENGINE<>; static constexpr TI TEST_SEQUENCE_LENGTH_ACTUAL = 5; static constexpr TI TEST_BATCH_SIZE_ACTUAL = 2; using ACTOR_TYPE_ORIGINAL = rlt::checkpoint::actor::TYPE; using POLICY_TEST = rlt::checkpoint::actor::TYPE::template CHANGE_BATCH_SIZE::template CHANGE_SEQUENCE_LENGTH; using POLICY = ACTOR_TYPE_ORIGINAL::template CHANGE_BATCH_SIZE; using T = typename POLICY::SPEC::T; static auto& policy() { return rlt::checkpoint::actor::module; } static constexpr TI ACTION_HISTORY_LENGTH = 1; static constexpr TI CONTROL_INTERVAL_INTERMEDIATE_NS = 2.5 * 1000 * 1000; // Inference is at 500hz static constexpr TI CONTROL_INTERVAL_NATIVE_NS = 10 * 1000 * 1000; // Training is 100hz static constexpr TI TIMING_STATS_NUM_STEPS = 100; static constexpr bool FORCE_SYNC_INTERMEDIATE = true; static constexpr TI FORCE_SYNC_NATIVE = 4; static constexpr bool DYNAMIC_ALLOCATION = false; using WARNING_LEVELS = rlt::inference::executor::WarningLevelsDefault; }; // #define RL_TOOLS_DISABLE_TEST #include ``` -------------------------------- ### Execute Raptor Foundation Policy Training Source: https://github.com/rl-tools/raptor/blob/master/README.md Executes the pre-training and post-training steps for the raptor foundation policy. This involves sampling dynamics parameters and then running the pre-training and post-training jobs, outputting experiment information. ```bash cd /rl-tools export RL_TOOLS_EXTRACK_EXPERIMENT=$(date '+%Y-%m-%d_%H-%M-%S') echo "Experiment: $RL_TOOLS_EXTRACK_EXPERIMENT" /build/src/foundation_policy/foundation_policy_pre_training_sample_dynamics_parameters seq 0 999 | xargs -I {} /build/src/foundation_policy/foundation_policy_pre_training ./src/foundation_policy/dynamics_parameters/{}.json /build/src/foundation_policy/foundation_policy_post_training ``` -------------------------------- ### Build Docker Image for Raptor Project Source: https://github.com/rl-tools/raptor/blob/master/README.md Builds the Docker image for the raptor project using the provided script. This image encapsulates the project's environment and dependencies, facilitating reproducible builds. ```bash cd rl-tools/docker ./build_all.sh cd ../../ ``` -------------------------------- ### Extract Checkpoints for Foundation Policy Post-Training Source: https://github.com/rl-tools/raptor/blob/master/README.md Extracts checkpoints generated during the pre-training phase for use in the post-training phase. This script creates a text file listing the checkpoints. ```bash cd rl-tools/src/foundation_policy ./extract_checkpoints.sh > checkpoints_$RL_TOOLS_EXTRACK_EXPERIMENT.txt ``` -------------------------------- ### Run Raptor Project within Docker Container Source: https://github.com/rl-tools/raptor/blob/master/README.md Launches a Docker container for the raptor project, mounting local directories for code and data persistence. This provides a controlled environment for running project commands. ```bash docker run -it --rm -v $(pwd)/rl-tools:/rl-tools -v data:/data rltools/rltools:ubuntu24.04_mkl_gcc_base ``` -------------------------------- ### Extract Foundation Policy Training Data Source: https://github.com/rl-tools/raptor/blob/master/README.md Concatenates and extracts the foundation policy training data archive. This step is crucial for preparing the dataset required for the training process. ```bash cat data/foundation-policy-v1-data.tar.gz.part_* > data/foundation-policy-v1-data.tar.gz cd rl-tools tar -xvf ../data/foundation-policy-v1-data.tar.gz cd .. ``` -------------------------------- ### C++ RL Tools Inference Control Function Call Source: https://github.com/rl-tools/raptor/blob/master/README.md This snippet shows how to call the `rl_tools_inference_applications_l2f_control` function for inference. This function should be invoked at the interval specified by `CONTROL_INTERVAL_INTERMEDIATE_NS`. It takes the current time, an observation pointer, and an action pointer as arguments, returning an executor status. ```c++ auto executor_status = rl_tools_inference_applications_l2f_control(current_time * 1000, &observation, &action); ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.