Marlin

https://docs.marlin.org/
Admin
Marlin is a verifiable computing protocol leveraging TEEs to deploy complex workloads like DeFi...
Tokens:201,335
Snippets:585
Trust Score:8.7
Update:1 year ago
Show doc for...
Tokens:
Raw
### C++ Project Setup for Marlin Multicast SDK

Source: https://docs.marlin.org/user-guides/multicast-sdk

This section covers the initial setup of a C++ project using CMake and Make, which is a prerequisite for integrating the Marlin Multicast SDK. Ensure you have a C++17 compliant compiler, CMake (v3.13+), and Make installed.

```bash
# Example CMakeLists.txt setup (conceptual)
cmake_minimum_required(VERSION 3.13)
project(MarlinProject)

# Add Marlin SDK include directories and libraries here
# target_include_directories(your_target PRIVATE ${MARLIN_SDK_INCLUDE_DIRS})
# target_link_libraries(your_target PRIVATE ${MARLIN_SDK_LIBRARIES})

add_executable(marlin_app main.cpp)
```

--------------------------------

### Install oyster-cvm Tool

Source: https://docs.marlin.org/oyster/build-cvm/quickstart

Installs the 'oyster-cvm' command-line tool for Linux (amd64, arm64) and macOS (arm64). This tool is essential for deploying and managing CVMs on the Oyster network.

```bash
sudowget https://artifacts.marlin.org/oyster/binaries/oyster-cvm_latest_linux_amd64 -O /usr/local/bin/oyster-cvm &&sudochmod +x /usr/local/bin/oyster-cvm
```

```bash
sudowget https://artifacts.marlin.org/oyster/binaries/oyster-cvm_latest_linux_arm64 -O /usr/local/bin/oyster-cvm &&sudochmod +x /usr/local/bin/oyster-cvm
```

```bash
sudowget https://artifacts.marlin.org/oyster/binaries/oyster-cvm_latest_darwin_arm64 -O /usr/local/bin/oyster-cvm &&sudochmod +x /usr/local/bin/oyster-cvm
```

--------------------------------

### Oyster Serverless Web2 API Quickstart

Source: https://docs.marlin.org/oyster/build-serverless

This guide provides a step-by-step walkthrough for interacting with Oyster Serverless using Web2 APIs. It's designed for developers whose dApps have off-chain components requiring secure computation.

```English
Follow this quickstart guide for a step-by-step walkthrough.
This guide simplifies the process, allowing you to quickly and efficiently get started with Web2 API-based interactions.
```

--------------------------------

### Clone Oyster Setup Repository

Source: https://docs.marlin.org/oyster/join/cvm/aws

Clones the oyster-setup-aws repository from GitHub and navigates into the cloned directory. This is the initial step to get the necessary files for AWS setup.

```bash
git clone https://github.com/marlinprotocol/oyster-setup-aws oyster-setup-aws &&cd oyster-setup-aws
```

--------------------------------

### Start Script for Eliza Agent

Source: https://docs.marlin.org/oyster/build-cvm/guides/eliza/build-docker

A bash script to set up the environment, install dependencies, build the project, and start the Eliza agent and its client. It includes commands for setting up Pulse Server, system updates, package installations, and running the application with specified characters.

```bash
#!/bin/bash

# Start the Pulse Server for inserting secrets
./pulse-server --listen-addr 0.0.0.0:1600

npminstall-g pnpm@9.4.0 &&\
apt-get update &&\
apt-get upgrade -y&&\
apt-getinstall-y\
git\
        python3 \
        python3-pip \
curl\
        node-gyp \
        ffmpeg \
        libtool-bin \
        autoconf \
        automake \
        libopus-dev \
make\
        g++ \
        build-essential \
        libcairo2-dev \
        libjpeg-dev \
        libpango1.0-dev \
        libgif-dev \
        openssl \
        libssl-dev &&\
apt-get clean &&\
rm-rf /var/lib/apt/lists/*

# Install dependencies
pnpminstall --no-frozen-lockfile

# Build the project
pnpm run build

# Start the application
# Example without a character: 
# pnpm start
npx concurrently "pnpm start --character=\"characters/trump.character.json\""
"pnpm start:client --host"
```

--------------------------------

### Verify Hardhat Installation

Source: https://docs.marlin.org/oyster/build-serverless/tutorials/subscription

This command verifies that Hardhat, a development environment for Ethereum, is correctly installed and configured. It's a crucial step to ensure your smart contract development setup is ready.

```Shell
npx hardhat
```

--------------------------------

### Define Applications with Docker Compose

Source: https://docs.marlin.org/oyster/build-cvm/quickstart

Creates a 'docker-compose.yml' file to define the applications that will run within the CVM. This example uses a simple echo-server image provided by Marlin.

```shell
cat> docker-compose.yml <<EOF
services:
  echo-server:
    image: marlinorg/oyster-cvm-tutorials-echo-server:latest
    init: true
    network_mode: host
    restart: unless-stopped
EOF
```

--------------------------------

### Setup Marlin Beacon Node

Source: https://docs.marlin.org/run-your-own-node/relay/instructions

Commands to set up a Marlin beacon node, either using the `marlinctl` utility or manually. This includes creating a keystore and starting the beacon with specified discovery, heartbeat, and bootstrap addresses.

```bash
sudo marlinctl beacon keystore create
```

```bash
sudo marlinctl beacon create --discovery-addr "0.0.0.0:8002" --heartbeat-addr "0.0.0.0:8003" --bootstrap-addr "34.82.79.68:8003"
```

```bash
./beacon/beacon --discovery-addr "0.0.0.0:8002" --heartbeat-addr "0.0.0.0:8003" --beacon-addr "34.82.79.68:8003"
```

--------------------------------

### Install NPM Dependencies

Source: https://docs.marlin.org/oyster/build-serverless/tutorials/subscription

This command installs the necessary Node Package Manager (npm) dependencies for the Oyster Serverless development tools. Ensure Node.js is installed before running this command.

```Shell
npminstall
```

--------------------------------

### Create Project Directory

Source: https://docs.marlin.org/user-guides/multicast-sdk/part1

Commands to create a new directory for the project and navigate into it. This sets up the initial workspace for the tutorial.

```Shell
$ mkdir tutorial
$ cd tutorial
```

--------------------------------

### Install Node Dependencies

Source: https://docs.marlin.org/oyster/join/cvm/aws

Installs the necessary Node.js dependencies for the project using npm. This command should be run after cloning the repository and installing Node.js.

```bash
npm i
```

--------------------------------

### Setting up Oyster Serverless Development Environment

Source: https://docs.marlin.org/oyster/build-serverless/tutorials

Configure your local machine for Oyster Serverless development. This tutorial covers the necessary tools and environment setup to begin building and deploying serverless functions.

```bash
# Example command for setting up development environment (conceptual)
# apt-get update && apt-get install -y nodejs npm docker.io
# npm install -g @oyster/cli
# oyster login
```

--------------------------------

### Complete CMakeLists.txt

Source: https://docs.marlin.org/user-guides/multicast-sdk/part1

The complete CMakeLists.txt file combining the minimum version, project details, and executable target definition for the C++ project.

```CMake
cmake_minimum_required(VERSION 3.13 FATAL_ERROR)
project(tutorial VERSION 0.0.1 LANGUAGES CXX)

add_executable(tutorial \
	main.cpp
)
```

--------------------------------

### Install Pulumi

Source: https://docs.marlin.org/oyster/join/cvm/aws

Installs Pulumi on Ubuntu/Debian-based systems by downloading and executing the official installation script. It also adds the Pulumi binary directory to the system's PATH.

```bash
curl -fsSL https://get.pulumi.com |sh&&export PATH=$PATH:$HOME/.pulumi/bin
```

--------------------------------

### Serverless Subscriptions Example

Source: https://docs.marlin.org/oyster/build-serverless/quickstart/subscriptions

This example demonstrates how to set up and manage serverless subscriptions using Oyster. It covers the basic functionality for automating periodic serverless requests.

```N/A
This section provides instructions to start with an example for Serverless Subscriptions. The specific code implementation details are not provided in this text.
```

--------------------------------

### Install Node.js and npm

Source: https://docs.marlin.org/oyster/join/cvm/aws

Installs Node.js and npm on Ubuntu/Debian-based systems, which are required for running Pulumi. Instructions for other operating systems are available on the official Node.js website.

```bash
sudo apt install nodejs npm
```

--------------------------------

### Set up Oyster Serverless Dev Environment

Source: https://docs.marlin.org/oyster/build-serverless/tutorials/web3-request

This snippet covers cloning the Oyster Serverless development tools, installing dependencies, and verifying the Hardhat installation. It's essential for setting up your local environment to build and deploy serverless functions.

```Bash
git clone https://github.com/marlinprotocol/oyster-serverless-devtools.git
cd oyster-serverless-devtools/user_contract_builder

npminstall

npx hardhat
```

--------------------------------

### Start Bor (without mining)

Source: https://docs.marlin.org/user-guides/polygon-mev/for-validators/loki-testnet

Starts the Bor node in full sync mode without mining enabled. It connects to the specified Heimdall endpoint and bootnodes.

```bash
/path/to/bor --datadir /path/to/datadir --syncmode 'full' --networkid 1137 --bor.heimdall 'http://13.57.231.182:8000' --bootnodes "enode://edf8c774a442603c497989529940b92489b98d0382ba6e9a01f5b1587e09f1a9d40faff5396a77e39ffc89ae0a0b424c5e933379f4dc34da41dcc3618e190e20@13.229.200.159:30303"  
```

--------------------------------

### Start Bor (with mining)

Source: https://docs.marlin.org/user-guides/polygon-mev/for-validators/loki-testnet

Starts the Bor node with mining enabled, including configurations for gas limit and target. It also specifies the sealing key and password for unlocking.

```bash
/path/to/bor --datadir /path/to/datadir --syncmode 'full' --networkid 1137 --miner.gaslimit '20000000' --miner.gastarget '20000000' --mine --unlock 0xSEALINGKEY --password /path/to/pass.txt --allow-insecure-unlock --bor.heimdall 'http://13.57.231.182:8000' --bootnodes "enode://edf8c774a442603c497989529940b92489b98d0382ba6e9a01f5b1587e09f1a9d40faff5396a77e39ffc89ae0a0b424c5e933379f4dc34da41dcc3618e190e20@13.229.200.159:30303"  
```

--------------------------------

### CMake Add Executable Target

Source: https://docs.marlin.org/user-guides/multicast-sdk/part1

Adds an executable target named 'tutorial' to the CMake build system, linking it with the 'main.cpp' source file.

```CMake
add_executable(tutorial \
	main.cpp
)
```

--------------------------------

### Minimal C++ Program

Source: https://docs.marlin.org/user-guides/multicast-sdk/part1

A basic C++ program with a main function. This serves as the entry point for the executable and will be compiled.

```C++
int main() {
	return 0;
}
```

--------------------------------

### Install mev-bor for Polygon Validators

Source: https://docs.marlin.org/user-guides/polygon-mev

Guide for Polygon POS validators to install the mev-bor client to accept Flashbots bundles. This involves setting up a gateway and configuring the client.

```Bash
# Example command to install mev-bor (actual command may vary based on installation method)
sudo apt-get update
sudo apt-get install mev-bor
```

--------------------------------

### Run Polkadot Relay Beacon (Manual Setup)

Source: https://docs.marlin.org/run-your-own-node/relay/relay-nodes/polkadot

Manually starts the beacon node with specified addresses, keystore path, and passphrase path. This is an alternative to using marlinctl for setup.

```bash
./beacon --discovery-addr "0.0.0.0:8002" --heartbeat-addr "0.0.0.0:8003" --beacon-addr "54.193.224.20:8003" --keystore-path /path/to/keystore --keystore-pass-path /path/to/passphrase  
```

--------------------------------

### Run Executable

Source: https://docs.marlin.org/user-guides/multicast-sdk/part1

Executes the compiled 'tutorial' program located in the build directory. Currently, it performs no action but confirms the build process was successful.

```Shell
./tutorial
```

--------------------------------

### Run Polkadot Relay Node (Manual Setup)

Source: https://docs.marlin.org/run-your-own-node/relay/relay-nodes/polkadot

Manually starts a Polkadot relay node, connecting it to the beacon using discovery and heartbeat addresses. This is an alternative to using marlinctl for setup.

```bash
./dot_relay --discovery-addrs "<beacon_public_ip>:8002" --heartbeat-addrs "<beacon_public_ip>:8003"  
```

--------------------------------

### setup.sh for Enclave Initialization

Source: https://docs.marlin.org/oyster/build-cvm/old-quickstart/build

This shell script is designed to run as the entrypoint for the enclave. It configures the loopback interface and adds a default route, ensuring basic network connectivity within the enclave environment.

```Shell
#!/bin/sh

# setting an address for loopback
ifconfig lo 127.0.0.1
ifconfig

# adding a default route
ip route add default via 127.0.0.1 dev lo
route -n

```

--------------------------------

### Create 'Hello World' Serverless Function

Source: https://docs.marlin.org/oyster/build-serverless/tutorials/dev-env

Creates a simple 'Hello World' serverless function in JavaScript. This function listens for fetch events and responds with 'Hello World'.

```JavaScript
addEventListener("fetch",event=>{
  event.respondWith(newResponse("Hello World"));
});
```

--------------------------------

### Start Cluster Selection Script - npm

Source: https://docs.marlin.org/run-your-own-node/relay/clusterselector/script

This command executes the NodeJS script to perform cluster selection after dependencies are installed and the .env file is configured.

```bash
npm start
```

--------------------------------

### Download genesis file

Source: https://docs.marlin.org/user-guides/polygon-mev/for-validators/loki-testnet

Downloads the genesis file required for initializing the Bor layer of the loki-testnet. This is the first step in setting up the testnet.

```bash
wget http://public.artifacts.marlin.pro/projects/mev-bor/1.0.1/genesis.json  
```

--------------------------------

### Set Up Local Pulumi Login

Source: https://docs.marlin.org/oyster/join/cvm/aws

Configures Pulumi to use the local filesystem for storing stack files. This is a crucial step before deploying infrastructure with Pulumi.

```bash
pulumi login --local
```

--------------------------------

### CMake Project Details

Source: https://docs.marlin.org/user-guides/multicast-sdk/part1

Defines the project name, version, and the programming language (C++). This helps CMake configure the build environment correctly.

```CMake
project(tutorial VERSION 0.0.1 LANGUAGES CXX)
```

--------------------------------

### Create Polygon Gateway Client Identity (Manual Setup)

Source: https://docs.marlin.org/user-guides/gateways/polygon-pos

This command generates a client identity for the Polygon gateway. It requires the marlinctl tool to be installed and prompts the user for a passphrase.

```bash
sudo marlinctl gateway polygon keystore create  

```

--------------------------------

### Deploy CVM on Oyster

Source: https://docs.marlin.org/oyster/build-cvm/quickstart

Deploys the defined CVM applications on the Oyster network. Requires a wallet private key with specific token balances (1 USDC and 0.001 ETH on Arbitrum One) and specifies a duration for the deployment.

```bash
oyster-cvm deploy --wallet-private-key <key> --duration-in-minutes 15 --docker-compose docker-compose.yml
```

--------------------------------

### Test Serverless Function Locally

Source: https://docs.marlin.org/oyster/build-serverless/tutorials/dev-env

Starts a local test server using the mock_serverless.sh script and then sends a POST request to it using curl to test the 'Hello World' function.

```Shell
./mock_serverless.sh 8090 hello.js
curl-X POST http://localhost:8090
```

--------------------------------

### Build and Run Marlin Executable

Source: https://docs.marlin.org/user-guides/multicast-sdk/part3

These shell commands demonstrate how to build and run the Marlin executable. First, CMake is used to configure the build with the Release type, followed by the `make` command to compile the project. Finally, the executable is run using `./tutorial`.

```Shell
cmake ..-DCMAKE_BUILD_TYPE=Release
make
./tutorial
```

--------------------------------

### Deploy AWS VPC with Pulumi

Source: https://docs.marlin.org/oyster/join/cvm/aws

Deploys dedicated VPCs and other necessary AWS components for running Oyster as a provider. This command requires a pre-configured AWS profile and prompts for stack creation and confirmation.

```bash
AWS_PROFILE=<profile> pulumi up
```

--------------------------------

### Initialize Marlin Multicast Client (C++)

Source: https://docs.marlin.org/user-guides/multicast-sdk/part3

This C++ code demonstrates how to include the necessary headers and namespaces for the Marlin Multicast SDK. It sets up a delegate class to handle client events and initializes a multicast client with specific options, including a keypair and network details.

```C++
#include <marlin/multicast/DefaultMulticastClient.hpp>
  
using namespace marlin::multicast;
using namespace marlin::net;

class MulticastDelegate {
public:
	void did_recv_message(
		DefaultMulticastClient<MulticastDelegate> &client,
		Buffer &&message,
		Buffer &&witness,
		std::string &channel,
		uint64_t message_id
	) {}

	void did_subscribe(
		DefaultMulticastClient<MulticastDelegate> &client,
		std::string &channel
	) {
		SPDLOG_INFO("Did subscribe to channel {}", channel);
	}

	void did_unsubscribe(
		DefaultMulticastClient<MulticastDelegate> &client,
		std::string &channel
	) {
		SPDLOG_INFO("Did unsubscribe from channel {}", channel);
	}
};

	uint8_t static_sk1[crypto_box_SECRETKEYBYTES];
	uint8_t static_pk1[crypto_box_PUBLICKEYBYTES];
	crypto_box_keypair(static_pk1, static_sk1);

	MulticastDelegate del;
	DefaultMulticastClientOptions clop1 {
		static_sk1,  // Secret key
		{"goldfish"},  // Pubsub channels
		"127.0.0.1:9002"  // Beacon address from the goldfish node
	};

	DefaultMulticastClient<MulticastDelegate> cl1(clop1);
	cl1.delegate = &del;

```

--------------------------------

### Build Executable with Make

Source: https://docs.marlin.org/user-guides/multicast-sdk/part1

Uses the Make utility to compile the C++ code and link it into an executable file based on the generated Makefile.

```Shell
$ make
```

--------------------------------

### Download and prepare verifier binary

Source: https://docs.marlin.org/oyster/build-cvm/guides/nodejs-server/verify

Downloads the verifier binary from a specified URL and makes it executable. This is the first step in verifying the enclave's attestation.

```Shell
wget http://public.artifacts.marlin.pro/projects/enclaves/verifier &&chmod +x ./verifier  

```

--------------------------------

### Configure and Run Attestation Verifier Enclave Setup

Source: https://docs.marlin.org/oyster/build-cvm/examples/attestation-verifier

This shell script configures the network interfaces, sets up iptables rules for traffic routing, generates identity and secp256k1 keypairs, and finally starts supervisord to manage the enclave processes.

```Shell
#!/bin/sh  
  
# setting an address for loopback  
ifconfig lo 127.0.0.1  
ifconfig  
  
# adding a default route  
ip route add default dev lo src 127.0.0.1  
route -n  
  
# iptables rules to route traffic to transparent proxy  
iptables -A OUTPUT -t nat -p tcp --dport1:65535 !-d127.0.0.1  -j DNAT --to-destination 127.0.0.1:1200  
iptables -L-t nat  
  
# generate identity key  
/app/keygen-ed25519 --secret /app/id.sec --public /app/id.pub  
  
# your custom setup goes here  
/app/keygen-secp256k1 --secret /app/secp256k1.sec --public /app/secp256k1.pub  
  
# starting supervisord  
cat /etc/supervisord.conf  
/app/supervisord  
```

--------------------------------

### Install Rust Toolchain

Source: https://docs.marlin.org/oyster/build-cvm/tutorials/create

Installs the Rust toolchain using the official installation script. Verifies the installation with the 'cargo' command.

```Shell
curl--proto'=https'--tlsv1.2-sSf https://sh.rustup.rs |sh  
```

```Shell
cargo  
```

--------------------------------

### Start Serverless Job Subscription

Source: https://docs.marlin.org/oyster/build-serverless/quickstart/subscriptions

Initiates a serverless job subscription by sending a transaction to the `startJobSubscription` method of the sample contract. This involves depositing USDC tokens and configuring subscription parameters.

```Solidity
pragma solidity ^0.8.0;

interface ISubscriptionRelay {
    function startJobSubscription(
        address userContract,
        bytes32 codeHash,
        bytes calldata codeInputs,
        uint64 periodicGap,
        uint64 terminationTimestamp,
        uint64 userTimeout,
        address callbackAddress
    ) external payable;
}

contract MyUserContract {
    ISubscriptionRelay subscriptionRelay = ISubscriptionRelay(0x...); // Replace with actual contract address

    function subscribeToServerlessJob() public payable {
        // Example parameters
        bytes32 exampleCodeHash = 0x6516be2032b475da2a96df1eefeb1679a8032faa434f8311a1441e92f2058fe5;
        bytes memory exampleCodeInputs = hex"00";
        uint64 examplePeriodicGap = 30; // 30 seconds
        uint64 exampleTerminationTimestamp = block.timestamp + 200; // 200 seconds from now
        uint64 exampleUserTimeout = 2000; // 2000 milliseconds
        address exampleCallbackAddress = address(this);

        // Deposit USDC tokens (assuming you have a function to approve and transfer USDC)
        // IERC20(0xaf88d065e77c8cC2239327C5EDb3A432268e5831).transfer(address(subscriptionRelay), 1000000);

        // Call the subscription method
        subscriptionRelay.startJobSubscription(
            address(this),
            exampleCodeHash,
            exampleCodeInputs,
            examplePeriodicGap,
            exampleTerminationTimestamp,
            exampleUserTimeout,
            exampleCallbackAddress
        ) payable;
    }

    // Callback function to receive results
    function oysterResultCall(bytes calldata output) external {
        // Process the output from the serverless function
    }
}
```

--------------------------------

### Dockerfile for AMD64 Enclave Image

Source: https://docs.marlin.org/oyster/build-cvm/old-quickstart/build

This Dockerfile sets up an enclave image for the AMD64 architecture. It starts with an Alpine base image, installs networking and utility tools, downloads and configures essential Marlin components like supervisord, keygen, and attestation server, and sets up.sh as the entrypoint.

```Dockerfile
FROM alpine:3.17

# install dependency tools
RUN apk add --no-cache net-tools iptables iproute2 wget

# working directory
WORKDIR /app

# supervisord to manage programs
RUN wget-O supervisord http://public.artifacts.marlin.pro/projects/enclaves/supervisord_master_linux_amd64
RUN chmod +x supervisord

# transparent proxy component inside the enclave to enable outgoing connections
RUN wget-O ip-to-vsock-transparent http://public.artifacts.marlin.pro/projects/enclaves/ip-to-vsock-transparent_v1.0.0_linux_amd64
RUN chmod +x ip-to-vsock-transparent

# key generator to generate static keys
RUN wget-O keygen http://public.artifacts.marlin.pro/projects/enclaves/keygen_v1.0.0_linux_amd64
RUN chmod +x keygen

# attestation server inside the enclave that generates attestations
RUN wget-O attestation-server http://public.artifacts.marlin.pro/projects/enclaves/attestation-server_v1.0.0_linux_amd64
RUN chmod +x attestation-server

# proxy to expose attestation server outside the enclave
RUN wget-O vsock-to-ip http://public.artifacts.marlin.pro/projects/enclaves/vsock-to-ip_v1.0.0_linux_amd64
RUN chmod +x vsock-to-ip

# dnsproxy to provide DNS services inside the enclave
RUN wget-O dnsproxy http://public.artifacts.marlin.pro/projects/enclaves/dnsproxy_v0.46.5_linux_amd64
RUN chmod +x dnsproxy

# supervisord config
COPY supervisord.conf /etc/supervisord.conf

# setup.sh script that will act as entrypoint
COPY setup.sh ./ 
RUN chmod +x setup.sh

# your custom setup goes here

# entry point
ENTRYPOINT ["/app/setup.sh"]

```

--------------------------------

### Create and Navigate Directory (Shell)

Source: https://docs.marlin.org/oyster/build-cvm/guides/private-compute/server

This command creates a new directory named 'my-server' and then changes the current directory to 'my-server'. This is a standard way to start a new project in a Unix-like environment.

```Shell
mkdir my-server &&cd my-server
```
Tokens:
Raw