SQD

https://docs.sqd.dev/en/
Admin
SQD is a decentralized data lake and query engine that enables developers to stream, query, and...
Tokens:2,417,176
Snippets:34,279
Trust Score:8.3
Update:2 months ago
Show doc for...
Tokens:
Raw
### Install and run the project

Source: https://docs.sqd.dev/en/sdk/alternative-clients/ponder/quickstart

Commands to install dependencies and start the development server.

```bash
npm install
npm run dev
```

--------------------------------

### Run the Uniswap portal example

Source: https://docs.sqd.dev/en/sdk/alternative-clients/ponder/examples

Commands to navigate to the Uniswap portal example directory and start the development server.

```bash
cd examples/uniswap-portal
npm install
npm run dev
```

--------------------------------

### Run the Euler multichain example

Source: https://docs.sqd.dev/en/sdk/alternative-clients/ponder/examples

Commands to navigate to the Euler multichain example directory and start the development server.

```bash
cd examples/euler-multichain
npm install
npm run dev
```

--------------------------------

### Clone and Install Dependencies

Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana/examples-tutorials/indexing-orca

Initial setup commands to retrieve the template project and install required Node.js dependencies.

```bash
git clone https://github.com/subsquid-labs/solana-example
cd solana-example
npm i
```

--------------------------------

### Run the Euler subgraph port example

Source: https://docs.sqd.dev/en/sdk/alternative-clients/ponder/examples

Commands to navigate to the Euler subgraph example directory and start the development server.

```bash
cd examples/euler-subgraph
npm install
npm run dev
```

--------------------------------

### Clone and install dependencies for SQD Firehose

Source: https://docs.sqd.dev/en/sdk/alternative-clients/subgraphs-support#content-area

Initial setup steps to prepare the graph-node-setup environment.

```bash
git clone https://github.com/subsquid-labs/graph-node-setup
cd graph-node-setup
npm ci
```

--------------------------------

### Build and Run Commands

Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana/quickstart#content-area

Commands to install dependencies, build the project, manage the database, and start the services.

```bash
npm i
npm run build
```

```bash
docker compose up -d
```

```bash
npx squid-typeorm-migration apply
```

```bash
node -r dotenv/config lib/main.js
```

```bash
npx squid-graphql-server
```

--------------------------------

### Clone the Ponder Portal repository

Source: https://docs.sqd.dev/en/sdk/alternative-clients/ponder/examples

Initial setup command to clone the repository containing all bounded examples.

```bash
git clone https://github.com/subsquid-labs/portal-ponder.git
cd portal-ponder
```

--------------------------------

### Clone and install project dependencies

Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron/examples-tutorials/indexing-usdt-transfers#download-the-project

Commands to clone the example repository and update the processor dependency to the latest version.

```bash
git clone https://github.com/subsquid-labs/tron-example
cd tron-example
npm install
npm install @subsquid/tron-processor@latest
```

--------------------------------

### Execute Setup Script

Source: https://docs.sqd.dev/en/network/worker#clone-the-worker-repository

Run the script by providing the paths for data storage and the key file.

```bash
./setup_worker.sh <DATA_DIR_PATH> <KEY_PATH>
```

--------------------------------

### Set Up Database

Source: https://docs.sqd.dev/en/sdk/squid-sdk/evm/examples-tutorials/build-indexer-from-scratch

Configuration files and commands to initialize and start the PostgreSQL database.

```text
DB_NAME=squid
DB_PORT=23798
```

```yaml
services:
  db:
    image: postgres:15
    environment:
      POSTGRES_DB: "${DB_NAME}"
      POSTGRES_PASSWORD: postgres
    ports:
      - "${DB_PORT}:5432"
```

```bash
docker compose up -d
```

```bash
npx tsc
```

```bash
npx squid-typeorm-migration generate
npx squid-typeorm-migration apply
```

--------------------------------

### Initialize a Substrate project from a template

Source: https://docs.sqd.dev/en/sdk/squid-sdk/substrate/guides/make-an-indexer

Use the sqd CLI to bootstrap a new project based on specific chain types.

```bash
sqd init my-squid-name -t substrate
```

```bash
sqd init my-squid-name -t ink
```

```bash
sqd init my-squid-name -t frontier-evm
```

--------------------------------

### GET /datasets/hyperliquid-replica-cmds/metadata

Source: https://docs.sqd.dev/en/api/hyperliquid-replica-cmds/metadata#response-start-block

Retrieves the metadata for the hyperliquid-replica-cmds dataset, including its start block and real-time status.

```APIDOC
## GET /datasets/hyperliquid-replica-cmds/metadata

### Description
Retrieves the metadata for the hyperliquid-replica-cmds dataset, including its start block and real-time status.

### Method
GET

### Endpoint
https://portal.sqd.dev/datasets/hyperliquid-replica-cmds/metadata

### Response
#### Success Response (200)
- **dataset** (string) - The default name used to reference this dataset.
- **aliases** (string[]) - Alternative names for the dataset.
- **real_time** (boolean) - Indicates if the dataset has real-time data.
- **start_block** (integer) - The block number of the first known block.

#### Response Example
{
  "dataset": "<string>",
  "aliases": [
    "<string>"
  ],
  "real_time": true,
  "start_block": 123
}

#### Error Response (404)
- **error** (object) - Error details when the dataset is not found.

#### Error Example
{
  "error": {
    "type": "invalid_request_error",
    "code": "unknown_dataset",
    "message": "Unknown dataset: no-such-chain"
  }
}
```

--------------------------------

### Fetch Dataset Metadata

Source: https://docs.sqd.dev/en/api/evm/metadata#response-aliases

Examples of how to perform a GET request to retrieve metadata for a specific dataset.

```bash
curl --compressed 'https://portal.sqd.dev/datasets/ethereum-mainnet/metadata'
```

```python
import requests

response = requests.get(
    "https://portal.sqd.dev/datasets/ethereum-mainnet/metadata"
)
metadata = response.json()
```

```javascript
const response = await fetch("https://portal.sqd.dev/datasets/ethereum-mainnet/metadata");
const metadata = await response.json();
```

```php
<?php

$curl = curl_init();

curl_setopt_array($curl, [
  CURLOPT_URL => "https://portal.sqd.dev/datasets/ethereum-mainnet/metadata",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "GET",
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}
```

```go
package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://portal.sqd.dev/datasets/ethereum-mainnet/metadata"

	req, _ := http.NewRequest("GET", url, nil)

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(string(body))

}
```

```java
HttpResponse<String> response = Unirest.get("https://portal.sqd.dev/datasets/ethereum-mainnet/metadata")
  .asString();
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://portal.sqd.dev/datasets/ethereum-mainnet/metadata")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)

response = http.request(request)
puts response.read_body
```

--------------------------------

### Get Dataset Metadata

Source: https://docs.sqd.dev/en/api/catalog/hyperliquid/openapi-replica-cmds.yaml

Retrieves dataset information such as name, aliases, start block, and status.

```shell
curl --compressed 'https://portal.sqd.dev/datasets/hyperliquid-replica-cmds/metadata'
```

```python
import requests

response = requests.get(
    "https://portal.sqd.dev/datasets/hyperliquid-replica-cmds/metadata"
)
data = response.json()
```

```javascript
const response = await fetch("https://portal.sqd.dev/datasets/hyperliquid-replica-cmds/metadata");
const data = await response.json();
```

--------------------------------

### Start and verify service

Source: https://docs.sqd.dev/en/portal/self-hosting#verify-routing

Launches the service using the generated key and checks logs for successful startup.

```bash
KEY_PATH=./portal.key docker compose up -d
docker compose logs -f portal
```

--------------------------------

### Initialize project with JSON configuration

Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/quickstart#content-area

Use a JSON configuration string to automate the project setup process without manual prompts.

```bash
pnpx @subsquid/pipes-cli@beta init --config '{
  "projectFolder": "usdc-example",
  "packageManager": "pnpm",
  "target": "postgresql",
  "networkType": "evm",
  "defaultNetwork": "ethereum-mainnet",
  "templates": [
    {
      "templateId": "erc20Transfers",
      "params": {
        "deployments": [
          {
            "address": "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
            "range": { "from": "latest" }
          }
        ]
      }
    }
  ]
}'
```

--------------------------------

### Initialize a project from a GitHub repository

Source: https://docs.sqd.dev/en/sdk/squid-sdk/evm/reference/create-squid

Scaffold a project using a custom GitHub repository URL.

```bash
sqd init my-squid -t https://github.com/subsquid-labs/solana-example
```

--------------------------------

### Initialize Project and Dependencies

Source: https://docs.sqd.dev/en/sdk/squid-sdk/evm/examples-tutorials/build-indexer-from-scratch

Commands to initialize the project, create a gitignore, and install necessary production and development dependencies.

```bash
npm init
```

```text
node_modules
lib
```

```bash
npm i dotenv typeorm @subsquid/evm-stream @subsquid/batch-processor @subsquid/evm-objects @subsquid/typeorm-store @subsquid/typeorm-migration @subsquid/graphql-server @subsquid/evm-abi @subsquid/evm-codec
```

```bash
npm i typescript @subsquid/typeorm-codegen @subsquid/evm-typegen --save-dev
```

--------------------------------

### Get Latest Ethereum Block

Source: https://docs.sqd.dev/en/api/catalog/evm/openapi.yaml

Examples for retrieving the latest block information using shell and Python.

```shell
curl --compressed 'https://portal.sqd.dev/datasets/ethereum-mainnet/head'
```

```python
import requests
```

--------------------------------

### Configure environment and start services

Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron/quickstart#what-you’ll-get

Commands to set the API key, manage the database, and run the processor and GraphQL server.

```bash
SQD_API_KEY=<YOUR_SQD_NETWORK_API_KEY>
```

```bash
docker compose up -d
```

```bash
npx squid-typeorm-migration apply
```

```bash
node -r dotenv/config lib/main.js
```

```bash
npx squid-graphql-server
```

--------------------------------

### Initialize solanaPortalStream with configuration

Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/basic-components/source

Example showing the setup of a portal stream with instruction decoding and local SQLite caching.

```typescript
import { solanaInstructionDecoder, solanaPortalStream } from "@subsquid/pipes/solana";
import { portalSqliteCache } from "@subsquid/pipes/portal-cache/node";
import * as orcaWhirlpool from "./abi/orca_whirlpool/index.js";

const stream = solanaPortalStream({
  id: "orca-swaps",
  portal: "https://portal.sqd.dev/datasets/solana-mainnet",
  outputs: solanaInstructionDecoder({
    range: { from: 200000000 },
    programId: orcaWhirlpool.programId,
    instructions: { swap: orcaWhirlpool.instructions.swap },
  }),
  cache: portalSqliteCache({ path: "./cache.sqlite" }),
});
```

--------------------------------

### GET /{firstBlock}/worker

Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron/reference/network-api#content-area

Retrieves a suitable worker URL capable of processing requests starting from the specified block.

```APIDOC
## GET /{firstBlock}/worker

### Description
Returns a worker URL capable of processing POST requests starting from the provided firstBlock.

### Method
GET

### Endpoint
/{firstBlock}/worker

### Parameters
#### Path Parameters
- **firstBlock** (number) - Required - The block number to start processing from.
```

--------------------------------

### Interactive Query Interface Example

Source: https://docs.sqd.dev/en/portal/evm/examples/contract-deployments

A cURL example demonstrating how to track both 'create' and 'create2' contract deployment types.

```bash
curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
 -H 'Content-Type: application/json' \
 -d '{
 "type": "evm",
 "fromBlock": 20000000,
 "toBlock": 20000003,
 "fields": {
 "block": {
 "number": true,
 "timestamp": true
 },
 "trace": {
 "type": true,
 "createFrom": true,
 "createResultAddress": true
 }
 },
 "traces": [
 {
 "type": [
 "create",
 "create2"
 ]
 }
 ]
}'
```

--------------------------------

### GET /{firstBlock}/worker

Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron/reference/network-api#block-fields

Retrieves a suitable worker URL capable of processing requests starting from the specified block.

```APIDOC
## GET /{firstBlock}/worker

### Description
Returns a worker URL capable of processing POST requests starting from the provided block number.

### Method
GET

### Endpoint
/{firstBlock}/worker

### Parameters
#### Path Parameters
- **firstBlock** (number) - Required - The starting block number for the requested worker.
```

--------------------------------

### Download and Prepare Setup Script

Source: https://docs.sqd.dev/en/network/worker#clone-the-worker-repository

Commands to download the worker setup script and grant it execution permissions.

```bash
curl -fO https://cdn.subsquid.io/worker/setup_worker.sh
```

```bash
chmod +x ./setup_worker.sh
```

--------------------------------

### Fetch Solana Dataset Metadata

Source: https://docs.sqd.dev/en/api/solana/metadata

Examples of how to perform a GET request to retrieve metadata for the Solana mainnet dataset.

```bash
curl --compressed 'https://portal.sqd.dev/datasets/solana-mainnet/metadata'
```

```python
import requests

response = requests.get(
    "https://portal.sqd.dev/datasets/solana-mainnet/metadata"
)
data = response.json()
```

```javascript
const response = await fetch("https://portal.sqd.dev/datasets/solana-mainnet/metadata");
const data = await response.json();
```

```php
<?php

$curl = curl_init();

curl_setopt_array($curl, [
  CURLOPT_URL => "https://portal.sqd.dev/datasets/solana-mainnet/metadata",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => "",
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => "GET",
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}
```

```go
package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://portal.sqd.dev/datasets/solana-mainnet/metadata"

	req, _ := http.NewRequest("GET", url, nil)

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(string(body))

}
```

```java
HttpResponse<String> response = Unirest.get("https://portal.sqd.dev/datasets/solana-mainnet/metadata")
  .asString();
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://portal.sqd.dev/datasets/solana-mainnet/metadata")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)

response = http.request(request)
puts response.read_body
```

--------------------------------

### Build Project

Source: https://docs.sqd.dev/en/sdk/squid-sdk/substrate/quickstart#content-area

Install dependencies and build the project.

```bash
npm i
npm install @subsquid/substrate-processor@latest
npm install --save-dev @subsquid/substrate-typegen@latest
npm run build
```

--------------------------------

### GET /datasets/hyperliquid-replica-cmds/metadata

Source: https://docs.sqd.dev/en/api/hyperliquid-replica-cmds/metadata#response-real-time

Retrieves the metadata for the Hyperliquid replica dataset, including its aliases, real-time status, and starting block.

```APIDOC
## GET /datasets/hyperliquid-replica-cmds/metadata

### Description
Retrieves the metadata for the Hyperliquid replica dataset.

### Method
GET

### Endpoint
https://portal.sqd.dev/datasets/hyperliquid-replica-cmds/metadata

### Response
#### Success Response (200)
- **dataset** (string) - The default name used to reference this dataset.
- **aliases** (string[]) - Alternative names for the dataset.
- **real_time** (boolean) - Indicates if the dataset has real-time data.
- **start_block** (integer) - The block number of the first known block.

#### Response Example
{
  "dataset": "hyperliquid-replica-cmds",
  "aliases": ["hyperliquid"],
  "real_time": true,
  "start_block": 123
}

#### Error Response (404)
- **error** (object) - Error details including type, code, and message.
```

--------------------------------

### Start Database and Run Migrations

Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron/guides/advanced/cli-cheatsheet#checking-out-the-indexed-data

Commands to initialize the local PostgreSQL database and apply schema migrations.

```bash
docker compose up -d
```

```bash
npx squid-typeorm-migration apply
```

--------------------------------

### GET /metadata

Source: https://docs.sqd.dev/en/api/evm/metadata

Retrieves metadata describing the dataset, including its name, aliases, start block, and real-time status.

```APIDOC
## GET /metadata

### Description
Retrieves metadata describing the dataset, including its name, aliases, start block, and real-time status.

### Method
GET

### Endpoint
https://portal.sqd.dev/datasets/{dataset}/metadata

### Response
#### Success Response (200)
- **dataset** (string) - The default name used to reference this dataset.
- **aliases** (array) - Alternative names for the dataset.
- **real_time** (boolean) - Indicates if the dataset has real-time data.
- **start_block** (integer) - The block number of the first known block.

#### Response Example
{
  "dataset": "ethereum-mainnet",
  "aliases": ["eth", "ethereum"],
  "real_time": true,
  "start_block": 0
}
```

--------------------------------

### Start the Docker stack

Source: https://docs.sqd.dev/en/portal/self-hosting

Initialize the services and monitor logs.

```bash
docker compose up -d
docker compose logs -f krown-data-service hotblocks-db
```
Tokens:
Raw