### Install Dependencies and Build

Source: https://docs.blobscan.com/docs/running-blobscan-locally

Commands to fetch and install Node.js dependencies and build the project.

```bash
pnpm fetch -r
pnpm install -r
SKIP_ENV_VALIDATION=true npm run build
```

--------------------------------

### Start Databases

Source: https://docs.blobscan.com/docs/running-blobscan-locally

Command to start the required PostgreSQL and Redis services using Docker Compose.

```bash
docker compose -f docker-compose.local.yml up -d postgres redis
```

--------------------------------

### Run Development Server

Source: https://docs.blobscan.com/docs/running-blobscan-locally

Command to start the development server.

```bash
pnpm dev
```

--------------------------------

### Install libssl-dev

Source: https://docs.blobscan.com/docs/indexer

Installs the libssl-dev package, a prerequisite for the indexer.

```bash
sudo apt install libssl-dev
```

--------------------------------

### Install Node.js and pnpm

Source: https://docs.blobscan.com/docs/running-blobscan-locally

Commands to install the required Node.js version and the pnpm package manager on a Linux system.

```bash
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs
curl -fsSL https://get.pnpm.io/install.sh | sh -
```

--------------------------------

### Install Dependencies with pnpm

Source: https://docs.blobscan.com/docs/testing

Install all project dependencies using pnpm. This command ensures all necessary packages are available for development and testing.

```bash
pnpm install

```

--------------------------------

### Create Command Usage

Source: https://docs.blobscan.com/docs/clis

Usage guide for creating blob propagation jobs.

```text
Create Command

  Create propagation jobs for blobs.

Options

  -h, --help                 Print this usage guide.
  -b, --blobHash blob-hash   Blob hash of the blobs to create jobs for.
  -s, --storage storage      Storage used to propagate the selected blobs.
                             Valid values are google, postgres or swarm.
  -f, --from from            Date from which to retrieve blobs to create jobs
                             for.
  -t, --to to                Date to which to retrieve blobs to create jobs
                             for.
```

--------------------------------

### Initialize Environment Configuration

Source: https://docs.blobscan.com/docs/installation

Create a local .env file from the provided example to define required environment variables.

```bash
cp .env.example .env
```

--------------------------------

### Streamline Test Setup and Execution

Source: https://docs.blobscan.com/docs/testing

Use 'pnpm test:dev' to combine the setup of Docker containers and the execution of tests, streamlining the development workflow.

```bash
pnpm test:dev

```

--------------------------------

### Initialize Docker Containers for Testing

Source: https://docs.blobscan.com/docs/testing

Set up and initialize the necessary Docker containers for integration testing. This includes starting a local PostgreSQL database and a fake GCS server.

```bash
pnpm test:setup

```

--------------------------------

### Deploy with Kubernetes Helm

Source: https://docs.blobscan.com/docs/installation

Add the Blobscan Helm repository and install the chart into a Kubernetes cluster.

```bash
helm repo add blobscan-helm-charts https://blobscan.github.io/blobscan-helm-charts
helm install blobscan blobscan-helm-charts/blobscan
```

--------------------------------

### Deploy with Docker Compose

Source: https://docs.blobscan.com/docs/installation

Start the Blobscan services in detached mode using Docker Compose.

```bash
docker compose up -d
```

--------------------------------

### Overall Stats Command Usage

Source: https://docs.blobscan.com/docs/clis

Usage guide for aggregating overall statistics.

```text
Overall Command

  Aggregate overall stats.

Options

  -d, --delete           Delete existing stats.
  -h, --help             Print this usage guide.
  -t, --to block-id      Block identifier up to which to aggregate data. It can
                         be a block number, "latest" for the last indexed
                         block or "finalized"  for the chain's most recent
                         finalized block. It defaults to "finalized"
  -s, --batchSize size   Number of blocks to process in a single batch. It
                         defaults to 100000
```

--------------------------------

### Retry Command Usage

Source: https://docs.blobscan.com/docs/clis

Usage guide for retrying failed propagation jobs.

```text
Retry Command

  Retries failed jobs.

Options

  -h, --help                 Print this usage guide.
  -q, --queue queue          Queue to retry failed jobs from. Valid values are
                             finalizer, google, postgres or swarm.
  -b, --blobHash blob-hash   Blob hash of the failed jobs to retry.
```

--------------------------------

### Daily Stats Command Usage

Source: https://docs.blobscan.com/docs/clis

Usage guide for aggregating daily statistics.

```text
Daily Command

  Aggregate daily stats.

Options

  -d, --delete        Delete existing stats.
  -h, --help          Print this usage guide.
  -e, --entity type   Entity type to aggregate. Valid values are blob, block or
                      tx.
  -f, --from date     Start date in ISO 8601 format.
  -t, --to date       End date in ISO 8601 format.
```

--------------------------------

### Index from Specific Slot with Multiple Threads

Source: https://docs.blobscan.com/docs/indexer

Starts the indexer from a specified slot and uses a defined number of threads for parallel processing.

```bash
cargo run -- -f 1000 -n 10
```

--------------------------------

### Uninstall Blobscan

Source: https://docs.blobscan.com/docs/running-blobscan-k8s

Commands to identify and remove the installed Blobscan release.

```bash
helm ls
helm uninstall blobscan
```

--------------------------------

### Remove Command Usage

Source: https://docs.blobscan.com/docs/clis

Usage guide for removing failed propagation jobs.

```text
Remove Command

  Removes failed jobs.

Options

  -h, --help                 Print this usage guide.
  -q, --queue queue          Queue to remove failed jobs from. Valid values are
                             finalizer, google, postgres or swarm.
  -b, --blobHash blob-hash   Blob hash of the failed jobs to retry.
  -f, --force                Force removal of jobs by obliterating the selected
                             queues.
```

--------------------------------

### Debug container with shell access

Source: https://docs.blobscan.com/docs/troubleshooting

Override the entrypoint to access a bash shell in a container that is failing to start.

```bash
docker compose run --entrypoint bash api
```

--------------------------------

### Create a local Kind cluster

Source: https://docs.blobscan.com/docs/running-blobscan-k8s

Use this to set up a local Kubernetes environment for testing Helm charts.

```bash
kind create cluster
kubectl cluster-info --context kind-kind
```

--------------------------------

### Run Database Migrations and Seeding

Source: https://docs.blobscan.com/docs/running-blobscan-locally

Commands to apply database migrations and seed the database with fixture data.

```bash
cd packages/db
pnpm db:migrate
pnpm db:seed
```

--------------------------------

### Launch Blobscan with Kurtosis

Source: https://docs.blobscan.com/docs/running-blobscan-kurtosis

Executes a local Ethereum testnet environment with the Blobscan service enabled using the ethPandaOps package.

```bash
kurtosis clean -a && kurtosis run github.com/ethpandaops/ethereum-package --image-download always '{"additional_services": ["blobscan"]}'
```

--------------------------------

### Run Blobscan from local repository

Source: https://docs.blobscan.com/docs/running-blobscan-kurtosis

Clones the ethereum-package repository and launches the environment locally, including the goomy_blob spammer service.

```bash
git clone https://github.com/ethpandaops/ethereum-package.git

cd ethereum-package
kurtosis clean -a && kurtosis run . --image-download always '{"additional_services": ["blobscan", "goomy_blob"]}'
```

--------------------------------

### Run Blob Propagation CLI

Source: https://docs.blobscan.com/docs/clis

Navigate to the propagation CLI directory and execute commands.

```bash
cd clis/blob-propagation-jobs-cli
pnpm start <command>
```

--------------------------------

### Build a Release Version

Source: https://docs.blobscan.com/docs/indexer

Builds an optimized release version of the Blobscan indexer using Cargo.

```bash
cargo build -r
```

--------------------------------

### Configure Environment Variables for Testing

Source: https://docs.blobscan.com/docs/testing

Copy the test environment file and rename it to .env. Adjust the .env file according to the project's specific testing requirements.

```bash
cp .env.test .env

```

--------------------------------

### Run the Indexer Locally

Source: https://docs.blobscan.com/docs/indexer

Compiles and runs the Blobscan indexer using Cargo.

```bash
cargo run
```

--------------------------------

### Run Indexer with Default Docker Image

Source: https://docs.blobscan.com/docs/indexer

Runs the Blobscan indexer using the master Docker image with default configurations.

```bash
docker run --rm blossomlabs/blobscan-indexer:master
```

--------------------------------

### Clone Repository

Source: https://docs.blobscan.com/docs/running-blobscan-locally

Commands to clone the Blobscan repository and enter the project directory.

```bash
git clone https://github.com/Blobscan/blobscan.git
cd blobscan
```

--------------------------------

### Run Stats Aggregation CLI

Source: https://docs.blobscan.com/docs/clis

Navigate to the stats aggregation CLI directory and execute commands.

```bash
cd clis/stats-aggregation-cli
pnpm start <command>
```

--------------------------------

### Search available Helm charts

Source: https://docs.blobscan.com/docs/running-blobscan-k8s

List all available charts within the added repository.

```bash
helm search repo blobscan-helm-charts
```

--------------------------------

### Execute Tests with pnpm

Source: https://docs.blobscan.com/docs/testing

Run all project tests using the pnpm test command. This executes the test suite defined in the project.

```bash
pnpm test

```

--------------------------------

### Run Indexer with .env File via Docker

Source: https://docs.blobscan.com/docs/indexer

Runs the Blobscan indexer using Docker, loading environment variables from a local .env file.

```bash
docker run --env-file=.env --rm blossomlabs/blobscan-indexer:master
```

--------------------------------

### Clone Blobscan Repository

Source: https://docs.blobscan.com/docs/testing

Clone the Blobscan repository to your local machine. This is the first step to setting up your development environment for testing.

```bash
git clone https://github.com/Blobscan/blobscan.git
cd blobscan

```

--------------------------------

### Run Indexer with Environment Variables via Docker

Source: https://docs.blobscan.com/docs/indexer

Runs the Blobscan indexer using Docker, configuring API endpoints and node connections via environment variables.

```bash
docker run --rm \
  -e SECRET_KEY=supersecret \
  -e BLOBSCAN_API_ENDPOINT=http://blobscan-api:3001 \
  -e BEACON_NODE_ENDPOINT=http://beacon:3500 \
  -e EXECUTION_NODE_ENDPOINT=http://execution:8545 \
  blossomlabs/blobscan-indexer:master
```

--------------------------------

### Apply pending database migrations

Source: https://docs.blobscan.com/docs/troubleshooting

Manually trigger pending migrations using the Prisma CLI.

```bash
pnpm prisma migrate deploy --schema packages/db/prisma/schema.prisma
```

--------------------------------

### Add and update the Helm repository

Source: https://docs.blobscan.com/docs/running-blobscan-k8s

Register the official Blobscan Helm repository to access the charts.

```bash
helm repo add blobscan-helm-charts https://blobscan.github.io/blobscan-helm-charts
```

```bash
helm repo update
helm install blobscan blobscan-helm-charts/blobscan
```

--------------------------------

### Access SQL shell

Source: https://docs.blobscan.com/docs/troubleshooting

Open a PostgreSQL shell directly within the running container.

```bash
docker compose exec postgres psql -h localhost blobscan_dev blobscan
```

--------------------------------

### Blob Propagation CLI

Source: https://docs.blobscan.com/docs/clis

Commands for propagating blobs across different storage systems.

```APIDOC
## POST /blob-propagation/create

### Description
Creates propagation jobs for blobs.

### Method
POST

### Endpoint
/blob-propagation/create

### Parameters
#### Query Parameters
- **blobHash** (string) - Required - Blob hash of the blobs to create jobs for.
- **storage** (string) - Required - Storage used to propagate the selected blobs. Valid values are google, postgres or swarm.
- **from** (date) - Optional - Date from which to retrieve blobs to create jobs for.
- **to** (date) - Optional - Date to which to retrieve blobs to create jobs for.

### Request Example
```json
{
  "blobHash": "exampleBlobHash",
  "storage": "google",
  "from": "2023-01-01T00:00:00Z",
  "to": "2023-01-31T23:59:59Z"
}
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message.

#### Response Example
```json
{
  "message": "Propagation jobs created successfully."
}
```
```

```APIDOC
## DELETE /blob-propagation/remove

### Description
Removes failed jobs from a specified queue.

### Method
DELETE

### Endpoint
/blob-propagation/remove

### Parameters
#### Query Parameters
- **queue** (string) - Required - Queue to remove failed jobs from. Valid values are finalizer, google, postgres or swarm.
- **blobHash** (string) - Optional - Blob hash of the failed jobs to retry.
- **force** (boolean) - Optional - Force removal of jobs by obliterating the selected queues.

### Request Example
```json
{
  "queue": "finalizer",
  "blobHash": "exampleBlobHash",
  "force": true
}
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message.

#### Response Example
```json
{
  "message": "Failed jobs removed successfully."
}
```
```

```APIDOC
## POST /blob-propagation/retry

### Description
Retries failed jobs from a specified queue.

### Method
POST

### Endpoint
/blob-propagation/retry

### Parameters
#### Query Parameters
- **queue** (string) - Required - Queue to retry failed jobs from. Valid values are finalizer, google, postgres or swarm.
- **blobHash** (string) - Optional - Blob hash of the failed jobs to retry.

### Request Example
```json
{
  "queue": "google",
  "blobHash": "exampleBlobHash"
}
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message.

#### Response Example
```json
{
  "message": "Failed jobs retried successfully."
}
```
```

--------------------------------

### Aggregate Metrics

Source: https://docs.blobscan.com/docs/running-blobscan-locally

Commands to manually trigger metrics aggregation for all data or specific timeframes.

```bash
# Aggregates all blob data since the beginning
cd clis/stats-aggregation-cli
pnpm start overall
```

```bash
# Aggregates all blob data for yesterday
cd clis/stats-aggregation-cli
pnpm start daily
```

--------------------------------

### Reset database and wipe data

Source: https://docs.blobscan.com/docs/troubleshooting

Perform a full reset of the database, which will delete all existing data.

```bash
pnpm prisma migrate reset --schema packages/db/prisma/schema.prisma
```

--------------------------------

### Generate Secret Key for .env

Source: https://docs.blobscan.com/docs/indexer

Generates a random base64 encoded secret key and saves it to the .env file for API authentication.

```bash
echo "SECRET_KEY=$(openssl rand -base64 32)" > .env
```

--------------------------------

### Filter Traces with RUST_LOG Environment Variable

Source: https://docs.blobscan.com/docs/indexer

Runs the indexer with custom trace filtering directives defined in the RUST_LOG environment variable.

```bash
RUST_LOG=blob_indexer[span{field=value}]=level cargo run
```

--------------------------------

### Pipe Indexer Output to Bunyan for Formatted Logs

Source: https://docs.blobscan.com/docs/indexer

Runs the indexer in quiet mode and pipes its output to the Bunyan CLI for formatted log display.

```bash
cargo run -q | bunyan
```

--------------------------------

### Stats Aggregation CLI

Source: https://docs.blobscan.com/docs/clis

Commands for aggregating daily and overall statistics.

```APIDOC
## POST /stats-aggregation/daily

### Description
Aggregates daily statistics.

### Method
POST

### Endpoint
/stats-aggregation/daily

### Parameters
#### Query Parameters
- **delete** (boolean) - Optional - Delete existing stats before aggregating.
- **entity** (string) - Required - Entity type to aggregate. Valid values are blob, block or tx.
- **from** (date) - Required - Start date in ISO 8601 format.
- **to** (date) - Required - End date in ISO 8601 format.

### Request Example
```json
{
  "delete": false,
  "entity": "blob",
  "from": "2023-01-01T00:00:00Z",
  "to": "2023-01-31T23:59:59Z"
}
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message.

#### Response Example
```json
{
  "message": "Daily stats aggregated successfully."
}
```
```

```APIDOC
## POST /stats-aggregation/overall

### Description
Aggregates overall statistics.

### Method
POST

### Endpoint
/stats-aggregation/overall

### Parameters
#### Query Parameters
- **delete** (boolean) - Optional - Delete existing stats before aggregating.
- **to** (string) - Required - Block identifier up to which to aggregate data. It can be a block number, "latest" for the last indexed block or "finalized" for the chain's most recent finalized block. Defaults to "finalized".
- **batchSize** (integer) - Optional - Number of blocks to process in a single batch. Defaults to 100000.

### Request Example
```json
{
  "delete": false,
  "to": "latest",
  "batchSize": 50000
}
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message.

#### Response Example
```json
{
  "message": "Overall stats aggregated successfully."
}
```
```

--------------------------------

### Clone Blobscan Indexer Repository

Source: https://docs.blobscan.com/docs/indexer

Clones the Blobscan indexer repository from GitHub and navigates into the directory.

```bash
git clone https://github.com/Blobscan/blobscan-indexer.rs.git
cd blobscan-indexer.rs
```

--------------------------------

### Delete local Kind cluster

Source: https://docs.blobscan.com/docs/running-blobscan-k8s

Clean up the local Kubernetes environment after testing.

```bash
kind delete cluster
```

--------------------------------

### Delete Aggregated Metrics

Source: https://docs.blobscan.com/docs/running-blobscan-locally

Commands to remove previously aggregated metrics using the stats aggregation CLI.

```bash
cd clis/stats-aggregation-cli
pnpm start daily --delete
pnpm start overall --delete
```

--------------------------------

### Blob Data Structure

Source: https://docs.blobscan.com/

Represents the JSON schema for blob metadata and storage references used within the Blobscan system.

```json
{
  "blob": {
    "versionedHash": "0x0100...1368",
    "commitment": "0xb4f67eb0...3803b983"
  },
  "blobDataStorage": [{
    "storage": "GOOGLE",
    "reference": "1/01/00/0100...1368.bin"
  }]
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.