### Install openECS and Go Dependencies

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

These commands guide the user through cloning the openECS repository, pulling large files with Git LFS, and setting up the Go module dependencies. It also includes steps to install the latest versions of `protoc-gen-go` and `protoc-gen-go-grpc` for protobuf and gRPC support, ensuring the project's build environment is complete.

```bash
git clone https://github.com/cipherowl.ai/openECS.git
git lfs pull
cd openECS
go mod tidy
```

```bash
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
go get google.golang.org/grpc@latest
go mod tidy
```

--------------------------------

### Run ECSD Service with Configuration Examples

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Demonstrates how to start the ECSD service using both command-line flags and environment variables to configure essential service parameters like chain, dataset, base URL, client ID, and client secret.

```bash
# Using command line flags
./ecsd --chain ethereum_mainnet --dataset co-demo --base-url https://api.example.com \
       --client-id your_client_id --client-secret your_client_secret

# Using environment variables
export CO_CHAIN=ethereum_mainnet
export CO_DATASET=co-demo
export CO_BASE_URL=https://api.example.com
export CO_CLIENT_ID=your_client_id
export CO_CLIENT_SECRET=your_client_secret
./ecsd
```

--------------------------------

### Run ECSD Service with Various Options

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Examples of running the ECSD service, demonstrating how to use default options, specify a custom Bloom filter file, change the HTTP port, set rate limits, and configure encryption keys via command-line flags and environment variables.

```bash
# Run with default options
./target/release/pa-ecsd

# Specify a different Bloom filter file
./target/release/pa-ecsd -f /path/to/bloomfilter.gob

# Change the port
./target/release/pa-ecsd -p 8081

# Set rate limits
./target/release/pa-ecsd -r 50 -b 10

# Use environment variables with GRPC port 9090 and HTTP port 8080, plus encryption keys
export KEY_PASSPHRASE=123456 # the passphrase for the private key, alternatively can put into the `.env`                   
./target/release/pa-ecsd -f bloomfilter.gob -p 8080 -gp 9090 -r 2000 -b 5000 --decrypt-key securedata/testdata/privkey.asc --signing-key securedata/testdata/pubkey.asc
```

--------------------------------

### Load Bloom Filter Store from File (Go)

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

This Go example illustrates how to load an existing Bloom filter store from a specified file path. After loading, it demonstrates checking for the presence of an Ethereum address within the loaded filter. This is useful for reusing pre-built Bloom filters without re-generating them.

```go
// Create a new BloomFilterStore and load from the file
store, _ := NewBloomFilterStoreFromFile(filePath, addressHandler)

if ok, _ := store.CheckAddress("0x1234567890123456789012345678901234567890"); ok {
    fmt.Println("Address found in the Bloom filter")
} else {
    fmt.Println("Address not found in the Bloom filter")
}
```

--------------------------------

### Start openECS Docker Container for Performance Testing

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

Launches the 'ecsd:latest' Docker container, configuring it for local performance testing. It maps ports 8080 and 9090, mounts a local keypair directory, and loads environment variables from 'docker.env' to set up the service with a high rate limit.

```Bash
docker run --env-file docker.env -p 8080:8080 -p 9090:9090 -v $(pwd)/ecsd/keypair/:/app/keys ecsd:latest
```

--------------------------------

### HTTP API Examples for Bloom Filter Service

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Illustrates how to interact with the Bloom filter service's HTTP API using `curl` for single address checks, batch address checks, and inspecting filter statistics.

```bash
curl "http://localhost:8080/check?address=0x96244d83dc15d36847c35209bbdc5bdde9bec3d8"
```

```bash
curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"addresses":["0x96244d83dc15d36847c35209bbdc5bdde9bec3d8","0x1111111111111111111111111111111111111111"]}' \
  "http://localhost:8080/batch-check"
```

```bash
curl "http://localhost:8080/inspect"
```

--------------------------------

### CLI: Generate Ethereum Addresses

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

This command-line interface (CLI) example demonstrates how to generate a large number of Ethereum addresses. The `-n` flag specifies the quantity of addresses to generate, which are then saved to a file named `address.txt` by default. This is an optional first step for populating a Bloom filter.

```bash
go run cmd/cli/main.go generate-addresses -n 1000000
```

--------------------------------

### Query Address with LLM Bot Caller Mode Enabled (True Example)

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Illustrates querying an Ethereum address using `curl` with the `__llm_bot_caller__: 1` header. Similar to the previous example, this enables the LLM bot explanation mode, but for an address that is found within the Bloom filter, demonstrating a `true` value for `in_set`.

```bash
curl "http://localhost:8080/check?address=0x96244d83dc15d36847c35209bbdc5bdde9bec3d8" -H "__llm_bot_caller__: 1" | jq
```

```json
{
  "query": "0x96244d83dc15d36847c35209bbdc5bdde9bec3d8",
  "in_set": true,
  "message": "",
  "__llm_explanations__data_dictionary__": {
    "in_set": "Whether the address was found in the Bloom filter (true) or not (false). Note that Bloom filters may have false positives but never false negatives",
    "message": "Additional information or error message",
    "query": "The address that was queried"
  }
}
```

--------------------------------

### Run ECSD Docker Container

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Examples for running the ECSD service within a Docker container, including default configuration, setting environment variables via a file, and mounting a custom Bloom filter file.

```bash
# Run with default configuration
docker run -p 8080:8080 ecsd

# Set environment variables
docker run docker run --env-file docker.env -p 8080:8080 -p 9090:9090 -v $(pwd)/ecsd/keypair/:/app/keys --rm ecsd:latest

# Mount your own Bloom filter file
docker run -p 8080:8080 \
  -v /path/to/your/bloomfilter.gob:/app/bloomfilter.gob \
  ecsd -f bloomfilter.gob
```

--------------------------------

### Query Address with LLM Bot Caller Mode Enabled (False Example)

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Demonstrates how to use `curl` to query an Ethereum address against the OpenECS service. By including the `__llm_bot_caller__: on` header, the service returns an extended JSON response that includes a `__llm_explanations__data_dictionary__` field, providing human-readable descriptions for each key in the response, tailored for LLM bots. This example shows an address not found in the set.

```bash
curl "http://localhost:8080/check?address=0xeb0baa3a556586192590cad296b1e48df62a8549" -H "__llm_bot_caller__: on" | jq
```

```json
{
  "query": "0xE5a00E3FccEfcCd9e4bA75955e12b6710eB254bE",
  "in_set": false,
  "message": "",
  "__llm_explanations__data_dictionary__": {
    "in_set": "Whether the address was found in the Bloom filter (true) or not (false). Note that Bloom filters may have false positives but never false negatives",
    "message": "Additional information or error message",
    "query": "The address that was queried"
  }
}
```

--------------------------------

### Load Bloom Filter from PGP Encrypted File (Go)

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

This Go example shows how to load a Bloom filter from a PGP encrypted file, enhancing data confidentiality. It involves initializing a `PGPSecureDataHandler` with public and private key paths and an optional passphrase. This handler is then passed to `NewBloomFilterStoreFromFile` to securely decrypt and load the Bloom filter.

```go
// Create the pgp secure data handler
pgpHandler := securedata.NewPGPSecureDataHandler(
	withPublicKeyPath(publicKeyPath),
	withPrivateKeyPath(privateKeyPath, passphrase),
)

// Create a new Bloom filter store from a pgp encrypted file
addressHandler := &address.EVMAddressHandler{}
store, _ := NewBloomFilterStoreFromFile(filePath, addressHandler, WithSecureDataHandler(pgpHandler))
```

--------------------------------

### CLI: Encode and Check Large-Scale Bloom Filter

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

This example demonstrates building and using a Bloom filter for a very large dataset, specifically 24 million Ethereum addresses. The first command encodes addresses from a CSV file into a Bloom filter, specifying the number of entries and false positive rate. The second command then checks addresses against the resulting large `bloomfilter.gob` file, showcasing the scalability of the solution.

```bash
go run cmd/cli/main.go encode -n 1000000000 -p 0.000001 -input ~/path/to/eth_all.csv
```

```bash
go run cmd/cli/main.go check -f bloomfilter.gob
```

--------------------------------

### Build ECSD Service Executable

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Instructions for building the ECSD service executable using the `make` command, typically resulting in a binary located in `target/*/pa-ecsd`.

```bash
make # target/*/pa-ecsd
```

--------------------------------

### Run Go Micro-benchmarks

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

Executes Go micro-benchmarks for the 'openECS/store' package. The '-bench=.' flag runs all benchmarks, and '-benchmem' enables memory allocation profiling, providing insights into operations per second, memory usage, and allocations per operation.

```Bash
go test -bench=. -benchmem
```

--------------------------------

### List Available Methods for proto.ECSd gRPC Service

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Shows how to use `grpcurl` to list all callable methods within the `proto.ECSd` gRPC service. This helps in understanding the specific operations supported by the service.

```bash
grpcurl -plaintext localhost:9090 list proto.ECSd
```

```bash
proto.ECSd.BatchCheckAddresses
proto.ECSd.CheckAddress
proto.ECSd.InspectFilter
```

--------------------------------

### Run ECSD Docker Container with Environment File and Port Mappings

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README_DOCKERFILE.md

This command runs the ECSD Docker container, loading environment variables from 'docker.env'. It maps the container's HTTP (8080) and gRPC (9090) ports to the host, and mounts a local 'keys' directory for application access.

```bash
docker run --env-file docker.env -p 8080:8080 -p 9090:9090 -v ~/keys/:/app/keys ecsd:latest
```

--------------------------------

### Build ECSD Docker Image

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README_DOCKERFILE.md

This command builds the Docker image for ECSD. Ensure the Dockerfile in the 'ecsd' directory is updated and ARG values are correctly set before execution. The image will be tagged as 'ecsd'.

```bash
docker build -t ecsd -f ecsd/Dockerfile .
```

--------------------------------

### Test PGP Encryption and Signing with Go

Source: https://github.com/cipherowl-ai/openecs/blob/main/securedata/README.md

This command executes the Go tests for the `securedata` package. It verifies the functionality of PGP encryption and signing implementations, ensuring that the cryptographic operations are working as expected. The `-v` flag provides verbose output, showing details of each test run.

```bash
go test -v ./securedata
```

--------------------------------

### Create and Add Addresses to Bloom Filter Store (Go)

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

This Go snippet demonstrates how to initialize a new Bloom filter store with a specified false positive rate and add an Ethereum address to it. It then checks if the added address is present in the filter and prints the result. Finally, the filter is saved to a specified file path for persistence.

```go
// Create an EVM address handler that will be used to validate and encode addresses
addressHandler := &address.EVMAddressHandler{}

// Create a new BloomFilterStore to store 10000 addresses with a false positive rate of 0.0000001
store, _ := NewBloomFilterStore(addressHandler)

store.AddAddress("0x1234567890123456789012345678901234567890")

if ok, _ := store.CheckAddress("0x1234567890123456789012345678901234567890"); ok {
    fmt.Println("Address found in the Bloom filter")
} else {
    fmt.Println("Address not found in the Bloom filter")
}

store.SaveToFile(filePath)
```

--------------------------------

### Configure pa-cli Options via Environment Variables

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This section outlines how to configure `pa-cli` command-line flags using environment variables. All variables are prefixed with `CO_` and use underscores, providing an alternative to command-line arguments for persistent or automated configurations.

```APIDOC
Environment Variables:
  CO_FILENAME: Corresponds to --filename
  CO_DECRYPT_KEY: Corresponds to --decrypt-key
  CO_SIGNING_KEY: Corresponds to --signing-key
  CO_DECRYPT_KEY_PASSPHRASE: Corresponds to --decrypt-key-passphrase
  CO_ENCRYPT_KEY: Corresponds to --encrypt-key
  CO_SIGNING_KEY_PASSPHRASE: Corresponds to --signing-key-passphrase
```

--------------------------------

### Run Python Client Performance Test

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

Executes a performance test using the Python client script 'bench.py'. This command loads addresses from 'demo_addresses.txt', runs a load test for 30 seconds, and uses 32 concurrent connections to measure the service's throughput and latency.

```Bash
python bench.py -f demo_addresses.txt --duration 30 --concurrency 32
```

--------------------------------

### Run ECSD Docker Container with Persistent Data Volume

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README_DOCKERFILE.md

These commands demonstrate how to create a Docker named volume for persistent storage and then run the ECSD container with this volume mounted. This ensures data persists across container restarts and removals.

```bash
docker volume create ecsd-data

docker run \
  --env-file docker.env \
  -v $(pwd)/ecsd/keypair/:/app/keys \
  -v $(pwd)/ecsd-data:/app/data \
  --rm \
  ecsd
```

--------------------------------

### Generate Ethereum Addresses for Testing with pa-cli

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command generates a specified number of Ethereum addresses, saving them to a designated output file. It's primarily used for creating large datasets for testing bloom filter operations and other address-related functionalities.

```bash
pa-cli generate-addresses --output ./addresses.txt -n 1000000
```

--------------------------------

### Export GPG Public and Private Keys

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

These commands export the public and private components of a GPG key pair to separate ASCII armored files. The `<KEY_ID>` placeholder must be replaced with the actual ID of the key to be exported, typically obtained from `gpg --list-keys`.

```bash
gpg --armor --export <KEY_ID> > test_public.asc
gpg --armor --export-secret-key <KEY_ID> > test_private.asc
```

--------------------------------

### Build Docker Image for ECSD Service

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Command to build a Docker image for the ECSD service from the repository root, tagging it as `ecsd`.

```bash
docker build -t ecsd -f cmd/ecsd/Dockerfile .
```

--------------------------------

### List Available gRPC Services

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Demonstrates how to use `grpcurl` to discover all available gRPC services exposed by the ECSD service running on port 9090. This is useful for initial exploration of the gRPC API.

```bash
grpcurl -plaintext localhost:9090 list
```

```bash
grpc.reflection.v1.ServerReflection
grpc.reflection.v1alpha.ServerReflection
proto.ECSd
```

--------------------------------

### Interactively Check Individual Ethereum Addresses against Bloom Filter

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command enables interactive verification of single Ethereum addresses against a pre-existing bloom filter. It requires specific decryption and signing keys for secure access and can perform checks on addresses as their hash values.

```bash
pa-cli check --filename ./bloomfilter.gob \
    --decrypt-key securedata/testdata/privkey.asc --signing-key securedata/testdata/pubkey.asc \
    --decrypt-key-passphrase "123456" --hash
```

--------------------------------

### Generate a New GPG Key Pair

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command initiates the process of generating a new GPG key pair (public and private keys). Users will be prompted to configure key parameters like type, size, expiration, and user identity, and to provide a passphrase for the private key.

```bash
gpg --full-generate-key
```

--------------------------------

### Batch Check Multiple Ethereum Addresses against Bloom Filter

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command facilitates checking multiple Ethereum addresses, supplied via standard input, against a specified bloom filter. Similar to the interactive checker, it supports decryption and signing, and can process addresses as hash values for efficient batch verification.

```bash
cat addresses.txt | ./pa-cli batch-check --filename ./bloomfilter.gob \
    --decrypt-key ./securedata/testdata/privkey.asc --signing-key ./securedata/testdata/pubkey.asc \
    --decrypt-key-passphrase "123456" --hash
```

--------------------------------

### List Existing GPG Keys

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command displays a list of all public keys managed by GPG. It's used to identify the Key ID or Fingerprint of a specific key, which is often needed for export or other key management operations.

```bash
gpg --list-keys
```

--------------------------------

### CLI: Check Addresses Against Bloom Filter

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

These CLI commands illustrate how to use a pre-built Bloom filter to check for the presence of Ethereum addresses. The first command shows interactive mode, allowing individual address checks. The second command demonstrates batch mode, where addresses are piped from a file for efficient bulk checking against the specified Bloom filter file.

```bash
go run cmd/cli/main.go check -f bloomfilter.gob
```

```bash
cat my_addresses.txt | go run cmd/cli/main.go batch-check -f bloomfilter.gob
```

--------------------------------

### Generate Test Addresses with pa-cli

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command generates a specified number of random addresses and saves them to a file. It's typically used for creating large datasets for testing bloom filter operations.

```bash
pa-cli generate-addresses -n 1000000 -o ./addresses.txt
```

--------------------------------

### Encode Ethereum Addresses into a Bloom Filter using pa-cli

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command constructs a bloom filter from a list of Ethereum addresses provided in an input file. It supports advanced features like signing and encryption of the resulting bloom filter, and can optionally convert addresses to hash values before encoding for privacy or efficiency.

```bash
pa-cli encode --input ./addresses.txt --output ./bloomfilter.gob \
    --signing-key securedata/testdata/privkey.asc --encrypt-key securedata/testdata/pubkey.asc \
    --signing-key-passphrase "123456" --hash
```

--------------------------------

### Create Encrypted Bloom Filter with pa-cli

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command creates an encrypted bloom filter from a list of addresses. It requires a signing key for integrity and an encryption key for confidentiality, along with their passphrases. The `--hash` flag enables hash mode.

```bash
pa-cli encode -i ./addresses.txt -o ./bloomfilter.gob \
    --signing-key ./keys/private.asc \
    --encrypt-key ./keys/public.asc \
    --signing-key-passphrase "123456" \
    --hash
```

--------------------------------

### Batch Check Addresses with pa-cli

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command performs a batch check of addresses against a bloom filter, reading addresses from standard input. Similar to interactive checking, it requires decryption and signing keys with passphrases and uses hash mode.

```bash
cat addresses_to_check.txt | pa-cli batch-check -f ./bloomfilter.gob \
    --decrypt-key ./keys/private.asc \
    --signing-key ./keys/public.asc \
    --decrypt-key-passphrase "123456" \
    --hash
```

--------------------------------

### Interactively Check Addresses with pa-cli

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command allows interactive checking of addresses against an existing bloom filter. It requires decryption and signing keys with their passphrases to access the filter. The `--hash` flag ensures consistency with the filter's creation mode.

```bash
pa-cli check -f ./bloomfilter.gob \
    --decrypt-key ./keys/private.asc \
    --signing-key ./keys/public.asc \
    --decrypt-key-passphrase "123456" \
    --hash
```

--------------------------------

### Bloom Filter Service Configuration Options

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Describes the various configuration parameters for the Bloom filter service, available via command-line flags or environment variables, covering Bloom filter settings, server ports, rate limiting, and required service credentials.

```APIDOC
Configuration Options:
  Command Line Flags:
    Bloom Filter Options:
      -f, --filename: string (default: "bloomfilter.gob")
        Description: Path to the Bloom filter file
      --decrypt-key: string (default: "")
        Description: Path to the decrypt key file
      --signing-key: string (default: "")
        Description: Path to the signing key file
      --decrypt-key-passphrase: string (default: "")
        Description: Passphrase for the decrypt key
    Server Options:
      --http-port: int (default: 8080)
        Description: HTTP port to listen on
      --grpc-port: int (default: 9090)
        Description: gRPC port to listen on
      --rate-limit: int (default: 20)
        Description: Rate limit for requests
      --burst: int (default: 5)
        Description: Burst limit for rate limiting
    Service Options (All Required):
      --chain: string
        Description: Chain name (e.g., "ethereum_mainnet")
      --dataset: string
        Description: Dataset name (e.g., "co-demo")
      --base-url: string
        Description: Base URL for the API (e.g., "https://api.example.com")
      --client-id: string
        Description: Client ID for authentication
      --client-secret: string
        Description: Client secret for authentication

  Environment Variables (prefixed with CO_):
    CO_FILENAME: string (default: "bloomfilter.gob")
      Description: Bloom filter file path
    CO_DECRYPT_KEY: string (default: "")
      Description: Path to decrypt key file
    CO_SIGNING_KEY: string (default: "")
      Description: Path to signing key file
    CO_DECRYPT_KEY_PASSPHRASE: string (default: "")
      Description: Decrypt key passphrase
    CO_HTTP_PORT: int (default: 8080)
      Description: HTTP port
    CO_GRPC_PORT: int (default: 9090)
      Description: gRPC port
    CO_RATE_LIMIT: int (default: 20)
      Description: Rate limit
    CO_BURST: int (default: 5)
      Description: Burst limit
    CO_CHAIN: string
      Description: Chain name
    CO_DATASET: string
      Description: Dataset name
    CO_BASE_URL: string
      Description: Base URL for the API
    CO_CLIENT_ID: string
      Description: Client ID
    CO_CLIENT_SECRET: string
      Description: Client secret
```

--------------------------------

### CLI: Performance Benchmarks for Bloom Filter Operations

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

These commands provide benchmarks for the performance of Bloom filter operations, specifically `check` and `encode`. They show how to measure the execution time for both debug and release builds of the CLI tool, highlighting the significant performance improvements achieved with optimized release builds for large datasets.

```bash
time target/debug/pa-cli check -input ~/Downloads/eth_all.csv
```

```bash
time target/release/pa-cli encode -input ~/Downloads/eth_all.csv
```

--------------------------------

### CLI: Encode Addresses into Bloom Filter

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

This CLI command is used to build a Bloom filter from a set of addresses. The `-n` flag specifies the expected number of entries, and the `-p` flag sets the desired false positive rate. The output is a `bloomfilter.gob` file containing the encoded Bloom filter.

```bash
go run cmd/cli/main.go encode -n 1000000 -p 0.000001
```

--------------------------------

### Inspect Bloom Filter Statistics with pa-cli

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command provides statistical information about a generated bloom filter. The `--json` flag outputs the statistics in a machine-readable JSON format.

```bash
pa-cli inspect -f ./bloomfilter.gob --json
```

--------------------------------

### Inspect Bloom Filter Statistics and Metadata with pa-cli

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command provides detailed insights into the statistics and metadata of a bloom filter. It supports secure access via decryption and signing keys, and offers an option to output the inspection results in a structured JSON format for programmatic parsing.

```bash
./pa-cli inspect --filename ./bloomfilter.gob \
    --decrypt-key ./securedata/testdata/privkey.asc --signing-key ./securedata/testdata/pubkey.asc \
    --decrypt-key-passphrase "123456" \
    --json
```

--------------------------------

### Encode Small Bloom Filter with Go CLI

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

Demonstrates how to use the Go command-line interface to encode a small Bloom filter. This command specifies the number of elements (-n) and the desired false positive probability (-p), resulting in a filter approximately 450KB in size.

```Bash
go run cmd/cli/main.go encode -n 100000 -p 0.0000001
```

--------------------------------

### Inspect Bloom Filter Statistics with LLM Bot Caller Mode

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Shows how to retrieve the current statistics of the Bloom filter using the `/inspect` endpoint via `curl`. The `__llm_bot_caller__: 1` header ensures that the response includes a `__llm_explanations__data_dictionary__` providing detailed explanations for each statistical metric, aiding LLM-driven analysis.

```bash
curl http://localhost:8080/inspect -H "__llm_bot_caller__: 1" | jq
```

```json
{
  "stats": {
    "k": 7,
    "m": 26168,
    "n": 1007,
    "estimated_capacity": 3738,
    "false_positive_rate": 4.094744429579303e-05
  },
  "last_update": "2025-03-15T22:54:07Z",
  "__llm_explanations__data_dictionary__": {
    "estimated_capacity": "Estimated maximum capacity of the Bloom filter before exceeding the false positive probability threshold",
    "false_positive_rate": "Estimated probability that the filter will incorrectly report that an element is in the set when it is not",
    "k": "Number of hash functions used in the Bloom filter",
    "last_update": "ISO 8601 timestamp of when the Bloom filter was last updated",
    "m": "Size of the bit array in the Bloom filter",
    "n": "Number of elements added to the Bloom filter"
  }
}
```

--------------------------------

### Batch Check Multiple Addresses via gRPC

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Illustrates how to use `grpcurl` to call the `BatchCheckAddresses` gRPC method, allowing for the simultaneous checking of multiple Ethereum addresses against the Bloom filter. This method is efficient for verifying the presence of a list of addresses.

```bash
grpcurl -plaintext -d '{"addresses": ["0x742d35Cc6634C0532925a3b844Bc454e4438f44e", "0x96244d83dc15d36847c35209bbdc5bdde9bec3d8"]}' localhost:9090 proto.ECSd.BatchCheckAddresses
```

--------------------------------

### Check Single Address via gRPC

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Demonstrates how to use `grpcurl` to invoke the `CheckAddress` gRPC method, passing an Ethereum address to determine if it is present in the Bloom filter. The response indicates whether the address is found (`isSet`).

```bash
grpcurl -plaintext -d '{"address": "0x96244d83dc15d36847c35209bbdc5bdde9bec3d8"}' localhost:9090 proto.ECSd.CheckAddress
```

```json
{
  "address": "0x742d35Cc6634C0532925a3b844Bc454e4438f44e",
  "isSet": true
}
```

--------------------------------

### ECSD Docker Container Environment Variables Reference

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README_DOCKERFILE.md

This section details the configurable environment variables for the ECSD Docker container, categorized by their function. These variables control bloom filter behavior, server ports, rate limiting, and essential service configurations.

```APIDOC
Bloom Filter Configuration:
  CO_FILENAME: string (default: "bloomfilter.gob") - Bloom filter file path
  CO_DECRYPT_KEY: string (default: "") - Path to decrypt key file
  CO_SIGNING_KEY: string (default: "") - Path to signing key file
  CO_DECRYPT_KEY_PASSPHRASE: string (default: "") - Decrypt key passphrase
  CO_HASH: boolean (default: false) - If true, the bloom filter stores address hashes instead of addresses strings

Server Configuration:
  CO_HTTP_PORT: integer (default: 8080) - HTTP port
  CO_GRPC_PORT: integer (default: 9090) - gRPC port
  CO_RATE_LIMIT: integer (default: 20) - Rate limit
  CO_BURST: integer (default: 5) - Burst limit

Service Configuration (All Required):
  CO_CHAIN: string - Chain name
  CO_DATASET: string - Dataset name
  CO_BASE_URL: string - Base URL for the API
  CO_CLIENT_ID: string - Client ID
  CO_CLIENT_SECRET: string - Client secret
```

--------------------------------

### Auto-Reload Bloom Filter on File Change (Go)

Source: https://github.com/cipherowl-ai/openecs/blob/main/README.md

This Go snippet demonstrates how to set up automatic reloading of a Bloom filter from a file whenever the file changes. It uses a `FileWatcherNotifier` to detect changes and a `ReloadManager` to manage the reloading process, ensuring the filter remains up-to-date with a specified minimum reload interval.

```go
// Create a file watcher notifier, that will reload the Bloom filter when the file changes.  
// But never more than once every 2 seconds.
addressHandler := &address.EVMAddressHandler{}
store, _ := NewBloomFilterStoreFromFile(filePath, addressHandler)
notifier, _ := reload.NewFileWatcherNotifier(filePath, 2*time.Second)

// Create the ReloadManager with the notifier.
manager := reload.NewReloadManager(store, notifier)
manager.Start(context.Background())
defer manager.Stop()
```

--------------------------------

### Retrieve Bloom Filter Statistics via gRPC

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Explains how to call the `InspectFilter` gRPC method using `grpcurl` to obtain the current statistics of the Bloom filter, such as `k`, `m`, `n`, estimated capacity, and false positive rate.

```bash
grpcurl -plaintext localhost:9090 proto.ECSd.InspectFilter
```

```json
{
  "k": 7,
  "m": "26168",
  "n": "1007",
  "estimatedCapacity": "3738",
  "falsePositiveRate": 4.094744429579303e-05,
  "lastUpdate": "2025-03-16T05:21:54Z"
}
```

--------------------------------

### Add Single Ethereum Address to an Existing Bloom Filter

Source: https://github.com/cipherowl-ai/openecs/blob/main/cmd/cli/README.md

This command allows the addition of a single, specified Ethereum address to an already existing bloom filter file. The address can be added directly or as its hash value, updating the filter in place.

```bash
./pa-cli add --input ./bloomfilter.gob --address 0x1234567890123456789012345678901234567890 --output ./bloomfilter.gob --hash
```

--------------------------------

### Bloom Filter Service API Endpoints

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Details the available HTTP and gRPC endpoints for interacting with the Bloom filter service, including single address checks, batch checks, and filter inspection.

```APIDOC
HTTP Endpoints:
  GET /check?address=[address]
    Description: Check if an address is in the Bloom filter
    Parameters:
      address: string (query) - The address to check
  POST /batch-check
    Description: Check multiple addresses (max 100)
    Body: {"addresses":["addr1","addr2"]}
    Parameters:
      addresses: array of string (body) - List of addresses to check
  GET /inspect
    Description: Get Bloom filter statistics and last update time
  GET /health
    Description: Check service health

gRPC Services:
  CheckAddress
    Description: Check single address
  BatchCheckAddresses
    Description: Check multiple addresses
  InspectFilter
    Description: Get filter statistics
```

--------------------------------

### Health Check for Bloom Filter Service

Source: https://github.com/cipherowl-ai/openecs/blob/main/ecsd/README.md

Demonstrates how to perform a health check on the Bloom filter service's HTTP endpoint using `curl`.

```bash
curl "http://localhost:8080/health"
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.