### Start IceGate Development Environment

Source: https://github.com/icegatetech/docs/blob/main/en/development/contributing.md

Start the development environment for IceGate using either Skaffold with a local Kubernetes setup or Docker Compose for hot-reloading.

```bash
skaffold dev
```

```bash
make dev
```

--------------------------------

### Install Rust

Source: https://github.com/icegatetech/docs/blob/main/en/development/setup.md

Installs Rust using the official script and sets up the environment. Verify the installed version.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
rustc --version  # Should be >= 1.92.0
```

--------------------------------

### Start Development Environment with Docker Compose

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Starts the development environment using Docker Compose with hot-reloading capabilities. Requires Docker and Docker Compose to be installed.

```bash
# Recommended: Skaffold with local Kubernetes
# skaffold dev

# Alternative: Docker Compose with hot-reload
make dev
```

--------------------------------

### Start Development Environment

Source: https://github.com/icegatetech/docs/blob/main/en/getting-started/configuration.md

Commands to start the core services with hot-reloading, in release mode, or with load generation and analytics.

```bash
# Start core services with hot-reload
make dev
```

```bash
# Start core services in release mode
make run-core-release
```

```bash
# Start with load generator
make run-load-release
```

```bash
# Start with monitoring (Jaeger, Prometheus, Grafana)
make run-analytics-release
```

--------------------------------

### Install Dependencies and Serve Locally

Source: https://github.com/icegatetech/docs/blob/main/README.md

Use these commands to install project dependencies and then build and serve the documentation locally for development. Open http://localhost:8080 to view the site.

```bash
# Install dependencies
npm install

# Build and serve locally
npm run serve
# Open http://localhost:8080
```

--------------------------------

### Complete Query Service Configuration Example

Source: https://github.com/icegatetech/docs/blob/main/en/getting-started/configuration.md

A full example of a query.yaml configuration file, demonstrating settings for catalog, storage, engine, queue, and observability services.

```yaml
catalog:
  backend: !rest
    uri: http://nessie:19120/iceberg
  warehouse: s3://warehouse/
  properties:
    prefix: main
  cache:
    memory_size_mb: 1024
    disk_dir: /tmp/icegate/cache
    disk_size_mb: 4096

storage:
  backend: !s3
    bucket: warehouse
    region: us-east-1
    endpoint: http://minio:9000

engine:
  batch_size: 8192
  target_partitions: 4
  catalog_name: iceberg
  refresh_interval_secs: 15
  max_age_secs: 30
  wal_query_enabled: false
  wal_metadata_size_hint: 65536

queue:
  common:
    base_path: s3://queue/

loki:
  enabled: true
  host: 0.0.0.0
  port: 3100

prometheus:
  enabled: true
  host: 0.0.0.0
  port: 9090

tempo:
  enabled: true
  host: 0.0.0.0
  port: 3200

metrics:
  enabled: true
  host: 0.0.0.0
  port: 9091
  path: /metrics

tracing:
  enabled: true
  service_name: icegate-query
  otlp_endpoint: http://jaeger:4317
  sample_ratio: 1.0

```

--------------------------------

### IceGate Catalog Configuration Example

Source: https://github.com/icegatetech/docs/blob/main/en/operations/troubleshooting.md

An example of the catalog configuration block in IceGate, specifying the Nessie REST backend and warehouse location. Ensure this matches your setup.

```yaml
catalog:
  backend: !rest
    uri: http://nessie:19120/iceberg
  warehouse: s3://warehouse/
```

--------------------------------

### Install IceGate with Helm Charts

Source: https://github.com/icegatetech/docs/blob/main/en/operations/deployment.md

Installs IceGate on Kubernetes using Helm charts from a local directory.

```bash
helm install icegate ./config/helm/icegate
```

--------------------------------

### Iceberg Table Partitioning Example

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Demonstrates identity partitioning on tenant_id for multi-tenancy in Iceberg tables. This setup is used for logs, spans, events, and metrics.

```sql
partitioning = ARRAY["tenant_id", "account_id", "day(timestamp)"]
```

--------------------------------

### Install IceGate from Local Charts using Helm

Source: https://github.com/icegatetech/docs/blob/main/en/getting-started/installation.md

Install the IceGate Helm chart from a local clone of the repository. This is useful for development or when offline installation is required.

```bash
git clone https://github.com/icegatetech/icegate.git
helm install icegate ./icegate/config/helm/icegate \
  --namespace icegate \
  --create-namespace \
  -f values.yaml
```

--------------------------------

### Install Rust Toolchain

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Commands to install Rust via rustup and verify the installation.

```bash
# Install via rustup
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Verify installation
rustc --version
cargo --version
```

--------------------------------

### Install Skaffold on Linux

Source: https://github.com/icegatetech/docs/blob/main/en/development/setup.md

Installs Skaffold on Linux by downloading the binary, making it executable, and moving it to the system's PATH.

```bash
# Linux
curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-linux-amd64
chmod +x skaffold && sudo mv skaffold /usr/local/bin/
```

--------------------------------

### Install IceGate Helm Chart

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Install the IceGate Helm chart from a local directory. Use custom values files or set individual parameters to customize the deployment.

```bash
helm install icegate ./config/helm/icegate
```

```bash
helm install icegate ./config/helm/icegate \
  -f my-values.yaml \
  --set storage.bucket=my-warehouse
```

--------------------------------

### Install IceGate with Custom Helm Values

Source: https://github.com/icegatetech/docs/blob/main/en/operations/deployment.md

Installs IceGate on Kubernetes using Helm charts with custom values specified in a file and via --set.

```bash
helm install icegate ./config/helm/icegate \
  -f my-values.yaml \
  --set storage.bucket=my-warehouse
```

--------------------------------

### Loki API Query with Partition Pruning

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Example of querying the Loki API with `start` and `end` parameters to leverage Iceberg partition pruning. Ensure `tenant_id` is correctly set in headers.

```http
curl -G http://localhost:3100/loki/api/v1/query_range \
  --data-urlencode 'query={service_name="api"}' \
  --data-urlencode 'start=1704067200' \
  --data-urlencode 'end=1704153600' \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Tempo Search Traces Example

Source: https://github.com/icegatetech/docs/blob/main/llms.txt

Examples of how to search for traces based on tags and duration.

```APIDOC
## Search Traces

### Description
Searches for traces based on specified criteria such as service name or minimum duration.

### Method
GET

### Endpoint
/api/search

### Query Parameters
- **tags** (string) - Optional - Filter traces by tags (e.g., `service.name=api-service`). Multiple tags can be specified.
- **minDuration** (string) - Optional - Filter traces that took at least this long to complete (e.g., `500ms`).
- **limit** (integer) - Optional - The maximum number of traces to return.

### Headers
- **X-Scope-OrgID** (string) - Required - The tenant ID for data isolation.

### Request Example (by service name)
```bash
curl -G http://localhost:3200/api/search \
  --data-urlencode 'tags=service.name=api-service' \
  --data-urlencode 'limit=10' \
  -H "X-Scope-OrgID: my-tenant"
```

### Request Example (by minimum duration)
```bash
curl -G http://localhost:3200/api/search \
  --data-urlencode 'tags=service.name=api-service' \
  --data-urlencode 'minDuration=500ms' \
  --data-urlencode 'limit=10' \
  -H "X-Scope-OrgID: my-tenant"
```
```

--------------------------------

### Install Rust via rustup

Source: https://github.com/icegatetech/docs/blob/main/en/development/building.md

Installs Rust and Cargo using the official rustup script. Verify the installation by checking the versions of `rustc` and `cargo`.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

rustc --version
cargo --version
```

--------------------------------

### GET /loki/api/v1/explain

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/loki.md

Get query execution plan.

```APIDOC
## GET /loki/api/v1/explain

### Description
Get query execution plan (IceGate extension).

### Method
GET

### Endpoint
/loki/api/v1/explain

### Parameters
#### Query Parameters
- **query** (string) - Required - LogQL query
```

--------------------------------

### Install IceGen Load Testing Tool

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/otlp.md

Clones the repository and builds the IceGen binary from source.

```bash
git clone https://github.com/icegatetech/icegen.git
cd icegen
cargo build --release
```

--------------------------------

### Loki Discover Labels Example

Source: https://github.com/icegatetech/docs/blob/main/llms.txt

Examples for discovering labels and their values in Loki.

```APIDOC
## Discover Labels

### Description
Endpoints to discover available labels and their corresponding values.

### Method
GET

### Endpoint
/loki/api/v1/labels

### Description
Lists all available label names.

### Request Example
```bash
curl http://localhost:3100/loki/api/v1/labels \
  -H "X-Scope-OrgID: my-tenant"
```

---

### Method
GET

### Endpoint
/loki/api/v1/label/{name}/values

### Description
Retrieves all values for a specific label name.

### Parameters
#### Path Parameters
- **name** (string) - Required - The name of the label to query values for.

### Request Example
```bash
curl http://localhost:3100/loki/api/v1/label/service_name/values \
  -H "X-Scope-OrgID: my-tenant"
```
```

--------------------------------

### Verify Service Installation

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Commands to check pod status, establish port forwarding, and test service readiness.

```bash
# Check pods are running
kubectl get pods -n icegate

# Port-forward to query service
kubectl port-forward -n icegate svc/icegate-query 3100:3100

# Test readiness
curl http://localhost:3100/ready
```

--------------------------------

### Install IceGate from OCI Registry using Helm

Source: https://github.com/icegatetech/docs/blob/main/en/getting-started/installation.md

Use this command to install the IceGate Helm chart directly from the OCI registry. Ensure you have Helm 3 and Kubernetes configured.

```bash
helm install icegate oci://ghcr.io/icegatetech/charts/icegate \
  --version 0.1.0 \
  --namespace icegate \
  --create-namespace \
  -f values.yaml
```

--------------------------------

### Interact with gRPC Services via grpcurl

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/otlp.md

Examples for listing services and exporting logs using grpcurl.

```bash
# Check available services
grpcurl -plaintext localhost:4317 list

# Send logs (requires proto file)
grpcurl -plaintext \
  -H "x-scope-orgid: my-tenant" \
  -d '{"resourceLogs": [...]}'
  localhost:4317 \
  opentelemetry.proto.collector.logs.v1.LogsService/Export
```

--------------------------------

### Explain Query Execution

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/loki.md

Get the execution plan for a LogQL query.

```bash
curl -G http://localhost:3100/loki/api/v1/explain \
  --data-urlencode 'query=count_over_time({service_name="api-service"}[5m])' \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Install IceGate via Helm

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Commands to deploy IceGate using either the OCI registry or a local chart clone.

```bash
helm install icegate oci://ghcr.io/icegatetech/charts/icegate \
  --version 0.1.0 \
  --namespace icegate \
  --create-namespace \
  -f values.yaml
```

```bash
git clone https://github.com/icegatetech/icegate.git
helm install icegate ./icegate/config/helm/icegate \
  --namespace icegate \
  --create-namespace \
  -f values.yaml
```

--------------------------------

### Rust Code Documentation Example

Source: https://github.com/icegatetech/docs/blob/main/en/development/contributing.md

Example of documenting a public Rust function using doc comments. Includes arguments, return values, and error handling.

```rust
/// Parses a LogQL query string into an AST.
///
/// # Arguments
///
/// * `query` - The LogQL query string
///
/// # Returns
///
/// The parsed LogQL expression or an error
pub fn parse(query: &str) -> Result<LogQLExpr> {
    // ...
}
```

--------------------------------

### Validate LogQL Syntax Example

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

This example demonstrates correct LogQL syntax, including using braces for labels and quotes for string values. It also highlights common unsupported features.

```logql
{service_name="api"}
```

--------------------------------

### Install Structurizr CLI and PlantUML

Source: https://github.com/icegatetech/docs/blob/main/c4/README.md

Prerequisites for local development on macOS or via Docker.

```bash
# Install structurizr-cli and plantuml (macOS)
brew install structurizr-cli plantuml

# Or use Docker for Structurizr Lite
docker pull structurizr/lite
```

--------------------------------

### Launch Interactive Editor

Source: https://github.com/icegatetech/docs/blob/main/c4/README.md

Command to start the Structurizr Lite server for live editing.

```bash
make lite
# Open http://localhost:8080
```

--------------------------------

### Install OpenSSL Development Libraries on Ubuntu/Debian

Source: https://github.com/icegatetech/docs/blob/main/en/development/building.md

Installs OpenSSL development libraries and `pkg-config` on Ubuntu/Debian systems. This is necessary for compiling software that links against OpenSSL.

```bash
apt install libssl-dev pkg-config
```

--------------------------------

### API: Available Labels Example

Source: https://github.com/icegatetech/docs/blob/main/en/guides/querying.md

Use curl to fetch a list of all available labels from the Loki API.

```bash
curl http://localhost:3100/loki/api/v1/labels \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Get Series Endpoint Example

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/prometheus.md

Retrieve series that match the provided selectors. Optional start and end timestamps can be specified. The X-Scope-OrgID header is required.

```bash
curl -G http://localhost:9090/api/v1/series \
  --data-urlencode 'match[]={__name__=~"http_.*"}' \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Configure Apache Iceberg Catalog

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

YAML configuration examples for different Iceberg catalog backends.

```yaml
catalog:
  backend: !rest
    uri: http://nessie:19120/iceberg
  warehouse: s3://warehouse/
  properties:
    prefix: main
```

```yaml
catalog:
  backend: !rest
    uri: http://nessie:19120/iceberg
  warehouse: s3://warehouse/
  properties:
    prefix: main
```

```yaml
catalog:
  backend: !s3tables
    table_bucket_arn: arn:aws:s3tables:us-east-1:123456789012:bucket/my-tables
  warehouse: s3://warehouse/
```

```yaml
catalog:
  backend: !glue
    catalog_id: "123456789012"
  warehouse: s3://warehouse/
```

```yaml
catalog:
  backend: !memory
  warehouse: /tmp/icegate/warehouse
```

--------------------------------

### Tempo Get Trace by ID Example

Source: https://github.com/icegatetech/docs/blob/main/llms.txt

Example of how to retrieve a specific trace by its ID using curl.

```APIDOC
## Get a Trace by ID

### Description
Retrieves a single trace identified by its unique trace ID.

### Method
GET

### Endpoint
/api/traces/{traceID}

### Parameters
#### Path Parameters
- **traceID** (string) - Required - The unique identifier of the trace.

### Headers
- **X-Scope-OrgID** (string) - Required - The tenant ID for data isolation.

### Request Example
```bash
curl http://localhost:3200/api/traces/5B8EFFF798038103D269B633813FC60C \
  -H "X-Scope-OrgID: my-tenant"
```
```

--------------------------------

### Build and Serve Documentation

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Commands to build and serve the user documentation locally.

```bash
# Build docs
cd docs && npm run build

# Serve docs locally
cd docs && npm run serve
```

--------------------------------

### Build and Serve User Documentation

Source: https://github.com/icegatetech/docs/blob/main/en/development/contributing.md

Build the user documentation using npm scripts within the 'docs' directory and serve it locally for preview. Requires Node.js and npm.

```bash
cd docs && npm run build

cd docs && npm run serve
```

--------------------------------

### Send Test Traces

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Send a sample trace via curl to verify the setup.

```APIDOC
## POST /v1/traces

### Description
Send a sample trace to the API.

### Method
POST

### Endpoint
http://localhost:4318/v1/traces

### Headers
- **Content-Type**: application/json
- **X-Scope-OrgID**: my-tenant

### Request Body
```json
{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {"key": "service.name", "value": {"stringValue": "api-gateway"}}
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "traceId": "5B8EFFF798038103D269B633813FC60C",
              "spanId": "EEE19B7EC3C1B174",
              "name": "GET /api/orders",
              "kind": 2,
              "startTimeUnixNano": "1704067200000000000",
              "endTimeUnixNano": "1704067200150000000",
              "status": {"code": 1},
              "attributes": [
                {"key": "http.method", "value": {"stringValue": "GET"}},
                {"key": "http.status_code", "value": {"intValue": "200"}},
                {"key": "http.url", "value": {"stringValue": "/api/orders"}}
              ]
            },
            {
              "traceId": "5B8EFFF798038103D269B633813FC60C",
              "spanId": "AABB19B7EC3C1B22",
              "parentSpanId": "EEE19B7EC3C1B174",
              "name": "SELECT orders",
              "kind": 3,
              "startTimeUnixNano": "1704067200050000000",
              "endTimeUnixNano": "1704067200120000000",
              "status": {"code": 1},
              "attributes": [
                {"key": "db.system", "value": {"stringValue": "postgresql"}},
                {"key": "db.statement", "value": {"stringValue": "SELECT * FROM orders WHERE user_id = $1"}}
              ]
            }
          ]
        }
      ]
    }
  ]
}
```
```

--------------------------------

### Send Test Traces

Source: https://github.com/icegatetech/docs/blob/main/en/cookbooks/traces-end-to-end.md

Send a sample trace via curl to verify the setup.

```APIDOC
## POST /v1/traces

### Description
Send a sample trace to the API.

### Method
POST

### Endpoint
http://localhost:4318/v1/traces

### Headers
- **Content-Type** (string) - Required - application/json
- **X-Scope-OrgID** (string) - Required - Your organization ID

### Request Body
- **resourceSpans** (array) - Required - Contains resource and scope spans.
  - **resource** (object) - Resource information.
    - **attributes** (array) - Resource attributes.
      - **key** (string) - Attribute key.
      - **value** (object) - Attribute value.
        - **stringValue** (string) - String value of the attribute.
  - **scopeSpans** (array) - Contains spans for a specific scope.
    - **spans** (array) - List of spans.
      - **traceId** (string) - Unique identifier for the trace.
      - **spanId** (string) - Unique identifier for the span.
      - **name** (string) - Operation name.
      - **kind** (int) - Type of span (e.g., 2 for Server, 3 for Client).
      - **startTimeUnixNano** (string) - Start time in Unix nanoseconds.
      - **endTimeUnixNano** (string) - End time in Unix nanoseconds.
      - **status** (object) - Span status.
        - **code** (int) - Status code (e.g., 1 for OK, 2 for Error).
      - **attributes** (array) - Span attributes.
        - **key** (string) - Attribute key.
        - **value** (object) - Attribute value.
          - **stringValue** (string) - String value of the attribute.
          - **intValue** (int) - Integer value of the attribute.

### Request Example
```json
{
  "resourceSpans": [{
    "resource": {
      "attributes": [
        {"key": "service.name", "value": {"stringValue": "api-gateway"}}
      ]
    },
    "scopeSpans": [{
      "spans": [
        {
          "traceId": "5B8EFFF798038103D269B633813FC60C",
          "spanId": "EEE19B7EC3C1B174",
          "name": "GET /api/orders",
          "kind": 2,
          "startTimeUnixNano": "1704067200000000000",
          "endTimeUnixNano": "1704067200150000000",
          "status": {"code": 1},
          "attributes": [
            {"key": "http.method", "value": {"stringValue": "GET"}},
            {"key": "http.status_code", "value": {"intValue": "200"}},
            {"key": "http.url", "value": {"stringValue": "/api/orders"}}
          ]
        },
        {
          "traceId": "5B8EFFF798038103D269B633813FC60C",
          "spanId": "AABB19B7EC3C1B22",
          "parentSpanId": "EEE19B7EC3C1B174",
          "name": "SELECT orders",
          "kind": 3,
          "startTimeUnixNano": "1704067200050000000",
          "endTimeUnixNano": "1704067200120000000",
          "status": {"code": 1},
          "attributes": [
            {"key": "db.system", "value": {"stringValue": "postgresql"}},
            {"key": "db.statement", "value": {"stringValue": "SELECT * FROM orders WHERE user_id = $1"}}
          ]
        }
      ]
    }]
  }]
}
```
```

--------------------------------

### Get All Labels Endpoint Example

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/prometheus.md

Retrieve all available label names from the metrics. Requires the X-Scope-OrgID header for authentication.

```bash
curl http://localhost:9090/api/v1/labels \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Get Label Values Endpoint Example

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/prometheus.md

Fetch all values for a specific label. Replace 'service_name' with the desired label. The X-Scope-OrgID header is mandatory.

```bash
curl http://localhost:9090/api/v1/label/service_name/values \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Run Development Server and Linter

Source: https://github.com/icegatetech/docs/blob/main/CONTRIBUTING.md

Builds and serves the documentation locally for preview. Validates documentation changes using the linter.

```bash
npm run serve
# Open http://localhost:8080

npm run lint
```

--------------------------------

### Run Query Service

Source: https://github.com/icegatetech/docs/blob/main/en/development/building.md

Starts the query service using a specific configuration file. Requires the 'query' binary to be built.

```bash
cargo run --bin query -- run -c config/docker/query.yaml
```

--------------------------------

### Configure Catalog Backends

Source: https://github.com/icegatetech/docs/blob/main/en/getting-started/configuration.md

Examples for different catalog backend types including REST, S3 Tables, Glue, and In-Memory.

```yaml
catalog:
  backend: !rest
    uri: http://nessie:19120/iceberg
  warehouse: s3://warehouse/
  properties:
    prefix: main
```

```yaml
catalog:
  backend: !s3tables
    table_bucket_arn: arn:aws:s3tables:us-east-1:123456789012:bucket/my-tables
  warehouse: s3://warehouse/
```

```yaml
catalog:
  backend: !glue
    catalog_id: "123456789012"
  warehouse: s3://warehouse/
```

```yaml
catalog:
  backend: !memory
  warehouse: /tmp/icegate/warehouse
```

--------------------------------

### GET /loki/api/v1/series

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/loki.md

Get label sets matching selectors.

```APIDOC
## GET /loki/api/v1/series

### Description
Get label sets matching selectors.

### Method
GET

### Endpoint
/loki/api/v1/series

### Parameters
#### Query Parameters
- **match[]** (string) - Required - Log stream selector(s)
- **start** (int) - Optional - Start timestamp
- **end** (int) - Optional - End timestamp
```

--------------------------------

### Install Skaffold on macOS

Source: https://github.com/icegatetech/docs/blob/main/en/development/setup.md

Installs Skaffold using Homebrew for macOS users.

```bash
# macOS
brew install skaffold
```

--------------------------------

### Configure IceGate Catalog Backends

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Example values.yaml configurations for different catalog backends including REST (Nessie), AWS Glue, and AWS S3 Tables.

```yaml
catalog:
  backend: rest
  rest:
    uri: http://nessie:19120/iceberg
  warehouse: "s3://warehouse/"

storage:
  s3:
    bucket: warehouse
    region: us-east-1
    endpoint: "http://minio:9000"

queue:
  common:
    basePath: "s3://queue/"

aws:
  existingSecret: icegate-aws-credentials
  region: us-east-1
```

```yaml
catalog:
  backend: glue
  glue:
    catalogId: "123456789012"
  warehouse: "s3://my-bucket/warehouse/"

storage:
  s3:
    bucket: my-bucket
    region: eu-central-1

aws:
  existingSecret: icegate-aws-credentials
  region: eu-central-1
```

```yaml
catalog:
  backend: s3tables
  s3tables:
    tableBucketArn: "arn:aws:s3tables:eu-central-1:123456789012:bucket/my-tables"

storage:
  s3:
    region: eu-central-1

aws:
  existingSecret: icegate-aws-credentials
  region: eu-central-1
```

--------------------------------

### otel-log-generator Examples

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Examples of using the otel-log-generator to send OTLP data with various configurations.

```APIDOC
## otel-log-generator Examples

### Send via gRPC with 8 tenants and 20 concurrent workers
```bash
otel-log-generator otel \
  --endpoint http://localhost:4317 \
  --transport grpc \
  --tenant-count 8 \
  --count 1000 \
  --concurrency 20
```

### Continuous mode with protobuf encoding
```bash
otel-log-generator otel \
  --endpoint http://localhost:4318/v1/logs \
  --use-protobuf \
  --continuous \
  --message-interval-ms 100 \
  --concurrency 10
```

### Aggregated messages (5 records per request)
```bash
otel-log-generator otel \
  --endpoint http://localhost:4318/v1/logs \
  --records-per-message 5 \
  --count 100
```

### Test error handling with 10% invalid records
```bash
otel-log-generator otel \
  --endpoint http://localhost:4318/v1/logs \
  --invalid-record-percent 10.0 \
  --count 100
```

### IceGen Parameters

| Parameter | Default | Description |
|-----------|---------|-------------|
| `--endpoint` | — | OTLP endpoint URL |
| `--transport` | `http` | Transport: `http` or `grpc` |
| `--use-protobuf` | `false` | Use protobuf encoding (HTTP only) |
| `--count` | `1` | Number of messages to send |
| `--concurrency` | `1` | Number of concurrent workers |
| `--message-interval-ms` | `0` | Delay between messages (ms) |
| `--records-per-message` | `1` | Log records per message |
| `--continuous` | `false` | Run continuously |
| `--tenant-id` | `default` | Tenant ID |
| `--tenant-count` | `1` | Number of random tenants |
| `--invalid-record-percent` | `0.0` | Percentage of invalid records |

```

--------------------------------

### Loki Range Query Example

Source: https://github.com/icegatetech/docs/blob/main/llms.txt

Example of how to perform a range query using curl.

```APIDOC
## Range Query

### Description
Performs a range query using LogQL to retrieve logs within a specified time frame.

### Method
GET

### Endpoint
/loki/api/v1/query_range

### Query Parameters
- **query** (string) - Required - The LogQL query string.
- **start** (string) - Required - The start timestamp (Unix epoch).
- **end** (string) - Required - The end timestamp (Unix epoch).
- **limit** (integer) - Optional - The maximum number of results to return.

### Headers
- **X-Scope-OrgID** (string) - Required - The tenant ID for data isolation.

### Request Example
```bash
curl -G http://localhost:3100/loki/api/v1/query_range \
  --data-urlencode 'query={service_name="api-service"}' \
  --data-urlencode 'start=1704067200' \
  --data-urlencode 'end=1704153600' \
  --data-urlencode 'limit=100' \
  -H "X-Scope-OrgID: my-tenant"
```
```

--------------------------------

### Apply Kustomize Overlays

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Commands to apply environment-specific Kustomize configurations or initiate development mode.

```bash
# Apply an overlay directly
kubectl apply -k config/kustomize/overlays/aws-glue

# Or use Skaffold for development (see Development Setup)
skaffold dev
```

--------------------------------

### Create and Apply a Custom Kustomize Overlay

Source: https://github.com/icegatetech/docs/blob/main/en/getting-started/installation.md

Steps to create a new custom Kustomize overlay by copying an existing one, modifying its configuration files, and then applying it to your cluster.

```bash
cp -r config/kustomize/overlays/orbstack config/kustomize/overlays/my-env
vi config/kustomize/overlays/my-env/values-icegate.yaml
vi config/kustomize/overlays/my-env/secret-aws.yaml
kubectl apply -k config/kustomize/overlays/my-env
```

--------------------------------

### Build Documentation Commands

Source: https://github.com/icegatetech/docs/blob/main/README.md

These npm scripts are available for building the documentation in various ways, including building all languages, specific languages, serving locally, linting, and cleaning the build directory.

```bash
npm run build
```

```bash
npm run build:en
```

```bash
npm run build:fr
```

```bash
npm run build:ru
```

```bash
npm run serve
```

```bash
npm run lint
```

```bash
npm run clean
```

--------------------------------

### Loki Metric Query Response Example

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Example JSON response structure for a successful metric query from the Loki API.

```json
{
  "status": "success",
  "data": {
    "resultType": "matrix",
    "result": [
      {
        "metric": {
          "service_name": "api-service"
        },
        "values": [
          [1704067200, "42"],
          [1704067500, "38"]
        ]
      }
    ]
  }
}
```

--------------------------------

### Loki Log Query Response Example

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Example JSON response structure for a successful log query from the Loki API.

```json
{
  "status": "success",
  "data": {
    "resultType": "streams",
    "result": [
      {
        "stream": {
          "service_name": "api-service",
          "severity_text": "INFO"
        },
        "values": [
          ["1704067200000000000", "Request processed successfully"]
        ]
      }
    ]
  }
}
```

--------------------------------

### Run IceGate with Skaffold

Source: https://github.com/icegatetech/docs/blob/main/en/development/setup.md

Starts the IceGate development environment using Skaffold with different profiles. The default profile includes MinIO and Nessie.

```bash
# Default profile (local k8s with MinIO + Nessie)
skaffold dev

# OrbStack profile
skaffold dev -p orbstack

# AWS Glue profile (pushes images to registry)
skaffold dev -p aws-glue

# External S3 profile
skaffold dev -p k3s-external-s3
```

--------------------------------

### Install OpenSSL on macOS

Source: https://github.com/icegatetech/docs/blob/main/en/development/building.md

Installs the OpenSSL library on macOS using Homebrew. This is often required to resolve linking errors for dependencies that rely on OpenSSL.

```bash
brew install openssl
```

--------------------------------

### LogQL Query Examples

Source: https://github.com/icegatetech/docs/blob/main/en/guides/querying.md

Examples of LogQL queries for various use cases, including recent errors, error rate by service, and log volume trends.

```APIDOC
## LogQL Query Examples

### Recent Errors
```logql
{service_name="api-service", severity_text="ERROR"}
```

### Error Rate by Service
```logql
sum by (service_name) (
  rate({severity_text="ERROR"}[5m])
)
```

### Log Volume Trends
```logql
sum(count_over_time({job="app"}[1h]))
```
```

--------------------------------

### Server and Routes Setup in Rust

Source: https://github.com/icegatetech/docs/blob/main/en/development/patterns.md

Defines the server state, including shared resources, and the main `run` function for server initialization. Separates routing logic into a dedicated module.

```rust
#[derive(Clone)]
pub struct ModuleState {
    pub resource: Arc<SharedResource>,
}

pub async fn run(
    resource: Arc<SharedResource>,
    config: ModuleConfig,
    cancel_token: CancellationToken,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    let addr: SocketAddr = format!("{}:{}", config.host, config.port).parse()?;
    let state = ModuleState { resource };
    let app = super::routes::routes(state);

    let listener = tokio::net::TcpListener::bind(addr).await?;
    axum::serve(listener, app)
        .with_graceful_shutdown(async move { cancel_token.cancelled().await })
        .await?;
    Ok(())
}
```

```rust
pub fn routes(state: ModuleState) -> Router {
    Router::new()
        .route("/v1/endpoint", post(handlers::endpoint_handler))
        .route("/health", get(handlers::health))
        .with_state(state)
}
```

--------------------------------

### Explaining Query Execution Plans with Loki API

Source: https://github.com/icegatetech/docs/blob/main/en/guides/performance-tuning.md

Demonstrates how to use the explain endpoint of the Loki API to retrieve query execution plans.

```bash
curl -G http://localhost:3100/loki/api/v1/explain \
  --data-urlencode 'query={service_name="api"} |= "error"' \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Enable S3 Versioning

Source: https://github.com/icegatetech/docs/blob/main/en/guides/data-retention.md

Configure bucket versioning for point-in-time recovery of warehouse storage.

```bash
aws s3api put-bucket-versioning \
  --bucket icegate-warehouse \
  --versioning-configuration Status=Enabled
```

--------------------------------

### Run Core Release Services with Docker Compose

Source: https://github.com/icegatetech/docs/blob/main/en/operations/deployment.md

Starts the core IceGate services including MinIO, Nessie, Ingest, Query, and Maintain using Docker Compose in a release configuration.

```bash
make run-core-release
```

--------------------------------

### Set Query Time Range with Start and End Timestamps

Source: https://github.com/icegatetech/docs/blob/main/en/operations/troubleshooting.md

Use --data-urlencode to specify the start and end timestamps for a query. This helps limit the data processed and improve performance.

```bash
--data-urlencode 'start=1704067200'
--data-urlencode 'end=1704153600'
```

--------------------------------

### GET /ready

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/loki.md

Health check endpoint.

```APIDOC
## GET /ready

### Description
Health check endpoint.

### Method
GET

### Endpoint
/ready
```

--------------------------------

### Execute Build Commands

Source: https://github.com/icegatetech/docs/blob/main/c4/README.md

Standard make commands for generating diagrams, validating syntax, and running the interactive UI.

```bash
# Generate PNG diagrams
make png

# Generate SVG diagrams
make svg

# Validate workspace syntax
make validate

# Run interactive UI (requires Docker)
make lite

# Show all commands
make help
```

--------------------------------

### GET /loki/api/v1/labels

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/loki.md

Retrieve all label names.

```APIDOC
## GET /loki/api/v1/labels

### Description
Get all label names.

### Method
GET

### Endpoint
/loki/api/v1/labels

### Parameters
#### Query Parameters
- **start** (int) - Optional - Start timestamp
- **end** (int) - Optional - End timestamp
```

--------------------------------

### Run IceGate services via CLI

Source: https://github.com/icegatetech/docs/blob/main/en/getting-started/configuration.md

Use the -c or --config flag to specify the configuration file path for various service commands.

```bash
# Ingest service
ingest run -c /etc/icegate/ingest.yaml

# Query service
query run -c /etc/icegate/query.yaml

# Maintain service (schema migration)
maintain migrate create -c /etc/icegate/maintain.yaml
maintain migrate upgrade -c /etc/icegate/maintain.yaml

# Show version
ingest version
query version
```

--------------------------------

### GET /api/v1/labels

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/prometheus.md

Retrieve all available label names.

```APIDOC
## GET /api/v1/labels

### Description
Get all label names.

### Method
GET

### Endpoint
/api/v1/labels

### Request Example
curl http://localhost:9090/api/v1/labels \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Go Service Resource and Exporter Setup

Source: https://github.com/icegatetech/docs/blob/main/en/cookbooks/centralized-logging.md

This Go code snippet shows how to set up the OpenTelemetry resource and an OTLP gRPC log exporter. It defines service attributes and configures the exporter to connect to the OpenTelemetry collector endpoint.

```go
import (
    "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc"
    "go.opentelemetry.io/otel/sdk/resource"
    semconv "go.opentelemetry.io/otel/semconv/v1.4.0"
)

res := resource.NewWithAttributes(
    semconv.SchemaURL,
    semconv.ServiceNameKey.String("payment-service"),
    semconv.ServiceVersionKey.String("2.0.1"),
)

exporter, _ := otlploggrpc.New(ctx,
    otlploggrpc.WithEndpoint("otel-collector:4317"),
    otlploggrpc.WithInsecure(),
)
```

--------------------------------

### GET /health

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Check the health status of the IceGate service.

```APIDOC
## GET /health

### Description
Check the health status of the service.

### Method
GET

### Endpoint
/health

### Response
#### Success Response (200)
- **status** (string) - The health status of the service.

#### Response Example
{
  "status": "healthy"
}
```

--------------------------------

### GET /api/v1/series

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/prometheus.md

Retrieve series matching specific selectors.

```APIDOC
## GET /api/v1/series

### Description
Get series matching selectors.

### Method
GET

### Endpoint
/api/v1/series

### Parameters
#### Query Parameters
- **match[]** (string) - Required - Series selector(s)
- **start** (float) - Optional - Start timestamp
- **end** (float) - Optional - End timestamp

### Request Example
curl -G http://localhost:9090/api/v1/series \
  --data-urlencode 'match[]={__name__=~"http_.*"}' \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Build and Lint Commands

Source: https://github.com/icegatetech/docs/blob/main/AGENTS.md

Standard npm commands for building, linting, serving, and cleaning the documentation project.

```bash
npm run build      # Build all languages to ./build (includes llms.txt files)
npm run lint       # Lint docs with --strict mode
npm run serve      # Build and serve locally on port 8080
npm run clean      # Remove build directory
```

--------------------------------

### Run Ingest Service

Source: https://github.com/icegatetech/docs/blob/main/en/development/building.md

Starts the ingest service using a specific configuration file. Requires the 'ingest' binary to be built.

```bash
cargo run --bin ingest -- run -c config/docker/ingest.yaml
```

--------------------------------

### GET /api/v1/query_range

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/prometheus.md

Query metrics over a specific time range.

```APIDOC
## GET /api/v1/query_range

### Description
Query metrics over a time range.

### Method
GET

### Endpoint
/api/v1/query_range

### Parameters
#### Query Parameters
- **query** (string) - Required - PromQL query
- **start** (float) - Required - Start timestamp (Unix seconds)
- **end** (float) - Required - End timestamp (Unix seconds)
- **step** (duration) - Required - Query resolution step

### Request Example
curl -G http://localhost:9090/api/v1/query_range \
  --data-urlencode 'query=http_requests_total{service="api"}' \
  --data-urlencode 'start=1704067200' \
  --data-urlencode 'end=1704153600' \
  --data-urlencode 'step=60' \
  -H "X-Scope-OrgID: my-tenant"
```

--------------------------------

### Get Tag Values

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Retrieve the possible values for a specific tag.

```APIDOC
## GET /api/search/tag/{tagName}/values

### Description
Retrieves all possible values for a given tag name.

### Method
GET

### Endpoint
http://localhost:3200/api/search/tag/{tagName}/values

### Path Parameters
- **tagName** (string) - Required - The name of the tag for which to retrieve values (e.g., `service.name`).

### Headers
- **X-Scope-OrgID**: my-tenant

### Response Example
(Response format not explicitly provided in the source text, but typically would be a JSON array of tag values.)
```

--------------------------------

### Run Core Services with Load Generator

Source: https://github.com/icegatetech/docs/blob/main/en/operations/deployment.md

Starts the core IceGate services along with a load generator for testing purposes using Docker Compose.

```bash
make run-load-release
```

--------------------------------

### Run Core Services with Monitoring and Analytics

Source: https://github.com/icegatetech/docs/blob/main/en/operations/deployment.md

Starts the core IceGate services along with monitoring tools (Jaeger, Prometheus, Grafana) and analytics (Trino) using Docker Compose.

```bash
make run-analytics-release
```

--------------------------------

### Get a Trace by ID

Source: https://github.com/icegatetech/docs/blob/main/llms-full.txt

Retrieve all spans for a specific trace ID.

```APIDOC
## GET /api/traces/{traceId}

### Description
Retrieves all spans associated with a given trace ID.

### Method
GET

### Endpoint
http://localhost:3200/api/traces/{traceId}

### Path Parameters
- **traceId** (string) - Required - The unique identifier of the trace.

### Headers
- **X-Scope-OrgID**: my-tenant

### Response
#### Success Response (200)
- **batches** (array) - Contains trace data organized into batches.
  - **resource** (object) - Resource information.
    - **attributes** (array) - Key-value pairs describing the resource.
  - **scopeSpans** (array) - Spans grouped by instrumentation scope.
    - **spans** (array) - Individual spans within the scope.
      - **traceId** (string) - The ID of the trace.
      - **spanId** (string) - The ID of the span.
      - **name** (string) - The name of the operation.
      - **kind** (integer) - The type of span (e.g., Server, Client).
      - **startTimeUnixNano** (string) - The start time of the span in nanoseconds.
      - **endTimeUnixNano** (string) - The end time of the span in nanoseconds.
      - **status** (object) - The status of the span.
        - **code** (integer) - The status code (e.g., OK, Error).

### Response Example
```json
{
  "batches": [
    {
      "resource": {
        "attributes": [
          {"key": "service.name", "value": {"stringValue": "api-gateway"}}
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "traceId": "5B8EFFF798038103D269B633813FC60C",
              "spanId": "EEE19B7EC3C1B174",
              "name": "GET /api/orders",
              "kind": 2,
              "startTimeUnixNano": "1704067200000000000",
              "endTimeUnixNano": "1704067200150000000",
              "status": {"code": 1}
            }
          ]
        }
      ]
    }
  ]
}
```
```

--------------------------------

### GET /loki/api/v1/label/{name}/values

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/loki.md

Retrieve values for a specific label.

```APIDOC
## GET /loki/api/v1/label/{name}/values

### Description
Get values for a specific label.

### Method
GET

### Endpoint
/loki/api/v1/label/{name}/values

### Parameters
#### Path Parameters
- **name** (string) - Required - The label name

#### Query Parameters
- **start** (int) - Optional - Start timestamp
- **end** (int) - Optional - End timestamp
```

--------------------------------

### Multi-Tenancy Ingest and Query Examples

Source: https://github.com/icegatetech/docs/blob/main/llms.txt

Demonstrates how to ingest data and query logs for a specific tenant using the X-Scope-OrgID header. The header enforces tenant isolation for both ingestion and retrieval.

```bash
# Ingest for a specific tenant
curl -X POST http://localhost:4318/v1/logs \
  -H "X-Scope-OrgID: team-backend" \
  -H "Content-Type: application/json" \
  -d '...'

# Query only sees that tenant's data
curl -G http://localhost:3100/loki/api/v1/query_range \
  --data-urlencode 'query={service_name="api"}' \
  -H "X-Scope-OrgID: team-backend"
```

--------------------------------

### Query Range Endpoint Example

Source: https://github.com/icegatetech/docs/blob/main/en/api-reference/prometheus.md

Use this endpoint to query metrics over a specified time range. Ensure the X-Scope-OrgID header is included for tenant identification.

```bash
curl -G http://localhost:9090/api/v1/query_range \
  --data-urlencode 'query=http_requests_total{service="api"}' \
  --data-urlencode 'start=1704067200' \
  --data-urlencode 'end=1704153600' \
  --data-urlencode 'step=60' \
  -H "X-Scope-OrgID: my-tenant"
```