Observability Platform Documentation

https://micromegas.info/docs/
Admin
This project documents an observability platform and backend services, guiding users through...
Tokens:117,774
Snippets:1,311
Trust Score:4.3
Update:4 months ago
Show doc for...
Tokens:
Raw
### Documentation Build and Serve

Source: https://micromegas.info/docs/development/build

Instructions for installing documentation dependencies, starting a development server, and building a static site for the documentation.

```bash
# Install dependencies

```

```bash
pip install -r mkdocs/docs-requirements.txt

```

```bash
# Start development server

```

```bash
cd mkdocs

```

```bash
mkdocs serve

```

```bash
# Build static site

```

```bash
mkdocs build

```

--------------------------------

### Python Development Setup and Execution

Source: https://micromegas.info/docs/development/build

Commands for installing Python dependencies using Poetry, running tests with pytest, and formatting code with black.

```bash
cd python/micromegas

# Install dependencies

```

```bash
poetry install

```

```bash
# Run tests

```

```bash
pytest

```

```bash
# Format code (required before commits)

```

```bash
black .

```

--------------------------------

### GitHub Actions CI/CD deployment example

Source: https://micromegas.info/docs/web-app/notebooks/screens-as-code

Example GitHub Actions workflow to deploy micromegas screens using the CLI. It checks out the code, installs micromegas, and applies changes with auto-approval.

```yaml
# GitHub Actions example
deploy-screens:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - run: pip install micromegas
    - run: |
        cd screens
        micromegas-screens apply --auto-approve
      env:
        MICROMEGAS_OIDC_ISSUER: ${{ secrets.OIDC_ISSUER }}
        MICROMEGAS_OIDC_CLIENT_ID: ${{ secrets.OIDC_CLIENT_ID }}
        MICROMEGAS_OIDC_CLIENT_SECRET: ${{ secrets.OIDC_CLIENT_SECRET }}

```

--------------------------------

### Install Build Tools on Linux

Source: https://micromegas.info/docs/getting-started

Install necessary C/C++ build tools, including the mold linker for faster linking, on Linux systems.

```bash
sudo apt-get update
sudo apt-get install build-essential clang mold

```

--------------------------------

### Quick Start Development Server

Source: https://micromegas.info/docs/admin/web-app

Navigate to the application directory and run the Python script to start the development server. The application will open on http://localhost:3000.

```bash
cd analytics-web-app
python start_analytics_web.py

```

--------------------------------

### Grafana Plugin Development Workflow

Source: https://micromegas.info/docs/contributing

Steps for setting up and running the Grafana plugin development environment, including installing dependencies, building binaries, starting the development server, running tests, and linting.

```bash
cd grafana

# Install dependencies
yarn install

# Build Go backend binaries
mage -v build

# Start development server with hot reload
yarn dev

# Run tests
yarn test:ci

# Run linting
yarn lint

# Build production bundle
yarn build

```

--------------------------------

### Development Data Source Configuration

Source: https://micromegas.info/docs/grafana/configuration

Configuration example for a local development setup. Typically uses a default host and port, disabled TLS/SSL, and a development API key.

```ini
Host: localhost:50051
TLS/SSL: Disabled
Auth Method: API Key
API Key: dev-key-12345

```

--------------------------------

### Notebook Execution Pattern Example

Source: https://micromegas.info/docs/web-app/notebooks/execution

A three-cell example demonstrating a typical notebook workflow: fetching raw data, defining thresholds in a reference table, and then joining these locally to generate alerts.

```SQL
Cell 1: "raw_data" (Table)
  SQL: SELECT time, name, value FROM measures

Cell 2: "thresholds" (Reference Table)
  CSV: name,warn,error
       cpu_usage,80,95
       memory_usage,70,90

Cell 3: "alerts" (Table)
  SQL: SELECT r.time, r.name, r.value, t.warn, t.error
       FROM raw_data r
       JOIN thresholds t ON r.name = t.name
       WHERE r.value > t.warn

```

--------------------------------

### Install Python Package

Source: https://micromegas.info/docs/contributing

Navigate to the Python Micromegas package directory and install dependencies using Poetry.

```bash
cd python/micromegas && poetry install   # Python doesn't need a build step
```

--------------------------------

### Start Grafana with Plugin

Source: https://micromegas.info/docs/contributing

Navigate to the Grafana directory and start Grafana with the plugin using Yarn and Docker Compose. Access Grafana at http://localhost:3000.

```bash
cd grafana
yarn server             # Starts Grafana with docker compose (includes --build)
# Access Grafana at http://localhost:3000
```

--------------------------------

### Test Micromegas Installation with Sample Query

Source: https://micromegas.info/docs/getting-started

Use the Python client to connect to a local Micromegas instance, execute a sample SQL query for log entries, and print the results to verify the installation.

```python
import datetime
import micromegas

# Connect to local Micromegas instance
client = micromegas.connect()

# Set up time range for query
now = datetime.datetime.now(datetime.timezone.utc)
begin = now - datetime.timedelta(days=1)
end = now

# Query recent log entries
sql = """
    SELECT time, process_id, level, target, msg
    FROM log_entries
    ORDER BY time DESC
    LIMIT 10;
"""

# Execute query and display results
df = client.query(sql, begin, end)
print(f"Found {len(df)} log entries")
print(df.head())

```

--------------------------------

### Start the HTTP Gateway Server

Source: https://micromegas.info/docs/gateway

Set the FlightSQL backend URL and start the gateway service on the specified HTTP endpoint.

```bash
# Set FlightSQL backend URL
export MICROMEGAS_FLIGHTSQL_URL=grpc://127.0.0.1:50051

# Start gateway on port 3000
cargo run --bin http-gateway-srv -- --listen-endpoint-http 0.0.0.0:3000

```

--------------------------------

### Install Python Dependencies

Source: https://micromegas.info/docs/contributing

Install Python dependencies using Poetry. Run this command from the `python/micromegas/` directory.

```bash
poetry install
```

--------------------------------

### Install Build Tools on macOS

Source: https://micromegas.info/docs/getting-started

Install the Xcode command-line tools on macOS to provide the necessary C/C++ build tools.

```bash
xcode-select --install

```

--------------------------------

### Install Micromegas Python Client

Source: https://micromegas.info/docs/getting-started

Install the Micromegas Python client library using pip to interact with the running Micromegas instance.

```bash
pip install micromegas

```

--------------------------------

### Start All Micromegas Services with tmux (Linux/macOS)

Source: https://micromegas.info/docs/getting-started

Use the development script with tmux to build and start all required Micromegas services, including PostgreSQL, ingestion, and flight-SQL servers, in separate panes.

```bash
# Start all services in a tmux session (debug mode)
python3 local_test_env/dev.py

# Or in release mode for better performance
python3 local_test_env/dev.py release

```

--------------------------------

### Start FlightSQL Server Manually

Source: https://micromegas.info/docs/getting-started

Manually start the FlightSQL server using Cargo, with authentication disabled.

```bash
cd rust
cargo run -p flight-sql-srv -- --disable-auth

```

--------------------------------

### Start PostgreSQL Database Manually

Source: https://micromegas.info/docs/getting-started

Manually start the PostgreSQL database service by navigating to its directory and running the provided Python script.

```bash
cd local_test_env/db
python run.py

```

--------------------------------

### Run Gateway with Default Configuration

Source: https://micromegas.info/docs/gateway/configuration

Start the HTTP gateway server using default settings. Ensure the FlightSQL backend URL is set as an environment variable.

```shell
export MICROMEGAS_FLIGHTSQL_URL=grpc://127.0.0.1:50051
cargo run --bin http-gateway-srv
```

--------------------------------

### Install micromegas CLI

Source: https://micromegas.info/docs/web-app/notebooks/screens-as-code

Install the micromegas package using pip. For development, use poetry install within the python/micromegas directory.

```bash
pip install micromegas
# or in development:
cd python/micromegas && poetry install

```

--------------------------------

### Build Backend Binaries

Source: https://micromegas.info/docs/grafana/installation

Build the backend binaries for the plugin using mage. Ensure mage is installed and available in your PATH.

```bash
mage -v build

```

--------------------------------

### Run Hard Link Script for Development Setup

Source: https://micromegas.info/docs/unreal/installation

Execute the unreal_hard_link_windows.py script from the micromegas/build directory to create hard links for development.

```bash
cd micromegas/build
python unreal_hard_link_windows.py
```

--------------------------------

### Start Admin Service Manually

Source: https://micromegas.info/docs/getting-started

Manually start the telemetry admin service using Cargo, running the 'crond' command.

```bash
cd rust
cargo run -p telemetry-admin -- crond

```

--------------------------------

### Example OIDC Migration Configuration

Source: https://micromegas.info/docs/admin/authentication

Demonstrates setting environment variables for API keys and OIDC configuration during a migration. This allows parallel operation before fully removing API keys.

```bash
# Step 1: Add OIDC configuration (API keys still work)
export MICROMEGAS_API_KEYS='[{"name": "service1", "key": "old-key"}]'
export MICROMEGAS_OIDC_CONFIG='{
  "issuers": [{
    "issuer": "https://accounts.google.com",
    "audience": "new-client-id.apps.googleusercontent.com"
  ) }]'

# Step 2: Update clients to use OIDC
# Test both authentication methods work

# Step 3: Remove API keys when all clients migrated
unset MICROMEGAS_API_KEYS
# Only OIDC remains

```

--------------------------------

### Example URL with Time Range Parameters

Source: https://micromegas.info/docs/web-app

This URL demonstrates how to specify a time range using 'from' and 'to' parameters. These parameters are evaluated at query time for relative ranges.

```url
/screen/my-dashboard?from=now-1h&to=now
```

--------------------------------

### Example Analytics Web Server Command

Source: https://micromegas.info/docs/admin/web-app

Demonstrates how to run the analytics-web-srv with a custom port, a specified frontend directory, and authentication disabled. Use `--disable-auth` only for development environments.

```bash
analytics-web-srv --port 8000 --frontend-dir ./dist --disable-auth

```

--------------------------------

### Install All Workspace Dependencies

Source: https://micromegas.info/docs/contributing

Install all dependencies for the monorepo, including Grafana plugin and shared types, using Yarn. Run this command from the repository root.

```bash
yarn install
```

--------------------------------

### Start Ingestion Server Manually

Source: https://micromegas.info/docs/getting-started

Manually start the telemetry ingestion server using Cargo, specifying the HTTP listen endpoint.

```bash
cd rust
cargo run -p telemetry-ingestion-srv -- --listen-endpoint-http 127.0.0.1:9000

```

--------------------------------

### Custom Detail Template Example

Source: https://micromegas.info/docs/web-app/notebooks/cell-types

An example of an extended detail template that includes a link to a process page using a 'process_id' column.

```markdown
### Event Details

---

**Location:** ($x, $y, $z)

**Process ID:** [$process_id](process?id=$process_id)

```

--------------------------------

### Initiate OAuth Login

Source: https://micromegas.info/docs/admin/web-app

Starts the OAuth login process.

```APIDOC
## GET /auth/login

### Description
Initiates the OAuth login flow.

### Method
GET

### Endpoint
/auth/login
```

--------------------------------

### Set Environment Variables for Development Setup

Source: https://micromegas.info/docs/unreal/installation

Set environment variables for the Unreal Engine root directory and the telemetry module directory for development setup on Windows.

```bash
set MICROMEGAS_UNREAL_ROOT_DIR=C:\Program Files\Epic Games\UE_5.3
set MICROMEGAS_UNREAL_TELEMETRY_MODULE_DIR=C:\YourProject\Plugins
```

--------------------------------

### Python Package Installation and Testing

Source: https://micromegas.info/docs/contributing

Manage Python dependencies and run tests using Poetry. Commands should be executed from the `python/micromegas/` directory.

```bash
poetry install
poetry run pytest
```

--------------------------------

### Properties Structure Example

Source: https://micromegas.info/docs/query-guide/schema-reference

Illustrates the optimized JSONB format for storing properties, which includes dictionary compression and native binary JSON for efficiency.

```sql
-- Properties structure (optimized JSONB format)
Dictionary(Int32, Binary)
```

--------------------------------

### Predicate Pushdown Example

Source: https://micromegas.info/docs/query-guide/performance

Micromegas automatically pushes filters down to the storage layer for efficiency. This example shows filters applied to time, level, and process ID.

```sql
-- These filters are pushed to Parquet reader for efficiency
WHERE time >= NOW() - INTERVAL '1 day'
  AND level <= 3
  AND process_id = 'my_process'
```

--------------------------------

### Micromegas Configuration File Example

Source: https://micromegas.info/docs/query-guide/python-api

A sample JSON configuration file for micromegas, specifying server URI and OIDC issuer details. Environment variables can override these settings.

```json
{
  "uri": "grpc+tls://analytics.example.com:50051",
  "client_id": "your-app-client-id",
  "issuers": [
    {
      "issuer": "https://accounts.example.com",
      "audience": "your-api-audience"
    }
  ]
}

```

--------------------------------

### Automatic Authentication with #[micromegas_main]

Source: https://micromegas.info/docs/admin/authentication

Applications using `#[micromegas_main]` automatically configure authentication from environment variables. This is the recommended approach for simplified setup.

```rust
use micromegas::micromegas_main;
use micromegas::tracing::prelude::*;

#[micromegas_main(interop_max_level = "info")]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    info!("Application starting");
    // Telemetry automatically authenticated based on environment variables
    Ok(())
}

```

--------------------------------

### Manual API Key Authentication Setup

Source: https://micromegas.info/docs/admin/authentication

Manually configure API key authentication for applications not using `#[micromegas_main]`. This involves setting the API key and configuring `HttpEventSink` with an `ApiKeyRequestDecorator`.

```rust
use micromegas_telemetry_sink::http_event_sink::HttpEventSink;
use micromegas_telemetry_sink::api_key_decorator::ApiKeyRequestDecorator;
use std::sync::Arc;

// From environment variable
std::env::set_var("MICROMEGAS_INGESTION_API_KEY", "secret-key-123");
let decorator = ApiKeyRequestDecorator::from_env().unwrap();

// Configure HttpEventSink with authentication
let sink = HttpEventSink::new(
    "http://localhost:9000",
    max_queue_size,
    metadata_retry,
    blocks_retry,
    Box::new(move || Arc::new(decorator.clone())),
);

```

--------------------------------

### Run Gateway with Custom Listen Address

Source: https://micromegas.info/docs/gateway/configuration

Start the HTTP gateway server and specify a custom HTTP listen endpoint using the --listen-endpoint-http flag.

```shell
cargo run --bin http-gateway-srv -- --listen-endpoint-http 0.0.0.0:3000
```

--------------------------------

### Run Gateway with Custom Headers

Source: https://micromegas.info/docs/gateway/configuration

Start the HTTP gateway server with custom header forwarding rules defined in MICROMEGAS_GATEWAY_HEADERS. The FlightSQL URL must also be specified.

```shell
export MICROMEGAS_GATEWAY_HEADERS='{
  "allowed_headers": ["Authorization", "X-Tenant-ID"],
  "allowed_prefixes": ["X-Custom-"],
  "blocked_headers": ["Cookie"]
}'
export MICROMEGAS_FLIGHTSQL_URL=grpc://127.0.0.1:50051
cargo run --bin http-gateway-srv
```

--------------------------------

### Get Processes from Last Day

Source: https://micromegas.info/docs/query-guide/schema-reference

Retrieves process ID, executable name, computer, and start time for all processes initiated within the last 24 hours. Results are ordered by start time in descending order.

```sql
SELECT process_id, exe, computer, start_time
FROM processes
WHERE start_time >= NOW() - INTERVAL '1 day'
ORDER BY start_time DESC;
```

--------------------------------

### Create Performance Histogram

Source: https://micromegas.info/docs/query-guide/functions-reference

Creates a histogram to analyze performance metrics like response time. The example bins response times in milliseconds into 10 bins between 0 and 100ms.

```sql
-- Create performance histogram (0-100ms, 10 bins)
SELECT make_histogram(0.0, 100.0, 10, duration / 1000000.0) as response_time_ms_histogram
FROM view_instance('thread_spans', 'web_server')
WHERE name = 'handle_request'
  AND duration > 1000000;  -- > 1ms

```

--------------------------------

### Classic Incoming Network Event Hierarchy

Source: https://micromegas.info/docs/unreal/network-tracing

Example of the event hierarchy for classic incoming network traffic, showing connections, objects, properties, and RPCs with their bit sizes.

```text
NetConnectionBeginEvent       connection="127.0.0.1:7777"  is_outgoing=false
  NetObjectBeginEvent         name="PlayerPawn_C_0"
  NetPropertyEvent            name="Health"            bit_size=16
  NetPropertyEvent            name="ReplicatedMovement" bit_size=112
  NetObjectEndEvent           bit_size=512
  NetObjectBeginEvent         name="InventoryComponent"   # peer, not nested
  NetPropertyEvent            name="GoldCount"         bit_size=24
  NetObjectEndEvent           bit_size=88
  NetRPCBeginEvent            name="ClientNotifyHit"
  NetPropertyEvent            name="HitLocation"       bit_size=96
  NetRPCEndEvent              bit_size=160
NetConnectionEndEvent         bit_size=760             # sum of root Object/RPC bits

```

--------------------------------

### Initialize screens directory

Source: https://micromegas.info/docs/web-app/notebooks/screens-as-code

Create a directory for your screens and initialize it with the micromegas-screens CLI, providing the server URL.

```bash
mkdir screens && cd screens
micromegas-screens init https://micromegas.example.com

```

--------------------------------

### Install Node.js Dependencies

Source: https://micromegas.info/docs/grafana/installation

Install the necessary Node.js dependencies for the Grafana plugin using yarn.

```bash
yarn install

```

--------------------------------

### Install Cargo Machete for Dependency Checking

Source: https://micromegas.info/docs/development/build

Installs the cargo-machete tool for checking unused dependencies in Rust projects.

```bash
# Install cargo-machete for unused dependency checking

```

```bash
cargo install cargo-machete

```

--------------------------------

### Initialize Game Instance with Telemetry

Source: https://micromegas.info/docs/unreal/examples

Sets up the game instance with Micromegas telemetry, including authentication and default context. Ensure API keys are stored securely and not hardcoded.

```cpp
// MyGameInstance.h
UCLASS()
class MYGAME_API UMyGameInstance : public UGameInstance
{
    GENERATED_BODY()

public:
    virtual void Init() override;
    virtual void Shutdown() override;

private:
    void SetupTelemetryContext();
};

// MyGameInstance.cpp
#include "MyGameInstance.h"
#include "MicromegasTelemetrySink/MicromegasTelemetrySinkModule.h"
#include "MicromegasTelemetrySink/ApiKeyAuthenticator.h"
#include "MicromegasTracing/Macros.h"
#include "MicromegasTracing/Dispatch.h"
#include "MicromegasTracing/DefaultContext.h"

void UMyGameInstance::Init()
{
    Super::Init();

    // Get API key from config (store securely, not hardcoded!)
    FString ApiKey = GetDefault<UGameSettings>()->TelemetryApiKey;

    // Initialize telemetry with API key authentication
    auto Auth = MakeShared<FApiKeyAuthenticator>(ApiKey);
    IMicromegasTelemetrySinkModule::LoadModuleChecked().InitTelemetry(
        TEXT("https://telemetry.example.com:9000"),
        Auth
    );

    SetupTelemetryContext();

    MICROMEGAS_LOG("Game", MicromegasTracing::LogLevel::Info,
                   TEXT("Game instance initialized"));
}

void UMyGameInstance::SetupTelemetryContext()
{
    if (auto* Ctx = MicromegasTracing::Dispatch::GetDefaultContext())
    {
        // Session info
        Ctx->Set(FName("session_id"), FName(*FGuid::NewGuid().ToString()));
        Ctx->Set(FName("timestamp"), FName(*FDateTime::UtcNow().ToString()));

        // Build info
        Ctx->Set(FName("build_version"), FName(TEXT(GAME_VERSION)));
        Ctx->Set(FName("build_config"), FName(TEXT(STRINGIFY(UE_BUILD_CONFIGURATION))));

        // Platform info
        Ctx->Set(FName("platform"), FName(*UGameplayStatics::GetPlatformName()));
        Ctx->Set(FName("cpu"), FName(*FPlatformMisc::GetCPUBrand()));
        Ctx->Set(FName("gpu"), FName(*GRHIAdapterName));

        // Player info (if available)
        if (ULocalPlayer* LocalPlayer = GetFirstGamePlayer())
        {
            Ctx->Set(FName("player_id"), FName(*LocalPlayer->GetPreferredUniqueNetId().ToString()));
        }
    }
}

void UMyGameInstance::Shutdown()
{
    MICROMEGAS_LOG("Game", MicromegasTracing::LogLevel::Info, 
                   TEXT("Game instance shutting down"));

    // Force flush before shutdown
    MicromegasTracing::Dispatch::FlushLogStream();
    MicromegasTracing::Dispatch::FlushMetricStream();
    MicromegasTracing::Dispatch::FlushCurrentThreadStream();

    Super::Shutdown();
}

```

--------------------------------

### Fetch Go Dependencies

Source: https://micromegas.info/docs/contributing

Download Go dependencies for the Grafana backend. Run this command from the `grafana/` directory.

```bash
go mod download
```

--------------------------------

### Production Data Source Configuration with OAuth 2.0

Source: https://micromegas.info/docs/grafana/configuration

Configuration example for a production environment using OAuth 2.0 Client Credentials. Requires OIDC issuer, client ID, and client secret for authentication.

```ini
Host: analytics.example.com:50051
TLS/SSL: Enabled
Auth Method: OAuth 2.0 Client Credentials
OIDC Issuer: https://accounts.google.com
Client ID: grafana-prod@project.iam.gserviceaccount.com
Client Secret: ********

```

--------------------------------

### Start Self-Hosted CI Runner Worker

Source: https://micromegas.info/docs/development/build

Starts the Micromegas self-hosted CI runner worker. The worker runs until interrupted (Ctrl+C).

```bash
# Start the worker (runs until Ctrl+C)

```

```bash
python3 build/dev_worker.py

```

--------------------------------

### Production Data Source Configuration with API Key and Metadata

Source: https://micromegas.info/docs/grafana/configuration

Configuration example for a production environment using an API key, with additional metadata for environment and region. TLS/SSL is enabled for secure communication.

```ini
Host: analytics.example.com:50051
TLS/SSL: Enabled
Auth Method: API Key
API Key: prod-key-67890
Metadata:
  environment: production
  region: us-east-1

```

--------------------------------

### Initialize screens directory command

Source: https://micromegas.info/docs/web-app/notebooks/screens-as-code

Command to initialize the screens directory. Must be run inside a git repository. Reads the git remote to construct the `managed_by` URL.

```bash
micromegas-screens init SERVER_URL [--remote REMOTE]

```

--------------------------------

### Start Self-Hosted CI Runner Worker with Resource Limits

Source: https://micromegas.info/docs/development/build

Starts the Micromegas self-hosted CI runner worker with specified CPU and memory limits.

```bash
# With resource limits

```

```bash
python3 build/dev_worker.py --cpus 8 --memory 16g

```

--------------------------------

### Configure Build Dependencies in YourGame.Build.cs

Source: https://micromegas.info/docs/unreal/installation

Add MicromegasTelemetrySink to the PrivateDependencyModuleNames in your game's Build.cs file.

```csharp
// YourGame.Build.cs
public class YourGame : ModuleRules
{
    public YourGame(ReadOnlyTargetRules Target) : base(Target)
    {
        PublicDependencyModuleNames.AddRange(new string[] { 
            "Core",  // MicromegasTracing is now included in Core
            "CoreUObject", 
            "Engine"
        });

        PrivateDependencyModuleNames.AddRange(new string[] {
            "MicromegasTelemetrySink"  // Add this plugin
        });
    }
}
```

--------------------------------

### Basic Log Entry Query

Source: https://micromegas.info/docs/query-guide

Start with a simple query to select all log entries, limiting the results to 10. This is a good starting point for exploring data.

```sql
-- Start with this
SELECT * FROM log_entries LIMIT 10;
```

--------------------------------

### Screens initialization configuration

Source: https://micromegas.info/docs/web-app/notebooks/screens-as-code

The micromegas-screens init command creates a JSON file with the server URL and a managed_by link derived from your git remote.

```json
{
  "managed_by": "https://github.com/org/repo/tree/main/screens",
  "server": "https://micromegas.example.com"
}

```

--------------------------------

### Configuring and Using Custom Authentication Headers

Source: https://micromegas.info/docs/gateway

This example shows how to configure the gateway to forward custom authentication headers and how a client can send these headers in a request. It involves setting an environment variable for allowed headers and using curl for the POST request.

```bash
# Configure custom header forwarding
export MICROMEGAS_GATEWAY_HEADERS='{
  "allowed_headers": ["X-Proprietary-Token", "X-Tenant-ID"],
  "blocked_headers": ["Cookie"]
}'

# Client sends custom headers
curl -X POST http://localhost:3000/gateway/query \
  -H "X-Proprietary-Token: custom-auth-token" \
  -H "X-Tenant-ID: tenant-123" \
  -d '{"sql": "SELECT * FROM logs"}'


```

--------------------------------

### Column Format Override Example

Source: https://micromegas.info/docs/web-app/notebooks/cell-types

Illustrates how to override column rendering in a table cell using markdown format strings and row macros. This example creates a clickable link for the 'exe' column that navigates to a process page.

```markdown
[$row.exe](/process/$row.process_id?from=$begin&to=$end)

```

--------------------------------

### Import existing screens

Source: https://micromegas.info/docs/web-app/notebooks/screens-as-code

Adopt existing screens from the server using the import command, specifying the screen names. This saves each screen as a JSON file and sets ownership on the server.

```bash
micromegas-screens import my-notebook performance-dashboard

```

--------------------------------

### Add Micromegas Dependency (Python)

Source: https://micromegas.info/docs/blog/2025/09/03/micromegas-v0120-async-span-tracing

Install the Micromegas library for Python applications using pip.

```python
pip install micromegas
```

--------------------------------

### Clone Micromegas Repository

Source: https://micromegas.info/docs/getting-started

Clone the Micromegas repository and navigate into the project directory to begin the installation process.

```bash
git clone https://github.com/madesroches/micromegas.git
cd micromegas

```

--------------------------------

### Example Metadata Configuration

Source: https://micromegas.info/docs/grafana/configuration

Optional key-value pairs sent to the FlightSQL server. Common use cases include environment or tenant identifiers, or custom headers. Format as key-value pairs.

```yaml
key1: value1
key2: value2

```

--------------------------------

### Get Sum from Histogram

Source: https://micromegas.info/docs/query-guide/functions-reference

Extracts the sum of all original values that were aggregated into the histogram. Useful for calculating totals.

```sql
-- Get total duration from histogram
SELECT sum_from_histogram(duration_histogram) as total_duration
FROM performance_histograms;
```

--------------------------------

### Stop All Micromegas Services

Source: https://micromegas.info/docs/getting-started

Execute the stop-dev.py script to gracefully shut down all Micromegas services started with the development script.

```bash
python3 local_test_env/stop-dev.py

```

--------------------------------

### Enable Telemetry System

Source: https://micromegas.info/docs/unreal/instrumentation-api

Initializes the telemetry system. This command should be run once to enable data collection.

```unrealconsole
telemetry.enable
```

--------------------------------

### Notebook URL Sharing Example

Source: https://micromegas.info/docs/web-app/notebooks

This URL demonstrates how variable values and time range are encoded as delta parameters for state sharing. Only values that differ from saved defaults appear in the URL.

```URL
/screen/my-notebook?from=now-1h&metric=cpu_usage
```

--------------------------------

### Gateway Error Response Format

Source: https://micromegas.info/docs/gateway

The gateway returns errors in a JSON format. This example shows a typical error structure.

```json
{
  "error": "Failed to execute query: table 'invalid' not found"
}

```

--------------------------------

### Get Total Count from Histogram

Source: https://micromegas.info/docs/query-guide/functions-reference

Extracts the total number of values that were aggregated into the histogram. Useful for understanding the sample size.

```sql
-- Get total sample count from histogram
SELECT count_from_histogram(duration_histogram) as total_samples
FROM performance_histograms;
```

--------------------------------

### Rust Workspace Build and Test Commands

Source: https://micromegas.info/docs/contributing

Run these commands from the `rust/` directory to build, test, format, and lint the Rust components of the project. Formatting is required before committing.

```bash
cargo build
cargo test
cargo fmt
    cargo clippy --workspace -- -D warnings
```

--------------------------------

### Get keys from process properties

Source: https://micromegas.info/docs/query-guide/functions-reference

This snippet shows how to retrieve the keys from the 'properties' column in the 'processes' table using jsonb_object_keys.

```sql
SELECT jsonb_object_keys(properties) as prop_keys
FROM processes
LIMIT 5;
```

--------------------------------

### Connect with Dictionary Preservation

Source: https://micromegas.info/docs/query-guide/python-api

Shows how to instantiate `FlightSQLClient` with dictionary preservation enabled for memory efficiency. Connects to a local server.

```python
from micromegas.flightsql.client import FlightSQLClient

# Connect with dictionary preservation for memory efficiency
client = FlightSQLClient(
    "grpc://localhost:50051",
    preserve_dictionary=True
)
```

--------------------------------

### Get CPU metrics over time

Source: https://micromegas.info/docs/query-guide/schema-reference

Retrieves CPU usage metrics over the last hour, including the timestamp, value, and unit.

```sql
SELECT time, value, unit
FROM measures
WHERE name = 'cpu_usage'
  AND time >= NOW() - INTERVAL '1 hour'
ORDER BY time;
```

--------------------------------

### Restart Grafana Server

Source: https://micromegas.info/docs/grafana/installation

Restart the Grafana server to apply the plugin installation. Use the appropriate command for your environment (Systemd or Docker).

```bash
# Systemd
sudo systemctl restart grafana-server

# Docker
docker restart grafana

```

--------------------------------

### Create a View Instance for Log Entries

Source: https://micromegas.info/docs/query-guide/functions-reference

Use `view_instance` to create a process-scoped view for log entries. Filter logs by severity level.

```sql
SELECT time, level, msg
FROM view_instance('log_entries', 'my_process_123')
WHERE level <= 3;
```

--------------------------------

### Plan screen changes command

Source: https://micromegas.info/docs/web-app/notebooks/screens-as-code

Command to preview what `apply` would change, showing creates, updates, deletes, and untracked screens. This is a read-only operation.

```bash
micromegas-screens plan [NAME...]

```

--------------------------------

### Rust Dependency Fetch and Compile

Source: https://micromegas.info/docs/contributing

Fetch and compile Rust dependencies. Run this command from the `rust/` directory.

```bash
cargo build
```

--------------------------------

### Validate SQL Query

Source: https://micromegas.info/docs/query-guide/python-api-advanced

Validates a SQL query against the database. Use this to check for syntax errors and get optimization recommendations before execution.

```python
sql = """
SELECT time, process_id, level, target, msg, properties 
FROM log_entries 
ORDER BY time DESC
"""

validation = validate_query(client, sql)
if validation['valid']:
    print("Query is valid")
    for rec in validation['recommendations']:
        print(f"⚠️  {rec}")
else:
    print(f"❌ Query validation failed: {validation['error']}")
```

--------------------------------

### Configure Go OTel SDK Environment Variables

Source: https://micromegas.info/docs/otlp

Set environment variables for the Go OpenTelemetry SDK to configure OTLP/HTTP endpoint, protocol, and service name.

```bash
export OTEL_EXPORTER_OTLP_ENDPOINT="http://127.0.0.1:9000/ingestion/otlp"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_SERVICE_NAME="my-service"

```

--------------------------------

### Get Recent Errors

Source: https://micromegas.info/docs/query-guide/query-patterns

Retrieves all log entries with a level of Fatal or Error from the last hour. Ordered by time in descending order.

```sql
-- Get all errors from the last hour
SELECT time, process_id, target, msg
FROM log_entries
WHERE level <= 2  -- Fatal and Error
  AND time >= NOW() - INTERVAL '1 hour'
ORDER BY time DESC;

```

--------------------------------

### Query Process Information

Source: https://micromegas.info/docs/query-guide/quick-start

Retrieves a list of processes sending telemetry, ordered by start time. Limits the results to the 10 most recent processes.

```python
processes = client.query("""
    SELECT process_id, exe, computer, start_time
    FROM processes
    ORDER BY start_time DESC
    LIMIT 10;
""")
print(processes)
```

--------------------------------

### Import screens command

Source: https://micromegas.info/docs/web-app/notebooks/screens-as-code

Command to import existing server screens by name. Downloads the screen and sets `managed_by` on the server. Prompts for confirmation if the screen is already managed by another repo.

```bash
micromegas-screens import NAME [NAME...]

```
Tokens:
Raw