### Docker Compose Commands for VIA v2 Source: https://context7.com/veristamp/via-typescript/llms.txt Provides essential commands for managing the VIA v2 services using Docker Compose. This includes starting services in detached mode, running database migrations, and starting the development server. ```bash # Start services docker-compose up -d # Run database migrations bun run db:migrate # Start development server bun run dev ``` -------------------------------- ### TOML Configuration Schema Example Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/via-core.md This TOML file defines the configuration structure for the Via core engine. It includes settings for the engine, various detectors (volume, distribution, cardinality, spectral, behavioral), thresholds, persistence, and observability. ```toml # via-core.toml [engine] profile_path = "/var/lib/via-core/profiles" max_memory_mb = 4096 memory_pressure_threshold = 0.85 shutdown_grace_period = "30s" [detectors.volume] enabled = true hw_alpha = 0.1 hw_beta = 0.05 hw_gamma = 0.1 period = 60 seasonality_auto_detect = true [detectors.volume.threshold] method = "ensemble" sensitivity = 0.7 adaptive_factor = 1.0 [detectors.distribution] enabled = true histogram_bins = 50 min_value = 0.0 max_value = 5000.0 decay_factor = 0.99 [detectors.cardinality] enabled = true hll_precision = 12 velocity_window = "60s" baseline_samples = 1000 [detectors.spectral] enabled = true salience_threshold = 2.0 window_size = 128 [detectors.behavioral] enabled = true features = ["mean", "stddev", "skewness", "kurtosis"] comparison_window = "1h" drift_tolerance = 0.1 [thresholds] volume = 0.8 distribution = 0.75 cardinality = 0.85 spectral = 0.7 behavioral = 0.8 overall_anomaly = 0.7 [persistence] enabled = true checkpoint_interval = "5m" storage_path = "/var/lib/via-core/checkpoints" max_checkpoints = 10 [observability] prometheus_port = 9090 log_level = "info" explainability = "full" # none, basic, full trace_sampling_rate = 0.1 ``` -------------------------------- ### Run VIA-Core Gatekeeper for Production Ingest Source: https://github.com/veristamp/via-typescript/blob/main/via-core/README.md Starts the VIA-Core gatekeeper, which serves as the primary entry point for live data ingestion. It exposes endpoints for ingesting single or batched data, retrieving metrics in Prometheus format, and checking system health. ```bash cargo run --release -p via-core --bin gatekeeper ``` -------------------------------- ### List All Policies Source: https://context7.com/veristamp/via-typescript/llms.txt Retrieves a list of all available policies, with an optional limit parameter to control the number of results. This is a GET request. ```bash curl "http://localhost:3000/control/policy?limit=20" ``` -------------------------------- ### Simulation Control API Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/via-sim-merge-plan.md Endpoints for starting, pausing, resuming, and managing the simulation generation. ```APIDOC ## POST /start ### Description Starts the simulation generation process. ### Method POST ### Endpoint /start ### Parameters #### Query Parameters None #### Request Body None ### Request Example ```json {} ``` ### Response #### Success Response (200) - **message** (string) - Confirmation message. #### Response Example ```json { "message": "Simulation started successfully." } ``` ## POST /pause ### Description Pauses the simulation generation process. ### Method POST ### Endpoint /pause ### Parameters #### Query Parameters None #### Request Body None ### Request Example ```json {} ``` ### Response #### Success Response (200) - **message** (string) - Confirmation message. #### Response Example ```json { "message": "Simulation paused successfully." } ``` ## POST /resume ### Description Resumes the simulation generation process. ### Method POST ### Endpoint /resume ### Parameters #### Query Parameters None #### Request Body None ### Request Example ```json {} ``` ### Response #### Success Response (200) - **message** (string) - Confirmation message. #### Response Example ```json { "message": "Simulation resumed successfully." } ``` ## POST /anomaly ### Description Injects an anomaly into the simulation. ### Method POST ### Endpoint /anomaly ### Parameters #### Query Parameters None #### Request Body - **anomalyConfig** (object) - Required - Configuration for the anomaly to be injected. - **type** (string) - Required - The type of anomaly. - **details** (object) - Optional - Specific details about the anomaly. ### Request Example ```json { "anomalyConfig": { "type": "network_latency", "details": { "target_service": "user-service", "latency_ms": 500 } } } ``` ### Response #### Success Response (200) - **message** (string) - Confirmation message. - **anomalyId** (string) - The ID of the injected anomaly. #### Response Example ```json { "message": "Anomaly injected successfully.", "anomalyId": "a1b2c3d4-e5f6-7890-1234-567890abcdef" } ``` ## DELETE /anomaly/{id} ### Description Removes a previously injected anomaly. ### Method DELETE ### Endpoint /anomaly/{id} ### Parameters #### Path Parameters - **id** (string) - Required - The ID of the anomaly to remove. #### Query Parameters None #### Request Body None ### Request Example ```json {} ``` ### Response #### Success Response (200) - **message** (string) - Confirmation message. #### Response Example ```json { "message": "Anomaly removed successfully." } ``` ## PUT /rate ### Description Sets the events per second for the simulation generation. ### Method PUT ### Endpoint /rate ### Parameters #### Query Parameters None #### Request Body - **rate** (number) - Required - The desired events per second. ### Request Example ```json { "rate": 1000 } ``` ### Response #### Success Response (200) - **message** (string) - Confirmation message. #### Response Example ```json { "message": "Generation rate set to 1000 events/sec." } ``` ## GET /state ### Description Retrieves the current state of the simulation generation. ### Method GET ### Endpoint /state ### Parameters #### Query Parameters None #### Request Body None ### Request Example ```json {} ``` ### Response #### Success Response (200) - **state** (string) - The current state (e.g., "running", "paused", "stopped"). - **currentRate** (number) - The current events per second. #### Response Example ```json { "state": "running", "currentRate": 1000 } ``` ## GET /events ### Description Streams simulation events using Server-Sent Events (SSE) or WebSocket. ### Method GET ### Endpoint /events ### Parameters #### Query Parameters None #### Request Body None ### Request Example ```json {} ``` ### Response #### Success Response (200) - **eventStream** (stream) - A stream of simulation events. #### Response Example (SSE Example) ``` event: simulation_update data: {"type": "progress", "percentage": 50} ``` (WebSocket Example) ```json { "type": "anomaly_detected", "details": { "anomalyId": "a1b2c3d4-e5f6-7890-1234-567890abcdef", "timestamp": "2023-10-27T10:00:00Z" } } ``` ``` -------------------------------- ### Start via-core HTTP Server (Bash) Source: https://github.com/veristamp/via-typescript/blob/main/via-core/crates/via-core/README.md Provides the command to build and run the via-core gatekeeper HTTP server using Cargo. This server allows ingesting data for anomaly detection via HTTP requests. ```bash cargo run --bin gatekeeper -- --port 8080 ``` -------------------------------- ### Via-TypeScript Implementation Roadmap Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/via-core.md Outlines the phased implementation plan for the Via-TypeScript project over 10 weeks. It details key features and infrastructure to be developed in each two-week phase, starting from foundational elements like schema and configuration to core enhancements, control APIs, performance optimizations, and finally testing and release preparation. ```plaintext Week 1-2: Foundation ├── via-schema (JSON Schema + Rust types) ├── via-config (TOML loader + validation) └── Hot-reload infrastructure Week 3-4: Core Enhancements ├── Memory pressure manager ├── ExplainableAnomaly struct ├── Model persistence └── Checkpoint/recovery Week 5-6: Control API ├── gRPC ControlService ├── Control client library └── Admin CLI Week 7-8: Performance ├── Adaptive ensemble ├── SIMD optimizations └── Profile compression Week 9-10: Testing & Polish ├── Integration tests ├── Documentation └── Release prep ``` -------------------------------- ### SimulationController HTTP API Endpoints (Rust) Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/via-sim-merge-plan.md Defines the HTTP API endpoints for the SimulationController, used to control and monitor the generation process. These endpoints allow for starting, pausing, resuming generation, injecting and removing anomalies, setting the event rate, and retrieving the current state or streaming events. ```rust // HTTP API endpoints for UI POST /start - Start generation POST /pause - Pause generation POST /resume - Resume generation POST /anomaly - Inject anomaly DELETE /anomaly/{id} - Remove anomaly PUT /rate - Set events per second GET /state - Get current state GET /events - Stream events (SSE or WebSocket) ``` -------------------------------- ### Example Explainable Anomaly JSON Output Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/via-core.md Illustrates a JSON object representing a detailed explainable anomaly result. This output format is used to convey the anomaly's score, contributing features, contextual factors, and counterfactual explanations. ```json { "entity_id": "123456789", "timestamp": 1699900800, "value": 450.2, "overall_score": 0.92, "is_anomaly": true, "severity": "high", "attribution": { "detector": "DistributionDetectorV2", "detector_score": 0.89, "contribution_to_overall": 0.85, "expected_value": 120.5, "deviation_magnitude": 274.2, "deviation_direction": "above", "features": [ { "feature_name": "latency_p99", "contribution": 0.65, "raw_value": 450.2, "expected_range": [80.0, 180.0], "percentile": 99.8 }, { "feature_name": "histogram_mass_shift", "contribution": 0.24, "raw_value": 0.73, "expected_range": [0.0, 0.2], "percentile": 99.5 } ] }, "counterfactual": { "description": "If latency were below 210.5ms, anomaly probability drops to 0.15", "threshold_value": 210.5, "minimal_change_required": "-53%", "probability_after_change": 0.15 }, "contextual_factors": [ { "factor_type": "TimeOfDay", "description": "Anomaly occurred during peak traffic hours (14:00-16:00)", "relevance_score": 0.3, "current_context": "peak", "historical_context": "typical" } ], "recommended_actions": [ "Check for upstream service degradation", "Review recent deployment changes", "Investigate potential DoS attack pattern" ] } ``` -------------------------------- ### Run VIA-Core Benchmarks Source: https://github.com/veristamp/via-typescript/blob/main/via-core/README.md Executes performance benchmarks for VIA-Core. Includes commands for running the full benchmark suite with batching enabled and for performing targeted throughput tests. ```bash # Full benchmark suite with batching cargo run --release -p via-bench --bin via-bench -- run-all -v -b 500 # Targeted throughput test cargo run --release -p via-bench --bin via-bench -- throughput -d 1 -b 500 ``` -------------------------------- ### Get Current Active Policy Source: https://context7.com/veristamp/via-typescript/llms.txt Retrieves information about the currently active policy. This is a simple GET request to the specified endpoint. ```bash curl http://localhost:3000/control/policy/current ``` -------------------------------- ### Get Saved Schema Source: https://context7.com/veristamp/via-typescript/llms.txt Retrieves a previously saved schema for a given source name. This is a GET request to the schema endpoint with the source name as a parameter. ```bash curl http://localhost:3000/schema/payment-service ``` -------------------------------- ### Forensic Analysis - Triage Similar Events API Source: https://context7.com/veristamp/via-typescript/llms.txt Uses vector similarity search to find events similar to positive examples while excluding negative examples. ```APIDOC ## POST /analysis/triage ### Description Uses vector similarity search to find events similar to known positive examples while excluding negative examples. Ideal for interactive incident investigation. ### Method POST ### Endpoint /analysis/triage ### Parameters #### Request Body - **positive_ids** (array of strings) - Required - IDs of known positive examples. - **negative_ids** (array of strings) - Required - IDs of known negative examples. - **start_ts** (integer) - Required - The start timestamp of the time window. - **end_ts** (integer) - Required - The end timestamp of the time window. ### Request Example ```json { "positive_ids": ["evt_12345", "evt_12346"], "negative_ids": ["evt_99999"], "start_ts": 1705308600, "end_ts": 1705312200 } ``` ### Response #### Success Response (200) - **triage_results** (array of objects) - Results of the triage search. - **id** (string) - The ID of the triaged event. - **score** (number) - The similarity score. - **payload** (object) - The payload of the triaged event. - **event_id** (string) - The event ID. - **entity_type** (string) - The type of entity. - **rhythm_hash** (string) - The rhythm hash. - **score** (number) - The score of the event. - **severity** (number) - The severity of the event. - **timestamp** (integer) - The timestamp of the event. #### Response Example ```json { "triage_results": [ { "id": "evt_12350", "score": 0.94, "payload": { "event_id": "evt_12350", "entity_type": "anomaly", "rhythm_hash": "api-gateway:ERROR:connection-timeout", "score": 0.88, "severity": 0.80, "timestamp": 1705311000 } } ] } ``` ``` -------------------------------- ### Benchmark Build and Run Commands (Bash) Source: https://github.com/veristamp/via-typescript/blob/main/via-core/BENCHMARK_v2.md Provides essential bash commands for building the release version of the `via-bench` project and running various benchmark tests. These include quick, throughput, performance-stress, mixed-workload, and a comprehensive run-all command. ```bash # Build release cargo build --release -p via-bench # Run benchmarks ./target/release/via-bench quick ./target/release/via-bench throughput --duration 1 ./target/release/via-bench performance-stress ./target/release/via-bench mixed-workload --duration 1 ./target/release/via-bench run-all ``` -------------------------------- ### Via-bench Cargo.toml Dependencies for Interactive Mode (TOML) Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/via-sim-merge-plan.md This TOML snippet outlines the necessary dependency additions to the `via-bench/Cargo.toml` file to support the new interactive benchmark mode. It includes `tokio` for async operations and `axum` for the HTTP API, along with `serde_json` for JSON handling. ```toml # via-bench/Cargo.toml (additions) via-sim = { path = "../via-sim" } # Already exists via-core = { workspace = true } # Already exists # Add for interactive mode tokio = { workspace = true, features = ["full"] } axum = "0.7" # For HTTP API serde_json = { workspace = true } ``` -------------------------------- ### Build Rust Project for Library and Release (Bash) Source: https://github.com/veristamp/via-typescript/blob/main/via-core/crates/via-core/README.md Commands to build the via-core Rust project. It covers building the library in release mode and also building it with C bindings (cdylib) for cross-language compatibility. ```bash # Library cargo build --release # With C bindings (cdylib) cargo build --release --lib # Gatekeeper server cargo build --release --bin gatekeeper ``` -------------------------------- ### Triage Similar Events Source: https://context7.com/veristamp/via-typescript/llms.txt Uses vector similarity search to identify events similar to provided positive examples and exclude those similar to negative examples. It requires lists of positive and negative event IDs, along with a time window. The response lists triage results with scores and event details. ```bash curl -X POST http://localhost:3000/analysis/triage \ -H "Content-Type: application/json" \ -d '{ "positive_ids": ["evt_12345", "evt_12346"], "negative_ids": ["evt_99999"], "start_ts": 1705308600, "end_ts": 1705312200 }' ``` -------------------------------- ### Docker Compose Deployment for VIA v2 Stack Source: https://context7.com/veristamp/via-typescript/llms.txt Sets up the complete VIA v2 stack using Docker Compose, including PostgreSQL for state persistence and Qdrant for vector storage. This configuration facilitates a robust deployment environment. ```yaml version: "3.8" services: via_api: image: ghcr.io/veristamp/via-api:latest ports: - "8080:8080" environment: DATABASE_URL: "postgresql://user:password@db:5432/via" VECTOR_DB_URL: "http://qdrant:6333" depends_on: - db - qdrant db: image: postgres:15 environment: POSTGRES_USER: user POSTGRES_PASSWORD: password POSTGRES_DB: via volumes: - postgres_data:/var/lib/postgresql/data qdrant: image: qdrant/qdrant ports: - "6333:6333" volumes: - qdrant_data:/qdrant/storage volumes: postgres_data: qdrant_data: ``` -------------------------------- ### List All Schemas Source: https://context7.com/veristamp/via-typescript/llms.txt Retrieves a list of all saved schemas. This is a GET request to the base schema endpoint. ```bash curl http://localhost:3000/schema/ ``` -------------------------------- ### Run VIA-Core Unit Tests Source: https://github.com/veristamp/via-typescript/blob/main/via-core/README.md Commands to execute the internal unit tests for VIA-Core. This includes running all core tests and running specific algorithmic tests, such as those for spectral residual analysis. ```bash # Run all core tests cargo test -p via-core # Run algorithmic specific tests cargo test algo::spectral_residual ``` -------------------------------- ### Get Dead Letters Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/TIER2_SOTA_HARDENING.md Retrieves a list of events that were moved to the dead-letter queue due to repeated processing failures. This endpoint is useful for debugging and replaying events. ```APIDOC ## GET /analysis/pipeline/dead-letters ### Description Retrieves a list of events that have been moved to the dead-letter queue after exhausting all retry attempts. This endpoint is crucial for diagnosing processing issues and enabling replay capabilities. ### Method GET ### Endpoint /analysis/pipeline/dead-letters ### Parameters #### Query Parameters - **limit** (integer) - Optional - The maximum number of dead-letter entries to return. - **offset** (integer) - Optional - The number of dead-letter entries to skip (for pagination). ### Response #### Success Response (200) - **deadLetters** (array) - A list of dead-letter entries. - **eventId** (string) - The ID of the event that ended up in the dead-letter queue. - **timestamp** (string) - The timestamp when the event was moved to the dead-letter queue. - **reason** (string) - The reason for moving the event to the dead-letter queue (e.g., 'max retries exceeded'). - **originalEvent** (object) - The original event payload. #### Response Example ```json { "deadLetters": [ { "eventId": "failed-event-id-789", "timestamp": "2023-10-27T11:00:00Z", "reason": "max retries exceeded", "originalEvent": { "eventId": "failed-event-id-789", "tenantId": "tenant-abc", "entityId": "entity-xyz", "payload": { "error": "processing failed" } } } ] } ``` ``` -------------------------------- ### Run Full Test Suite (Bash) Source: https://github.com/veristamp/via-typescript/blob/main/via-core/INTEGRATION_FIX.md Command to execute the complete test suite for the `via-core` Rust project. This is essential for verifying the integrity and functionality of the implemented changes. ```bash cargo test --lib -p via-core ``` -------------------------------- ### Get Pipeline Statistics Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/TIER2_SOTA_HARDENING.md Retrieves statistics about the Tier-2 pipeline's current operational state, including queue status. This endpoint aids in pipeline observability. ```APIDOC ## GET /analysis/pipeline/stats ### Description Provides statistics and operational metrics for the Tier-2 intelligence pipeline, including details about the in-memory queue. ### Method GET ### Endpoint /analysis/pipeline/stats ### Parameters None ### Response #### Success Response (200) - **queueStats** (object) - Statistics related to the in-memory queue. - **currentSize** (integer) - The current number of items in the queue. - **maxSize** (integer) - The maximum capacity of the queue. - **acceptedCount** (integer) - The total number of events accepted into the queue. - **rejectedCount** (integer) - The total number of events rejected due to backpressure. - **dedupeWindowCount** (integer) - The number of items dropped due to deduplication. #### Response Example ```json { "queueStats": { "currentSize": 50, "maxSize": 1000, "acceptedCount": 15000, "rejectedCount": 5, "dedupeWindowCount": 12 } } ``` ``` -------------------------------- ### Get Incident Detail Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/TIER2_SOTA_HARDENING.md Retrieves detailed information about a specific incident. This endpoint is part of Phase C, providing access to the incident lifecycle and associated decisions. ```APIDOC ## GET /analysis/incidents/:incidentId ### Description Retrieves detailed information for a specific incident, including its state, associated events, and decisions made by the policy engine. ### Method GET ### Endpoint /analysis/incidents/:incidentId ### Parameters #### Path Parameters - **incidentId** (string) - Required - The unique identifier of the incident to retrieve. ### Response #### Success Response (200) - **incidentId** (string) - The unique identifier for the incident. - **status** (string) - The current lifecycle state of the incident. - **createdAt** (string) - The timestamp when the incident was created. - **updatedAt** (string) - The timestamp when the incident was last updated. - **events** (array) - A list of event IDs associated with this incident. - **decisions** (array) - A list of decisions made by the policy engine for this incident. #### Response Example ```json { "incidentId": "incident-abc-789", "status": "new", "createdAt": "2023-10-27T10:05:00Z", "updatedAt": "2023-10-27T10:05:00Z", "events": [ "unique-event-id-123", "unique-event-id-456" ], "decisions": [ { "policyId": "policy-cpu-alert", "action": "escalate", "timestamp": "2023-10-27T10:05:00Z" } ] } ``` ``` -------------------------------- ### GET /analysis/incidents/{incidentId} Source: https://context7.com/veristamp/via-typescript/llms.txt Retrieves a single incident with its associated decisions. This endpoint is useful for detailed investigation of a specific incident. ```APIDOC ## GET /analysis/incidents/{incidentId} ### Description Retrieves a single incident with its associated decisions. This endpoint is useful for detailed investigation of a specific incident. ### Method GET ### Endpoint /analysis/incidents/{incidentId} ### Parameters #### Path Parameters - **incidentId** (string) - Required - The unique identifier of the incident. #### Query Parameters None #### Request Body None ### Request Example ```bash curl http://localhost:3000/analysis/incidents/inc_abc123 ``` ### Response #### Success Response (200) - **incident** (object) - Details of the incident (schema similar to the 'incidents' array in the list endpoint). - **decisions** (array) - An array of decision objects associated with the incident. Each object contains: - **id** (integer) - The unique ID of the decision. - **incidentId** (string) - The ID of the incident this decision relates to. - **decision** (string) - The action taken (e.g., 'escalated'). - **reason** (string) - The reason for the decision. - **confidence** (integer) - The confidence score of the decision. - **policyVersion** (string) - The version of the policy used. - **createdAt** (string) - The timestamp when the decision was made. #### Response Example ```json { "incident": { "id": 1, "incidentId": "inc_abc123", "status": "escalated", "entityKey": "trace:xyz-789", "firstSeenTs": 1705310400, "lastSeenTs": 1705312200, "severityMax": 85, "scoreMax": 92, "confidence": 82, "evidence": { "trace_id": "xyz-789", "member_point_ids": ["evt_12345", "evt_12346"] }, "policyVersion": "tier2-policy-v1" }, "decisions": [ { "id": 1, "incidentId": "inc_abc123", "decision": "escalated", "reason": "reason=trace;members=5;severity_max=0.850;score_max=0.920", "confidence": 82, "policyVersion": "tier2-policy-v1", "createdAt": "2024-01-15T10:30:00.000Z" } ] } ``` ``` -------------------------------- ### GET /analysis/incidents Source: https://context7.com/veristamp/via-typescript/llms.txt Lists incidents with their decision history. Incidents are automatically created from anomaly signals and can be escalated, merged, or dismissed. ```APIDOC ## GET /analysis/incidents ### Description Lists incidents with their decision history. Incidents are automatically created from anomaly signals and can be escalated, merged, or dismissed. ### Method GET ### Endpoint /analysis/incidents ### Parameters #### Query Parameters - **limit** (integer) - Optional - The maximum number of incidents to return. #### Request Body None ### Request Example ```bash curl "http://localhost:3000/analysis/incidents?limit=50" ``` ### Response #### Success Response (200) - **incidents** (array) - An array of incident objects. Each object contains: - **id** (integer) - The internal ID of the incident. - **incidentId** (string) - The unique identifier for the incident. - **status** (string) - The current status of the incident (e.g., 'escalated'). - **entityKey** (string) - The key identifying the entity associated with the incident. - **firstSeenTs** (integer) - The timestamp when the incident was first detected. - **lastSeenTs** (integer) - The timestamp when the incident was last seen. - **severityMax** (integer) - The maximum severity score observed for the incident. - **scoreMax** (integer) - The maximum score observed for the incident. - **confidence** (integer) - The confidence level of the incident detection. - **evidence** (object) - Details about the evidence supporting the incident. - **policyVersion** (string) - The version of the policy used for detection. #### Response Example ```json { "incidents": [ { "id": 1, "incidentId": "inc_abc123", "status": "escalated", "entityKey": "trace:xyz-789", "firstSeenTs": 1705310400, "lastSeenTs": 1705312200, "severityMax": 85, "scoreMax": 92, "confidence": 82, "evidence": { "trace_id": "xyz-789", "member_point_ids": ["evt_12345", "evt_12346"] }, "policyVersion": "tier2-policy-v1" } ] } ``` ``` -------------------------------- ### Run Rust Project Tests (Bash) Source: https://github.com/veristamp/via-typescript/blob/main/via-core/crates/via-core/README.md Command to execute all tests within the via-core Rust project. This is useful for verifying the integrity and correctness of the anomaly detection engine. ```bash cargo test --lib # 66 tests, 0 failures ``` -------------------------------- ### Add Real-time Benchmark Mode Command (Rust) Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/via-sim-merge-plan.md This Rust code snippet shows the addition of a new `Interactive` subcommand to the CLI for the benchmark tool. This command enables an interactive benchmarking mode with real-time anomaly injection and monitoring capabilities, configurable via duration and port. ```rust // In main.rs #[derive(Subcommand)] enum Commands { // ... existing commands ... /// Interactive benchmark with real-time anomaly injection Interactive { /// Duration in minutes #[arg(short, long, default_value = "10")] duration: u64, /// Web UI port #[arg(short, long, default_value = "8081")] port: u16, }, } fn run_interactive_benchmark(duration: u64, port: u16) { // Start HTTP server for: // - Inject anomalies during run // - View real-time metrics // - Export results println!("Interactive benchmark mode on port {}", port); } ``` -------------------------------- ### GET /analysis/pipeline/dead-letters Source: https://context7.com/veristamp/via-typescript/llms.txt Retrieves failed events from the dead letter queue. This is crucial for debugging and understanding why certain events could not be processed. ```APIDOC ## GET /analysis/pipeline/dead-letters ### Description Retrieves failed events from the dead letter queue. This is crucial for debugging and understanding why certain events could not be processed. ### Method GET ### Endpoint /analysis/pipeline/dead-letters ### Parameters #### Query Parameters - **limit** (integer) - Optional - The maximum number of dead letter entries to return. #### Request Body None ### Request Example ```bash curl "http://localhost:3000/analysis/pipeline/dead-letters?limit=50" ``` ### Response #### Success Response (200) - **dead_letters** (array) - An array of dead letter objects. Each object contains: - **id** (integer) - The unique ID of the dead letter entry. - **eventId** (string) - The ID of the event that failed. - **reason** (string) - The reason for the event failure. - **payload** (object) - The original payload of the failed event. - **createdAt** (string) - The timestamp when the event was added to the dead letter queue. #### Response Example ```json { "dead_letters": [ { "id": 1, "eventId": "evt_failed_123", "reason": "embedding_service_unavailable", "payload": {}, "createdAt": "2024-01-15T10:30:00.000Z" } ] } ``` ``` -------------------------------- ### GET /analysis/pipeline/stats Source: https://context7.com/veristamp/via-typescript/llms.txt Retrieves statistics for the Tier-2 processing pipeline. This endpoint helps monitor the health and throughput of the anomaly detection system. ```APIDOC ## GET /analysis/pipeline/stats ### Description Retrieves statistics for the Tier-2 processing pipeline. This endpoint helps monitor the health and throughput of the anomaly detection system. ### Method GET ### Endpoint /analysis/pipeline/stats ### Parameters #### Query Parameters None #### Request Body None ### Request Example ```bash curl http://localhost:3000/analysis/pipeline/stats ``` ### Response #### Success Response (200) - **queue** (object) - Contains statistics about the processing queue: - **length** (integer) - The current number of items in the queue. - **processed** (integer) - The total number of items processed. - **dropped** (integer) - The number of items dropped from the queue. - **isRunning** (boolean) - Indicates if the pipeline is currently running. #### Response Example ```json { "queue": { "length": 42, "processed": 15000, "dropped": 3, "isRunning": true } } ``` ``` -------------------------------- ### Production Profile Configuration (Rust) Source: https://github.com/veristamp/via-typescript/blob/main/via-core/BENCHMARK_v2.md Defines the `ProfileConfig` struct for production settings, including Holt-Winters smoothing parameters (hw_alpha, hw_beta, hw_gamma), seasonal period, histogram bins, and histogram decay factor. These settings are crucial for time-series analysis and anomaly detection. ```rust ProfileConfig { hw_alpha: 0.1, // Holt-Winters smoothing hw_beta: 0.05, // Trend smoothing hw_gamma: 0.1, // Seasonal smoothing period: 60, // Seasonal period (seconds) hist_bins: 50, // Histogram resolution hist_decay: 0.95, // Histogram decay factor } ``` -------------------------------- ### Rust Library Usage for Anomaly Detection Source: https://github.com/veristamp/via-typescript/blob/main/via-core/crates/via-core/README.md Demonstrates how to use the via-core engine as a library in Rust. It shows the process of creating or retrieving a profile for an entity and updating it with new data to detect anomalies. ```rust use via_core::{ViaProfile, engine::Engine}; let mut engine = Engine::new(); let profile = engine.get_or_create_profile(entity_id); let (score, is_anomaly) = profile.update(value, timestamp_ns); ``` -------------------------------- ### GET /analysis/incident/{incidentId} Source: https://context7.com/veristamp/via-typescript/llms.txt Retrieves the incident graph for a specific incident ID. This provides a visual representation of related events and their connections. ```APIDOC ## GET /analysis/incident/{incidentId} ### Description Retrieves the incident graph for a specific incident ID. This provides a visual representation of related events and their connections. ### Method GET ### Endpoint /analysis/incident/{incidentId} ### Parameters #### Path Parameters - **incidentId** (string) - Required - The unique identifier of the incident. #### Query Parameters None #### Request Body None ### Request Example ```bash curl http://localhost:3000/analysis/incident/inc_abc123 ``` ### Response #### Success Response (200) - **graph** (array) - An array of objects representing the incident graph. Each object contains: - **id** (integer) - The unique ID of the graph node. - **metaIncidentId** (string) - The ID of the meta-incident. - **qdrantPointId** (string) - The ID of the corresponding point in Qdrant. - **linkType** (string) - The type of link between nodes (e.g., 'trace'). - **confidence** (integer) - The confidence score of the link. - **createdAt** (string) - The timestamp when the graph node was created. #### Response Example ```json { "graph": [ { "id": 1, "metaIncidentId": "inc_abc123", "qdrantPointId": "evt_12345", "linkType": "trace", "confidence": 100, "createdAt": "2024-01-15T10:30:00.000Z" } ] } ``` ``` -------------------------------- ### Metrics API Source: https://github.com/veristamp/via-typescript/blob/main/via-core/README.md Exposes system metrics in Prometheus format for monitoring. ```APIDOC ## GET /metrics ### Description Retrieves system and performance metrics in Prometheus exposition format. ### Method GET ### Endpoint /metrics ### Response #### Success Response (200) - Metrics in Prometheus text format. #### Response Example ```text # HELP http_requests_total Total number of HTTP requests # TYPE http_requests_total counter http_requests_total{method="POST",path="/ingest/batch"} 1500 # HELP gatekeeper_processed_events_total Total number of events processed by the gatekeeper # TYPE gatekeeper_processed_events_total counter gatekeeper_processed_events_total 1000000 ``` ``` -------------------------------- ### Rust - AnomalyProfile::default Source: https://context7.com/veristamp/via-typescript/llms.txt Creates a new AnomalyProfile with default configuration settings. This is the simplest way to start using the anomaly detection engine. ```APIDOC ## Rust - AnomalyProfile::default ### Description Creates a new AnomalyProfile with default configuration settings. This is the simplest way to start using the anomaly detection engine. ### Method Rust function call ### Endpoint N/A ### Parameters None ### Request Example ```rust use via_core::AnomalyProfile; let mut profile = AnomalyProfile::default(); ``` ### Response #### Success Response - **AnomalyProfile** (struct) - An instance of the AnomalyProfile struct initialized with default parameters. #### Response Example ```rust // profile variable now holds an AnomalyProfile instance ``` ``` -------------------------------- ### Model Persistence and Recovery in Rust Source: https://github.com/veristamp/via-typescript/blob/main/docs/plans/via-core.md Implements mechanisms for saving and restoring learned anomaly profiles to enable zero-warmup restarts. It includes structures for managing model storage, defining recovery strategies, and performing the actual recovery process. Dependencies include `uuid`, `tokio`, `serde`, and `bincode`. ```rust pub struct ModelStorage { storage_dir: PathBuf, checkpoint_interval: Duration, max_checkpoints: usize, encoder: BincodeEncoder, compression: Lz4, } impl ModelStorage { pub async fn checkpoint(&self, profiles: &HashMap) -> Result { let checkpoint = Checkpoint { id: Uuid::new_v4(), timestamp: Instant::now(), version: VERSION, profiles: profiles .iter() .filter(|(id, p)| p.should_persist()) .map(|(id, p)| (id, p.serialize())) .collect(), metadata: self.collect_metrics(), }; // Atomic write with rename let temp_path = self.storage_dir.join(format!("checkpoint_{}.tmp", checkpoint.id)); self.encode_to_file(&checkpoint, &temp_path).await?; fs::rename(&temp_path, self.path_for_id(checkpoint.id)).await?; // Maintain only N checkpoints self.prune_old_checkpoints().await?; Ok(checkpoint.id) } } pub struct RecoveryManager { storage: ModelStorage, recovery_strategy: RecoveryStrategy, } pub enum RecoveryStrategy { Full, PriorityBased(usize), Adaptive { min_entities: usize, max_recovery_time: Duration, }, } impl RecoveryManager { pub async fn recover(&self) -> Result, RecoveryError> { let latest = self.storage.latest_checkpoint().await?; if let Some(checkpoint) = latest { info!("Recovering {} profiles from checkpoint {}", checkpoint.profiles.len(), checkpoint.id); let profiles = match self.recovery_strategy { RecoveryStrategy::Full => self.load_all(&checkpoint).await, RecoveryStrategy::PriorityBased(n) => self.load_by_priority(&checkpoint, n).await, RecoveryStrategy::Adaptive { .. } => self.load_adaptive(&checkpoint).await, }?; info!("Recovered {} profiles", profiles.len()); Ok(profiles) } else { info!("No checkpoint found, starting fresh"); Ok(HashMap::new()) } } } ``` -------------------------------- ### Get Single Incident with Decisions (API) Source: https://context7.com/veristamp/via-typescript/llms.txt Retrieves details of a single incident, including its decision history. This is useful for understanding the lifecycle and resolution of an incident. ```bash curl http://localhost:3000/analysis/incidents/inc_abc123 ```