### Quick Start: Chaos Lab Setup and Experiment Execution Source: https://github.com/tracer-cloud/opensre/blob/main/tests/chaos_engineering/README.md Use these Make commands for a quick setup of the chaos lab environment, listing experiments, running a specific experiment, and tearing down the lab. Ensure Docker, Kind, kubectl, Helm, and a Python virtual environment are installed. ```bash export DD_API_KEY='your-api-key' # omit if using CHAOS_LAB_FLAGS=--skip-datadog # export KUBECTL_CONTEXT=kind-tracer-k8s-test # optional make chaos-lab-up make chaos-experiment-list make chaos-experiment-up EXPERIMENT=pod-failure make chaos-experiment-down EXPERIMENT=pod-failure make chaos-lab-down ``` -------------------------------- ### OpenSRE Quick Start Commands Source: https://github.com/tracer-cloud/opensre/blob/main/README.md Executes the initial setup and investigation commands for OpenSRE. This sequence is used to onboard, investigate an incident, and update the system. ```bash opensre onboard opensre investigate -i tests/e2e/kubernetes/fixtures/datadog_k8s_alert.json opensre update ``` -------------------------------- ### Manual Chaos Mesh and Baseline Workload Setup Source: https://github.com/tracer-cloud/opensre/blob/main/tests/chaos_engineering/README.md Apply Chaos Mesh and baseline workloads to the Kind cluster. This involves installing Chaos Mesh, verifying its pods, applying the base chaos-demo and pod-kill-demo, and verifying the default namespace pods. ```bash make chaos-mesh-up kubectl get pods -n chaos-mesh --context=kind-tracer-k8s-test make chaos-engineering-apply kubectl get pods -n default --context=kind-tracer-k8s-test ``` -------------------------------- ### Install Make using Chocolatey Source: https://github.com/tracer-cloud/opensre/blob/main/SETUP.md Install the 'make' utility on Windows using Chocolatey. This command should be run after Chocolatey has been installed. ```powershell choco install make ``` -------------------------------- ### Verify Setup with Make Commands Source: https://github.com/tracer-cloud/opensre/blob/main/SETUP.md Run linting, type checking, and tests with coverage to verify the development environment setup. All checks must pass. ```bash make lint && make typecheck && make test-cov ``` -------------------------------- ### Install Make using winget Source: https://github.com/tracer-cloud/opensre/blob/main/SETUP.md Install the 'make' utility on Windows using the winget package manager. This is an alternative to using Chocolatey. ```powershell winget install GnuWin32.Make ``` -------------------------------- ### Install OpenSRE Source: https://github.com/tracer-cloud/opensre/blob/main/docs/introduction-monitoring.mdx Run this command to install OpenSRE. Once installed, it begins tracing all jobs and processes in real time. ```bash curl -sSL https://install.tracer.cloud | sh -s ``` -------------------------------- ### Install Mintlify CLI Source: https://github.com/tracer-cloud/opensre/blob/main/docs/README.md Installs the Mintlify CLI globally. This is a prerequisite for local development. ```bash npm i -g mint ``` -------------------------------- ### Run Mintlify Development Server Source: https://github.com/tracer-cloud/opensre/blob/main/docs/README.md Starts the local development server for previewing documentation. Access the docs at http://localhost:3000. ```bash mint dev ``` -------------------------------- ### OpenSRE Onboard Wizard Source: https://context7.com/tracer-cloud/opensre/llms.txt Initiates an interactive setup wizard to configure LLM credentials and validate integrations. Use `local_llm` for zero-config local LLM setup via Ollama. ```bash # Full interactive wizard opensre onboard ``` ```bash # Zero-config local LLM via Ollama (no API key needed) opensre onboard local_llm ``` -------------------------------- ### Setup AWS Infrastructure for RNA-seq Pipeline Source: https://github.com/tracer-cloud/opensre/blob/main/docs/environments/aws-batch.mdx Run the setup script to provision the necessary AWS infrastructure, including VPC, S3 buckets, AWS Batch compute environment, and IAM roles. This script automates the creation of all required AWS resources. ```bash cd nextflow-test-pipelines/pipelines/aws-batch/rnaseq ./run.sh setup ``` -------------------------------- ### Clone Repository and Install OpenSRE Source: https://github.com/tracer-cloud/opensre/blob/main/README.md Clones the OpenSRE repository from GitHub and installs the project locally using make. This is the initial step for setting up a development environment. ```bash git clone https://github.com/Tracer-Cloud/opensre cd opensre make install ``` -------------------------------- ### Start OpenSRE Agent Source: https://github.com/tracer-cloud/opensre/blob/main/docs/environments/codespaces.mdx Initialize the OpenSRE agent to start tracking pipelines. You will be prompted to configure a pipeline name. Obtain your personal token from the onboarding page. ```bash tracer init --token ``` -------------------------------- ### Install Hub helpers for OpenSRE Source: https://github.com/tracer-cloud/opensre/blob/main/tests/benchmarks/openrca_scenarios/README.md Install the OpenSRE package with the 'opensre-hub' extra, which includes dependencies for interacting with the Hugging Face Hub. ```bash pip install 'opensre[opensre-hub]' ``` -------------------------------- ### Install OpenSRE with Homebrew Source: https://github.com/tracer-cloud/opensre/blob/main/README.md Installs the OpenSRE framework using the Homebrew package manager. Ensure Homebrew is installed and configured before running this command. ```bash brew install Tracer-Cloud/opensre/opensre ``` -------------------------------- ### Manual Datadog Installation via Helm Source: https://github.com/tracer-cloud/opensre/blob/main/tests/chaos_engineering/README.md Install and configure the Datadog agent on the Kind cluster using Helm. This includes setting the API key, namespace, and potentially the Datadog site. Ensure the Datadog Helm repository is added and updated. ```bash export DD_API_KEY='your-api-key' # export DD_SITE=datadoghq.eu # if not US1 kubectl create namespace tracer-test --context=kind-tracer-k8s-test --dry-run=client -o yaml \ | kubectl apply -f - --context=kind-tracer-k8s-test helm repo add datadog https://helm.datadoghq.com helm repo update datadog helm upgrade --install datadog datadog/datadog \ --kube-context kind-tracer-k8s-test \ -n tracer-test \ -f tests/e2e/kubernetes/k8s_manifests/datadog-values.yaml \ --set "datadog.apiKey=${DD_API_KEY}" \ ${DD_SITE:+--set datadog.site=${DD_SITE}} \ --wait --timeout 10m ``` -------------------------------- ### Initialize OpenSRE Agent Source: https://github.com/tracer-cloud/opensre/blob/main/docs/tutorials/fastquorum-debugging.mdx Starts the OpenSRE agent to begin tracking pipeline metrics. Replace '(your token)' with your actual user token. ```bash tracer init --token eyJh---- (your token) ``` -------------------------------- ### Verify All Local Integrations with CLI Source: https://github.com/tracer-cloud/opensre/blob/main/docs/integrations-overview.mdx Run this command after setup to confirm that all configured local integrations are functioning correctly. It checks the status of all connected services. ```bash opensre integrations verify ``` -------------------------------- ### Initialize Pipeline Tracking Source: https://github.com/tracer-cloud/opensre/blob/main/docs/environments/macos.mdx Run this command to configure and start tracking a new pipeline. You will be prompted to enter a unique pipeline name for identification and searchability. ```bash sudo tracer init ``` -------------------------------- ### Install OpenSRE CLI Source: https://context7.com/tracer-cloud/opensre/llms.txt Choose the installation method appropriate for your operating system. For macOS, Homebrew is recommended. Linux and WSL users can use curl, while Windows users can use PowerShell. Building from source requires Python 3.11+. ```bash # macOS (Homebrew) brew install Tracer-Cloud/opensre/opensre ``` ```bash # Linux / WSL (curl) curl -fsSL https://raw.githubusercontent.com/Tracer-Cloud/opensre/main/install.sh | bash ``` ```powershell # Windows PowerShell irm https://raw.githubusercontent.com/Tracer-Cloud/opensre/main/install.ps1 | iex ``` ```bash # From source (Python 3.11+) git clone https://github.com/Tracer-Cloud/opensre && cd opensre make install ``` -------------------------------- ### Setup ClickHouse via Interactive CLI Source: https://github.com/tracer-cloud/opensre/blob/main/docs/clickhouse.mdx Use the OpenSRE CLI to set up the ClickHouse integration by selecting it and providing host and credentials when prompted. ```bash opensre integrations setup ``` -------------------------------- ### Setup RabbitMQ Integration via CLI Source: https://github.com/tracer-cloud/opensre/blob/main/docs/rabbitmq.mdx Use the OpenSRE CLI to set up the RabbitMQ integration interactively. You will be prompted for necessary connection details. ```bash opensre integrations setup rabbitmq ``` -------------------------------- ### Initialize OpenSRE Agent Source: https://github.com/tracer-cloud/opensre/blob/main/docs/environments/linux-cloud.mdx Start tracking a pipeline by initializing the OpenSRE agent with your unique token. You will be prompted to configure the pipeline name. ```bash sudo tracer init --token ``` -------------------------------- ### Setup MariaDB Integration via CLI Source: https://github.com/tracer-cloud/opensre/blob/main/docs/mariadb.mdx Use the OpenSRE CLI to interactively set up the MariaDB integration. You will be prompted for connection details and SSL preferences. ```bash opensre integrations setup mariadb ``` -------------------------------- ### Setup Bitbucket Integration via Environment Variables Source: https://github.com/tracer-cloud/opensre/blob/main/docs/bitbucket.mdx Configure Bitbucket integration by setting the BITBUCKET_WORKSPACE, BITBUCKET_USERNAME, and BITBUCKET_APP_PASSWORD environment variables in your .env file. ```bash BITBUCKET_WORKSPACE=your-workspace-slug BITBUCKET_USERNAME=your-username BITBUCKET_APP_PASSWORD=your-app-password ``` -------------------------------- ### AWS Batch EC2 LaunchTemplate UserData Example Source: https://github.com/tracer-cloud/opensre/blob/main/docs/environments/aws-batch.mdx This is an example of the UserData field within an EC2 Launch Template for AWS Batch. It shows where to append OpenSRE installation commands. ```yaml Resources: NextflowLaunchTemplate: Properties: LaunchTemplateData: UserData: !Base64 | #cloud-config runcmd: - ... ``` -------------------------------- ### Get OpenSRE Version Information Source: https://context7.com/tracer-cloud/opensre/llms.txt Displays the installed version of OpenSRE, the Python version, and operating system details. The `--json` flag provides this information in a machine-readable format. ```bash opensre version ``` ```bash # opensre 2026.4.5 # Python 3.13.0 # OS linux (x86_64) ``` ```bash opensre version --json ``` ```json # {"opensre": "2026.4.5", "python": "3.13.0", "os": "linux", "arch": "x86_64"} ``` -------------------------------- ### Context Source Annotations for Investigation Source: https://github.com/tracer-cloud/opensre/blob/main/tests/AGENTS.md This Python dictionary example shows how to use annotations to declare available evidence sources and provide specific details for each source. This guides the agent's investigation strategy by indicating which data is accessible. ```python annotations={ # Evidence source declarations "context_sources": "s3,lambda,cloudwatch", # S3 context "s3_bucket": "landing-bucket", "s3_key": "raw/data/2024/file.json", # Lambda context "function_name": "processor-function", "lambda_log_group": "/aws/lambda/processor", # CloudWatch context "cloudwatch_log_group": "/ecs/pipeline", "correlation_id": "run-123", } ``` -------------------------------- ### Launch Demo Pipeline Source: https://github.com/tracer-cloud/opensre/blob/main/docs/environments/codespaces.mdx Execute this command to launch a prepared nf-core fastquorum pipeline for demonstration purposes. This pipeline has specific resource requirements. ```bash tracer demo ``` -------------------------------- ### Run Demo Pipeline Source: https://github.com/tracer-cloud/opensre/blob/main/docs/environments/linux-cloud.mdx Launch a sample nf-core fastquorum pipeline to test OpenSRE tracking. This pipeline has specific resource requirements. ```bash sudo tracer demo ``` -------------------------------- ### Install OpenSRE Source: https://github.com/tracer-cloud/opensre/blob/main/docs/environments/codespaces.mdx Run this command to install OpenSRE in your Codespaces environment. Ensure you have curl installed. ```bash curl -sSL https://install.tracer.cloud | sh ``` -------------------------------- ### Run Documentation Development with Make Source: https://github.com/tracer-cloud/opensre/blob/main/docs/README.md Alternative command to run the documentation development server from the project root. ```bash make docs-dev ``` -------------------------------- ### Manual Cluster Setup with Kind Source: https://github.com/tracer-cloud/opensre/blob/main/tests/chaos_engineering/README.md Manually create and configure a Kind Kubernetes cluster. This involves creating the cluster with a specified name and waiting for it to become ready, then setting the kubectl context. ```bash kind create cluster --name tracer-k8s-test --wait 60s kubectl config use-context kind-tracer-k8s-test ``` -------------------------------- ### Onboard OpenSRE and Investigate Alerts Source: https://github.com/tracer-cloud/opensre/blob/main/README.md Runs the OpenSRE onboarding process to configure local LLM providers and integrations. It also demonstrates how to investigate alerts using a provided JSON fixture. ```bash opensre onboard opensre investigate -i tests/e2e/kubernetes/fixtures/datadog_k8s_alert.json ``` -------------------------------- ### Install OpenSRE with cURL Source: https://github.com/tracer-cloud/opensre/blob/main/README.md Installs the OpenSRE framework using a cURL command. This is a common method for fetching and executing installation scripts. ```bash curl -fsSL https://raw.githubusercontent.com/Tracer-Cloud/opensre/main/install.sh | bash ``` -------------------------------- ### Install Development Dependencies Source: https://github.com/tracer-cloud/opensre/blob/main/SETUP.md Install the project's development dependencies using pip. This command installs the package in editable mode with development extras. ```bash pip install -e ".[dev]" ``` -------------------------------- ### Install OpenSRE with PowerShell Source: https://github.com/tracer-cloud/opensre/blob/main/README.md Installs the OpenSRE framework using PowerShell. This is suitable for Windows environments where PowerShell is available. ```powershell irm https://raw.githubusercontent.com/Tracer-Cloud/opensre/main/install.ps1 | iex ``` -------------------------------- ### Start MCP Server Source: https://context7.com/tracer-cloud/opensre/llms.txt Command to start the MCP server. This server exposes the `run_rca` tool for root cause analysis. ```bash opensre-mcp # or via uvx: uvx opensre-mcp ``` -------------------------------- ### Install OpenSRE Agent in UserData Source: https://github.com/tracer-cloud/opensre/blob/main/docs/environments/aws-batch.mdx Append this command to the 'runcmd' section of your EC2 Launch Template UserData to install the OpenSRE agent. ```bash - curl -sSL https://install.tracer.cloud | sh ``` -------------------------------- ### Setup Honeycomb Integration via Environment Variables Source: https://github.com/tracer-cloud/opensre/blob/main/docs/honeycomb.mdx Configure Honeycomb integration by setting environment variables. HONEYCOMB_DATASET is optional and defaults to '__all__'. HONEYCOMB_API_URL can be overridden for specific regions or proxies. ```bash HONEYCOMB_API_KEY=your-api-key HONEYCOMB_DATASET=your-dataset # optional, defaults to __all__ HONEYCOMB_API_URL=https://api.honeycomb.io # optional ``` -------------------------------- ### Setup Alertmanager Integration via CLI Source: https://github.com/tracer-cloud/opensre/blob/main/docs/alertmanager.mdx Use the interactive CLI wizard to set up the Alertmanager integration. Follow the prompts to provide the Alertmanager URL and authentication method. ```bash opensre integrations setup alertmanager ``` -------------------------------- ### Install Chocolatey Package Manager Source: https://github.com/tracer-cloud/opensre/blob/main/SETUP.md Install Chocolatey, a package manager for Windows, by running a PowerShell script. Review the script before execution. ```powershell Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1')) ``` -------------------------------- ### Start OpenSRE MCP Server Source: https://github.com/tracer-cloud/opensre/blob/main/SETUP.md Start the OpenSRE MCP server. This command makes the 'run_rca' tool available for MCP clients. ```bash opensre-mcp ``` -------------------------------- ### Install OpenSRE Agent Source: https://github.com/tracer-cloud/opensre/blob/main/docs/tutorials/fastquorum-debugging.mdx Installs the OpenSRE agent using a curl command. Ensure you are using the correct CLI branch and user token. ```bash curl -sSL https://install.tracer.cloud | CLI_BRANCH=dev sh -s user_35Fukh3QxSAxJLgfyE9SwPoPy9K ``` -------------------------------- ### Setup Better Stack Integration via CLI Source: https://github.com/tracer-cloud/opensre/blob/main/docs/betterstack.mdx Use this command to interactively set up the Better Stack integration. You will be prompted for necessary credentials and configuration details. ```bash opensre integrations setup betterstack ``` -------------------------------- ### Update OpenSRE Source: https://context7.com/tracer-cloud/opensre/llms.txt Manages OpenSRE self-updates. Use `--check` to see if a newer version is available without installing, and `--yes` to download and install the latest release. ```bash # Check if a newer version is available (no install) opensre update --check ``` ```bash # Download and install the newest release opensre update --yes ``` -------------------------------- ### Nextflow Pipeline Example Source: https://github.com/tracer-cloud/opensre/blob/main/docs/quickstart-monitoring.mdx This is an example of a Nextflow pipeline configuration file that includes various modules and subworkflows for bioinformatics analysis. It demonstrates how different components are included and potentially integrated with monitoring tools. ```groovy /* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ IMPORT MODULES / SUBWORKFLOWS / FUNCTIONS ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ */ include { paramsSummaryMap } from 'plugin/nf-schema' include { paramsSummaryMultiqc } from '../subworkflows/nf-core/utils_nfcore_pipeline' include { softwareVersionsToYAML } from '../subworkflows/nf-core/utils_nfcore_pipeline' include { methodsDescriptionText } from '../subworkflows/local/utils_nfcore_fastquorum_pipeline' include { ALIGN_BAM as ALIGN_RAW_BAM } from '../modules/local/align_bam/main' include { ALIGN_BAM as ALIGN_CONSENSUS_BAM } from '../modules/local/align_bam/main' include { FASTQC } from '../modules/nf-core/fastqc/main' include { FGBIO_FASTQTOBAM as FASTQTOBAM } from '../modules/local/fgbio/fastqtobam/main' include { FGBIO_GROUPREADSBYUMI as GROUPREADSBYUMI } from '../modules/local/fgbio/groupreadsbyumi/main' include { FGBIO_CALLMOLECULARCONSENSUSREADS as CALLMOLECULARCONSENSUSREADS } from '../modules/local/fgbio/callmolecularconsensusreads/main' include { FGBIO_CALLDDUPLEXCONSENSUSREADS as CALLDDUPLEXCONSENSUSREADS } from '../modules/local/fgbio/callduplexconsensusreads/main' include { FGBIO_FILTERCONSENSUSREADS as FILTERCONSENSUSREADS } from '../modules/local/fgbio/filterconsensusreads/main' include { FGBIO_COLLECTDUPLEXSEQMETRICS as COLLECTDUPLEXSEQMETRICS } from '../modules/local/fgbio/collectduplexseqmetrics/main' include { FGBIO_CALLANDFILTERMOLECULARCONSENSUSREADS as CALLANDFILTERMOLECULARCONSENSUSREADS } from '../modules/local/fgbio/callandfiltermolecularconsensusreads/main' include { FGBIO_CALLANDFILTERDUPLEXCONSENSUSREADS as CALLANDFILTERDUPLEXCONSENSUSREADS } from '../modules/local/fgbio/callandfilterduplexconsensusreads/main' include { SAMTOOLS_MERGE as MERGE_BAM } from '../modules/nf-core/samtools/merge/main' include { MULTIQC } from '../modules/nf-core/multiqc/main' /* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``` -------------------------------- ### LLMSettings.from_env Source: https://context7.com/tracer-cloud/opensre/llms.txt Builds LLM configuration from environment variables, reading provider and API keys. ```APIDOC ## LLMSettings.from_env ### Description Reads LLM configuration from environment variables, specifically `LLM_PROVIDER` and the corresponding API key. It validates the combination and returns a Pydantic model. Providers that do not require an API key are whitelisted. ### Method Python Function Call ### Parameters None (reads from environment variables) ### Request Example ```python import os from app.config import LLMSettings # Example for Anthropic os.environ["LLM_PROVIDER"] = "anthropic" os.environ["ANTHROPIC_API_KEY"] = "sk-ant-..." settings = LLMSettings.from_env() print(settings.provider) # Output: "anthropic" # Example for Ollama (local, no API key) os.environ["LLM_PROVIDER"] = "ollama" os.environ["OLLAMA_MODEL"] = "llama3.2" os.environ["OLLAMA_HOST"] = "http://localhost:11434" settings = LLMSettings.from_env() # Example for Amazon Bedrock (IAM auth, no API key) os.environ["LLM_PROVIDER"] = "bedrock" os.environ["AWS_REGION"] = "us-east-1" settings = LLMSettings.from_env() print(settings.bedrock_reasoning_model) # Output: "us.anthropic.claude-sonnet-4-6" ``` ### Response #### Success Response (200) Returns a Pydantic model representing the LLM configuration. * **provider** (str) - The configured LLM provider (e.g., "anthropic", "ollama"). * **anthropic_reasoning_model** (str) - The reasoning model for Anthropic. * **bedrock_reasoning_model** (str) - The reasoning model for Bedrock. #### Response Example ```json { "provider": "anthropic", "anthropic_reasoning_model": "claude-sonnet-4-6", "anthropic_toolcall_model": "claude-haiku-4-5-20251001", "bedrock_reasoning_model": "us.anthropic.claude-sonnet-4-6" } ``` ``` -------------------------------- ### Bug Report Example - Expected vs. Actual Behavior Source: https://github.com/tracer-cloud/opensre/blob/main/CONTRIBUTING.md This example demonstrates how to structure the 'Expected Behavior' and 'Actual Behavior' sections of a bug report, including specific commands and error messages. ```markdown ### Expected Behavior `opensre investigate --org myorg` should return investigation results ### Actual Behavior Command exits silently with no output Error: exit code 0 ``` -------------------------------- ### Launch nf-core/fastquorum Pipeline Source: https://github.com/tracer-cloud/opensre/blob/main/docs/tutorials/fastquorum-debugging.mdx Executes the nf-core/fastquorum pipeline with specified inputs and configurations. Includes options for trace and report generation. ```bash nextflow run . --input samplesheet.csv \ --fasta data/chr17.fa \ --outdir results \ --duplex_seq true \ -profile test,docker \ -with-trace \ -with-report results/report.html ``` -------------------------------- ### Bug Report Example - Steps to Reproduce and Environment Source: https://github.com/tracer-cloud/opensre/blob/main/CONTRIBUTING.md Provides clear, minimal steps to consistently trigger a bug and details the necessary environment information for effective debugging. ```markdown ### Steps to Reproduce 1. Run `opensre investigate --org myorg` 2. Observe output ### Environment - OS: macOS 14.2 - Python: 3.11.5 - opensre version: v0.2.1 ``` -------------------------------- ### Update OpenSRE Source: https://github.com/tracer-cloud/opensre/blob/main/docs/quickstart.mdx Run this command to update your OpenSRE installation to the latest version. ```bash opensre update ``` -------------------------------- ### Verify OpsGenie Integration Source: https://github.com/tracer-cloud/opensre/blob/main/docs/opsgenie.mdx Run this command to verify the OpsGenie integration is set up correctly and can connect to the service. ```bash opensre integrations verify opsgenie ```