### Clone and Build ComposeFlux Source: https://github.com/veerendra2/composeflux/blob/main/docs/Development.md Clone the repository, download dependencies, and build the application binary. This is the initial setup for development. ```bash git clone https://github.com/veerendra2/composeflux.git cd composeflux go mod download task install # Install dev tools task build ./dist/composeflux --help ``` -------------------------------- ### Install MkDocs Material Theme Source: https://github.com/veerendra2/composeflux/blob/main/docs/Development.md Install the MkDocs Material theme using pip. This is required for serving the documentation locally. ```bash pip install mkdocs-material ``` -------------------------------- ### Stack Configuration Example Source: https://github.com/veerendra2/composeflux/blob/main/docs/Introduction.md An example of a stack.yml configuration file. This file controls deployment order and defines common environment variables for all stacks within the STACK_PATH. ```yaml # Only list stacks that need specific order # Everything else deploys in whatever order startup_order: - traefik # Must match the directory name in STACK_PATH # Common variables available to all stacks envs: DOMAIN: homeserver.local TZ: America/New_York ENVIRONMENT: production ``` -------------------------------- ### Set Up Python Virtual Environment for MkDocs Source: https://github.com/veerendra2/composeflux/blob/main/docs/Development.md Manually set up a Python virtual environment for installing MkDocs dependencies. This is an alternative to using direnv. ```bash python3 -m venv venv/ source venv/bin/activate ``` -------------------------------- ### Go Import Organization Example Source: https://github.com/veerendra2/composeflux/blob/main/AGENTS.md Demonstrates the standard Go import grouping: standard library, third-party packages, and internal module packages, separated by blank lines. ```go import ( // 1. Standard library "context" "fmt" "log/slog" // 2. Third-party "github.com/alecthomas/kong" mobyClient "github.com/moby/moby/client" // 3. Internal module "github.com/veerendra2/composeflux/internal/reconcile" "github.com/veerendra2/composeflux/pkg/secrets" ) ``` -------------------------------- ### Start ComposeFlux Service Source: https://github.com/veerendra2/composeflux/blob/main/docs/GettingStarted.md Commands to start the ComposeFlux service in daemon mode and view its logs. Assumes Docker Compose is set up. ```bash # Default: Run in daemon mode (continuous reconciliation) docker compose up -d docker compose logs -f ``` -------------------------------- ### Stack Discovery Structure Example Source: https://github.com/veerendra2/composeflux/blob/main/docs/Introduction.md Illustrates the directory structure for stack discovery. ComposeFlux only discovers stacks one level deep, so nested directories are not processed. ```text stacks/ ├── app1/ ← Discovered ✓ │ └── compose.yml ├── app2/ ← Discovered ✓ │ └── compose.yml └── nested/ └── app3/ ← NOT discovered ✗ └── compose.yml ``` -------------------------------- ### Running Specific Go Tests Source: https://github.com/veerendra2/composeflux/blob/main/AGENTS.md Examples for executing individual Go test functions or all tests within a package, including options for verbose output and the race detector. ```bash # Run one test function in a specific package go test -v ./internal/reconcile/... -run TestFunctionName # Run all tests in a package go test ./pkg/secrets/... # Run with race detector go test -race ./... ``` -------------------------------- ### Go Interface and Concrete Struct Example Source: https://github.com/veerendra2/composeflux/blob/main/AGENTS.md Illustrates the pattern of defining an exported interface and an unexported concrete struct for package implementations, promoting dependency injection. ```go type Client interface { Deploy(ctx context.Context, project *types.Project) error } type client struct { /* unexported */ } ``` -------------------------------- ### Go Error Handling: Wrapping with Context Source: https://github.com/veerendra2/composeflux/blob/main/AGENTS.md Example of wrapping an error with additional context using fmt.Errorf and the %w verb for error chaining. ```go return nil, fmt.Errorf("failed to clone: %w", err) ``` -------------------------------- ### Go Error Handling: Fatal at Command Layer Source: https://github.com/veerendra2/composeflux/blob/main/AGENTS.md Example of using FatalIfErrorf at the command-line interface layer to terminate the application on critical errors. ```go ctx.FatalIfErrorf(ctx.Run()) ``` -------------------------------- ### Configure SSH Key Fetching from Secrets Manager Source: https://github.com/veerendra2/composeflux/blob/main/docs/GettingStarted.md Example of disabling secrets manager fetch for SSH deploy keys by setting GIT_DEPLOY_KEY_SECRET_REF to an empty string. Optionally specifies a custom path for the mounted SSH key. ```yaml environment: GIT_DEPLOY_KEY_SECRET_REF: "" # Disable fetch from secrets manager # GIT_SSH_KEY_PATH: /.ssh/composeflux_id_rsa # Optional: custom path volumes: - ~/.ssh/id_rsa:/.ssh/composeflux_id_rsa:ro ``` -------------------------------- ### Serve MkDocs Documentation Locally Source: https://github.com/veerendra2/composeflux/blob/main/docs/Development.md Serve the MkDocs documentation locally with live reload enabled. This task allows for previewing documentation changes in real-time. ```bash task docs # or task serve-docs ``` -------------------------------- ### Go Error Handling: Log and Return Source: https://github.com/veerendra2/composeflux/blob/main/AGENTS.md Shows how to log a structured error message with relevant context before returning the error up the call stack. ```go slog.Error("Failed to create client", "provider", name, "error", err) return nil, err ``` -------------------------------- ### Available Task Commands Source: https://github.com/veerendra2/composeflux/blob/main/docs/Development.md Lists all available tasks for the ComposeFlux project, including build, test, lint, and documentation serving commands. ```bash task: Available tasks for this project: * all: Run comprehensive checks; format, lint, security and test * build: Build the application binary for the current platform * build-docker: Build Docker image * compose: Run the application in development mode using Docker Compose * fmt: Formats all Go source files * install: Install required tools and dependencies * lint: Run static analysis and code linting using golangci-lint * run: Runs the main application * security: Run security vulnerability scan * serve-docs: Serve MkDocs documentation locally with live reload (aliases: docs) * test: Runs all tests in the project (aliases: tests) * vet: Examines Go source code and reports suspicious constructs ``` -------------------------------- ### Go Error Handling: Sentinel Comparison Source: https://github.com/veerendra2/composeflux/blob/main/AGENTS.md Demonstrates checking for specific sentinel errors using errors.Is, ensuring non-fatal errors are handled appropriately. ```go if err != nil && !errors.Is(err, git.NoErrAlreadyUpToDate) { ... } ``` -------------------------------- ### ComposeFlux Build and Test Commands Source: https://github.com/veerendra2/composeflux/blob/main/AGENTS.md A list of Task commands for building, running, formatting, linting, testing, and security checking the ComposeFlux project. These commands streamline the development workflow. ```bash task build # Compile binary to dist/composeflux (injects version via ldflags) task run # go run ./cmd/composeflux (runs vet first) task fmt # go fmt ./... task vet # go vet ./... task lint # golangci-lint run --timeout 3m task test # go test ./... task security # govulncheck ./... task all # fmt + lint + vet + security + test (full local CI) task build-docker # docker build -t composeflux . task install # Install govulncheck and golangci-lint ``` -------------------------------- ### Go Error Handling: Warn and Continue Source: https://github.com/veerendra2/composeflux/blob/main/AGENTS.md Illustrates handling non-fatal errors within reconciliation loops by logging a warning and continuing the loop, preventing a single failure from stopping the entire process. ```go if err := r.Deploy(ctx, project); err != nil { slog.Warn("Failed to deploy stack", "stack_name", name, "error", err) continue } ``` -------------------------------- ### Performing a One-Shot Sync with ComposeFlux Source: https://github.com/veerendra2/composeflux/blob/main/docs/GettingStarted.md Manually trigger an immediate synchronization and deployment using the `sync` command. This is useful for applying changes like updated secrets without a Git commit. ```bash # One-shot mode - manually trigger sync composeflux sync ``` -------------------------------- ### Configure Bitwarden Environment Variables Source: https://github.com/veerendra2/composeflux/blob/main/docs/how-to-guides/Bitwarden.md Set these environment variables in your .env file or compose file to connect ComposeFlux to Bitwarden Secrets Manager. Include your access token, organization ID, and project ID. Optional variables are available for self-hosted Bitwarden instances and for fetching SSH deploy keys. ```bash SECRETS_PROVIDER=bitwarden BITWARDEN_ACCESS_TOKEN= BITWARDEN_ORGANIZATION_ID= BITWARDEN_PROJECT_ID= # Optional: only if fetching SSH deploy key from Bitwarden GIT_DEPLOY_KEY_SECRET_REF= # Optional: only for self-hosted Bitwarden # BITWARDEN_API_URL=https://vault.bitwarden.com/api # BITWARDEN_IDENTITY_URL=https://vault.bitwarden.com/identity ``` -------------------------------- ### Environment Variables for ComposeFlux Deployment Source: https://github.com/veerendra2/composeflux/blob/main/docs/GettingStarted.md Configure Git repository URL, stack path, and secrets provider settings in the .env file. Supports Bitwarden or Infisical for secret management. ```bash # Required - Common GIT_REPO_URL=git@github.com:user/stacks-repo.git STACK_PATH=stacks # Optional - Choose a secrets provider (omit to run without secrets): # Option A: Bitwarden SECRETS_PROVIDER=bitwarden GIT_DEPLOY_KEY_SECRET_REF=aaaaaaa-bbbbb-bbbb-cccc-ddddd BITWARDEN_ACCESS_TOKEN=your-access-token BITWARDEN_ORGANIZATION_ID=your-org-id BITWARDEN_PROJECT_ID=your-project-id # Option B: Infisical # SECRETS_PROVIDER=infisical # GIT_DEPLOY_KEY_SECRET_REF=SSH_PRIVATE_KEY # INFISICAL_CLIENT_ID=your-client-id # INFISICAL_CLIENT_SECRET=your-client-secret # INFISICAL_ENVIRONMENT=prod # INFISICAL_PROJECT_ID=your-project-id ``` -------------------------------- ### Verify ComposeFlux Deployment Source: https://github.com/veerendra2/composeflux/blob/main/docs/GettingStarted.md Commands to check ComposeFlux logs, list containers managed by ComposeFlux, and list all Docker Compose projects. ```bash # Check logs docker compose logs -f # List managed stacks (should show containers with composeflux label) docker ps --filter "label=composeflux.managed=true" # List all compose projects docker compose ls ``` -------------------------------- ### ComposeFlux Command Line Usage Source: https://github.com/veerendra2/composeflux/blob/main/docs/GettingStarted.md This is the general usage information for the ComposeFlux CLI, outlining available flags and commands. ```bash Usage: composeflux [flags] A GitOps continuous deployment tool for Docker Compose. Flags: -h, --help Show context-sensitive help. --log-format="console" Set the output format of the logs. Must be "console" or "json" ($LOG_FORMAT). --log-level=INFO Set the log level. Must be "DEBUG", "INFO", "WARN" or "ERROR" ($LOG_LEVEL). --log-add-source Whether to add source file and line number to log records ($LOG_ADD_SOURCE). --version Print version information and exit Commands: run Run ComposeFlux in daemon mode (continuous reconciliation) sync Perform a one-shot sync and deploy Run "composeflux --help" for more information on a command. ``` -------------------------------- ### Test GitHub SSH Access Source: https://github.com/veerendra2/composeflux/blob/main/docs/how-to-guides/GithubDeployKeys.md Tests the SSH connection to GitHub using the generated deploy key. A successful authentication will result in a confirmation message from GitHub. ```bash # Test GitHub connection with your key ssh -i ~/.ssh/composeflux_deploy -T git@github.com # Expected output: # Hi user/repo! You've successfully authenticated, but GitHub does not provide shell access. ``` -------------------------------- ### Running ComposeFlux in Daemon Mode Source: https://github.com/veerendra2/composeflux/blob/main/docs/GettingStarted.md Execute ComposeFlux in daemon mode for continuous reconciliation. It performs an initial sync and then periodically checks for Git changes. ```bash # Daemon mode (initial sync at startup, then checks Git every 5 minutes) composeflux run ``` -------------------------------- ### Configure ComposeFlux with Secrets Manager Source: https://github.com/veerendra2/composeflux/blob/main/docs/how-to-guides/GithubDeployKeys.md Configures ComposeFlux to fetch the SSH private key from a secrets manager. Ensure the `GIT_REPO_URL` uses the SSH protocol and `SECRETS_PROVIDER` and `GIT_DEPLOY_KEY_SECRET_REF` are set correctly. ```yaml # In compose.yml environment: GIT_REPO_URL: git@github.com:user/repo.git # SSH URL SECRETS_PROVIDER: bitwarden # or infisical GIT_DEPLOY_KEY_SECRET_REF: SSH_PRIVATE_KEY # Change if using a different name ``` -------------------------------- ### Configure ComposeFlux with Mounted Local Key Source: https://github.com/veerendra2/composeflux/blob/main/docs/how-to-guides/GithubDeployKeys.md Configures ComposeFlux to use a locally mounted SSH private key, bypassing the secrets manager. The `GIT_SSH_KEY_PATH` environment variable specifies where the key will be available inside the container, and the volume mounts the private key to that location. ```yaml # In compose.yml environment: GIT_SSH_KEY_PATH: /.ssh/composeflux_id_rsa # Where the key will be mounted volumes: - ~/.ssh/composeflux_deploy:/.ssh/composeflux_id_rsa:ro ``` -------------------------------- ### ComposeFlux Infisical Environment Variables Source: https://github.com/veerendra2/composeflux/blob/main/docs/how-to-guides/Infisical.md Configure ComposeFlux to use Infisical as its secrets provider by setting these environment variables in your .env file or compose file. Ensure you replace placeholders with your actual Infisical credentials and settings. ```bash SECRETS_PROVIDER=infisical INFISICAL_CLIENT_ID= INFISICAL_CLIENT_SECRET= INFISICAL_ENVIRONMENT=prod INFISICAL_PROJECT_ID= # Optional (supports comma-separated paths) INFISICAL_SECRET_PATH=/ # INFISICAL_SITE_URL=https://app.infisical.com # Optional: only if using a custom key name for SSH deploy key # GIT_DEPLOY_KEY_SECRET_REF=SSH_PRIVATE_KEY ``` -------------------------------- ### Expose Bitwarden Secrets in Compose Stack Source: https://github.com/veerendra2/composeflux/blob/main/docs/how-to-guides/Bitwarden.md Define your services in a compose file and reference secrets from Bitwarden using environment variable syntax. ComposeFlux will fetch these secrets and make them available to your application containers. ```yaml services: app: image: myapp:latest environment: DATABASE_URL: ${DATABASE_URL} API_KEY: ${API_KEY} ``` -------------------------------- ### Docker Compose Configuration for ComposeFlux Source: https://github.com/veerendra2/composeflux/blob/main/docs/GettingStarted.md Defines the ComposeFlux service, including image, container name, restart policy, and environment variables for Git and secrets management. Mounts the Docker socket and optionally SSH keys. ```yaml services: composeflux: image: ghcr.io/veerendra2/composeflux:latest container_name: composeflux restart: unless-stopped environment: # Git Configuration GIT_REPO_URL: ${GIT_REPO_URL} STACK_PATH: ${STACK_PATH} # GIT_INTERVAL: 5m # Sync interval # GIT_BRANCH: main # Secrets Manager - Bitwarden (optional) # SECRETS_PROVIDER: ${SECRETS_PROVIDER} # GIT_DEPLOY_KEY_SECRET_REF: ${GIT_DEPLOY_KEY_SECRET_REF} # BITWARDEN_ACCESS_TOKEN: ${BITWARDEN_ACCESS_TOKEN} # BITWARDEN_ORGANIZATION_ID: ${BITWARDEN_ORGANIZATION_ID} # BITWARDEN_PROJECT_ID: ${BITWARDEN_PROJECT_ID} # Secrets Manager - Infisical (comment out Bitwarden above if using this) # SECRETS_PROVIDER: infisical # GIT_DEPLOY_KEY_SECRET_REF: ${GIT_DEPLOY_KEY_SECRET_REF} # INFISICAL_CLIENT_ID: ${INFISICAL_CLIENT_ID} # INFISICAL_CLIENT_SECRET: ${INFISICAL_CLIENT_SECRET} # INFISICAL_ENVIRONMENT: ${INFISICAL_ENVIRONMENT} # INFISICAL_PROJECT_ID: ${INFISICAL_PROJECT_ID} # INFISICAL_SITE_URL: https://app.infisical.com # Logging # LOG_LEVEL: info # LOG_FORMAT: console # console or json # Metrics # METRICS_ADDR: ":9090" # Prometheus metrics endpoint, empty to disable ports: - "9090:9090" # Prometheus metrics volumes: - /var/run/docker.sock:/var/run/docker.sock # Optional: Custom SSH known_hosts # - ./ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro # Optional: Mount local SSH key instead of fetching from secrets manager # - ~/.ssh/id_rsa:/.ssh/composeflux_id_rsa:ro ``` -------------------------------- ### Exposing Infisical Secrets as Environment Variables in Compose Source: https://github.com/veerendra2/composeflux/blob/main/docs/how-to-guides/Infisical.md Define your service in a Docker Compose file to utilize secrets fetched from Infisical. Secrets are exposed as environment variables within the service container, allowing your application to access them directly. ```yaml services: app: image: myapp:latest environment: DATABASE_PASSWORD: ${DATABASE_PASSWORD} API_KEY: ${API_KEY} ``` -------------------------------- ### Generate SSH Key Pair Source: https://github.com/veerendra2/composeflux/blob/main/docs/how-to-guides/GithubDeployKeys.md Generates a new SSH key pair using ED25519. It's recommended to use a key without a passphrase for automation. This command creates both a private and a public key file. ```bash # Generate new key pair (no passphrase for automation) ssh-keygen -t ed25519 -C "composeflux-deploy-key" -f ~/.ssh/composeflux_deploy # This creates: # - Private key: ~/.ssh/composeflux_deploy # - Public key: ~/.ssh/composeflux_deploy.pub ``` -------------------------------- ### Exclude Stack from Image Updates Source: https://github.com/veerendra2/composeflux/blob/main/docs/Introduction.md Add this label to any service within a stack to prevent automatic image updates for the entire stack. If any service has this label, the whole stack is excluded from image updates. ```yaml services: db: image: postgres:15 labels: composeflux.image-update.exclude: "true" ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.