kubechecks

https://github.com/zapier/kubechecks
Admin
kubechecks is a Kubernetes deployment validation tool that integrates with GitHub and GitLab to...
Tokens:23,403
Snippets:273
Trust Score:9.7
License:MPL-2.0
Update:5 months ago
Show doc for...
Tokens:
Raw
### Development Setup

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Run this command to install development tools like golangci-lint and verify Go and Docker installations.

```bash
make dev-setup
```

--------------------------------

### Install and Initialize Linux `pass`

Source: https://github.com/zapier/kubechecks/blob/main/docs/contributing.md

Install the `pass` utility using your distribution's package manager and initialize it with your GPG key. This is a one-time setup.

```sh
# Ubuntu/Debian
sudo apt install pass

# Fedora/RHEL
sudo dnf install pass

# Arch
sudo pacman -S pass

# Initialize with your GPG key (one-time setup)
gpg --gen-key           # if you don't have a GPG key yet
pass init "your-email@example.com"
```

--------------------------------

### Setup Buildx Builder and Verify QEMU

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Use these commands to set up the buildx builder and verify QEMU installation, which is necessary for multi-arch builds.

```bash
# Setup buildx builder
make setup-buildx
```

```bash
# Verify QEMU is installed (for ARM emulation)
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
```

--------------------------------

### Start Tilt Development Environment

Source: https://github.com/zapier/kubechecks/blob/main/docs/contributing.md

Run `tilt up` from the root directory of the repository to start the Tilt development environment. This command builds and deploys resources.

```sh
tilt up
```

--------------------------------

### Install kubechecks with Helm

Source: https://context7.com/zapier/kubechecks/llms.txt

Add the Helm repository and install kubechecks, providing required secrets and configuration inline.

```bash
helm repo add kubechecks https://zapier.github.io/kubechecks/
helm repo update

helm install kubechecks kubechecks/kubechecks \
  --namespace kubechecks \
  --create-namespace \
  --set secrets.create=true \
  --set "secrets.env.KUBECHECKS_VCS_TOKEN=ghp_yourGitHubToken" \
  --set "secrets.env.KUBECHECKS_ARGOCD_API_TOKEN=argocd-api-token" \
  --set "configMap.create=true" \
  --set "configMap.env.KUBECHECKS_VCS_TYPE=github" \
  --set "configMap.env.KUBECHECKS_ARGOCD_API_SERVER_ADDR=argocd-server.argocd:443" \
  --set "configMap.env.KUBECHECKS_WEBHOOK_URL_BASE=https://kubechecks.mycompany.com"
```

--------------------------------

### Sequential Processing Example with Archive Mode

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Illustrates the timing of sequential processing with archive mode, highlighting cache hit benefits.

```text
Time 0s: PR #5 arrives → Download archive (3s)
Time 3s: PR #5 processing starts → Run checks (15s)
Time 18s: PR #5 completes, PR #6 starts → Download archive (cache hit = 0s)
Time 18s: PR #6 processing starts → Run checks (12s)
```

--------------------------------

### Full Helm values.yaml Configuration Example

Source: https://context7.com/zapier/kubechecks/llms.txt

A comprehensive values.yaml file for production deployments, illustrating all major configuration options for ArgoCD integration, environment variables, secrets, deployment, service, and ingress.

```yaml
# values.yaml
argocd:
  namespace: argocd

configMap:
  create: true
  env:
    KUBECHECKS_VCS_TYPE: "github"
    KUBECHECKS_ARGOCD_API_SERVER_ADDR: "argocd-server.argocd:443"
    KUBECHECKS_ARGOCD_API_NAMESPACE: "argocd"
    KUBECHECKS_ARGOCD_REPOSITORY_ENDPOINT: "argocd-repo-server.argocd:8081"
    KUBECHECKS_WEBHOOK_URL_BASE: "https://kubechecks.mycompany.com"
    KUBECHECKS_WEBHOOK_URL_PREFIX: ""
    KUBECHECKS_FALLBACK_K8S_VERSION: "1.28.0"
    KUBECHECKS_ENABLE_KUBECONFORM: "true"
    KUBECHECKS_ENABLE_PREUPGRADE: "true"
    KUBECHECKS_ENABLE_CONFTEST: "true"
    KUBECHECKS_POLICIES_LOCATION: "./policies"
    KUBECHECKS_MONITOR_ALL_APPLICATIONS: "true"
    KUBECHECKS_LOG_LEVEL: "info"
    KUBECHECKS_MAX_CONCURRENT_CHECKS: "32"
    KUBECHECKS_TIDY_OUTDATED_COMMENTS_MODE: "hide"
    KUBECHECKS_REPO_CACHE_ENABLED: "true"
    KUBECHECKS_REPO_CACHE_TTL: "24h"
    KUBECHECKS_OTEL_ENABLED: "false"

secrets:
  create: true
  env:
    KUBECHECKS_VCS_TOKEN: "<github-token>"
    KUBECHECKS_ARGOCD_API_TOKEN: "<argocd-token>"
    KUBECHECKS_WEBHOOK_SECRET: "<webhook-hmac-secret>"

deployment:
  replicaCount: 1
  image:
    name: "ghcr.io/zapier/kubechecks"
    tag: "latest"
    pullPolicy: Always
  resources:
    limits:
      memory: 1Gi
    requests:
      memory: 256Mi
      cpu: 200m
  args: ["controller"]

service:
  create: true
  type: ClusterIP
  port: 8080

ingress:
  create: true
  className: "nginx"
  annotations:
    kubernetes.io/tls-acme: "true"
  hosts:
    kubechecks.mycompany.com:
      paths:
        - path: "/hooks"
          pathType: Prefix
  tls:
    - secretName: kubechecks-tls
      hosts:
        - kubechecks.mycompany.com

```

--------------------------------

### Configure `.secret` File

Source: https://github.com/zapier/kubechecks/blob/main/docs/contributing.md

For backwards compatibility, copy the example `.secret` file and edit it with your specific values. This method is loaded before keychain values.

```sh
cp .secret.example .secret
# Edit .secret with your values
```

--------------------------------

### Build Production Image Locally

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Builds the production Docker image for the local platform. This is a common starting point for local development.

```bash
make build
```

--------------------------------

### Run kubechecks controller Command

Source: https://context7.com/zapier/kubechecks/llms.txt

Starts the webhook server, application watcher, and check workers. Configuration is primarily done via environment variables, but command-line flags can override some settings.

```bash
# Run directly (all config via env vars)
kubechecks controller \
  --vcs-type github \
  --vcs-token "$GITHUB_TOKEN" \
  --argocd-api-server-addr argocd-server.argocd:443 \
  --argocd-api-token "$ARGOCD_TOKEN" \
  --webhook-url-base https://kubechecks.mycompany.com \
  --webhook-secret "$WEBHOOK_SECRET" \
  --enable-kubeconform \
  --enable-preupgrade \
  --monitor-all-applications \
  --log-level info
```

```text
# Expected startup log output:
# INF Starting KubeChecks git-tag=v1.2.3 git-commit=abc1234
# INF parsing configuration
# INF Server Configuration:
# INF Webhook URL Base: https://kubechecks.mycompany.com
# INF VCS Type: github
# INF ArgoCD Namespace: argocd
# INF starting web server
# INF listening for requests
```

--------------------------------

### Programmatic ServerConfig Loading

Source: https://context7.com/zapier/kubechecks/llms.txt

Demonstrates loading kubechecks configuration programmatically using Viper, useful for testing or custom setups. It reads environment variables automatically when using the CLI.

```go
package config_example

import (
    "fmt"
    "github.com/zapier/kubechecks/pkg/config"
    "github.com/spf13/viper"
)

func loadConfig() {
    // When using the CLI, config.New() reads all KUBECHECKS_* env vars automatically.
    cfg, err := config.New()
    if err != nil {
        panic(err)
    }

    // Or build programmatically for testing:
    v := viper.New()
    v.Set("vcs-type", "github")
    v.Set("argocd-api-server-addr", "argocd-server.argocd:443")
    v.Set("enable-kubeconform", true)
    v.Set("enable-ai-review", true)
    v.Set("ai-review-provider", "anthropic")
    v.Set("ai-review-model", "claude-sonnet-4-6")
    v.Set("anthropic-api-key", "sk-ant-...")
    v.Set("worst-ai-review-state", "warning")

    cfg, err = config.NewWithViper(v)
    if err != nil {
        panic(err)
    }

    fmt.Printf("VCS: %s, ArgoCD: %s, AI: %v (provider: %s)\n",
        cfg.VcsType, cfg.ArgoCDServerAddr, cfg.EnableAIReview, cfg.AIReviewProvider)

    // Check if GitHub App auth is configured (vs. token auth):
    if cfg.IsGithubApp() {
        fmt.Println("Using GitHub App authentication")
    }
}
```

--------------------------------

### GitHub Actions CI/CD Integration

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Examples of how to use Makefile targets within GitHub Actions for testing, CI checks, documentation rebuilds, and multi-arch builds.

```yaml
# Example GitHub Actions usage
- name: Run tests
  run: make test
```

```yaml
- name: Run CI checks
  run: make ci
```

```yaml
- name: Rebuild docs
  run: make rebuild-docs
```

```yaml
- name: Build and push
  run: make push-multiarch
  env:
    IMAGE_REGISTRY: ghcr.io/${{ github.repository_owner }}
```

--------------------------------

### Install Kubechecks via Helm

Source: https://github.com/zapier/kubechecks/blob/main/docs/usage.md

Install the kubechecks Helm chart into your Kubernetes cluster. Ensure you have configured secrets as per the chart's documentation.

```console
helm install kubechecks charts/kubechecks -n kubechecks --create-namespace
```

--------------------------------

### Configure VCS Type for Tilt

Source: https://context7.com/zapier/kubechecks/llms.txt

Create a tilt_config.json file to specify the Version Control System (VCS) type. The default is 'gitlab', but this example sets it to 'github'.

```bash
echo '{"vcs-type": "github"}' > tilt_config.json
```

--------------------------------

### Add Kubechecks Helm Repository

Source: https://github.com/zapier/kubechecks/blob/main/docs/usage.md

Add the kubechecks Helm chart repository to your Helm configuration. This is a prerequisite for installing kubechecks.

```console
helm repo add kubechecks https://zapier.github.io/kubechecks/
```

--------------------------------

### Build Docs Locally with Mkdocs

Source: https://github.com/zapier/kubechecks/blob/main/docs/contributing.md

Run `mkdocs serve` from the repository root to build and serve the documentation locally. Changes will auto-reload, and docs are viewable at `http://localhost:8000`.

```bash
mkdocs serve
```

--------------------------------

### Show All Make Targets

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Displays a list of all available targets and their descriptions in the Makefile. Useful for discovering build commands.

```bash
make help
```

--------------------------------

### Run Conftest/Rego Policy Check

Source: https://context7.com/zapier/kubechecks/llms.txt

Enforce organizational standards by running OPA/conftest Rego policies against rendered manifests. Configure policies via `config.ServerConfig`, specifying locations like local paths or Git repositories.

```go
package rego_example

import (
    "context"
    "fmt"

    "github.com/zapier/kubechecks/pkg/checks"
    "github.com/zapier/kubechecks/pkg/checks/rego"
    "github.com/zapier/kubechecks/pkg/config"
)

func runConftestCheck(ctx context.Context, request checks.Request) {
    cfg := config.ServerConfig{
        EnableConfTest:   true,
        PoliciesLocation: []string{"./policies", "git::https://github.com/myorg/k8s-policies.git"},
        ShowDebugInfo:    false,
    }

    checker, err := rego.NewChecker(cfg)
    if err != nil {
        fmt.Printf("failed to create rego checker: %v\n", err)
        return
    }

    // Policies are Rego files that receive input = the YAML manifest content.
    // Example policy (policies/require-labels.rego):
    //   package main
    //   deny[msg] {
    //     not input.metadata.labels["app"]
    //     msg := sprintf("resource %v must have label 'app'", [input.metadata.name])
    //   }

    result, err := checker.Check(ctx, request)
    if err != nil {
        fmt.Printf("conftest check failed: %v\n", err)
        return
    }

    // result.State: StateSuccess | StateWarning | StateFailure
    // result.Details: ASCII table of policy violations per file
    // | ✅/⚠️/❌ | file | message |
    fmt.Printf("Conftest: %s\n%s\n", result.Summary, result.Details)
}
```

--------------------------------

### Run Unit Tests for Archive Package

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Execute unit tests for the archive package using the 'go test' command. Options for coverage and race detection are available.

```bash
# Test archive package
go test ./pkg/archive/...

# Test with coverage
go test -cover ./pkg/archive/...

# Test with race detector
go test -race ./pkg/archive/...
```

--------------------------------

### GitHub Get Pull Request Files

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Fetches the list of changed files for a pull request from GitHub. This information is crucial for determining which files need to be checked.

```go
// GET /repos/{owner}/{repo}/pulls/{number}/files
files, err := githubClient.GetPullRequestFiles(ctx, pr)
```

--------------------------------

### GitHub Get Pull Request Files

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Retrieves a list of all changed file paths for a given GitHub pull request. It handles pagination to ensure all files are fetched.

```go
func (c *Client) GetPullRequestFiles(ctx context.Context, pr vcs.PullRequest) ([]string, error) {
    // Get all changed files
    var allFiles []string
    opts := &github.ListOptions{PerPage: 100}

    for {
        files, resp, err := c.c.PullRequests.ListFiles(ctx, owner, repo, pr.CheckID, opts)
        for _, file := range files {
            allFiles = append(allFiles, file.GetFilename())
        }
        if resp.NextPage == 0 {
            break
        }
        opts.Page = resp.NextPage
    }

    return allFiles, nil
}
```

--------------------------------

### Build and Push Multi-Architecture Image

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Builds the Docker image for multiple architectures and pushes it to a specified container registry. Requires `IMAGE_REGISTRY` to be set.

```bash
make push-multiarch IMAGE_REGISTRY=ghcr.io/zapier
```

--------------------------------

### Build with Custom Image Tag

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Shows how to specify a custom tag for the Docker image using the `IMAGE_TAG` environment variable.

```bash
make build IMAGE_TAG=v1.0.0
```

--------------------------------

### GitLab Get Pull Request Files

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Retrieves a list of all changed file paths for a given GitLab merge request by parsing its diffs. It ensures each file path is added only once.

```go
func (c *Client) GetPullRequestFiles(ctx context.Context, pr vcs.PullRequest) ([]string, error) {
    // Get all diffs
    diffs, _, err := c.c.MergeRequests.ListMergeRequestDiffs(pr.FullName, pr.CheckID, nil)

    // Extract file paths
    var allFiles []string
    filesSeen := make(map[string]bool)

    for _, diff := range diffs {
        filePath := diff.NewPath
        if filePath == "" || filePath == "/dev/null" {
            filePath = diff.OldPath
        }
        if filePath != "" && filePath != "/dev/null" && !filesSeen[filePath] {
            allFiles = append(allFiles, filePath)
            filesSeen[filePath] = true
        }
    }

    return allFiles, nil
}
```

--------------------------------

### Enable Archive Mode Configuration

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Use these flags or environment variables to enable and configure archive mode.

```bash
# Enable archive mode
--archive-mode=true

# Archive cache directory
--archive-cache-dir=/tmp/kubechecks/archives

# Archive cache TTL
--archive-cache-ttl=1h
```

```bash
KUBECHECKS_ARCHIVE_MODE=true
KUBECHECKS_ARCHIVE_CACHE_DIR=/tmp/kubechecks/archives
KUBECHECKS_ARCHIVE_CACHE_TTL=1h
```

--------------------------------

### GitLab Get Pull Request Files

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Retrieves the list of changed files for a merge request from GitLab using its API. This data is used to scope the checks performed on the extracted archive.

```go
// GET /projects/{id}/merge_requests/{iid}/diffs
files, err := gitlabClient.GetPullRequestFiles(ctx, pr)
```

--------------------------------

### Push to Custom Registry

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Illustrates pushing a multi-architecture build to a custom container registry by setting the `IMAGE_REGISTRY` environment variable.

```bash
make push-multiarch IMAGE_REGISTRY=docker.io/myorg
```

--------------------------------

### Configure Queue Size

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Sets the maximum buffered requests per repository. This can be configured via a command-line flag or an environment variable.

```go
Config{
    QueueSize: 100,  // Max buffered requests per repo
}
```

--------------------------------

### Cache Lookup Logic

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Demonstrates how to check a cache for a previously downloaded and extracted archive using a merge commit SHA as the cache key. If found, it returns the cached path, avoiding redundant downloads.

```go
// Use merge commit SHA as cache key
cacheKey := mergeCommitSHA

// Check cache
if entry, exists := cache.entries[cacheKey]; exists {
    // Cache hit! Return immediately
    return entry.extractedPath, nil
}
```

--------------------------------

### Archive Mode vs. Legacy Git Mode

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Illustrates the conditional logic for selecting between archive mode (using Downloader) and legacy git mode based on configuration. This determines how a repository is cloned.

```go
// check.go
if cfg.ArchiveMode {
    // Use archive mode
    repo, err := archiveManager.Clone(ctx, cloneURL, baseBranch, pr)
} else {
    // Legacy git mode
    repo, err := repoManager.Clone(ctx, cloneURL, baseBranch)
}
```

--------------------------------

### Run Go Tests

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Executes the Go test suite. This command is essential for verifying code correctness.

```bash
make test
```

--------------------------------

### PromQL: Calculate Download Success Rate

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Calculates the download success rate over the last 5 minutes. Aim for a rate above 99%.

```promql
# Download success rate (should be >99%)
rate(kubechecks_archive_download_success_total[5m]) /
rate(kubechecks_archive_download_total[5m])
```

--------------------------------

### Run Tests with Coverage

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Executes the Go test suite and generates a coverage report. This helps identify areas of the codebase not adequately tested.

```bash
make test-coverage
```

--------------------------------

### Build Multi-Architecture Image

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Builds the Docker image for multiple architectures, specifically amd64 and arm64. This is crucial for supporting diverse deployment environments.

```bash
make build-multiarch
```

--------------------------------

### Configure VCS Type in `tilt_config.json`

Source: https://github.com/zapier/kubechecks/blob/main/docs/contributing.md

Specify the version control system type (e.g., 'github') in `tilt_config.json` if you are not using the default 'gitlab'.

```json
{
    "vcs-type": "github"
}
```

--------------------------------

### Run Linter

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Executes golangci-lint to perform static analysis and linting on the Go codebase. This enforces code quality and style guidelines.

```bash
make lint
```

--------------------------------

### PromQL: Calculate P95 Download Time

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Calculates the 95th percentile of download duration over the last 5 minutes.

```promql
# P95 download time
histogram_quantile(0.95, rate(kubechecks_archive_download_duration_seconds_bucket[5m]))
```

--------------------------------

### Build Custom Docker Image

Source: https://context7.com/zapier/kubechecks/llms.txt

Build a Docker image with custom name and tag. This allows for specific versioning or naming conventions for the image.

```bash
make build IMAGE_NAME=my-kubechecks IMAGE_TAG=v2.0.0
```

--------------------------------

### PromQL: Calculate Average Download Time

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Calculates the average download duration over the last 5 minutes. Aim for less than 5 seconds.

```promql
# Average download time (should be <5s)
rate(kubechecks_archive_download_duration_seconds_sum[5m]) /
rate(kubechecks_archive_download_duration_seconds_count[5m])
```

--------------------------------

### Download and Extract Archive (Cache Miss)

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Executes the download and extraction process when an archive is not found in the cache. It then caches the newly downloaded and extracted content for future use.

```go
// Download archive with auth
zipData, err := downloader.download(ctx, archiveURL, authHeaders)

// Extract to target directory
extractedPath, err := downloader.extract(zipData, targetDir)

// Cache for future use
cache.entries[cacheKey] = &CacheEntry{
    extractedPath: extractedPath,
    lastUsed:      time.Now(),
    refCount:      1,
}
```

--------------------------------

### Build with Custom Image Name

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Demonstrates how to override the default Docker image name using the `IMAGE_NAME` environment variable.

```bash
make build IMAGE_NAME=my-kubechecks
```

--------------------------------

### Configure Archive Cache Settings

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Optionally configure the cache directory and TTL (Time To Live) for archive mode. Adjust these based on performance and disk usage needs.

```bash
KUBECHECKS_ARCHIVE_CACHE_DIR=/fast/ssd/path
KUBECHECKS_ARCHIVE_CACHE_TTL=1h
```

--------------------------------

### Run Integration Tests with VCS APIs

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Execute integration tests that interact with real VCS APIs. This requires setting appropriate authentication tokens for GitHub and GitLab.

```bash
# Test with real VCS APIs (requires tokens)
GITHUB_TOKEN=xxx GITLAB_TOKEN=xxx go test -tags=integration ./pkg/archive/...
```

--------------------------------

### Monitor Archive Mode Metrics with PromQL

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Use these PromQL queries to monitor cache hit rate, average download time, and cache disk usage in archive mode. Ensure metrics are scraped at an appropriate interval for accurate rate calculations.

```promql
# Cache hit rate
rate(kubechecks_archive_cache_hits_total[5m]) /
(rate(kubechecks_archive_cache_hits_total[5m]) + rate(kubechecks_archive_cache_misses_total[5m]))
```

```promql
# Average download time
rate(kubechecks_archive_download_duration_seconds_sum[5m]) /
rate(kubechecks_archive_download_duration_seconds_count[5m])
```

```promql
# Cache disk usage
kubechecks_archive_cache_size_bytes / (1024 * 1024 * 1024)  # GB
```

--------------------------------

### Format Code

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Formats Go source code according to standard Go conventions. This ensures code consistency across the project.

```bash
make fmt
```

--------------------------------

### VCS Provider Environment Variables

Source: https://context7.com/zapier/kubechecks/llms.txt

Configure the Version Control System (VCS) provider by setting environment variables. This includes specifying the VCS type, authentication tokens, and optional base URLs for enterprise or self-hosted instances.

```bash
# ── VCS provider ──────────────────────────────────────────────────────────────
export KUBECHECKS_VCS_TYPE="github"          # "github" | "gitlab"
export KUBECHECKS_VCS_TOKEN="ghp_..."        # personal access or app token
export KUBECHECKS_VCS_USERNAME="bot-user"
export KUBECHECKS_VCS_EMAIL="bot@mycompany.com"
export KUBECHECKS_VCS_BASE_URL=""            # override for GitHub Enterprise / self-hosted GitLab
export KUBECHECKS_VCS_UPLOAD_URL=""          # GitHub Enterprise upload URL

# GitHub App authentication (alternative to token)
export KUBECHECKS_GITHUB_APP_ID="12345"
export KUBECHECKS_GITHUB_INSTALLATION_ID="67890"
export KUBECHECKS_GITHUB_PRIVATE_KEY="$(cat /path/to/app.private-key.pem)"

```

--------------------------------

### Configure Kubechecks Webhook Registration

Source: https://context7.com/zapier/kubechecks/llms.txt

Set environment variables to enable automatic webhook registration for ArgoCD-monitored repositories. Configure the base URL, HMAC secret, and label filters for triggering checks.

```bash
export KUBECHECKS_ENSURE_WEBHOOKS="true"
export KUBECHECKS_WEBHOOK_URL_BASE="https://kubechecks.mycompany.com"
export KUBECHECKS_WEBHOOK_SECRET="my-hmac-secret"
export KUBECHECKS_LABEL_FILTER="run"
```

--------------------------------

### RepoQueue Worker Goroutine

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Illustrates the main loop for a RepoQueue's worker goroutine, handling incoming requests from the queue and responding to shutdown signals.

```go
func (rq *RepoQueue) startWorker() {
    for {
        select {
        case request := <-rq.queue:
            rq.processRequest(request)  // Sequential processing
        case <-rq.done:
            return  // Graceful shutdown
        }
    }
}
```

--------------------------------

### Inspect Built Image

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Allows inspection of the locally built Docker image. Use this to examine the contents or layers of the image.

```bash
make inspect
```

--------------------------------

### Enqueueing a Check Request

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Demonstrates how to enqueue a new PR check request using the QueueManager. Ensure the context is valid and EnqueueParams are correctly populated.

```go
queueManager.Enqueue(ctx, EnqueueParams{
    PullRequest: pr,
    Container:   ctr,
    Processors:  processors,
})
```

--------------------------------

### Run Full CI Checks

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Performs all checks required for continuous integration, including formatting, validation, linting, and testing.

```bash
make ci
```

--------------------------------

### Run Built Image

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Executes the locally built Docker image. This command is used to run the application in a container.

```bash
make run
```

--------------------------------

### Rebuild Documentation

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Rebuilds the project's documentation from source code comments and other relevant files. This ensures documentation stays up-to-date with the codebase.

```bash
make rebuild-docs
```

--------------------------------

### Kubechecks Server Configuration Struct

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Defines the Go struct for server configuration, including settings for archive mode, cache directory, and cache TTL, with environment variable mappings and default values.

```go
type ServerConfig struct {
    // Archive mode settings
    ArchiveMode      bool          `env:"ARCHIVE_MODE" default:"false"`
    ArchiveCacheDir  string        `env:"ARCHIVE_CACHE_DIR" default:"/tmp/kubechecks/archives"`
    ArchiveCacheTTL  time.Duration `env:"ARCHIVE_CACHE_TTL" default:"1h"`
}
```

--------------------------------

### Cache Key Methods

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Key methods of the Cache struct for managing archives. GetOrDownload retrieves or downloads an archive, while Release decrements its reference count.

```go
// GetOrDownload retrieves cached archive or downloads if not present
func (c *Cache) GetOrDownload(ctx context.Context, archiveURL, mergeCommitSHA string, authHeaders map[string]string) (string, error)
```

```go
// Release decrements reference count
func (c *Cache) Release(archiveURL, mergeCommitSHA string)
```

--------------------------------

### Validate Code with Go Vet

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Runs the 'go vet' tool to statically analyze Go code and detect suspicious constructs. This helps catch potential errors early.

```bash
make validate
```

--------------------------------

### Run Kubechecks CheckEvent Orchestration

Source: https://context7.com/zapier/kubechecks/llms.txt

Initializes and runs the CheckEvent, which orchestrates repository cloning, app discovery, concurrent worker dispatch, and comment assembly for PR/MR checks. Ensure all necessary dependencies like container and VCS interfaces are provided.

```go
package main

import (
    "context"

    "github.com/zapier/kubechecks/pkg/checks"
    "github.com/zapier/kubechecks/pkg/container"
    "github.com/zapier/kubechecks/pkg/events"
    "github.com/zapier/kubechecks/pkg/vcs"
)

func runChecks(ctx context.Context, ctr container.Container, pr vcs.PullRequest, processors []checks.ProcessorEntry) error {
    // repoManager wraps the git clone/cache logic
    repoManager := ctr.RepoManager

    // aiReviewChecker is nil when AI review is disabled
    var aiReviewChecker events.AIReviewChecker // set via getAIReviewChecker(ctr) in cmd/processors.go

    ce := events.NewCheckEvent(pr, ctr, repoManager, processors, aiReviewChecker)

    // Process downloads the PR archive, discovers affected apps,
    // dispatches concurrent workers, and posts the comment.
    return ce.Process(ctx)
}

// vcs.PullRequest fields that drive a check run:
// pr := vcs.PullRequest{
//     Name:      "myorg/myrepo",
//     CheckID:   42,
//     SHA:       "abc1234def5678",
//     HeadRef:   "feature/update-deployment",
//     BaseRef:   "main",
//     CloneURL:  "https://github.com/myorg/myrepo.git",
//     Username:  "author-login",
//     Email:     "author@mycompany.com",
// }
```

--------------------------------

### Manual GitHub Archive Download Test

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Manually test downloading a GitHub repository archive using curl. Ensure you provide a valid Authorization header with your token and the correct repository and SHA.

```bash
# Test GitHub archive download
curl -H "Authorization: Bearer $GITHUB_TOKEN" \
  "https://api.github.com/repos/owner/repo/zipball/SHA" \
  -o test.zip
```

--------------------------------

### PromQL: Calculate Cache Hit Rate

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Calculates the cache hit rate over the last 5 minutes. Aim for a rate above 70%.

```promql
# Cache hit rate (should be >70%)
rate(kubechecks_archive_cache_hits_total[5m]) /
(rate(kubechecks_archive_cache_hits_total[5m]) + rate(kubechecks_archive_cache_misses_total[5m]))
```

--------------------------------

### Inspect BuildKit Builder Status

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Provides detailed information about the current BuildKit builder instance, including its configuration and status. Helpful for troubleshooting build environments.

```bash
docker buildx inspect
```

--------------------------------

### Run Individual CI Checks

Source: https://github.com/zapier/kubechecks/blob/main/docs/contributing.md

Use specific `make` targets to run individual checks like tests, linting, formatting, or validation.

```bash
make test      # Run all tests
make lint      # Run golangci-lint (includes go vet)
make fmt       # Check code formatting
make validate  # Run go vet (also included in lint)
```

--------------------------------

### Manager Key Methods

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Key methods of the Manager struct for handling repository archives. These include cloning archives, retrieving changed files, and validating pull requests for archive mode.

```go
// Clone downloads archive and returns extracted path
func (m *Manager) Clone(ctx context.Context, cloneURL, branchName string, pr vcs.PullRequest) (*git.Repo, error)
```

```go
// GetChangedFiles retrieves changed files from VCS API
func (m *Manager) GetChangedFiles(ctx context.Context, pr vcs.PullRequest) ([]string, error)
```

```go
// ValidatePullRequest checks if PR can use archive mode
func (m *Manager) ValidatePullRequest(ctx context.Context, pr vcs.PullRequest) error
```

--------------------------------

### Enable Archive Mode Configuration

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Set the KUBECHECKS_ARCHIVE_MODE environment variable to true to enable archive mode. This is the primary step for migrating from Git mode.

```bash
KUBECHECKS_ARCHIVE_MODE=true
```

--------------------------------

### Run kubechecks process Command

Source: https://context7.com/zapier/kubechecks/llms.txt

Manually triggers a check for a single PR/MR by reference, bypassing the webhook listener. Can process by number or URL.

```bash
# Process a GitHub pull request by number
kubechecks process "123"
```

```bash
# Process a GitLab merge request by URL
kubechecks process "https://gitlab.com/myorg/myrepo/-/merge_requests/42"
```

```text
# Expected output:
# INF parsing configuration
# INF cloning repo branchName=feature/update-deployment
# INF Changed files retrieved from VCS API file_count=3 files="apps/myapp/deployment.yaml, ..."
# INF adding 2 apps to the queue
# INF Finished
```

--------------------------------

### PromQL Query for Total Backlog

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Displays the total number of items across all queues, representing the overall backlog.

```promql
# Total backlog
kubechecks_queue_repo_worker_size_total
```

--------------------------------

### RepoQueue Fields

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Details the fields of a repository-specific queue, including its buffered channel, worker goroutine signal, and processing function.

```go
type RepoQueue struct {
    repoURL     string                 // Repository URL
    queue       chan *CheckRequest     // Buffered channel (default: 100)
    done        chan struct{}          // Shutdown signal
    wg          sync.WaitGroup         // Wait for worker completion
    processFunc ProcessFunc            // Processing function
    processed   int                    // Total processed count
    queuedAt    time.Time              // When first request was queued
}
```

--------------------------------

### Check BuildKit Cache Size

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Displays the disk usage of the BuildKit cache. This is useful for diagnosing slow builds or disk space issues related to caching.

```bash
docker buildx du
```

--------------------------------

### GitHub Authentication Headers

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Provides the necessary authentication headers for interacting with the GitHub API.

```go
func (c *Client) GetAuthHeaders() map[string]string {
    return map[string]string{
        "Authorization": fmt.Sprintf("Bearer %s", c.cfg.VcsToken),
    }
}
```

--------------------------------

### PromQL: Monitor Cache Disk Usage

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Monitors the total disk space used by the cache in Gigabytes.

```promql
# Cache disk usage (GB)
kubechecks_archive_cache_size_bytes / (1024 * 1024 * 1024)
```

--------------------------------

### Clean Docker Images and Build Cache

Source: https://github.com/zapier/kubechecks/blob/main/docs/contributing.md

Use `docker image prune` to remove dangling images and `docker buildx prune` to clean the build cache, helping to manage local Docker storage.

```bash
docker image prune      # Remove dangling images
docker buildx prune     # Remove build cache
```

--------------------------------

### PromQL Query for Active Queues

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Shows the count of currently active repository queues.

```promql
# Number of active queues
kubechecks_queue_repo_worker_count
```

--------------------------------

### Running Checks on Extracted Archive

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Illustrates how checks are executed against the contents of the extracted archive. The extracted directory is treated similarly to a standard git checkout, allowing checks to run directly on the files.

```go
// The extracted archive looks like a git checkout
// /tmp/kubechecks/archives/{sha}/{repo-name}-{sha}/
//   - manifests/
//   - charts/
//   - values.yaml
//   etc.

// Checks run normally
results, err := runChecks(ctx, repo.Directory, changedFiles)
```

--------------------------------

### PromQL: Calculate Extraction Success Rate

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Calculates the extraction success rate over the last 5 minutes. Aim for a rate above 99.9%.

```promql
# Extraction success rate (should be >99.9%)
rate(kubechecks_archive_extract_success_total[5m]) /
rate(kubechecks_archive_extract_total[5m])
```

--------------------------------

### OpenTelemetry Integration Configuration

Source: https://context7.com/zapier/kubechecks/llms.txt

Environment variables to enable OpenTelemetry tracing and metrics, directing them to a specified collector. Traces cover major spans, and Prometheus metrics are exposed on port 8080.

```bash
# Enable OTEL and point to a Tempo/Jaeger/OTLP collector
export KUBECHECKS_OTEL_ENABLED="true"
export KUBECHECKS_OTEL_COLLECTOR_HOST="otel-collector.monitoring.svc.cluster.local"
export KUBECHECKS_OTEL_COLLECTOR_PORT="4317"

# Traces are emitted for all major spans:
#   - GenerateListOfAffectedApps
#   - getDiff / getResources / getArgoSettings
#   - Conftest
#   - CommitStatus
#   - createNote / createAIReviewNote
#
# Prometheus metrics are exposed at :8080/metrics (via labstack/echo-contrib)
# including queue depth, worker utilization, and per-check latencies.
```

--------------------------------

### GitLab Authentication Headers

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Provides the necessary authentication headers for interacting with the GitLab API using a private token.

```go
func (c *Client) GetAuthHeaders() map[string]string {
    return map[string]string{
        "PRIVATE-TOKEN": c.cfg.VcsToken,
    }
}
```

--------------------------------

### PromQL Query for Average Queue Size

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Calculates the average number of items currently in the queue for each repository.

```promql
# Average queue size per repo
avg(kubechecks_queue_repo_worker_size)
```

--------------------------------

### Graceful Shutdown Implementation

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Implements a graceful shutdown for the QueueManager by stopping new requests, draining in-flight work, and notifying about dropped items. It uses a sync.WaitGroup to wait for all workers to finish, with a context for timeout.

```go
func (qm *QueueManager) Shutdown(ctx context.Context) error {
    // Close all queues
    for _, queue := range qm.queues {
        close(queue.done)
    }

    // Wait for all workers (with timeout)
    done := make(chan struct{})
    go func() {
        for _, queue := range qm.queues {
            queue.wg.Wait()
        }
        close(done)
    }()

    select {
    case <-done:
        return nil  // Clean shutdown
    case <-ctx.Done():
        return ctx.Err()  // Timeout
    }
}
```

--------------------------------

### PromQL: Calculate Average Extraction Time

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Calculates the average extraction duration over the last 5 minutes. Aim for less than 1 second.

```promql
# Average extraction time (should be <1s)
rate(kubechecks_archive_extract_duration_seconds_sum[5m]) /
rate(kubechecks_archive_extract_duration_seconds_count[5m])
```

--------------------------------

### Manual GitLab Archive Download Test

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Manually test downloading a GitLab repository archive using curl. Use the PRIVATE-TOKEN header and specify the project path and SHA or merge request details.

```bash
# Test GitLab archive download
curl -H "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "https://gitlab.com/api/v4/projects/owner%2Frepo/repository/archive.zip?sha=refs/merge-requests/123/merge" \
  -o test.zip
```

--------------------------------

### Run Diff Check for Manifest Comparison

Source: https://context7.com/zapier/kubechecks/llms.txt

Executes the Diff Check to compare ArgoCD applied manifests with live cluster state. Requires a checks.Request object containing rendered manifests, the ArgoCD Application CRD, and an ArgoCD gRPC client.

```go
package diff_example

import (
    "context"
    "fmt"

    "github.com/zapier/kubechecks/pkg/checks"
    "github.com/zapier/kubechecks/pkg/checks/diff"
)

func runDiffCheck(ctx context.Context, request checks.Request) {
    // request.JsonManifests contains rendered JSON manifests from ArgoCD repo-server
    // request.App is the live ArgoCD Application CRD
    // request.Container.ArgoClient connects to the ArgoCD gRPC API

    result, err := diff.Check(ctx, request)
    if err != nil {
        fmt.Printf("diff check failed: %v\n", err)
        return
    }

    // result.Summary example: "2 added, 1 modified, 0 removed"
    // result.Details contains a markdown fenced diff block, e.g.:
    // ```diff
    // ===== apps/Deployment default/myapp ======
    // -  replicas: 1
    // +  replicas: 3
    // ```
    fmt.Println(result.Summary)
    fmt.Println(result.Details)

    if result.NoChangesDetected {
        fmt.Println("No changes between PR branch and live cluster state.")
    }
}
```

--------------------------------

### PromQL Query for Maximum Queue Size

Source: https://github.com/zapier/kubechecks/blob/main/pkg/queue/README.md

Finds the maximum number of items in any single repository's queue.

```promql
# Maximum queue size across all repos
max(kubechecks_queue_repo_worker_size)
```

--------------------------------

### Downloader Struct Definition

Source: https://github.com/zapier/kubechecks/blob/main/pkg/archive/README.md

Defines the Downloader struct, which holds an HTTP client for making download requests. This struct is the foundation for all download and extraction operations.

```go
type Downloader struct {
    httpClient *http.Client
}
```

--------------------------------

### Build Debug Image

Source: https://github.com/zapier/kubechecks/blob/main/docs/build-system.md

Builds a debug version of the Docker image, including the delve debugger. This is useful for local development and debugging.

```bash
make build-debug
```

--------------------------------

### Sign DCO Commit

Source: https://github.com/zapier/kubechecks/blob/main/docs/contributing.md

Append a trailer to your commit message to sign off, indicating agreement with the Developer Certificate of Origin. Use `-s` or `--signoff`.

```bash
git commit -s
```
Tokens:
Raw