### Configuration Example: Cache Map Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Demonstrates configuring multiple cache mappings. ```json { "cacheMap": [ { "source": "**/node_modules/**", "target": ".npm" }, { "source": "**/target/**", "target": ".maven" } ] } ``` -------------------------------- ### CLI Command Example: Builder Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Shows how to specify a custom builder for the build process. ```bash buildkit-cache-dance --cache-map "**/node_modules/** -> .npm" --dockerfile Dockerfile --cache-dir cache --scratch-dir scratch --builder my-builder:latest ``` -------------------------------- ### Configuration Example: Full Options Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Shows a comprehensive configuration object for BuildKit Cache Dance. ```json { "cacheMap": [ { "source": "**/node_modules/**", "target": ".npm" } ], "dockerfile": "Dockerfile.prod", "cacheDir": "./build-cache", "scratchDir": "./scratch", "skipExtraction": false, "extract": true, "utilityImage": "mcr.microsoft.com/dotnet/sdk:6.0", "builder": "docker buildx build" } ``` -------------------------------- ### CLI Command Example: Utility Image Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Demonstrates specifying a utility image for the build process. ```bash buildkit-cache-dance --cache-map "**/node_modules/** -> .npm" --dockerfile Dockerfile --cache-dir cache --scratch-dir scratch --utility-image my-utility-image:latest ``` -------------------------------- ### GitHub Actions Workflow Example Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Example of a GitHub Actions workflow demonstrating integration with BuildKit Cache Dance. ```yaml name: Cache Dance Example on: [push] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup BuildKit Cache Dance uses: ./ with: cache-map: | **/node_modules/** -> .npm **/target/** -> .maven dockerfile: Dockerfile cache-dir: cache scratch-dir: scratch - name: Build with Docker run: docker build -t my-image . ``` -------------------------------- ### Example SourcePath Usage Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/types.md Illustrates the declaration of SourcePath variables with relative and absolute host paths. ```typescript const source1: SourcePath = 'cache-mount/apt'; const source2: SourcePath = 'cache-mount/go-build'; const source3: SourcePath = '/tmp/cache-data'; ``` -------------------------------- ### CLI Command Example: Basic Usage Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Demonstrates basic CLI usage for the BuildKit Cache Dance tool. ```bash buildkit-cache-dance --cache-map "**/node_modules/** -> .npm" --dockerfile Dockerfile --cache-dir cache --scratch-dir scratch ``` -------------------------------- ### CLI Command Example: Extract Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Illustrates using the CLI with the --extract flag. ```bash buildkit-cache-dance --cache-map "**/node_modules/** -> .npm" --dockerfile Dockerfile --cache-dir cache --scratch-dir scratch --extract ``` -------------------------------- ### Complete Injection Workflow Example Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-inject-cache.md This TypeScript example shows a complete workflow for injecting caches using the injectCaches function. It includes parsing options and handling potential errors during the cache injection process. Ensure you have the necessary imports for parseOpts and injectCaches. ```typescript import { parseOpts } from './opts.js'; import { injectCaches } from './inject-cache.js'; async function main() { // Parse options from GitHub Actions inputs or CLI args const opts = parseOpts(process.argv); if (opts.help) { console.log('Usage: buildkit-cache-dance [options]'); return; } // Inject caches from host into Docker layer try { await injectCaches(opts); console.log('Successfully injected caches'); } catch (error) { console.error('Failed to inject caches:', error); process.exit(1); } } main(); ``` -------------------------------- ### Dockerfile with Cache Mounts Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Example Dockerfile demonstrating the use of RUN --mount=type=cache instructions with both 'id' and 'target' specified. ```dockerfile RUN --mount=type=cache,id=apt-cache,target=/var/cache/apt \ apt-get update && apt-get install -y gcc RUN --mount=type=cache,target=/root/.cache/go-build \ go build . ``` -------------------------------- ### Example TargetPath Usage Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/types.md Illustrates the declaration of TargetPath variables with absolute container paths. ```typescript const target1: TargetPath = '/var/cache/apt'; const target2: TargetPath = '/var/lib/apt'; const target3: TargetPath = '/root/.cache/go-build'; ``` -------------------------------- ### CacheMap Example Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-opts.md Illustrates how to define a CacheMap, associating host directories like 'cache-mount/apt' with container paths or specific cache options. ```typescript const cacheMap: CacheMap = { 'cache-mount/apt': '/var/cache/apt', 'cache-mount/go-build': { target: '/root/.cache/go-build', id: 'go-build' } }; ``` -------------------------------- ### CLI Command Example: Skip Extraction Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Shows how to use the CLI with the --skip-extraction flag. ```bash buildkit-cache-dance --cache-map "**/node_modules/** -> .npm" --dockerfile Dockerfile --cache-dir cache --scratch-dir scratch --skip-extraction ``` -------------------------------- ### Parsing Opts with Example Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/types.md Demonstrates how to parse command-line arguments into an Opts configuration object using the parseOpts function. Ensure the 'opts.js' module is imported. ```typescript import { parseOpts, Opts } from './opts.js'; const opts: Opts = parseOpts(['--extract', '--cache-map', '{"apt": "/var/cache/apt"}']); ``` -------------------------------- ### Example CacheMap Usage Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/types.md Illustrates how to define a CacheMap with both string target paths and detailed object configurations for cache mounts. ```typescript const cacheMap: CacheMap = { 'cache-mount/apt': '/var/cache/apt', 'cache-mount/go': { target: '/root/.cache/go-build', id: 'go-build', uid: 1000, gid: 1000 } }; ``` -------------------------------- ### GitLab CI Integration for Cache Management Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md Integrates buildkit-cache-dance into a GitLab CI pipeline for managing build caches. This example shows how to install dependencies, download the tool, inject, build, and extract caches. ```yaml variables: DOCKER_TLS_CERTDIR: "/certs" DOCKER_HOST: "tcp://docker:2376" DOCKER_TLS_VERIFY: 1 DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client" cache: paths: - cache-mount stages: - build build: stage: build image: docker:24-dind before_script: - apk add --no-cache nodejs npm curl tar - mkdir -p cache-mount - curl -LJO https://github.com/reproducible-containers/buildkit-cache-dance/releases/download/v3.1.0/buildkit-cache-dance-3.1.0.tar.gz - tar xf buildkit-cache-dance-3.1.0.tar.gz script: # Inject caches - node ./buildkit-cache-dance-3.1.0/dist/index.js \ --cache-dir cache-mount \ --cache-map '{"apt": "/var/cache/apt"}' # Build image - docker build -f Dockerfile -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA . # Extract caches - node ./buildkit-cache-dance-3.1.0/dist/index.js \ --extract \ --cache-dir cache-mount \ --cache-map '{"apt": "/var/cache/apt"}' artifacts: paths: - cache-mount ``` -------------------------------- ### Sequential Cache Processing Example Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md Demonstrates the current sequential processing of multiple independent caches. This is the default behavior and can be parallelized using external tools if needed. ```bash # Current: Sequential processing for cache_id in apt go pip; do node dist/index.js --cache-map "{\"$cache_id\": \"...\"}" done # Could be parallelized by: # - Running multiple injection processes (not yet supported) # - Using GNU parallel or xargs ``` -------------------------------- ### Error Handling Example: Cache Configuration Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Demonstrates handling errors related to cache configuration. ```typescript try { // Logic that might throw cache configuration errors if (!options.cacheMap || options.cacheMap.length === 0) { throw new Error("Cache map is required."); } } catch (error) { console.error("Cache configuration error:", error.message); // Handle the error appropriately } ``` -------------------------------- ### Error Handling Example: Docker Command Failure Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Illustrates handling errors when a Docker command fails. ```typescript import { spawn } from 'child_process'; const dockerProcess = spawn('docker', ['build', './nonexistent-dir']); dockerProcess.on('error', (err) => { console.error('Failed to start Docker process:', err); // Handle the error appropriately }); dockerProcess.on('close', (code) => { if (code !== 0) { console.error(`Docker process exited with code ${code}`); // Handle the error appropriately } }); ``` -------------------------------- ### Error Handling Example: File System Error Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Shows how to catch and handle file system related errors. ```typescript import * as fs from 'fs'; try { fs.readFileSync('/path/to/nonexistent/file'); } catch (error) { if (error.code === 'ENOENT') { console.error('File not found error.'); } else { console.error('An unexpected file system error occurred:', error); } // Handle the error appropriately } ``` -------------------------------- ### Custom Utility Images Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Examples of using different custom container images as the utility image for cache operations. ```bash # Use Alpine buildkit-cache-dance --utility-image alpine:3.20 ``` ```bash # Use Ubuntu buildkit-cache-dance --utility-image ubuntu:24.04 ``` ```bash # Use custom image buildkit-cache-dance --utility-image myregistry.com/custom-base:latest ``` -------------------------------- ### Get Target Path and Mount Arguments Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/types.md Demonstrates how to extract the target path and construct the mount arguments string from a CacheOptions object. ```typescript import { getTargetPath, getMountArgsString } from './opts.js'; const options: CacheOptions = { target: '/var/cache/apt', id: 'apt', sharing: 'locked' }; const target = getTargetPath(options); // '/var/cache/apt' const mount = getMountArgsString(options); // 'type=cache,target=/var/cache/apt,id=apt,sharing=locked' ``` -------------------------------- ### Example of parseOpts Usage Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/types.md Illustrates how to use the parseOpts function and access the parsed arguments. It shows how positional arguments are captured in the '_' array and other options are parsed as key-value pairs. ```typescript import { parseOpts } from './opts.js'; const result = parseOpts([ '--cache-map', '{"apt": "/var/cache/apt"}', 'extra-positional-arg' ]); console.log(result._); // ['extra-positional-arg'] console.log(result['cache-map']); // '{"apt": "/var/cache/apt"}' ``` -------------------------------- ### Dockerfile: Valid Cache Mount with Target Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md An example of a correctly configured cache mount using the 'target' option. This is a valid way to specify the cache location. ```dockerfile RUN --mount=type=cache,target=/var/cache/apt apt-get update ``` -------------------------------- ### CLI Injection Mode Usage Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/README.md Example of using the buildkit-cache-dance tool via the command line in injection mode, specifying cache mapping and directory. ```bash node dist/index.js \ --cache-map '{"var-cache-apt": "/var/cache/apt"}' \ --cache-dir cache-mount ``` -------------------------------- ### Specify Docker Builder with GitHub Actions Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Example of how to dynamically set the builder name using GitHub Actions outputs. ```bash buildkit-cache-dance --builder "${{ steps.setup-buildx.outputs.name }}" ``` -------------------------------- ### CLI Extraction Mode Usage Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/README.md Example of using the buildkit-cache-dance tool via the command line in extraction mode, including cache mapping and directory. ```bash node dist/index.js \ --extract \ --cache-map '{"var-cache-apt": "/var/cache/apt"}' \ --cache-dir cache-mount ``` -------------------------------- ### Dockerfile: Incomplete Cache Mount Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md This example shows an incomplete cache mount instruction in a Dockerfile, missing the required 'target' or 'id' option. Ensure one of these is present to avoid errors. ```dockerfile RUN --mount=type=cache echo "test" # ❌ Missing target or id ``` -------------------------------- ### Generated Dancefile Example Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-extract-cache.md A sample Dockerfile generated by the extract-cache module. It uses a busybox utility image, mounts a cache target, copies cache content to a temporary directory, and uses '|| true' to handle cases where the cache might be empty or missing. ```dockerfile FROM ghcr.io/containerd/busybox:latest COPY buildstamp buildstamp RUN --mount=type=cache,target=/var/cache/apt \ mkdir -p /var/dance-cache/ \ && cp -p -R /var/cache/apt/. /var/dance-cache/ || true ``` -------------------------------- ### Complete Cache Extraction Workflow Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-extract-cache.md This TypeScript example shows a full extraction process. It parses command-line options, checks if extraction is skipped, and then calls `extractCaches`. It includes basic error handling and logging, making it suitable for CI environments. ```typescript import { parseOpts } from './opts.js'; import { extractCaches } from './extract-cache.js'; async function main() { // In a GitHub Actions post-step, environment variables are automatically set const opts = parseOpts(process.argv); if (opts.skip-extraction) { console.log('Cache has not changed, skipping extraction'); return; } try { await extractCaches(opts); console.log('Successfully extracted caches'); } catch (error) { console.error('Failed to extract caches:', error); // In GitHub Actions, this would be logged but may not fail the workflow // depending on how the action is configured process.exit(1); } } main(); ``` -------------------------------- ### Configure BuildKit Cache Dance in GitHub Actions Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Example of how to use the buildkit-cache-dance action in a GitHub Actions workflow. Specify builder, cache directory, cache mapping, Dockerfile, scratch directory, extraction behavior, and utility image. ```yaml - name: Restore Docker cache mounts uses: reproducible-containers/buildkit-cache-dance@v3 with: builder: ${{ steps.setup-buildx.outputs.name }} cache-dir: cache-mount cache-map: | { "var-cache-apt": "/var/cache/apt", "var-lib-apt": "/var/lib/apt" } dockerfile: Dockerfile scratch-dir: scratch skip-extraction: false utility-image: ghcr.io/containerd/busybox:latest ``` -------------------------------- ### Function Usage: Extract Cache Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Example of using the extractCache function from the library. ```typescript import { extractCache } from "@buildkit-cache-dance/extract-cache"; import { Opts } from "@buildkit-cache-dance/opts"; async function runExtractCache(options: Opts) { await extractCache(options); } // Example usage: // const opts: Opts = { ... }; // runExtractCache(opts); ``` -------------------------------- ### Function Usage: Inject Cache Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Example of using the injectCache function from the library. ```typescript import { injectCache } from "@buildkit-cache-dance/inject-cache"; import { Opts } from "@buildkit-cache-dance/opts"; async function runInjectCache(options: Opts) { await injectCache(options); } // Example usage: // const opts: Opts = { ... }; // runInjectCache(opts); ``` -------------------------------- ### Local Development Workflow Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md This script demonstrates the full local development cycle: building the tool, injecting caches, performing a Docker build, extracting caches, and performing a second build to verify cache usage. ```bash #!/bin/bash # Build the tool npm install npm run build # Create cache directory mkdir -p cache-mount # Inject caches node dist/index.js \ --cache-dir cache-mount \ --dockerfile Dockerfile # Build with docker buildx (requires setup) docker buildx build \ -f Dockerfile \ -t myimage:latest \ --load . # Extract caches for next build node dist/index.js \ --extract \ --cache-dir cache-mount \ --dockerfile Dockerfile # Second build (should use cached content) docker buildx build \ -f Dockerfile \ -t myimage:latest \ --load . ``` -------------------------------- ### Usage of Simple Path Mapping Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Demonstrates how to apply a simple cache map configuration using the buildkit-cache-dance command line. ```bash buildkit-cache-dance --cache-map '{"var-cache-apt": "/var/cache/apt"}' ``` -------------------------------- ### help Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-opts.md Prints the command-line help message to standard output. This function provides users with information on how to use the build-cache-dance utility. ```APIDOC ## help ### Description Prints the command-line help message to standard output. This function provides users with information on how to use the build-cache-dance utility. ### Function Signature ```typescript function help(): void ``` ### Returns `void` — No return value; output is written to console. ### Example ```typescript import { help } from './opts.js'; help(); // Outputs: // build-cache-dance [options] // Save 'RUN --mount=type=cache' caches on GitHub Actions or other CI platforms // ... ``` ``` -------------------------------- ### Error Handling Example: JSON Parsing Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/GENERATION_SUMMARY.txt Illustrates a potential JSON parsing error scenario. ```typescript try { // Attempt to parse invalid JSON JSON.parse("{ invalid json "); } catch (error) { console.error("JSON parsing error:", error.message); // Handle the error appropriately } ``` -------------------------------- ### Command-Line Argument Format Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Illustrates the general format for passing configuration options as command-line arguments. Options are parsed using the `mri` library. ```bash buildkit-cache-dance --option-name value --boolean-flag ``` -------------------------------- ### Run a Simple Docker Command Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-run.md Executes a basic 'docker buildx build' command. Ensure 'docker' is in your PATH. ```typescript import { run } from './run.js'; // Run: docker buildx build -f Dockerfile --tag myimage . await run('docker', ['buildx', 'build', '-f', 'Dockerfile', '--tag', 'myimage', '.']); ``` -------------------------------- ### Index Module Flow Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/module-architecture.md Outlines the execution flow for the main entry point, including argument parsing, mode determination, and module orchestration. ```mermaid graph TD Process starts subgraph index.ts A[Parse arguments via parseOpts()] B[Check for --help flag] C{Yes → Call help() and exit} D{No → Continue} E[Check for --extract flag (or GITHUB_STATE env var)] F{Yes → Call extractCaches()} G{No → Call injectCaches()} H[Write POST=true to GITHUB_STATE for post-step] I[Process completes or errors] end Process starts --> A --> B B --> C B --> D --> E E --> F E --> G --> H H --> I G --> I ``` -------------------------------- ### Manage Docker Containers and Copy Files Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-run.md Demonstrates creating, copying files from, and removing a Docker container. Ensure Docker is running and accessible. ```typescript import { run } from './run.js'; // Create a container from an image await run('docker', ['create', '-ti', '--name', 'cache-container', 'dance:extract']); // Copy files from container await run('docker', ['cp', '-L', 'cache-container:/var/dance-cache', '-']); // Remove container await run('docker', ['rm', '-f', 'cache-container']); ``` -------------------------------- ### run Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/README.md Executes a given command with specified arguments. ```APIDOC ## run(command, args) ### Description Executes a command with its arguments, providing a utility for running external processes. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Method Function Call ### Endpoint N/A ### Request Example ```javascript await run('ls', ['-l']); ``` ### Response #### Success Response This function typically executes a command and may not return a specific value, or it might return the command's exit code or output depending on implementation details not specified. #### Response Example N/A ``` -------------------------------- ### Basic GitHub Actions Usage Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/README.md Example of using the buildkit-cache-dance action in a GitHub Actions workflow with basic cache configuration. ```yaml - uses: reproducible-containers/buildkit-cache-dance@v3 with: cache-dir: cache-mount cache-map: | { "var-cache-apt": "/var/cache/apt" } ``` -------------------------------- ### Migrating from Deprecated Cache Options Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md Demonstrates the migration from deprecated `--cache-source` and `--cache-target` arguments to the current `--cache-map` option for configuring cache directories. This ensures compatibility with newer versions of the tool. ```bash buildkit-cache-dance --cache-source ./cache-apt --cache-target /var/cache/apt ``` ```bash buildkit-cache-dance --cache-map '{"./cache-apt": "/var/cache/apt"}' ``` -------------------------------- ### Usage of Object Configuration for Cache Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Shows how to use the buildkit-cache-dance command with a cache map defined by detailed object configurations. ```bash buildkit-cache-dance --cache-map '{ "var-cache-apt": { "target": "/var/cache/apt", "id": "apt-cache" } }' ``` -------------------------------- ### Check Docker Status Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md Verify the status of your Docker installation and available builders. This is a crucial step in diagnosing Docker command execution errors. ```bash docker info ``` ```bash docker buildx ls ``` -------------------------------- ### Manual cache-map configuration for apt-get Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/README.md This YAML snippet shows how to use the buildkit-cache-dance action with a manual `cache-map` to specify cache mount targets for apt-get. It allows for more fine-grained control over cache paths. ```yaml - name: Restore Docker cache mounts uses: reproducible-containers/buildkit-cache-dance@v3 with: builder: ${{ steps.setup-buildx.outputs.name }} cache-map: | { "var-cache-apt": "/var/cache/apt", "var-lib-apt": "/var/lib/apt" } skip-extraction: ${{ steps.cache.outputs.cache-hit }} ``` -------------------------------- ### Docker Command Execution Error Example Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md This error occurs when a Docker command fails to execute or exits with a non-zero status. It is thrown by the `run()` function. ```bash Error running command: docker buildx build [args] [Underlying docker error details] ``` ```bash Error: spawn docker ENOENT ``` ```bash docker: error: invalid builder: mybuilder ``` ```bash Error: ENOENT: no such file or directory, open 'Dockerfile' ``` ```bash Error: EACCES: permission denied ``` ```bash docker: error: failed to allocate cache ``` -------------------------------- ### Build Docker Image with BuildKit Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-run.md Use this function to build a Docker image using buildx, specifying the Dockerfile, tag, context, and an optional builder. Ensure the 'run' function is imported from './run.js'. ```typescript import { run } from './run.js'; async function buildImage(dockerfilePath: string, tag: string, context: string, builder?: string) { const args = ['buildx', 'build', '-f', dockerfilePath, '--tag', tag]; if (builder) { args.push('--builder', builder); } args.push(context); await run('docker', args); } // Usage await buildImage('Dancefile.inject', 'dance:inject', 'cache-mount', 'my-builder'); ``` -------------------------------- ### Configure Cache Map via Command-line Argument Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Demonstrates setting the cache map using a command-line argument. This has the highest precedence for configuration. ```bash buildkit-cache-dance --cache-map '{"x": "/y"}' ``` -------------------------------- ### Create Missing Scratch Directory Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md Use 'mkdir -p' to create the scratch directory if it does not exist. This resolves ENOENT errors when the scratch directory is missing. ```bash mkdir -p scratch ``` -------------------------------- ### Skip Extraction with GitHub Actions Cache Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Example demonstrating how to conditionally skip the extraction phase based on the cache hit status from actions/cache in GitHub Actions. ```yaml - name: Cache uses: actions/cache@v4 id: cache with: path: cache-mount key: cache-mount-${{ hashFiles('Dockerfile') }} - name: Extract Docker cache mounts uses: reproducible-containers/buildkit-cache-dance@v3 with: skip-extraction: ${{ steps.cache.outputs.cache-hit }} ``` -------------------------------- ### BuildKit Cache Dance CLI Options Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/README.md This section lists the available command-line options for the BuildKit Cache Dance script. Use these to customize cache injection, extraction, and other behaviors. ```shell build-cache-dance [options] Save 'RUN --mount=type=cache' caches on GitHub Actions or other CI platforms Options: --extract Extract the cache from the docker container (extract step). Otherwise, inject the cache (main step) --cache-map The map of actions source to container destination paths for the cache paths --dockerfile The Dockerfile to use for the auto-discovery of cache-map. Default: 'Dockerfile' --cache-dir The root directory where cache content is injected from/extracted to when using auto-discovery of the cache-map. --scratch-dir Where the action is stores some temporary files for its processing. Default: 'scratch' --skip-extraction Skip the extraction of the cache from the docker container --builder The name of the buildx builder. Default: 'default' --help Show this help ``` -------------------------------- ### Cache Python pip Cache in Dockerfile Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md This Dockerfile caches pip's download cache using buildkit mounts. This speeds up subsequent package installations. ```dockerfile FROM python:3.11-slim WORKDIR /app COPY requirements.txt . RUN --mount=type=cache,target=/root/.cache/pip \ pip install --cache-dir /root/.cache/pip -r requirements.txt COPY . . RUN --mount=type=cache,target=/root/.cache/pip \ pip install --cache-dir /root/.cache/pip -e . ``` -------------------------------- ### Run with Scratch Directory in Root-Owned Path Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md This command attempts to use a scratch directory located in '/etc', which is typically owned by root, leading to EACCES permission denied errors. Ensure the scratch directory is in a location writable by the current user. ```bash buildkit-cache-dance --scratch-dir /etc/scratch # /etc is root-owned ``` -------------------------------- ### Configuration Flow Diagram Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/module-architecture.md Illustrates how user configuration is processed from various sources into a consolidated options object. This flow is central to setting up the cache dance behavior. ```text User provides configuration via: ├── GitHub Actions inputs (via @actions/core) ├── CLI arguments (--option value) └── Environment variables (HOME, PATH, DOCKER_HOST, etc.) ↓ parseOpts() consolidates sources: ├── Reads GitHub Actions inputs ├── Parses CLI arguments with mri ├── Applies defaults └── Converts deprecated options ↓ Options object (Opts type): ├── cache-map (JSON string) ├── dockerfile (path) ├── cache-dir (path or null) ├── scratch-dir (path) ├── builder (name) ├── utility-image (image name) ├── skip-extraction (boolean) ├── extract (boolean) └── help (boolean) ``` -------------------------------- ### Get Cache Map from Options Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-opts.md Parses the cache map from provided options or auto-discovers it from a Dockerfile. It handles JSON parsing errors and extracts cache mount instructions. ```typescript import { parseOpts, getCacheMap } from './opts.js'; const opts = parseOpts(['--cache-map', '{"var-cache-apt": "/var/cache/apt"}']); const cacheMap = await getCacheMap(opts); console.log(cacheMap); // { 'var-cache-apt': '/var/cache/apt' } ``` ```typescript const opts = parseOpts(['--dockerfile', 'Dockerfile', '--cache-dir', 'cache-mount']); const cacheMap = await getCacheMap(opts); // For a Dockerfile containing: RUN --mount=type=cache,id=go-build,target=/root/.cache/go-build console.log(cacheMap); // { 'cache-mount/go-build': { id: 'go-build', target: '/var/cache-target' } } ``` -------------------------------- ### JSON: Cache Options with Misspelled Target Key Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md An example of cache options where the 'target' key is misspelled as 'dest'. Correcting the key name to 'target' is necessary for proper parsing. ```json { "dest": "/var/cache/apt", // ❌ Should be "target" "id": "apt-cache" } ``` -------------------------------- ### Create Missing Cache Directory Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md Use 'mkdir -p' to create the cache directory if it does not exist. This resolves ENOENT errors when the cache directory is missing. ```bash mkdir -p cache-mount ``` -------------------------------- ### Example Usage of ToStringable with CacheOptions Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/types.md Demonstrates how numeric values for uid and gid, and string values for id, can be used directly within CacheOptions due to the implicit toString() implementation. ```typescript const opts: CacheOptions = { target: '/var/cache/apt', uid: 1000, // number, implements implicit toString() gid: 1000, // number, implements implicit toString() id: 'apt-cache' // string, already stringifiable }; ``` -------------------------------- ### GitHub Actions Workflow for Python pip Cache Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md This GitHub Actions workflow uses buildkit-cache-dance to restore pip's cache directory. This speeds up Python package installations. ```yaml - name: Restore Docker cache mounts uses: reproducible-containers/buildkit-cache-dance@v3 with: builder: ${{ steps.buildx.outputs.name }} cache-dir: cache-mount cache-map: | { "pip-cache": "/root/.cache/pip" } ``` -------------------------------- ### Piped Command Error Example Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md This error occurs when either command in a piped operation, such as `docker cp | tar`, fails. It is thrown by the `runPiped()` function when a process exits with a non-zero code. ```bash process exited with code 127 ``` ```bash Error: No such container or object ``` ```bash Error: tar: Cannot open: No such file or directory ``` ```bash Error: tar: invalid option ``` -------------------------------- ### Fixing Cache Map JSON Syntax Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md Shows an example of fixing an invalid cache map argument by correcting JSON syntax. The corrected version ensures proper double-quoting for string values. ```bash # Before (invalid) buildkit-cache-dance --cache-map '{"var-cache-apt": /var/cache/apt}' # After (valid) buildkit-cache-dance --cache-map '{"var-cache-apt": "/var/cache/apt"}' ``` -------------------------------- ### Get Target Path from Cache Options Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-opts.md Extracts the Docker container target path from cache options, supporting both string and object formats. Throws an error if the object format lacks a 'target' property. ```typescript import { getTargetPath } from './opts.js'; // String format const path1 = getTargetPath('/var/cache/apt'); console.log(path1); // '/var/cache/apt' // Object format const path2 = getTargetPath({ target: '/var/cache/apt', id: 'apt-cache' }); console.log(path2); // '/var/cache/apt' // Error case try { getTargetPath({ id: 'apt-cache' }); // Missing 'target' } catch (e) { console.error(e.message); // "Expected the 'target' key in the cache options..." } ``` -------------------------------- ### Monitor Cache Size Before and After Build Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md Use shell commands to check the cache directory size before and after a build process. This helps in understanding cache growth and identifying potential issues. ```bash #!/bin/bash # Check cache size before build before=$(du -sh cache-mount | cut -f1) echo "Cache size before: $before" # Run build... # Check cache size after after=$(du -sh cache-mount | cut -f1) echo "Cache size after: $after" # If cache is too large, consider: # - Using cache eviction policies # - Splitting caches by layer # - Cleaning unused packages ``` -------------------------------- ### Usage of Mixed Cache Map Configuration Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Illustrates applying a mixed cache map configuration, including both string paths and object definitions, via the buildkit-cache-dance command. ```bash buildkit-cache-dance --cache-map '{ "var-cache-apt": "/var/cache/apt", "go-build": {"target": "/root/.cache/go-build", "uid": 1000, "gid": 1000} }' ``` -------------------------------- ### Generated Dancefile for Cache Injection with Ownership Restoration Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/api-reference-inject-cache.md This example shows a generated Dancefile that copies cache content and restores original file ownership using `chown`. It includes cache busting via a `buildstamp` file and mounts the cache target path. ```dockerfile FROM ghcr.io/containerd/busybox:latest COPY buildstamp buildstamp RUN --mount=type=cache,target=/var/cache/apt,id=apt-cache \ --mount=type=bind,source=.,target=/var/dance-cache \ cp -p -R /var/dance-cache/. /var/cache/apt && chown -R 1000:1000 /var/cache/apt || true ``` -------------------------------- ### Dockerfile with Multiple Caches and Ownership Restoration Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md This Dockerfile demonstrates caching multiple directories, including one that requires specific ownership restoration. It uses buildkit's cache mounts with UID/GID specified. ```dockerfile FROM ubuntu:22.04 RUN useradd -m build WORKDIR /app RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ apt-get update && apt-get install -y build-essential COPY . . RUN --mount=type=cache,target=/build-cache,uid=1000,gid=1000 \ chown build:build /build-cache && \ sudo -u build gcc -O3 -o app main.c ``` -------------------------------- ### Manual Docker Operations for Debugging Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md Manually perform operations similar to what the tool does, using Docker buildx. This is helpful for debugging complex cache and bind mount scenarios. ```bash # Create a test Dancefile cat > Dancefile << 'EOF' FROM busybox:latest COPY buildstamp buildstamp RUN --mount=type=cache,target=/var/cache/apt \ --mount=type=bind,source=.,target=/var/dance-cache \ cp -p -R /var/dance-cache/. /var/cache/apt || true EOF # Build manually docker buildx build -f Dancefile --tag dance:test . # Or with a specific builder docker buildx build --builder mybuilder -f Dancefile --tag dance:test . ``` -------------------------------- ### Run with Non-Existent Cache Directory Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md This error occurs if the cache directory does not exist and cannot be created due to insufficient permissions or incorrect path. Ensure the cache directory exists or can be created. ```bash buildkit-cache-dance --cache-dir /read-only-mount/cache ``` -------------------------------- ### GitHub Actions workflow for BuildKit cache dance Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/README.md This GitHub Actions workflow demonstrates how to use the buildkit-cache-dance action to restore Docker cache mounts. It sets up buildx, caches the mount directory, restores the cache using the action, and then builds the Docker image. ```yaml --- name: Build on: push: jobs: Build: runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 - uses: docker/setup-buildx-action@v3 id: setup-buildx - uses: docker/metadata-action@v5 id: meta with: images: Build - name: Cache uses: actions/cache@v4 id: cache with: path: cache-mount key: cache-mount-${{ hashFiles('Dockerfile') }} - name: Restore Docker cache mounts uses: reproducible-containers/buildkit-cache-dance@v3 with: builder: ${{ steps.setup-buildx.outputs.name }} cache-dir: cache-mount dockerfile: Dockerfile skip-extraction: ${{ steps.cache.outputs.cache-hit }} - name: Build and push uses: docker/build-push-action@v5 with: context: . cache-from: type=gha cache-to: type=gha,mode=max file: Dockerfile push: ${{ github.event_name != 'pull_request' }} tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }} ``` -------------------------------- ### Run with Scratch Directory Permission Issues Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/errors.md This error arises when the scratch directory lacks the necessary write permissions for the current user. Ensure the scratch directory is writable. ```bash buildkit-cache-dance --scratch-dir /root/scratch # No permission ``` -------------------------------- ### Configure Scratch Directory Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Specify a temporary directory for storing Dancefiles and intermediate build artifacts. This directory is cleaned on each operation. ```bash buildkit-cache-dance --scratch-dir /tmp/buildkit-scratch ``` -------------------------------- ### Download and Extract BuildKit Cache Dance Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/README.md Use these commands to download the release archive and extract the script for local execution. ```shell curl -LJO https://github.com/reproducible-containers/buildkit-cache-dance/archive/refs/tags/v3.1.0.tar.gz tar xvf buildkit-cache-dance-3.1.0.tar.gz ``` -------------------------------- ### GitHub Actions Workflow for apt-get Cache Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md This GitHub Actions workflow demonstrates how to use buildkit-cache-dance to restore apt-get caches. It combines with GitHub Actions cache for further optimization. ```yaml name: Build with apt-get Cache on: push: branches: [main] jobs: build: runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 - uses: docker/setup-buildx-action@v3 id: buildx - uses: actions/cache@v4 id: cache with: path: cache-mount key: cache-mount-${{ hashFiles('Dockerfile') }} restore-keys: | cache-mount- - name: Restore Docker cache mounts uses: reproducible-containers/buildkit-cache-dance@v3 with: builder: ${{ steps.buildx.outputs.name }} cache-dir: cache-mount cache-map: | { "var-cache-apt": "/var/cache/apt", "var-lib-apt": "/var/lib/apt" } skip-extraction: ${{ steps.cache.outputs.cache-hit }} - name: Build and push uses: docker/build-push-action@v5 with: context: . file: Dockerfile cache-from: type=gha cache-to: type=gha,mode=max push: ${{ github.event_name != 'pull_request' }} tags: myimage:latest ``` -------------------------------- ### Test Dockerfile Parsing with Auto-discovery Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/usage-patterns.md Create a test Dockerfile and use a Node.js script to test its parsing and cache auto-discovery. This helps verify Dockerfile syntax and cache mount configurations. ```bash # Create test Dockerfile cat > test.Dockerfile << 'EOF' FROM ubuntu:22.04 RUN --mount=type=cache,target=/var/cache/apt apt-get update EOF # Test auto-discovery node dist/index.js \ --dockerfile test.Dockerfile \ --cache-dir cache-mount ``` -------------------------------- ### Map apt Cache Directories Source: https://github.com/reproducible-containers/buildkit-cache-dance/blob/main/_autodocs/configuration.md Maps the /var/cache/apt and /var/lib/apt directories for caching. This is useful for ensuring package manager caches are preserved across builds. ```bash buildkit-cache-dance --cache-map '{ "var-cache-apt": "/var/cache/apt", "var-lib-apt": "/var/lib/apt" }' ```