sccache (mozilla/sccache)

sccache

https://github.com/mozilla/sccache
Admin
sccache is a ccache-like compiler caching tool that avoids recompilation by storing cached results...

Tokens:29,461
Snippets:368
Trust Score:9
Update:2 weeks ago
Show doc for...
Context Summary (auto-generated)
Raw
# sccache - Shared Compilation Cache

sccache is a ccache-like compiler caching tool that wraps compiler invocations and avoids recompilation by storing and retrieving cached build artifacts. It operates as a client-server daemon: the first invocation spawns a background server process (listening on `127.0.0.1:4226` by default), and subsequent compilations communicate with that server. When a cache hit is found, sccache serves the stored output directly, skipping the actual compilation. Supported languages include C, C++, Assembler, Rust, NVIDIA CUDA (via `nvcc` and `clang`), AMD ROCm HIP (`hipcc`), and Wind River's Diab compiler. Both GCC and MSVC support Response Files.

sccache supports a wide range of cache storage backends — local disk, Amazon S3 (and S3-compatible services like MinIO, Cloudflare R2), Redis, Memcached, Google Cloud Storage, Azure Blob Storage, GitHub Actions cache, WebDAV, Alibaba OSS, and Tencent COS — as well as multi-level hierarchical caching that chains these backends together. Additionally, sccache includes an icecream-style distributed compilation system (`sccache-dist`) that supports automatic toolchain packaging, transport-layer encryption, JWT-based authentication, and sandboxed build server execution.

---

## CLI Usage

### Wrapping a compiler invocation

sccache acts as a transparent compiler wrapper. Prefix any compile command with `sccache`, or configure your build system to use it as a launcher.

```bash
# Direct C/C++ compilation
sccache gcc -o foo.o -c foo.c
sccache clang++ -std=c++17 -O2 -o bar.o -c bar.cpp

# With CMake (3.4+)
cmake -DCMAKE_C_COMPILER_LAUNCHER=sccache \
      -DCMAKE_CXX_COMPILER_LAUNCHER=sccache \
      ..

# MSVC on CMake 3.25+
cmake -DCMAKE_C_COMPILER_LAUNCHER=sccache \
      -DCMAKE_CXX_COMPILER_LAUNCHER=sccache \
      -DCMAKE_MSVC_DEBUG_INFORMATION_FORMAT=Embedded \
      -DCMAKE_POLICY_CMP0141=NEW \
      ..

# Rust via environment variable
export RUSTC_WRAPPER=/path/to/sccache
cargo build --release

# Rust via ~/.cargo/config.toml
# [build]
# rustc-wrapper = "/path/to/sccache"
```

---

### Server lifecycle commands

sccache runs as a local daemon. These commands manage the server process.

```bash
# Start the server (without performing any compilation)
sccache --start-server

# Start on a custom port
SCCACHE_SERVER_PORT=4242 sccache --start-server

# Start on a Unix domain socket
SCCACHE_SERVER_UDS=$HOME/sccache.sock sccache --start-server

# Start on an abstract Unix socket
SCCACHE_SERVER_UDS=\\x00sccache.sock sccache --start-server

# Stop the running server
sccache --stop-server

# Print cache statistics
sccache --show-stats

# Zero the statistics counters
sccache --zero-stats
```

---

### Debugging the server

Run the server in the foreground with verbose logging for local debugging.

```bash
# Foreground mode with debug logging to stderr
SCCACHE_LOG=debug SCCACHE_START_SERVER=1 SCCACHE_NO_DAEMON=1 sccache

# Redirect server logs to a file during builds
SCCACHE_ERROR_LOG=/tmp/sccache_log.txt SCCACHE_LOG=debug cmake --build ./build

# Enable millisecond precision timestamps in logs
SCCACHE_LOG_MILLIS=1 SCCACHE_LOG=trace SCCACHE_START_SERVER=1 SCCACHE_NO_DAEMON=1 sccache
```

---

### Cache control variables

Control cache behavior without modifying the config file.

```bash
# Force recompilation, overwriting cached entries (useful for broken cache artifacts)
SCCACHE_RECACHE=1 sccache gcc -o foo.o -c foo.c

# Separate cache namespaces between different invocations (mixed into the hash)
SCCACHE_C_CUSTOM_CACHE_BUSTER=project-a sccache gcc -o foo.o -c foo.c
SCCACHE_C_CUSTOM_CACHE_BUSTER=project-b sccache gcc -o foo.o -c foo.c

# Gracefully fall back to local compiler if the sccache server is unreachable
SCCACHE_IGNORE_SERVER_IO_ERROR=1 sccache gcc -o foo.o -c foo.c

# Set zstd compression level for cached entries (range 1–22, default 3)
SCCACHE_CACHE_ZSTD_LEVEL=10 sccache gcc -o foo.o -c foo.c
```

---

### Path normalization with SCCACHE_BASEDIRS

Strip a base directory prefix from source paths before hashing, enabling cache sharing across different checkout locations (similar to ccache's `CCACHE_BASEDIR`).

```bash
# Single base directory
export SCCACHE_BASEDIRS=/home/user/project
sccache gcc -o foo.o -c /home/user/project/src/foo.c
# Hash uses "src/foo.c" instead of the absolute path

# Multiple base directories (longest prefix wins)
# Separated by ':' on Linux/macOS, ';' on Windows
export SCCACHE_BASEDIRS="/home/user/project:/home/user/workspace"

# Via config file (~/.config/sccache/config on Linux)
# basedirs = ["/home/user/project", "/home/user/workspace"]
```

---

## Configuration File

sccache reads its configuration from a TOML file. The path can be set with `SCCACHE_CONF`; default locations are `~/.config/sccache/config` (Linux), `~/Library/Application Support/Mozilla.sccache/config` (macOS), and `%APPDATA%\Mozilla\sccache\config\config` (Windows).

```toml
# ~/.config/sccache/config

# Time to wait for the server to start (milliseconds)
server_startup_timeout_ms = 10000

# Strip these path prefixes from source file paths before hashing
basedirs = ["/home/user/project"]

# ── Local disk cache ──────────────────────────────────────────────────────────
[cache.disk]
dir = "/tmp/.cache/sccache"
size = 10737418240  # 10 GiB

# Preprocessor cache mode settings (C/C++ only, local disk only)
[cache.disk.preprocessor_cache_mode]
use_preprocessor_cache_mode = true
file_stat_matches = false        # false = hash contents; true = size+ctime+mtime
use_ctime_for_stat = true
ignore_time_macros = false
skip_system_headers = false
hash_working_directory = true

# ── S3 ────────────────────────────────────────────────────────────────────────
[cache.s3]
bucket = "my-sccache-bucket"
endpoint = "s3-us-east-1.amazonaws.com"
use_ssl = true
key_prefix = "sccache/"
server_side_encryption = false

# ── Redis ─────────────────────────────────────────────────────────────────────
[cache.redis]
endpoint = "redis://127.0.0.1:6379"
# cluster_endpoints = "redis://10.0.0.1:6379,redis://10.0.0.2:6379"
username = "user"
password = "passwd"
db = 0
expiration = 86400
key_prefix = "sccache/"

# ── Multi-level (disk → redis → s3) ──────────────────────────────────────────
[cache.multilevel]
chain = ["disk", "redis", "s3"]
write_error_policy = "l0"  # ignore | l0 (default) | all
```

---

## Local Disk Cache

The default storage backend. The cache location and size are controlled by environment variables or config file entries.

```bash
# Change disk cache directory and size
export SCCACHE_DIR=/var/cache/sccache
export SCCACHE_CACHE_SIZE=20G   # suffix: B, K, M, G, T
sccache --start-server

# Set to read-only (only serve existing cache entries, never write new ones)
export SCCACHE_LOCAL_RW_MODE=READ_ONLY

# Enable preprocessor cache mode (C/C++ only; default: on)
export SCCACHE_DIRECT=true

# Verify disk cache is active
sccache --show-stats
# Cache location   Local disk: "/var/cache/sccache"
```

---

## S3 / S3-Compatible Storage

Use Amazon S3, MinIO, DigitalOcean Spaces, or Cloudflare R2 as the cache backend. Credentials are loaded from static env vars, `~/.aws/credentials`, EC2 instance metadata (IMDSv2), or AssumeRole.

```bash
# ── AWS S3 ────────────────────────────────────────────────────────────────────
export SCCACHE_BUCKET=my-sccache-bucket
export SCCACHE_REGION=us-east-1
export SCCACHE_S3_USE_SSL=true
export SCCACHE_S3_KEY_PREFIX=sccache/
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
sccache --start-server

# ── MinIO / custom S3-compatible endpoint ────────────────────────────────────
export SCCACHE_BUCKET=build-cache
export SCCACHE_ENDPOINT=http://minio.internal:9000
export SCCACHE_REGION=auto
export AWS_ACCESS_KEY_ID=minioadmin
export AWS_SECRET_ACCESS_KEY=minioadmin
sccache --start-server

# ── Cloudflare R2 ─────────────────────────────────────────────────────────────
export SCCACHE_BUCKET=my-r2-bucket
export SCCACHE_ENDPOINT=https://<ACCOUNT_ID>.r2.cloudflarestorage.com
export SCCACHE_REGION=auto
export AWS_ACCESS_KEY_ID=<R2_ACCESS_KEY>
export AWS_SECRET_ACCESS_KEY=<R2_SECRET_KEY>
sccache --start-server

# Enable server-side encryption (SSE-S3)
export SCCACHE_S3_SERVER_SIDE_ENCRYPTION=true

# Public read-only access (no credentials needed, e.g. for PR builds)
export SCCACHE_S3_NO_CREDENTIALS=true
sccache --start-server
```

---

## Redis Cache

Use a single Redis node or a Redis Cluster as the cache backend. Supports TLS and Unix sockets.

```bash
# ── Single node ───────────────────────────────────────────────────────────────
export SCCACHE_REDIS_ENDPOINT=redis://127.0.0.1:6379
sccache --start-server

# With authentication and custom DB
export SCCACHE_REDIS_ENDPOINT=redis://cache.internal:6379
export SCCACHE_REDIS_USERNAME=alice
export SCCACHE_REDIS_PASSWORD=secret
export SCCACHE_REDIS_DB=2
export SCCACHE_REDIS_EXPIRATION=86400   # 24 hours; 0 = never expire
export SCCACHE_REDIS_KEY_PREFIX=sccache/
sccache --start-server

# ── TLS ───────────────────────────────────────────────────────────────────────
export SCCACHE_REDIS_ENDPOINT=rediss://cache.internal:6380

# ── Redis Cluster ─────────────────────────────────────────────────────────────
export SCCACHE_REDIS_CLUSTER_ENDPOINTS=redis://10.0.0.1:6379,redis://10.0.0.2:6379,redis://10.0.0.3:6379
sccache --start-server

# Recommended Redis LRU configuration (redis.conf)
# maxmemory 50gb
# maxmemory-policy allkeys-lru
```

---

## Memcached Cache

Use a Memcached instance as the cache backend.

```bash
# Basic configuration
export SCCACHE_MEMCACHED_ENDPOINT=tcp://127.0.0.1:11211
sccache --start-server

# With authentication and TTL
export SCCACHE_MEMCACHED_ENDPOINT=tcp://memcached.internal:11211
export SCCACHE_MEMCACHED_USERNAME=user
export SCCACHE_MEMCACHED_PASSWORD=passwd
export SCCACHE_MEMCACHED_EXPIRATION=3600       # default: 86400 (1 day); max: 2592000 (30 days); 0 = disable
export SCCACHE_MEMCACHED_KEY_PREFIX=sccache/
sccache --start-server
```

---

## Google Cloud Storage Cache

Use a GCS bucket as the cache backend. Credentials are loaded from `SCCACHE_GCS_KEY_PATH`, `SCCACHE_GCS_CREDENTIALS_URL`, `GOOGLE_APPLICATION_CREDENTIALS`, well-known credential files, or VM metadata.

```bash
# Read-only (default)
export SCCACHE_GCS_BUCKET=my-sccache-bucket
export SCCACHE_GCS_RW_MODE=READ_ONLY
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa-key.json
sccache --start-server

# Read-write with explicit key file
export SCCACHE_GCS_BUCKET=my-sccache-bucket
export SCCACHE_GCS_RW_MODE=READ_WRITE
export SCCACHE_GCS_KEY_PATH=/path/to/service-account.json
export SCCACHE_GCS_KEY_PREFIX=sccache/
sccache --start-server

# Verify
sccache --show-stats
# Cache location   GCS, bucket: Bucket(name=my-sccache-bucket), key_prefix: sccache/
```

---

## Azure Blob Storage Cache

Use an Azure Blob Storage container as the cache backend. The container must already exist.

```bash
export SCCACHE_AZURE_CONNECTION_STRING="BlobEndpoint=https://myaccount.blob.core.windows.net/;SharedAccessSignature=sv=..."
export SCCACHE_AZURE_BLOB_CONTAINER=sccache
export SCCACHE_AZURE_KEY_PREFIX=builds/
sccache --start-server

# Note: environment variables are only read when the server starts.
# Always restart the server after changing these variables:
sccache --stop-server && sccache --start-server
```

---

## GitHub Actions Cache

Use the GitHub Actions cache API as the storage backend. Set `SCCACHE_GHA_ENABLED=on` and export the required GHA environment variables.

```yaml
# .github/workflows/build.yml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install sccache
        uses: mozilla-actions/sccache-action@v0.0.5

      - name: Export GHA cache variables
        uses: actions/github-script@v7
        with:
          script: |
            core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || '');
            core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');

      - name: Build
        env:
          SCCACHE_GHA_ENABLED: "on"
          SCCACHE_GHA_CACHE_TO: "sccache-${{ runner.os }}-${{ github.ref_name }}"
          SCCACHE_GHA_CACHE_FROM: "sccache-${{ runner.os }}-"
          RUSTC_WRAPPER: sccache
        run: cargo build --release
```

---

## WebDAV Cache

Use any WebDAV-compatible service (Ccache HTTP backend, Bazel Remote Cache, Gradle Build Cache) as a storage backend.

```bash
# Basic endpoint (no auth)
export SCCACHE_WEBDAV_ENDPOINT=http://bazel-cache.internal:8080/cache
sccache --start-server

# With basic authentication
export SCCACHE_WEBDAV_ENDPOINT=http://webdav.internal:80/sccache.php
export SCCACHE_WEBDAV_KEY_PREFIX=/custom/subfolder/
export SCCACHE_WEBDAV_USERNAME=alice
export SCCACHE_WEBDAV_PASSWORD=secret12
sccache --start-server

# With bearer token authentication
export SCCACHE_WEBDAV_ENDPOINT=https://remote-cache.example.com/
export SCCACHE_WEBDAV_TOKEN=eyJhbGciOiJSUzI1NiJ9...
sccache --start-server
```

---

## Alibaba Cloud OSS Cache

Use Alibaba Cloud Object Storage Service as the cache backend.

```bash
export SCCACHE_OSS_BUCKET=my-sccache-bucket
export SCCACHE_OSS_ENDPOINT=oss-us-east-1.aliyuncs.com
export SCCACHE_OSS_KEY_PREFIX=sccache/
export ALIBABA_CLOUD_ACCESS_KEY_ID=<your-access-key-id>
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=<your-access-key-secret>
sccache --start-server

# Public read-only (no credentials, useful for PR builds)
export SCCACHE_OSS_NO_CREDENTIALS=true
sccache --start-server
```

---

## Tencent Cloud COS Cache

Use Tencent Cloud Object Storage as the cache backend.

```bash
export SCCACHE_COS_BUCKET=my-sccache-bucket
export SCCACHE_COS_ENDPOINT=cos.na-siliconvalley.myqcloud.com
export SCCACHE_COS_KEY_PREFIX=sccache/
export TENCENTCLOUD_SECRET_ID=<your-secret-id>
export TENCENTCLOUD_SECRET_KEY=<your-secret-key>
sccache --start-server
```

---

## Multi-Level (Hierarchical) Cache

Chain multiple backends together in order from fastest to slowest. On a cache miss at level N, sccache checks level N+1 and backfills all faster levels asynchronously.

```bash
# ── Via environment variables ─────────────────────────────────────────────────
export SCCACHE_MULTILEVEL_CHAIN="disk,redis,s3"

# Level 0: local disk
export SCCACHE_DIR=/var/cache/sccache
export SCCACHE_CACHE_SIZE=10G

# Level 1: shared Redis
export SCCACHE_REDIS_ENDPOINT=redis://cache.internal:6379
export SCCACHE_REDIS_EXPIRATION=86400

# Level 2: S3 long-term storage
export SCCACHE_BUCKET=team-sccache
export SCCACHE_REGION=us-east-1
export SCCACHE_S3_USE_SSL=true

# Write error policy: ignore | l0 (default) | all
export SCCACHE_MULTILEVEL_WRITE_ERROR_POLICY=l0

sccache --start-server

# ── Via config file ───────────────────────────────────────────────────────────
# ~/.config/sccache/config
# [cache.multilevel]
# chain = ["disk", "redis", "s3"]
# write_error_policy = "l0"
#
# [cache.disk]
# dir = "/var/cache/sccache"
# size = 10737418240
#
# [cache.redis]
# endpoint = "redis://cache.internal:6379"
# expiration = 86400
#
# [cache.s3]
# bucket = "team-sccache"
# endpoint = "s3-us-east-1.amazonaws.com"
# use_ssl = true
```

---

## Rust Compilation Caching

Cache `rustc` invocations produced by cargo. Crates that invoke the system linker (`bin`, `dylib`, `cdylib`, `proc-macro`) and incrementally compiled crates cannot be cached.

```bash
# Option 1: RUSTC_WRAPPER environment variable
export RUSTC_WRAPPER=/usr/local/bin/sccache
cargo build --release

# Option 2: ~/.cargo/config.toml (persistent)
# [build]
# rustc-wrapper = "/usr/local/bin/sccache"

# Disable incremental compilation (required for caching)
# In ~/.cargo/config.toml or Cargo.toml:
# [profile.dev]
# incremental = false

# Full CI example with S3 backend
export RUSTC_WRAPPER=sccache
export SCCACHE_BUCKET=ci-rust-cache
export SCCACHE_REGION=us-east-1
export AWS_ACCESS_KEY_ID=$CI_AWS_KEY
export AWS_SECRET_ACCESS_KEY=$CI_AWS_SECRET
cargo build --release
sccache --show-stats
```

---

## Distributed Compilation

`sccache-dist` provides distributed compilation with encrypted transport, JWT-based authentication, and sandboxed execution (via bubblewrap or Docker on Linux).

### Scheduler setup

```toml
# scheduler.conf
public_addr = "127.0.0.1:10600"

[client_auth]
type = "token"
token = "my-client-secret-token"

[server_auth]
type = "jwt_hs256"
secret_key = "my-scheduler-secret-key"
```

```bash
# Generate a JWT HS256 key
sccache-dist auth generate-jwt-hs256-key

# Generate a per-server token (bind to IP:port)
sccache-dist auth generate-jwt-hs256-server-token \
    --config scheduler.conf \
    --server 192.168.1.10:10501

# Start the scheduler
sccache-dist scheduler --config scheduler.conf
```

### Build server setup

```toml
# server.conf
cache_dir = "/tmp/toolchains"
toolchain_cache_size = 10737418240   # 10 GB
public_addr = "192.168.1.10:10501"
scheduler_url = "https://192.168.1.1"

[builder]
type = "overlay"
build_dir = "/tmp/build"
bwrap_path = "/usr/bin/bwrap"

[scheduler_auth]
type = "jwt_token"
token = "<token from generate-jwt-hs256-server-token>"
```

```bash
# Start the build server (must run as root for bubblewrap)
sudo sccache-dist server --config server.conf

# Systemd unit for auto-start
# /etc/systemd/system/sccache-server.service
# [Service]
# ExecStart=/usr/local/bin/sccache-dist server --config /etc/sccache/server.conf
```

### Client setup for distributed compilation

```toml
# ~/.config/sccache/config
[dist]
scheduler_url = "https://192.168.1.1"
toolchains = []
toolchain_cache_size = 5368709120  # 5 GB

[dist.auth]
type = "token"
token = "my-client-secret-token"

# Custom toolchain (required for macOS/Windows cross-compilation)
[[dist.toolchains]]
type = "path_override"
compiler_executable = "/home/me/.mozbuild/clang/bin/clang"
archive = "/home/me/.mozbuild/toolchains/clang-dist-toolchain.tar.gz"
archive_compiler_executable = "/builds/worker/toolchains/clang/bin/clang"
```

```bash
# Check distributed compilation status
sccache --dist-status
# {"SchedulerStatus":["https://sccache1.internal/",{"num_servers":3,"num_cpus":56,"in_progress":24}]}

# Authenticate (OAuth2 flow)
sccache --dist-auth
```

---

## Jenkins Integration

Prevent parallel Jenkins jobs from sharing and killing each other's sccache server by using the Port Allocator Plugin to give each job its own port.

```bash
# In the Jenkins job shell step, the Port Allocator Plugin exports SCCACHE_SERVER_PORT
# Each job spawns its own isolated sccache server:
export SCCACHE_SERVER_PORT=${SCCACHE_SERVER_PORT}   # set by Port Allocator Plugin
export SCCACHE_DIR=${WORKSPACE}/.sccache
sccache --start-server

# Build steps here...
make -j$(nproc) CC="sccache gcc" CXX="sccache g++"

sccache --show-stats
sccache --stop-server

# Alternatively, run sccache as a persistent system service with:
# SCCACHE_IDLE_TIMEOUT=0 (never auto-exit)
```

---

## Building sccache from Source

```bash
# Build with all backends (default)
cargo build --release

# Build with specific backends only
cargo build --release --no-default-features --features=s3,redis,gcs

# Build distributed client + server
cargo build --release --features="dist-client dist-server"

# Static linking (portable binary, Linux with musl)
cargo build --release --target x86_64-unknown-linux-musl --features=dist-server

# Static OpenSSL (for portable dist-server builds)
cargo build --release --features="dist-server,openssl/vendored"

# Available feature flags:
# s3, redis, gcs, azure, memcached, gha, webdav, oss, cos
# dist-client, dist-server, openssl/vendored
```

---

sccache's most common use case is accelerating CI/CD pipelines and developer builds by caching compiler output in a shared remote store. Teams typically configure a single S3 bucket or Redis cluster as the shared backend, set `RUSTC_WRAPPER=sccache` or the CMake launcher flags, and let sccache transparently serve cache hits. The multi-level caching feature allows combining a fast local disk cache with a shared remote backend, dramatically improving hit rates: individual machines get near-instant local hits while still benefiting from team-shared builds on first-time compilations.

For large distributed teams or organizations running many parallel CI jobs, sccache-dist extends the caching benefit with distributed compilation, offloading compile work to a pool of build servers. This is especially valuable for C++ and Rust projects with large codebases. sccache integrates directly with CMake, Cargo, Jenkins, and GitHub Actions, and its pluggable backend architecture makes it straightforward to adopt in any existing build infrastructure without modifying source code or build scripts.