### Install stargz store for CRI-O/Podman

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/integration.md

Follow the installation guide for CRI-O and Podman to set up stargz store. This is required for using stargz with these runtimes.

```bash
./INSTALL.md#install-stargz-store-for-cri-opodman-with-systemd
```

--------------------------------

### Install stargz snapshotter for containerd

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/integration.md

Follow the installation guide for containerd to set up stargz snapshotter. This is required for using stargz on Kubernetes nodes with containerd.

```bash
./INSTALL.md#install-stargz-snapshotter-for-containerd-with-systemd
```

--------------------------------

### Install and enable Stargz Store service

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md

Extracts stargz-store binary, downloads the systemd service file, enables and starts the service, and restarts CRI-O if applicable.

```bash
tar -C /usr/local/bin -xvf stargz-snapshotter-${version}-linux-${arch}.tar.gz stargz-store
wget -O /etc/systemd/system/stargz-store.service https://raw.githubusercontent.com/containerd/stargz-snapshotter/main/script/config-cri-o/etc/systemd/system/stargz-store.service
systemctl enable --now stargz-store
systemctl restart cri-o # if you are using CRI-O

```

--------------------------------

### Install and Enable Stargz Snapshotter Service

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md

Extract stargz-snapshotter binaries, download the systemd service file, enable, and start the stargz-snapshotter service. Finally, restart containerd and Docker.

```bash
tar -C /usr/local/bin -xvf stargz-snapshotter-${version}-linux-${arch}.tar.gz containerd-stargz-grpc ctr-remote
wget -O /etc/systemd/system/stargz-snapshotter.service https://raw.githubusercontent.com/containerd/stargz-snapshotter/${version}/script/config/etc/systemd/system/stargz-snapshotter.service
systemctl enable --now stargz-snapshotter
systemctl restart containerd
systemctl restart docker

```

--------------------------------

### Start Stargz Store Systemd User Service

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/rootless.md

After installing the systemd unit file to `$HOME/.config/systemd/user/`, use this command to start the Stargz Store service for the current user.

```bash
$ systemctl --user start podman-rootless-stargz-store
```

--------------------------------

### Install stargz snapshotter for Docker/Moby

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/integration.md

Follow the installation guide for Docker/Moby to set up stargz snapshotter. This enables lazy pulling of eStargz images.

```bash
./INSTALL.md#install-stargz-snapshotter-for-dockermoby-with-systemd
```

--------------------------------

### Install and enable Stargz Snapshotter service

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md

Extracts stargz snapshotter binaries, downloads the systemd service file, enables and starts the service, and restarts containerd.

```bash
tar -C /usr/local/bin -xvf stargz-snapshotter-${version}-linux-${arch}.tar.gz containerd-stargz-grpc ctr-remote
wget -O /etc/systemd/system/stargz-snapshotter.service https://raw.githubusercontent.com/containerd/stargz-snapshotter/main/script/config/etc/systemd/system/stargz-snapshotter.service
systemctl enable --now stargz-snapshotter
systemctl restart containerd

```

--------------------------------

### Build and Run Demo Environment

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

Commands to build and launch the containerd demo environment using docker-compose. This setup requires privileged containers.

```bash
$ cd ${GOPATH}/src/github.com/containerd/stargz-snapshotter/script/demo
$ docker-compose build containerd_demo
$ docker-compose up -d
$ docker exec -it containerd_demo /bin/bash
(inside container) # ./script/demo/run.sh
```

--------------------------------

### Start Stargz Store with Podman Unshare

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/rootless.md

Run this command to start the Stargz Store in a namespace managed by Podman. Ensure the paths match your configuration.

```bash
$ podman unshare stargz-store --root $HOME/.local/share/stargz-store/data $HOME/.local/share/stargz-store/store &
```

--------------------------------

### Start containerd-stargz-grpc with Configuration

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/passthrough.md

After updating the configuration, restart the containerd-stargz-grpc service, specifying the path to your config.toml file.

```bash
$ containerd-stargz-grpc -config config.toml
```

--------------------------------

### Install FUSE on Ubuntu

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md

Installs the FUSE package on Ubuntu and loads the FUSE kernel module.

```bash
apt-get install fuse
modprobe fuse

```

--------------------------------

### Start IPFS Daemon with Private Network

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ipfs.md

Starts the IPFS daemon, enforcing the use of the private network. The daemon will fail to start if the private network is not configured.

```bash
export LIBP2P_FORCE_PNET=1
ipfs daemon
```

--------------------------------

### Go Program for Compilation

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

A simple Go program that prints 'hello world'. This file is used as an example source file to be mounted into a container for compilation.

```golang
package main

import "fmt"

func main() {
     fmt.Println("hello world")
}
```

--------------------------------

### Example OCI Descriptor for IPFS Image

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ipfs.md

This is an example OCI descriptor pointing to the image index of an IPFS-enabled image, obtained using `ipfs cat` and `jq`.

```console
# ipfs cat bafkreie7754qk7fl56ebauawdgfuqqa3kdd7sotvuhsm6wbz3qin6ssw3a | jq
{
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "digest": "sha256:80d6aec48c0a74635a5f3dc106328c1673afaa21ed6e1270a9a44de66e8ffa55",
  "size": 314,
  "urls": [
    "ipfs://bafkreiea22xmjdakorrvuxz5yeddfdawoox2uipnnyjhbknejxtg5d72ku"
  ]
}
```

--------------------------------

### Start IPFS Daemon

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ipfs.md

Ensure your IPFS daemon is running before proceeding with IPFS operations.

```bash
ipfs daemon
```

--------------------------------

### Generate IPFS Swarm Key

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ipfs.md

Installs and runs a utility to generate a swarm key for creating a private IPFS network. Save the output to `~/.ipfs/swarm.key`.

```bash
go install github.com/Kubuxu/go-ipfs-swarm-key-gen/ipfs-swarm-key-gen@latest
~/go/bin/ipfs-swarm-key-gen > ~/.ipfs/swarm.key
```

--------------------------------

### Docker-based Authentication Example

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/overview.md

Enables stargz snapshotter to access private registries using docker login credentials. Ensure you have logged in via `docker login` or `nerdctl login`.

```console
# docker login
(Enter username and password)
# ctr-remote image rpull --user <username>:<password> docker.io/<your-repository>/ubuntu:18.04
```

--------------------------------

### Accessing and Viewing Stargz Snapshotter Metadata

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/overview.md

This example demonstrates how to run a container with Stargz snapshotter, list the hidden state directory contents, and view the metadata JSON files for each layer. The metadata includes layer digest, size, and fetch progress.

```console
# ctr-remote run --rm -t --snapshotter=stargz docker.io/stargz/golang:1.12.9-esgz test /bin/bash
root@1d43741b8d29:/go# ls -a /
.   bin   dev  go    lib    media  opt     root  sbin  sys  usr
..  boot  etc  home  lib64  mnt    proc  run   srv   tmp  var
root@1d43741b8d29:/go# ls /.stargz-snapshotter/*
/.stargz-snapshotter/sha256:2b1fc65cafe05b65acc9e9f186df4dd81ae74c58ef73d89ecfc15e7286b3e960.json
/.stargz-snapshotter/sha256:42d56485c1f672e394a02855048774621731c8fd44a54dc816a421a3a52b8482.json
/.stargz-snapshotter/sha256:6a5826d877de5c93fb4a9e1d0369cfdef6d43df2610562501ebf42e4bcb2ef73.json
/.stargz-snapshotter/sha256:a4d35801573274df19d9c2ae2aed80eba96d5aa69a38c464e1f01f9abf81e34e.json
/.stargz-snapshotter/sha256:ab13100112faac6e04d2da2281db3df942efc8cef2532ab2cac688c6232944d8.json
/.stargz-snapshotter/sha256:e8cc31024eb09fe216ad906392aec139038330c6d29dfd3fe5c81c4b2dd21430.json
/.stargz-snapshotter/sha256:f077511be7d385c17ba88980379c5cd0aab7068844dffa7a1cefbf68cc3daea3.json
root@1d43741b8d29:/go# cat /.stargz-snapshotter/*
{"digest":"sha256:2b1fc65cafe05b65acc9e9f186df4dd81ae74c58ef73d89ecfc15e7286b3e960","size":131339690,"fetchedSize":7939690,"fetchedPercent":6.045156646859757}
{"digest":"sha256:42d56485c1f672e394a02855048774621731c8fd44a54dc816a421a3a52b8482","size":10047608,"fetchedSize":2047608,"fetchedPercent":20.379059374131632}
{"digest":"sha256:6a5826d877de5c93fb4a9e1d0369cfdef6d43df2610562501ebf42e4bcb2ef73","size":54352828,"fetchedSize":2302828,"fetchedPercent":4.236813584014432}
{"digest":"sha256:a4d35801573274df19d9c2ae2aed80eba96d5aa69a38c464e1f01f9abf81e34e","size":70359295,"fetchedSize":2259295,"fetchedPercent":3.211082487395588}
{"digest":"sha256:ab13100112faac6e04d2da2281db3df942efc8cef2532ab2cac688c6232944d8","size":7890588,"fetchedSize":2140588,"fetchedPercent":27.12837116828302}
{"digest":"sha256:e8cc31024eb09fe216ad906392aec139038330c6d29dfd3fe5c81c4b2dd21430","size":52934435,"fetchedSize":2634435,"fetchedPercent":4.976788738748227}
{"digest":"sha256:f077511be7d385c17ba88980379c5cd0aab7068844dffa7a1cefbf68cc3daea3","size":580,"fetchedSize":580,"fetchedPercent":100}
```

--------------------------------

### Install FUSE on CentOS

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md

Installs the FUSE package on CentOS 7 or 8 and loads the FUSE kernel module.

```bash
# centos 7
yum install fuse
# centos 8
dnf install fuse

modprobe fuse

```

--------------------------------

### Pull image using 'ctr' with Stargz Snapshotter

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/transfer.md

Example command to pull an image using the 'ctr' client with the Stargz Snapshotter enabled, leveraging Transfer Service.

```bash
# ctr image pull --snapshotter=stargz ghcr.io/stargz-containers/ubuntu:24.04-esgz
```

--------------------------------

### Prefetch List Glob Patterns for Image Optimization

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

Examples of glob patterns that can be used in a prefetch list file for `ctr-remote image optimize`. These patterns define which files and directories should be prefetched.

```bash
/usr/bin/*
```

```bash
/usr/lib/**/*.so
```

```bash
/var/log/**/app.*.log
```

```bash
/opt/data/img_[ab].png
```

--------------------------------

### Log of Accessed Files During Optimization

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

This is an example of the JSON log file generated by `ctr-remote image optimize --record-out`. Each entry details a file path, its manifest digest, and layer index that was accessed during the optimization.

```json
{"path":"usr/bin/bash","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"usr/bin/bash","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"etc/ld.so.cache","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"usr/lib/x86_64-linux-gnu/libtinfo.so.6.4","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"usr/lib/x86_64-linux-gnu/libc.so.6","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"etc/nsswitch.conf","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"etc/nsswitch.conf","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"etc/passwd","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"usr/bin/ls","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"etc/ld.so.cache","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"usr/lib/x86_64-linux-gnu/libselinux.so.1","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"usr/lib/x86_64-linux-gnu/libc.so.6","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
{"path":"usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.11.2","manifestDigest":"sha256:5d070ad5f7fe63623cbb99b4fc0fd997f5591303d4b03ccce50f403957d0ddc4","layerIndex":0}
```

--------------------------------

### Pull image using 'crictl' with Stargz Snapshotter

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/transfer.md

Example command to pull an image using the 'crictl' client with Stargz Snapshotter enabled, suitable for CRI environments utilizing Transfer Service.

```bash
# crictl image pull ghcr.io/stargz-containers/ubuntu:24.04-esgz
```

--------------------------------

### Create Buildx Builder with Specific BuildKit Version

Source: https://github.com/containerd/stargz-snapshotter/blob/main/README.md

If your Docker version does not include a recent BuildKit, use `docker buildx create` to specify an older version like v0.10.3. This allows building eStargz images with older Docker installations.

```bash
docker buildx create --use --name v0.10.3 --driver docker-container --driver-opt image=moby/buildkit:v0.10.3
```

```bash
docker buildx inspect --bootstrap v0.10.3
```

--------------------------------

### Configure containerd for Stargz Snapshotter and Transfer Service (CRI)

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/transfer.md

Add this TOML configuration to containerd's config.toml to enable Stargz Snapshotter with Transfer Service for CRI clients. This includes basic CRI setup and the necessary Transfer Service and Stargz Snapshotter plugin configurations.

```toml
version = 2

# Basic CRI configuration with enabling Stargz Snapshotter
[plugins."io.containerd.grpc.v1.cri".containerd]
  default_runtime_name = "runc"
  snapshotter = "stargz"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"

# Enable Stargz Snapshotter in Transfer Service
[[plugins."io.containerd.transfer.v1.local".unpack_config]]
  platform = "linux"
  snapshotter = "stargz"

# Plugin Stargz Snapshotter
[proxy_plugins]
  [proxy_plugins.stargz]
    type = "snapshot"
    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"
  [proxy_plugins.stargz.exports]
    root = "/var/lib/containerd-stargz-grpc/"
    enable_remote_snapshot_annotations = "true"
```

--------------------------------

### Quick demo using k3d with stargz-snapshotter

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/integration.md

This demonstrates creating a k3d cluster with stargz-snapshotter enabled, deploying a Node.js application using an eStargz image, and accessing it.

```bash
$ k3d cluster create mycluster --k3s-arg='--snapshotter=stargz@server:*;agent:*'
$ cat <<'EOF' | kubectl --context=k3d-mycluster apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: nodejs
spec:
  containers:
  - name: nodejs-stargz
    image: ghcr.io/stargz-containers/node:17.8.0-esgz
    command: ["node"]
    args:
    - -e
    - var http = require('http');
      http.createServer(function(req, res) {
        res.writeHead(200);
        res.end('Hello World!\n');
      }).listen(80);
    ports:
    - containerPort: 80
EOF
$ kubectl --context=k3d-mycluster get po nodejs -w
$ kubectl --context=k3d-mycluster port-forward nodejs 8080:80 &
$ curl 127.0.0.1:8080
Hello World!
$ k3d cluster delete mycluster
```

--------------------------------

### Configure Podman for Stargz Store

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/rootless.md

Add this configuration to `/etc/containers/storage.conf` or `$HOME/.config/containers/storage.conf` to enable Podman to use Stargz Store. Replace `/path/to/home` with your actual home directory.

```toml
[storage]
driver = "overlay"

[storage.options]
additionallayerstores = ["/path/to/homedir/.local/share/stargz-store/store:ref"]
```

--------------------------------

### Example eStargz TOC JSON Structure

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/estargz.md

This JSON structure represents an example Table of Contents for an eStargz image. It lists directory and file entries with their metadata, including name, type, size, modification time, and digest.

```json
{
  "version": 1,
  "entries": [
    {
      "name": "bin/",
      "type": "dir",
      "modtime": "2019-08-20T10:30:43Z",
      "mode": 16877,
      "NumLink": 0
    },
    {
      "name": "bin/busybox",
      "type": "reg",
      "size": 833104,
      "modtime": "2019-06-12T17:52:45Z",
      "mode": 33261,
      "offset": 126,
      "NumLink": 0,
      "digest": "sha256:8b7c559b8cccca0d30d01bc4b5dc944766208a53d18a03aa8afe97252207521f",
      "chunkDigest": "sha256:8b7c559b8cccca0d30d01bc4b5dc944766208a53d18a03aa8afe97252207521f"
    },
    {
      "name": "lib/",
      "type": "dir",
      "modtime": "2019-08-20T10:30:43Z",
      "mode": 16877,
      "NumLink": 0
    },
    {
      "name": "lib/ld-musl-x86_64.so.1",
      "type": "reg",
      "size": 580144,
      "modtime": "2019-08-07T07:15:30Z",
      "mode": 33261,
      "offset": 512427,
      "NumLink": 0,
      "digest": "sha256:45c6ee3bd1862697eab8058ec0e462f5a760927331c709d7d233da8ffee40e9e",
      "chunkDigest": "sha256:45c6ee3bd1862697eab8058ec0e462f5a760927331c709d7d233da8ffee40e9e"
    },
    {
      "name": ".prefetch.landmark",
      "type": "reg",
      "size": 1,
      "offset": 886633,
      "NumLink": 0,
      "digest": "sha256:dc0e9c3658a1a3ed1ec94274d8b19925c93e1abb7ddba294923ad9bde30f8cb8",
      "chunkDigest": "sha256:dc0e9c3658a1a3ed1ec94274d8b19925c93e1abb7ddba294923ad9bde30f8cb8"
    },
    ...
  ]
}
```

--------------------------------

### Enable Stargz Snapshotter with `--snapshotter=stargz`

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/lima.md

Use the `--snapshotter=stargz` flag with `nerdctl.lima` to verify if stargz-snapshotter is enabled and check the storage driver.

```bash
$ nerdctl.lima --snapshotter=stargz system info | grep stargz
 Storage Driver: stargz
```

--------------------------------

### Pull and Run IPFS Image with OverlayFS

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ipfs.md

This command pulls an image from IPFS using the overlayfs snapshotter and then runs a container from it. It demonstrates a slower cold-start due to not using stargz snapshotter.

```console
# time ( ctr-remote i rpull --snapshotter=overlayfs --ipfs bafkreienbir4knaofs3o5f57kqw2the2v7zdhdlzpkq346mipuopwvqhty && \
         ctr-remote run --snapshotter=overlayfs --rm -t bafkreienbir4knaofs3o5f57kqw2the2v7zdhdlzpkq346mipuopwvqhty foo python -c 'print("Hello, World!")' )
fetching sha256:7240ac9f... application/vnd.oci.image.index.v1+json
fetching sha256:17dc54f4... application/vnd.oci.image.manifest.v1+json
fetching sha256:6f1289b1... application/vnd.oci.image.config.v1+json
fetching sha256:9476e460... application/vnd.oci.image.layer.v1.tar+gzip
fetching sha256:64c0f10e... application/vnd.oci.image.layer.v1.tar+gzip
fetching sha256:4c25b309... application/vnd.oci.image.layer.v1.tar+gzip
fetching sha256:942374d5... application/vnd.oci.image.layer.v1.tar+gzip
fetching sha256:3fff52a3... application/vnd.oci.image.layer.v1.tar+gzip
fetching sha256:5cf06daf... application/vnd.oci.image.layer.v1.tar+gzip
fetching sha256:419e258e... application/vnd.oci.image.layer.v1.tar+gzip
fetching sha256:1acf5650... application/vnd.oci.image.layer.v1.tar+gzip
fetching sha256:b95c0dd0... application/vnd.oci.image.layer.v1.tar+gzip
Hello, World!

real	0m11.320s
user	0m0.556s
sys	0m0.280s
```

--------------------------------

### Build eStargz Image with Docker Buildx

Source: https://github.com/containerd/stargz-snapshotter/blob/main/README.md

Use this command to build an eStargz image and push it to a registry. Ensure BuildKit v0.10 or later is used. Flags `oci-mediatypes=true` and `compression=estargz` are essential.

```bash
docker buildx build -t ghcr.io/ktock/hello:esgz \
    -o type=registry,oci-mediatypes=true,compression=estargz,force-compression=true \
    /tmp/buildctx/
```

--------------------------------

### Apply Pod and Test Service

Source: https://github.com/containerd/stargz-snapshotter/blob/main/README.md

These commands apply the stargz-pod.yaml to the Kind cluster, monitor the pod's status, forward a local port to the pod's service, and then test the service with curl.

```console
kubectl --context kind-stargz-demo apply -f stargz-pod.yaml && kubectl --context kind-stargz-demo get po nodejs -w
kubectl --context kind-stargz-demo port-forward nodejs 8080:80 &
curl 127.0.0.1:8080

```

--------------------------------

### Go program for testing lazy pulling

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/integration.md

A simple Go program that prints 'Hello, world!'. This program is used in conjunction with the Dockerfile to test the lazy pulling of eStargz images.

```go
package main

import "fmt"

func main() {
	fmt.Println("Hello, world!")
}
```

--------------------------------

### Create Kind Cluster with Stargz Snapshotter Image

Source: https://github.com/containerd/stargz-snapshotter/blob/main/README.md

Use this command to create a Kind Kubernetes cluster with a prebuilt node image that includes the stargz snapshotter configuration.

```console
kind create cluster --name stargz-demo --image ghcr.io/containerd/stargz-snapshotter:0.12.1-kind
```

--------------------------------

### TOC Image Layer Descriptor Example

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/estargz.md

This JSON object represents a layer descriptor within a TOC image. It includes the layer's digest and an annotation specifying the digest of the corresponding eStargz layer blob.

```json
{
  "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
  "digest": "sha256:64dedefd539280a5578c8b94bae6f7b4ebdbd12cb7a7df0770c4887a53d9af70",
  "size": 154425,
  "annotations": {
    "containerd.io/snapshot/stargz/layer.digest": "sha256:5da5601c1f2024c07f580c11b2eccf490cd499473883a113c376d64b9b10558f"
  }
}
```

--------------------------------

### Build and run image with lazy pulling

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/integration.md

Commands to build a Docker image using the Dockerfile and then run the container to verify the application works. This demonstrates the end-to-end lazy pulling workflow.

```console
$ docker buildx build --load -t hello /tmp/ctx/
$ docker run --rm hello
```

--------------------------------

### Create Docker Buildx Builder with stargz-snapshotter

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/integration.md

Use this command to create a new Docker Buildx builder instance that is configured to use stargz-snapshotter for lazy pulling of eStargz images.

```bash
$ docker buildx create --use --name lazy-builder --buildkitd-flags '--oci-worker-snapshotter=stargz'
$ docker buildx inspect --bootstrap lazy-builder
```

--------------------------------

### Import Stargz Snapshotter as Go Module

Source: https://github.com/containerd/stargz-snapshotter/blob/main/README.md

Instructions for importing the stargz-snapshotter and its estargz sub-module into a Go project. Ensure both imports point to the same commit version.

```go
import (
	_ "github.com/containerd/stargz-snapshotter"
	_ "github.com/containerd/stargz-snapshotter/estargz"
)
```

--------------------------------

### Optimize Image with Host File Mount

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

Command to optimize a container image by mounting a local Go source file into the container. This allows the container to compile the program using the mounted source.

```bash
ctr-remote image optimize --oci \
           --mount=type=bind,source=/tmp/hello.go,destination=/hello.go,options=bind:ro \
           --entrypoint='[ "/bin/bash", "-c" ]' --args='[ "go build -o /hello /hello.go && /hello" ]' \
           ghcr.io/stargz-containers/golang:1.15.3-buster-org \
           registry2:5000/golang:1.15.3-esgz-hello-world
```

--------------------------------

### Configure Registry Mirror with Custom Headers

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/overview.md

Sets custom headers to be sent to a specific registry mirror host. Note that headers are not passed to redirected locations.

```toml
[[resolver.host."registry2:5000".mirrors]]
  host = "registry2:5000"
  [resolver.host."registry2:5000".mirrors.header]
    x-custom-2 = ["value3", "value4"]
```

--------------------------------

### Building eStargz images using BuildKit

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/integration.md

BuildKit version 0.10 and later supports creating eStargz images. Refer to the README for detailed instructions.

```bash
README.md#building-estargz-images-using-buildkit
```

--------------------------------

### Create KinD Cluster with Lazy Pulling Enabled

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/pre-converted-images.md

Use this command to create a KinD cluster with a prebuilt node image that has lazy pulling enabled for eStargz images. Replace `${VERSION}` with the desired stargz-snapshotter version.

```console
$ kind create cluster --name stargz-demo --image ghcr.io/containerd/stargz-snapshotter:0.12.1-kind
```

--------------------------------

### Enable IPFS in Stargz Snapshotter Config

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ipfs.md

Add this configuration to `config.toml` to enable IPFS support in Stargz Snapshotter.

```toml
ipfs = true
```

--------------------------------

### Default /etc/resolv.conf Configuration

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

This is the default content for the /etc/resolv.conf file within the container when CNI networking is enabled.

```text
nameserver 8.8.8.8
```

--------------------------------

### Configure Registry Mirror with Insecure Connection

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/overview.md

Specifies a mirror host for a registry and enables plain HTTP connection. Also sets a custom request timeout for the mirror.

```toml
# Use `mirrorhost.io` as a mirrored host of `exampleregistry.io` and
# use plain HTTP for connecting to the mirror host.
[[resolver.host."exampleregistry.io".mirrors]]
host = "mirrorhost.io"
insecure = true
request_timeout_sec = 60

# Use plain HTTP for connecting to `exampleregistry.io`.
[[resolver.host."exampleregistry.io".mirrors]]
host = "exampleregistry.io"
insecure = true
```

--------------------------------

### Dockerfile using eStargz base image

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/integration.md

A sample Dockerfile that utilizes an eStargz-formatted Go image as a base for building an application. This enables lazy pulling during the build process.

```dockerfile
FROM ghcr.io/stargz-containers/golang:1.18-esgz AS dev
COPY ./hello.go /hello.go
RUN go build -o /hello /hello.go

FROM scratch
COPY --from=dev /hello /
ENTRYPOINT [ "/hello" ]
```

--------------------------------

### Configure Docker for Stargz Snapshotter

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md

Enable the containerd-snapshotter feature and set 'stargz' as the storage driver in Docker's daemon.json configuration.

```json
{
  "features": {
    "containerd-snapshotter": true
  },
  "storage-driver": "stargz"
}
```

--------------------------------

### Build eStargz Image with Kaniko

Source: https://github.com/containerd/stargz-snapshotter/blob/main/README.md

Kaniko v1.5.0+ experimentally supports building eStargz images. Set the environment variable `GGCR_EXPERIMENT_ESTARGZ=1` when running Kaniko.

```bash
docker run --rm -e GGCR_EXPERIMENT_ESTARGZ=1 \
    -v /tmp/buildctx:/workspace -v ~/.docker/config.json:/kaniko/.docker/config.json:ro \
    gcr.io/kaniko-project/executor:v1.8.1 --destination ghcr.io/ktock/hello:esgz
```

--------------------------------

### Configure CRI-O/Podman storage for Stargz Store

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md

Add this configuration to CRI-O/Podman's storage.conf to enable Stargz Store as an additional layer store.

```toml
[storage]
  driver = "overlay"
  graphroot = "/var/lib/containers/storage"
  runroot = "/run/containers/storage"

[storage.options]
  additionallayerstores = ["/var/lib/stargz-store/store:ref"]

```

--------------------------------

### Optimize Image with Custom Workload Configuration

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

Optimize a container image against a specific workload, such as running a command. This command specifies the entrypoint and arguments for the optimized workload.

```bash
ctr-remote image optimize --oci \
           --entrypoint='[ "/bin/bash", "-c" ]' --args='[ "go version" ]' \
           ghcr.io/stargz-containers/golang:1.15.3-buster-org \
           registry2:5000/golang:1.15.3-esgz-go-version
```

--------------------------------

### Converting All Platforms in a Multi-Platform Image

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

Use the `--all-platforms` option with `ctr-remote image optimize` to convert all images contained within a multi-platform image. This ensures all variants are optimized.

```bash
ctr-remote image optimize --oci \
           --all-platforms \
           ghcr.io/stargz-containers/golang:1.15.3-buster-org \
           registry2:5000/golang:1.15.3-esgz-fat
```

--------------------------------

### Run Python eStargz Image with Stargz Snapshotter

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/lima.md

Demonstrates lazy pulling and running a Python eStargz image using `nerdctl.lima` with the stargz snapshotter enabled.

```bash
$ nerdctl.lima --snapshotter=stargz run --rm -it --name python ghcr.io/stargz-containers/python:3.13-esgz
Python 3.13.2 (main, Feb  6 2025, 22:37:13) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
```

--------------------------------

### Build and Convert Image to eStargz with nerdctl

Source: https://github.com/containerd/stargz-snapshotter/blob/main/README.md

Use nerdctl to build an image and then convert it to eStargz format. The `--estargz` flag must be used with `--oci` during conversion.

```bash
nerdctl build -t ghcr.io/ktock/hello:1 /tmp/buildctx
```

```bash
nerdctl image convert --estargz --oci ghcr.io/ktock/hello:1 ghcr.io/ktock/hello:esgz
```

```bash
nerdctl push ghcr.io/ktock/hello:esgz
```

--------------------------------

### Push Image to IPFS with ctr-remote

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ipfs.md

Converts an image to IPFS-enabled eStargz and stores it to IPFS using `ctr-remote i ipfs-push`.

```console
# ctr-remote i pull ghcr.io/stargz-containers/python:3.9-org
# ctr-remote i ipfs-push ghcr.io/stargz-containers/python:3.9-org
bafkreie7754qk7fl56ebauawdgfuqqa3kdd7sotvuhsm6wbz3qin6ssw3a
```

--------------------------------

### Configure containerd for Stargz Snapshotter

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md

Add this configuration to containerd's config.toml to enable the stargz snapshotter.

```toml
version = 2

# Enable stargz snapshotter for CRI
[plugins."io.containerd.grpc.v1.cri".containerd]
  snapshotter = "stargz"
  disable_snapshot_annotations = false

# Plug stargz snapshotter into containerd
[proxy_plugins]
  [proxy_plugins.stargz]
    type = "snapshot"
    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"
  [proxy_plugins.stargz.exports]
    root = "/var/lib/containerd-stargz-grpc/"

```

--------------------------------

### Configure containerd for Stargz Snapshotter

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/INSTALL.md

Add stargz snapshotter as a proxy plugin in containerd's config.toml to enable its integration.

```toml
version = 2

# Plug stargz snapshotter into containerd
[proxy_plugins]
  [proxy_plugins.stargz]
    type = "snapshot"
    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"
  [proxy_plugins.stargz.exports]
    root = "/var/lib/containerd-stargz-grpc/"

```

--------------------------------

### Enable Database Metadata Store

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/overview.md

Configures stargz-snapshotter to use a disk-based database for storing filesystem metadata, which can reduce memory consumption compared to the default in-memory store.

```toml
metadata_store = "db"
```

--------------------------------

### Configure containerd for Stargz Snapshotter and Transfer Service (non-CRI)

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/transfer.md

Add this TOML configuration to containerd's config.toml to enable Stargz Snapshotter with Transfer Service for clients like 'ctr'. Ensure containerd-stargz-grpc is running and set `enable_remote_snapshot_annotations` to 'true'.

```toml
version = 2

# Enable Stargz Snapshotter in Transfer Service
[[plugins."io.containerd.transfer.v1.local".unpack_config]]
  platform = "linux"
  snapshotter = "stargz"

# Plugin Stargz Snapshotter
[proxy_plugins]
  [proxy_plugins.stargz]
    type = "snapshot"
    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"
  [proxy_plugins.stargz.exports]
    root = "/var/lib/containerd-stargz-grpc/"
    enable_remote_snapshot_annotations = "true"
```

--------------------------------

### Enable Fuse Manager Configuration

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/overview.md

Add this TOML configuration to enable the fuse manager. Specify the socket address and optionally the binary path if it's not in the system's PATH.

```toml
[fuse_manager]
enable = true
# address must be an absolute path; default is "/run/containerd-stargz-grpc/fuse-manager.sock"
address = "/run/containerd-stargz-grpc/fuse-manager.sock"
# set a custom binary path if the executable is not in PATH
path = "/usr/local/bin/stargz-fuse-manager"

```

--------------------------------

### Accelerate Optimization with pigz

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

Uses the 'pigz' command to speed up gzip decompression during image optimization. This can significantly reduce the overall optimization time.

```bash
# ctr-remote image optimize --oci --estargz-gzip-helper pigz ghcr.io/stargz-containers/golang:1.15.3-buster-org registry2:5000/golang:1.15.3-esgz
```

--------------------------------

### Configure Stargz Snapshotter in containerd

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/overview.md

This TOML configuration snippet shows how to integrate the stargz snapshotter as a proxy plugin into containerd. It specifies the plugin type, socket address, and root directory for snapshot operations. It also demonstrates how to set stargz as the default snapshotter for the containerd CRI plugin.

```toml
version = 2

# Plug stargz snapshotter into containerd
# Containerd recognizes stargz snapshotter through specified socket address.
# The specified address below is the default which stargz snapshotter listen to.
[proxy_plugins]
  [proxy_plugins.stargz]
    type = "snapshot"
    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"
  [proxy_plugins.stargz.exports]
    root = "/var/lib/containerd-stargz-grpc/"

# Use stargz snapshotter through CRI
[plugins."io.containerd.grpc.v1.cri".containerd]
  snapshotter = "stargz"
  disable_snapshot_annotations = false
```

--------------------------------

### Optimize Image and Record Accessed Files

Source: https://github.com/containerd/stargz-snapshotter/blob/main/docs/ctr-remote.md

This command optimizes an image and records the accessed files during the optimization process to a specified log file. It uses `ctr-remote image pull` to fetch the image and then `ctr-remote image optimize` with `--record-out` to log file access.

```bash
ctr-remote image pull docker.io/library/ubuntu:24.04
c tr-remote image optimize --record-out=/tmp/log.json \
  --entrypoint='[ "/bin/bash", "-c" ]' --args='[ "ls" ]' \
  docker.io/library/ubuntu:24.04 registry2:5000/ubuntu:24.04
```