### Get Started with ZeroFS Source: https://github.com/barre/zerofs/blob/main/README.md Basic steps to get ZeroFS running. This involves generating a configuration file, editing it with S3 credentials, and then running the ZeroFS service. ```bash zerofs init ``` ```bash $EDITOR zerofs.toml ``` ```bash zerofs run -c zerofs.toml ``` -------------------------------- ### Install Go Client Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/client-libraries/page.mdx Get the Go client library module. Note that this does not include the native library; you must build the `zerofs-ffi` cdylib separately and configure cgo. ```bash go get github.com/Barre/ZeroFS/zerofs/zerofs-ffi/bindings/go ``` -------------------------------- ### Install ZeroFS Source: https://github.com/barre/zerofs/blob/main/website/index.html Use this command to install ZeroFS. It fetches and executes the installation script. ```bash curl -sSfL https://sh.zerofs.net | sh ``` -------------------------------- ### Basic ZeroFS Start Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/quickstart/page.mdx Start ZeroFS using a configuration file. Supports both the full command and a shorthand. ```bash # Start ZeroFS with configuration file zerofs run --config zerofs.toml # Or use the shorthand zerofs run -c zerofs.toml ``` -------------------------------- ### Install ZeroFS Source: https://github.com/barre/zerofs/blob/main/website/index.html Run this command to install ZeroFS. ```shell $ curl -sSfL https://sh.zerofs.net | sh ``` -------------------------------- ### Deploy ZeroFS Gateway Example Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/kubernetes-csi/page.mdx Download and apply the example manifest for the ZeroFS gateway. Ensure to edit the Secret with your storage URL, encryption password, and credentials before applying. ```bash curl -fsSLO https://raw.githubusercontent.com/Barre/ZeroFS/main/zerofs/zerofs-csi/deploy/gateway-example.yaml # Edit the Secret: storage url, encryption_password, credentials. kubectl apply -f gateway-example.yaml ``` -------------------------------- ### Enable and Start Systemd Mount Unit Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/9p-access/page.mdx Use systemctl to enable the mount unit to start on boot and start it immediately. ```bash sudo systemctl enable mnt-zerofs-9p.mount sudo systemctl start mnt-zerofs-9p.mount ``` -------------------------------- ### Start ZeroFS with Configuration File Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/configuration/page.mdx Use the `run` command with the `--config` or `-c` flag to specify a configuration file. This is the primary method for starting ZeroFS with custom settings. ```bash # Start ZeroFS with a config file zerofs run --config /etc/zerofs/zerofs.toml # Or use the shorthand zerofs run -c zerofs.toml ``` -------------------------------- ### Install Python Client Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/client-libraries/page.mdx Install the official Python client library using pip. The wheel includes the native library, making the installation self-contained. ```bash pip install zerofs-client ``` -------------------------------- ### Deploy ZeroFS CSI StorageClass Example Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/kubernetes-csi/page.mdx Apply the example StorageClass manifest for the ZeroFS CSI driver. This defines how volumes will be provisioned. ```bash kubectl apply -f $DEPLOY/storageclass-example.yaml ``` -------------------------------- ### Create and Mount ZeroFS Source: https://github.com/barre/zerofs/blob/main/README.md Steps to start the ZeroFS server and mount it via NFS or 9P to manage devices. ```bash # Start ZeroFS zerofs run -c zerofs.toml # Mount ZeroFS via NFS or 9P to manage devices mount -t nfs 127.0.0.1:/ /mnt/zerofs # or mount -t 9p -o trans=tcp,port=5564 127.0.0.1 /mnt/zerofs ``` -------------------------------- ### Cache Allocation Log Example Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/caching/page.mdx Example log output showing the resolved disk and memory cache allocations after configuration is applied. ```text Cache allocation - Disk: 10.00GB total (8926 MB decoded-blocks + 1073 MB raw-parts), Memory: 2.00GB ``` -------------------------------- ### Install script (Recommended) Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/quickstart/page.mdx Installs ZeroFS using a curl script. You can pin a specific version and install to a custom directory without root privileges. ```bash curl -sSfL https://sh.zerofs.net | sh ``` ```bash curl -sSfL https://sh.zerofs.net | VERSION=v1.2.5 INSTALL_DIR=$HOME/.local/bin sh ``` -------------------------------- ### Run ZeroFS Read-Write Instance Source: https://github.com/barre/zerofs/blob/main/README.md Start the default read-write ZeroFS instance using a configuration file. ```bash zerofs run -c zerofs.toml ``` -------------------------------- ### Install ZeroFS using curl Source: https://github.com/barre/zerofs/blob/main/website/social/card.html This command downloads and executes the ZeroFS installation script. Ensure you review the script before execution. ```bash $curl -sSfL https://sh.zerofs.net | sh ``` -------------------------------- ### Start the Standalone Compactor Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/standalone-compactor/page.mdx Run this command on a separate instance to start the dedicated ZeroFS compactor process. It uses the same configuration file as the main server. ```bash zerofs compactor -c zerofs.toml ``` -------------------------------- ### ZeroFS Start with NBD Support Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/quickstart/page.mdx Configure and start ZeroFS with Network Block Device (NBD) support. Includes steps to add NBD configuration and create NBD devices. ```bash # Add NBD configuration to your TOML file cat >> zerofs.toml < Result<(), zerofs_client::ZeroFsError> { let fs = Client::connect("unix:/run/zerofs/9p.sock").await?; fs.create_dir_all("/projects/demo", 0o755).await?; fs.write("/projects/demo/hello.txt", b"hello from zerofs").await?; let data = fs.read("/projects/demo/hello.txt").await?; assert_eq!(&data[..], b"hello from zerofs"); for entry in fs.read_dir("/projects/demo").await? { let size = entry.metadata.as_ref().map_or(0, |m| m.size); println!("{size:>8} {}", entry.name); } let file = fs .open("/projects/demo/big.bin", OpenOptions::read_write().create(true)) .await?; file.write_at(0, &vec![0u8; 4 << 20]).await?; file.sync_all().await?; file.close().await; Ok(()) } ``` -------------------------------- ### Install ZeroFS via Docker Source: https://github.com/barre/zerofs/blob/main/README.md Installs and runs ZeroFS using Docker. This includes pulling the image, generating a starter configuration, and running the container with the configuration mounted. ```bash docker pull ghcr.io/barre/zerofs:latest ``` ```bash docker run --rm ghcr.io/barre/zerofs:latest init - > zerofs.toml ``` ```bash $EDITOR zerofs.toml ``` ```bash docker run --rm -v "$PWD/zerofs.toml:/zerofs.toml" \ ghcr.io/barre/zerofs:latest run -c /zerofs.toml ``` -------------------------------- ### Setting up PostgreSQL in the Browser Terminal Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/postgresql-in-the-browser/page.mdx After copying PostgreSQL binaries to the ZeroFS filesystem, set the PATH and LD_LIBRARY_PATH environment variables within the browser terminal. Then, initialize the database, start the PostgreSQL server, and connect using psql. ```bash # From another machine or the file manager, copy a static PostgreSQL # build onto the ZeroFS filesystem, e.g. into /postgres # Then from the terminal tab: export PATH=/mnt/postgres/bin:$PATH export LD_LIBRARY_PATH=/mnt/postgres/lib initdb -D /mnt/pgdata pg_ctl -D /mnt/pgdata -l /mnt/pgdata/logfile start psql -d postgres ``` -------------------------------- ### Python ZeroFS Client Example Source: https://github.com/barre/zerofs/blob/main/zerofs/zerofs-ffi/README.md Demonstrates connecting to a ZeroFS client, writing a file, reading a file, and listing directory contents. Ensure the ZeroFS server is running and accessible via the specified Unix socket. ```python import asyncio, zerofs_ffi as z async def main(): fs = await z.Client.connect("unix:/run/zerofs/9p.sock") await fs.write("/hello.txt", b"hi") print(await fs.read("/hello.txt")) for e in await fs.read_dir("/"): print(e.name, e.metadata.size if e.metadata else "?") await fs.close() asyncio.run(main()) ``` -------------------------------- ### Configure NBD Server in zerofs.toml Source: https://github.com/barre/zerofs/blob/main/README.md Example configuration for enabling the NBD server in the ZeroFS configuration file. Uncomment and adjust addresses or socket paths as needed. ```toml # Configure NBD in zerofs.toml # [servers.nbd] # addresses = ["127.0.0.1:10809"] # unix_socket = "/tmp/zerofs.nbd.sock" # Optional ``` -------------------------------- ### Animate Install Command Typing Source: https://github.com/barre/zerofs/blob/main/website/index.html Simulates a command-line interface typing effect for the install command in the hero section. Only runs if reduced motion is not preferred. ```javascript const motionOK = !matchMedia("(prefers-reduced-motion: reduce)").matches; // install command types itself if (motionOK) { const t = document.querySelector(".hero .typer"); const full = t.textContent; t.style.minWidth = `${full.length}ch`; t.textContent = ""; const tn = document.createTextNode(""); const caret = document.createElement("i"); caret.className = "caret"; t.append(tn, caret); let n = 0; const tick = () => { tn.nodeValue = full.slice(0, ++n); if (n < full.length) setTimeout(tick, 16 + Math.random() * 26); }; setTimeout(tick, 1350); } ``` -------------------------------- ### Python Client Example Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/client-libraries/page.mdx Demonstrates basic filesystem operations using the Python client library, including directory creation, file writing, reading, metadata retrieval, positioned I/O, directory streaming, renaming, and removal. Ensure the client is connected to a running ZeroFS server. ```python import zerofs_client async with await zerofs_client.Client.connect("unix:/run/zerofs/9p.sock") as fs: await fs.create_dir_all("/demo", 0o755) await fs.write("/demo/notes.txt", b"hello from zerofs") data = await fs.read("/demo/notes.txt") meta = await fs.metadata("/demo/notes.txt") print(len(data), meta.size, meta.is_file()) # A File handle for positioned I/O. async with await fs.open("/demo/data.bin", zerofs_client.OpenOptions(write=True, create=True)) as f: await f.write_at(0, b"\x00" * 4096) await f.sync_all() # Stream the directory entry by entry. async with await fs.open_dir("/demo") as d: async for entry in d: print(entry.name) await fs.rename("/demo/notes.txt", "/demo/notes.md") await fs.remove_file("/demo/notes.md") await fs.remove_dir_all("/demo") ``` -------------------------------- ### Run ZeroFS Read-Only Instance Source: https://github.com/barre/zerofs/blob/main/README.md Start a ZeroFS instance in read-only mode using a configuration file. ```bash zerofs run -c zerofs.toml --read-only ``` -------------------------------- ### Start ZeroFS Writer Without Compaction Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/standalone-compactor/page.mdx Use this command to start the main ZeroFS server while disabling its built-in compactor. This is the first step in setting up a standalone compactor. ```bash zerofs run -c zerofs.toml --no-compactor ``` -------------------------------- ### systemd Environment Variables Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/configuration/page.mdx Example environment file for systemd service substitution. Sensitive information should be protected. ```bash ZEROFS_PASSWORD=your-secure-password AWS_ACCESS_KEY_ID=your-key AWS_SECRET_ACCESS_KEY=your-secret ``` -------------------------------- ### ZeroFS Data Layout Example (SlateDB) Source: https://github.com/barre/zerofs/blob/main/README.md Shows the key-value store layout used by ZeroFS within SlateDB, demonstrating how inodes, directory entries, and file data chunks are organized. ```text Key-Value Store: ├── inode:0 → {type: directory, ...} ├── direntry:0/"file1.txt" → inode 1 ├── inode:1 → {type: file, size: 1024, ...} ├── chunk:1/0 → [first 32K of file data] ├── chunk:1/1 → [second 32K of file data] └── next_inode_id → 2 ``` -------------------------------- ### Stream Stats with grpcurl Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/monitoring/page.mdx This example demonstrates how to stream statistics from the ZeroFS server using `grpcurl`. Ensure the `admin.proto` file is available in the specified import path. The command streams `StatsSnapshot` messages at a 1-second interval. ```bash # admin.proto lives at zerofs/proto/admin.proto in the ZeroFS repository grpcurl -plaintext \ -import-path zerofs/proto -proto admin.proto \ -d '{"interval_ms": 1000}' \ 127.0.0.1:7000 zerofs.admin.AdminService/StreamStats ``` -------------------------------- ### ZeroFS Configuration for RPC Server Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/checkpoints/page.mdx Example TOML configuration to enable the ZeroFS RPC server, which is required for checkpoint management commands. ```toml [servers.rpc] addresses = ["127.0.0.1:7000"] unix_socket = "/tmp/zerofs.rpc.sock" ``` -------------------------------- ### GCS Configuration with GOOGLE_APPLICATION_CREDENTIALS Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/configuration/page.mdx Environment variable setup for GCS authentication using GOOGLE_APPLICATION_CREDENTIALS, followed by the command to run ZeroFS. ```bash export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json" zerofs run -c zerofs.toml ``` -------------------------------- ### ZeroFS Start with Environment Variables Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/quickstart/page.mdx Override configuration values using environment variables. The configuration file can reference these variables. ```bash # You can override config values with environment variables export ZEROFS_PASSWORD='override-password' export AWS_ACCESS_KEY_ID='override-key' export AWS_SECRET_ACCESS_KEY='override-secret' # Config file can reference environment variables # [storage] # encryption_password = "${ZEROFS_PASSWORD}" zerofs run -c zerofs.toml ``` -------------------------------- ### Basic ZeroFS TOML Configuration Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/configuration/page.mdx Example of a basic ZeroFS configuration file structure, including required cache, storage, and server settings. Unknown keys are rejected, and sizes are interpreted as decimal gigabytes. ```toml # Cache configuration (required) [cache] dir = "/var/cache/zerofs" # Directory for caching data disk_size_gb = 10.0 # Maximum disk cache size in GB memory_size_gb = 2.0 # Memory cache size in GB (optional) # Storage configuration (required) [storage] url = "s3://bucket/path" # Storage backend URL encryption_password = "your-secure-password" # Encryption password # Server configuration (required table, every subsection optional) [servers.nfs] addresses = ["127.0.0.1:2049"] ``` -------------------------------- ### Standalone Compactor Configuration Example Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/standalone-compactor/page.mdx This TOML snippet shows the relevant configuration sections for a standalone compactor. Ensure the storage URL and credentials are correct, and adjust LSM settings as needed. ```toml [storage] url = "s3://my-bucket/zerofs-data" encryption_password = "your-password" [aws] access_key_id = "..." secret_access_key = "..." [lsm] max_concurrent_compactions = 8 # Number of parallel compaction jobs ``` -------------------------------- ### TypeScript/Node Client Example Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/client-libraries/page.mdx Demonstrates basic filesystem operations using the TypeScript/Node.js client library, including directory creation, file writing, reading, metadata retrieval, positioned I/O, directory streaming, renaming, and removal. Ensure the client is connected to a running ZeroFS server. ```typescript import { Client, openOptions, isFile } from "zerofs-client"; await using fs = await Client.connect("unix:/run/zerofs/9p.sock"); await fs.createDirAll("/demo", 0o755); await fs.write("/demo/notes.txt", new TextEncoder().encode("hello from zerofs")); const data = await fs.read("/demo/notes.txt"); const meta = await fs.metadata("/demo/notes.txt"); console.log(data.length, meta.size, isFile(meta)); // A File handle for positioned I/O. await using f = await fs.open("/demo/data.bin", openOptions({ write: true, create: true })); await f.writeAt(0n, new Uint8Array(4096)); await f.syncAll(); // Stream the directory entry by entry. await using dir = await fs.openDir("/demo"); for await (const entry of dir) console.log(entry.name); await fs.rename("/demo/notes.txt", "/demo/notes.md"); await fs.removeFile("/demo/notes.md"); await fs.removeDirAll("/demo"); ``` -------------------------------- ### Running ZeroFS Instances Across Multiple Regions Source: https://github.com/barre/zerofs/blob/main/README.md This bash script demonstrates how to start ZeroFS instances in different regions (US East, EU West, Asia Pacific) using their respective configuration files. ```bash # Machine 1 - US East (10.0.1.5) zerofs run -c zerofs-us-east.toml # Machine 2 - EU West (10.0.2.5): same config with # url = "s3://zerofs-eu-west/db", default_region = "eu-west-1", addresses on 10.0.2.5 zerofs run -c zerofs-eu-west.toml # Machine 3 - Asia Pacific (10.0.3.5): same config with # url = "s3://zerofs-ap-southeast/db", default_region = "ap-southeast-1", addresses on 10.0.3.5 zerofs run -c zerofs-asia.toml ``` -------------------------------- ### Generate Bindings for Multiple Languages Source: https://github.com/barre/zerofs/blob/main/zerofs/zerofs-ffi/README.md This command sequence shows how to build the zerofs-ffi crate and then generate bindings for Python, Node.js/TypeScript, and Go. Ensure uniffi-bindgen tools are installed beforehand. ```sh cargo build -p zerofs-ffi ./zerofs-ffi/bindings/generate.sh python out/python cargo install uniffi-bindgen-node-js # once, for the Node/TypeScript package ./zerofs-ffi/bindings/generate.sh typescript out/ts # bundles the lib + facade cargo install \ --git https://github.com/NordSecurity/uniffi-bindgen-go \ --tag v0.7.0+v0.31.0 # once, for the Go package ./zerofs-ffi/bindings/generate.sh go out/go # generated pkg + facade ``` -------------------------------- ### Writer and Replicas Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/read-replicas/page.mdx Start a read-write ZeroFS instance and any number of read-only instances using the same configuration file. Replicas require read access to the object store and the same encryption password as the writer. ```bash # Read-write instance (exactly one) zerofs run -c zerofs.toml # Read-only instances (any number) zerofs run -c zerofs.toml --read-only ``` -------------------------------- ### Set up Geo-Distributed NBD Devices and ZFS Pool Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/nbd-devices/page.mdx Create NBD devices on both geo-distributed machines, then connect to them from a client machine to form a mirrored ZFS pool across continents. This setup ensures availability even if one region fails. ```bash # Create devices on both machines mount -t nfs 10.0.1.5:/ /mnt/zerofs truncate -s 100G /mnt/zerofs/.nbd/storage umount /mnt/zerofs mount -t nfs 10.0.2.5:/ /mnt/zerofs truncate -s 100G /mnt/zerofs/.nbd/storage umount /mnt/zerofs # From a client machine, connect to both regions with optimal settings nbd-client 10.0.1.5 10809 /dev/nbd0 -N storage -persist -timeout 600 -connections 8 nbd-client 10.0.2.5 10809 /dev/nbd1 -N storage -persist -timeout 600 -connections 8 # Create mirrored pool across continents zpool create global-pool mirror /dev/nbd0 /dev/nbd1 ``` -------------------------------- ### Install Command Block Styling Source: https://github.com/barre/zerofs/blob/main/website/index.html Styles the '.install' container for command-line interface elements, using flexbox for alignment and spacing. Includes styling for the '.cmd' element itself, featuring borders, background, and box-shadow. ```css .install { margin-top: 44px; display: flex; flex-wrap: wrap; align-items: center; justify-content: center; gap: 14px; } .cmd { display: flex; align-items: center; gap: 14px; border: 1px solid var(--line-bright); border-radius: var(--r); background: rgba(255,255,255,0.045); box-shadow: inset 0 1px 0 rgba(255,255,255,0.06), 0 8px 32px rgba(0,0,0,0.45); padding: 14px 18px; font-family: var(--mono); font-size: 14px; } ``` -------------------------------- ### Run ZeroFS with Environment Variable Configuration Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/encryption/page.mdx Starts the ZeroFS service using a configuration file that references an environment variable for the encryption password. This allows for dynamic password management without hardcoding. ```bash zerofs run --config zerofs.toml ``` -------------------------------- ### Go Client Example Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/client-libraries/page.mdx Demonstrates basic filesystem operations using the Go client library, including directory creation, file writing, reading, metadata retrieval, and directory streaming. This requires building the `zerofs-ffi` cdylib and configuring cgo. Ensure the client is connected to a running ZeroFS server. ```go import ( "log" zerofs "github.com/Barre/ZeroFS/zerofs/zerofs-ffi/bindings/go" ) fs, err := zerofs.ClientConnect("unix:/run/zerofs/9p.sock") if err != nil { log.Fatal(err) } deffer fs.Close() must := func(err error) { if err != nil { log.Fatal(err) } } must(fs.CreateDirAll("/demo", 0o755)) must(fs.Write("/demo/notes.txt", []byte("hello from zerofs"))) data, err := fs.Read("/demo/notes.txt") must(err) meta, err := fs.Metadata("/demo/notes.txt") must(err) log.Printf("%d bytes, size=%d, isFile=%v", len(data), meta.Size, meta.IsFile()) // Stream the directory with a range-over-func iterator. dir, err := fs.OpenDir("/demo") must(err) deffer dir.Close() for entry, err := range dir.Entries() { must(err) log.Println(entry.Name) } ``` -------------------------------- ### Install ZeroFS Go Module Source: https://github.com/barre/zerofs/blob/main/zerofs/zerofs-ffi/bindings/go/README.md Add the ZeroFS Go module to your project using go get. Replace vX.Y.Z with the desired version tag. ```sh go get github.com/Barre/ZeroFS/zerofs/zerofs-ffi/bindings/go@vX.Y.Z ``` -------------------------------- ### Initialize and Run ZeroFS with Configuration Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/troubleshooting/page.mdx Use 'zerofs init' to create a new configuration file. Specify a config file using '--config' or '-c' when running ZeroFS. ```bash # Error: Failed to load config # Solution: Create or specify config file # Initialize a new config zerofs init # Run with specific config file zerofs run --config /path/to/zerofs.toml # Or use short form zerofs run -c zerofs.toml ``` -------------------------------- ### Build Python Wheel with Maturin Source: https://github.com/barre/zerofs/blob/main/zerofs/zerofs-ffi/README.md Install necessary build tools and then build the Python wheel for the ZeroFS client. Finally, install the built wheel. ```sh pip install "maturin>=1.10,<2.0" "uniffi-bindgen==0.31.0" (cd zerofs-ffi/bindings/python && maturin build --profile wheel) pip install target/wheels/zerofs_client-*.whl ``` -------------------------------- ### Create and Boot VM using NBD Device Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/nbd-devices/page.mdx Create a raw disk image for a virtual machine on an NBD device and then boot a QEMU VM using this device. The cache option is set to writeback for performance. ```bash # Create VM disk qemu-img create -f raw /dev/nbd0 50G # Boot VM using NBD device qemu-system-x86_64 \ -drive file=/dev/nbd0,format=raw,cache=writeback \ -m 4G -enable-kvm ``` -------------------------------- ### Check ZFS Pool Setup Source: https://github.com/barre/zerofs/blob/main/README.md Verify the ZFS pool configuration and I/O statistics, including cache devices. Use this command to confirm the L2ARC setup. ```bash zpool iostat -v mypool ``` -------------------------------- ### Initialize ZeroFS Configuration Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/quickstart/page.mdx Use the `zerofs init` command to create a configuration template. You can write it to a file or to standard output. ```bash # Write a config template to ./zerofs.toml # (overwrites an existing file) zerofs init # Or write the template to stdout zerofs init - > /path/to/zerofs.toml ``` -------------------------------- ### Initialize ZeroFS Configuration Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/configuration/page.mdx Use the `init` command to create a template TOML configuration file for ZeroFS. ```bash zerofs init ``` -------------------------------- ### Get ZeroFS Checkpoint Info Source: https://github.com/barre/zerofs/blob/main/README.md Retrieve detailed information about a specific named checkpoint. ```bash zerofs checkpoint info -c zerofs.toml my-snapshot ``` -------------------------------- ### Connect to NBD Device in ZeroFS Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/troubleshooting/page.mdx Ensure NBD is configured in 'zerofs.toml', check if the port is listening, load the NBD module, create a device file, and then connect using 'nbd-client'. ```bash # Error: nbd-client connection failed # Solution: Check NBD configuration and availability # Verify NBD is configured in zerofs.toml: # [servers.nbd] # addresses = ["127.0.0.1:10809"] # Check if port is listening ss -tlnp | grep 10809 # Ensure NBD module is loaded (Linux) sudo modprobe nbd lsmod | grep nbd # Create NBD device file sudo mkdir -p /mnt/zerofs/.nbd sudo truncate -s 10G /mnt/zerofs/.nbd/my-device # Connect to the device sudo nbd-client 127.0.0.1 10809 /dev/nbd0 -N my-device ``` -------------------------------- ### GitHub Action Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/quickstart/page.mdx Example of using the ZeroFS GitHub Action in a workflow. Requires setting the object store URL and encryption password. ```yaml # Use in your workflow - uses: Barre/zerofs@main with: object-store-url: 's3://bucket/path' encryption-password: ${{ secrets.ZEROFS_PASSWORD }} ``` -------------------------------- ### ZeroFS Web UI Configuration Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/postgresql-in-the-browser/page.mdx Enable the Web UI by adding this configuration to your ZeroFS setup. This allows access to the terminal tab. ```toml [servers.webui] addresses = ["127.0.0.1:8080"] uid = 1000 gid = 1000 ``` -------------------------------- ### Configure Prometheus Endpoint Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/prometheus/page.mdx Add a [prometheus] section to your configuration file to start an HTTP server for metrics. Multiple bind addresses are supported. ```toml [prometheus] addresses = ["127.0.0.1:9091"] ``` ```toml [prometheus] addresses = ["127.0.0.1:9091", "0.0.0.0:9091"] ``` -------------------------------- ### Create & Connect NBD Device Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/quickstart/page.mdx This snippet demonstrates how to create a new NBD block device file, connect it to the NBD server, format it with ext4, and mount it. Ensure NBD is configured in your TOML file and the filesystem is mounted. ```bash # First, ensure NBD is configured in your TOML: # [servers.nbd] # addresses = ["127.0.0.1:10809"] # Mount the filesystem (if not already mounted) sudo mount -t nfs -o vers=3,async,nolock 127.0.0.1:/ /mnt/zerofs # Create a block device file sudo mkdir -p /mnt/zerofs/.nbd sudo truncate -s 10G /mnt/zerofs/.nbd/my-device # Install nbd-client if needed sudo apt-get install nbd-client # Debian/Ubuntu sudo yum install nbd # RHEL/CentOS # Connect to the NBD device sudo nbd-client 127.0.0.1 10809 /dev/nbd0 -N my-device # Create filesystem sudo mkfs.ext4 /dev/nbd0 # Mount the block device sudo mkdir -p /mnt/nbd0 sudo mount /dev/nbd0 /mnt/nbd0 ``` -------------------------------- ### Get ZeroFS Checkpoint Info and Delete Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/checkpoints/page.mdx Retrieve details about a specific checkpoint or delete an old checkpoint. The RPC server must be running. ```bash # Get details about a specific checkpoint zerofs checkpoint info -c zerofs.toml my-snapshot # Delete a checkpoint zerofs checkpoint delete -c zerofs.toml old-snapshot ``` -------------------------------- ### Build Release Binary with Web UI Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/quickstart/page.mdx Use this command to build a release binary that includes the Web UI. This process first builds the frontend assets and then compiles the Rust code with the 'webui' feature enabled. ```bash make build-release ``` -------------------------------- ### Run ZeroFS from a Checkpoint (Read-Only) Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/checkpoints/page.mdx Starts the ZeroFS filesystem in read-only mode from a specified checkpoint. This is useful for inspecting historical data without modifying it. ```bash # Start ZeroFS from a checkpoint (read-only) zerofs run -c zerofs.toml --checkpoint my-snapshot # Mount and access historical data mount -t nfs 127.0.0.1:/ /mnt/snapshot ls /mnt/snapshot # See data as it was at checkpoint time ``` -------------------------------- ### S3FS Data Layout Example Source: https://github.com/barre/zerofs/blob/main/README.md Illustrates the typical object storage layout for S3FS, where files are single objects and directories are marked by zero-byte objects. ```text s3://bucket/ ├── file1.txt (complete file as single object) ├── dir1/ (zero-byte marker) ├── dir1/file2.txt (complete file) └── .metadata/ (optional metadata storage) ``` -------------------------------- ### Build ZeroFS from Source Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/troubleshooting/page.mdx If your CPU lacks AVX2 support, build ZeroFS from source to target a baseline x86-64 architecture. This bypasses the need for advanced CPU instruction sets. ```bash git clone https://github.com/Barre/ZeroFS cd ZeroFS/zerofs cargo build --release ``` -------------------------------- ### ZeroFS Configuration for Encryption Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/encryption/page.mdx Example TOML configuration for ZeroFS, specifying the storage URL and the encryption password. The DEK is generated by ZeroFS on its first run. ```toml # ZeroFS generates a new DEK on first run [storage] url = "s3://bucket/path" encryption_password = "strong-password" ``` -------------------------------- ### Connect NBD Device for Database Storage Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/nbd-devices/page.mdx Connect to a pre-created NBD device for database storage, format it, mount it, and initialize a PostgreSQL database. Ensure the NBD client is configured with appropriate settings like block size and connection parameters. ```bash # Connect the database device created in .nbd/ nbd-client 127.0.0.1 10809 /dev/nbd0 \ -N database \ -persist \ -timeout 600 \ -connections 4 \ -block-size 4096 mkfs.ext4 /dev/nbd0 mount /dev/nbd0 /var/lib/postgresql # Initialize database sudo -u postgres initdb -D /var/lib/postgresql/16/main ``` -------------------------------- ### ZeroFS Cache Configuration Source: https://github.com/barre/zerofs/blob/main/documentation/src/app/caching/page.mdx Example TOML configuration for ZeroFS cache settings, specifying the cache directory, total disk size, and decoded-block memory size. ```toml [cache] dir = "/var/cache/zerofs" disk_size_gb = 10.0 memory_size_gb = 2.0 # optional ``` -------------------------------- ### Example ZeroFS TOML Configuration Source: https://github.com/barre/zerofs/blob/main/README.md This TOML configuration illustrates settings for cache, storage, filesystem limits, compression, various server protocols (NFS, 9P, NBD), LSM tuning, and cloud credentials. Environment variables are used for sensitive information like passwords and access keys. ```toml [cache] dir = "${HOME}/.cache/zerofs" disk_size_gb = 10.0 memory_size_gb = 1.0 # Optional, defaults to 0.25 [storage] url = "s3://my-bucket/zerofs-data" encryption_password = "${ZEROFS_PASSWORD}" [filesystem] max_size_gb = 100.0 # Optional: limit filesystem to 100 GB (defaults to 16 EiB) compression = "zstd-3" # Optional: "zstd-{1-22}" (default: "zstd-3") or "lz4" [servers.nfs] addresses = ["127.0.0.1:2049"] # Can specify multiple addresses [servers.ninep] addresses = ["127.0.0.1:5564"] unix_socket = "/tmp/zerofs.9p.sock" # Optional [servers.nbd] addresses = ["127.0.0.1:10809"] unix_socket = "/tmp/zerofs.nbd.sock" # Optional [lsm] wal_enabled = false # WAL reduces compaction churn from frequent fsyncs (default: false) # Enable for fsync-heavy workloads to reduce compaction overhead [aws] access_key_id = "${AWS_ACCESS_KEY_ID}" secret_access_key = "${AWS_SECRET_ACCESS_KEY}" # endpoint = "https://s3.us-east-1.amazonaws.com" # For S3-compatible services # default_region = "us-east-1" # allow_http = "true" # For non-HTTPS endpoints # conditional_put = "redis://localhost:6379" # For S3-compatible stores without conditional put support # [azure] # storage_account_name = "${AZURE_STORAGE_ACCOUNT_NAME}" # storage_account_key = "${AZURE_STORAGE_ACCOUNT_KEY}" ```