Twelf (bnjjj/twelf)

Twelf

https://github.com/bnjjj/twelf
Admin
Twelf is a configuration solution for Rust with 12-Factor support, enabling multiple layers to read...

Tokens:7,458
Snippets:50
Trust Score:7.2
Update:3 weeks ago
Show doc for...
Context Summary (auto-generated)
Raw
# Twelf

Twelf is a configuration solution for Rust with 12-Factor support, designed around the concept of configuration layers. It enables developers to build application configurations from multiple sources including JSON, TOML, YAML, DHALL, INI files, environment variables, and command-line arguments. The library uses a simple proc-macro `#[config]` attribute to transform any struct into a layered configuration structure.

The core design principle is layered priority: later layers in the configuration chain override values from earlier layers, allowing flexible configuration strategies where defaults can be set in files and overridden by environment variables or CLI arguments. Twelf also supports environment variable expansion within configuration files, nested structs with serde's `#[serde(flatten)]`, and complex types like `HashMap` and `Vec` from environment variables.

## Configuration Layer Enum

The `Layer` enum defines the available configuration sources. Each layer variant specifies how to load configuration values, and layers are processed in order with later layers overriding earlier ones.

```rust
use twelf::{config, Layer};

#[config]
#[derive(Debug, Default)]
struct AppConfig {
    db_host: String,
    db_port: usize,
    debug: bool,
}

fn main() -> Result<(), twelf::Error> {
    // Layers are processed in order: JSON first, then environment variables override
    let config = AppConfig::with_layers(&[
        Layer::Json("config.json".into()),           // Base configuration from file
        Layer::Env(Some("APP_".to_string())),        // Override with APP_DB_HOST, APP_DB_PORT, etc.
    ])?;

    println!("Database: {}:{}", config.db_host, config.db_port);
    Ok(())
}
```

## with_layers Method

The `with_layers` method is automatically generated by the `#[config]` attribute. It takes a slice of `Layer` values and returns a `Result<Self, twelf::Error>` containing the fully merged configuration.

```rust
use twelf::{config, Layer};

#[config]
#[derive(Debug, Default)]
struct ServerConfig {
    host: String,
    port: usize,
    #[serde(default = "default_timeout")]
    timeout_ms: usize,
}

fn default_timeout() -> usize { 30000 }

fn main() -> Result<(), twelf::Error> {
    // Layer order matters: later layers override earlier ones
    let config = ServerConfig::with_layers(&[
        Layer::Toml("server.toml".into()),   // Priority 1: lowest
        Layer::Yaml("server.yaml".into()),   // Priority 2
        Layer::Json("server.json".into()),   // Priority 3
        Layer::Env(None),                    // Priority 4: highest (no prefix)
    ])?;

    // If HOST=production.example.com is set, it overrides file values
    println!("Server: {}:{} (timeout: {}ms)", config.host, config.port, config.timeout_ms);
    Ok(())
}
```

## Environment Variable Layer

The `Layer::Env` variant reads configuration from environment variables. It supports an optional prefix to namespace variables and handles complex types like `HashMap` and `Vec` using comma-separated formats.

```rust
use std::collections::HashMap;
use twelf::{config, Layer};

#[config]
#[derive(Debug, Default)]
struct Config {
    database_url: String,
    allowed_origins: Vec<String>,
    feature_flags: HashMap<String, String>,
}

fn main() -> Result<(), twelf::Error> {
    // Set environment variables
    std::env::set_var("APP_DATABASE_URL", "postgres://localhost/mydb");
    std::env::set_var("APP_ALLOWED_ORIGINS", "http://localhost:3000,https://example.com");
    std::env::set_var("APP_FEATURE_FLAGS", "dark_mode=enabled,beta_api=disabled");

    let config = Config::with_layers(&[
        Layer::Env(Some("APP_".to_string())),  // Reads APP_* variables
    ])?;

    // allowed_origins: ["http://localhost:3000", "https://example.com"]
    // feature_flags: {"dark_mode": "enabled", "beta_api": "disabled"}
    println!("DB: {}", config.database_url);
    println!("Origins: {:?}", config.allowed_origins);
    println!("Flags: {:?}", config.feature_flags);
    Ok(())
}
```

## JSON Configuration Layer

The `Layer::Json` variant loads configuration from a JSON file. It supports environment variable expansion with the `shellexpand` feature.

```rust
use twelf::{config, Layer};

#[config]
#[derive(Debug, Default)]
struct Config {
    db_host: String,
    threads: usize,
}

fn main() -> Result<(), twelf::Error> {
    // config.json contents:
    // {
    //     "db_host": "localhost",
    //     "threads": 5
    // }

    let config = Config::with_layers(&[
        Layer::Json("./config.json".into()),
        Layer::Env(Some("APP_".to_string())),  // APP_THREADS=10 would override
    ])?;

    println!("Host: {}, Threads: {}", config.db_host, config.threads);
    Ok(())
}
```

## TOML Configuration Layer

The `Layer::Toml` variant loads configuration from TOML files with support for environment variable expansion using `${VAR:-default}` syntax.

```rust
use std::collections::HashMap;
use twelf::{config, Layer};

#[config]
#[derive(Debug, Default)]
pub struct Settings {
    pub server_url: Option<String>,
    pub api_endpoint: Option<String>,
    pub profiles: HashMap<String, HashMap<String, String>>,
}

fn main() -> Result<(), twelf::Error> {
    // config.toml contents:
    // server_url = "127.0.0.1:3000"
    // api_endpoint = "http://localhost:3000/api"
    //
    // [profiles.s3]
    // type = "s3"
    // bucket = "test"
    // region = "${AWS_REGION:-us-east-1}"  # Environment variable expansion

    let config = Settings::with_layers(&[
        Layer::Toml("./config.toml".into()),
        Layer::Env(Some("APP_".to_string())),
    ])?;

    println!("Server: {:?}", config.server_url);
    println!("Profiles: {:?}", config.profiles);
    Ok(())
}
```

## Clap Command Line Layer

The `Layer::Clap` variant integrates with the clap crate for command-line argument parsing. The `#[config]` macro automatically generates a `clap_args()` method that creates clap arguments from struct fields.

```rust
use clap_rs as clap;
use twelf::{config, Layer};

#[config]
#[derive(Debug, Default)]
struct Config {
    /// Database host address
    db_host: String,
    /// Number of worker threads
    threads: usize,
    /// Enable verbose output
    verbose: bool,
}

fn main() -> Result<(), twelf::Error> {
    // Auto-generated clap arguments from struct fields
    let app = clap::Command::new("myapp")
        .args(&Config::clap_args());

    let matches = app.get_matches();

    // CLI args override environment variables which override file config
    let config = Config::with_layers(&[
        Layer::Json("config.json".into()),
        Layer::Env(Some("APP_".to_string())),
        Layer::Clap(matches),
    ])?;

    // Run: cargo run -- --db-host localhost --threads 8 --verbose
    println!("Config: {:?}", config);
    Ok(())
}
```

## Clap Derive Integration

Twelf works with clap's derive macros for more control over CLI argument definitions, including custom help text, default values, and short flags.

```rust
use clap::{CommandFactory, Parser};
use clap_rs as clap;
use twelf::{config, Layer};

#[config]
#[derive(Parser, Debug, Default)]
#[clap(author, version, about)]
struct Config {
    #[clap(long, help = "Database connection string")]
    db_host: String,

    #[clap(long, short, help = "Number of threads", default_value_t = 4)]
    threads: usize,

    #[clap(long, short, help = "Enable verbose mode")]
    verbose: bool,
}

fn main() -> Result<(), twelf::Error> {
    let matches = Config::command().get_matches();

    let config = Config::with_layers(&[
        Layer::Env(Some("APP_".to_string())),
        Layer::Clap(matches),
    ])?;

    // Clap defaults are respected but can be overridden by earlier layers
    println!("Threads: {} (default was 4)", config.threads);
    Ok(())
}
```

## Nested Structs with Serde Flatten

Twelf supports nested configuration structures using serde's `#[serde(flatten)]` attribute, allowing modular configuration organization.

```rust
use std::collections::HashMap;
use twelf::reexports::serde::{Deserialize, Serialize};
use twelf::{config, Layer};

#[derive(Debug, Default, Deserialize, Serialize)]
struct DatabaseConfig {
    host: String,
    port: usize,
}

#[derive(Debug, Default, Deserialize, Serialize)]
struct CacheConfig {
    redis_url: String,
}

#[config]
#[derive(Debug, Default)]
struct AppConfig {
    app_name: String,
    #[serde(flatten)]
    database: DatabaseConfig,
    #[serde(flatten)]
    cache: CacheConfig,
    labels: HashMap<String, String>,
}

fn main() -> Result<(), twelf::Error> {
    // Environment variables map to flattened fields
    std::env::set_var("APP_HOST", "db.example.com");
    std::env::set_var("APP_PORT", "5432");
    std::env::set_var("APP_REDIS_URL", "redis://localhost:6379");
    std::env::set_var("APP_LABELS", "env=prod,region=us-west");
    std::env::set_var("APP_APP_NAME", "MyService");

    let config = AppConfig::with_layers(&[
        Layer::Env(Some("APP_".to_string())),
    ])?;

    println!("Database: {}:{}", config.database.host, config.database.port);
    println!("Cache: {}", config.cache.redis_url);
    Ok(())
}
```

## Default Trait Layer

The `Layer::DefaultTrait` variant uses Rust's `Default` trait to provide fallback values. Requires the `default_trait` feature.

```rust
use twelf::{config, Layer};

#[config]
#[derive(Debug)]
struct Config {
    host: String,
    port: usize,
    max_connections: usize,
}

impl Default for Config {
    fn default() -> Self {
        Self {
            host: "localhost".to_owned(),
            port: 8080,
            max_connections: 100,
        }
    }
}

fn main() -> Result<(), twelf::Error> {
    std::env::set_var("PORT", "3000");  // Only override port

    // DefaultTrait provides base values, Env overrides specific fields
    let config = Config::with_layers(&[
        Layer::DefaultTrait,
        Layer::Env(None),
    ])?;

    // host: "localhost" (from Default)
    // port: 3000 (from Env)
    // max_connections: 100 (from Default)
    println!("Server: {}:{} (max: {})", config.host, config.port, config.max_connections);
    Ok(())
}
```

## Custom Function Layer

The `Layer::CustomFn` variant allows programmatic configuration through closures that return `serde_json::Value`. Requires the `custom_fn` feature.

```rust
use twelf::{config, Layer};
use serde_json::json;

#[config]
#[derive(Debug, Default)]
struct Config {
    api_key: String,
    service_url: String,
    retry_count: usize,
}

fn main() -> Result<(), twelf::Error> {
    // Custom function to load config from any source (vault, remote API, etc.)
    let load_secrets = || {
        // In production, this might call HashiCorp Vault, AWS Secrets Manager, etc.
        json!({
            "api_key": "secret-key-from-vault",
            "service_url": "https://api.example.com",
            "retry_count": 3
        })
    };

    std::env::set_var("RETRY_COUNT", "5");  // Override retry count

    let config = Config::with_layers(&[
        Layer::CustomFn(load_secrets.into()),
        Layer::Env(None),
    ])?;

    // api_key and service_url from CustomFn, retry_count from Env
    println!("API: {} (retries: {})", config.service_url, config.retry_count);
    Ok(())
}
```

## Serde Default Attribute

Use serde's `#[serde(default)]` attribute to provide default values for individual fields without implementing the full `Default` trait.

```rust
use twelf::{config, Layer};

fn default_port() -> usize { 8080 }
fn default_log_level() -> String { "info".to_owned() }

#[config]
#[derive(Debug, Default)]
struct Config {
    host: String,

    #[serde(default = "default_port")]
    port: usize,

    #[serde(default = "default_log_level")]
    log_level: String,

    #[serde(default)]  // Uses type's Default (empty string)
    optional_field: String,
}

fn main() -> Result<(), twelf::Error> {
    std::env::set_var("HOST", "0.0.0.0");
    // PORT and LOG_LEVEL not set - will use serde defaults

    let config = Config::with_layers(&[
        Layer::Env(None),
    ])?;

    // host: "0.0.0.0", port: 8080, log_level: "info"
    println!("{}:{} [{}]", config.host, config.port, config.log_level);
    Ok(())
}
```

## Error Handling

Twelf provides a comprehensive `Error` enum for handling configuration loading failures including I/O errors, parsing errors, and deserialization errors.

```rust
use twelf::{config, Layer, Error};

#[config]
#[derive(Debug, Default)]
struct Config {
    required_field: String,
    port: usize,
}

fn main() {
    let result = Config::with_layers(&[
        Layer::Json("nonexistent.json".into()),
        Layer::Env(None),
    ]);

    match result {
        Ok(config) => println!("Loaded: {:?}", config),
        Err(Error::Io(e)) => eprintln!("File error: {}", e),
        Err(Error::Json(e)) => eprintln!("JSON parse error: {}", e),
        Err(Error::Deserialize(msg)) => eprintln!("Config error: {}", msg),
        Err(e) => eprintln!("Other error: {}", e),
    }
}
```

## Feature Flags Configuration

Twelf is modular through Cargo features. Enable only the layers you need to minimize dependencies.

```toml
# Cargo.toml

# Minimal: only JSON and environment variables
[dependencies]
twelf = { version = "0.15", default-features = false, features = ["json", "env"] }

# Full configuration support
[dependencies]
twelf = { version = "0.15", features = ["json", "toml", "yaml", "env", "clap", "shellexpand"] }

# With default trait and custom function support
[dependencies]
twelf = { version = "0.15", features = ["default_trait", "custom_fn"] }

# Available features:
# - env: Environment variable support (default)
# - clap: Command-line argument support (default)
# - shellexpand: Environment variable expansion in files (default)
# - json: JSON file support
# - toml: TOML file support
# - yaml: YAML file support
# - ini: INI file support
# - dhall: Dhall file support
# - default_trait: Layer::DefaultTrait support
# - custom_fn: Layer::CustomFn support
```

Twelf's primary use case is building 12-Factor compliant applications where configuration comes from multiple sources with clear precedence. Common patterns include: using file-based configuration for development defaults, environment variables for container deployments, and CLI arguments for one-off overrides. The layered approach makes it easy to provide sensible defaults while allowing runtime customization without code changes.

The library integrates seamlessly with the Rust ecosystem through serde for serialization and clap for CLI parsing. For production deployments, teams typically combine `Layer::Json` or `Layer::Toml` for base configuration, `Layer::Env` for secrets and deployment-specific values, and optionally `Layer::Clap` for debugging or administrative overrides. The `custom_fn` feature enables integration with secret managers like HashiCorp Vault or AWS Secrets Manager for sensitive configuration values.