### Running the Full Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/quick-reference.md Command-line execution of the full Draccus example. Demonstrates how to specify configuration values via a YAML file and command-line overrides, including nested parameters. ```bash python train.py \ --config_path config.yaml \ --batch_size 64 \ --model.type bert \ --model.num_layers 24 ``` -------------------------------- ### Install Pyrallis Source: https://github.com/marin-community/draccus/blob/main/docs/index.md Install the pyrallis library using pip. ```bash pip install pyrallis ``` -------------------------------- ### Command-line Argument Parsing Example Source: https://github.com/marin-community/draccus/blob/main/README.md This example shows how to pass arguments via the command line, overriding default values or values from a config file. The output reflects the provided arguments. ```console $ python train_model.py --config_path=some_config.yaml --exp_name=my_first_exp Training my_first_exp with 42 workers... ``` -------------------------------- ### Basic Draccus Example with Dataclass Source: https://github.com/marin-community/draccus/blob/main/README.md Use this snippet to parse command-line arguments or YAML configurations directly into a Python dataclass. Ensure draccus is installed and imported. ```python from dataclasses import dataclass import draccus @dataclass class TrainConfig: """Training Config for Machine Learning""" workers: int = 8 # The number of workers for training exp_name: str = 'default_exp' # The experiment name @draccus.wrap() def main(cfg: TrainConfig): print(f"Training {cfg.exp_name} with {cfg.workers} workers...") if __name__ == "__main__": main() ``` -------------------------------- ### Custom Choice Type Implementation Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/types.md Demonstrates how to implement a custom choice type by defining the required class methods. This example shows a basic structure for custom logic to resolve names to classes, list choices, and get registration names. ```python from draccus.choice_types import ChoiceType from draccus.utils import is_choice_type from dataclasses import dataclass @dataclass class CustomChoiceType: @classmethod def get_choice_class(cls, name: str): # Custom logic to resolve name to class pass @classmethod def get_known_choices(cls): # Custom logic to list choices pass @classmethod def get_choice_name(cls, subcls): # Custom logic to get name for class pass @classmethod def default_choice_name(cls): return None # Check if a type implements the protocol if is_choice_type(CustomChoiceType): print("It's a choice type!") ``` -------------------------------- ### Install Pre-commit Hooks Source: https://github.com/marin-community/draccus/blob/main/CONTRIBUTING.md Installs the pre-commit hooks for the project. These hooks help maintain code style and quality by running checks on each commit. ```bash uvx pre-commit install ``` -------------------------------- ### Full Draccus Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/quick-reference.md A comprehensive example demonstrating nested configurations, choice registries for different model types (GPT, BERT), and a main training configuration. Includes a post-init validation for batch size divisibility by GPU count. ```python from dataclasses import dataclass, field from typing import List import draccus @dataclass class ModelConfig(draccus.ChoiceRegistry): hidden_size: int = 768 @ModelConfig.register_subclass("gpt") @dataclass class GPTConfig(ModelConfig): num_layers: int = 12 @ModelConfig.register_subclass("bert") @dataclass class BertConfig(ModelConfig): num_layers: int = 12 dropout: float = 0.1 @dataclass class TrainingConfig: # Data dataset: str = "wikitext" batch_size: int = 32 # Model model: ModelConfig = GPTConfig() # Training epochs: int = 10 learning_rate: float = 0.001 gpu_ids: List[int] = field(default_factory=lambda: [0, 1]) def __post_init__(self): if self.batch_size % len(self.gpu_ids) != 0: raise ValueError("Batch size must be divisible by GPU count") @draccus.wrap() def main(cfg: TrainingConfig): print(f"Training {cfg.model} on {cfg.dataset}") print(f"Config: {draccus.dump(cfg)}") if __name__ == "__main__": main() ``` -------------------------------- ### Install uv Package Manager Source: https://github.com/marin-community/draccus/blob/main/CONTRIBUTING.md Installs the uv package manager, which is used for Python dependency management in this project. Refer to the uv documentation for alternative installation methods. ```bash pip install uv ``` -------------------------------- ### ParserEnum Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/types.md Demonstrates the usage of ParserEnum, showing how ConfigType extends it to represent different configuration formats like YAML, JSON, and TOML. ```python from draccus.options import ConfigType, ParserEnum # ConfigType extends ParserEnum print(ConfigType.YAML.value) # YAMLParser print(ConfigType.JSON.value) # JSONParser print(ConfigType.TOML.value) # TOMLParser ``` -------------------------------- ### Configuration Merging Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/configuration.md Demonstrates how command-line arguments override values from a configuration file. Ensure the config file is specified via `config_path`. ```yaml # config.yaml workers: 8 exp_name: default_exp model: type: gpt layers: 12 ``` ```bash # Command-line overrides file python app.py --config_path=config.yaml --workers=16 --model.layers=24 # Result: # workers: 16 (from CLI, overrides file) # exp_name: default_exp (from file) # model.type: gpt (from file) # model.layers: 24 (from CLI, overrides file) ``` -------------------------------- ### Quick Start: Parse and Dump Configuration Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/main-exports.md Demonstrates how to parse a configuration from command-line arguments and a file, and then save the configuration to a file. Ensure you have a config.yaml file for parsing. ```python from dataclasses import dataclass import draccus @dataclass class Config: workers: int = 8 name: str = "default" # Parse from command-line and file cfg = draccus.parse(Config, config_path="config.yaml") # Save to file with open("out.yaml", "w") as f: draccus.dump(cfg, f) # Use as decorator @draccus.wrap() def main(cfg: Config): print(f"Config: {cfg}") if __name__ == "__main__": main() ``` -------------------------------- ### Example Configuration for Choice Type Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/choice-registry.md Illustrates how to use the CHOICE_TYPE_KEY in a configuration file to select a specific choice type, such as 'bert'. ```yaml # Configuration using the choice type key model: type: bert # This selects BERTConfig num_layers: 24 hidden_size: 1024 ``` -------------------------------- ### TOML Parser Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-type.md TOML format support example. If a string cannot be parsed as TOML, it is returned as-is. ```toml workers = 8 [model] type = "gpt" layers = 12 ``` -------------------------------- ### Load YAML Configuration Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/parsers.md Shows how to load YAML configuration from a file path, a file object, or directly from a string using YAMLParser.load_config(). Supports !include directives and merge keys. ```python # From file path config = YAMLParser.load_config("config.yaml") # From file object with open("config.yaml") as f: config = YAMLParser.load_config(f) # From string config = YAMLParser.load_config("key: value\nother: 123") ``` -------------------------------- ### Command-line Argument Parsing Example Source: https://github.com/marin-community/draccus/blob/main/docs/api.md Example of how to pass complex data structures like lists and dictionaries as command-line arguments using Draccus. ```console $ python train_model.py --worker_inds=[2,18,42] --worker_names="{2: 'George', 18: 'Ben'}" ``` -------------------------------- ### Include Config File Example (model_config.yaml) Source: https://github.com/marin-community/draccus/blob/main/README.md This YAML file defines a model configuration that can be included in another configuration file using the `!include` tag. ```yaml # model_config.yaml type: bert num_layers: 24 num_heads: 24 hidden_size: 1024 dropout: 0.2 ``` -------------------------------- ### YAML Parser Example with !include Directive Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-type.md Demonstrates YAML parsing with the custom !include directive for incorporating content from other YAML files. ```yaml # model.yaml type: bert num_layers: 24 # train.yaml model: !include model.yaml learning_rate: 0.001 ``` -------------------------------- ### Pyrallis Parse Function Source: https://github.com/marin-community/draccus/blob/main/docs/index.md A basic example of using `pyrallis.parse` to initialize a configuration dataclass from command-line arguments. ```python def main(): cfg = pyrallis.parse(config_class=TrainConfig) print(f'Training {cfg.exp_name} with {cfg.workers} workers...') ``` -------------------------------- ### Include Config via Command Line Argument Source: https://github.com/marin-community/draccus/blob/main/README.md This example shows how to use the `include` keyword in command-line arguments to dynamically include a configuration file for a nested parameter. This overrides values defined in the main config file. ```yaml # model_config.yaml type: bert_cli num_layers: 48 num_heads: 48 ``` ```yaml # train_config.yaml exp_name: my_yaml_exp workers: 42 model: type: bert num_layers: 24 num_heads: 24 ``` -------------------------------- ### Include Config File Example (train_config.yaml) Source: https://github.com/marin-community/draccus/blob/main/README.md This YAML file demonstrates including another configuration file (`model_config.yaml`) using the `!include` tag. This allows for modular configuration management. ```yaml # train_config.yaml exp_name: my_yaml_exp workers: 42 model: !include model_config.yaml ``` -------------------------------- ### Command-line Execution with Nested Configurations Source: https://github.com/marin-community/draccus/blob/main/README.md Demonstrates how to pass arguments for nested configurations using dot notation on the command line. This example sets the experiment name and evaluation workers for the nested log and compute configurations. ```console $ python train_model.py --log.exp_name=my_third_exp --compute.eval_workers=2 Training my_third_exp... Using 8 workers and 2 evaluation workers Saving to /share/experiments/my_third_exp ``` -------------------------------- ### YAML Configuration Example Source: https://github.com/marin-community/draccus/blob/main/README.md This snippet shows a typical YAML configuration file structure that Draccus can parse. Ensure your YAML file adheres to valid syntax. ```yaml compute: worker_inds: [0,2,3] ``` -------------------------------- ### Draccus Example with Configuration Type Switching Source: https://github.com/marin-community/draccus/blob/main/docs/api.md Demonstrates setting the global config type and then using a context manager to switch to a different type for a specific operation. Includes a dataclass definition and a main function. ```python import draccus dracccus.set_config_type('json') @dataclass class TrainConfig: """ Training config for Machine Learning """ workers: int = 8 # The number of workers for training exp_name: str = 'default_exp' # The experiment name @draccus.wrap() def main(cfg: TrainConfig): draccus.dump(cfg) with draccus.config_type('yaml'): draccus.dump(cfg) ``` -------------------------------- ### Manage Global Configuration Options Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/main-exports.md Provides static methods to set and get the global configuration type. Defaults to YAML. ```python class Options: _config_type: ConfigType = ConfigType.YAML @staticmethod def set_config_type(new_type: Union[ConfigType, str]) @staticmethod def get_config_type() -> ConfigType ``` -------------------------------- ### YAML Configuration File Example Source: https://github.com/marin-community/draccus/blob/main/README.md This YAML file defines configuration parameters that can be loaded by Draccus. It supports nested structures for complex configurations. ```yaml exp_name: my_yaml_exp workers: 42 model: type: bert num_layers: 24 num_heads: 24 hidden_size: 1024 dropout: 0.2 ``` -------------------------------- ### Plugin Discovery Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/choice-registry.md Demonstrates how to define a subclass of PluginRegistry to automatically discover model subclasses from a specified package. Plugins are registered automatically upon import using the @register_subclass decorator. ```python @dataclass class ModelConfig(draccus.PluginRegistry, discover_packages_path="my_models"): """Dynamically discovers model subclasses from the my_models package""" type: str = "gpt" # Plugin files (e.g., my_models/gpt.py, my_models/bert.py) automatically # register themselves on import via @ModelConfig.register_subclass decorator ``` -------------------------------- ### Instantiate ArgumentParser and parse arguments Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/parse-and-wrap.md Use the `ArgumentParser` class for more control over argument parsing, especially with nested dataclasses. This example shows basic instantiation and parsing. ```python parser = ArgumentParser(config_class=MyConfig) config = parser.parse_args(["--field", "value"]) ``` -------------------------------- ### Draccus TOML Configuration Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/quick-reference.md TOML is another supported configuration format for Draccus, useful for its clear syntax and hierarchical structure. ```toml workers = 8 flags = ["debug", "verbose"] [model] type = "gpt" layers = 12 ``` -------------------------------- ### Example Usage of Draccus dump() Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-loading.md Demonstrates serializing a dataclass instance to a string, a file, and in JSON format. Shows how to use omit_defaults and the config_type context manager. ```python from dataclasses import dataclass import draccus @dataclass class AppConfig: workers: int = 8 exp_name: str = "default" cfg = AppConfig(workers=16, exp_name="custom") # Dump to string (YAML format by default) yaml_str = draccus.dump(cfg) print(yaml_str) # Output: # workers: 16 # exp_name: custom # Dump to file with open("output.yaml", "w") as f: draccus.dump(cfg, f) # Dump without default values compact_yaml = draccus.dump(cfg, omit_defaults=True) # Dump as JSON with draccus.config_type("json"): json_str = draccus.dump(cfg) ``` -------------------------------- ### Run All Tests with Pytest Source: https://github.com/marin-community/draccus/blob/main/CONTRIBUTING.md Executes all tests in the project using pytest. Ensure uv is installed and dependencies are managed through it. ```bash uv run pytest ``` -------------------------------- ### Listing All Registered Choices Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/choice-registry.md Demonstrates how to get a dictionary of all choices currently registered within a ChoiceRegistry. This can be helpful for introspection or debugging. ```python choices = ModelConfig.get_known_choices() # Returns: {"gpt": GPTConfig, "bert": BERTConfig, ...} ``` -------------------------------- ### Draccus YAML Configuration Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/quick-reference.md Default configuration format for Draccus is YAML. Supports nested structures and lists. ```yaml workers: 8 model: type: gpt layers: 12 flags: - debug - verbose ``` -------------------------------- ### Save YAML Configuration Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/parsers.md Illustrates saving a Python dictionary to YAML format using YAMLParser.save_config(). Can return the YAML as a string or write directly to a file object. Supports custom dump arguments. ```python config = {"workers": 8, "model": {"type": "gpt"}} # Return as string yaml_str = YAMLParser.save_config(config) print(yaml_str) # Write to file with open("output.yaml", "w") as f: YAMLParser.save_config(config, f) # Custom formatting options yaml_str = YAMLParser.save_config(config, default_flow_style=False, sort_keys=False) ``` -------------------------------- ### Get Choice Class by Name Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/choice-registry.md Demonstrates retrieving a registered class using its short name or its fully qualified class name. ```python # Registered short name cls = MyRegistry.get_choice_class("myimpl") # Also supports fully qualified names cls = MyRegistry.get_choice_class("my_package.my_module.MyImplementation") ``` -------------------------------- ### Define Dataclass Configuration with Post-Init and Properties Source: https://github.com/marin-community/draccus/blob/main/README.md Enhance dataclass configurations with features like `__post_init__` for post-processing and `@property` for derived attributes. This example defines a `TrainConfig` with worker counts, experiment name, and root path, including logic to set evaluation workers and derive the experiment directory. ```python from dataclasses import dataclass, field from pathlib import Path from typing import Optional import draccus @dataclass class TrainConfig: """ Training config for Machine Learning """ # The number of workers for training workers: int = field(default=8) # The number of workers for evaluation eval_workers: Optional[int] = field(default=None) # The experiment name exp_name: str = field(default='default_exp') # The experiment root folder path exp_root: Path = field(default=Path('/share/experiments')) def __post_init__(self): # A builtin method of dataclasses, used for post-processing our configuration. self.eval_workers = self.eval_workers or self.workers @property def exp_dir(self) -> Path: # Properties are great for arguments that can be derived from existing ones return self.exp_root / self.exp_name @draccus.wrap() def main(cfg: TrainConfig): print(f'Training {cfg.exp_name}...) print(f' Using {cfg.workers} workers and {cfg.eval_workers} evaluation workers') print(f' Saving to {cfg.exp_dir}') ``` -------------------------------- ### Include Config at Top Level with Merge Source: https://github.com/marin-community/draccus/blob/main/README.md This example demonstrates including a base configuration file at the top level using PyYAML's `<<` merge key syntax combined with `!include`. This merges keys from the included file into the current configuration. ```yaml # base_config.yaml type: bert lr: 0.001 ``` ```yaml # train_config.yaml <<: !include base_config.yaml exp_name: my_yaml_exp ``` -------------------------------- ### Polymorphic Configuration Loading with ChoiceRegistry Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/choice-registry.md Example of using ChoiceRegistry to load polymorphic configurations from YAML. It demonstrates defining a base configuration class and registering different implementations, then parsing a configuration file to instantiate the correct subtype. ```python from dataclasses import dataclass import draccus @dataclass class OptimizerConfig(draccus.ChoiceRegistry): """Base optimizer configuration""" learning_rate: float = 0.001 @OptimizerConfig.register_subclass("adam") @dataclass class AdamConfig(OptimizerConfig): beta1: float = 0.9 beta2: float = 0.999 epsilon: float = 1e-8 @OptimizerConfig.register_subclass("sgd") @dataclass class SGDConfig(OptimizerConfig): momentum: float = 0.9 nesterov: bool = False @dataclass class TrainConfig: optimizer: OptimizerConfig = AdamConfig() # YAML configuration # optimizer: # type: sgd # learning_rate: 0.01 # momentum: 0.95 cfg = draccus.parse(TrainConfig, config_path="config.yaml") print(type(cfg.optimizer).__name__) # SGDConfig print(cfg.optimizer.momentum) # 0.95 ``` -------------------------------- ### Parse YAML String Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/parsers.md Demonstrates parsing simple scalar values and complex YAML structures from strings using YAMLParser.parse_string(). Handles empty strings by returning them as-is. ```python import draccus from draccus.parsers.config_parsers import YAMLParser # Parse simple value result = YAMLParser.parse_string("42") # 42 result = YAMLParser.parse_string("true") # True result = YAMLParser.parse_string("") # "" # Parse complex structure yaml_str = """ model: type: gpt layers: 12 workers: 8 """ result = YAMLParser.parse_string(yaml_str) # Returns: {"model": {"type": "gpt", "layers": 12}, "workers": 8} ``` -------------------------------- ### Set and Get Configuration Type Source: https://github.com/marin-community/draccus/blob/main/_autodocs/quick-reference.md Globally set the configuration file type (e.g., 'json', 'toml') or retrieve the current setting. Use 'config_type' context manager for temporary changes. ```python import draccus # Set globally dracccus.set_config_type("json") # Get current current = draccus.get_config_type() # Temporary change with draccus.config_type("toml"): cfg = draccus.load(MyConfig, "config.toml") toml_str = draccus.dump(cfg) ``` -------------------------------- ### Parse Nested Configurations from Command Line Source: https://github.com/marin-community/draccus/blob/main/docs/step_by_step.md Command-line arguments can directly access nested configuration values using dot notation. This example shows how to override default values. ```console $ python train_model.py --log.exp_name=my_third_exp --compute.eval_workers=2 Training my_third_exp... Using 8 workers and 2 evaluation workers Saving to /share/experiments/my_third_exp ``` -------------------------------- ### Define Hierarchical Dataclass Configurations Source: https://github.com/marin-community/draccus/blob/main/README.md Supports nested dataclasses for complex configurations. This example defines `ComputeConfig`, `LogConfig`, and a main `TrainConfig` that nests the other two. Use `default_factory` for initializing nested dataclasses. ```python from dataclasses import dataclass, field from pathlib import Path from typing import Optional import draccus @dataclass class ComputeConfig: """ Config for training resources """ # The number of workers for training workers: int = field(default=8) # The number of workers for evaluation eval_workers: Optional[int] = field(default=None) def __post_init__(self): # A builtin method of dataclasses, used for post-processing our configuration. self.eval_workers = self.eval_workers or self.workers @dataclass class LogConfig: """ Config for logging arguments """ # The experiment name exp_name: str = field(default='default_exp') # The experiment root folder path exp_root: Path = field(default=Path('/share/experiments')) @property def exp_dir(self) -> Path: # Properties are great for arguments that can be derived from existing ones return self.exp_root / self.exp_name # TrainConfig will be our main configuration class. # Notice that default_factory is the standard way to initialize a class argument in dataclasses @dataclass class TrainConfig: log: LogConfig = field(default_factory=LogConfig) compute: ComputeConfig = field(default_factory=ComputeConfig) @pyrallis.wrap() def main(cfg: TrainConfig): print(f'Training {cfg.log.exp_name}...) print(f' Using {cfg.compute.workers} workers and {cfg.compute.eval_workers} evaluation workers') print(f' Saving to {cfg.log.exp_dir}') ``` -------------------------------- ### JSON Parser Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-type.md Standard JSON parsing example. If a string cannot be parsed as JSON, it is returned as-is. ```json { "workers": 8, "model": { "type": "gpt", "layers": 12 } } ``` -------------------------------- ### Register and Load Subclasses with QNamePluginRegistry Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/choice-registry.md Shows how to define a configuration class using QNamePluginRegistry, register subclasses with short names, and then load instances using either short or fully qualified names. ```python @dataclass class ExecutorConfig(draccus.QNamePluginRegistry, discover_packages_path="executors"): """Supports both short names and fully qualified class names""" pass @ExecutorConfig.register_subclass("local") @dataclass class LocalExecutor(ExecutorConfig): workers: int = 1 # Can use short name executor1 = draccus.loads(ExecutorConfig, "type: local\nworkers: 4") # Or fully qualified name executor2 = draccus.loads(ExecutorConfig, "type: my_pkg.custom.CustomExecutor") ``` -------------------------------- ### Custom Integer Decoding Function Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/types.md Example of a custom decoding function that converts raw values to integers. It includes error handling for invalid conversions and uses the DecodingFunction protocol. ```python from draccus.parsers.decoding import DecodingFunction from typing import Any, Sequence def my_decoder(raw_value: Any, path: Sequence[str]) -> int: """Decoding function that converts strings to int""" try: return int(raw_value) except ValueError as e: raise draccus.DecodingError(path, f"Cannot convert {raw_value} to int") from e ``` -------------------------------- ### load(), loads(), dump() Source: https://github.com/marin-community/draccus/blob/main/_autodocs/MANIFEST.txt Documentation for configuration loading and dumping functions. Includes details on loading from files (load()) and strings (loads()), and dumping configurations (dump()). ```APIDOC ## load() ### Description Loads configuration from a file path. ### Method `load(path: str, ...)` ### Parameters (Details on parameters, types, defaults, and descriptions would be here if available in source) ### Example ```python # Example usage of load() ``` ## loads() ### Description Loads configuration from a string. ### Method `loads(config_string: str, ...)` ### Parameters (Details on parameters, types, defaults, and descriptions would be here if available in source) ### Example ```python # Example usage of loads() ``` ## dump() ### Description Dumps configuration to a string or file. ### Method `dump(config_object, ...)` ### Parameters (Details on parameters, types, defaults, and descriptions would be here if available in source) ### Example ```python # Example usage of dump() ``` ``` -------------------------------- ### Running with Dataclass Features Source: https://github.com/marin-community/draccus/blob/main/docs/step_by_step.md Execute the script with arguments to see the effect of `__post_init__` and `@property` in action. ```console $ python -m train_model.py --exp_name=my_second_exp --workers=42 Training my_second_exp... Using 42 workers and 42 evaluation workers Saving to /share/experiments/my_second_exp ``` -------------------------------- ### Enum Decoding Error Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/errors.md Demonstrates catching a DecodingError when a value cannot be parsed into a defined enum member. ```python from dataclasses import dataclass from enum import Enum import draccus class Status(Enum): ACTIVE = "active" INACTIVE = "inactive" @dataclass class Config: status: Status try: cfg = draccus.loads(Config, "status: pending") except draccus.DecodingError as e: print(e) # `status`: Couldn't parse 'pending' into an enum ``` -------------------------------- ### get_config_type Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-type.md Get the current global configuration type. This method returns the active ConfigType enum value. ```APIDOC ## get_config_type() -> ConfigType ### Description Get the current global configuration type. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Method staticmethod ### Endpoint N/A (This is a Python method) ### Parameters None ### Returns The active ConfigType ### Example ```python import draccus current = draccus.get_config_type() print(current) # ConfigType.YAML ``` ``` -------------------------------- ### Parsing from Command Line Source: https://github.com/marin-community/draccus/blob/main/docs/step_by_step.md Demonstrates how to pass arguments directly from the command line to the Python script. ```console $ python train_model.py --exp_name=my_first_model Training my_first_model with 8 workers... ``` -------------------------------- ### Union Type Decoding Error Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/errors.md Illustrates catching a DecodingError when a value does not match any of the types specified in a Union. ```python from dataclasses import dataclass from typing import Union import draccus @dataclass class Config: value: Union[int, str] # Must be int or str try: cfg = draccus.loads(Config, "value: [1,2,3]") # List doesn't match except draccus.DecodingError as e: print(e) # Error indicating list doesn't match int or str ``` -------------------------------- ### Choice Type Unknown Model Error Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/errors.md Shows how to catch a DecodingError when a specified type for a choice class is not registered. ```python from dataclasses import dataclass import draccus @dataclass class ModelConfig(draccus.ChoiceRegistry): pass @ModelConfig.register_subclass("gpt") @dataclass class GPT(ModelConfig): layers: int = 12 @dataclass class Config: model: ModelConfig = GPT() try: cfg = draccus.loads(Config, """ model: type: unknown_model layers: 24 """) except draccus.DecodingError as e: print(e) # `model`: Couldn't find a choice class for 'unknown_model' ``` -------------------------------- ### Accessing and Using Parser Classes Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/parsers.md Demonstrates how to access parser classes from the ConfigType enum and how to load a configuration file using the appropriate parser. Ensure Draccus is imported and a configuration file exists. ```python from draccus.options import ConfigType # Access parser classes yaml_parser = ConfigType.YAML.value json_parser = ConfigType.JSON.value toml_parser = ConfigType.TOML.value # Get parser from config type import draccus current_type = draccus.get_config_type() parser = current_type.value config = parser.load_config("config.yaml") ``` -------------------------------- ### Dataclass Missing Required Field Error Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/errors.md Illustrates catching a DecodingError when a required field is missing in the configuration for a dataclass. ```python from dataclasses import dataclass import draccus @dataclass class Config: required_field: int try: cfg = draccus.loads(Config, "other_field: 5") except draccus.DecodingError as e: print(e) # `root`: Missing required field(s) `required_field` for Config ``` -------------------------------- ### Basic Integer Decoding Error Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/errors.md Demonstrates catching a DecodingError when attempting to parse a non-integer string into an integer field. ```python from dataclasses import dataclass import draccus @dataclass class Config: value: int try: cfg = draccus.loads(Config, "value: not_an_integer") except draccus.DecodingError as e: print(e) # Output: `value`: Couldn't parse 'not_an_integer' into an int print(e.key_path) # ("value",) print(e.message) # Couldn't parse 'not_an_integer' into an int ``` -------------------------------- ### Load Configuration with Command-Line Argument Source: https://github.com/marin-community/draccus/blob/main/README.md Specify the configuration file path using the --config_path argument when running the script. Command-line arguments have higher priority. ```bash python my_script.py --log.exp_name=readme_exp --config_path=/share/configs/config.yaml ``` -------------------------------- ### Processing Dataclass Instances Source: https://github.com/marin-community/draccus/blob/main/_autodocs/types.md Example of a function that accepts any object conforming to the DataclassInstance protocol and iterates over its fields. Requires importing DataclassInstance. ```python from draccus.utils import DataclassInstance def process_dataclass(obj: DataclassInstance) -> None: for field_name in obj.__dataclass_fields__: print(field_name) ``` -------------------------------- ### PluginRegistry - __init_subclass__ Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/choice-registry.md Initializes a PluginRegistry subclass, configuring it for plugin discovery. The `discover_packages_path` parameter specifies the package namespace to search for plugins. This path is required either during subclass definition or when initializing the subclass. ```APIDOC ## __init_subclass__ (classmethod) ### Description Initializes a PluginRegistry subclass with plugin discovery configuration. It ensures that the `discover_packages_path` is set, either directly or through the subclass definition. ### Method classmethod ### Parameters #### Parameters - **discover_packages_path** (Optional[str]) - No - Package path for plugin discovery. Must be provided if not set by subclass ### Raises ValueError if discover_packages_path is not specified ``` -------------------------------- ### Draccus JSON Configuration Example Source: https://github.com/marin-community/draccus/blob/main/_autodocs/quick-reference.md Draccus can also parse JSON configuration files. The structure mirrors YAML and Python dictionaries. ```json { "workers": 8, "model": { "type": "gpt", "layers": 12 }, "flags": ["debug", "verbose"] } ``` -------------------------------- ### Getting the Registered Name of a Subclass Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/choice-registry.md Illustrates how to find the registered choice name associated with a given subclass. This is the inverse operation of `get_choice_class`. ```python name = ModelConfig.get_choice_name(GPTConfig) # Returns: "gpt" ``` -------------------------------- ### Get Current Global Configuration Type Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-type.md Retrieve the current global configuration type. Returns the active ConfigType enum value. ```python import draccus current = draccus.get_config_type() print(current) # ConfigType.YAML ``` -------------------------------- ### Load Configuration from File Path, File Object, or String Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-loading.md Use `load()` to populate a dataclass instance from a file path, file object, or string. File type is auto-detected by extension for paths; otherwise, the configured parser is used. ```python from dataclasses import dataclass import draccus @dataclass class AppConfig: workers: int name: str # From file path cfg = draccus.load(AppConfig, "config.yaml") # From file object with open("config.json") as f: cfg = draccus.load(AppConfig, f) # From string cfg = draccus.load(AppConfig, "workers: 8\nname: myapp") ``` -------------------------------- ### DecodingError Example Usage Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/encoding-decoding.md Demonstrates how to catch and handle `DecodingError`, including accessing its attributes and using the `strip_prefix` method to modify the error path. ```python from draccus.utils import DecodingError try: # Decoding fails pass except DecodingError as e: print(e) # Output: `model.layers`: Value must be int, got str print(e.key_path) # ("model", "layers") print(e.message) # Value must be int, got str # Remove the "model" prefix when re-raising new_error = e.strip_prefix(["model"]) print(new_error) # Output: `layers`: Value must be int, got str ``` -------------------------------- ### Basic Configuration Parsing Source: https://github.com/marin-community/draccus/blob/main/_autodocs/README.md Parses configuration from both command-line arguments and a specified config file into a dataclass. Use this for a combined parsing approach. ```python from dataclasses import dataclass import draccus @dataclass class Config: workers: int = 8 name: str = "default" # Parse from CLI and config file cfg = draccus.parse(Config, config_path="config.yaml") # Use decorator instead @draccus.wrap() def main(cfg: Config): print(f"Workers: {cfg.workers}") if __name__ == "__main__": main() ``` -------------------------------- ### Handle ParsingError During Config Loading Source: https://github.com/marin-community/draccus/blob/main/_autodocs/types.md Example of how to catch a ParsingError when loading a configuration file. Ensure the file path is correct and the format is valid YAML. ```python import draccus try: cfg = draccus.load(MyConfig, "invalid.yaml") except draccus.ParsingError as e: print(f"Failed to parse config: {e}") ``` -------------------------------- ### CLI File Inclusion with 'include' Keyword Source: https://github.com/marin-community/draccus/blob/main/_autodocs/configuration.md Incorporate external configuration files into the main configuration via the command line using the `include` keyword. This merges the specified file's content into the configuration at the given path. ```bash # train_config.yaml contains: model.type: bert python app.py --model="include model_config.yaml" ``` -------------------------------- ### ConfigType and Options Classes Source: https://github.com/marin-community/draccus/blob/main/_autodocs/MANIFEST.txt Documentation for ConfigType and Options classes, used for defining and managing configuration types and their associated options. ```APIDOC ## ConfigType ### Description Represents a type of configuration. ### Methods (Details on methods and attributes would be here if available in source) ### Example ```python # Example usage of ConfigType ``` ## Options ### Description Holds options related to configuration parsing and processing. ### Methods (Details on methods and attributes would be here if available in source) ### Example ```python # Example usage of Options ``` ``` -------------------------------- ### Simple Configuration Parsing and Dumping Source: https://github.com/marin-community/draccus/blob/main/_autodocs/README.md Use `draccus.parse` to load configuration into a dataclass and `draccus.dump` to serialize it. Ensure the dataclass is defined with appropriate types and default values. ```python from dataclasses import dataclass import draccus @dataclass class AppConfig: debug: bool = False workers: int = 8 output_dir: str = "./output" cfg = draccus.parse(AppConfig) print(draccus.dump(cfg)) ``` -------------------------------- ### Dataclass Field Type Resolution Flow Source: https://github.com/marin-community/draccus/blob/main/_autodocs/module-structure.md Illustrates the process of resolving types for dataclass fields, starting from the field type and proceeding through canonicalization and decoder instantiation. ```text field.type → canonicalize_union() ↓ get_decoding_fn() [decoding.py] ↓ Check for generic parameters ↓ Instantiate decoder for specific type ↓ DecodingFunction[T] ``` -------------------------------- ### Handle DraccusException Source: https://github.com/marin-community/draccus/blob/main/_autodocs/errors.md Use this snippet to catch and handle any draccus-specific exceptions during configuration parsing. This is useful for gracefully managing errors related to configuration setup or decorator arguments. ```python import draccus try: cfg = draccus.parse(MyConfig, preferred_help="invalid_value") except draccus.DraccusException as e: print(f"Draccus error: {e}") ``` -------------------------------- ### Dumping Dataclass Object to YAML Source: https://github.com/marin-community/draccus/blob/main/_autodocs/types.md Example function demonstrating how to use the draccus.dump function with an object typed as Dataclass. This function is designed to work with any object that adheres to the DataclassInstance protocol. ```python from draccus.utils import Dataclass def dump_to_yaml(obj: Dataclass) -> str: """Works with any dataclass""" return draccus.dump(obj) ``` -------------------------------- ### Load Configuration from String Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-loading.md Use `loads()` for explicit loading of configuration from a string into a dataclass instance. It utilizes the currently configured parser, defaulting to YAML. ```python from dataclasses import dataclass import draccus @dataclass class Config: value: int yaml_str = """ value: 42 """ cfg = draccus.loads(Config, yaml_str) assert cfg.value == 42 ``` -------------------------------- ### Instantiate ArgumentParser Directly Source: https://github.com/marin-community/draccus/blob/main/_autodocs/configuration.md Create an ArgumentParser instance for fine-grained control over parsing behavior. Use this when you need to specify the config class, default config path, help formatter, and error handling. ```python from draccus.argparsing import ArgumentParser parser = ArgumentParser( config_class=MyConfig, config_path="config.yaml", prog="myapp", exit_on_error=True, preferred_help="inline" ) # Parse with custom args config = parser.parse_args(["--field", "value"]) # Parse allowing unknown args config, unknown = parser.parse_known_args(["--field", "value", "--unknown"]) ``` -------------------------------- ### Load Raw Configuration Dictionary Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-loading.md Loads a configuration dictionary from a file path, file object, or string. The file parameter can be used to infer the configuration type by its extension. ```python def load_config( stream: Union[str, TextIO, os.PathLike], *, file: Optional[Union[str, Path, os.PathLike]] = None ) -> dict ``` -------------------------------- ### Parse Configuration with Draccus Source: https://github.com/marin-community/draccus/blob/main/_autodocs/quick-reference.md Demonstrates various ways to parse configuration into dataclasses using draccus, including from command-line arguments, files, custom arguments, and decorators. Ensure the dataclass is defined before parsing. ```python import draccus from dataclasses import dataclass @dataclass class MyConfig: field1: int field2: str = "default" # From command-line arguments cfg = draccus.parse(MyConfig) # From file cfg = draccus.parse(MyConfig, config_path="config.yaml") # With custom arguments cfg = draccus.parse(MyConfig, args=["--field1", "42"]) # With decorator @draccus.wrap(config_path="config.yaml") def main(cfg: MyConfig): print(cfg.field1) if __name__ == "__main__": main() ``` -------------------------------- ### Detailed Error Information with DecodingError Source: https://github.com/marin-community/draccus/blob/main/_autodocs/configuration.md Utilize DecodingError to get detailed path information for configuration errors. This helps in pinpointing the exact location of the error within the configuration structure. ```python import draccus try: cfg = draccus.load(MyConfig, "config.yaml") except draccus.DecodingError as e: print(f"Error at {'.'.join(e.key_path)}: {e.message}") # Example: "model.layers: Expected int, got str" ``` -------------------------------- ### Basic Parsing with pyrallis.parse Source: https://github.com/marin-community/draccus/blob/main/docs/step_by_step.md Define a dataclass for configuration and parse it using `draccus.parse`. This is useful for simple configuration needs. ```python from dataclasses import dataclass, field import draccus @dataclass class TrainConfig: """ Training config for Machine Learning """ # The number of workers for training workers: int = field(default=8) # The experiment name exp_name: str = field(default='default_exp') def main(): cfg = draccus.parse(config_class=TrainConfig) print(f'Training {cfg.exp_name} with {cfg.workers} workers...') if __name__ == '__main__': main() ``` -------------------------------- ### Handle DecodingError During Config Loading Source: https://github.com/marin-community/draccus/blob/main/_autodocs/types.md Example of catching a DecodingError when loading configuration from a string. This demonstrates accessing the key path and message, and using strip_prefix to re-raise errors with a modified path. ```python import draccus try: cfg = draccus.loads(Config, yaml_string) except draccus.DecodingError as e: print(e) # `model.layers`: Expected int print(e.key_path) # ("model", "layers") print(e.message) # Expected int # Remove prefix when re-raising new_error = e.strip_prefix(["model"]) print(new_error) # `layers`: Expected int ``` -------------------------------- ### Automatic Format Detection by File Extension Source: https://github.com/marin-community/draccus/blob/main/_autodocs/api-reference/config-type.md Load configuration from a file path, with the format automatically detected by its extension (e.g., .yaml, .json, .toml). ```python import draccus @dataclass class Config: value: int # Automatically uses YAML parser cfg = draccus.load(Config, "config.yaml") # Automatically uses JSON parser cfg = draccus.load(Config, "config.json") # Automatically uses TOML parser cfg = draccus.load(Config, "config.toml") # Can also pass file path hint to load_config d = draccus.cfgparsing.load_config(stream_obj, file="config.json") ``` -------------------------------- ### Load and Re-save Configurations Source: https://github.com/marin-community/draccus/blob/main/_autodocs/configuration.md Load a configuration from a YAML file, modify its properties, and then save it back to a file. ```python # Load from file cfg = draccus.load(MyConfig, "input.yaml") # Modify cfg.workers = 32 # Save back with open("output.yaml", "w") as f: draccus.dump(cfg, f) ``` -------------------------------- ### parse() and wrap() Source: https://github.com/marin-community/draccus/blob/main/_autodocs/MANIFEST.txt Documentation for the parse() and wrap() functions, which are central to Draccus's configuration parsing and object wrapping capabilities. This includes details on their signatures, parameters, return types, and usage examples. ```APIDOC ## parse() ### Description Parses configuration data from various sources into Python objects. ### Method `parse()` ### Parameters (Details on parameters, types, defaults, and descriptions would be here if available in source) ### Return Type (Details on return type would be here if available in source) ### Exceptions (Details on exceptions raised would be here if available in source) ### Example ```python # Example usage of parse() ``` ## wrap() ### Description Wraps Python objects with configuration and parsing logic. ### Method `wrap()` ### Parameters (Details on parameters, types, defaults, and descriptions would be here if available in source) ### Return Type (Details on return type would be here if available in source) ### Exceptions (Details on exceptions raised would be here if available in source) ### Example ```python # Example usage of wrap() ``` ``` -------------------------------- ### Basic Argparse Configuration Source: https://github.com/marin-community/draccus/blob/main/docs/index.md Define a dataclass for training configuration and parse arguments using pyrallis. This approach offers type hints and automatic code completion. ```python from argparse import ArgumentParser, Namespace def get_config() -> Namespace: parser = ArgumentParser(description='Training config for Machine Learning') parser.add_argument('--workers', type=int, default=8, help='The number of workers for training') parser.add_argument('--exp_name', type=str, default='default_exp', help='The experiment name') return parser.parse_args() def main(): cfg = get_config() print(f'Training {cfg.exp_name} with {cfg.workers} workers...') ``` -------------------------------- ### Configuration Loading/Dumping Functions Source: https://github.com/marin-community/draccus/blob/main/_autodocs/INDEX.md Functions for loading, dumping, and parsing configuration data. ```APIDOC ## load() ### Description Loads configuration data. ### Method `load()` ### Endpoint N/A (Function) ``` ```APIDOC ## loads() ### Description Loads configuration data from a string. ### Method `loads()` ### Endpoint N/A (Function) ``` ```APIDOC ## dump() ### Description Dumps configuration data. ### Method `dump()` ### Endpoint N/A (Function) ``` ```APIDOC ## load_config() ### Description Loads a configuration. ### Method `load_config()` ### Endpoint N/A (Function) ``` ```APIDOC ## save_config() ### Description Saves a configuration. ### Method `save_config()` ### Endpoint N/A (Function) ``` ```APIDOC ## parse_string() ### Description Parses a string into configuration data. ### Method `parse_string()` ### Endpoint N/A (Function) ```