### Install OmegaConf Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Use pip to install the library. ```bash pip install omegaconf ``` -------------------------------- ### Config interpolation example Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Demonstrates primitive, string, and relative interpolation using a loaded configuration. ```python >>> conf = OmegaConf.load('source/config_interpolation.yaml') >>> def show(x): ... print(f"type: {type(x).__name__}, value: {repr(x)}") >>> # Primitive interpolation types are inherited from the reference >>> show(conf.client.server_port) type: int, value: 80 >>> # String interpolations concatenate fragments into a string >>> show(conf.client.url) type: str, value: 'http://localhost:80/' >>> # Relative interpolation example >>> show(conf.client.description) type: str, value: 'Client of http://localhost:80/' ``` -------------------------------- ### Basic custom resolver usage Source: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html A simple example of a resolver that adds 10 to a provided value. ```python >>> OmegaConf.register_new_resolver("plus_10", lambda x: x + 10) >>> c = OmegaConf.create({'key': '${plus_10:990}'}) >>> c.key 1000 ``` -------------------------------- ### Define configuration with interpolation in YAML Source: https://omegaconf.readthedocs.io/en/2.3_branch/usage.html Example YAML structure demonstrating absolute and relative variable interpolation. ```yaml server: host: localhost port: 80 client: url: http://${server.host}:${server.port}/ server_port: ${server.port} # relative interpolation description: Client of ${.url} ``` -------------------------------- ### Example of Clearing a Built-in Resolver Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/custom_resolvers.rst.txt Shows how to remove a default resolver like oc.env using clear_resolver. ```python >>> OmegaConf.has_resolver("oc.env") True >>> # This will remove the default resolver: oc.env >>> OmegaConf.clear_resolver("oc.env") True >>> OmegaConf.has_resolver("oc.env") False ``` -------------------------------- ### Providing default values Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Uses the get method to provide a fallback value if a key is missing. ```python >>> conf.get('missing_key', 'a default value') 'a default value' ``` -------------------------------- ### Handle Configuration Cache Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Utility methods to get, set, clear, or copy the resolver cache for a given configuration container. ```python @staticmethod def get_cache(conf: BaseContainer) -> Dict[str, Any]: return conf._metadata.resolver_cache ``` ```python @staticmethod def set_cache(conf: BaseContainer, cache: Dict[str, Any]) -> None: conf._metadata.resolver_cache = copy.deepcopy(cache) ``` ```python @staticmethod def clear_cache(conf: BaseContainer) -> None: OmegaConf.set_cache(conf, defaultdict(dict, {})) ``` ```python @staticmethod def copy_cache(from_config: BaseContainer, to_config: BaseContainer) -> None: OmegaConf.set_cache(to_config, OmegaConf.get_cache(from_config)) ``` -------------------------------- ### Define example2.yaml configuration file Source: https://omegaconf.readthedocs.io/en/2.3_branch/usage.html A sample YAML configuration file defining server port and user list. ```yaml server: port: 80 users: - user1 - user2 ``` -------------------------------- ### Define example3.yaml configuration file Source: https://omegaconf.readthedocs.io/en/2.3_branch/usage.html A sample YAML configuration file defining log settings. ```yaml log: file: log.txt ``` -------------------------------- ### Create from List Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Initialize a configuration from a Python list. ```python >>> conf = OmegaConf.create([1, {"a":10, "b": {"a":10, 123: "int_key"}}]) >>> print(OmegaConf.to_yaml(conf)) - 1 - a: 10 b: a: 10 123: int_key ``` -------------------------------- ### Get object type Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Retrieves the type of the object or a specific child node within the configuration. ```python @staticmethod def get_type(obj: Any, key: Optional[str] = None) -> Optional[Type[Any]]: if key is not None: c = obj._get_child(key) else: c = obj return OmegaConf._get_obj_type(c) ``` -------------------------------- ### Create from YAML String Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Initialize a configuration from a raw YAML string. ```python >>> s = """ ... a: b ... b: c ... list: ... - item1 ... - item2 ... 123: 456 ... """ >>> conf = OmegaConf.create(s) >>> print(OmegaConf.to_yaml(conf)) a: b b: c list: - item1 - item2 123: 456 ``` -------------------------------- ### Create from Dictionary Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Initialize a configuration from a standard Python dictionary. ```python >>> conf = OmegaConf.create({"k" : "v", "list" : [1, {"a": "1", "b": "2", 3: "c"}]}) >>> print(OmegaConf.to_yaml(conf)) k: v list: - 1 - a: '1' b: '2' 3: c ``` -------------------------------- ### Create from Dot-List Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Initialize a configuration from a list of dot-notation strings. ```python >>> dot_list = ["a.aa.aaa=1", "a.aa.bbb=2", "a.bb.aaa=3", "a.bb.bbb=4"] >>> conf = OmegaConf.from_dotlist(dot_list) >>> print(OmegaConf.to_yaml(conf)) a: aa: aaa: 1 bbb: 2 bb: aaa: 3 bbb: 4 ``` -------------------------------- ### Initialize and customize structured configs Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/structured_config.rst.txt Demonstrates creating a config from a class or instance, and overriding default values during initialization. ```python conf1 = OmegaConf.structured(SimpleTypes) conf2 = OmegaConf.structured(SimpleTypes()) # The two configs are identical in this case assert conf1 == conf2 # But the second form allow for easy customization of the values: conf3 = OmegaConf.structured( SimpleTypes(num=20, height=Height.TALL)) print(OmegaConf.to_yaml(conf3)) ``` -------------------------------- ### Merge configurations from files and CLI Source: https://omegaconf.readthedocs.io/en/2.3_branch/usage.html Demonstrates loading configurations from files and command line arguments, then merging them into a single object. ```python >>> from omegaconf import OmegaConf >>> import sys >>> >>> # Simulate command line arguments >>> sys.argv = ['program.py', 'server.port=82'] >>> >>> base_conf = OmegaConf.load('source/example2.yaml') >>> second_conf = OmegaConf.load('source/example3.yaml') >>> cli_conf = OmegaConf.from_cli() >>> >>> # merge them all >>> conf = OmegaConf.merge(base_conf, second_conf, cli_conf) >>> print(OmegaConf.to_yaml(conf)) server: port: 82 users: - user1 - user2 log: file: log.txt ``` -------------------------------- ### Accessing configuration elements Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Demonstrates object-style, dictionary-style, and list-index access patterns for configuration nodes. ```python >>> # object style access of dictionary elements >>> conf.server.port 80 >>> # dictionary style access >>> conf['log']['rotation'] 3600 >>> # items in list >>> conf.users[0] 'user1' ``` -------------------------------- ### Create from Structured Config Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Initialize configurations using dataclasses for type safety. ```python >>> from dataclasses import dataclass >>> @dataclass ... class MyConfig: ... port: int = 80 ... host: str = "localhost" >>> # For strict typing purposes, prefer OmegaConf.structured() when creating structured configs >>> conf = OmegaConf.structured(MyConfig) >>> print(OmegaConf.to_yaml(conf)) port: 80 host: localhost ``` ```python >>> conf = OmegaConf.structured(MyConfig(port=443)) >>> print(OmegaConf.to_yaml(conf)) port: 443 host: localhost ``` -------------------------------- ### Manipulating configuration Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Shows how to update existing keys, add new keys, and insert new dictionary structures. ```python >>> # Changing existing keys >>> conf.server.port = 81 >>> # Adding new keys >>> conf.server.hostname = "localhost" >>> # Adding a new dictionary >>> conf.database = {'hostname': 'database01', 'port': 3306} ``` -------------------------------- ### Create Empty Config Source: https://omegaconf.readthedocs.io/en/2.3_branch/usage.html Initialize an empty configuration object. ```python >>> from omegaconf import OmegaConf >>> conf = OmegaConf.create() >>> print(OmegaConf.to_yaml(conf)) {} ``` -------------------------------- ### Create Empty OmegaConf Object Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Initialize an empty configuration object. ```python >>> from omegaconf import OmegaConf >>> conf = OmegaConf.create() >>> print(OmegaConf.to_yaml(conf)) {} ``` -------------------------------- ### OmegaConf.from_cli Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Creates a configuration object from command line arguments. ```APIDOC ## OmegaConf.from_cli(args_list=None) ### Description Creates a DictConfig from the provided list of arguments, or from sys.argv if no list is provided. ### Parameters - **args_list** (Optional[List[str]]) - Optional - A list of dotlist-style strings. If None, defaults to sys.argv[1:]. ``` -------------------------------- ### Create from CLI Arguments Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Parse command-line arguments into a configuration object. ```python >>> # Simulating command line arguments >>> sys.argv = ['your-program.py', 'server.port=82', 'log.file=log2.txt'] >>> conf = OmegaConf.from_cli() >>> print(OmegaConf.to_yaml(conf)) server: port: 82 log: file: log2.txt ``` -------------------------------- ### Using oc.select for defaults and special keys Source: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html Demonstrates using oc.select to provide default values, access keys containing colons, and handle missing values. ```python >>> cfg = OmegaConf.create({ ... "a": "Saving output to ${oc.select:output,/tmp}" ... }) >>> print(cfg.a) Saving output to /tmp >>> cfg.output = "/etc/config" >>> print(cfg.a) Saving output to /etc/config ``` ```python >>> cfg = OmegaConf.create({ ... # yes, there is a : in this key ... "a:b": 10, ... "bad": "${a:b}", ... "good": "${oc.select:'a:b'}", ... }) >>> print(cfg.bad) Traceback (most recent call last): ... UnsupportedInterpolationType: Unsupported interpolation type a >>> print(cfg.good) 10 ``` ```python >>> cfg = OmegaConf.create({ ... "missing": "???", ... "interpolation": "${missing}", ... "select": "${oc.select:missing}", ... "with_default": "${oc.select:missing,default value}", ... } ... ) ... >>> print(cfg.interpolation) Traceback (most recent call last): ... InterpolationToMissingValueError: MissingMandatoryValue while ... >>> print(cfg.select) None >>> print(cfg.with_default) default value ``` -------------------------------- ### Using lists in structured configurations Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/structured_config.rst.txt Demonstrates defining and validating lists of types within structured configurations. ```python >>> from dataclasses import dataclass, field >>> from typing import List, Tuple >>> @dataclass ... class User: ... name: str = MISSING >>> @dataclass ... class ListsExample: ... # Typed list can hold Any, int, float, bool, str, ... # bytes, pathlib.Path and Enums as well as arbitrary Structured configs. ... ints: List[int] = field(default_factory=lambda: [10, 20, 30]) ... bools: Tuple[bool, bool] = field(default_factory=lambda: (True, False)) ... users: List[User] = field(default_factory=lambda: [User(name="omry")]) ``` ```python >>> conf: ListsExample = OmegaConf.structured(ListsExample) >>> # Okay, 10 is an int >>> conf.ints.append(10) >>> # Okay, "20" can be converted to an int >>> conf.ints.append("20") >>> conf.bools.append(True) >>> conf.users.append(User(name="Joe")) >>> # Not okay, 10 cannot be converted to a User >>> with raises(ValidationError): ... conf.users.append(10) ``` -------------------------------- ### Select configuration nodes with OmegaConf.select Source: https://omegaconf.readthedocs.io/en/2.3_branch/usage.html Use dot-notation or brackets to retrieve specific keys from a configuration object. Supports default values and error handling for missing nodes. ```python >>> cfg = OmegaConf.create({ ... "foo" : { ... "missing" : "???", ... "bar": { ... "zonk" : 10, ... } ... } ... }) >>> assert OmegaConf.select(cfg, "foo") == { ... "missing" : "???", ... "bar": { ... "zonk" : 10, ... } ... } >>> assert OmegaConf.select(cfg, "foo.bar") == { ... "zonk" : 10, ... } >>> assert OmegaConf.select(cfg, "foo.bar.zonk") == 10 # dots >>> assert OmegaConf.select(cfg, "foo[bar][zonk]") == 10 # brackets >>> assert OmegaConf.select(cfg, "no_such_key", default=99) == 99 >>> assert OmegaConf.select(cfg, "foo.missing") is None >>> assert OmegaConf.select(cfg, "foo.missing", default=99) == 99 >>> OmegaConf.select(cfg, ... "foo.missing", ... throw_on_missing=True ... ) Traceback (most recent call last): ... omegaconf.errors.MissingMandatoryValue: missing node selected full_key: foo.missing ``` -------------------------------- ### Load from YAML File Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Load configuration data directly from a YAML file. ```python >>> conf = OmegaConf.load('source/example.yaml') >>> # Output is identical to the YAML file >>> print(OmegaConf.to_yaml(conf)) server: port: 80 log: file: ??? rotation: 3600 users: - user1 - user2 ``` -------------------------------- ### Define and instantiate nested structured configs Source: https://omegaconf.readthedocs.io/en/2.3_branch/structured_config.html Demonstrates nesting dataclasses within a parent class and initializing them using OmegaConf.structured. ```python >>> @dataclass ... class User: ... # A simple user class with two missing fields ... name: str = MISSING ... height: Height = MISSING >>> >>> @dataclass ... class DuperUser(User): ... duper: bool = True ... >>> # Group class contains two instances of User. >>> @dataclass ... class Group: ... name: str = MISSING ... # data classes can be nested ... admin: User = field(default_factory=User) ... ... # You can also specify different defaults for nested classes ... manager: User = field(default_factory=lambda: User(name="manager", height=Height.TALL)) >>> conf: Group = OmegaConf.structured(Group) >>> print(OmegaConf.to_yaml(conf)) name: ??? admin: name: ??? height: ??? manager: name: manager height: TALL ``` -------------------------------- ### Register and Manage Custom Resolvers Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/custom_resolvers.rst.txt Demonstrates registering a new resolver, checking its existence, and clearing all custom resolvers. ```python >>> # register a new resolver: str.lower >>> OmegaConf.register_new_resolver( ... name='str.lower', ... resolver=lambda x: str(x).lower(), ... ) >>> # check if resolver exists (after adding, before removal) >>> OmegaConf.has_resolver("str.lower") True >>> # clear all custom-resolvers >>> OmegaConf.clear_resolvers() >>> # check if resolver exists (after removal) >>> OmegaConf.has_resolver("str.lower") False >>> # default resolvers are not affected >>> OmegaConf.has_resolver("oc.env") True ``` -------------------------------- ### OmegaConf.load Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Loads a configuration from a file path or file-like object. ```APIDOC ## OmegaConf.load ### Description Loads a configuration from a YAML file or a file-like object. ### Parameters - **file_** (Union[str, pathlib.Path, IO[Any]]) - Required - The file path or file-like object to load from. ### Returns - **Union[DictConfig, ListConfig]** - The loaded configuration object. ``` -------------------------------- ### Nesting structured configurations Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/structured_config.rst.txt Illustrates how to nest dataclasses and handle type validation for nested assignments. ```python >>> @dataclass ... class User: ... # A simple user class with two missing fields ... name: str = MISSING ... height: Height = MISSING >>> >>> @dataclass ... class DuperUser(User): ... duper: bool = True ... >>> # Group class contains two instances of User. >>> @dataclass ... class Group: ... name: str = MISSING ... # data classes can be nested ... admin: User = field(default_factory=User) ... ... # You can also specify different defaults for nested classes ... manager: User = field(default_factory=lambda: User(name="manager", height=Height.TALL)) >>> conf: Group = OmegaConf.structured(Group) >>> print(OmegaConf.to_yaml(conf)) name: ??? admin: name: ??? height: ??? manager: name: manager height: TALL ``` ```python >>> with raises(ValidationError): ... conf.manager = 10 ``` ```python >>> conf.manager = DuperUser() >>> assert conf.manager.duper == True ``` -------------------------------- ### OmegaConf.from_dotlist Source: https://omegaconf.readthedocs.io/en/2.3_branch/api_reference.html Creates a configuration object from a list of dot-notation strings. ```APIDOC ## OmegaConf.from_dotlist(dotlist: List[str]) -> DictConfig ### Description Creates a configuration object from a list of dotlist-style strings (e.g., ['foo.bar=1']). ### Parameters - **dotlist** (List[str]) - Required - A list of strings in dotlist format. ### Returns - **DictConfig** - The created configuration object. ``` -------------------------------- ### Create configuration from CLI arguments Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Generates a DictConfig from command line arguments or a provided list of strings. ```python @staticmethod def from_cli(args_list: Optional[List[str]] = None) -> DictConfig: if args_list is None: # Skip program name args_list = sys.argv[1:] return OmegaConf.from_dotlist(args_list) ``` -------------------------------- ### OmegaConf.create Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Creates a DictConfig or ListConfig from a dictionary, list, or string. ```APIDOC ## OmegaConf.create ### Description Creates a configuration object from the provided input. The input can be a dictionary, list, or a string representation. ### Parameters - **obj** (Any) - Optional - The input object to convert into a configuration (dict, list, or string). - **parent** (BaseContainer) - Optional - The parent container for the new configuration. - **flags** (Dict[str, bool]) - Optional - Configuration flags to apply. ### Returns - **Union[DictConfig, ListConfig]** - The resulting configuration object. ``` -------------------------------- ### OmegaConf.select Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Selects a config node or value using dot-notation or bracket-notation. Supports a default value if the key is missing. ```APIDOC ## OmegaConf.select(cfg, key, default=None, throw_on_missing=False) ### Description Allows you to select a config node or value, using either a dot-notation or brackets to denote sub-keys. ### Parameters - **cfg** (Any) - The configuration object to search. - **key** (str) - The key path to select. - **default** (Any, optional) - The value to return if the key is missing. - **throw_on_missing** (bool, optional) - If True, raises a MissingMandatoryValue error if the key is missing. ``` -------------------------------- ### Dynamically generate nodes with oc.create Source: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html Use oc.create to convert Python objects or YAML strings into configuration nodes. ```python >>> OmegaConf.register_new_resolver("make_dict", lambda: {"a": 10}) >>> cfg = OmegaConf.create( ... { ... "plain_dict": "${make_dict:}", ... "dict_config": "${oc.create:${make_dict:}}", ... "dict_config_env": "${oc.create:${oc.env:YAML_ENV}}", ... } ... ) >>> os.environ["YAML_ENV"] = "A: 10\nb: 20\nC: ${.A}" >>> show(cfg.plain_dict) # `make_dict` returns a Python dict type: dict, value: {'a': 10} >>> show(cfg.dict_config) # `oc.create` converts it to DictConfig type: DictConfig, value: {'a': 10} >>> show(cfg.dict_config_env) # YAML string to DictConfig type: DictConfig, value: {'A': 10, 'b': 20, 'C': '${.A}'} >>> cfg.dict_config_env.C # interpolations work in a DictConfig 10 ``` -------------------------------- ### Internal configuration creation implementation Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Low-level implementation for creating configuration objects from various input types. ```python @staticmethod def _create_impl( # noqa F811 obj: Any = _DEFAULT_MARKER_, parent: Optional[BaseContainer] = None, flags: Optional[Dict[str, bool]] = None, ) -> Union[DictConfig, ListConfig]: try: from ._utils import get_yaml_loader from .dictconfig import DictConfig from .listconfig import ListConfig if obj is _DEFAULT_MARKER_: obj = {} if isinstance(obj, str): obj = yaml.load(obj, Loader=get_yaml_loader()) if obj is None: return OmegaConf.create({}, parent=parent, flags=flags) elif isinstance(obj, str): return OmegaConf.create({obj: None}, parent=parent, flags=flags) else: assert isinstance(obj, (list, dict)) return OmegaConf.create(obj, parent=parent, flags=flags) ``` -------------------------------- ### Define structured configs with lists and tuples Source: https://omegaconf.readthedocs.io/en/2.3_branch/structured_config.html Shows how to use typing.List and typing.Tuple within dataclasses to enforce types for collection fields. ```python >>> from dataclasses import dataclass, field >>> from typing import List, Tuple >>> @dataclass ... class User: ... name: str = MISSING >>> @dataclass ... class ListsExample: ... # Typed list can hold Any, int, float, bool, str, ... # bytes, pathlib.Path and Enums as well as arbitrary Structured configs. ... ints: List[int] = field(default_factory=lambda: [10, 20, 30]) ... bools: Tuple[bool, bool] = field(default_factory=lambda: (True, False)) ... users: List[User] = field(default_factory=lambda: [User(name="omry")]) ``` -------------------------------- ### oc.create Source: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html Dynamically generate configuration nodes from Python objects or YAML strings. ```APIDOC ## oc.create ### Description Used for dynamic generation of config nodes from Python dictionaries, lists, or YAML strings. ### Syntax `${oc.create:value}` ### Parameters - **value** (string/object) - Required - The input data to be converted into a DictConfig or ListConfig. ``` -------------------------------- ### Define YAML configuration file Source: https://omegaconf.readthedocs.io/en/2.3_branch/structured_config.html A sample YAML configuration file used for merging with structured schemas. ```yaml server: port: 80 log: file: ??? rotation: 3600 users: - user1 - user2 ``` -------------------------------- ### Handling variadic arguments and escaping Source: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html Demonstrates how to pass multiple arguments and use escaping or quotes for special characters. ```python >>> OmegaConf.register_new_resolver("concat", lambda x, y: x+y) >>> c = OmegaConf.create({ ... 'key1': '${concat:Hello,World}', ... 'key_trimmed': '${concat:Hello , World}', ... 'escape_whitespace': '${concat:Hello,\ World}', ... 'quoted': '${concat:"Hello,", " World"}', ... }) >>> c.key1 'HelloWorld' >>> c.key_trimmed 'HelloWorld' >>> c.escape_whitespace 'Hello World' >>> c.quoted 'Hello, World' ``` -------------------------------- ### Static Type Checking with Structured Configs Source: https://omegaconf.readthedocs.io/en/2.3_branch/structured_config.html Demonstrates how structured configurations enable static type checking and validation for attribute assignments. ```python >>> conf: SimpleTypes = OmegaConf.structured(SimpleTypes) >>> # Passes static type checking >>> conf.description = "text" >>> # Fails static type checking (but will also raise a Validation error) >>> with raises(ValidationError): ... conf.num = "foo" ``` -------------------------------- ### Saving and loading pickle Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Uses pickle to persist configuration while retaining type information. ```python >>> conf = OmegaConf.create({"foo": 10, "bar": 20, 123: 456}) >>> with tempfile.TemporaryFile() as fp: ... pickle.dump(conf, fp) ... fp.flush() ... assert fp.seek(0) == 0 ... loaded = pickle.load(fp) ... assert conf == loaded ``` -------------------------------- ### omegaconf.read_write Source: https://omegaconf.readthedocs.io/en/2.3_branch/api_reference.html Context manager to toggle read/write access for a configuration node. ```APIDOC ## omegaconf.read_write() ### Description Provides a generator to manage the read/write state of a configuration node. ``` -------------------------------- ### Merging configurations Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Combines multiple configuration objects into a single instance. ```python conf = OmegaConf.merge(base_cfg, model_cfg, optimizer_cfg, dataset_cfg) ``` ```python conf = OmegaConf.merge(server_cfg, plugin1_cfg, site1_cfg, site2_cfg) ``` ```python >>> from omegaconf import OmegaConf >>> import sys >>> >>> # Simulate command line arguments >>> sys.argv = ['program.py', 'server.port=82'] >>> >>> base_conf = OmegaConf.load('source/example2.yaml') >>> second_conf = OmegaConf.load('source/example3.yaml') >>> cli_conf = OmegaConf.from_cli() >>> >>> # merge them all >>> conf = OmegaConf.merge(base_conf, second_conf, cli_conf) >>> print(OmegaConf.to_yaml(conf)) server: port: 82 users: - user1 - user2 log: file: log.txt ``` -------------------------------- ### Select configuration value Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Selects a value from a configuration node using a key, with options for default values and error handling. ```python @staticmethod def select( cfg: Container, key: str, *, default: Any = _DEFAULT_MARKER_, throw_on_resolution_failure: bool = True, throw_on_missing: bool = False, ) -> Any: """ :param cfg: Config node to select from :param key: Key to select :param default: Default value to return if key is not found :param throw_on_resolution_failure: Raise an exception if an interpolation resolution error occurs, otherwise return None :param throw_on_missing: Raise an exception if an attempt to select a missing key (with the value '???') is made, otherwise return None :return: selected value or None if not found. """ from ._impl import select_value try: return select_value( cfg=cfg, key=key, default=default, throw_on_resolution_failure=throw_on_resolution_failure, throw_on_missing=throw_on_missing, ) except Exception as e: format_and_raise(node=cfg, key=key, value=None, cause=e, msg=str(e)) ``` -------------------------------- ### Manage configuration state with context managers Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Use these context managers to temporarily modify the read-only or struct status of a configuration node. ```python @contextmanager def read_write(config: Node) -> Generator[Node, None, None]: prev_state = config._get_node_flag("readonly") try: OmegaConf.set_readonly(config, False) yield config finally: OmegaConf.set_readonly(config, prev_state) ``` ```python @contextmanager def open_dict(config: Container) -> Generator[Container, None, None]: prev_state = config._get_node_flag("struct") try: OmegaConf.set_struct(config, False) yield config finally: OmegaConf.set_struct(config, prev_state) ``` -------------------------------- ### Select configuration nodes or values Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Use dot-notation or brackets to retrieve values from a configuration object. Supports default values and mandatory value error handling. ```python >>> cfg = OmegaConf.create({ ... "foo" : { ... "missing" : "???", ... "bar": { ... "zonk" : 10, ... } ... } ... }) >>> assert OmegaConf.select(cfg, "foo") == { ... "missing" : "???", ... "bar": { ... "zonk" : 10, ... } ... } >>> assert OmegaConf.select(cfg, "foo.bar") == { ... "zonk" : 10, ... } >>> assert OmegaConf.select(cfg, "foo.bar.zonk") == 10 # dots >>> assert OmegaConf.select(cfg, "foo[bar][zonk]") == 10 # brackets >>> assert OmegaConf.select(cfg, "no_such_key", default=99) == 99 >>> assert OmegaConf.select(cfg, "foo.missing") is None >>> assert OmegaConf.select(cfg, "foo.missing", default=99) == 99 >>> OmegaConf.select(cfg, ... "foo.missing", ... throw_on_missing=True ... ) Traceback (most recent call last): ... omegaconf.errors.MissingMandatoryValue: missing node selected full_key: foo.missing ``` -------------------------------- ### Define Nested Container Annotations Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/structured_config.rst.txt Shows how to nest Dict and List annotations within a structured configuration. ```python >>> @dataclass ... class NestedContainers: ... dict_of_dict: Dict[str, Dict[str, int]] ... list_of_list: List[List[int]] = field(default_factory=lambda: [[123]]) ... dict_of_list: Dict[str, List[int]] = MISSING ... list_of_dict: List[Dict[str, int]] = MISSING ... ... >>> cfg = OmegaConf.structured(NestedContainers(dict_of_dict={"foo": {"bar": 123}})) >>> print(OmegaConf.to_yaml(cfg)) dict_of_dict: foo: bar: 123 list_of_list: - - 123 dict_of_list: ??? list_of_dict: ??? >>> with raises(ValidationError): ... cfg.list_of_dict = [["whoops"]] # not a list of dicts ``` -------------------------------- ### Define a structured config with simple types Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/structured_config.rst.txt Defines a dataclass using various primitive types, Enums, and pathlib.Path to serve as a configuration schema. ```python class Height(Enum): SHORT = 0 TALL = 1 @dataclass class SimpleTypes: num: int = 10 pi: float = 3.1415 is_awesome: bool = True height: Height = Height.SHORT description: str = "text" data: bytes = b"bin_data" path: pathlib.Path = pathlib.Path("hello.txt") ``` -------------------------------- ### OmegaConf.save Source: https://omegaconf.readthedocs.io/en/2.3_branch/api_reference.html Saves a configuration object to a file. ```APIDOC ## OmegaConf.save(config: Any, f: Union[str, Path, IO], resolve: bool = False) -> None ### Description Saves the provided configuration object to a file or file-like object. ### Parameters - **config** (Any) - Required - The configuration object to save. - **f** (Union[str, Path, IO]) - Required - The destination filename or file object. - **resolve** (bool) - Optional - If True, saves the resolved version of the configuration. ``` -------------------------------- ### Serialize configuration to YAML Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Converts a configuration object into a YAML string representation. ```python @staticmethod def to_yaml(cfg: Any, *, resolve: bool = False, sort_keys: bool = False) -> str: """ returns a yaml dump of this config object. :param cfg: Config object, Structured Config type or instance :param resolve: if True, will return a string with the interpolations resolved, otherwise interpolations are preserved :param sort_keys: If True, will print dict keys in sorted order. default False. :return: A string containing the yaml representation. """ cfg = _ensure_container(cfg) container = OmegaConf.to_container(cfg, resolve=resolve, enum_to_str=True) return yaml.dump( # type: ignore container, default_flow_style=False, allow_unicode=True, sort_keys=sort_keys, Dumper=get_omega_conf_dumper(), ) ``` -------------------------------- ### Using oc.dict.keys and oc.dict.values Source: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html Converts DictConfig keys or values into ListConfig objects for easier manipulation. ```python >>> cfg = OmegaConf.create( ... { ... "workers": { ... "node3": "10.0.0.2", ... "node7": "10.0.0.9", ... }, ... "nodes": "${oc.dict.keys: workers}", ... "ips": "${oc.dict.values: workers}", ... } ... ) >>> # Keys are copied from the DictConfig: >>> show(cfg.nodes) type: ListConfig, value: ['node3', 'node7'] >>> # Values are dynamically fetched through interpolations: >>> show(cfg.ips) type: ListConfig, value: ['${workers.node3}', '${workers.node7}'] >>> assert cfg.ips == ["10.0.0.2", "10.0.0.9"] ``` -------------------------------- ### Convert to Object Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Recursively converts config objects to plain Python types, resolving interpolations and instantiating Structured Configs. ```python >>> container = OmegaConf.to_object(conf) >>> show(container) type: dict, value: {'structured_config': MyConfig(port=80, host='localhost')} >>> show(container["structured_config"]) type: MyConfig, value: MyConfig(port=80, host='localhost') ``` -------------------------------- ### Set configuration value implementation Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Internal logic for setting values in a configuration container, handling structural overrides and merging. ```python assert isinstance( root, Container ), f"Unexpected type for root: {type(root).__name__}" last_key: Union[str, int] = last if isinstance(root, ListConfig): last_key = int(last) ctx = flag_override(root, "struct", False) if force_add else nullcontext() with ctx: if merge and (OmegaConf.is_config(value) or is_primitive_container(value)): assert isinstance(root, BaseContainer) node = root._get_child(last_key) if OmegaConf.is_config(node): assert isinstance(node, BaseContainer) node.merge_with(value) return if OmegaConf.is_dict(root): assert isinstance(last_key, str) root.__setattr__(last_key, value) elif OmegaConf.is_list(root): assert isinstance(last_key, int) root.__setitem__(last_key, value) else: assert False ``` -------------------------------- ### Configure Read-Only and Struct Flags Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Methods to toggle and check the read-only and struct status of configuration nodes. ```python @staticmethod def set_readonly(conf: Node, value: Optional[bool]) -> None: # noinspection PyProtectedMember conf._set_flag("readonly", value) ``` ```python @staticmethod def is_readonly(conf: Node) -> Optional[bool]: # noinspection PyProtectedMember return conf._get_flag("readonly") ``` ```python @staticmethod def set_struct(conf: Container, value: Optional[bool]) -> None: # noinspection PyProtectedMember conf._set_flag("struct", value) ``` ```python @staticmethod def is_struct(conf: Container) -> Optional[bool]: # noinspection PyProtectedMember return conf._get_flag("struct") ``` -------------------------------- ### Save configuration to file Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Saves a DictConfig or ListConfig object to a file, optionally resolving variables. ```python @staticmethod def save( config: Any, f: Union[str, pathlib.Path, IO[Any]], resolve: bool = False ) -> None: """ Save as configuration object to a file :param config: omegaconf.Config object (DictConfig or ListConfig). :param f: filename or file object :param resolve: True to save a resolved config (defaults to False) """ if is_dataclass(config) or is_attr_class(config): config = OmegaConf.create(config) data = OmegaConf.to_yaml(config, resolve=resolve) if isinstance(f, (str, pathlib.Path)): with io.open(os.path.abspath(f), "w", encoding="utf-8") as file: file.write(data) elif hasattr(f, "write"): f.write(data) f.flush() else: raise TypeError("Unexpected file type") ``` -------------------------------- ### omegaconf.open_dict Source: https://omegaconf.readthedocs.io/en/2.3_branch/api_reference.html Context manager to allow modification of a dictionary-like configuration container. ```APIDOC ## omegaconf.open_dict() ### Description Allows adding new keys to a configuration container that is otherwise restricted. ``` -------------------------------- ### OmegaConf.to_yaml Source: https://omegaconf.readthedocs.io/en/2.3_branch/api_reference.html Returns a YAML dump of the provided config object. ```APIDOC ## OmegaConf.to_yaml(cfg, resolve=False, sort_keys=False) ### Description Returns a YAML dump of this config object. ### Parameters - **cfg** (Any) - Required - Config object, Structured Config type or instance. - **resolve** (bool) - Optional - If True, will return a string with the interpolations resolved, otherwise interpolations are preserved. - **sort_keys** (bool) - Optional - If True, will print dict keys in sorted order. Default False. ### Returns A string containing the yaml representation. ``` -------------------------------- ### Accessing underlying configuration type Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/structured_config.rst.txt Demonstrates how to retrieve the actual type of a configuration object using OmegaConf.get_type(). ```python >>> type(conf).__name__ 'DictConfig' >>> OmegaConf.get_type(conf).__name__ 'SimpleTypes' ``` -------------------------------- ### Working with Frozen Classes Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/structured_config.rst.txt Frozen dataclasses are supported, making the configuration node and its children read-only. ```python >>> from dataclasses import dataclass, field >>> from typing import List >>> @dataclass(frozen=True) ... class FrozenClass: ... x: int = 10 ... list: List = field(default_factory=lambda: [1, 2, 3]) >>> conf = OmegaConf.structured(FrozenClass) >>> with raises(ReadonlyConfigError): ... conf.x = 20 ``` ```python >>> with raises(ReadonlyConfigError): ... conf.list[0] = 20 ``` -------------------------------- ### Define Interpolation Helpers Source: https://omegaconf.readthedocs.io/en/2.3_branch/_modules/omegaconf/omegaconf.html Utility functions for creating interpolation strings within configuration values. ```python def II(interpolation: str) -> Any: """ Equivalent to ``${interpolation}`` :param interpolation: :return: input ``${node}`` with type Any """ return "${" + interpolation + "}" ``` ```python def SI(interpolation: str) -> Any: """ Use this for String interpolation, for example ``"http://${host}:${port}"`` :param interpolation: interpolation string :return: input interpolation with type ``Any`` """ return interpolation ``` -------------------------------- ### Apply Field Modifiers Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/structured_config.rst.txt Demonstrates the use of MISSING and Optional modifiers within a structured configuration. ```python >>> from typing import Optional >>> from omegaconf import MISSING >>> @dataclass ... class Modifiers: ... num: int = 10 ... optional_num: Optional[int] = 10 ... another_num: int = MISSING ... optional_dict: Optional[Dict[str, int]] = None ... list_optional: List[Optional[int]] = field(default_factory=lambda: [10, MISSING, None]) >>> conf: Modifiers = OmegaConf.structured(Modifiers) ``` -------------------------------- ### Saving and loading YAML Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Persists configuration to a YAML file; note that type information is not preserved. ```python >>> conf = OmegaConf.create({"foo": 10, "bar": 20, 123: 456}) >>> with tempfile.NamedTemporaryFile() as fp: ... OmegaConf.save(config=conf, f=fp.name) ... loaded = OmegaConf.load(fp.name) ... assert conf == loaded ``` -------------------------------- ### Accessing Environment Variables with oc.env Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/custom_resolvers.rst.txt Use oc.env to interpolate environment variables into configuration values, with optional fallback defaults. ```python >>> conf = OmegaConf.load('source/env_interpolation.yaml') >>> conf.user.name 'omry' >>> conf.user.home '/home/omry' ``` ```python >>> cfg = OmegaConf.create( ... { ... "database": { ... "password1": "${oc.env:DB_PASSWORD,password}", ... "password2": "${oc.env:DB_PASSWORD,12345}", ... "password3": "${oc.env:DB_PASSWORD,null}", ... }, ... } ... ) >>> # default is already a string >>> show(cfg.database.password1) type: str, value: 'password' >>> # default is converted to a string automatically >>> show(cfg.database.password2) type: str, value: '12345' >>> # unless it's None >>> show(cfg.database.password3) type: NoneType, value: None ``` -------------------------------- ### Registering a new resolver Source: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html The signature for registering a custom resolver function. ```python def register_new_resolver( name: str, resolver: Resolver, *, replace: bool = False, use_cache: bool = False, ) -> None ``` -------------------------------- ### Retrieve missing keys Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Returns a set of all keys in the configuration that are currently set to missing (???). ```python >>> missings = OmegaConf.missing_keys({ ... "foo": {"bar": "???"}, ... "missing": "???", ... "list": ["a", None, "???"] ... }) >>> assert missings == {'list[2]', 'foo.bar', 'missing'} ``` -------------------------------- ### Assign subclasses to structured fields Source: https://omegaconf.readthedocs.io/en/2.3_branch/structured_config.html Demonstrates that subclasses of the defined type can be assigned to structured configuration fields. ```python >>> conf.manager = DuperUser() >>> assert conf.manager.duper == True ``` -------------------------------- ### oc.deprecated Source: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html Mark a configuration node as deprecated and provide a migration path. ```APIDOC ## oc.deprecated ### Description Enables deprecation warnings when a specific config node is accessed. ### Syntax `${oc.deprecated:key,message}` ### Parameters - **key** (string) - Required - The new key to migrate to. - **message** (string) - Optional - A custom warning message. Defaults to '$OLD_KEY is deprecated. Change your code and config to use '$NEW_KEY''. ``` -------------------------------- ### OmegaConf.merge Source: https://omegaconf.readthedocs.io/en/2.3_branch/api_reference.html Merges multiple configuration objects into a single one. ```APIDOC ## OmegaConf.merge(*configs) -> Union[ListConfig, DictConfig] ### Description Merges a list of previously created configuration objects into a single configuration. ### Parameters - **configs** (Union[DictConfig, ListConfig, Dict, List, Any]) - Required - The input configurations to merge. ### Returns - **Union[ListConfig, DictConfig]** - The resulting merged configuration object. ``` -------------------------------- ### Runtime Type Validation Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Demonstrates how structured configs enforce type constraints. ```python >>> conf.port = 42 # Ok, type matches >>> conf.port = "1080" # Ok! "1080" can be converted to an int >>> conf.port = "oops" # "oops" cannot be converted to an int Traceback (most recent call last): ... omegaconf.errors.ValidationError: Value 'oops' could not be converted to Integer ``` -------------------------------- ### Caching resolver output Source: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html Use use_cache=True to cache results based on the input string literals. ```python >>> import random >>> random.seed(1234) >>> OmegaConf.register_new_resolver( ... "cached", random.randint, use_cache=True ... ) >>> OmegaConf.register_new_resolver("uncached", random.randint) >>> c = OmegaConf.create( ... { ... "uncached": "${uncached:0,10000}", ... "cached_1": "${cached:0,10000}", ... "cached_2": "${cached:0, 10000}", ... "cached_3": "${cached:0,${uncached}}", ... } ... ) >>> # not the same since the cache is disabled by default >>> assert c.uncached != c.uncached >>> # same value on repeated access thanks to the cache >>> assert c.cached_1 == c.cached_1 == 122 >>> # same input as `cached_1` => same value >>> assert c.cached_2 == c.cached_1 == 122 >>> # same string literal "${uncached}" => same value >>> assert c.cached_3 == c.cached_3 == 1192 ``` -------------------------------- ### omegaconf.II Source: https://omegaconf.readthedocs.io/en/2.3_branch/api_reference.html Creates an interpolation equivalent to ${interpolation}. ```APIDOC ## omegaconf.II(interpolation) ### Description Equivalent to ${interpolation}. ### Parameters - **interpolation** (Any) - Required - The interpolation string. ### Returns - (Any) - The input node with type Any. ``` -------------------------------- ### Define Structured Config with Dictionaries Source: https://omegaconf.readthedocs.io/en/2.3_branch/structured_config.html Use typing.Dict to define dictionary fields in a dataclass, ensuring keys and values adhere to supported types. ```python >>> from dataclasses import dataclass, field >>> from typing import Dict >>> @dataclass ... class DictExample: ... ints: Dict[str, int] = field(default_factory=lambda: {"a": 10, "b": 20, "c": 30}) ... bools: Dict[str, bool] = field(default_factory=lambda: {"Uno": True, "Zoro": False}) ... users: Dict[str, User] = field(default_factory=lambda: {"omry": User(name="omry")}) ``` -------------------------------- ### OmegaConf.missing_keys Source: https://omegaconf.readthedocs.io/en/2.3_branch/_sources/usage.rst.txt Returns a set of missing keys present in the input configuration. ```APIDOC ## OmegaConf.missing_keys(cfg) ### Description Returns a set of missing keys present in the input cfg, represented as a dotlist style string. ### Parameters - **cfg** (Any) - The configuration object to inspect. ```