### Merge Server, Plugin, and Site Configurations Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Example of merging server and site-specific configurations. ```python conf = OmegaConf.merge(server_cfg, plugin1_cfg, site1_cfg, site2_cfg) ``` -------------------------------- ### Install OmegaConf Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Install the OmegaConf library using pip. Ensure you are using Python 3.8 or newer. ```bash pip install omegaconf ``` -------------------------------- ### Install OmegaConf Source: https://omegaconf.readthedocs.io/en/latest/usage.html Install the OmegaConf library using pip. This command is the standard way to add the package to your Python environment. ```bash pip install omegaconf ``` -------------------------------- ### String Interpolation Example Source: https://omegaconf.readthedocs.io/en/latest/_sources/grammar.rst.txt Demonstrates basic string interpolation in OmegaConf. The input string is resolved to its final value. ```python '\''Hi you\', I said\'' resolves to the string '\''\''Hi you\', I said\''\'' ``` -------------------------------- ### Installing omegaconf-pydevd Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Provides the command to install the pydevd plugin for enhanced debugger integration with OmegaConf. ```bash pip install omegaconf-pydevd ``` -------------------------------- ### Providing Default Values in OmegaConf Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Shows how to use the `get` method to provide a default value for a potentially missing key in the configuration. ```python >>> conf.get('missing_key', 'a default value') 'a default value' ``` -------------------------------- ### Create Empty OmegaConf Object Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Create an empty OmegaConf configuration object. This is useful as a starting point. ```python from omegaconf import OmegaConf conf = OmegaConf.create() print(OmegaConf.to_yaml(conf)) ``` -------------------------------- ### Example of Eager Interpolation Resolution Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates how OmegaConf.resolve() eagerly resolves interpolations, changing the config in-place. ```python cfg = OmegaConf.create({"a": 10, "b": "${a}"}) show(cfg) assert cfg.a == cfg.b == 10 # lazily resolving interpolation OmegaConf.resolve(cfg) show(cfg) ``` -------------------------------- ### Finding Missing Keys in Configuration Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates how to use OmegaConf.missing_keys to get a set of all missing keys within a configuration object. ```python missings = OmegaConf.missing_keys({ "foo": {"bar": "???"}, "missing": "???", "list": ["a", None, "???"] }) assert missings == {'list[2]', 'foo.bar', 'missing'} ``` -------------------------------- ### Merge Base, Model, Optimizer, and Dataset Configurations Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Example of merging multiple configuration objects for machine learning experiments. ```python conf = OmegaConf.merge(base_cfg, model_cfg, optimizer_cfg, dataset_cfg) ``` -------------------------------- ### Basic Custom Resolver Example Source: https://omegaconf.readthedocs.io/en/latest/custom_resolvers.html Register a simple resolver that adds 10 to its input. This demonstrates the basic usage of registering a lambda function as a resolver. ```python >>> OmegaConf.register_resolver("plus_10", lambda x: x + 10) >>> c = OmegaConf.create({'key': '${plus_10:990}'}) >>> c.key 1000 ``` -------------------------------- ### omegaconf.SI() Source: https://omegaconf.readthedocs.io/en/latest/api_reference.html Use this for String interpolation, for example `"http://${host}:${port}"`. ```APIDOC ## omegaconf.SI() ### Description Use this for String interpolation, for example `"http://${host}:${port}"`. ### Parameters * **interpolation** (str) - interpolation string ### Returns * input interpolation with type `Any` ``` -------------------------------- ### YAML Flow Style Serialization with OmegaConf Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Example of using `OmegaConf.to_yaml` with `default_flow_style=None` to keep nested collections compact in the output. ```python >>> conf = OmegaConf.create({"nhood": [[-1, 0, 0], [0, -1, 0], [0, 0, -1]]}) >>> print(OmegaConf.to_yaml(conf, default_flow_style=None), end="") nhood: - [-1, 0, 0] - [0, -1, 0] - [0, 0, -1] ``` -------------------------------- ### Perform Arithmetic with eval Source: https://omegaconf.readthedocs.io/en/latest/_sources/how_to_guides.rst.txt Use the 'eval' resolver to perform arithmetic operations directly in your configuration. This example calculates 10 squared. ```yaml ten_squared: ${eval:'10 ** 2'} ``` ```python cfg = OmegaConf.create({ "ten_squared": "${eval:'10 ** 2'}", }) ``` -------------------------------- ### Create Structured Config from Class and Instance Source: https://omegaconf.readthedocs.io/en/latest/structured_config.html Demonstrates creating OmegaConf configurations from a structured class and an instance of that class. The instance-based creation allows for easy customization of field values during construction. ```python >>> conf1 = OmegaConf.structured(SimpleTypes) >>> conf2 = OmegaConf.structured(SimpleTypes()) >>> # The two configs are identical in this case >>> assert conf1 == conf2 ``` ```python >>> # But the second form allow for easy customization of the values: >>> conf3 = OmegaConf.structured( ... SimpleTypes(num=20, ... height=Height.TALL)) >>> print(OmegaConf.to_yaml(conf3)) num: 20 pi: 3.1415 is_awesome: true height: TALL description: text data: !!binary |- YmluX2RhdGE= path: !!python/object/apply:pathlib.PosixPath - hello.txt ``` -------------------------------- ### Structured Config Initialization and Basic Assignment Source: https://omegaconf.readthedocs.io/en/latest/_sources/structured_config.rst.txt Demonstrates initializing a structured config and assigning values. It shows how static type checking works and how runtime validation catches type errors. ```python from omegaconf import OmegaConf, ValidationError from dataclasses import dataclass, field from enum import Enum from typing import List, Tuple, Literal class Height(Enum): TALL = 1 SHORT = 2 class SimpleTypes: description: str num: int conf: SimpleTypes = OmegaConf.structured(SimpleTypes) # Passes static type checking conf.description = "text" # Fails static type checking (but will also raise a Validation error) with raises(ValidationError): conf.num = "foo" ``` -------------------------------- ### Create and Access Interpolated Configuration Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates creating a configuration with interpolation and accessing its values. ```python >>> cfg = OmegaConf.create( ... { ... "john": {"height": 180, "weight": 75}, ... "player": "${john}", ... } ... ) >>> (cfg.player.height, cfg.player.weight) (180, 75) ``` -------------------------------- ### Using oc.env with default values Source: https://omegaconf.readthedocs.io/en/latest/custom_resolvers.html Demonstrates using oc.env with default values, including handling null defaults. The default value is converted to a string unless it is null. ```python >>> conf = OmegaConf.load('source/env_interpolation.yaml') >>> conf.user.name 'omry' >>> conf.user.home '/home/omry' ``` ```python >>> cfg = OmegaConf.create( ... { ... "database": { ... "password1": "${oc.env:DB_PASSWORD,password}", ... "password2": "${oc.env:DB_PASSWORD,12345}", ... "password3": "${oc.env:DB_PASSWORD,null}", ... }, ... } ... ) >>> # default is already a string >>> show(cfg.database.password1) type: str, value: 'password' >>> # default is converted to a string automatically >>> show(cfg.database.password2) type: str, value: '12345' >>> # unless it's None >>> show(cfg.database.password3) type: NoneType, value: None ``` -------------------------------- ### Create OmegaConf with Various Key Types Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates creating an OmegaConf object with various supported key types including strings, integers, booleans, floats, Enums, and bytes. ```python from enum import Enum class Color(Enum): RED = 1 BLUE = 2 conf = OmegaConf.create( {"key": "str", 123: "int", True: "bool", 3.14: "float", Color.RED: "Color", b"123": "bytes"} ) print(conf) ``` -------------------------------- ### Create OmegaConf from Command Line Arguments Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Parse command-line arguments to create an OmegaConf object. Simulates sys.argv for demonstration. ```python # Simulating command line arguments sys.argv = ['your-program.py', 'server.port=82', 'log.file=log2.txt'] conf = OmegaConf.from_cli() print(OmegaConf.to_yaml(conf)) ``` -------------------------------- ### Using Environment Variables with OmegaConf Source: https://omegaconf.readthedocs.io/en/latest/_sources/custom_resolvers.rst.txt Demonstrates how to use environment variables with OmegaConf interpolations, including decoding and providing default values. ```python >>> cfg = OmegaConf.create( ... { ... "database": { ... "port": '${oc.decode:${oc.env:DB_PORT}}', ... "nodes": '${oc.decode:${oc.env:DB_NODES}}', ... "timeout": '${oc.decode:${oc.env:DB_TIMEOUT,null}}', ... } ... } ... ) >>> os.environ["DB_PORT"] = "3308" >>> show(cfg.database.port) # converted to int type: int, value: 3308 >>> os.environ["DB_NODES"] = "[host1, host2, host3]" >>> show(cfg.database.nodes) # converted to a Python list type: list, value: ['host1', 'host2', 'host3'] >>> show(cfg.database.timeout) # keeping `None` as is type: NoneType, value: None >>> os.environ["DB_TIMEOUT"] = "${.port}" >>> show(cfg.database.timeout) # resolving interpolation type: int, value: 3308 ``` -------------------------------- ### Get Resolver Cache Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Retrieves the resolver cache associated with a given OmegaConf container. The cache stores resolved values for resolvers. ```python from omegaconf import OmegaConf, DictConfig conf = DictConfig({'a': '${env:HOME}'}) cache = OmegaConf.get_cache(conf) print(cache) ``` -------------------------------- ### Using oc.dict.keys and oc.dict.values Source: https://omegaconf.readthedocs.io/en/latest/custom_resolvers.html Demonstrates how oc.dict.keys and oc.dict.values can extract keys or values from a DictConfig as a ListConfig. This simplifies manipulation of dictionary-like configurations. ```python >>> cfg = OmegaConf.create( ... { ... "workers": { ... "node3": "10.0.0.2", ... "node7": "10.0.0.9", ... }, ... "nodes": "${oc.dict.keys: workers}", ... "ips": "${oc.dict.values: workers}", ... } ... ) >>> # Keys are copied from the DictConfig: >>> show(cfg.nodes) type: ListConfig, value: ['node3', 'node7'] >>> # Values are dynamically fetched through interpolations: >>> show(cfg.ips) type: ListConfig, value: ['${workers.node3}', '${workers.node7}'] >>> assert cfg.ips == ["10.0.0.2", "10.0.0.9"] ``` -------------------------------- ### Nested Arithmetic with eval Source: https://omegaconf.readthedocs.io/en/latest/_sources/how_to_guides.rst.txt Perform calculations involving other configuration values using nested interpolation with the 'eval' resolver. This example calculates the area of a rectangle. ```yaml side_1: 5 side_2: 6 rectangle_area: ${eval:'${side_1} * ${side_2}'} ``` ```python cfg = OmegaConf.create({ "side_1": 5, "side_2": 6, "rectangle_area": "${eval:'${side_1} * ${side_2}'}", }) ``` -------------------------------- ### Type Coercion with Structured Configs Source: https://omegaconf.readthedocs.io/en/latest/structured_config.html Normally, values assigned to structured config fields are coerced to the field's type. This example shows an integer being converted to a string. ```python @dataclass class HasStr: s: str ``` ```python cfg = OmegaConf.structured(HasStr) cfg.s = 10.1 assert cfg.s == "10.1" # The assigned value has been converted to a string ``` -------------------------------- ### Perform Arithmetic with Nested Interpolation Source: https://omegaconf.readthedocs.io/en/latest/how_to_guides.html Use nested interpolation with the 'eval' resolver to perform calculations involving other configuration values, like multiplying side_1 and side_2 to get the rectangle_area. ```yaml side_1: 5 side_2: 6 rectangle_area: ${eval:'${side_1} * ${side_2}'} ``` ```python cfg = OmegaConf.create({ "side_1": 5, "side_2": 6, "rectangle_area": "${eval:'${side_1} * ${side_2}'}", }) assert cfg.rectangle_area == 30 ``` -------------------------------- ### select Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Selects a value from a configuration using a key path. ```APIDOC ## select ### Description Select a value from a config using a key path. The key path uses dot notation (``"a.b.c"``) or bracket notation (``"a[b][c]"``), or a mix of both. **Keys containing special characters** (``.``, ``[``, ``]``, ``=``) can be expressed by escaping them with a backslash: - ``r"a\.b"`` — selects the key ``"a.b"`` (single key with a literal dot) - ``r"a\[0\]"`` — selects the key ``"a[0]"`` - ``r"a\=b"`` — selects the key ``"a=b"`` A backslash before any other character passes through unchanged (``r"a\b"`` selects the key ``"a\\b"`` — a backslash followed by ``b``). ### Parameters - **cfg** (Container) - Config node to select from - **key** (str) - Key path to select (dot/bracket notation, backslash-escapable) - **default** (Any) - Default value to return if key is not found. Defaults to a marker value. - **throw_on_resolution_failure** (bool) - Raise an exception if an interpolation resolution error occurs, otherwise return None. Defaults to True. - **throw_on_missing** (bool) - Raise an exception if an attempt to select a missing key (with the value '???') is made, otherwise return None. Defaults to False. ### Returns - **Any** - selected value or None if not found. ``` -------------------------------- ### Value Coercion in Structured Configs Source: https://omegaconf.readthedocs.io/en/latest/_sources/structured_config.rst.txt OmegaConf automatically coerces assigned values to the field's declared type when possible. For example, an integer assigned to a string field will be converted to a string. ```python from dataclasses import dataclass from omegaconf import OmegaConf @dataclass class HasStr: s: str cfg = OmegaConf.structured(HasStr) cfg.s = 10.1 assert cfg.s == "10.1" # The assigned value has been converted to a string ``` -------------------------------- ### omegaconf.omegaconf methods Source: https://omegaconf.readthedocs.io/en/latest/genindex.html Instance methods available on OmegaConf objects. ```APIDOC ## omegaconf Methods ### `flag_override()` Flags an override for a configuration value. ### `II()` An interpolation helper function. ### `open_dict()` Opens the configuration for modification in a context manager. ### `read_write()` Enables read-write mode for the configuration. ### `SI()` An interpolation helper function. ``` -------------------------------- ### Resolving Ambiguity with Explicitly Typed Containers Source: https://omegaconf.readthedocs.io/en/latest/structured_config.html Use OmegaConf.typed_list() or OmegaConf.typed_dict() to create explicitly typed containers and resolve ambiguity in union assignments. This example uses typed_list to assign an empty list. ```python cfg.value = OmegaConf.typed_list([], element_type=str) assert cfg.value == [] cfg.value.append("hello") assert cfg.value == ["hello"] ``` -------------------------------- ### OmegaConf._from_cli Source: https://omegaconf.readthedocs.io/en/latest/api_reference.html Creates a DictConfig object from command-line arguments provided as a list of strings. ```APIDOC ## OmegaConf._from_cli ### Description Create a config from command-line arguments (`sys.argv[1:]` by default). Each argument must be a dotlist-style string such as `"foo.bar=1"`. ### Parameters #### Path Parameters - **args_list** (List[str] | None) - Optional - Explicit list of dotlist strings; defaults to `sys.argv[1:]`. ### Returns A `DictConfig` built from the arguments. ``` -------------------------------- ### Pass String Data to eval Source: https://omegaconf.readthedocs.io/en/latest/how_to_guides.html When passing string data to eval, use a nested pair of quotes to ensure the data is interpreted as a string literal. This example demonstrates multiplying a string by an integer. ```yaml cow_say: moo three_cows: ${eval:'3 * "${cow_say}"'} ``` ```python cfg = OmegaConf.create({ "cow_say": "moo", "three_cows": "${eval:'3 * \"${cow_say}\"'} " }) assert cfg.three_cows == "moomoomoo" ``` -------------------------------- ### Type Validation with Interpolated Values Source: https://omegaconf.readthedocs.io/en/latest/_sources/structured_config.rst.txt Interpolated values are validated and converted to the annotated type when accessed. This example shows a type mismatch that fails validation, and then how assigning a compatible string allows successful conversion. ```python from omegaconf import II from dataclasses import dataclass @dataclass class Interpolation: str_key: str = "string" int_key: int = II("str_key") cfg = OmegaConf.structured(Interpolation) # cfg.int_key # fails due to type mismatch # omegaconf.errors.InterpolationValidationError: Value 'string' could not be converted to Integer # full_key: int_key # object_type=Interpolation cfg.str_key = "1234" # string value assert cfg.int_key == 1234 # automatically convert str to int ``` -------------------------------- ### Handling Ambiguous Union Assignments Source: https://omegaconf.readthedocs.io/en/latest/structured_config.html When a value is valid for more than one union member (e.g., an empty container), OmegaConf raises a ValidationError. This example shows an ambiguous assignment to Union[List[int], List[str]]. ```python @dataclass class ListUnion: value: Union[List[int], List[str]] = field(default_factory=lambda: [1]) cfg = OmegaConf.structured(ListUnion) cfg.value = [] ``` -------------------------------- ### Create Structured Config from Instance Source: https://omegaconf.readthedocs.io/en/latest/_sources/structured_config.rst.txt Create an OmegaConf configuration object from an instance of a structured config class. This allows for setting initial values for specific fields during construction. ```python from omegaconf import OmegaConf conf2 = OmegaConf.structured(SimpleTypes()) ``` -------------------------------- ### OmegaConf.from_cli Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Creates a DictConfig from command-line arguments, typically sys.argv[1:]. Arguments should be in dotlist format (e.g., 'foo.bar=1'). ```APIDOC ## OmegaConf.from_cli ### Description Creates a DictConfig from command-line arguments, typically sys.argv[1:]. Arguments should be in dotlist format (e.g., 'foo.bar=1'). ### Parameters #### Path Parameters - **args_list** (Optional[List[str]], optional) - Explicit list of dotlist strings. Defaults to ``sys.argv[1:]``. ### Returns - **DictConfig**: A DictConfig built from the provided arguments. ``` -------------------------------- ### Type Validation After Union Branch Selection Source: https://omegaconf.readthedocs.io/en/latest/structured_config.html Once a union branch is selected, the field is fully typed and rejects values that violate the selected container's element type. This example shows appending an incompatible type to a List[int]. ```python cfg.value = [1, 2] cfg.value.append(1) # ok — List[int] cfg.value.append("x") # not ok — str is not int ``` -------------------------------- ### Merging Structured Configs with Type Validation Source: https://omegaconf.readthedocs.io/en/latest/_sources/structured_config.rst.txt OmegaConf configs created from structured configs enforce type information at runtime. This example demonstrates merging a loaded config with a schema, where an intentional type error in the schema causes a ValidationError. ```yaml server: port: 8080 log: file: "/var/log/app.log" rotation: 10 users: - 1 - 2 ``` ```python from dataclasses import dataclass, field from typing import List from omegaconf import OmegaConf, MISSING from omegaconf.errors import ValidationError @dataclass class Server: port: int = MISSING @dataclass class Log: file: str = MISSING rotation: int = MISSING @dataclass class MyConfig: server: Server = field(default_factory=Server) log: Log = field(default_factory=Log) users: List[int] = field(default_factory=list) schema = OmegaConf.structured(MyConfig) conf = OmegaConf.load("source/example.yaml") # with raises(ValidationError): # OmegaConf.merge(schema, conf) ``` -------------------------------- ### Creating and Printing a DictConfig Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates the creation of a DictConfig object and its YAML representation. ```python conf = OmegaConf.create({"a": {"b": 10}, "c":20}) print(OmegaConf.to_yaml(conf)) ``` -------------------------------- ### Get the type of an OmegaConf object or its child Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Retrieve the Python type of an OmegaConf node or a specific child element identified by a key. For structured configs, it returns the underlying dataclass or attrs class; for plain containers, it returns dict or list. ```python from typing import Any, Optional, Type from omegaconf import Container @staticmethod def get_type(obj: Any, key: Optional[str] = None) -> Optional[Type[Any]]: """ Return the type of ``obj``, or of ``obj[key]`` when ``key`` is provided. For structured configs this is the underlying dataclass or attrs class. For plain containers it is ``dict`` or ``list``. :param obj: An OmegaConf node or container. :param key: Optional key within ``obj`` to inspect. :return: The Python type, or ``None`` if not determinable. """ if key is not None: c = obj._get_child(key) else: c = obj return OmegaConf._get_obj_type(c) ``` -------------------------------- ### Load OmegaConf from YAML File Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Load an OmegaConf configuration directly from a YAML file. The output is identical to the file's content. ```python conf = OmegaConf.load('source/example.yaml') print(OmegaConf.to_yaml(conf)) ``` -------------------------------- ### Assigning to Union[str, float] Source: https://omegaconf.readthedocs.io/en/latest/_sources/structured_config.rst.txt When assigning values to a Union-typed field, the value must precisely match one of the types in the Union annotation. Conversion is disabled to avoid ambiguity. For example, assigning an int to a Union[str, float] field will raise a ValidationError. ```python from typing import Union from dataclasses import dataclass from omegaconf import OmegaConf @dataclass class StrOrInt: u: Union[str, float] cfg = OmegaConf.structured(StrOrInt) cfg.u = 10.1 assert cfg.u == 10.1 cfg.u = "10.1" assert cfg.u == "10.1" # cfg.u = 123 # Conversion from `int` to `float` does not occur. # Traceback (most recent call last): # ... # omegaconf.errors.ValidationError: Value '123' of type 'int' is incompatible with type hint 'Union[str, float]' # full_key: u # object_type=StrOrInt ``` -------------------------------- ### OmegaConf.create (list/tuple overload) Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Creates an OmegaConf ListConfig object from a list or tuple. ```APIDOC ## OmegaConf.create (list/tuple overload) ### Description Creates an OmegaConf ListConfig object from a list or tuple. ### Method Signature ```python @staticmethod @overload create(obj: Union[List[Any], Tuple[Any, ...]], parent: Optional[BaseContainer] = None, flags: Optional[Dict[str, bool]] = None, *, max_yaml_expanded_nodes: Optional[int] = _DEFAULT_MAX_YAML_EXPANDED_NODES) -> ListConfig ``` ### Parameters * **obj** (Union[List[Any], Tuple[Any, ...]]) - The list or tuple to convert into a ListConfig. * **parent** (Optional[BaseContainer]) - Optional parent node. * **flags** (Optional[Dict[str, bool]]) - Optional flags dict (e.g. ``{"readonly": True}``). * **max_yaml_expanded_nodes** (Optional[int]) - Maximum YAML nodes after alias expansion when ``obj`` is a YAML string. By default, OmegaConf uses the ``OMEGACONF_MAX_YAML_EXPANDED_NODES`` environment variable if set, otherwise ``10_000``. Explicit arguments override the environment. Pass ``None`` only for trusted input. ### Returns A ``ListConfig`` instance. ``` -------------------------------- ### Merge Configurations from Files and CLI Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates merging configurations loaded from YAML files and command-line arguments. The port is updated via CLI. ```python >>> from omegaconf import OmegaConf >>> import sys >>> >>> # Simulate command line arguments >>> sys.argv = ['program.py', 'server.port=82'] >>> >>> base_conf = OmegaConf.load('source/example2.yaml') >>> second_conf = OmegaConf.load('source/example3.yaml') >>> cli_conf = OmegaConf.from_cli() >>> >>> # merge them all >>> conf = OmegaConf.merge(base_conf, second_conf, cli_conf) >>> print(OmegaConf.to_yaml(conf)) ``` -------------------------------- ### Create OmegaConf DictConfig from various inputs Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Use the create method to instantiate a DictConfig from different object types like dictionaries, existing configs, or None. It supports optional parent, flags, and YAML expansion limits. ```python from omegaconf import OmegaConf # Example usage (assuming DictConfig is defined elsewhere) # config = OmegaConf.create({'key': 'value'}) # print(config) # Example with None # empty_config = OmegaConf.create(None) # print(empty_config) # Example with existing DictConfig # existing_cfg = OmegaConf.create({'a': 1}) # new_cfg = OmegaConf.create(existing_cfg) # print(new_cfg) # Example with flags # read_only_cfg = OmegaConf.create({'b': 2}, flags={'readonly': True}) # print(read_only_cfg.is_readonly()) # Example with max_yaml_expanded_nodes # cfg_with_limit = OmegaConf.create('a: 1', max_yaml_expanded_nodes=5000) # print(cfg_with_limit) # The actual implementation of create: @staticmethod def create( obj: Any = _DEFAULT_MARKER_, parent: Optional[BaseContainer] = None, flags: Optional[Dict[str, bool]] = None, *, max_yaml_expanded_nodes: Optional[int] = _DEFAULT_MAX_YAML_EXPANDED_NODES, ) -> Optional[Union[DictConfig, ListConfig]]: return OmegaConf._create_impl( obj=obj, parent=parent, flags=flags, max_yaml_expanded_nodes=max_yaml_expanded_nodes, ) ``` -------------------------------- ### Accessing Dictionary Elements in OmegaConf Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates object-style and dictionary-style access for configuration elements. Also shows how to access items within a list. ```python >>> # object style access of dictionary elements >>> conf.server.port 80 ``` ```python >>> # dictionary style access >>> conf['log']['rotation'] 3600 ``` ```python >>> # items in list >>> conf.users[0] 'user1' ``` -------------------------------- ### Create OmegaConf from Dotlist Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Create an OmegaConf object from a list of strings in 'key=value' format. Uses dot/bracket notation for nested keys. ```python dot_list = ["a.aa.aaa=1", "a.aa.bbb=2", "a.bb.aaa=3", "a.bb.bbb=4"] conf = OmegaConf.from_dotlist(dot_list) print(OmegaConf.to_yaml(conf)) ``` -------------------------------- ### OmegaConf.create (string overload) Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Creates an OmegaConf configuration object from a string, typically a YAML or JSON representation. ```APIDOC ## OmegaConf.create (string overload) ### Description Creates an OmegaConf configuration object from a string, typically a YAML or JSON representation. ### Method Signature ```python @staticmethod @overload create(obj: str, parent: Optional[BaseContainer] = None, flags: Optional[Dict[str, bool]] = None, *, max_yaml_expanded_nodes: Optional[int] = _DEFAULT_MAX_YAML_EXPANDED_NODES) -> Union[DictConfig, ListConfig] ``` ### Parameters * **obj** (str) - The string containing the configuration data (YAML or JSON). * **parent** (Optional[BaseContainer]) - Optional parent node. * **flags** (Optional[Dict[str, bool]]) - Optional flags dict (e.g. ``{"readonly": True}``). * **max_yaml_expanded_nodes** (Optional[int]) - Maximum YAML nodes after alias expansion when ``obj`` is a YAML string. By default, OmegaConf uses the ``OMEGACONF_MAX_YAML_YAML_EXPANDED_NODES`` environment variable if set, otherwise ``10_000``. Explicit arguments override the environment. Pass ``None`` only for trusted input. ### Returns A ``DictConfig`` or ``ListConfig`` instance. ``` -------------------------------- ### Load OmegaConf from a file Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Use the load static method to load configuration from a file path or a file-like object. It supports specifying the maximum number of YAML nodes after alias expansion. ```python from omegaconf import OmegaConf import pathlib from typing import IO, Union # Example usage (assuming a file named 'config.yaml' exists): # try: # cfg = OmegaConf.load('config.yaml') # print(cfg) # except FileNotFoundError: # print("config.yaml not found.") # Example with a file-like object: # import io # yaml_string = "key: value" # file_obj = io.StringIO(yaml_string) # cfg_from_obj = OmegaConf.load(file_obj) # print(cfg_from_obj) # Example with max_yaml_expanded_nodes: # cfg_with_limit = OmegaConf.load('complex_config.yaml', max_yaml_expanded_nodes=5000) # print(cfg_with_limit) # The actual implementation of load: @staticmethod def load( file_: Union[str, pathlib.Path, IO[Any]], *, max_yaml_expanded_nodes: Optional[int] = _DEFAULT_MAX_YAML_EXPANDED_NODES, ) -> Union[DictConfig, ListConfig]: """ Load a YAML config from a file path or file-like object. """ # Implementation details omitted for brevity in this example ``` -------------------------------- ### Create Config from CLI Arguments Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Creates a DictConfig from command-line arguments. Arguments should be in dotlist format. ```python if args_list is None: # Skip program name args_list = sys.argv[1:] return OmegaConf.from_dotlist(args_list) ``` -------------------------------- ### Saving and Loading OmegaConf to/from Pickle File Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Shows how to use Python's `pickle` module to save and load OmegaConf objects, which preserves type information. Be aware of potential cross-version compatibility issues. ```python >>> conf = OmegaConf.create({"foo": 10, "bar": 20, 123: 456}) >>> with tempfile.TemporaryFile() as fp: ... pickle.dump(conf, fp) ... fp.flush() ... assert fp.seek(0) == 0 ... loaded = pickle.load(fp) ... assert conf == loaded ``` -------------------------------- ### _select Source: https://omegaconf.readthedocs.io/en/latest/api_reference.html Selects a value from a configuration object using a dot or bracket notation key path. ```APIDOC ## _select ### Description Select a value from a config using a key path. The key path uses dot notation (`"a.b.c"`) or bracket notation (`"a[b][c]"`), or a mix of both. **Keys containing special characters** (`.`, `[`, `]`, `=`) can be expressed by escaping them with a backslash: * `r"a\.b"` — selects the key `"a.b"` (single key with a literal dot) * `r"a\[0\]"` — selects the key `"a[0]"` * `r"a\=b"` — selects the key `"a=b"` A backslash before any other character passes through unchanged (`r"a\b"` selects the key `"a\\b"` — a backslash followed by `b`). ### Parameters * **cfg** (Container) - The configuration object to select from. * **key** (str) - The key path to select. * **default** (Any) - The default value to return if the key is not found. Defaults to a special marker indicating no default. * **throw_on_resolution_failure** (bool) - Whether to throw an exception if interpolation resolution fails. Defaults to True. * **throw_on_missing** (bool) - Whether to throw an exception if the key is missing. Defaults to False. ``` -------------------------------- ### OmegaConf Configuration Creation from String Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Creates an OmegaConf configuration object from a string, typically YAML or JSON. This overload is specifically for string inputs. ```python from omegaconf import OmegaConf, DictConfig, ListConfig from typing import Optional, Dict, Union # @overload # def create( # obj: str, # parent: Optional[BaseContainer] = None, # flags: Optional[Dict[str, bool]] = None, # *, # max_yaml_expanded_nodes: Optional[int] = _DEFAULT_MAX_YAML_EXPANDED_NODES, # ) -> Union[DictConfig, ListConfig]: ... ``` -------------------------------- ### OmegaConf._create Source: https://omegaconf.readthedocs.io/en/latest/api_reference.html Creates an OmegaConf config object from various input types like YAML strings, dictionaries, lists, or existing OmegaConf objects. ```APIDOC ## OmegaConf._create ### Description Create an OmegaConf config from `obj`. `obj` may be a YAML string, a dict, a list or tuple, a dataclass or attrs class (type or instance), an existing `DictConfig` / `ListConfig`, or `None`. Omitting `obj` (or passing `{}` explicitly) returns an empty `DictConfig`. ### Parameters #### Path Parameters - **obj** (str | List[Any] | Tuple[Any, ...] | DictConfig | ListConfig | None | Dict[Any, Any]) - Required - Source object to build the config from. - **parent** (BaseContainer | None) - Optional - Optional parent node. - **flags** (Dict[str, bool] | None) - Optional - Optional flags dict (e.g. `{"readonly": True}`). - **max_yaml_expanded_nodes** (int | None) - Optional - Maximum YAML nodes after alias expansion when `obj` is a YAML string. By default, OmegaConf uses the `OMEGACONF_MAX_YAML_EXPANDED_NODES` environment variable if set, otherwise `10_000`. Explicit arguments override the environment. Pass `None` only for trusted input. See https://omegaconf.readthedocs.io/en/latest/yaml_aliases.html. ### Returns A `DictConfig`, `ListConfig`, or `None` (when `obj` is `None`). ``` -------------------------------- ### Dynamic node creation with oc.create Source: https://omegaconf.readthedocs.io/en/latest/custom_resolvers.html Use oc.create for dynamic generation of config nodes from Python dict/list objects or YAML strings. Resolvers can be registered to provide data for oc.create. ```python >>> OmegaConf.register_resolver("make_dict", lambda: {"a": 10}) >>> cfg = OmegaConf.create( ... { ... "plain_dict": "${make_dict:}", ... "dict_config": "${oc.create:${make_dict:}}", ... "dict_config_env": "${oc.create:${oc.env:YAML_ENV}}", ... } ... ) >>> os.environ["YAML_ENV"] = "A: 10\nb: 20\nC: ${.A}" >>> show(cfg.plain_dict) # `make_dict` returns a Python dict type: dict, value: {'a': 10} >>> show(cfg.dict_config) # `oc.create` converts it to DictConfig type: DictConfig, value: {'a': 10} >>> show(cfg.dict_config_env) # YAML string to DictConfig type: DictConfig, value: {'A': 10, 'b': 20, 'C': '${.A}'} >>> cfg.dict_config_env.C # interpolations work in a DictConfig 10 ``` -------------------------------- ### Saving and Loading OmegaConf to/from YAML File Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates saving an OmegaConf object to a temporary YAML file and then loading it back. Note that type information is not retained. ```python >>> conf = OmegaConf.create({"foo": 10, "bar": 20, 123: 456}) >>> with tempfile.NamedTemporaryFile() as fp: ... OmegaConf.save(config=conf, f=fp.name) ... loaded = OmegaConf.load(fp.name) ... assert conf == loaded ``` -------------------------------- ### OmegaConf.load Source: https://omegaconf.readthedocs.io/en/latest/api_reference.html Loads a YAML configuration from a file path or a file-like object. ```APIDOC ## OmegaConf.load ### Description Load a YAML config from a file path or file-like object. ### Parameters #### Path Parameters * **file** – A file path (str or `pathlib.Path`) or an open file object. * **max_yaml_expanded_nodes** – Maximum YAML nodes after alias expansion. By default, OmegaConf uses the `OMEGACONF_MAX_YAML_EXPANDED_NODES` environment variable if set, otherwise `10_000`. Explicit arguments override the environment. Pass `None` only for trusted input. See https://omegaconf.readthedocs.io/en/latest/yaml_aliases.html. ### Returns A `DictConfig` or `ListConfig` parsed from the YAML content. ``` -------------------------------- ### Collecting Missing Keys with OmegaConf.missing_keys Source: https://omegaconf.readthedocs.io/en/latest/usage.html Demonstrates how to find all missing keys ('???') within a configuration object, including those within nested structures or referenced by interpolations. Optionally resolves custom resolvers. ```python >>> missings = OmegaConf.missing_keys({ ... "foo": {"bar": "???"}, ... "missing": "???", ... "list": ["a", None, "???"] ... }) >>> assert missings == {'list[2]', 'foo.bar', 'missing'} ``` -------------------------------- ### Create OmegaConf from List Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Create an OmegaConf object from a Python list. Tuples are also supported. ```python conf = OmegaConf.create([1, {"a":10, "b": {"a":10, 123: "int_key"}}]) print(OmegaConf.to_yaml(conf)) ``` -------------------------------- ### Type Checking with OmegaConf Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Shows how to use OmegaConf.is_config, OmegaConf.is_dict, and OmegaConf.is_list to determine the type of OmegaConf objects. ```python # dict: d = OmegaConf.create({"foo": "bar"}) assert OmegaConf.is_config(d) assert OmegaConf.is_dict(d) assert not OmegaConf.is_list(d) # list: l = OmegaConf.create([1,2,3]) assert OmegaConf.is_config(l) assert OmegaConf.is_list(l) assert not OmegaConf.is_dict(l) ``` -------------------------------- ### Union Operator for Merging Configurations Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates using the Python 3.9+ dict-style union operator '|' to merge configurations, equivalent to OmegaConf.merge(). ```python cfg1 | cfg2 ``` -------------------------------- ### OmegaConf String Interpolation Helper Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Provides a utility function to create string interpolations in the format '${interpolation}'. This is useful when constructing configuration strings programmatically. ```python from omegaconf import OmegaConf def II(interpolation: str) -> Any: """ Equivalent to ``${interpolation}`` :param interpolation: :return: input ``${node}`` with type Any """ return "$" + "{" + interpolation + "}" ``` -------------------------------- ### omegaconf.MISSING Source: https://omegaconf.readthedocs.io/en/latest/api_reference.html Alias for an undefined or missing configuration value. ```APIDOC ## omegaconf.MISSING ### Description Alias for an undefined or missing configuration value. ### Type ??? (The source indicates '???' for the type) ``` -------------------------------- ### Create Customized Structured Config Source: https://omegaconf.readthedocs.io/en/latest/_sources/structured_config.rst.txt Create a structured config with specific fields customized during instantiation by passing a structured class instance with overridden values. ```python from omegaconf import OmegaConf conf3 = OmegaConf.structured( SimpleTypes(num=20, height=Height.TALL)) print(OmegaConf.to_yaml(conf3)) ``` -------------------------------- ### Configuration Conversion and Manipulation Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Utilities for converting and manipulating OmegaConf configurations. ```APIDOC ## masked_copy ### Description Create a masked copy of a DictConfig that contains a subset of the keys. ### Method `staticmethod` ### Parameters * **conf** (DictConfig) - DictConfig object. * **keys** (Union[str, List[str]]) - Keys to preserve in the copy. ### Returns * DictConfig - The masked ``DictConfig`` object. ### Raises * ValueError - If conf is not a DictConfig. ``` ```APIDOC ## to_container ### Description Recursively converts an OmegaConf config to a primitive container (dict or list). ### Method `staticmethod` ### Parameters * **cfg** (Any) - The OmegaConf configuration to convert. * **resolve** (bool, optional) - Whether to resolve interpolations. Defaults to False. * **throw_on_missing** (bool, optional) - Whether to throw an error if a key is missing. Defaults to False. * **enum_to_str** (bool, optional) - Whether to convert enums to strings. Defaults to False. * **structured_config_mode** (SCMode, optional) - The mode for structured config conversion. Defaults to SCMode.DICT. ### Returns * Union[Dict[DictKeyType, Any], List[Any], None, str, Any] - The converted primitive container. ``` -------------------------------- ### Perform Basic Arithmetic with eval Source: https://omegaconf.readthedocs.io/en/latest/how_to_guides.html Define a configuration using YAML or Python and use the 'eval' resolver to compute values, such as '10 ** 2'. Ensure the input to eval is trusted as it can execute arbitrary code. ```yaml ten_squared: ${eval:'10 ** 2'} ``` ```python cfg = OmegaConf.create({ "ten_squared": "${eval:'10 ** 2'}", }) assert cfg.ten_squared == 100 ``` -------------------------------- ### Register and Use Custom Resolver Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Shows how to register a custom resolver for new interpolation types and use it in a configuration. ```python >>> OmegaConf.register_resolver( ... "add", lambda *numbers: sum(numbers) ... ) >>> c = OmegaConf.create({'total': '${add:1,2,3}'}) >>> c.total 6 ``` -------------------------------- ### omegaconf.omegaconf.OmegaConf.create Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Creates an OmegaConf configuration object from various input types such as YAML strings, dictionaries, lists, dataclasses, or existing OmegaConf objects. It can also create an empty DictConfig if no object is provided or an empty dictionary is passed. ```APIDOC ## omegaconf.omegaconf.OmegaConf.create ### Description Creates an OmegaConf configuration object from various input types such as YAML strings, dictionaries, lists, dataclasses, or existing OmegaConf objects. It can also create an empty DictConfig if no object is provided or an empty dictionary is passed. ### Method Signature ```python @staticmethod def create( obj: Any = _DEFAULT_MARKER_, parent: Optional[BaseContainer] = None, flags: Optional[Dict[str, bool]] = None, *, max_yaml_expanded_nodes: Optional[int] = _DEFAULT_MAX_YAML_EXPANDED_NODES, ) -> Optional[Union[DictConfig, ListConfig]] ``` ### Parameters * **obj** (Any): Source object to build the config from. Defaults to a marker indicating no object was passed. * **parent** (Optional[BaseContainer]): Optional parent node. * **flags** (Optional[Dict[str, bool]]): Optional flags dict (e.g. ``{"readonly": True}``). * **max_yaml_expanded_nodes** (Optional[int]): Maximum YAML nodes after alias expansion when ``obj`` is a YAML string. Overrides environment variables. Pass ``None`` only for trusted input. ### Returns Optional[Union[DictConfig, ListConfig]]: A ``DictConfig``, ``ListConfig``, or ``None`` (when ``obj`` is ``None``). ``` -------------------------------- ### Checking for Missing Values Source: https://omegaconf.readthedocs.io/en/latest/_sources/usage.rst.txt Demonstrates how to use OmegaConf.is_missing to check if a configuration value is marked as missing ('???'). ```python cfg = OmegaConf.create({ "foo" : 10, "bar": "???" }) assert not OmegaConf.is_missing(cfg, "foo") assert OmegaConf.is_missing(cfg, "bar") ``` -------------------------------- ### Load YAML from File or Object Source: https://omegaconf.readthedocs.io/en/latest/_modules/omegaconf/omegaconf.html Loads configuration from a file path or an open file object. Supports custom YAML loader with node expansion limits. ```python obj = yaml.load( f, Loader=get_yaml_loader( max_yaml_expanded_nodes=max_yaml_expanded_nodes ), ) ``` -------------------------------- ### Define Structured Config with Lists and Tuples Source: https://omegaconf.readthedocs.io/en/latest/structured_config.html Use `typing.List` and `typing.Tuple` to define fields that can hold various OmegaConf-supported types. Default factories are used to initialize empty collections. ```python from dataclasses import dataclass, field from typing import List, Tuple @dataclass class User: name: str = MISSING @dataclass class ListsExample: # Typed list can hold Any, int, float, bool, str, # bytes, pathlib.Path and Enums as well as arbitrary Structured configs. ints: List[int] = field(default_factory=lambda: [10, 20, 30]) bools: Tuple[bool, bool] = field(default_factory=lambda: (True, False)) users: List[User] = field(default_factory=lambda: [User(name="omry")]) ``` -------------------------------- ### Merge Configurations from Files and CLI Source: https://omegaconf.readthedocs.io/en/latest/usage.html Load configurations from YAML files and command-line arguments, then merge them into a single OmegaConf object. Note how CLI arguments can override file values. ```yaml server: port: 80 users: - user1 - user2 ``` ```yaml log: file: log.txt ``` ```python >>> from omegaconf import OmegaConf >>> import sys >>> >>> # Simulate command line arguments >>> sys.argv = ['program.py', 'server.port=82'] >>> >>> base_conf = OmegaConf.load('source/example2.yaml') >>> second_conf = OmegaConf.load('source/example3.yaml') >>> cli_conf = OmegaConf.from_cli() >>> >>> # merge them all >>> conf = OmegaConf.merge(base_conf, second_conf, cli_conf) >>> print(OmegaConf.to_yaml(conf)) server: port: 82 users: - user1 - user2 log: file: log.txt ``` -------------------------------- ### Static Type Checking with Structured Configs Source: https://omegaconf.readthedocs.io/en/latest/structured_config.html Demonstrates using Python type annotations with OmegaConf structured configs for static type checking. Shows valid and invalid assignments that would be caught by type checkers or raise validation errors. ```python >>> conf: SimpleTypes = OmegaConf.structured(SimpleTypes) >>> # Passes static type checking >>> conf.description = "text" >>> # Fails static type checking (but will also raise a Validation error) >>> with raises(ValidationError): ... conf.num = "foo" ``` -------------------------------- ### Creating Explicitly Typed Containers with Initial Content Source: https://omegaconf.readthedocs.io/en/latest/structured_config.html Both OmegaConf.typed_list() and OmegaConf.typed_dict() accept an optional initial content argument to create the container with pre-existing values. ```python lst = OmegaConf.typed_list([1, 2, 3], element_type=int) d = OmegaConf.typed_dict({"x": 1}, key_type=str, element_type=int) ```