### Compose Sequence Node Examples - YAML Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-composer.md Examples demonstrating block and flow style sequences in YAML. ```yaml # Block style sequence - item1 - item2 - nested: key: value # Flow style sequence [item1, item2, {nested: {key: value}}] ``` -------------------------------- ### Instantiate Constructor Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-constructor.md Examples of creating a basic Constructor instance and one with custom single constructors defined in a dictionary. ```julia # Create a basic constructor cons = YAML.Constructor() # Create with custom constructors custom_cons = Dict("!custom" => my_constructor_func) cons = YAML.Constructor(custom_cons) ``` -------------------------------- ### Compose Mapping Node Examples - YAML Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-composer.md Examples showing simple and complex mappings in YAML, including explicit tags and nested structures. ```yaml # Simple mapping key1: value1 key2: value2 # Complex keys and values ? !!int 1 : !!seq [a, b, c] ? !!map {x: 1} : nested_mapping: key: value ``` -------------------------------- ### Handle Implicit Document Start Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-parser.md Handles the start of a YAML document, whether explicit or implicit. Creates a `DocumentStartEvent` and processes optional byte order marks, YAML version, and TAG directives. ```julia parse_implicit_document_start(stream::EventStream) -> DocumentStartEvent ``` -------------------------------- ### Sequence Node Examples Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-nodes.md Illustrates YAML sequences in both block style (using hyphens) and flow style (using brackets). ```yaml # Block style - item1 - item2 - item3 # Flow style [item1, item2, item3] ``` -------------------------------- ### Mapping Node Examples Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-nodes.md Illustrates YAML mappings in both block style (indented key-value pairs) and flow style (using braces). ```yaml # Block style key1: value1 key2: value2 # Flow style {key1: value1, key2: value2} ``` -------------------------------- ### Stage 3: Composer - Node Structure Example Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/parsing-pipeline.md Illustrates the structure of a node tree representing a YAML document. ```text Document with YAML: person: name: Alice items: [1, 2, 3] Becomes MappingNode: ├─ key: ScalarNode("tag:yaml.org,2002:str", "person") └─ value: MappingNode ├─ key: ScalarNode("tag:yaml.org,2002:str", "name") │ value: ScalarNode("tag:yaml.org,2002:str", "Alice") └─ key: ScalarNode("tag:yaml.org,2002:str", "items") value: SequenceNode ├─ ScalarNode("tag:yaml.org,2002:int", "1") ├─ ScalarNode("tag:yaml.org,2002:int", "2") └─ ScalarNode("tag:yaml.org,2002:int", "3") ``` -------------------------------- ### Handle ConstructorError Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-constructor.md Example of catching a `ConstructorError` during object construction. It demonstrates how to access the problem description and its location within the YAML document. ```julia try construct_object(constructor, node) catch e::ConstructorError println("Failed to construct: $(e.problem)") println("At: $(e.problem_mark)") end ``` -------------------------------- ### YAML Example with Anchor and Alias Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-composer.md This YAML demonstrates how to define an anchor `&anchor` for a node and then reference it using an alias `*anchor`. Both `first_item` and `second_item` will point to the same node object after composition. ```yaml first_item: &anchor x: 1 y: 2 second_item: *anchor # References the same node as first_item ``` -------------------------------- ### Accessing Node Source Location Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-nodes.md Retrieve and display the starting line and column of a node in the source YAML document using `start_mark`. This is useful for debugging and error reporting. ```julia if node.start_mark !== nothing println("Node starts at line $(node.start_mark.line), column $(node.start_mark.column)") end ``` -------------------------------- ### Handle ScannerError Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-scanner.md Demonstrates how to catch and handle ScannerError exceptions during tokenization. This example shows how to access the problem description and the location where the error occurred. ```julia try ts = YAML.TokenStream(IOBuffer("invalid: [")) # Tokenization would fail due to unclosed bracket catch e::ScannerError println("Scanning error: $(e.problem)") println("At: $(e.problem_mark)") end ``` -------------------------------- ### Access Token Span Information Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Demonstrates how to retrieve and print the start and end Mark objects associated with a token's span. ```julia token_span = token.span println("Token spans from $(token_span.start_mark) to $(token_span.end_mark)") ``` -------------------------------- ### Define DocumentStartEvent and DocumentEndEvent Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Structs representing the start and end boundaries of a YAML document. They can optionally include explicit flags, version information, and tags. ```julia struct DocumentStartEvent <: Event start_mark::Mark end_mark::Mark explicit::Bool version::Union{Tuple, Nothing} tags::Union{Dict{String, String}, Nothing} end struct DocumentEndEvent <: Event start_mark::Mark end_mark::Mark explicit::Bool end ``` -------------------------------- ### Define Flow Sequence and Mapping Tokens Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Represent the start and end tokens for flow-style sequences (`[` and `]`) and mappings (`{` and `}`). ```julia struct FlowSequenceStartToken <: Token span::Span end struct FlowSequenceEndToken <: Token span::Span end struct FlowMappingStartToken <: Token span::Span end struct FlowMappingEndToken <: Token span::Span end ``` -------------------------------- ### DocumentStartEvent, DocumentEndEvent Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md These event types delineate the start and end of YAML documents, optionally including version and tag information. ```APIDOC ## DocumentStartEvent, DocumentEndEvent ```julia struct DocumentStartEvent <: start_mark::Mark end_mark::Mark explicit::Bool version::Union{Tuple, Nothing} tags::Union{Dict{String, String}, Nothing} end struct DocumentEndEvent <: start_mark::Mark end_mark::Mark explicit::Bool end ``` Represent YAML document boundaries. `version` is a tuple like `(1, 1)` or `(1, 2)`. ``` -------------------------------- ### Define StreamStartEvent and StreamEndEvent Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Structs representing the start and end boundaries of a token stream. They include mark information and encoding for the start event. ```julia struct StreamStartEvent <: Event start_mark::Mark end_mark::Mark encoding::String end struct StreamEndEvent <: Event start_mark::Mark end_mark::Mark end ``` -------------------------------- ### Span Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Represents a range in a YAML document where a token is located, defined by start and end marks. ```APIDOC ## Struct: Span ### Description Represents a range in a YAML document where a token is located. ### Fields - **start_mark** (`Mark`) - Position where the token/span begins - **end_mark** (`Mark`) - Position where the token/span ends ### Usage Every token includes a span for precise source location information. ```julia token_span = token.span println("Token spans from $(token_span.start_mark) to $(token_span.end_mark)") ``` ``` -------------------------------- ### Create and Use EventStream Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-parser.md Demonstrates how to create an `EventStream` from a `TokenStream` and then use `peek` to look at the next event or `forward!` to advance to the next event. ```julia import YAML ts = YAML.TokenStream(IOBuffer("key: value")) events = YAML.EventStream(ts) # Peek at next event event = peek(events) # Advance to next event event = forward!(events) ``` -------------------------------- ### Basic YAML Parsing and Writing in Julia Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Demonstrates common operations like parsing YAML from a string or file, handling multiple documents, and writing Julia data structures back to YAML format. Ensure the 'YAML' package is imported before use. ```julia import YAML # Parse YAML string data = YAML.load("key: value\nlist: [1, 2, 3]") # Parse YAML file data = YAML.load_file("config.yml") # Parse multiple documents for doc in YAML.load_all_file("data.yml") println(doc) end # Write to YAML YAML.write_file("output.yml", data) yaml_string = YAML.yaml(data) ``` -------------------------------- ### Scanning Tokens with TokenStream Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-scanner.md Demonstrates how to initialize and use the TokenStream to iterate through YAML tokens. You can peek at the next token or move forward to consume it. ```APIDOC ## Scanning Tokens The TokenStream is used to sequentially retrieve tokens from input: ```julia import YAML ts = YAML.TokenStream(IOBuffer("key: [1, 2, 3]")) # Peek at the next token without consuming it token = peek(ts) # Returns StreamStartToken # Move to the next token token = forward!(ts) # Returns StreamStartToken # Continue scanning while !ts.done token = forward!(ts) println(token) end ``` ``` -------------------------------- ### Catch ComposerError in YAML Parsing Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/errors.md A general example of catching a ComposerError during YAML loading, printing the problem and its location. ```julia import YAML try data = YAML.load("item: *undefined") catch e::ComposerError println("Composer error: $(e.problem)") println("At: $(e.problem_mark)") end ``` -------------------------------- ### Initialize and Use TokenStream Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-scanner.md Demonstrates how to initialize a YAML.jl TokenStream from an IOBuffer and sequentially retrieve tokens using peek and forward!. The stream is considered done when all tokens are consumed. ```julia import YAML ts = YAML.TokenStream(IOBuffer("key: [1, 2, 3]")) # Peek at the next token without consuming it token = peek(ts) # Returns StreamStartToken # Move to the next token token = forward!(ts) # Returns StreamStartToken # Continue scanning while !ts.done token = forward!(ts) println(token) end ``` -------------------------------- ### compose_sequence_node Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-composer.md Composes a sequence node from a sequence start event. It recursively composes child nodes until a SequenceEndEvent is encountered. ```APIDOC ## compose_sequence_node Composes a sequence node from a sequence start event. ```julia compose_sequence_node(composer::Composer, anchor::Union{String, Nothing}) -> SequenceNode ``` #### Parameters | Parameter | Type | Description | |-----------|------|-------------| | composer | `Composer` | Composer instance | | anchor | `Union{String, Nothing}` | Anchor name if this node is anchored | #### Returns A `SequenceNode` containing child nodes. #### Process 1. Extracts tag from event or resolves implicitly 2. Creates SequenceNode with empty value vector 3. Registers in anchors if anchored 4. Recursively composes each element until `SequenceEndEvent` 5. Updates node's end_mark 6. Returns the node ``` -------------------------------- ### Access Error Location with Mark Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Example of how to access and print the line and column number from a Mark object when an error occurs. ```julia if error.problem_mark !== nothing println("Error at line $(error.problem_mark.line), column $(error.problem_mark.column)") end ``` -------------------------------- ### YAML Loading with Custom Constructors Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/errors.md Illustrates how to use custom constructors with YAML.load and handle potential ConstructorErrors, specifically for custom types like dates. ```julia import YAML function safe_load(yaml_string) custom_constructors = Dict( "!date" => construct_date ) try YAML.load(yaml_string; more_constructors=custom_constructors) catch e::ConstructorError if "!date" in e.problem @warn "Custom date constructor failed" else throw(e) end return nothing end end ``` -------------------------------- ### compose_mapping_node Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-composer.md Composes a mapping node from a mapping start event. It recursively composes key and value pairs until a MappingEndEvent is encountered. ```APIDOC ## compose_mapping_node Composes a mapping node from a mapping start event. ```julia compose_mapping_node(composer::Composer, anchor::Union{String, Nothing}) -> MappingNode ``` #### Parameters | Parameter | Type | Description | |-----------|------|-------------| | composer | `Composer` | Composer instance | | anchor | `Union{String, Nothing}` | Anchor name if this node is anchored | #### Returns A `MappingNode` containing (key, value) node pairs. #### Process 1. Extracts tag from event or resolves implicitly 2. Creates MappingNode with empty value vector 3. Registers in anchors if anchored 4. Recursively composes key and value pairs until `MappingEndEvent` 5. Updates node's end_mark 6. Returns the node ``` -------------------------------- ### Constructor System Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Custom constructors for YAML tags and type conversion. ```APIDOC ## Custom Constructors - Define custom constructors for specific YAML tags. - Type conversion functions for mapping YAML types to Julia types. - Default constructors for standard YAML types. - Support for custom mapping in constructors. ``` -------------------------------- ### Load YAML Document from File Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/configuration.md Parses the first YAML document from a specified file. Allows customization of dictionary types and constructors. ```julia load_file(filename::AbstractString; dicttype::Union{Type, Function} = Dict{Any, Any}, more_constructors::Union{Nothing, Dict} = nothing, multi_constructors::Dict = Dict(), constructorType::Function = SafeConstructor) ``` -------------------------------- ### Load YAML Configuration File Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Loads a YAML configuration file and accesses nested database settings. Ensure 'config.yml' exists. ```julia import YAML config = YAML.load_file("config.yml") db_host = config["database"]["host"] db_port = config["database"]["port"] ``` -------------------------------- ### Load YAML with Custom Dictionary Types in Julia Source: https://github.com/juliadata/yaml.jl/blob/master/README.md Demonstrates how to specify custom dictionary types when loading YAML files. This includes using `Dict{Symbol,Any}`, `OrderedDict{String,Any}`, and `DefaultDict{String,Any}`. ```julia # using Symbol keys data = YAML.load_file("test.yml"; dicttype=Dict{Symbol,Any}) ``` ```julia # maintaining the order from the YAML file using OrderedCollections data = YAML.load_file("test.yml"; dicttype=OrderedDict{String,Any}) ``` ```julia # specifying a default value using DataStructures data = YAML.load_file("test.yml"; dicttype=()->DefaultDict{String,Any}(Missing)) ``` -------------------------------- ### Configuration Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Options for configuring YAML.jl behavior. ```APIDOC ## `dicttype` - Specify a custom dictionary type to be used for YAML mappings. ## `more_constructors` - Add custom constructors for specific YAML tags. ## `multi_constructors` - Define constructors that handle multiple YAML tags. ## `constructorType` - Factory function for creating constructor instances. ``` -------------------------------- ### Compose Mapping Node - YAML.jl Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-composer.md Composes a mapping node from a mapping start event. Keys can be complex structures, and both keys and values are recursively composed. ```julia compose_mapping_node(composer::Composer, anchor::Union{String, Nothing}) -> MappingNode ``` -------------------------------- ### Stage 1: Scanner - TokenStream Operations Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/parsing-pipeline.md Create a token stream from input, peek at the next token, and advance to the next token. ```julia ts = YAML.TokenStream(input) token = peek(ts) token = forward!(ts) ``` -------------------------------- ### Main API Reference Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Core functions for loading and writing YAML data. ```APIDOC ## `load()` ### Description Parse YAML from a string or IO stream. ### Method `load(yaml_string_or_io)` ### Parameters - `yaml_string_or_io`: A string containing YAML content or an IO object. ## `load_file()` ### Description Parse YAML from a file. ### Method `load_file(filepath)` ### Parameters - `filepath`: The path to the YAML file. ## `load_all()` / `load_all_file()` ### Description Parse multiple YAML documents from a string, IO stream, or file. ### Method `load_all(yaml_string_or_io)` `load_all_file(filepath)` ### Parameters - `yaml_string_or_io`: A string containing YAML content or an IO object. - `filepath`: The path to the YAML file. ## `write()` / `write_file()` ### Description Serialize Julia data structures to YAML format. ### Method `write(data)` `write_file(filepath, data)` ### Parameters - `data`: The Julia data structure to serialize. - `filepath`: The path to the output YAML file (for `write_file`). ## `yaml()` ### Description Get the YAML string representation of Julia data. ### Method `yaml(data)` ### Parameters - `data`: The Julia data structure. ## `YAMLDocIterator` ### Description An iterator for processing multiple YAML documents. ### Usage Iterate over documents obtained from `load_all` or `load_all_file`. ``` -------------------------------- ### Write Julia Data to YAML File Source: https://github.com/juliadata/yaml.jl/blob/master/README.md Use `YAML.write_file` to serialize Julia objects into a YAML file. This example shows how to write the `data` variable to `test-output.yml`. ```julia import YAML YAML.write_file("test-output.yml", data) ``` -------------------------------- ### Input YAML Document Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/parsing-pipeline.md The initial YAML content to be parsed. ```yaml person: name: Alice age: 30 active: true ``` -------------------------------- ### Define DocumentStartToken and DocumentEndToken Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Represent YAML document markers (`---` and `...`) in the token stream, each storing its source location. ```julia struct DocumentStartToken <: Token span::Span end struct DocumentEndToken <: Token span::Span end ``` -------------------------------- ### Define Span Type for Token Range Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md The Span struct represents a range within a YAML document, defined by start and end Mark objects. It is used to locate tokens. ```julia struct Span start_mark::Mark end_mark::Mark end ``` -------------------------------- ### Compose Sequence Node - YAML.jl Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-composer.md Composes a sequence node from a sequence start event. Child nodes are composed recursively, and flow vs. block style is preserved in the node. ```julia compose_sequence_node(composer::Composer, anchor::Union{String, Nothing}) -> SequenceNode ``` -------------------------------- ### Stage 4: Constructor - Tag to Julia Type Mapping Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/parsing-pipeline.md Shows the default mapping from YAML node tags to native Julia types. ```text ScalarNode with tag: tag:yaml.org,2002:null → nothing tag:yaml.org,2002:bool → true/false tag:yaml.org,2002:int → 42 tag:yaml.org,2002:float → 3.14 tag:yaml.org,2002:str → "text" tag:yaml.org,2002:binary → Vector{UInt8} tag:yaml.org,2002:timestamp → Date/DateTime SequenceNode → Vector{Any} MappingNode → Dict{Any, Any} (or custom dicttype) ``` -------------------------------- ### Type System Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Reference for node, token, event, error, and support types. ```APIDOC ## Node Types - `ScalarNode`: Represents a scalar value in YAML. - `SequenceNode`: Represents a YAML sequence (list). - `MappingNode`: Represents a YAML mapping (dictionary). ## Token Types - All YAML tokens and their fields are defined. ## Event Types - All YAML events and their fields are defined. ## Error Types - `ConstructorError`: Errors during type construction. - `ParserError`: Errors during YAML parsing. - `ComposerError`: Errors during node composition. - `ScannerError`: Errors during lexical analysis (scanning). ## Support Types - `Mark`: Represents a position in the source document. - `Span`: Represents a range of text in the source document. - `Constructor`: Base type for YAML constructors. - `Resolver`: Handles type resolution. ``` -------------------------------- ### Override Standard YAML Tags Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/configuration.md Override standard YAML tags like `tag:yaml.org,2002:int` by providing a custom constructor function in `more_constructors`. This example converts integers to strings. ```julia function construct_int_as_string(constructor, node) return string(YAML.construct_scalar(constructor, node)) end more_constructors = Dict("tag:yaml.org,2002:int" => construct_int_as_string) data = YAML.load("value: 42"; more_constructors=more_constructors) # data["value"] => "42" (as string, not integer) ``` -------------------------------- ### Process YAML Directives Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-parser.md Processes YAML directives at the start of a document to determine the YAML version and tag handles. It validates the YAML version and processes tag handle mappings, merging them with default tags. ```julia process_directives(stream::EventStream) -> Tuple ``` -------------------------------- ### load() Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/configuration.md Parses the first YAML document from a string or IO stream. Allows customization of dictionary types and constructors. ```APIDOC ## load() ### Description Parses the first YAML document from a string or IO stream. Allows customization of dictionary types and constructors. ### Parameters - `input` (Union{AbstractString, IO}): The source of the YAML data (string or IO stream). - `dicttype` (Union{Type, Function}, optional): The type to use for mappings. Defaults to `Dict{Any, Any}`. - `more_constructors` (Union{Nothing, Dict}, optional): Custom single-tag constructors. - `multi_constructors` (Dict, optional): Custom multi-tag constructors. Defaults to an empty `Dict`. - `constructorType` (Function, optional): Constructor factory function. Defaults to `SafeConstructor`. ``` -------------------------------- ### _print (String) Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-writer.md Serializes strings to YAML format, applying appropriate quoting and escaping, and handling multi-line strings using literal block scalars. ```APIDOC ## _print (String) Prints string values with appropriate quoting and escaping. ```julia _print(io::IO, str::AbstractString, level::Int = 0, ignore_level::Bool = false) ``` ### Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | io | `IO` | — | Output stream | | str | `AbstractString` | — | String to serialize | | level | `Int` | `0` | Current indentation level | | ignore_level | `Bool` | `false` | (not used for strings) | ### Output Format Single-line string: ```yaml "simple string" ``` Multi-line string with literal block scalar (`|`): ```yaml |- First line Second line ``` With trailing newlines (`|+`): ```yaml |+ Text with trailing newlines ``` ### Quoting Rules - Strings with embedded newlines use literal block scalar style (`|`, `|-`, `|+`) - Strings with quotes use literal style to avoid escaping - Simple strings are quoted using Julia's `repr()` function - Empty strings are always quoted ### Trailing Newline Handling - No trailing newlines: `|-` (strip) - One trailing newline: `|` (clip, default) - Multiple trailing newlines: `|+` (keep) ### Examples ```yaml # Simple quoted string text: "hello world" # Literal block (preserves newlines) description: | This is a longer description spread across multiple lines # Keep trailing newlines script: |+ #!/bin/bash echo "done" ``` ``` -------------------------------- ### Handle Custom Types with YAML Loading Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Illustrates how to define and use custom constructors for loading YAML data that represents custom Julia types. The `more_constructors` argument is used to map YAML tags to Julia functions. ```julia import YAML struct Point x::Float64 y::Float64 end function construct_point(constructor, node) m = YAML.construct_mapping(constructor, node) Point(m["x"], m["y"]) end yaml_str = """ location: !point x: 10.0 y: 20.0 """ data = YAML.load(yaml_str; more_constructors=Dict("!point" => construct_point)) println(data["location"]) ``` -------------------------------- ### Define Fallback Constructor for Unknown Tags Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/configuration.md Use `nothing` as a key in `multi_constructors` to define a fallback constructor that handles any tags not explicitly defined. This allows for logging or default handling of unknown tags. ```julia function construct_unknown(constructor, tag, node) value = YAML.construct_scalar(constructor, node) @warn "Unknown tag: $tag, value: $value" return value end multi_constructors = Dict(nothing => construct_unknown) data = YAML.load("value: !unknown data"; multi_constructors=multi_constructors) ``` -------------------------------- ### Initialize Stream Parsing Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-parser.md The initial state function for stream parsing. Consumes the `StreamStartToken` and emits a `StreamStartEvent`, then sets the state to `parse_implicit_document_start`. ```julia parse_stream_start(stream::EventStream) -> StreamStartEvent ``` -------------------------------- ### Load YAML Text Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Demonstrates loading YAML formatted text into a Julia dictionary. Ensure the YAML.jl package is imported. ```julia import YAML yaml_text = """ name: Alice age: 30 skills: - programming - writing """ data = YAML.load(yaml_text) println(data["name"]) println(data["age"]) println(data["skills"][1]) ``` -------------------------------- ### add_multi_constructor! Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-constructor.md Registers a multi-tag constructor function that handles a tag prefix and its suffix. ```APIDOC ## add_multi_constructor! ### Description Registers a multi-tag constructor function that handles a tag prefix and its suffix. ### Method Signature ```julia add_multi_constructor!(func::Function, constructor::Constructor, tag::Union{String, Nothing}) ``` ### Parameters #### Path Parameters - **func** (`Function`) - Constructor function with signature `(::Constructor, ::String, ::Node) -> Any` - **constructor** (`Constructor`) - Constructor instance to register with - **tag** (`Union{String, Nothing}`) - Tag prefix (e.g., "!custom/") — `nothing` for fallback ### Returns The modified `Constructor` object. ### Request Example ```julia function multi_constructor(constructor::Constructor, tag_suffix::String, node::Node) # tag_suffix contains what came after the registered prefix construct_scalar(constructor, node) end add_multi_constructor!(multi_constructor, cons, "!custom/") ``` ``` -------------------------------- ### Efficiently Load All Documents from a File Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/parsing-pipeline.md Use `load_all()` for memory-efficient loading of files with many YAML documents, processing one document at a time. Avoid `collect(YAML.load_all_file(...))` for large files as it loads all documents into memory. ```julia # Memory efficient: one document at a time for doc in YAML.load_all_file("many_docs.yml") process(doc) # Each doc loaded/unloaded individually end # Less efficient: loads all documents into memory docs = collect(YAML.load_all_file("many_docs.yml")) ``` -------------------------------- ### load Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-main.md Parses the first YAML document from a string or IO stream and returns it as a Julia object. Supports custom constructors and dictionary types. ```APIDOC ## load ### Description Parses the first YAML document from a string or IO stream and returns it as a Julia object. Supports custom constructors and dictionary types. ### Method ```julia load(x::Union{AbstractString, IO}) load(x::Union{AbstractString, IO}, more_constructors::Union{Nothing, Dict}) load(x::Union{AbstractString, IO}, more_constructors::Union{Nothing, Dict}, multi_constructors::Dict) load(ts::TokenStream, more_constructors::Union{Nothing, Dict} = nothing, multi_constructors::Dict = Dict(); dicttype::Union{Type, Function} = Dict{Any, Any}, constructorType::Function = SafeConstructor) ``` ### Parameters #### Path Parameters - **x** (Union{AbstractString, IO}) - Required - YAML string or IO stream to parse - **more_constructors** (Union{Nothing, Dict}) - Optional - Dictionary of custom constructors to use for specific tags - **multi_constructors** (Dict) - Optional - Dictionary of multi-constructors for tag prefixes - **dicttype** (Union{Type, Function}) - Optional - Type to use for all parsed dictionaries (e.g., `OrderedDict{String,Any}`) - **constructorType** (Function) - Optional - Constructor type to use (must be compatible with the Constructor interface) ### Returns The first YAML document as a Julia object. Returns `nothing` if the stream contains no document. ### Request Example ```julia import YAML # Load from a string yaml_str = """ name: John age: 30 """ data = YAML.load(yaml_str) println(data["name"]) # Load from a file via IO data = YAML.load(open("config.yml", "r")) # Load with custom dictionary type using OrderedCollections data = YAML.load(yaml_str; dicttype=OrderedDict{String,Any}) # Load with Symbol keys data = YAML.load(yaml_str; dicttype=Dict{Symbol,Any}) ``` ``` -------------------------------- ### Load YAML with DefaultDict and Missing Values Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/configuration.md Loads YAML content into a `DefaultDict`, allowing for missing keys to be represented by a default value (e.g., `missing`). Handles both present and absent keys gracefully. ```julia import YAML using DataStructures yaml_str = """ present_key: value missing_key: ??? """ data = YAML.load(yaml_str; dicttype=()->DefaultDict{String, Any}(missing)) println(data["present_key"]) # "value" println(data["nonexistent"]) # missing ``` -------------------------------- ### YAML Error Handling Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Demonstrates how to catch and handle specific YAML parsing errors, including syntax and type errors, using try-catch blocks. ```julia import YAML try data = YAML.load_file("config.yml") catch e::YAML.ScannerError println("Syntax error: $(e.problem)") catch e::YAML.ConstructorError println("Type error: $(e.problem)") end ``` -------------------------------- ### Stage 3: Composer - Node Tree Construction Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/parsing-pipeline.md Compose a node tree from an event stream using a resolver. ```julia resolver = YAML.Resolver() node = YAML.compose(events, resolver) ``` -------------------------------- ### Unimplemented Constructors Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-constructor.md These constructors are not yet implemented and throw `ConstructorError` if encountered. ```APIDOC ## Unimplemented Constructors These constructors are not yet implemented and throw `ConstructorError` if encountered. ### `construct_yaml_omap` ```julia construct_yaml_omap(constructor::Constructor, node::Node) # throws ``` ### `construct_yaml_pairs` ```julia construct_yaml_pairs(constructor::Constructor, node::Node) # throws ``` ### `construct_yaml_set` ```julia construct_yaml_set(constructor::Constructor, node::Node) # throws ``` ### `construct_yaml_object` ```julia construct_yaml_object(constructor::Constructor, node::Node) # throws ``` ``` -------------------------------- ### Catch ConstructorError during YAML Loading Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/errors.md Demonstrates how to use a try-catch block to handle ConstructorError when loading YAML data. It specifically catches errors related to invalid integer formats and prints relevant problem details and line numbers. ```julia import YAML try data = YAML.load("\n !!int \"not_a_number\" ") catch e::ConstructorError println("Construction error: $(e.problem)") if e.problem_mark !== nothing println("At line $(e.problem_mark.line)") end end ``` ```julia # Catch parse errors within constructor context try data = YAML.load("value: !!int abc") catch e println("Error type: $(typeof(e))") end ``` -------------------------------- ### Configuration Keywords Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Keywords that can be used to configure the YAML parsing and writing behavior. ```APIDOC ## Configuration Keywords ### `dicttype` - **Type**: `Type/Function` - **Default**: `Dict{Any,Any}` - **Purpose**: Specifies the dictionary type to use for YAML mappings. ### `more_constructors` - **Type**: `Dict` - **Default**: `nothing` - **Purpose**: Allows defining custom constructors for single-tagged YAML types. ### `multi_constructors` - **Type**: `Dict` - **Default**: `Dict()` - **Purpose**: Allows defining custom constructors for multi-tagged YAML types. ### `constructorType` - **Type**: `Function` - **Default**: `SafeConstructor` - **Purpose**: Factory function for creating YAML constructors. ``` -------------------------------- ### Load All YAML Documents from File Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/configuration.md Returns an iterator for all YAML documents found in a file. Supports custom dictionary types and constructors for each document. ```julia load_all_file(filename::AbstractString; dicttype::Union{Type, Function} = Dict{Any, Any}, more_constructors::Union{Nothing, Dict} = nothing, multi_constructors::Dict = Dict(), constructorType::Function = SafeConstructor) ``` -------------------------------- ### TokenStream Constructor Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-scanner.md Creates a new TokenStream from an IO object, automatically detecting the character encoding. ```APIDOC ## TokenStream Constructor ### Description Creates a new TokenStream from an IO object. Automatically detects the character encoding. ### Signature ```julia TokenStream(input::IO) ``` ### Examples ```julia import YAML # Create a token stream from a file ts = YAML.TokenStream(open("document.yml", "r")) # Or from a string (wrapped in IOBuffer) ts = YAML.TokenStream(IOBuffer("key: value")) ``` ``` -------------------------------- ### Constructor Output (Julia) Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/parsing-pipeline.md The final output of the constructor stage, representing the YAML data as Julia objects. ```julia Dict{Any, Any}( "person" => Dict{Any, Any}( "name" => "Alice", "age" => 30, "active" => true ) ) ``` -------------------------------- ### Load YAML Data with Standard API Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-composer.md Use the standard `YAML.load` function for typical YAML parsing. This function internally utilizes the composer to build the node tree. ```julia import YAML # Standard API uses composer internally data = YAML.load("key: value") ``` -------------------------------- ### Define Key, Value, and Entry Tokens Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Represent separators within YAML structures: `:` for values, `-` for block entries, and `,` for flow entries. ```julia struct KeyToken <: Token span::Span end struct ValueToken <: Token span::Span end struct BlockEntryToken <: Token span::Span end struct FlowEntryToken <: Token span::Span end ``` -------------------------------- ### Parser and Events Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Details on the parsing pipeline stages and event handling. ```APIDOC ## Event Stream - Conversion from tokens to YAML events. ## Parsing State Machine - Describes the internal state machine of the parser. ## Directive Processing - Handling of YAML directives (e.g., `%YAML`, `%TAG`). ``` -------------------------------- ### add_constructor! Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-constructor.md Registers a single-tag constructor function that handles a specific YAML tag. ```APIDOC ## add_constructor! ### Description Registers a single-tag constructor function. ### Method Signature ```julia add_constructor!(func::Function, constructor::Constructor, tag::Union{String, Nothing}) ``` ### Parameters #### Path Parameters - **func** (`Function`) - Constructor function with signature `(::Constructor, ::Node) -> Any` - **constructor** (`Constructor`) - Constructor instance to register with - **tag** (`Union{String, Nothing}`) - YAML tag this constructor handles (e.g., "!mytype") ### Returns The modified `Constructor` object. ### Request Example ```julia function my_constructor(constructor::Constructor, node::Node) # Custom logic to construct object from node construct_scalar(constructor, node) end add_constructor!(my_constructor, cons, "!mytype") ``` ``` -------------------------------- ### Constructor constructor Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-constructor.md Creates a new Constructor instance. Optionally accepts custom single and multi-tag constructors to customize YAML tag handling. ```julia Constructor(single_constructors = Dict(), multi_constructors = Dict()) ``` -------------------------------- ### Default YAML Constructors Dictionary Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-constructor.md A dictionary mapping YAML tags to their default constructor functions. Used for parsing standard YAML types. ```julia const default_yaml_constructors = Dict{Union{String, Nothing}, Function}( "tag:yaml.org,2002:null" => construct_yaml_null, "tag:yaml.org,2002:bool" => construct_yaml_bool, "tag:yaml.org,2002:int" => construct_yaml_int, "tag:yaml.org,2002:float" => construct_yaml_float, "tag:yaml.org,2002:binary" => construct_yaml_binary, "tag:yaml.org,2002:timestamp" => construct_yaml_timestamp, "tag:yaml.org,2002:omap" => construct_yaml_omap, "tag:yaml.org,2002:pairs" => construct_yaml_pairs, "tag:yaml.org,2002:set" => construct_yaml_set, "tag:yaml.org,2002:str" => construct_yaml_str, "tag:yaml.org,2002:seq" => construct_yaml_seq, "tag:yaml.org,2002:map" => construct_yaml_map, nothing => construct_undefined, ) ``` -------------------------------- ### _print (Pair) Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-writer.md Serializes a single key-value pair to YAML format, determining whether to use a single line or multi-line representation based on the value's complexity. ```APIDOC ## _print (Pair) Prints a single key-value pair. ```julia _print(io::IO, pair::Pair, level::Int = 0, ignore_level::Bool = false) ``` ### Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | io | `IO` | — | Output stream | | pair | `Pair` | — | Key-value pair to serialize | | level | `Int` | `0` | Current indentation level | | ignore_level | `Bool` | `false` | Skip indentation (continuation) | ### Output Format Simple value: ```yaml key: value ``` Complex value (with newline): ```yaml key: nested: data: value ``` ### Key Handling - `nothing` keys are serialized as `null` - Keys requiring quotes are automatically quoted (special characters, looks like YAML syntax, etc.) - Normal keys are printed as-is ### Notes - Determines if value is simple (fits on same line) or complex (requires newline) - Complex values (dicts and non-empty arrays) cause line break after key - Simple values are followed by space before value - Recursively prints both key and value ``` -------------------------------- ### Load YAML with OrderedDict and Symbol Keys Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/configuration.md Loads YAML content into an OrderedDict, preserving the order of keys and using Symbols for keys. Useful when key order and type are important. ```julia import YAML using OrderedCollections yaml_str = """ first: 1 second: 2 third: 3 """ # Preserve order with Symbol keys data = YAML.load(yaml_str; dicttype=OrderedDict{Symbol, Any}) for (key, value) in data println("$key => $value") end ``` -------------------------------- ### Parsing Pipeline Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Overview of the four-stage YAML parsing process. ```APIDOC ## Four-Stage Pipeline 1. **Scanner**: Converts input characters into tokens. 2. **Parser**: Converts tokens into events. 3. **Composer**: Converts events into a node tree. 4. **Constructor**: Converts the node tree into Julia objects. ## Data Flow - Diagram illustrating the data flow through the pipeline. ## Customization Points - Areas within the pipeline where custom logic can be injected. ## Performance Considerations - Notes on performance aspects of the parsing pipeline. ``` -------------------------------- ### load_file() Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/configuration.md Parses a YAML file. Supports customization of dictionary types and constructors. ```APIDOC ## load_file() ### Description Parses a YAML file. Supports customization of dictionary types and constructors. ### Parameters - `filename` (AbstractString): The path to the YAML file. - `dicttype` (Union{Type, Function}, optional): The type to use for mappings. Defaults to `Dict{Any, Any}`. - `more_constructors` (Union{Nothing, Dict}, optional): Custom single-tag constructors. - `multi_constructors` (Dict, optional): Custom multi-tag constructors. Defaults to an empty `Dict`. - `constructorType` (Function, optional): Constructor factory function. Defaults to `SafeConstructor`. ``` -------------------------------- ### Load YAML from File Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-main.md Parses a YAML file into a Julia object. Supports custom dictionary types and symbol keys via keyword arguments. Includes a check to handle cases where the file contains no YAML documents. ```julia import YAML # Basic file loading data = YAML.load_file("config.yml") ``` ```julia using OrderedCollections data = YAML.load_file("config.yml"; dicttype=OrderedCollections.OrderedDict{String,Any}) ``` ```julia # Check if data was loaded if data !== nothing println(data) else println("File contains no YAML documents") end ``` -------------------------------- ### Direct EventStream Usage Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-parser.md For low-level direct usage, create a TokenStream from an IOBuffer and then an EventStream from the token stream. Process events by repeatedly calling forward!. ```julia import YAML # Direct usage (uncommon) ts = YAML.TokenStream(IOBuffer("key: value")) events = YAML.EventStream(ts) while !isnothing(peek(events)) event = forward!(events) println(typeof(event)) end ``` -------------------------------- ### Direct Node Access in YAML Parsing Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Shows how to access YAML data at the node level using TokenStream, EventStream, and compose. Useful for low-level manipulation. ```julia import YAML ts = YAML.TokenStream(IOBuffer("key: value")) events = YAML.EventStream(ts) resolver = YAML.Resolver() node = YAML.compose(events, resolver) # Process node tree directly if node isa MappingNode for (key_node, value_node) in node.value println("Key: $(key_node.value), Value: $(value_node.value)") end end ``` -------------------------------- ### Write Data to YAML File Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/README.md Shows how to write Julia data structures to a YAML file. Requires importing YAML and using a Dict for data. ```julia import YAML data = Dict( "person" => Dict( "name" => "Bob", "age" => 25 ), "active" => true ) YAML.write_file("output.yml", data) ``` -------------------------------- ### Define StreamStartToken and StreamEndToken Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Represent the beginning and end of a token stream. `StreamStartToken` includes encoding information. ```julia struct StreamStartToken <: Token span::Span encoding::String end struct StreamEndToken <: Token span::Span end ``` -------------------------------- ### Basic YAML Error Handling Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/errors.md Demonstrates a basic try-catch block for handling common YAML parsing errors like ScannerError, ParserError, ComposerError, and ConstructorError. ```julia import YAML try data = YAML.load(yaml_string) catch e::Union{ScannerError, ParserError, ComposerError, ConstructorError} @error "YAML parsing failed" error = e return nothing end ``` -------------------------------- ### Constructor Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-constructor.md Manages the construction of Julia objects from YAML nodes, supporting custom types and handling circular references. ```APIDOC ## Constructor Mutable struct that manages object construction from YAML nodes. ```julia mutable struct Constructor constructed_objects::Dict{Node, Any} recursive_objects::Set{Node} yaml_constructors::Dict{Union{String, Nothing}, Function} yaml_multi_constructors::Dict{Union{String, Nothing}, Function} end ``` ### Fields | Field | Type | Description | |---|---|---| | constructed_objects | `Dict{Node, Any}` | Cache of already-constructed objects to handle circular references | | recursive_objects | `Set{Node}` | Set of nodes currently being constructed, used to detect infinite recursion | | yaml_constructors | `Dict{Union{String, Nothing}, Function}` | Maps YAML tags (e.g., "tag:yaml.org,2002:int") to constructor functions | | yaml_multi_constructors | `Dict{Union{String, Nothing}, Function}` | Maps tag prefixes to constructor functions that handle multiple related tags | ### Constructor ```julia Constructor(single_constructors = Dict(), multi_constructors = Dict()) ``` Creates a new Constructor with optional custom constructors. The `single_constructors` dict maps tags to functions, and `multi_constructors` dict maps tag prefixes. ### Examples ```julia # Create a basic constructor cons = YAML.Constructor() # Create with custom constructors custom_cons = Dict("!custom" => my_constructor_func) cons = YAML.Constructor(custom_cons) ``` ``` -------------------------------- ### Stage 2: Parser - EventStream Operations Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/parsing-pipeline.md Create an event stream from a token stream, peek at the next event, and advance to the next event. ```julia events = YAML.EventStream(ts) event = peek(events) event = forward!(events) ``` -------------------------------- ### DocumentStartToken and DocumentEndToken Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Represent YAML document markers (`---` and `...`). Each token stores its source location. ```APIDOC ## DocumentStartToken, DocumentEndToken ### Description Represent YAML document markers (`---` and `...`). Each token stores its source location. ### Fields - **span** (`Span`) - Source location of the token ``` -------------------------------- ### ConstructorError Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/types.md Exception raised during object construction. It includes context, problem, and optional notes about the error. ```APIDOC ### ConstructorError Exception raised during object construction. ```julia struct ConstructorError <: Exception context::Union{String, Nothing} context_mark::Union{Mark, Nothing} problem::Union{String, Nothing} problem_mark::Union{Mark, Nothing} note::Union{String, Nothing} end ``` ``` -------------------------------- ### Stage 4: Constructor - Document Construction Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/parsing-pipeline.md Construct Julia objects from a node tree using a safe constructor. ```julia cons = YAML.SafeConstructor() obj = YAML.construct_document(cons, node) ``` -------------------------------- ### Load YAML Data from String and File Source: https://github.com/juliadata/yaml.jl/blob/master/_autodocs/api-reference-parser.md Use YAML.load to parse a YAML string or YAML.load_file to parse a YAML file. These functions utilize the event stream internally for parsing. ```julia import YAML # These use EventStream internally: data = YAML.load("key: value") data = YAML.load_file("file.yml") ```