### Combinator Composition Example Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Shows how combinators are built by prepending to a list, demonstrating the use of `string` and `ascii_char` combinators. ```elixir empty() # [] |> string("hello") # [{:string, "hello"}] |> ascii_char([?a..?z]) # [{:bin_segment, [?a..?z], [], :integer}, {:string, "hello"}] ``` -------------------------------- ### Example of NimbleParsec Parse Success Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/00-index.md A practical example showing a successful parse operation using a hypothetical MyParser, demonstrating the {:ok, ...} return tuple. ```elixir MyParser.parse("hello world") # {:ok, ["hello"], " world", %{}, {1, 0}, 5} ``` -------------------------------- ### Position Tracking Example Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Illustrates how line and column information is tracked during parsing, with the column representing the byte offset from the start of the line. ```elixir {line, column} = position # line: 1-indexed line number # column: byte offset from line start ``` -------------------------------- ### Building a Date Parser with Nimble Parsec Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/02-types.md An example demonstrating how to build a date parser using combinators, including defining custom components and post-processing for validation. ```elixir defmodule DateParser do import NimbleParsec # Define a combinator for parsing a date component (min: 1, max: 2 digits) # This returns type t defcombinatorp :component, integer(min: 1, max: 2) # Define the main parser defparsec :date, parsec(:component) |> ignore(string("/")) |> parsec(:component) |> ignore(string("/")) |> parsec(:component) |> post_traverse(:validate_date) # Callback receives rest, acc, context, position, offset # Note: acc is reversed, so [day, month, year] is in reverse order defp validate_date(rest, [year, month, day], context, _pos, _offset) do if day >= 1 and day <= 31 and month >= 1 and month <= 12 and year > 0 do {rest, [{day, month, year}], context} else {:error, "invalid date"} end end end # Usage DateParser.date("25/12/2025") # {:ok, [{25, 12, 2025}], "", %{}, {1, 0}, 8} ``` -------------------------------- ### Entry Point Options Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Details the options that can be passed when calling a parser function generated by `defparsec/3`, including starting byte offset, line information, and initial context. ```APIDOC ## Entry Point Options When calling a parser function generated by `defparsec/3`, you can pass options: ```elixir MyParser.parse(binary, opts) ``` **Available options:** | Option | Type | Default | Description | |--------|------|---------|-------------| | :byte_offset | non_neg_integer | 0 | Starting byte offset for position tracking | | :line | pos_integer | {1, byte_offset} | Starting line and offset info | | :context | keyword | [] | Initial context map (converted to map) | **Example:** ```elixir MyParser.parse("hello", byte_offset: 10, line: {2, 0}, context: [state: :initial]) ``` ``` -------------------------------- ### NimbleParsec Installation Dependency Source: https://github.com/dashbitco/nimble_parsec/blob/master/README.md Specifies the dependency to add to your `mix.exs` file for installing the NimbleParsec library. ```elixir def deps do [ {:nimble_parsec, "~> 1.0"} ] end ``` -------------------------------- ### Example of NimbleParsec Parse Failure Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/00-index.md An example illustrating a failed parse operation with MyParser, showcasing the {:error, ...} return tuple and a sample error reason. ```elixir MyParser.parse("!") # {:error, "expected ...", "!", %{}, {1, 0}, 0} ``` -------------------------------- ### Combinator Inlining Example Source: https://github.com/dashbitco/nimble_parsec/blob/master/README.md Demonstrates how combinators are inlined and compiled multiple times when reused directly, potentially impacting compile times and memory usage. ```elixir date_then_time = concat(date, time) time_then_date = concat(time, date) defparsec :combinations, choice([date_then_time, time_then_date]) ``` -------------------------------- ### Unit Testing Parsers with ExUnit Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/03-patterns-and-examples.md Demonstrates how to unit test Nimble Parsec parsers using ExUnit. Includes examples for parsing positive integers, handling errors, and parsing with options like byte offset and line number. ```elixir defmodule NumberParserTest do use ExUnit.Case import NimbleParsec defparsec :number, integer(min: 1) test "parses positive integers" do assert {:ok, [42], "", %{}, {1, 0}, 2} = number("42") end test "returns error for non-digits" do {:error, _reason, _rest, _context, _line, _offset} = number("abc") end test "parses with options" do assert {:ok, [123], rest, %{}, line, offset} = number("123remaining", byte_offset: 10, line: {2, 5}) end end ``` -------------------------------- ### Nimble Parsec Parser Call with Options Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Provides an example of calling a parser with specific options for byte offset, line number, and initial context. ```elixir MyParser.parse("hello", byte_offset: 10, line: {2, 0}, context: [state: :initial]) ``` -------------------------------- ### Context Map Modification Example Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Shows how the `context` map can be initialized from parser options and updated during parsing by traverse callbacks. ```elixir # Initial context from parser options context = Map.new(Keyword.get(opts, :context, [])) # Traverse callbacks can update context {rest, acc, new_context} = callback(rest, acc, context, pos, offset) ``` -------------------------------- ### Parsed Values Accumulator Example Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Demonstrates how the accumulator (`acc`) stores parsed results in reverse order for efficient O(1) prepending, with a final reversal at the end of parsing. ```elixir # Parsing "abc" with three ascii_char calls: # After first char: acc = [?a] # After second char: acc = [?b, ?a] # After third char: acc = [?c, ?b, ?a] # Final result (reversed): [?a, ?b, ?c] ``` -------------------------------- ### Create an empty combinator Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Use `empty/0` to create a starting combinator for building parsers with the pipe operator. It returns an empty list internally. ```elixir @spec empty() :: t def empty() ``` ```elixir empty() |> ascii_char([?a..?z]) |> ascii_char([?0..?9]) ``` -------------------------------- ### Understand Position Calculation in Multiline Parsing Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/07-errors-and-troubleshooting.md Be aware that line and column information is automatically updated but may not always match user expectations. Line starts at 1, column at 0 (byte offset from line start). ```elixir defparsec :multiline, string("line1\n") |> string("line2") |> post_traverse(:check_position) defp check_position(rest, acc, context, {line, column}, offset) do # line and column are updated automatically # but may not match user expectations {rest, acc, context} end ``` -------------------------------- ### Parser Success Return Value Example Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/02-types.md Demonstrates the structure of a successful parser return tuple, including parsed terms, remaining input, context, and position. ```elixir MyParser.parse("hello123") # {:ok, ["hello", 123], "", %{}, {1, 0}, 8} ``` -------------------------------- ### Error Handling Example Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Demonstrates how Nimble Parsec automatically generates error messages from combinators and labels, providing an example of an input that triggers an error. ```APIDOC ## Error Handling The library automatically generates error messages from combinators and labels. For example: ```elixir defparsec :digit, ascii_char([?0..?9]) |> label("a digit") # Input "a" produces: # {:error, "expected a digit", "a", %{}, {1, 0}, 0} ``` ``` -------------------------------- ### empty/0 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/06-module-reference.md Returns an empty combinator, typically used as a starting point for building more complex parsers. ```APIDOC ## empty/0 ### Description Returns an empty combinator used as a starting point. ### Syntax ```elixir @spec empty() :: t def empty() ``` ``` -------------------------------- ### Catch-All Error Handling Clause Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Provides an example of a catch-all clause in generated parsers that handles parsing failures by returning an {:error, ...} tuple. ```elixir # Success case def parse__0(<>, acc, ...) when guards do ... end # Catch-all for failure def parse__0(rest, _acc, _stack, context, line, column) do {:error, "expected ...", rest, context, line, column} end ``` -------------------------------- ### Stateful Parsing with Custom Context Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/05-advanced-usage.md Implement stateful parsing by using `post_traverse` callbacks to modify the parser's context. This example tracks indentation levels during parsing. ```elixir defmodule StatefulParser do import NimbleParsec # Track indentation level defparsec :indented_block, post_traverse(:push_indent) |> repeat( lookahead(ascii_char([?\s])) |> times(ascii_char([?\s]), min: 2) |> ascii_string([{:not, ?\n}], min: 1) |> optional(string("\n")) ) |> post_traverse(:pop_indent) defp push_indent(rest, _acc, context, _pos, _offset) do indent = Map.get(context, :indent_level, 0) + 1 context = Map.put(context, :indent_level, indent) {rest, [], context} end defp pop_indent(rest, acc, context, _pos, _offset) do indent = Map.get(context, :indent_level, 1) - 1 context = Map.put(context, :indent_level, indent) {rest, acc, context} end end # Usage with initial context StatefulParser.indented_block(input, context: [indent_level: 0]) ``` -------------------------------- ### Nimble Parsec Error Handling Example Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Demonstrates how Nimble Parsec generates error messages, including the use of the `label` combinator. ```elixir defparsec :digit, ascii_char([?0..?9]) |> label("a digit") # Input "a" produces: # {:error, "expected a digit", "a", %{}, {1, 0}, 0} ``` -------------------------------- ### Parser Error Return Value Example Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/02-types.md Illustrates the structure of a parser error return tuple, containing the error reason, input at failure, context, and position. ```elixir MyParser.parse("not a digit") # {:error, "expected ASCII character...", "not a digit", %{}, {1, 0}, 0} ``` -------------------------------- ### Tag Validation with Context Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/05-advanced-usage.md Validate parsed data using context state. This example tracks encountered tags and performs a final validation check. ```elixir defmodule TagValidator do import NimbleParsec defparsec :tagged_list, repeat( string("<") |> ascii_string([{:not, ?>}], min: 1) |> string(">") |> post_traverse(:track_tag) ) |> post_traverse(:validate_tags) defp track_tag(rest, [tag], context, _pos, _offset) do tags = Map.get(context, :tags, []) context = Map.put(context, :tags, [tag | tags]) {rest, [tag], context} end defp validate_tags(rest, acc, context, _pos, _offset) do case context do %{tags: tags} when is_list(tags) -> {rest, [Enum.reverse(tags)], context} _ -> {:error, "Invalid tag structure"} end end end ``` -------------------------------- ### Create a Configuration Language DSL Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/05-advanced-usage.md Builds a domain-specific language for parsing configuration files using Nimble Parsec's combinators. Demonstrates keyword, identifier, integer, and string value parsers. ```elixir defmodule ConfigDSL do import NimbleParsec # High-level helpers for a configuration language def keyword(word) do string(word) |> lookahead_not(ascii_char([?a..?z, ?0..?9])) end def identifier do ascii_string([?a..?z, ?A..?Z, ?_], min: 1) end def integer_value do integer(min: 1) end def string_value do ignore(string("\"")) |> ascii_string([{:not, ?"}], min: 0) |> ignore(string("\"")) end # Configuration parser defparsec :config, repeat( choice([ keyword("debug") |> ignore(string(":")) |> choice([string("true") |> replace(true), string("false") |> replace(false)]) |> tag(:debug), keyword("port") |> ignore(string(":")) |> integer_value() |> tag(:port), keyword("host") |> ignore(string(":")) |> string_value() |> tag(:host) ]) |> optional(ignore(string("\n"))) ) end ConfigDSL.config("debug: true\nport: 8080\nhost: \"localhost\"") ``` -------------------------------- ### Step-by-Step Parser Testing Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/07-errors-and-troubleshooting.md Defines two parser steps and demonstrates how to test each step individually using `IO.inspect`. This helps isolate issues in complex parsers. ```elixir defparsec :step1, ascii_char([?a..?z]) defparsec :step2, step1 |> integer(min: 1) # Test each step IO.inspect(MyParser.step1("a123")) IO.inspect(MyParser.step2("a123")) ``` -------------------------------- ### Nimble Parsec Parser Entry Point Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Shows how to call a parser function generated by `defparsec/3` with optional arguments. ```elixir MyParser.parse(binary, opts) ``` -------------------------------- ### Empty Combinator Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/06-module-reference.md Returns an empty combinator, typically used as a starting point for building more complex parsers. ```Elixir @spec empty() :: t def empty() ``` -------------------------------- ### string/1, string/2 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Matches a literal string value. This combinator can be chained with previous combinators or used to start a new parsing sequence. ```APIDOC ## string/1, string/2 ### Description Matches a literal string value. ### Method `string(combinator \ empty(), binary)` ### Parameters #### Path Parameters - **combinator** (t) - Optional - The previous combinator to chain. Defaults to `empty()`. - **binary** (binary) - Required - The exact string to match. ### Return Value A combinator that matches the given binary string. ### Example ```elixir defparsec :greeting, string("hello") # "hello" -> {:ok, ["hello"], "", %{}, {1, 0}, 5} ``` ``` -------------------------------- ### Entry Points Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/00-index.md Functions for defining public and private parsers and combinators. ```APIDOC ## Entry Points ### Description Functions for defining public and private parsers and combinators. ### API Reference | Function | Purpose | |----------|---------| | `defparsec/3` | Define public parser function | | `defparsecp/3` | Define private parser function | | `defcombinator/3` | Define public combinator only | | `defcombinatorp/3` | Define private combinator only | ``` -------------------------------- ### choice/2, choice/3 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/06-module-reference.md Tries multiple alternative combinators in the order they are provided. Requires a minimum of two choices. ```APIDOC ## choice/2, choice/3 ### Description Tries multiple alternatives in order. ### Function Signature ```elixir @spec choice(nonempty_list(t)) :: t @spec choice(t, nonempty_list(t)) :: t @spec choice(t, nonempty_list(t), opts) :: t def choice(combinator \ empty(), [_, _ | _] = choices, opts \ []) ``` ### Requirements Minimum 2 choices. ### Options - `:gen_weights` — List of positive integers for weighting choices in `generate/1` ### Implementation Details Uses `{:choice, choices, weights}` instruction. ``` -------------------------------- ### Line Combinator Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Wraps the result of a combinator with line and offset information. Use this to get line numbers and column offsets for parsed data. ```elixir @spec line(t) :: t @spec line(t, t) :: t def line(combinator \ empty(), to_wrap) ``` ```elixir defparsec :with_line, string("hello") |> line() # "hello" -> {:ok, [{["hello"], {1, 0}}], "", %{}, {1, 0}, 5} ``` -------------------------------- ### line/1, line/2 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Wraps the result of a combinator with line information, returning a tuple `{result, {line, offset_to_line_start}}`. It takes an optional previous combinator and the combinator to wrap, returning a new combinator that adds line/column information. ```APIDOC ## line/1, line/2 ### Description Wraps the result of a combinator with line information: `{result, {line, offset_to_line_start}}`. ### Method N/A (Elixir function) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example ```elixir defparsec :with_line, string("hello") |> line() # "hello" -> {:ok, [{["hello"], {1, 0}}], "", %{}, {1, 0}, 5} ``` ### Response #### Success Response (200) A combinator that adds line/column information. #### Response Example N/A ``` -------------------------------- ### Optimized Combinator Reuse with defpcombinatorp Source: https://github.com/dashbitco/nimble_parsec/blob/master/README.md Shows how to use `defpcombinatorp` to define and reuse combinators, improving compile-time performance by avoiding redundant compilation. ```elixir defcombinatorp :date, ... defcombinatorp :time, ... date_then_time = concat(parsec(:date), parsec(:time)) time_then_date = concat(parsec(:time), parsec(:date)) defparsec :combinations, choice([date_then_time, time_then_date]) ``` -------------------------------- ### Binary Tail Handling for Memory Efficiency Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Captures the rest of the input as a binary tail (`::binary`) for zero-copy efficiency when possible. ```elixir <> # rest is a reference to the original binary tail, not a copy ``` -------------------------------- ### Create Reusable Combinator Abstractions with Macros Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/05-advanced-usage.md Defines custom combinators like `optional_whitespace`, `csv`, and `bracketed` using macros for reusable parsing logic. Demonstrates how to build more complex parsers from simpler ones. ```elixir defmodule CombinatorHelpers do import NimbleParsec # Helper for optional whitespace def optional_whitespace(combinator \ empty()) do repeat(ascii_char([?\s, ?\t])) |> concat(combinator) end # Helper for comma-separated values def csv(item_parser) do item_parser |> repeat( ignore(string(",")) |> optional(optional_whitespace()) |> concat(item_parser) ) end # Helper for bracketed content def bracketed(combinator, open \ "[", close \ "]") do ignore(string(open)) |> optional_whitespace() |> concat(combinator) |> optional_whitespace() |> ignore(string(close)) end end # Usage defmodule MyParser do import NimbleParsec import CombinatorHelpers item = integer(min: 1) array = bracketed(csv(item)) defparsec :array, array end MyParser.array("[1, 2, 3]") ``` -------------------------------- ### Optimize Many Alternatives with Specialized Structures Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/07-errors-and-troubleshooting.md For parsers with many alternatives, consider using tries or specialized structures instead of a large `choice` list to improve performance. ```elixir # Slow: tries 100 choices defparsec :slow, choice([keyword1, keyword2, ..., keyword100]) # Better: use trie or specialized structure ``` -------------------------------- ### Custom Transform using post_traverse Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Demonstrates creating a custom transform `as_atom` using `post_traverse/3` to convert a parsed string to an atom. ```elixir defmodule Transform do import NimbleParsec defparsec :to_atom, ascii_string([?a..?z], min: 1) |> post_traverse(:as_atom) defp as_atom(rest, [str], context, _pos, _offset) do {rest, [String.to_atom(str)], context} end end ``` -------------------------------- ### Generating Random Data for Property-Based Testing Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/03-patterns-and-examples.md Shows how to use `export_metadata: true` and `NimbleParsec.generate/1` to create random, valid inputs for testing parsers. ```elixir defmodule IDParser do import NimbleParsec # Generates identifiers like "foo_bar_123" defparsec :identifier, ascii_string([?a..?z, ?_], min: 1) |> repeat(ascii_string([?a..?z, ?0..?9, ?_], min: 1)), export_metadata: true end # Generate random identifiers for testing 1..10 |> Enum.map(fn _ -> NimbleParsec.parsec({IDParser, :identifier}) |> NimbleParsec.generate() end) # Generates valid identifiers like "a", "foo_bar", "x_1_y_2", etc. ``` -------------------------------- ### Ignore Combinator Result Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Use `ignore` to match a combinator but exclude its result from the final output. This is useful for parsing delimiters or other structural elements that are necessary for the parse but not part of the desired data. The example shows parsing a date string while ignoring the hyphens. ```elixir defparsec :date_parts, integer(4) |> ignore(string("-")) |> integer(2) |> ignore(string("-")) |> integer(2) ``` -------------------------------- ### Reusable Parsers with `defcombinatorp` for Performance Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/03-patterns-and-examples.md Compares inefficient repeated compilation of parsers with the performant approach using `defcombinatorp` to define and reuse compiled parser combinators. ```elixir defmodule GoodPerf do import NimbleParsec defcombinatorp :date, integer(4) |> ignore(string("-")) |> integer(2) |> ignore(string("-")) |> integer(2) defcombinatorp :time, integer(2) |> ignore(string(":")) |> integer(2) |> ignore(string(":")) |> integer(2) # These reuse the compiled combinators defparsec :date_time_1, parsec(:date) |> parsec(:time) defparsec :date_time_2, parsec(:time) |> parsec(:date) end ``` -------------------------------- ### Primitives Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/00-index.md Basic combinators for matching literal strings, characters, integers, and byte sequences. ```APIDOC ## Primitives ### Description Basic combinators for matching literal strings, characters, integers, and byte sequences. ### API Reference | Function | Matches | |----------|---------| | `string/2` | Literal string | | `ascii_char/2` | Single ASCII character in ranges | | `utf8_char/2` | Single UTF-8 codepoint in ranges | | `integer/2` | Integer (fixed or variable length) | | `ascii_string/3` | ASCII string of specified length | | `utf8_string/3` | UTF-8 string of specified length | | `bytes/2` | Raw bytes (no interpretation) | | `eos/1` | End of string assertion | | `empty/0` | Empty combinator (starting point) | ``` -------------------------------- ### Custom Context State Management Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/00-index.md Demonstrates passing and updating custom context state during parsing using post-traversal hooks. ```elixir defparsec :parser, post_traverse(:init_context) |> repeat(item) |> post_traverse(:finalize) ``` -------------------------------- ### bytes/1, bytes/2 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/06-module-reference.md Matches exactly `count` bytes without interpretation. It uses the `{:bytes, count}` instruction, compiled to `<<_::binary-size(count)>>` pattern. ```APIDOC ## bytes/1, bytes/2 ### Description Matches exactly `count` bytes without interpretation. ### Function Signature ```elixir @spec bytes(pos_integer) :: t @spec bytes(t, pos_integer) :: t def bytes(combinator \ empty(), count) when is_combinator(combinator) and is_integer(count) and count > 0 ``` ### Implementation Details Uses `{:bytes, count}` instruction, compiled to `<<_::binary-size(count)>>` pattern. ``` -------------------------------- ### Choice Combinator Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Use `choice/2` or `choice/3` to try multiple combinators in order until one succeeds. Requires at least two choices. ```Elixir def choice(combinator \ empty(), [_, _ | _] = choices, opts \ []) ``` ```Elixir defparsec :keyword_or_id, choice([ string("if") |> replace(:if), string("while") |> replace(:while), ascii_string([?a..?z], min: 1) ]) ``` -------------------------------- ### Correct Function Definition Order with Variable Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Shows the correct way to define a parser by assigning it to a variable before using it in `defparsec`. ```elixir # RIGHT date = integer(4) |> ignore(string("-")) |> integer(2) defparsec :good, date ``` -------------------------------- ### Choice Between Alternatives Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/06-module-reference.md Tries multiple combinators in sequence until one succeeds. Requires at least two choices. Supports weighted choices for generation. ```Elixir @spec choice(nonempty_list(t)) :: t @spec choice(t, nonempty_list(t)) :: t @spec choice(t, nonempty_list(t), opts) :: t def choice(combinator \ empty(), [_, _ | _] = choices, opts \ []) ``` -------------------------------- ### Generated Parser Entry Point Signature Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md The typical signature for a generated parser entry point function, accepting a binary and options. ```elixir def name(binary, opts \ \ []) when is_binary(binary) ``` -------------------------------- ### Descriptive Error Messages with Labels Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/03-patterns-and-examples.md Defines a parser for configuration lines (key=value) and demonstrates how labels improve error messages when parsing fails. ```elixir defmodule ConfigParser do import NimbleParsec key = ascii_string([?a..?z, ?A..?Z, ?_], min: 1) |> label("configuration key") value = ascii_string([{:not, ?\n}], min: 1) |> label("configuration value") defparsec :config_line, key |> ignore(string("=") |> label("equals sign")) |> concat(value) |> label("key=value pair") end ConfigParser.config_line("timeout=30") # {:ok, ["timeout", "30"], "", %{}, {1, 0}, 11} ConfigParser.config_line("timeout") # {:error, "expected equals sign", "", %{}, {1, 0}, 7} ``` -------------------------------- ### Eager vs Lazy Evaluation for Performance Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/05-advanced-usage.md Demonstrates eager evaluation where multiple `repeat` combinators are chained. Eager evaluation typically leads to slower compilation but potentially faster runtime compared to lazy alternatives. ```elixir # Eager: slower compilation, faster runtime defparsec :eager, repeat(ascii_char([?a..?z])) |> repeat(ascii_char([?0..?9])) |> repeat(ascii_char([?a..?z])) ``` -------------------------------- ### Internal Combinator Representation Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Illustrates the internal structure of combinators as a list of instructions, stored in reverse order for efficient prepending. ```elixir [ {:instruction_type, arg1, arg2, ...}, {:another_type, arg1, arg2}, ... ] ``` -------------------------------- ### Defining a Parser for a Literal String Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/07-errors-and-troubleshooting.md Defines a parser that expects the literal string 'hello'. This is useful for matching fixed text. ```elixir defparsec :greeting, string("hello") ``` -------------------------------- ### Define a Basic Date Parser Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/00-index.md Defines a parser for a YYYY-MM-DD date format using NimbleParsec's `defparsec`, `integer`, and `string` combinators. It ignores the hyphens and returns the year, month, and day as a list. ```elixir defmodule MyParser do import NimbleParsec defparsec :date, integer(4) |> ignore(string("-")) |> integer(2) |> ignore(string("-")) |> integer(2) end MyParser.date("2025-06-07") # {:ok, [2025, 6, 7], "", %{}, {1, 0}, 10} ``` -------------------------------- ### defcombinatorp/3 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/06-module-reference.md Creates a private combinator, similar to `defcombinator/3` but generates a private combinator. ```APIDOC ## defcombinatorp/3 ### Description Creates a private combinator. ### Syntax ```elixir defmacro defcombinatorp(name, combinator, opts \ []) ``` ``` -------------------------------- ### bytes/1, bytes/2 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Consumes the next `n` bytes from the input without interpreting them. It can chain with a previous combinator and requires the exact number of bytes to consume. ```APIDOC ## bytes/1, bytes/2 ### Description Consumes the next `n` bytes from the input without interpreting them. ### Parameters #### Path Parameters - None #### Query Parameters - None #### Request Body - None ### Parameters - **combinator** (t) - Optional - The previous combinator to chain. Defaults to `empty()`. - **count** (pos_integer) - Required - The exact number of bytes to consume. ### Return Value A combinator that matches exactly `count` bytes and returns them as a binary. ### Example ```elixir defparsec :three_bytes, bytes(3) # "abc" -> {:ok, ["abc"], "", %{}, {1, 0}, 3} ``` ``` -------------------------------- ### Maintaining Parsing State with Context Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/03-patterns-and-examples.md Shows how to use the `context` option to maintain and update state during a parsing process, such as accumulating parsed items. ```elixir defmodule ContextParser do import NimbleParsec defparsec :items, repeat( integer(min: 1) |> post_traverse(:track_items) ) defp track_items(rest, [item], context, _pos, _offset) do items = Map.get(context, :items, []) new_context = Map.put(context, :items, [item | items]) {rest, [item], new_context} end end ContextParser.items("1 2 3", context: [state: :counting]) # {:ok, [1, 2, 3], " ", %{items: [3, 2, 1], state: :counting}, {1, 0}, 5} ``` -------------------------------- ### Test Error Recovery with `choice` Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/07-errors-and-troubleshooting.md Demonstrate how the `choice` combinator can recover from errors by trying alternative parsers. ```elixir test "error recovery with choice" do # choice tries alternatives on error {:ok, result, _, _, _, _} = MyParser.flexible_parser("fallback_value") assert result == ["fallback"] end ``` -------------------------------- ### Implement Manual Line Tracking Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/07-errors-and-troubleshooting.md Manually track line numbers by using `post_traverse/2` within a loop that processes lines and updates context with the current line number. ```elixir defmodule LineTracker do import NimbleParsec defparsec :lines, repeat( ascii_string([{:not, ?\n}], min: 0) |> post_traverse(:update_line) |> optional(string("\n")) ) defp update_line(rest, acc, context, {line, _column}, _offset) do new_context = Map.put(context, :current_line, line) {rest, acc, new_context} end end ``` -------------------------------- ### Infinite Loop with `repeat(optional(...))` Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/07-errors-and-troubleshooting.md Demonstrates a parser that can enter an infinite loop because `optional/2` can always match an empty value, preventing `repeat/2` from terminating. This is an anti-pattern. ```elixir defparsec :bad, repeat(optional(ascii_char([?a..?z]))) ``` -------------------------------- ### Adding Line Information to Parse Results Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/03-patterns-and-examples.md Demonstrates how to use the `line()` combinator to enrich parse results with line and column tracking information, useful for error reporting. ```elixir defmodule JSONParser do import NimbleParsec # Simplified JSON value parser string_value = ignore(string("\"")) |> ascii_string([{:not, ?"}], min: 0) |> ignore(string("\"")) number_value = integer(min: 1) defparsec :value, choice([string_value, number_value]) |> line() # Add line information end JSONParser.value("\"hello\"", line: {5, 10}) # {:ok, [{["hello"], {5, 10}}], "", %{}, {5, 10}, 17} ``` -------------------------------- ### Defparsec Macros for Parser Compilation Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md These macros initiate the NimbleParsec compilation process by expanding combinator specifications and calling the compiler. ```elixir defmacro defparsec(name, combinator, opts \ \ []) defmacro defparsecp(name, combinator, opts \ \ []) ``` -------------------------------- ### Generic Options (opts) Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/02-types.md A generic keyword list for passing options to various Nimble Parsec functions. Specific functions define their own valid option keys. ```elixir @type opts :: Keyword.t() defparsec :my_parser, string("hello"), inline: true, debug: false, export_metadata: true ``` -------------------------------- ### Using Labels for Context in Parsers Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/07-errors-and-troubleshooting.md Applies multiple labels to different stages of a parser pipeline to provide detailed context in error messages. This improves the clarity of parsing failures. ```elixir defparsec :parsed, ascii_char([?a..?z]) |> label("first letter") |> integer(min: 1) |> label("following number") |> label("letter-number pair") ``` -------------------------------- ### Parse URL Query Parameters for Phoenix/Plug Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/05-advanced-usage.md Define a parser for URL query strings and a helper function to integrate with Phoenix controllers. ```elixir defmodule QueryParser do import NimbleParsec # Parse URL query parameters key = ascii_string([?a..?z, ?A..?Z, ?_], min: 1) value = ascii_string([{:not, ?&}, {:not, ?=}], min: 1) param = key |> ignore(string("=")) |> concat(value) |> tag(:param) defparsec :query_string, param |> repeat(ignore(string("&")) |> concat(param)) # Helper for Phoenix def parse_query(query_string) do case query_string(query_string) do {:ok, params, "", _context, _line, _offset} -> {:ok, Enum.map(params, &extract_param/1) |> Enum.into(%{})} {:error, reason, _rest, _context, _line, _offset} -> {:error, reason} end end defp extract_param({:param, [value, key]}), do: {key, value} end # Usage in Phoenix controller defmodule MyController do def search(conn, _params) do query = conn.query_string case QueryParser.parse_query(query) do {:ok, params} -> # Use parsed params json(conn, params) {:error, reason} -> send_resp(conn, 400, "Invalid query: #{reason}") end end end ``` -------------------------------- ### generate/1 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Generates a random binary that would be parseable by the given combinator, useful for testing parsers. It takes a combinator to generate from and returns a randomly generated binary. ```APIDOC ## generate/1 ### Description Generates a random binary that would be parseable by the given combinator. Useful for testing parsers. ### Method N/A (Elixir function) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example ```elixir defmodule MyParser do import NimbleParsec defparsec :greeting, choice([string("hello"), string("hi")]), export_metadata: true end NimbleParsec.parsec({MyParser, :greeting}) |> NimbleParsec.generate() # Might return "hello" or "hi" ``` ### Response #### Success Response (200) A randomly generated binary that matches the combinator pattern. #### Response Example N/A ``` -------------------------------- ### Inspect Generated Functions in IEx Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Use `erlang.fun_info/1` in IEx to inspect details about compiled generated functions. ```elixir iex> :erlang.fun_info(:erlang.fun_info(&MyParser.date__0/6)) ``` -------------------------------- ### Recorder Module replay/2 Function Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md The `replay/2` function is used by the `mix nimble_parsec.compile` task to inject compiled parsers back into source files, looking for specific markers. ```elixir def replay(contents, id) ``` -------------------------------- ### Property-Based Testing for Parsers Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/05-advanced-usage.md Use PropCheck to generate random inputs and verify parser behavior, ensuring correctness across a wide range of cases. ```elixir defmodule ParserProperties do use ExUnit.Case use PropCheck defmodule TestParser do import NimbleParsec defparsec :identifier, ascii_string([?a..?z, ?_], min: 1), export_metadata: true end property "generated identifiers parse successfully" do forall _sample <- 1..100 do input = NimbleParsec.parsec({TestParser, :identifier}) |> NimbleParsec.generate() {:ok, _result, "", _context, _line, _offset} = TestParser.identifier(input) true end end property "parsed identifiers have correct length" do forall length <- :propcheck.integer(1..20) do defparsec_opts = [export_metadata: true] parser = ascii_string([?a..?z], length) input = "a" |> String.duplicate(length) |> Kernel.<>("extra") # length matches exactly n characters {:ok, [result], "extra", _context, _line, _offset} = # Parse exactly length characters result == String.duplicate("a", length) end end end ``` -------------------------------- ### NimbleParsec.Compiler.entry_point/1 Function Signature Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md The signature for the function that generates the public entry point for a parser, returning documentation, type specification, and definition tuple. ```elixir @spec entry_point(name :: atom) :: {doc :: binary, spec :: Macro.t, def :: tuple} ``` -------------------------------- ### byte_offset/1, byte_offset/2 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Wraps the result of a combinator with its byte offset, returning a tuple `{result, offset}`. It takes an optional previous combinator and the combinator to wrap, returning a new combinator that adds byte offset information. ```APIDOC ## byte_offset/1, byte_offset/2 ### Description Wraps the result of a combinator with its byte offset: `{result, offset}`. ### Method N/A (Elixir function) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example ```elixir defparsec :with_offset, string("hello") |> byte_offset() # "hello" -> {:ok, [{["hello"], 0}], "", %{}, {1, 0}, 5} ``` ### Response #### Success Response (200) A combinator that adds byte offset information. #### Response Example N/A ``` -------------------------------- ### Times Combinator (Min/Max Range) Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Use `times/2` or `times/3` with a range to repeat a combinator a minimum and maximum number of times. This provides flexibility for variable-length patterns. ```Elixir defparsec :two_to_four, times(ascii_char([?a..?z]), min: 2, max: 4) ``` -------------------------------- ### repeat/1, repeat/2, repeat/3 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/01-api-reference.md Allows a combinator to appear zero or more times. ```APIDOC ## repeat/1, repeat/2, repeat/3 ### Description Allows a combinator to appear zero or more times. ### Method N/A (Elixir function) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example ```elixir defparsec :digits, repeat(ascii_char([?0..?9])) # "12345" -> {:ok, [?1, ?2, ?3, ?4, ?5], "", %{}, {1, 0}, 5} # "abc" -> {:ok, [], "abc", %{}, {1, 0}, 0} ``` ### Response #### Success Response (200) A combinator that matches the given combinator zero or more times. #### Response Example N/A ``` -------------------------------- ### NimbleParsec Success Return Format Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/00-index.md Demonstrates the expected return format for a successful parse operation in NimbleParsec, including the parsed tokens, remaining input, context, line/column, and byte offset. ```elixir # Success {:ok, tokens, rest, context, {line, column}, byte_offset} ``` -------------------------------- ### Custom DSL Keyword Parser Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/00-index.md Creates a custom DSL helper function 'keyword/1' that parses a given word followed by a negative lookahead to ensure it's not part of a larger word. ```elixir def keyword(word) do string(word) |> lookahead_not(ascii_char([?a..?z])) end ``` -------------------------------- ### pre_traverse/2, pre_traverse/3 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/06-module-reference.md Similar to `post_traverse`, but the position is taken from BEFORE the combinators are applied. It also allows for pre-processing of results. ```APIDOC ## pre_traverse/2, pre_traverse/3 ### Description Same as `post_traverse` but with position from BEFORE the combinators. ### Method N/A (Elixir function signature) ### Endpoint N/A (Elixir function signature) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example N/A ### Response #### Success Response (200) Callback must return: `{rest, acc, context}` or `{:error, reason}` #### Response Example N/A ``` -------------------------------- ### repeat_while/2, repeat_while/3, repeat_while/4 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/06-module-reference.md Repeats a combinator based on a callback function that returns either `{:cont, context}` or `{:halt, context}` to control the repetition. ```APIDOC ## repeat_while/2, repeat_while/3, repeat_while/4 ### Description Repeats based on a callback function returning `{:cont, context}` or `{:halt, context}`. ### Function Signature ```elixir @spec repeat_while(t, call) :: t @spec repeat_while(t, t, call) :: t @spec repeat_while(t, t, call, opts) :: t def repeat_while(combinator \ empty(), to_repeat, while, opts \ []) ``` ### Callback Signature ```elixir def my_while(rest, context, {line, column}, byte_offset, ...extra_args) do {:cont, context} | {:halt, context} end ``` ``` -------------------------------- ### ignore/1, ignore/2 Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/06-module-reference.md Matches the input but discards the results. This is useful when you need to consume input but don't care about its value. ```APIDOC ## ignore/1, ignore/2 ### Description Matches but discards results. ### Method N/A (Elixir function signature) ### Endpoint N/A (Elixir function signature) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example N/A ### Response #### Success Response (200) N/A #### Response Example N/A ``` -------------------------------- ### Binary Matching Pattern for Datetime Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/04-compiler-internals.md Demonstrates how NimbleParsec generates Erlang binary pattern matching for parsing datetime strings. It includes guards for validating byte values and a fallback clause for errors. ```elixir def datetime__0( <>, acc, stack, context, line, offset) when x0 >= 48 and x0 <= 57 and (x1 >= 48 and x1 <= 57) and ... do datetime__1(rest, [...acc list...], stack, context, line, offset) end def datetime__0(rest, _acc, _stack, context, line, offset) do {:error, "...", rest, context, line, offset} end ``` -------------------------------- ### Labeled Error for Configuration Key Source: https://github.com/dashbitco/nimble_parsec/blob/master/_autodocs/00-index.md Parses a configuration key consisting of lowercase letters and underscores, and labels potential errors with 'configuration key'. ```elixir defparsec :config_key, ascii_string([?a..?z, ?_], min: 1) |> label("configuration key") ```