### Run LALRPOP Executable Manually

Source: https://lalrpop.github.io/lalrpop/index.html/quick_start_guide

This command shows how to execute the `lalrpop` binary directly after installation via `cargo install lalrpop`. Running `lalrpop file.lalrpop` will generate `file.rs`. The command only executes if the `.lalrpop` file is newer than the generated `.rs` file, but this can be overridden with the `-f` flag.

```Shell
lalrpop file.lalrpop
```

--------------------------------

### Add LALRPOP Dependencies to Cargo.toml

Source: https://lalrpop.github.io/lalrpop/index.html/quick_start_guide

This snippet shows how to add the necessary `lalrpop-util` and `lalrpop` dependencies to your `Cargo.toml` file. `lalrpop-util` is a runtime dependency for the generated code, while `lalrpop` is a build-time dependency. An option to disable default features for `lalrpop` is also provided for custom lexer scenarios.

```TOML
# The generated code depends on lalrpop-util.
[dependencies]
lalrpop-util = "0.22.2"

# Add a build-time dependency on the lalrpop library:
[build-dependencies]
lalrpop = "0.22.2"
# If you are supplying your own external lexer you can disable default features so that the
# built-in lexer feature is not included
# lalrpop = { version = "0.22.2", default-features = false }
```

--------------------------------

### Create build.rs for LALRPOP Source Processing

Source: https://lalrpop.github.io/lalrpop/index.html/quick_start_guide

This Rust code snippet demonstrates the minimal `build.rs` file content required to process LALRPOP grammar files. The `lalrpop::process_src()` function automatically finds `.lalrpop` files in `src/` and generates corresponding Rust source files in `OUT_DIR`. For multi-crate workspaces, `process_root()` can be used.

```Rust
fn main() {
    lalrpop::process_src().unwrap();
}
```

--------------------------------

### Example: Binding Variables with String Literals

Source: https://lalrpop.github.io/lalrpop/index.html/print

Illustrates a simple language where single-character variables are bound to string literals, demonstrating the input format for a lexer.

```Example Language Input
x = "a"
y = "bc"

```

--------------------------------

### Use lalrpop_mod! Macro for Parser Module Generation

Source: https://lalrpop.github.io/lalrpop/index.html/quick_start_guide

This Rust snippet illustrates how to use the `lalrpop_mod!` macro to generate a wrapper module for your LALRPOP parser. By adding `lalrpop_mod!(grammar);` to `lib.rs` (or another module), a `grammar` submodule is created, allowing you to use the generated parser from your Rust code.

```Rust
#![allow(unused)]
fn main() {
lalrpop_mod!(grammar);
}
```

--------------------------------

### Default LALRPOP Build Configuration

Source: https://lalrpop.github.io/lalrpop/index.html/advanced_setup

Illustrates the basic `build.rs` setup for LALRPOP, which processes `.lalrpop` files in the `src` directory, updating them in-place only when changes are detected.

```Rust
fn main() {
    lalrpop::process_src().unwrap();
}
```

--------------------------------

### LALRPOP Shorthand Expansion Examples

Source: https://lalrpop.github.io/lalrpop/index.html/print

Table illustrating how various LALRPOP shorthand notations for action code and value selection are expanded into their equivalent explicit forms, including cases with struct constructors.

```LALRPOP
| Alternative | Equivalent to |
| --- | --- |
| `A => bar(<>)` | `<a:A> => bar(a)` |
| `A B => bar(<>)` | `<a:A> <b:B> => bar(a, b)` |
| `A B => (<>)` | `<a:A> <b:B> => (a, b)` |
| `<A> B => bar(<>)` | `<a:A> B => bar(a)` |
| `<p:A> B => bar(<>)` | `<p:A> B => bar(p)` |
| `<A> <B> => bar(<>)` | `<a:A> <b:B> => bar(a, b)` |
| `<p:A> <q:B> => bar(<>)` | `<p:A> <q:B> => bar(p, q)` |
| `<p:A> B => Foo {<>}` | `<p:A> B => Foo {p:p}` |
| `<p:A> <q:B> => Foo {<>}` | `<p:A> <q:B> => Foo {p:p, q:q}` |
```

--------------------------------

### Example of Simple Variable Binding Language

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/002_raw_delimited_content

Demonstrates the syntax of a simple custom language designed for variable binding, where variables are single characters and values are string literals. This serves as the target language for the LALRPOP grammar definition.

```Custom Language Example
x = "a"
y = "bc"

```

--------------------------------

### LALRPOP: Examples of '<>' shorthand expansion in action code

Source: https://lalrpop.github.io/lalrpop/index.html/tutorial/003_type_inference

A reference table demonstrating how LALRPOP's '<>' shorthand expressions within grammar alternatives are expanded into their equivalent explicit forms, showing how values are implicitly selected and named.

```LALRPOP
Alternative | Equivalent to
A => bar(<>) | <a:A> => bar(a)
A B => bar(<>) | <a:A> <b:B> => bar(a, b)
A B => (<>) | <a:A> <b:B> => (a, b)
<A> B => bar(<>) | <a:A> B => bar(a)
<p:A> B => bar(<>) | <p:A> B => bar(p)
<A> <B> => bar(<>) | <a:A> <b:B> => bar(a, b)
<p:A> <q:B> => bar(<>) | <p:A> <q:B> => bar(p, q)
<p:A> B => Foo {<>} | <p:A> B => Foo {p:p}
<p:A> <q:B> => Foo {<>} | <p:A> <q:B> => Foo {p:p, q:q}
```

--------------------------------

### Example of Calling the Calculator Parser in Rust

Source: https://lalrpop.github.io/lalrpop/index.html/print

This Rust snippet demonstrates how to instantiate and use the generated calculator parser. It initializes the lexer with an input string, parses the expression, and then asserts the correctness of the parsed Abstract Syntax Tree (AST) representation.

```Rust
let input = "22 * pi + 66";
let lexer = Lexer::new(input);
let expr = calculator9::ExprParser::new()
    .parse(input,lexer)
    .unwrap();
assert_eq!(&format!("{:?}", expr), "((\"22\" * \"pi\") + \"66\")");
```

--------------------------------

### Example Program to Parse with Logos

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/005_external_lib

This code snippet shows a simple program written in the toy language that the lexer, built using Logos, will be able to parse. It includes variable declarations, comments, and an arithmetic expression.

```Text
var a = 42;
var b = 23;

# a comment
print (a - b);

```

--------------------------------

### LALRPOP: Example grammar rules using custom tokens

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/003_writing_custom_lexer

Provides an example of LALRPOP grammar rules for `FlowCtrl` that utilize the custom terminal tokens defined in the `extern` block. This demonstrates how the parser automatically substitutes string literals in rules with their corresponding `Tok` enum variants, allowing existing rules to work without modification.

```LALRPOP
FlowCtrl: ast::Stmt = {
    " " " " <Label> => ast::Stmt::Mark(<>),
    " " "\t" <Label> => ast::Stmt::Call(<>),
    " " "\n" <Label> => ast::Stmt::Jump(<>),
    "\t" " " <Label> => ast::Stmt::Jz(<>),
    "\t" "\t" <Label> => ast::Stmt::Js(<>),
    "\t" "\n" => ast::Stmt::Return,
    "\n" "\n" => ast::Stmt::Exit,
};
```

--------------------------------

### Define Basic LALRPOP Grammar for Terms

Source: https://lalrpop.github.io/lalrpop/index.html/print

This snippet illustrates a simple context-free grammar in LALRPOP notation. It defines a `Term` as either a `Num` or another `Term` enclosed in parentheses, serving as a foundational example for parsing expressions.

```LALRPOP
Term = Num | "(" Term ")"
```

--------------------------------

### Lalrpop Cheatsheet: Grammar Features and Syntax

Source: https://lalrpop.github.io/lalrpop/index.html/print

A quick reference guide to various Lalrpop features, including syntax for position tracking, error recovery, grammar parameters, custom errors, macros, and external definitions.

```APIDOC
position: <left: @L> T <right: @R>
  Description: Captures the offset of the first byte and the offset of the last byte plus one (as `left` and `right` respectively).
error_recovery: ! => { ... }
  Description: Recovers from parser errors.
grammar_parameter: grammar(scale: isize);
  Description: Input parameters usable in the generated parser.
custom_error: "e" =>? Err(ParseError::User { error: "an error" })
  Description: Makes an action fallible.
custom_macros: Comma<T> = { ... }
  Description: Makes a non-terminal generic in other non-terminals.
quantifier_macros: <Num?> <Num*> <Num+>
  Description: A non-terminal which can appear 0..1, 0+, 1+ times.
tuple_macro: <a:(<Num> ",")*>
  Description: Applies a quantifier to a group of matches.
extern: extern { }
  Description: Allows to override some parts of the generated parser.
extern_error: type Error = MyError;
  Description: Sets the error to use in the `ParseError::User` variant.
extern_location: type Location = MyLoc;
  Description: Sets the type to for locations instead of `usize`.
extern_tok: enum MyToken { }
  Description: Declares the type of lexer tokens to be consumed by the generated parser.
auto_parameters: <>
  Description: Refers to all the parameters of the non-terminal as a tuple.
conditional actions: Expr<I> = { ... , <T> if I == "a" => (), ...}
  Description: Conditional definition of a macro's alternative.
precedence: #[precedence(level="0")]
  Description: Creates a hierarchy to parser actions for which ones should be applied first.
```

--------------------------------

### LALRPOP Tokenization Example with Mismatched Grammar

Source: https://lalrpop.github.io/lalrpop/index.html/print

An example of how the LALRPOP lexer tokenizes a string, even if it doesn't conform to the grammar. It visually breaks down the string into individual terminals like '(', ')', and numeric regex matches, demonstrating the lexer's independence from the parser's grammar rules and how the parser would later report an error.

```Diagram
  (  22   44  )     )
  ^  ^^   ^^  ^     ^
  |  |    |   |     ")" terminal
  |  |    |   |
  |  |    |   ")" terminal
  |  +----+
  |  |
  |  2 r"[0-9]+" terminals
  |
  "(" terminal
```

--------------------------------

### Example Usage: Parsing an Expression with Custom Lexer

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/004_token_references

Demonstrates how to use the custom lexer and the generated LALRPOP parser to parse an input expression. It initializes the lexer with the input, creates a parser instance, and then calls `parse` with both the input and the lexer, asserting the correctness of the parsed AST.

```Rust
let input = "22 * pi + 66";
let lexer = Lexer::new(input);
let expr = calculator9::ExprParser::new()
    .parse(input,lexer)
    .unwrap();
assert_eq!(&format!("{:?}", expr), "((\"22\" * \"pi\") + \"66\")");
```

--------------------------------

### LALRPOP Nonterminal Definition Example

Source: https://lalrpop.github.io/lalrpop/index.html/print

Illustrates the structure of a nonterminal definition in LALRPOP grammar, showing the nonterminal name, return type, matching symbols, and associated action code. This specific example defines a 'Num' nonterminal that matches a regular expression for digits.

```LALRPOP Grammar
   Num: i32 = r"[0-9]+" => i32::from_str(<>).unwrap();
// ~~~  ~~~   ~~~~~~~~~    ~~~~~~~~~~~~~~~~~~~~~~~~~~
// |    |     |                Action code
// |    |     Symbol(s) that should match
// |    Return type
// Name of nonterminal
```

--------------------------------

### LALRPOP Lexer Tokenization Example

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/001_lexer_gen

This example demonstrates how LALRPOP's lexer tokenizes an input string, identifying individual terminals like `"("`, `")"`, and `r"[0-9]+"`. It highlights that the lexer operates independently of the grammar, producing tokens even if the sequence does not form a valid grammatical structure, which the parser would then detect as an error.

```APIDOC
  (  22   44  )     )
  ^  ^^   ^^  ^     ^
  |  |    |   |     ")" terminal
  |  |    |   |
  |  |    |   ")" terminal
  |  +----
  |  |
  |  2 r"[0-9]+" terminals
  |
  "(" terminal
```

--------------------------------

### LALRPOP Grammar Example with Terminals and Nonterminals

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/001_lexer_gen

This LALRPOP grammar example defines two nonterminals, `Term` and `Num`. It demonstrates how string literals (`"("`, `")"`) and regular expressions (`r"[0-9]+"`) are used as terminals within the grammar rules, which are then processed by the lexer.

```Rust
pub Term: i32 = {
    <n:Num> => n,
    "(" <t:Term> ")" => t,
};

Num: i32 = <s:r"[0-9]+"> => i32::from_str(s).unwrap();
```

--------------------------------

### Track Token Locations in LALRPOP Grammar Rules

Source: https://lalrpop.github.io/lalrpop/index.html/print

Explains the use of `@L` and `@R` macros within LALRPOP grammar rules to capture the byte-level start and end positions of tokens. `@L` binds the start position of the token to its right, while `@R` binds the position one byte past the end of the token to its left, both as `usize`.

```LALRPOP
Symbol = {
    <start: @L> <s: r"a-z"> <end: @R> => {
        // `start` is the byte location of the start of our string
        // `s` is the string itself
        // `end` is the byte location of the end
    }
}

```

--------------------------------

### Add Lalrpop Dependency to Cargo.toml

Source: https://lalrpop.github.io/lalrpop/index.html/print

Specifies the Lalrpop dependency in a Cargo.toml file, disabling default features for a minimal setup.

```TOML
lalrpop = { version = "0.22.2", default-features = false }
```

--------------------------------

### LALRPOP Rule for Basic Location Tracking

Source: https://lalrpop.github.io/lalrpop/index.html/location_tracking

This LALRPOP grammar rule demonstrates the use of `@L` and `@R` macros. `@L` binds the start byte position of the token immediately to its right, and `@R` binds the byte position one past the end of the token immediately to its left. Both bind as `usize`.

```LALRPOP
Symbol = {
    <start: @L> <s: r"a-z"> <end: @R> => {
        // `start` is the byte location of the start of our string
        // `s` is the string itself
        // `end` is the byte location of the end
    }
}
```

--------------------------------

### Generate LALRPOP Parser Module with lalrpop_mod!

Source: https://lalrpop.github.io/lalrpop/index.html/print

The `lalrpop_mod!` macro, typically placed in `lib.rs` or another module, generates a Rust module corresponding to a `.lalrpop` grammar file. For example, `lalrpop_mod!(grammar)` creates a `grammar` submodule, allowing the generated parser to be used within your Rust code.

```Rust
#![allow(unused)]
fn main() {
lalrpop_mod!(grammar);
}
```

--------------------------------

### LALRPOP: Customizing Token Skipping with Ignore Patterns

Source: https://lalrpop.github.io/lalrpop/index.html/print

Explains how to define `ignore patterns` within a `match` declaration to skip non-token characters. This example shows how to disable default whitespace skipping and explicitly define patterns for single-line (`//`) and multi-line (`/* */`) comments.

```LALRPOP
match {
    r"\s*" => { }, // The default whitespace skipping is disabled if an `ignore pattern` is specified
    r"//[^\n\r]*[\n\r]*" => { }, // Skip `// comments`
    r"/\*[^*]*\*+(?:[^/*][^*]*\*+)*/" => { },  // Skip `/* comments */`
}
```

--------------------------------

### LALRPOP: Return Result from Num rule

Source: https://lalrpop.github.io/lalrpop/index.html/tutorial/007_fallible_actions

Demonstrates how to modify a LALRPOP grammar rule to return a `Result<i32, ParseError>` instead of a plain `i32` by using `=>?`. This example shows mapping a `from_str` error to a `ParseError::User` for numbers that are too large for `i32`.

```LALRPOP Grammar
Num: i32 = {
    r"[0-9]+" =>? i32::from_str(<>
        .map_err(|_| ParseError::User {
            error: "number is too big"
        })
};
```

--------------------------------

### Demonstrate Unrecoverable Lexer Error Example

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/006_error_recovery_custom_lexer

This example shows an input that causes a lexer error, specifically an `InvalidToken` error, which the LALRPOP parser cannot recover from directly because it's a lexer-level issue, not a parser-level one.

```LALRPOP Input
print (a - $);

// Err(User { error: InvalidToken })
```

--------------------------------

### Create New Rust Project with Cargo

Source: https://lalrpop.github.io/lalrpop/index.html/print

This command initializes a new binary Rust project named 'calculator' using Cargo. It sets up the basic project structure, including `Cargo.toml` and `src` directory, ready for development.

```Shell
cargo new --bin calculator
```

--------------------------------

### Run LALRPOP Parser with Lexer in Rust

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/005_external_lib

This final snippet demonstrates how to use the implemented lexer and the generated LALRPOP parser. It reads source code, creates a lexer instance, initializes the parser, and then parses the input to produce an Abstract Syntax Tree (AST).

```Rust
let source_code = std::fs::read_to_string("myscript.toy")?;
let lexer = Lexer::new(&source_code);
let parser = ScriptParser::new();
let ast = parser.parse(lexer)?;

println!("{:?}", ast);
```

--------------------------------

### Default LALRPOP Build Script Configuration

Source: https://lalrpop.github.io/lalrpop/index.html/print

Shows the standard `build.rs` file for LALRPOP projects, which uses `lalrpop::process_src()` to compile `.lalrpop` files into `.rs` files with default settings.

```Rust
fn main() {
    lalrpop::process_src().unwrap();
}
```

--------------------------------

### Create new Rust project with Cargo

Source: https://lalrpop.github.io/lalrpop/index.html/tutorial/001_adding_lalrpop

Initializes a new binary Rust project named 'calculator' using the Cargo build system.

```Shell
cargo new --bin calculator
```

--------------------------------

### Run LALRPOP Parser in Rust

Source: https://lalrpop.github.io/lalrpop/index.html/print

This Rust code demonstrates how to use the generated LALRPOP parser. It reads source code from a file, initializes a lexer, creates a parser instance, and then parses the input, printing the resulting Abstract Syntax Tree (AST).

```Rust
let source_code = std::fs::read_to_string("myscript.toy")?;
let lexer = Lexer::new(&source_code);
let parser = ScriptParser::new();
let ast = parser.parse(lexer)?;

println!("{:?}", ast);
```

--------------------------------

### LALRPOP Num Rule with Explicit Regex Capture (Calculator1)

Source: https://lalrpop.github.io/lalrpop/index.html/print

Defines the 'Num' nonterminal, capturing a regular expression match into a named variable 's' and then parsing it into an i32. This is from the calculator1 example.

```LALRPOP
Num: i32 = <s:r"[0-9]+"> => i32::from_str(s).unwrap();
```

--------------------------------

### Configure Cargo.toml for LALRPOP Project

Source: https://lalrpop.github.io/lalrpop/index.html/print

This `Cargo.toml` configuration snippet adds the necessary dependencies for a LALRPOP project. It includes `lalrpop-util` for runtime utilities and `lalrpop` as a build-time dependency, enabling the generation of parsers.

```TOML
# The generated code depends on lalrpop-util.
[dependencies]
lalrpop-util = "0.22.2"

# Add a build-time dependency on the lalrpop library:
[build-dependencies]
lalrpop = "0.22.2"
# If you are supplying your own external lexer you can disable default features so that the
# built-in lexer feature is not included
```

--------------------------------

### LALRPOP Term Rule with Explicit Action Code (Calculator1)

Source: https://lalrpop.github.io/lalrpop/index.html/print

Defines the 'Term' nonterminal in LALRPOP, explicitly showing how to match numbers or parenthesized terms and select their values using action code. This is from the calculator1 example.

```LALRPOP
pub Term: i32 = {
    <n:Num> => n,
    "(" <t:Term> ")" => t,
};
```

--------------------------------

### Create build.rs for LALRPOP preprocessing

Source: https://lalrpop.github.io/lalrpop/index.html/tutorial/001_adding_lalrpop

Defines the `build.rs` script, placed alongside `Cargo.toml`, which invokes `lalrpop::process_root()` to convert `.lalrpop` grammar files into Rust source files during the build process.

```Rust
fn main() {
    lalrpop::process_root().unwrap();
}
```

--------------------------------

### Configure Cargo.toml for LALRPOP Integration

Source: https://lalrpop.github.io/lalrpop/index.html/print

This `Cargo.toml` configuration adds `lalrpop` as a build dependency and `lalrpop-util` as a runtime dependency. The `build-dependencies` section is crucial for LALRPOP's preprocessor, while `lalrpop-util` provides necessary utilities for the generated parser.

```TOML
[package]
name = "calculator"
version = "0.1.0"
authors = ["Niko Matsakis <niko@alum.mit.edu>"]
edition = "2021"

[build-dependencies] # <-- We added this and everything after!
lalrpop = "0.22.2"

[dependencies]
lalrpop-util = { version = "0.21.0", features = ["lexer", "unicode"] }
```

--------------------------------

### Lalrpop Build Script (build.rs)

Source: https://lalrpop.github.io/lalrpop/index.html/print

Defines a `build.rs` file to automatically process Lalrpop grammar files using default settings, generating corresponding Rust source files in `OUT_DIR`.

```Rust
fn main() {
    lalrpop::process_src().unwrap();
}
```

--------------------------------

### LALRPOP Num Rule with Shorthand Regex Capture (Calculator2)

Source: https://lalrpop.github.io/lalrpop/index.html/print

Shows a shorthand for the 'Num' nonterminal using `<>` to automatically synthesize a variable for the matched regular expression. A type annotation is still required due to custom action code. This is from the calculator2 example.

```LALRPOP
Num: i32 = r"[0-9]+" => i32::from_str(<>)unwrap();
```

--------------------------------

### Implement build.rs for LALRPOP Processing

Source: https://lalrpop.github.io/lalrpop/index.html/print

This `build.rs` script is executed by Cargo before compilation. It calls `lalrpop::process_root()` to find and convert `.lalrpop` grammar files into Rust source files, which are then compiled with the rest of the project. The `unwrap()` handles potential file system errors.

```Rust
fn main() {
    lalrpop::process_root().unwrap();
}
```

--------------------------------

### Parse Tree Visualization for `((123))`

Source: https://lalrpop.github.io/lalrpop/index.html/crash_course

This ASCII art snippet visualizes how the input `((123))` would be parsed according to a simple grammar. It demonstrates the hierarchical structure of a parse tree, showing how numbers and parenthesized terms are recognized and nested.

```Textual Diagram
(  (  1  2  3  )  )
|  |  |     |  |  |
|  |  +-Num-+  |  |
|  |     |     |  |
|  |   Term    |  |
|  |     |     |  |
|  +---Term----+  |
|        |        |
+------Term-------+
```

--------------------------------

### LALRPOP Term Rule with Implicit Action Code (Calculator2 Shorthand)

Source: https://lalrpop.github.io/lalrpop/index.html/print

Demonstrates LALRPOP's shorthand for 'Term' where action code is omitted. LALRPOP automatically synthesizes action code to take the value of matched items, inferring the type. This is from the calculator2 example.

```LALRPOP
pub Term = { Num, "(" <Term> ")" };
```

--------------------------------

### LALRPOP: Case-Insensitive Keyword Matching with Match Declaration

Source: https://lalrpop.github.io/lalrpop/index.html/print

Demonstrates how to use a case-insensitive regular expression (`r"(?i)begin"`) within a `match` declaration and map it to a canonical string literal (`"BEGIN"`). This allows the grammar to reference the keyword consistently regardless of its casing in the input.

```LALRPOP
match {
    r"(?i)begin" => "BEGIN",
    ...
}
```

--------------------------------

### LALRPOP Conditional Non-Terminal Declarations

Source: https://lalrpop.github.io/lalrpop/index.html/conditional-compilation

Demonstrates how to apply `#[cfg]` attributes to LALRPOP non-terminal declarations (`pub MyRule: () = { ... };`) to conditionally include or exclude them based on Cargo features. It shows examples using `feature`, `any`, `all`, and `not` for complex conditional logic.

```Rust
#![allow(unused)]
fn main() {
#[cfg(feature = "FEATURE") ]
pub MyRule1 : () = {
    ...
};

#[cfg(any(feature = "FEATURE_A", all(not(feature = "FEATURE_B"), feature = "FEATURE_C")))]
pub MyRule2 : () = {
    ...
};

pub MyRule3: () = {
    #[cfg(feature = "FEATURE_A") ]
    "A" => (),
    #[cfg(feature = "FEATURE_B") ]
    "B" => (),
};
}
```

--------------------------------

### Define LALRPOP Grammar Rules for Script, Statement, and Expression

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/005_external_lib

This snippet defines the core grammar rules for a simple language using LALRPOP. It includes rules for `Script`, `Statement` (variable declaration, print), and `Expression` with operator precedence and associativity.

```LALRPOP
pub Script: Vec<ast::Statement> = {
  <stmts:Statement*> => stmts
}

pub Statement: ast::Statement = {
  "var" <name:"identifier"> "=" <value:Expression> ";" => {
    ast::Statement::Variable { name, value }
  },
  "print" <value:Expression> ";" => {
    ast::Statement::Print { value }
  },
}

pub Expression: Box<ast::Expression> = {
  #[precedence(level="1")]
  Term,

  #[precedence(level="2")] #[assoc(side="left")]
  <lhs:Expression> "*" <rhs:Expression> => {
    Box::new(ast::Expression::BinaryOperation {
      lhs,
      operator: ast::Operator::Mul,
      rhs
    })
  },
  <lhs:Expression> "/" <rhs:Expression> => {
    Box::new(ast::Expression::BinaryOperation {
      lhs,
      operator: ast::Operator::Div,
      rhs
    })
  },

  #[precedence(level="3")] #[assoc(side="left")]
  <lhs:Expression> "+" <rhs:Expression> => {
    Box::new(ast::Expression::BinaryOperation {
      lhs,
      operator: ast::Operator::Add,
      rhs
    })
  },
  <lhs:Expression> "-" <rhs:Expression> => {
    Box::new(ast::Expression::BinaryOperation {
      lhs,
      operator: ast::Operator::Sub,
      rhs
    })
  },
}

pub Term: Box<ast::Expression> = {
  <val:"int"> => {
    Box::new(ast::Expression::Integer(val))
  },
  <name:"identifier"> => {
    Box::new(ast::Expression::Variable(name))
  },
  "(" <Expression> ")",
}
```

--------------------------------

### Implement Lexer Constructor in Rust

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/005_external_lib

This code block provides the `new` constructor for the `Lexer` struct, which initializes the token stream using the `Logos` trait's `lexer()` method on the input string.

```Rust
impl<'input> Lexer<'input> {
  pub fn new(input: &'input str) -> Self {
    Self { token_stream: Token::lexer(input).spanned() }
  }
}
```

--------------------------------

### LALRPOP Grammar with Terminals and Nonterminals

Source: https://lalrpop.github.io/lalrpop/index.html/print

An example LALRPOP grammar demonstrating the definition of a 'Term' nonterminal that can be a 'Num' or a parenthesized 'Term', and a 'Num' nonterminal matching a regular expression. This snippet highlights the presence of both string literal terminals ('(' and ')') and a regex terminal ('r"[0-9]+"').

```LALRPOP Grammar
pub Term: i32 = {
    <n:Num> => n,
    "(" <t:Term> ")" => t,
};

Num: i32 = <s:r"[0-9]+"> => i32::from_str(s).unwrap();
```

--------------------------------

### Parsing multiple expressions with LALRPOP

Source: https://lalrpop.github.io/lalrpop/index.html/print

Demonstrates parsing multiple arithmetic expressions using a LALRPOP-generated parser (calculator5) and asserting the formatted output. This snippet shows basic parser usage before error handling is introduced.

```Rust
    let expr = calculator5::ExprsParser::new()
        .parse("22 * 44 + 66, 13*3,")
        .unwrap();
    assert_eq!(&format!("{:?}", expr), "[((22 * 44) + 66), (13 * 3)]");
}
}
```

--------------------------------

### Use lalrpop_mod! macro to import generated parser

Source: https://lalrpop.github.io/lalrpop/index.html/tutorial/001_adding_lalrpop

Demonstrates how to use the `lalrpop_mod!` macro in a Rust source file (e.g., `lib.rs`) to create a module for the parser generated from a `.lalrpop` grammar file, making it accessible within the crate.

```Rust
#![allow(unused)]
fn main() {
lalrpop_mod!(grammar);
}
```

--------------------------------

### LALRPOP: Concise Term definition with implicit action code and type inference

Source: https://lalrpop.github.io/lalrpop/index.html/tutorial/003_type_inference

Demonstrates LALRPOP's shorthand for 'Term' where action code is omitted. LALRPOP automatically synthesizes action code to take the value of matched items and infers the return type, making the definition more concise.

```LALRPOP
pub Term = { Num, "(" <Term> ")" };
```

--------------------------------

### Define Spanned Type for Lexer Stream Format in Rust

Source: https://lalrpop.github.io/lalrpop/index.html/print

This Rust type alias defines the `Spanned` type, which represents an item in the lexer's output stream. It encapsulates a token (`Tok`), its start and end locations (`Loc`), and a potential error (`Error`), following Rust's `Result` convention.

```Rust
pub type Spanned<Tok, Loc, Error> = Result<(Loc, Tok, Loc), Error>;
```

--------------------------------

### Configure Cargo.toml for LALRPOP build and runtime dependencies

Source: https://lalrpop.github.io/lalrpop/index.html/tutorial/001_adding_lalrpop

Modifies the `Cargo.toml` file to include `lalrpop` as a build dependency and `lalrpop-util` as a runtime dependency, enabling LALRPOP preprocessing and utility functions.

```TOML
[package]
name = "calculator"
version = "0.1.0"
authors = ["Niko Matsakis <niko@alum.mit.edu>"]
edition = "2021"

[build-dependencies] # <-- We added this and everything after!
lalrpop = "0.22.2"

[dependencies]
lalrpop-util = { version = "0.21.0", features = ["lexer", "unicode"] }
```

--------------------------------

### Rust Type Definition for Spanned Token Stream

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/003_writing_custom_lexer

Defines the `Spanned` type, which represents an item in the token stream accepted by the parser. It's a `Result` containing a tuple of start location, token, end location, or an error. `Loc` typically represents a byte offset, and `Tok` defines the token values.

```Rust
pub type Spanned<Tok, Loc, Error> = Result<(Loc, Tok, Loc), Error>;
```

--------------------------------

### Rust Test for LALRPOP Grammar - Successful Parse

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/002_raw_delimited_content

A Rust test case demonstrating a successful parse of the input `z = "xyz"` using the `EqlParser` generated by LALRPOP. This test passes with the initial grammar, confirming expected behavior for certain well-formed inputs.

```Rust
#![allow(unused)]
fn main() {
#[test]
fn fair_ball() {
    assert_eq!(nobol2::EqlParser::new().parse(r#"z = "xyz""#), Ok(('z', "xyz").into()));
}
}
```

--------------------------------

### Implement Iterator Trait for Rust Lexer

Source: https://lalrpop.github.io/lalrpop/index.html/print

This Rust code implements the `Iterator` trait for the `Lexer` struct, making it compatible with LALRPOP's lexer requirements. The `next` method retrieves the next token and its span from the internal `token_stream`, then maps it into a `Result` tuple containing the start location, the token itself, and the end location, handling potential token errors.

```Rust
impl<'input> Iterator for Lexer<'input> {
  type Item = Spanned<Token, usize, LexicalError>;

  fn next(&mut self) -> Option<Self::Item> {
    self.token_stream
      .next()
      .map(|(token, span)| Ok((span.start, token?, span.end)))
  }
}
```

--------------------------------

### LALRPOP Grammar for Expr and ExprOp Nonterminals

Source: https://lalrpop.github.io/lalrpop/index.html/print

LALRPOP grammar rules for parsing expressions (`Expr`) and their operators (`ExprOp`). It demonstrates importing Rust types, defining the return type for nonterminals (`Box<Expr>`), and using the `<>` expansion for constructor arguments.

```LALRPOP
use std::str::FromStr;
use ast::{Expr, Opcode}; // (0)

grammar;

pub Expr: Box<Expr> = { // (1)
    Expr ExprOp Factor => Box::new(Expr::Op(<>)), // (2)
    Factor,
};

ExprOp: Opcode = { // (3)
    "+" => Opcode::Add,
    "-" => Opcode::Sub,
};

```

--------------------------------

### Define Lexer Structure and Spanned Type for Rust Parser

Source: https://lalrpop.github.io/lalrpop/index.html/print

This Rust code defines the `Lexer` struct and a `Spanned` type alias, essential for integrating with LALRPOP. The `Lexer` holds a `SpannedIter` from the `logos` crate, providing an iterator over tokens with their start and end spans. The `Spanned` type simplifies error handling by encapsulating the token, its location, and potential errors.

```Rust
use logos::{Logos, SpannedIter};

use crate::tokens::{Token, LexicalError}; // your Token enum, as above

pub type Spanned<Tok, Loc, Error> = Result<(Loc, Tok, Loc), Error>;

pub struct Lexer<'input> {
  // instead of an iterator over characters, we have a token iterator
  token_stream: SpannedIter<'input, Token>,
}
```

--------------------------------

### LALRPOP Grammar with Match Declaration for Token Precedence

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/002_raw_delimited_content

Defines a LALRPOP grammar for variables, literals, and equality expressions. It includes a `match` declaration to establish precedence for `r"[x-z]"` over `r"[a-z]*"`, aiming to resolve potential ambiguities in token recognition.

```LALRPOP
use super::{Var, Lit, Eql};

grammar;

match {
   r"[x-z]"
} else {
   r"[a-z]*",
   _
}

pub Var: Var = <r"[x-z]"> => <>.chars().next().unwrap().into();

pub Lit: Lit = "\"" <r"[a-z]*"> "\"" => <>.into();

pub Eql: Eql = <Var> "=" <Lit> => (<>).into();
```

--------------------------------

### Rust: Parsing input with a custom lexer

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/003_writing_custom_lexer

Demonstrates how to use the generated LALRPOP parser with a custom `Lexer` instance in Rust. It shows the typical invocation of a parser function (e.g., `parser::parse_Program`) by passing the initialized lexer as input instead of a raw string slice.

```Rust
#![allow(unused)]
fn main() {
    let lexer = lexer::Lexer::new("\n\n\n");
    match parser::parse_Program(lexer) {
        ...
    }
}
```

--------------------------------

### LALRPOP: Initial String Literal Rule

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/002_raw_delimited_content

Shows the initial LALRPOP grammar rule for string literals, highlighting the implicit distinct tokens for delimiter and content, which leads to incorrect lexing.

```LALRPOP
pub Lit: Lit = "\"" <r"[a-z]*"> "\"" => <>.into();
```

--------------------------------

### LALRPOP: Mapping Case-Insensitive Regex to Fixed Literals

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/001_lexer_gen

This `match` declaration illustrates how to map a case-insensitive regular expression (`r"(?i)begin"`) to a fixed string literal (`"BEGIN"`). This allows the grammar to refer to the keyword consistently while the lexer handles various capitalizations.

```LALRPOP
match {
    r"(?i)begin" => "BEGIN",
    ...
}
```

--------------------------------

### Basic Context-Free Grammar for Parenthesized Numbers

Source: https://lalrpop.github.io/lalrpop/index.html/crash_course

This snippet shows a simple context-free grammar defining a 'Term' as either a number ('Num') or a parenthesized term. It illustrates the basic structure for parsing expressions like '123' or '(123)'.

```LALRPOP
Term = Num | "(" Term ")"
```

--------------------------------

### Rust: Using Custom Lexer with LALRPOP Parser

Source: https://lalrpop.github.io/lalrpop/index.html/print

Demonstrates how to instantiate a custom `lexer::Lexer` and pass it as input to the LALRPOP-generated parser function, `parser::parse_Program`, instead of a raw string slice.

```Rust
#![allow(unused)]
fn main() {
    let lexer = lexer::Lexer::new("\n\n\n");
    match parser::parse_Program(lexer) {
        ...
    }
}
```

--------------------------------

### Run Lalrpop Executable Manually

Source: https://lalrpop.github.io/lalrpop/index.html/print

Executes the `lalrpop` binary from the command line to generate a Rust file from a `.lalrpop` grammar file. It only executes if the grammar file is newer than the generated Rust file, unless the `-f` flag is used.

```Shell
lalrpop file.lalrpop
```

--------------------------------

### Rust: Tests for LALRPOP Improved Grammar Rules

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/002_raw_delimited_content

Rust unit tests validating the parsing behavior of the improved LALRPOP grammar rules for variables, string literals, and equality expressions, ensuring correct tokenization and parsing.

```Rust
#![allow(unused)]
fn main() {
#[test]
fn homerun() {
    assert_eq!(nobol5::VarParser::new().parse("x"), Ok('x'.into()));
    assert_eq!(nobol5::LitParser::new().parse(r#""abc""#), Ok("abc".into()));
    assert_eq!(nobol5::EqlParser::new().parse(r#"x = "a""#), Ok(('x', "a").into()));
    assert_eq!(nobol5::EqlParser::new().parse(r#"y = "bc""#), Ok(('y', "bc").into()));
    assert_eq!(nobol5::EqlParser::new().parse(r#"z = "xyz""#), Ok(('z', "xyz").into()));
    assert_eq!(nobol5::EqlParser::new().parse(r#"z = "x""#), Ok(('z', "x").into()));
    assert_eq!(nobol5::EqlParser::new().parse(r#"z = "x y z""#), Ok(('z', "x y z").into()));
}
}
```

--------------------------------

### LALRPOP: Improved String Literal and Assignment Rules

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/002_raw_delimited_content

Demonstrates a revised LALRPOP grammar where string delimiters and content are treated as a single token, resolving lexer ambiguity. Includes rules for variables, literals, and equality expressions.

```LALRPOP
pub Var: Var = <r"[a-z]"> => <>.chars().next().unwrap().into();

pub Lit: Lit = <l:r#""[a-z ]*""#> => l[1..l.len()-1].into();

pub Eql: Eql = <Var> "=" <Lit> => (<>).into();
```

--------------------------------

### Add Logos Dependency to Cargo.toml

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/005_external_lib

To use the Logos library, add the specified dependency to your project's `Cargo.toml` file. This provides access to the `logos` crate and the `Logos` trait for lexer generation.

```TOML
logos = "0.14"

```

--------------------------------

### LALRPOP Lexer and Parser Workflow

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/001_lexer_gen

This diagram visualizes the two-phase process of LALRPOP. The Lexer applies regular expressions to raw text to produce a stream of terminals, which are then consumed by the Parser to recognize nonterminals and execute user-defined action code.

```APIDOC
        +-------------------+    +---------------------+
Text -> | Lexer             | -> | Parser              |
|                   |    |                     |
| Applies regex to  |    | Consumes terminals, |
| produce terminals |    | executes your code  |
+-------------------+    | as it recognizes    |
                         | nonterminals        |
                         +---------------------+
```

--------------------------------

### Implement Constructor for Rust Lexer

Source: https://lalrpop.github.io/lalrpop/index.html/print

This Rust code provides the `new` constructor for the `Lexer` struct. It takes an input string slice and initializes the `token_stream` by calling `Token::lexer(input).spanned()`, leveraging the `Logos` trait to create a spanned iterator over the tokens derived from the input.

```Rust
impl<'input> Lexer<'input> {
  pub fn new(input: &'input str) -> Self {
    // the Token::lexer() method is provided by the Logos trait
    Self { token_stream: Token::lexer(input).spanned() }
  }
}
```

--------------------------------

### LALRPOP Grammar with Generalized Lit and Match for Whitespace

Source: https://lalrpop.github.io/lalrpop/index.html/print

An attempt to generalize the `Lit` production to include whitespace (`[a-z ]*`) while maintaining the `match` declaration. This variant aims to allow more flexible string literals but introduces new tokenization problems for other parts of the input, demonstrating the complexity of lexical analysis.

```LALRPOP
match {
   r"[x-z]"
} else {
   r"[a-z ]*",
   _
}

pub Var: Var = <r"[x-z]"> => <>.chars().next().unwrap().into();

pub Lit: Lit = {
    "\"" <r"[x-z]"> "\"" => <>.into(),
    "\"" <r"[a-z ]*"> "\"" => <>.into(),
};

pub Eql: Eql = <Var> "=" <Lit> => (<>).into();
```

--------------------------------

### Define Abstract Syntax Tree (AST) for Toy Language

Source: https://lalrpop.github.io/lalrpop/index.html/lexer_tutorial/005_external_lib

This Rust code defines the Abstract Syntax Tree (AST) for the toy programming language. It uses enums `Statement`, `Expression`, and `Operator` to represent the structure of parsed code, including variable declarations, print statements, integers, variables, and binary operations.

```Rust
#![allow(unused)]
fn main() {
#[derive(Clone, Debug, PartialEq)]
pub enum Statement {
  Variable { name: String, value: Box<Expression> },
  Print { value: Box<Expression> },
}

#[derive(Clone, Debug, PartialEq)]
pub enum Expression {
  Integer(i64),
  Variable(String),
  BinaryOperation {
    lhs: Box<Expression>,
    operator: Operator,
    rhs: Box<Expression>,
  },
}

#[derive(Clone, Debug, PartialEq)]
pub enum Operator {
  Add,
  Sub,
  Mul,
  Div,
}
}

```

--------------------------------

### Implement Lexer Struct and Constructor in Rust

Source: https://lalrpop.github.io/lalrpop/index.html/print

This Rust struct defines the lexer, holding an iterator over character indices and a reference to the input string. The `new` method initializes the lexer, preparing it to process the input and generate tokens.

```Rust
use std::str::CharIndices;

pub struct Lexer<'input> {
    chars: std::iter::Peekable<CharIndices<'input>>,
    input: &'input str,
}

impl<'input> Lexer<'input> {
    pub fn new(input: &'input str) -> Self {
        Lexer {
            chars: input.char_indices().peekable(),
            input,
        }
    }
}
```

--------------------------------

### Visualize Parse Tree for Parenthesized Numbers

Source: https://lalrpop.github.io/lalrpop/index.html/print

This text diagram visualizes the parse tree for the input `((123))`. It demonstrates how the grammar rules are applied hierarchically to break down the input into `Num` and `Term` components, showing the parsing process.

```Text Diagram
(  (  1  2  3  )  )
|  |  |     |  |  |
|  |  +-Num-+  |  |
|  |     |     |  |
|  |   Term    |  |
|  |     |     |  |
|  +---Term----+  |
|        |        |
+------Term-------+
```

--------------------------------

### Forcing Color Output in LALRPOP Build

Source: https://lalrpop.github.io/lalrpop/index.html/advanced_setup

Demonstrates how to configure LALRPOP to always output colors, overriding default TTY settings, by using the `Configuration` struct and `always_use_colors()` method.

```Rust
fn main() {
    lalrpop::Configuration::new()
        .always_use_colors()
        .process_current_dir();
}
```

--------------------------------

### Rust Test Case for Initial LALRPOP Grammar (Fair Ball)

Source: https://lalrpop.github.io/lalrpop/index.html/print

A Rust test case demonstrating successful parsing of a simple assignment `z = "xyz"` using the LALRPOP grammar defined previously. It asserts the correct parsing of a variable and a literal, confirming the grammar's intended behavior for this specific input.

```Rust
#![allow(unused)]
fn main() {
#[test]
fn fair_ball() {
    assert_eq!(nobol2::EqlParser::new().parse(r#"z = "xyz""#), Ok(('z', "xyz").into()));
}
}
```