### C/C++ Macro Example Source: https://lukaswirth.dev/tlborm/syntax-extensions/source-analysis Demonstrates C/C++ macro usage where macros are processed during the initial compilation stages. This example uses macros to define types and blocks, highlighting the preprocessor's role. ```c #define SUB int #define BEGIN { #define END } SUB main() BEGIN printf("Oh, the horror!\n"); END ``` -------------------------------- ### Rust Macro Expansion Output Example Source: https://lukaswirth.dev/tlborm/print Presents the expanded code after applying the `-Zunpretty=expanded` flag to a Rust macro definition, illustrating the generated code. ```rust #![feature(prelude_import)] #[prelude_import] use std::prelude::rust_2018::*; #[macro_use] extern crate std; // Shorthand for initializing a `String`. macro_rules! S { ($e : expr) => { String :: from($e) } ; } fn main() { let world = String::from("World"); { ::std::io::_print( ::core::fmt::Arguments::new_v1( &["Hello, ", "!\n"], &match (&world,) { (arg0,) => [ ::core::fmt::ArgumentV1::new(arg0, ::core::fmt::Display::fmt) ], } ) ); } } ``` -------------------------------- ### Rust Item Parsing Macro Example Source: https://lukaswirth.dev/tlborm/print This example demonstrates Rust macros designed to parse specific Rust items like structs and functions. The focus is on extracting useful parts of these items without attempting to parse the entire grammar, ignoring complex features like generics. ```Rust # // This section will show a few macros that can parse some of Rust's more complex items like structs and functions to a certain extent. # // The goal of these macros is not to be able to parse the entire grammar of the items but to parse parts that are in general quite useful without being too complex to parse. # // This means we ignore things like generics and such. # // The main points of interest of these macros are their `matchers`. The transcribers are only there for example purposes and are usually not that impressive. ``` -------------------------------- ### Rust Macro Invocation Examples Source: https://lukaswirth.dev/tlborm/syntax-extensions/ast Demonstrates various ways macros can be invoked in Rust code, including attributes, function-like macros, and macro_rules!. ```rust use Color::*; let colors = vec![RED, GREEN, BLUE]; println!("Hello, World!"); ``` ```rust #[derive(Clone)] #[no_mangle] #![allow(dead_code)] #![crate_name="blang"] println!("Hi!") ``` ```rust macro_rules! dummy { () => {}; } macro_rules! bitflags { struct Color: u8 { const RED = 0b0001, const GREEN = 0b0010, const BLUE = 0b0100, const BRIGHT = 0b1000, } } lazy_static! { static ref FIB_100: u32 = { fn fib(a: u32) -> u32 { match a { 0 => 0, 1 => 1, a => fib(a-1) + fib(a-2) } } fib(100) }; } ``` -------------------------------- ### Abstract Syntax Tree (AST) Visualization Source: https://lukaswirth.dev/tlborm/syntax-extensions/source-analysis Presents the Abstract Syntax Tree (AST) corresponding to the example expression. This visualization highlights the difference in structure and representation compared to the token tree. ```text ┌─────────┐ │ BinOp │ │ op: Add │ ┌╴│ lhs: ◌ │ ┌─────────┐ │ │ rhs: ◌ │╶┐ ┌─────────┐ │ BinOp │╶┘ └─────────┘ └╴│ Var │ │ op: Add │ │ name: e │ ┌╴│ lhs: ◌ │ └─────────┘ ┌─────────┐ ┌─────────┐ │ │ rhs: ◌ │╶┐ ┌─────────┐ │ Var │╶┐ │ BinOp │╶┘ └─────────┘ └╴│ BinOp │ │ name: a │ │ │ op: Add │ │ op: Add │ └─────────┘ └╴│ lhs: ◌ │ ┌╴│ lhs: ◌ │ ┌─────────┐ ┌╴│ rhs: ◌ │ ┌─────────┐ │ │ rhs: ◌ │╶┐ ┌─────────┐ │ Var │╶┘ └─────────┘ │ Var │╶┘ └─────────┘ └╴│ Index │ │ name: b │ │ name: c │ ┌╴│ arr: ◌ │ └─────────┘ └─────────┘ ┌─────────┐ │ │ ind: ◌ │╶┐ │ Var │╶┘ └─────────┘ │ │ name: d │ ┌─────────┐ │ └─────────┘ │ LitInt │╶┘ │ val: 0 │ └─────────┘ ``` -------------------------------- ### Token Tree Example Expression Source: https://lukaswirth.dev/tlborm/syntax-extensions/source-analysis Illustrates a simple arithmetic expression as a sequence of tokens that will be parsed into token trees. This serves as a visual aid to understand tokenization before structural parsing. ```text a + b + (c + d[0]) + e ``` -------------------------------- ### Rust: Example Expansion of Recurrence Macro Rule Source: https://lukaswirth.dev/tlborm/decl-macros/macros-practical Illustrates the step-by-step process of how the `recurrence!` macro expands a given input based on its defined rule. It shows how tokens are consumed and meta-variables are bound. ```rust a[n] = $($inits:expr),+ , ... , $recur:expr ``` -------------------------------- ### Rust macro_rules! example with empty matcher Source: https://lukaswirth.dev/tlborm/decl-macros/macros-methodical This is a simple example of a `macro_rules!` macro named `four`. It has an empty matcher `()` and expands to `1 + 3`. This macro can be invoked with empty parentheses, brackets, or braces, e.g., `four!()`, `four![]`, or `four!{}`. ```rust macro_rules! four { () => { 1 + 3 }; } ``` -------------------------------- ### Token Tree Structure Visualization Source: https://lukaswirth.dev/tlborm/syntax-extensions/source-analysis Visualizes the parsed structure of the example expression as token trees. It shows how grouping symbols like parentheses and brackets form interior nodes, while other tokens act as leaves. ```text «a» «+» «b» «+» «( )» «+» «e» ╭────────┴──────────╮ «c» «+» «d» «[ ]» ╭─┴─╮ «0» ``` -------------------------------- ### Rust macro_rules! meta fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Demonstrates the `meta` fragment specifier for matching attribute contents. This can capture simple paths or paths followed by delimited token trees or literals. ```rust macro_rules! metas { ($($meta:meta)*) => (); } metas! { ASimplePath super::man path = "home" foo(bar) } fn main() {} ``` -------------------------------- ### Rust macro_rules! pat_param fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Shows the `pat_param` fragment specifier, which allows `|` tokens to follow it, providing an alternative to the `pat` fragment when top-level or-patterns are disallowed. ```rust macro_rules! patterns { ($( $( $pat:pat_param )|+ )*) => (); } patterns! { "literal" _ 0..5 ref mut PatternsAreNice 0 | 1 | 2 | 3 } fn main() {} ``` -------------------------------- ### Rust Macro Hygiene Example: `use_local!` Source: https://lukaswirth.dev/tlborm/print Illustrates a non-hygienic macro `use_local!` that incorrectly modifies a pre-existing local identifier, showing it does not respect its surrounding context. ```rust __ let mut local = 0; use_local!(); ``` -------------------------------- ### Rust Macro Hygiene Example: `make_local!` Source: https://lukaswirth.dev/tlborm/print Demonstrates a non-hygienic macro `make_local!` that creates a local identifier which is then unexpectedly accessible outside its intended scope. ```rust __ make_local!(); assert_eq!(local, 0); ``` -------------------------------- ### Rust Macro `ty` Example: Matching Type Expressions Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Illustrates the `ty` fragment's ability to match various Rust type expressions. The example macro `types!` captures different forms of types, including paths, primitives, arrays, and generic trait implementations. ```Rust macro_rules! types { ($($type:ty)*) => (); } types! { foo::bar bool [u8] impl IntoIterator } fn main() {} ``` -------------------------------- ### Rust Macro for Recurrence Relations (Fibonacci Example) Source: https://lukaswirth.dev/tlborm/decl-macros/macros-practical This Rust macro `recurrence!` is designed to define sequences using initialization values and a recurrence expression. The example provided demonstrates its use for generating a Fibonacci sequence. It highlights a past compiler issue with the '...' token and the subsequent adjustment to use semicolons. ```rust macro_rules! recurrence { ( a[n]: $sty:ty = $($inits:expr),+ ; ... ; $recur:expr ) => { // ^~~~~~^ changed /* ... */ // Cheat :D (vec![0u64, 1, 2, 3, 5, 8, 13, 21, 34]).into_iter() }; } fn main() { let fib = recurrence![a[n]: u64 = 0, 1; ...; a[n-2] + a[n-1]]; // ^~~~~^ changed for e in fib.take(10) { println!("{}", e) } } ``` -------------------------------- ### Rust Main Function with Recurrence Macro Source: https://lukaswirth.dev/tlborm/print Example of using the `recurrence!` macro in a Rust `main` function to define and iterate over a Fibonacci sequence. It demonstrates how to initialize the sequence and consume it using `take` and `println`. ```rust fn main() { let fib = recurrence![a[n]: u64 = 1, 1; ...; a[n-2] + a[n-1]]; for e in fib.take(10) { println!("{}", e) } } ``` -------------------------------- ### Rust macro_rules! path fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Demonstrates the `path` fragment specifier for matching TypePath style paths, including function-style trait forms like `Fn() -> ()`. ```rust macro_rules! paths { ($($path:path)*) => (); } paths! { ASimplePath ::A::B::C::D G::::C FnMut(u32) -> () } fn main() {} ``` -------------------------------- ### Rust: Example Usage of a Recurrence Macro Source: https://lukaswirth.dev/tlborm/decl-macros/macros-practical Demonstrates how to use a hypothetical `recurrence!` macro to define a sequence and iterate over its elements. This macro is intended for defining mathematical recurrence relations. ```rust let fib = recurrence![a[n] = 0, 1, ..., a[n-2] + a[n-1]]; for e in fib.take(10) { println("{}", e) } ``` -------------------------------- ### Rust Macro Invocations and Token Trees Source: https://lukaswirth.dev/tlborm/print Demonstrates various Rust macro invocations, including attributes, function-like macros, and macro_rules! constructs. The subsequent example illustrates how the parser treats these as opaque token trees (represented by ⬚) without understanding their internal structure. ```rust bitflags! { struct Color: u8 { const RED = 0b0001, const GREEN = 0b0010, const BLUE = 0b0100, const BRIGHT = 0b1000, } } lazy_static! { static ref FIB_100: u32 = { fn fib(a: u32) -> u32 { match a { 0 => 0, 1 => 1, a => fib(a-1) + fib(a-2) } } fib(100) }; } fn main() { use Color::*; let colors = vec![RED, GREEN, BLUE]; println!("Hello, World!"); } ``` ```rust bitflags! ⬚ lazy_static! ⬚ fn main() { let colors = vec! ⬚; println! ⬚; } ``` -------------------------------- ### Recursive Macro Expansion Example Source: https://lukaswirth.dev/tlborm/print Demonstrates the recursive nature of syntax extension expansion. An initial expansion might contain another macro invocation, which is then expanded in a subsequent pass until all extensions are resolved or the recursion limit is hit. ```text let x = four!(); // Expands to: let x = 1 + three!(); // After second expansion pass: let x = 1 + 3; ``` -------------------------------- ### Rust macro_rules! expr fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Illustrates the `expr` fragment specifier for matching any Rust expression. This is useful for capturing and manipulating expressions within macros. ```rust macro_rules! expressions { ($($expr:expr)*) => (); } expressions! { "literal" funcall() future.await break 'foo bar } fn main() {} ``` -------------------------------- ### Rust Macro Hygiene Example: Basic Usage Source: https://lukaswirth.dev/tlborm/decl-macros/macros-practical Demonstrates a basic Rust macro `using_a!` that defines a local variable `a` and uses it within an expression. It highlights how hygienic macros prevent identifier collisions by creating distinct compilation contexts. ```rust macro_rules! using_a { ($e:expr) => { { let a = 42; $e } } } let four = using_a!(a / 10); fn main() {} ``` -------------------------------- ### Rust Crate-Wide Macro Accessibility Source: https://lukaswirth.dev/tlborm/print Provides advice on placing macro_rules! macros at the top of the root module to ensure they are consistently available 'crate wide'. This example shows the recommended structure using #[macro_use] on a module definition. ```rust #[macro_use] mod some_mod_that_defines_macros; mod some_mod_that_uses_those_macros; ``` -------------------------------- ### Rust Macro for Fibonacci Sequence Generation Source: https://lukaswirth.dev/tlborm/decl-macros/macros-practical This Rust macro defines a way to generate sequences, specifically demonstrated with the Fibonacci sequence. It initializes with given values and defines a recursive rule for subsequent elements. The example shows its usage in a `main` function to print the first 10 Fibonacci numbers. ```rust macro_rules! recurrence { ( a[n]: $sty:ty = $($inits:expr),+ , ... , $recur:expr ) => { { /* What follows here is *literally* the code from before, cut and pasted into a new position. No other changes have been made. */ use std::ops::Index; struct Recurrence { mem: [u64; 2], pos: usize, } struct IndexOffset<'a> { slice: &'a [u64; 2], offset: usize, } impl<'a> Index for IndexOffset<'a> { type Output = u64; fn index<'b>(&'b self, index: usize) -> &'b u64 { use std::num::Wrapping; let index = Wrapping(index); let offset = Wrapping(self.offset); let window = Wrapping(2); let real_index = index - offset + window; &self.slice[real_index.0] } } impl Iterator for Recurrence { type Item = u64; fn next(&mut self) -> Option { if self.pos < 2 { let next_val = self.mem[self.pos]; self.pos += 1; Some(next_val) } else { let next_val = { let n = self.pos; let a = IndexOffset { slice: &self.mem, offset: n }; (a[n-2] + a[n-1]) }; { use std::mem::swap; let mut swap_tmp = next_val; for i in (0..2).rev() { swap(&mut swap_tmp, &mut self.mem[i]); } } self.pos += 1; Some(next_val) } } } Recurrence { mem: [0, 1], pos: 0 } } }; } fn main() { let fib = recurrence![a[n]: u64 = 0, 1, ..., a[n-2] + a[n-1]]; for e in fib.take(10) { println!("{}", e) } } ``` -------------------------------- ### Rust Scoping Rules for Functions and Macros Source: https://lukaswirth.dev/tlborm/print Demonstrates that the scoping rules discussed for macros also apply to functions, with the exception of #[macro_use]. This example shows nested macro definitions and calls within functions to illustrate scope. ```rust macro_rules! X { () => { Y!() }; } fn a() { macro_rules! Y { () => {"Hi!"} } assert_eq!(X!(), "Hi!"); { assert_eq!(X!(), "Hi!"); macro_rules! Y { () => {"Bye!"} } assert_eq!(X!(), "Bye!"); } assert_eq!(X!(), "Hi!"); } fn b() { macro_rules! Y { () => {"One more"} } assert_eq!(X!(), "One more"); } fn main() { a(); b(); } ``` -------------------------------- ### Rust macro_rules! Visibility in Sub-modules Source: https://lukaswirth.dev/tlborm/print Demonstrates that function-like macros defined with macro_rules! are visible in sub-modules, unlike other language constructs. This example shows a macro X! being defined and then used within multiple modules. ```rust macro_rules! X { () => {}; } mod a { X!(); // defined } mod b { X!(); // defined } mod c { X!(); // defined } fn main() {} ``` -------------------------------- ### Example AST Transformation with Macro Expansion Source: https://lukaswirth.dev/tlborm/syntax-extensions/expansion Illustrates the structural transformation of an Abstract Syntax Tree (AST) when a macro invocation like `four!()` is expanded. The initial AST segment shows a `Macro` node for `four!()`, which is replaced by a `BinOp` node representing `1 + 3` after expansion, demonstrating that the expansion is treated as a complete AST node. ```text ┌─────────────┐ │ Let │ │ name: eight │ ┌─────────┐ │ init: ◌ │╶─╴│ BinOp │ └─────────────┘ │ op: Mul │ ┌╴│ lhs: ◌ │ ┌────────┐ │ │ rhs: ◌ │╶┐ ┌────────────┐ │ LitInt │╶┘ └─────────┘ └╴│ Macro │ │ val: 2 │ │ name: four │ └────────┘ │ body: () │ └────────────┘ ┌─────────────┐ │ Let │ │ name: eight │ ┌─────────┐ │ init: ◌ │╶─╴│ BinOp │ └─────────────┘ │ op: Mul │ ┌╴│ lhs: ◌ │ ┌────────┐ │ │ rhs: ◌ │╶┐ ┌─────────┐ │ LitInt │╶┘ └─────────┘ └╴│ BinOp │ │ val: 2 │ │ op: Add │ └────────┘ ┌╴│ lhs: ◌ │ ┌────────┐ │ │ rhs: ◌ │╶┐ ┌────────┐ │ LitInt │╶┘ └─────────┘ └╴│ LitInt │ │ val: 1 │ │ val: 3 │ └────────┘ └────────┘ ``` -------------------------------- ### Recursive Macro Expansion Example Source: https://lukaswirth.dev/tlborm/syntax-extensions/expansion Demonstrates the handling of nested macro expansions. An initial invocation `four!()` expands to `1 + three!()`. The compiler then performs a second pass to expand `three!()`, resulting in the final code `let x = 1 + 3;`. This illustrates that expansion occurs in multiple passes until all macro invocations are resolved. ```text let x = four!(); let x = 1 + three!(); let x = 1 + 3; ``` -------------------------------- ### Rust macro_rules! pat fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Illustrates the `pat` fragment specifier for matching any Rust pattern, including or-patterns. Useful for destructuring in match statements or function arguments. ```rust macro_rules! patterns { ($($pat:pat)*) => (); } patterns! { "literal" _ 0..5 ref mut PatternsAreNice 0 | 1 | 2 | 3 } fn main() {} ``` -------------------------------- ### Rust Macro Hygiene Example: Expanded Code Source: https://lukaswirth.dev/tlborm/decl-macros/macros-practical Shows the expanded version of the `using_a!` macro invocation. This clearly illustrates that the `a` defined within the macro and the `a` used in the expression are treated as separate identifiers due to macro hygiene. ```rust let four = { let a = 42; a / 10 }; ``` -------------------------------- ### Rust macro_rules! - Rule Specificity and Backtracking Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/metavar-and-expansion Demonstrates how `macro_rules!` parsers cannot backtrack, leading to dead rules if not ordered correctly from most-specific to least-specific. This example shows a macro where the second rule can never be matched due to the first rule consuming all valid input. ```rust #![allow(unused)] macro_rules! dead_rule { ($e:expr) => { ... }; ($i:ident +) => { ... }; } ``` -------------------------------- ### Rust Macro: Alternative Push-down Accumulation Syntax Source: https://lukaswirth.dev/tlborm/decl-macros/patterns/push-down-acc An alternative syntax for Rust macros using push-down accumulation, aiming for a more direct expression of array initialization. This example shows a different way to structure the `init_array` and `accum` macros. Dependencies include the standard Rust macro system. ```rust macro_rules! init_array { [$e:expr; $n:tt] => { { let e = $e; [accum!($n, e.clone())] } }; } macro_rules! accum { (3, $e:expr) => { $e, accum!(2, $e) }; (2, $e:expr) => { $e, accum!(1, $e) }; (1, $e:expr) => { $e }; } ``` -------------------------------- ### Rust macro_rules! Lexical Order Dependency Source: https://lukaswirth.dev/tlborm/print Illustrates that macro_rules! macros are only accessible after their definition in the source code. This example shows a macro X! being defined within a module and then used, highlighting that it's undefined before its declaration. ```rust mod a { // X!(); // undefined } mod b { // X!(); // undefined macro_rules! X { () => {}; } X!(); // defined } mod c { // X!(); // undefined } fn main() {} ``` -------------------------------- ### Rust macro_rules! Macro Redefinition and Shadowing Source: https://lukaswirth.dev/tlborm/print Demonstrates that defining macro_rules! macros multiple times is allowed, with the most recent declaration shadowing previous ones within its scope. This example shows macro X! being redefined and called with different arguments. ```rust #![allow(unused)] fn main() { macro_rules! X { (1) => {}; } X!(1); macro_rules! X { (2) => {}; } // X!(1); // Error: no rule matches `1` X!(2); mod a { macro_rules! X { (3) => {}; } // X!(2); // Error: no rule matches `2` X!(3); } // X!(3); // Error: no rule matches `3` X!(2); } ``` -------------------------------- ### Rust macro_rules! item fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Demonstrates the `item` fragment specifier for matching Rust item definitions, including visibility modifiers. This captures complete items like structs, enums, or functions. ```rust macro_rules! items { ($($item:item)*) => (); } items! { struct Foo; enum Bar { Baz } impl Foo {} pub use crate::foo; /*...*/ } fn main() {} ``` -------------------------------- ### Rust macro_rules! matcher with literal token trees Source: https://lukaswirth.dev/tlborm/decl-macros/macros-methodical This example demonstrates how to use literal token trees within a `macro_rules!` matcher. The matcher `(4 fn ['spang "whammo"] @_@)` must exactly match the input token sequence for the corresponding expansion to be used. ```rust macro_rules! gibberish { (4 fn ['spang "whammo"] @_@) => {...}; } ``` -------------------------------- ### Rust macro_rules! unifying macros with internal rules Source: https://lukaswirth.dev/tlborm/decl-macros/patterns/internal-rules This Rust example shows how to unify two `macro_rules!` macros, `as_expr!` and `foo!`, into a single macro using internal rules. The `foo` macro now internally handles the expression matching previously done by `as_expr!`, thus avoiding the need for a separate `as_expr!` macro and reducing global namespace pollution. ```rust #[macro_export] macro_rules! as_expr { ($e:expr) => {$e} } #[macro_export] macro_rules! foo { ($($tts:tt)*) => { as_expr!($($tts)*) }; } fn main() { assert_eq!(foo!(42), 42); } ``` -------------------------------- ### Use len() to get repetition count in Rust macros Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/metavar-expr The `len()` metavariable expression expands to the iteration count of a repetition at a specified depth. It defaults to depth 0. The expression expands to an unsuffixed integer literal token. This example demonstrates `len()` within nested macro repetitions. ```rust #![feature(macro_metavar_expr)] macro_rules! lets_count { ( $( $outer:ident ( $( $inner:ident ),* ) ; )* ) => { $( $( println!( "'{}' in inner iteration {}/{}", stringify!($inner), ${index()}, ${len()} ); println!( "'{}' in outer iteration {}/{}", stringify!($outer), ${index(1)}, ${len(1)} ); )* )* }; } fn main() { lets_count!( many (small , things) ; none () ; exactly ( one ) ; ); } ``` -------------------------------- ### Rust: Alternative Array Initialization (Illustrative - Not Supported) Source: https://lukaswirth.dev/tlborm/print This macro demonstrates a more direct, but unsupported, approach to array initialization that would require intermediate expressions to be complete syntax elements, which Rust macros do not allow. It is presented to contrast with the push-down accumulation method. ```rust macro_rules! init_array { [$e:expr; $n:tt] => { { let e = $e; [accum!($n, e.clone())] } }; } macro_rules! accum { (3, $e:expr) => { $e, accum!(2, $e) }; (2, $e:expr) => { $e, accum!(1, $e) }; (1, $e:expr) => { $e }; } ``` -------------------------------- ### Rust Metavariable Expressions: Count and Index Source: https://lukaswirth.dev/tlborm/print Provides examples of metavariable expressions in Rust macros, specifically focusing on `${count(ident, depth)}` and `${index(depth)}`. These expressions allow macros to access the number of repetitions and the current index within nested repetitions, enabling more dynamic macro behavior. ```rust #![feature(macro_metavar_expr)] macro_rules! example_macro { ($($e:expr),*) => { // Example using ${count} and ${index} // This is illustrative and requires a macro definition that utilizes these expressions. // For a full working example, refer to Rust documentation on macro_metavar_expr. // For instance, a macro that prints repetition info: // macro_rules! print_repetition_info { // ($($i:ident),*) => { // $( println!("Index: {}, Count: {}", ${index()}, ${count($i)} ) )* // } // } // print_repetition_info!(a, b, c); }; } fn main() { // Placeholder for a macro invocation that would use these features. // example_macro!(); } ``` -------------------------------- ### Rust macro_rules! lifetime fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Illustrates the `lifetime` fragment specifier, which matches Rust lifetimes or labels, similar to `ident` but with a preceding apostrophe. ```rust macro_rules! lifetimes { ($($lifetime:lifetime)*) => (); } lifetimes! { 'static 'shiv '_ } fn main() {} ``` -------------------------------- ### Parse Rust Tokens into Syntax Tree with syn (Rust) Source: https://lukaswirth.dev/tlborm/print The `syn` crate parses Rust token streams into a structured syntax tree, simplifying the process for procedural macros. It offers features for parsing standard Rust syntax, handling derive macro inputs, and a generic parsing API for custom syntax. It utilizes `proc_macro2` and provides location information for error reporting. ```Rust # This chapter talks about procedural macro hygiene and the type that encodes it, `Span`. Every token in a `TokenStream` has an associated `Span` holding some additional info. A span, as its documentation states, is `A region of source code, along with macro expansion information`. It points into a region of the original source code(important for displaying diagnostics at the correct places) as well as holding the kind of _hygiene_ for this location. The hygiene is relevant mainly for identifiers, as it allows or forbids the identifier from referencing things or being referenced by things defined outside of the invocation. There are 3 kinds of hygiene(which can be seen by the constructors of the `Span` type): * `definition site`(_**unstable**_): A span that resolves at the macro definition site. Identifiers with this span will not be able to reference things defined outside or be referenced by things outside of the invocation. This is what one would call "hygienic". * `mixed site`: A span that has the same hygiene as `macro_rules` declarative macros, that is it may resolve to definition site or call site depending on the type of identifier. See here for more information. * `call site`: A span that resolves to the invocation site. Identifiers in this case will behave as if written directly at the call site, that is they freely resolve to things defined outside of the invocation and can be referenced from the outside as well. This is what one would call "unhygienic". ``` -------------------------------- ### Rust Declarative Macros 2.0 (decl_macro) Introduction Source: https://lukaswirth.dev/tlborm/print This section introduces the experimental 'declarative macros 2.0' feature in Rust, intended as a replacement for `macro_rules!`. It notes that the system is incomplete and subject to change. -------------------------------- ### Rust Macro Expansion with rustc Source: https://lukaswirth.dev/tlborm/syntax-extensions/debugging Shows the command to compile a Rust file and view the expanded macro code using the unstable -Zunpretty=expanded argument. This is useful for debugging macro behavior by seeing the code generated by the macro. ```bash rustc +nightly -Zunpretty=expanded hello.rs ``` -------------------------------- ### Rust macro_rules! literal fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Shows the `literal` fragment specifier, which matches any Rust literal expression, including integers, floats, strings, bytes, and booleans. ```rust macro_rules! literals { ($($literal:literal)*) => (); } literals! { -1 "hello world" 2.3 b'b' true } fn main() {} ``` -------------------------------- ### Rust Crate-Wide Macro Placement Recommendation Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/scoping Provides a common piece of advice for managing macro visibility: place all `macro_rules!` macros intended for crate-wide access at the very top of the root module, before any other modules. This ensures consistent availability throughout the crate. ```rust #![allow(unused)] #[macro_use] mod some_mod_that_defines_macros; mod some_mod_that_uses_those_macros; ``` -------------------------------- ### Rust Macro Definition and Usage Source: https://lukaswirth.dev/tlborm/syntax-extensions/debugging Defines a shorthand macro `S!` for initializing a String and demonstrates its usage in a simple Rust program. This macro simplifies the creation of String objects from string literals. ```rust // Shorthand for initializing a `String`. macro_rules! S { ($e:expr) => {String::from($e)}; } fn main() { let world = S!("World"); println!("Hello, {}!", world); } ``` -------------------------------- ### Rust macro_rules! ident fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Shows the `ident` fragment specifier, which matches Rust identifiers and keywords. This is commonly used for capturing variable names, function names, etc. ```rust macro_rules! idents { ($($ident:ident)*) => (); } idents! { // _ <- This is not an ident, it is a pattern foo async O_________O _____O_____ } fn main() {} ``` -------------------------------- ### Rust macro_rules! block fragment example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/fragment-specifiers Demonstrates the `block` fragment specifier in Rust's `macro_rules!`. It matches a block expression enclosed in curly braces, which can contain zero or more statements. ```rust macro_rules! blocks { ($($block:block)*) => (); } blocks! { {} { let zig; } { 2 } } fn main() {} ``` -------------------------------- ### AST Representation of '1 + 2' Source: https://lukaswirth.dev/tlborm/print Illustrates the Abstract Syntax Tree (AST) structure for a simple arithmetic expression '1 + 2'. This shows how binary operations and integer literals are represented hierarchically. ```text ┌─────────┐ ┌─────────┐ │ BinOp │ ┌╴│ LitInt │ │ op: Add │ │ │ val: 1 │ │ lhs: ◌ │╶┘ └─────────┘ │ rhs: ◌ │╶┐ ┌─────────┐ └─────────┘ └╴│ LitInt │ │ val: 2 │ └─────────┘ ``` -------------------------------- ### Rust macro_rules! - Ambiguity Example Source: https://lukaswirth.dev/tlborm/print Illustrates a common ambiguity error in macro_rules! where the parser cannot determine the correct rule due to insufficient lookahead. This occurs when multiple parsing options are available for the same input sequence. ```Rust #[allow(unused)] fn main() { macro_rules! ambiguity { ($($i:ident)* $i2:ident) => { }; } // error: local ambiguity: multiple parsing options: built-in NTs ident ('i') or ident ('i2'). ambiguity!(an_identifier); } ``` -------------------------------- ### Define a single rule for Rust macro_rules! Source: https://lukaswirth.dev/tlborm/decl-macros/macros-methodical This illustrates a single rule within a `macro_rules!` macro. It specifies a matcher pattern `($matcher)` and a corresponding expansion `{$expansion}`. The parentheses around the matcher and braces around the expansion are conventional but can be varied. ```rust ($matcher) => {$expansion} ``` -------------------------------- ### Rust macro_rules! - Ambiguity Error Example Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/metavar-and-expansion Illustrates a common ambiguity error in `macro_rules!` where the parser cannot determine between two similar patterns, leading to a compilation failure. This occurs because the parser does not perform sufficient lookahead. ```rust #![allow(unused)] fn main() { macro_rules! ambiguity { ($($i:ident)* $i2:ident) => { }; } // error: local ambiguity: multiple parsing options: built-in NTs ident ('i') or ident ('i2'). ambiguity!(an_identifier); } ``` -------------------------------- ### Rust Macro: Getting Length with len() Source: https://lukaswirth.dev/tlborm/print Demonstrates the len() metavariable expression in Rust macros, which expands to the repetition count of a metavariable up to a specified depth. It's similar to count() but specifically focuses on length. ```Rust #![feature(macro_metavar_expr)] macro_rules! foo { ( $( $bar:ident ),* ) => { println!("The length of '$bar' is {}", ${len($bar)} ); }; } fn main() { foo!( one, two, three ); foo!(); } ``` -------------------------------- ### Define Rust macro_rules! with multiple rules Source: https://lukaswirth.dev/tlborm/decl-macros/macros-methodical This defines a `macro_rules!` macro named `$name` that accepts a sequence of rules ($rule0 to $ruleN). Each rule consists of a matcher and an expansion. A minimum of one rule is required. ```rust macro_rules! $name { $rule0 ; $rule1 ; // … $ruleN ; } ``` -------------------------------- ### Usage Example of a Derive Procedural Macro in Rust Source: https://lukaswirth.dev/tlborm/proc-macros/methodical/derive This snippet shows how to use a derive procedural macro in Rust. The `#[derive(TlbormDerive)]` attribute is applied to a struct, indicating that the `TlbormDerive` macro should be invoked to generate code for it. ```rust use tlborm_proc::TlbormDerive; #[derive(TlbormDerive)] struct Foo; ``` -------------------------------- ### Rust Declarative Macros vs. macro_rules! Syntax Source: https://lukaswirth.dev/tlborm/decl-macros/macros2 Compares the syntax of `macro` and `macro_rules!` for defining macros. Demonstrates `replace_expr` and `count_tts` macros. Note: requires `#![feature(decl_macro)]` which is not stable. ```rust #![feature(decl_macro)] macro_rules! replace_expr_ { ($_t:tt $sub:expr) => { $sub } } macro replace_expr($_t:tt $sub:expr) { $sub } macro_rules! count_tts_ { () => { 0 }; ($odd:tt $($a:tt $b:tt)*) => { (count_tts!($($a)*) << 1) | 1 }; ($($a:tt $even:tt)*) => { count_tts!($($a)*) << 1 }; } macro count_tts { () => { 0 }, ($odd:tt $($a:tt $b:tt)*) => { (count_tts!($($a)*) << 1) | 1 }, ($($a:tt $even:tt)*) => { count_tts!($($a)*) << 1 }, } ``` -------------------------------- ### Manual Expansion Trace for count_tts macro Source: https://lukaswirth.dev/tlborm/decl-macros/building-blocks/counting This section manually traces the expansion of the `count_tts` macro with a specific input, demonstrating the recursive application of its rules and bitwise operations. ```rust count_tts!(0 0 0 0 0 0 0 0 0 0); ``` ```rust count_tts!(0 0 0 0 0) << 1; ``` ```rust ((count_tts!(0 0) << 1) | 1) << 1; ``` ```rust ((count_tts!(0) << 1 << 1) | 1) << 1; ``` ```rust ((((count_tts!() << 1) | 1) << 1 << 1) | 1) << 1; ``` ```rust ((((0 << 1) | 1) << 1 << 1) | 1) << 1; ``` ```rust ((((0 << 1) | 1) << 1 << 1) | 1) << 1; ``` -------------------------------- ### Rust macro_rules! - Token Capture and Substitution Behavior Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/metavar-and-expansion Explains Rust macro substitution, which is not token-based but AST-node based. This example highlights how capturing with certain fragment specifiers makes the result 'un-destructible', preventing further matching or examination. ```rust macro_rules! capture_then_match_tokens { ($e:expr) => {match_tokens!($e)}; } macro_rules! match_tokens { ($a:tt + $b:tt) => {"got an addition"}; (($i:ident)) => {"got an identifier"}; ($($other:tt)*) => {"got something else"}; } fn main() { println!("{}\n{}\n{}\\n", match_tokens!((caravan)), match_tokens!(3 + 6), match_tokens!(5)); println!("{}\n{}\n{}", capture_then_match_tokens!((caravan)), capture_then_match_tokens!(3 + 6), capture_then_match_tokens!(5)); } ``` -------------------------------- ### Rust #[macro_use] on extern crate vs. Modules Source: https://lukaswirth.dev/tlborm/print Explains that #[macro_use] applied to an 'extern crate' behaves differently than when applied to a module; external crate macros are 'hoisted' to the top of the module. This example contrasts this behavior with module-level #[macro_use]. ```rust mod a { // X!(); // undefined } macro_rules! Y { () => {}; } mod b { X!(); // defined, and so is Y! } #[macro_use] extern crate macs; mod c { X!(); // defined, and so is Y! } fn main() {} ``` -------------------------------- ### Rust 2018: Importing Macros from External Crate via `use` Source: https://lukaswirth.dev/tlborm/decl-macros/minutiae/import-export Explains the improved way to import macro_rules! macros from external crates in Rust 2018 Edition using the `use` keyword for namespaced access. ```rust use some_crate::some_macro; fn main() { some_macro!("hello"); // as well as some_crate::some_other_macro!("macro world"); } ``` -------------------------------- ### Rust macro_rules! Macro Dependency Resolution Source: https://lukaswirth.dev/tlborm/print Explains that the lexical order dependency does not apply to macros themselves, meaning a macro can call another macro that is defined later. This example shows macro X! calling macro Y!, where Y! is defined after X!. ```rust mod a { // X!(); // undefined } macro_rules! X { () => { Y!(); }; } mod b { // X!(); // defined, but Y! is undefined } macro_rules! Y { () => {}; } mod c { X!(); // defined, and so is Y! } fn main() {} ``` -------------------------------- ### Log All Tokens with Rust's log_syntax! Source: https://lukaswirth.dev/tlborm/print The `log_syntax!` macro causes the Rust compiler to output all tokens passed to it. This provides more targeted debugging than `trace_macros!`. Note that this feature requires a nightly compiler channel. ```rust #![feature(log_syntax)] macro_rules! sing { () => {}; ($tt:tt $($rest:tt)*) => {log_syntax!($tt); sing!($($rest)*);}; } sing! { ^ < @ < . @ * '\x08' '{' '"' _ # ' ' - @ '$' && / _ % ! ( '\t' @ | = > ; '\x08' ''' + '$' ? '\x7f' , # '"' ~ | ) '\x07' } fn main() {} ``` -------------------------------- ### Rust macro_rules! Lexical Order Dependency Across Scopes Source: https://lukaswirth.dev/tlborm/print Shows that the lexical order dependency for macro_rules! applies even when the macro is moved to an outer scope. This example demonstrates that a macro defined globally is accessible in subsequent modules. ```rust mod a { // X!(); // undefined } macro_rules! X { () => {}; } mod b { X!(); // defined } mod c { X!(); // defined } fn main() {} ``` -------------------------------- ### Rust Recurrence Macro Definition and Usage Source: https://lukaswirth.dev/tlborm/decl-macros/macros-practical Defines a Rust macro `recurrence!` that generates a sequence based on initial values and a recurrence formula. It implements an iterator to provide the sequence elements, handling memory for previous values. The example demonstrates its use for calculating factorials. ```rust ( $seq:ident [ $ind:ident ]: $sty:ty = $($inits:expr),+ ; ... ; $recur:expr ) => { { use std::ops::Index; const MEM_SIZE: usize = count_exprs!($($inits),+); struct Recurrence { mem: [$sty; MEM_SIZE], pos: usize, } struct IndexOffset<'a> { slice: &'a [$sty; MEM_SIZE], offset: usize, } impl<'a> Index for IndexOffset<'a> { type Output = $sty; #[inline(always)] fn index<'b>(&'b self, index: usize) -> &'b $sty { use std::num::Wrapping; let index = Wrapping(index); let offset = Wrapping(self.offset); let window = Wrapping(MEM_SIZE); let real_index = index - offset + window; &self.slice[real_index.0] } } impl Iterator for Recurrence { type Item = $sty; #[inline] fn next(&mut self) -> Option<$sty> { if self.pos < MEM_SIZE { let next_val = self.mem[self.pos]; self.pos += 1; Some(next_val) } else { let next_val = { let $ind = self.pos; let $seq = IndexOffset { slice: &self.mem, offset: $ind }; $recur }; { use std::mem::swap; let mut swap_tmp = next_val; for i in (0..MEM_SIZE).rev() { swap(&mut swap_tmp, &mut self.mem[i]); } } self.pos += 1; Some(next_val) } } } Recurrence { mem: [$($inits),+], pos: 0 } } }; } fn main() { for e in recurrence!(f[i]: f64 = 1.0; ...; f[i-1] * i as f64).take(10) { println!("{}", e) } } ``` -------------------------------- ### Rust macro_rules! - Macro Rule Definition Source: https://lukaswirth.dev/tlborm/print Demonstrates a macro definition with two rules where the second rule is unreachable due to the parser's inability to backtrack. This illustrates the importance of ordering macro rules from most specific to least specific. ```Rust #[macro_export] macro_rules! dead_rule { ($e:expr) => { ... }; ($i:ident +) => { ... }; } ```