phplrt

https://phplrt.org/docs
Admin
phplrt is a PHP library for developing domain-specific languages by defining grammar rules to...
Tokens:18,941
Snippets:244
Trust Score:5.2
Update:6 months ago
Show doc for...
Tokens:
Raw
### Example Composer.json Configuration

Source: https://phplrt.org/docs/installation

This is an example of how your composer.json file might look after installing the Phplrt packages. It specifies the PHP version and the required runtime and development packages.

```json
{
    "require": {
        "php": "^8.1",
        "phplrt/runtime": "^3.6"
    },
    "require-dev": {
        "phplrt/phplrt": "^3.6"
    }
}
```

--------------------------------

### Install Phplrt Runtime and Development Packages

Source: https://phplrt.org/docs/installation

Use these Composer commands to add the phplrt/runtime for running applications and phplrt/phplrt for assembly and compilation to your project. The --dev flag installs the latter as a development dependency.

```bash
$ composer require phplrt/runtime
$ composer require phplrt/phplrt --dev
```

--------------------------------

### Basic Lexer Usage

Source: https://phplrt.org/docs/lexer

Demonstrates how to install and use the Lexer class to tokenize a simple string with defined token patterns.

```APIDOC
## Basic Lexer Usage

### Description
This section shows how to install the lexer package and use its basic functionality to tokenize a string.

### Installation
```bash
composer require phplrt/lexer
```

### Usage Example
```php
$lexer = new Phplrt\Lexer\Lexer([
    'T_WHITESPACE'  => '\s+',
    'T_PLUS'        => '\+',
    'T_DIGIT'       => '\d+'
]);

foreach ($lexer->lex('23 + 42') as $token) {
    echo $token . "\n";
}

// Expected output:
// "23" (T_DIGIT)
// " " (T_WHITESPACE)
// "+" (T_PLUS)
// " " (T_WHITESPACE)
// "42" (T_DIGIT)
// \0
```

### Method
`Phplrt\Lexer\Lexer::lex(string $string): Iterator<TokenInterface>`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
None

### Response
#### Success Response (200)
An iterator of `Phplrt\Contracts\Lexer\TokenInterface` objects.

#### Response Example
```json
{
  "example": "Iterator<TokenInterface>"
}
```
```

--------------------------------

### Example Output

Source: https://phplrt.org/docs/guide/creating-project

The expected output structure when parsing a valid expression.

```text
array:3 [
  0 => Phplrt\Lexer\Token\Token {
    -bytes: null
    -offset: 0
    -value: "2"
    -name: "number"
  }
  1 => Phplrt\Lexer\Token\Token {
    -bytes: null
    -offset: 4
    -value: "2"
    -name: "number"
  }
  2 => Phplrt\Lexer\Token\Token {
    -bytes: null
    -offset: 8
    -value: "4"
    -name: "number"
  }
]
```

--------------------------------

### Implement AST node class

Source: https://phplrt.org/docs/compiler/code

Example implementation of an AST node class that satisfies the NodeInterface.

```php
<?php

use Phplrt\Contracts\Ast\NodeInterface;
use Phplrt\Contracts\Lexer\TokenInterface;

class ExampleAstNode implements NodeInterface
{
    private int $digit;

    public function __construct($state, TokenInterface $digit, int $offset) 
    {
        $this->digit = (int)$digit->getValue();
    }

    /**
     * The required method of NodeInterface, which should return 
     * children AST nodes.
     * 
     * In this case, the node is empty, so the iterator returns nothing.
     * 
     * @return \Traversable|NodeInterface[]
     */
    public function getIterator(): \Traversable
    {
        return new \EmptyIterator();
    }
}
```

--------------------------------

### PhpDoc Type Grammar Example

Source: https://phplrt.org/docs/examples/phpdoc-types

Illustrates the grammar for describing complex types in PhpDoc, compatible with Psalm and PHPStan. This example showcases nested arrays, callables, lists, iterables with key-value pairs, constant references, and variadic type hints.

```php
array {
    field1: callable(Example, int): mixed,
    field2: list<Some>,
    field3: iterable<array-key, array{ int, non-empty-string } >,
    Some::CONST_*,
    ...
}
```

--------------------------------

### Initialize Multistate Lexer with Lexers

Source: https://phplrt.org/docs/lexer/multistate

Create a multistate lexer by providing an array of lexers, each assigned to a state name. This example initializes a lexer with 'html' and 'php' states.

```php
$lexer = new \Phplrt\Lexer\Multistate([
    'html' => new \Phplrt\Lexer\Lexer([ ... ]),
    'php' => new \Phplrt\Lexer\Lexer([ ... ])
]);
```

--------------------------------

### Load and Parse Grammar

Source: https://phplrt.org/docs/compiler

Loads a grammar file into memory and parses example text using the compiler. Ensure the 'phplrt/compiler' package is installed as a dev dependency.

```php
<?php

use Phplrt\Source\File;
use Phplrt\Compiler\Compiler;

$compiler = new Compiler();
$compiler->load(File::fromPatname(__DIR__ . '/path/to/file.pp2'));

echo $compiler->parse(File::fromSources('example text'));

```

--------------------------------

### Configure Multistate Lexer Transitions

Source: https://phplrt.org/docs/lexer/multistate

Define transition rules for a multistate lexer to specify how to switch between states based on token recognition. This example sets transitions from 'html' to 'php' on 'T_PHP_OPEN' and back from 'php' to 'html' on 'T_PHP_CLOSE'.

```php
$lexer = new \Phplrt\Lexer\Multistate(
    states: [
        'html' => new \Phplrt\Lexer\Lexer([ ... ]),
        'php' => new \Phplrt\Lexer\Lexer([ ... ])
    ],
    transitions: [
        'html' => [ 'T_PHP_OPEN'  => 'php'  ],
        'php'  => [ 'T_PHP_CLOSE' => 'html' ],
    ]
);
```

--------------------------------

### Define EBNF-style grammar rules

Source: https://phplrt.org/docs/compiler/grammar

Example of defining basic arithmetic rules using standard EBNF notation.

```text
(* "sum" is a rule that determines the sequence of a number, an addition symbol and one more number *)
sum = digit plus digit ;

(* "digit" is one of the available numeric characters *)
digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;

(* "plus" is a plus sign. Incredibly! *)
plus = "+" ;
```

--------------------------------

### Handle Syntax Errors

Source: https://phplrt.org/docs/guide/creating-project

Examples of error messages generated when parsing invalid input strings.

```php
$math->parse('2 + 2 & 4');

// Syntax error, unrecognized "&"
//    1. | 2 + 2 & 4
//       |       ^ in /.../lexer/src/Exception/UnrecognizedTokenException.php:19

$math->parse('2 + ');

// Syntax error, unexpected end of input, "number" is expected 
//    1. | 2 + 
//       |      in /.../parser/src/Exception/UnexpectedTokenException.php:35

$math->parse('+ 42');

// Syntax error, unexpected "+" (plus), "number" is expected
//    1. | + 42
//       | ^ in /.../parser/src/Exception/UnexpectedTokenException.php:35
```

--------------------------------

### Default Unknown Token Handling (Lexing Error)

Source: https://phplrt.org/docs/lexer/unknown-token

By default, the lexer stops with an error when an unrecognized token is encountered. This example demonstrates the default behavior.

```php
$lexer = new Phplrt\Lexer\Lexer(
    tokens: ['T_DIGIT' => '\d+'],
);

foreach ($lexer->lex('42 unknown') as $token) {
    echo $token->getName() . "\n";
}

// T_DIGIT
// Uncaught Phplrt\Lexer\Exception\UnrecognizedTokenException: Syntax error, unrecognized " unknown"
```

--------------------------------

### Define Fully Qualified Name Rule

Source: https://phplrt.org/docs/examples/phpdoc-types

Defines 'FullQualifiedName' starting with a namespace delimiter.

```grammar
FullQualifiedName
  : ::T_NS_DELIMITER:: NamePart() (::T_NS_DELIMITER:: NamePart())*
  ;
```

--------------------------------

### Define Relative Name Rule

Source: https://phplrt.org/docs/examples/phpdoc-types

Defines 'RelativeName' starting with a name part.

```grammar
RelativeName
  : NamePart() (::T_NS_DELIMITER:: NamePart())*
  ;
```

--------------------------------

### Token Interface Definition

Source: https://phplrt.org/docs/lexer

The TokenInterface defines the methods available for interacting with lexer tokens, including getting the name, offset, value, and byte length.

```php
interface TokenInterface
{
    public function getName(): string;
    public function getOffset(): int;
    public function getValue(): string;
    public function getBytes(): int;
}

```

--------------------------------

### Assemble Grammar Binary

Source: https://phplrt.org/docs/guide/creating-project

Create a build script to compile grammar definitions into a configuration file.

```php
<?php

require __DIR__ . '/vendor/autoload.php';

$compiler = new \Phplrt\Compiler\Compiler();

// Source grammar loading.
$compiler->load('

    // TODO

');

// Compiling the grammar into a set of
// instructions for the parser.
$assembly = $compiler->build();

// Saving an assembly to a file.
file_put_contents(__DIR__ . '/grammar.php', $assembly->generate());
```

--------------------------------

### Initialize Parser Lexer

Source: https://phplrt.org/docs/guide/creating-project

Set up the Lexer component within a custom Parser class using the generated grammar configuration.

```php
<?php

declare(strict_types=1);

namespace Calculator;

use Phplrt\Contracts\Lexer\LexerInterface;
use Phplrt\Lexer\Lexer;

final class Parser
{
    private LexerInterface $lexer;

    public function __construct()
    {
        // This will be the path to our configuration file
        $config = require __DIR__ . '/grammar.php';

        $this->lexer = new Lexer(
            //
            // The "tokens" array field contains a list of lexer
            // states with list of regular expression of tokens.
            //
            // Since we use only one state, we can load the main one,
            // it is called "default" and is always available.
            //
            $config['tokens']['default'],
            //
            // This array field contains a list of token names
            // to skip.
            //
            $config['skip'],
        );
    }
}
```

--------------------------------

### Basic Lexer Usage

Source: https://phplrt.org/docs/lexer

Instantiate the Lexer with token definitions and lex a string. The lexer returns an iterator of tokens.

```php
$lexer = new Phplrt\Lexer\Lexer([
    'T_WHITESPACE'  => '\s+',
    'T_PLUS'        => '\+',
    'T_DIGIT'       => '\d+'
]);

foreach ($lexer->lex('23 + 42') as $token) {
    echo $token . "\n";
}

//
// Expected output:
//
// > "23" (T_DIGIT)
// > " " (T_WHITESPACE)
// > "+" (T_PLUS)
// > " " (T_WHITESPACE)
// > "42" (T_DIGIT)
// > \0
//

```

--------------------------------

### Initialize a Lexer

Source: https://phplrt.org/docs/parser

Define token patterns and skip whitespace using the Lexer class.

```php
use Phplrt\Lexer\Lexer;

$lexer = (new Lexer())
    ->append('T_NUMBER', '\\d+')
    ->append('T_PLUS', '\\+')
    ->append('T_WHITESPACE', '\\s+')
        ->skip('T_WHITESPACE')
;
```

--------------------------------

### JSON5 Identifier Token

Source: https://phplrt.org/docs/examples/json5

Defines the token for identifiers in JSON5, which can start with '$' or an alphabet character, followed by alphanumeric characters or '$'.

```json5
%token T_IDENTIFIER             ["$_A-Za-z]["$_0-9A-Za-z]*
```

--------------------------------

### Execute Parser

Source: https://phplrt.org/docs/guide/creating-project

Instantiate the custom parser and process an input string.

```php
$math = new Calculator\Parser();

$ast = $math->parse('2 + 2');

var_dump($ast);
```

--------------------------------

### Instantiate Lexer with a Custom Driver

Source: https://phplrt.org/docs/lexer/drivers

Instantiate the Lexer class with a custom driver implementation using the constructor. Alternatively, you can set the driver after instantiation using the `setDriver` method.

```php
<?php

// Usage
$lexer = new \Phplrt\Lexer\Lexer($tokens, $skip, new MyDriver());

// Or alternative
$lexer->setDriver(new MyDriver());

```

--------------------------------

### Create an ArrayBuffer for Testing

Source: https://phplrt.org/docs/parser/rules

Initializes an ArrayBuffer with a sequence of Tokens. This buffer is used to test the reduce method of grammar rules.

```php
<?php

use Phplrt\Buffer\ArrayBuffer;use Phplrt\Lexer\Token\Token;

$buffer = new ArrayBuffer([
    new Token('T_DIGIT', '2', 0),
    new Token('T_PLUS', '+', 2),
    new Token('T_DIGIT', '2', 4),
]);
```

--------------------------------

### Implement Custom Lexer Driver Interface

Source: https://phplrt.org/docs/lexer/drivers

Implement the `DriverInterface` to create a custom lexer driver. The `run` method should yield tokens, and the `reset` method is for clearing any internal memoized state.

```php
<?php

use Phplrt\Lexer\Driver\DriverInterface;
use Phplrt\Contracts\Source\ReadableInterface;

class MyDriver implements DriverInterface
{
    public function run(array $tokens, ReadableInterface $source, int $offset = 0): iterable
    {
        // The method accepts an array of tokens, an object with the source 
        // file and a shift in bytes relative to the beginning of the text 
        // from which you want to start parsing. 
    
        // The result should be a token iterator.
        yield new Token($tokenName, $tokenValue, $tokenOffset);
    }

    public function reset(): void 
    {
        //
        // Method for resetting internal memoized state, if required.
        //
        // For example, in cases where a set of tokens is copied to a whole 
        // regular expression after the first run.
        //
    }
}

```

--------------------------------

### Lexing with Prepended Token (Correct Order)

Source: https://phplrt.org/docs/lexer/modification

Shows correct lexing behavior when T_FLOAT is prepended, ensuring floating-point numbers are correctly identified.

```php
foreach ($lexer->lex('42.0') as $i => $token) {
    echo $i . ' => ' . $token . "\n";
}

//
// Expected Output:
//
// 0 => "42.0" (T_FLOAT)
// 1 => \0
//

```

--------------------------------

### Configure composer.json

Source: https://phplrt.org/docs/guide/creating-project

Define project dependencies and PSR-4 autoloading for the parser implementation.

```json
{
    "require": {
        "phplrt/runtime": "^3.6"
    },
    "autoload": {
        "psr-4": {
            "Calculator\\": "src"
        }
    },
    "require-dev": {
        "phplrt/phplrt": "^3.6"
    }
}
```

--------------------------------

### Accessing Token Properties

Source: https://phplrt.org/docs/lexer

Demonstrates how to retrieve specific properties of a token object using its interface methods.

```php
echo $token->getName();
// Excepted Output: string("T_DIGIT")

echo $token->getOffset();
// Excepted Output: int(0)

echo $token->getValue();
// Excepted Output: string("2")

echo $token->getBytes();
// Excepted Output: int(1)

```

--------------------------------

### Initialize Lexer with Default EOI

Source: https://phplrt.org/docs/lexer/end-of-input

Demonstrates the default behavior where a T_EOI token is appended to the token stream.

```php
$lexer = new Phplrt\Lexer\Lexer(
    tokens: [ 'T_DIGIT' => '\d+' ],
);

foreach ($lexer->lex('42') as $token) {
    echo $token->getName() . "\n";
}

// T_DIGIT
// T_EOI
```

--------------------------------

### Configure Parser with AST Builder

Source: https://phplrt.org/docs/guide/creating-project

Set the `Runtime::CONFIG_AST_BUILDER` option in the parser configuration to use a `SimpleBuilder` instance. This builder will use the 'reducers' section of the compiled grammar to construct the AST.

```php
use Phplrt\Parser\SimpleBuilder;

final class Parser
{
    /* ... */

    public function __construct()
    {
        /* ... lexer initialization ... */

        $this->parser = new Runtime($this->lexer, $config['grammar'], [
            /** ... other options ... */
            
            //
            // This option contains a reference to the tree builder instance.
            //
            // All construction rules will be loaded from the "reducers" section
            // of the compiled grammar.
            //
            Runtime::CONFIG_AST_BUILDER => new SimpleBuilder($config['reducers']),
        ]);
    }
}
```

--------------------------------

### Configuring Unknown Token Handler (PassthroughHandler)

Source: https://phplrt.org/docs/lexer/unknown-token

Using `PassthroughHandler` allows unknown tokens to be returned like any other token, including their name and value.

```php
$lexer = new Phplrt\Lexer\Lexer(
    tokens: ['T_DIGIT' => '\d+'],
    onUnknownToken: new \Phplrt\Lexer\Config\PassthroughHandler(),
);

foreach ($lexer->lex('42 unknown 23') as $token) {
    echo $token->getName() . ' with value (' . $token->getValue() . ")\n";
}

// T_DIGIT with value (42)
// T_UNKNOWN with value ( unknown )
// T_DIGIT with value (23)
// T_EOI with value (\0)
```

--------------------------------

### Parse Source Code

Source: https://phplrt.org/docs/parser

Execute the parser with the defined lexer and grammar.

```php
$parser = new \Phplrt\Parser\Parser($lexer, $grammar, $options);

var_dump($parser->parse('2 + 2'));
```

--------------------------------

### Token Object Interface

Source: https://phplrt.org/docs/lexer

Details the methods available on the `TokenInterface` for retrieving information about each token.

```APIDOC
## Token Objects

### Description
This section describes the `TokenInterface` and provides examples of how to use its methods to get information about a token.

### Interface Definition
```php
interface TokenInterface
{
    public function getName(): string;
    public function getOffset(): int;
    public function getValue(): string;
    public function getBytes(): int;
}
```

### Usage Example
```php
// Assuming $token is an instance of TokenInterface

echo $token->getName();
// Expected Output: string("T_DIGIT")

echo $token->getOffset();
// Expected Output: int(0)

echo $token->getValue();
// Expected Output: string("2")

echo $token->getBytes();
// Expected Output: int(1)
```

### Methods
- `getName(): string` - Returns the name of the token.
- `getOffset(): int` - Returns the starting position (offset) of the token in the input string.
- `getValue(): string` - Returns the actual string value of the token.
- `getBytes(): int` - Returns the number of bytes the token occupies.
```

--------------------------------

### Delegate rule to class

Source: https://phplrt.org/docs/compiler/code

Use the -> operator to map a grammar rule to a specific PHP class constructor.

```text
#Digit -> ExampleAstNode
  : <T_DIGIT> 
  ;
```

--------------------------------

### Configure Parser Runtime

Source: https://phplrt.org/docs/guide/creating-project

Extend the Parser class to include the runtime parser logic using the grammar configuration.

```php
// ...
use Phplrt\Contracts\Parser\ParserInterface;
use Phplrt\Parser\Parser as Runtime;

final class Parser
{
    private LexerInterface $lexer;
    private ParserInterface $parser;

    public function __construct()
    {
        $config = require __DIR__ . '/grammar.php';

        $this->lexer = /* ... lexer loading ... */
        
        $this->parser = new Runtime(
            //
            // The first required argument is a
            // reference to the lexer.
            //
            $this->lexer,
            //
            // The second argument is a list of parser rules.
            //
            // This list can also be loaded from a "grammar.php"
            // config file.
            //
            $config['grammar'],
            // Additional options
            [
                //
                // It is worth paying attention to one option, which
                // is also desirable to set: This is the name of the main
                // rule from which the analysis of grammar will begin.
                //
                Runtime::CONFIG_INITIAL_RULE => $config['initial'],
            ]
        );
    }
}
```

--------------------------------

### Configuring Unknown Token Handler (NullHandler)

Source: https://phplrt.org/docs/lexer/unknown-token

The `NullHandler` can be used to skip unknown tokens entirely, preventing them from being included in the lexed output.

```php
$lexer = new Phplrt\Lexer\Lexer(
    tokens: ['T_DIGIT' => '\d+'],
    onUnknownToken: new \Phplrt\Lexer\Config\NullHandler(),
);

foreach ($lexer->lex('42 unknown 23') as $token) {
    echo $token->getName() . ' with value (' . $token->getValue() . ")\n";
}

// T_DIGIT with value (42)
// T_DIGIT with value (23)
// T_EOI with value (\0)
```

--------------------------------

### Parse Expression and Dump AST

Source: https://phplrt.org/docs/guide/creating-project

Execute the parser with an arithmetic expression string and then use `var_dump` to inspect the resulting Abstract Syntax Tree.

```php
// Execution "2 + 2 + 4" expression:

$ast = $math->parse('2 + 2 + 4');

var_dump($ast);

```

--------------------------------

### Configure Parser with Custom AST Builder

Source: https://phplrt.org/docs/parser/ast

Instantiate the PHPLRT Parser, providing your custom AST builder via the `Parser::CONFIG_AST_BUILDER` configuration option. This allows the parser to use your custom node creation logic.

```php
use Phplrt\Parser\Parser;

$parser = new Parser($lexer, $grammar, [
    Parser::CONFIG_AST_BUILDER => new MyBuilder()
]);
```

--------------------------------

### Test Lexeme Rule Reduction

Source: https://phplrt.org/docs/parser/rules

Applies a Lexeme rule to a buffer and iterates through it, demonstrating how the reduce method processes tokens. The output shows which tokens are matched by the rule.

```php
<?php

$rule = new \Phplrt\Parser\Grammar\Lexeme('T_PLUS');

while ($buffer->valid()) {
    var_dump($buffer->key(), $rule->reduce($buffer));

    $buffer->next();
}

//
// Approximate Output:
//
// int(0)   NULL
//
// int(1)   object(Phplrt\Lexer\Token\Token)#7 (4) {
//              ["offset":private] => int(2)
//              ["value":private]  => string(1) "+"
//              ["name":private]   => string(6) "T_PLUS"
//          }
//
// int(2)   NULL
//
```

--------------------------------

### Configuring Unknown Token Handler (ThrowErrorHandler)

Source: https://phplrt.org/docs/lexer/unknown-token

The `onUnknownToken` argument in the Lexer constructor allows overriding the default behavior. `ThrowErrorHandler` is the default, explicitly shown here.

```php
$lexer = new Phplrt\Lexer\Lexer(
    tokens: ['T_DIGIT' => '\d+'],
    onUnknownToken: new \Phplrt\Lexer\Config\ThrowErrorHandler(),
);
```

--------------------------------

### Configure Parser Grammar

Source: https://phplrt.org/docs/parser

Map grammar rules to Concatenation and Lexeme objects for the parser.

```php
use Phplrt\Parser\Grammar\Concatenation;
use Phplrt\Parser\Grammar\Lexeme;
use Phplrt\Parser\Parser;

$options = [Parser::CONFIG_INITIAL_RULE => 'expression'];

//
// This (e)BNF construction:
// expression = T_NUMBER T_PLUS T_NUMBER ;
// 
// Looks like:
// Concatenation = Token1 Token2 Token1
//
$grammar = [
    // Concat: 1 then 2 then 1
    'expression' => new Concatenation([1, 2, 1]),

    // 1 is a T_NUMBER token
    1 => new Lexeme('T_NUMBER'),

    // 2 is a T_PLUS lexeme
    2 => new Lexeme('T_PLUS'),
];
```

--------------------------------

### Prepend Multiple New Tokens Individually

Source: https://phplrt.org/docs/lexer/modification

To prepend each token from a list individually to the beginning of the lexer's token definitions, specify the second argument of `prependMany`.

```php
$lexer = new \Phplrt\Lexer\Lexer(['T_WHITESPACE' => '\s+']);

$lexer->prependMany([
    'T_INT'   => '\d+',
    'T_FLOAT' => '\d+\.\d+',
]);

// Expected Token Definitions:
// [
//     'T_FLOAT' => '\d+\.\d+',
//     'T_INT' => '\d+',
//     'T_WHITESPACE' => '\s+',
// ]

```

--------------------------------

### Define Pointcut Grammar Rules

Source: https://phplrt.org/docs/examples/go-aop

Grammar rules for parsing pointcut references and dynamic execution points.

```text
    ExecutionPointcut()
    ::T_RIGHT_PAREN::
  ;

DynamicExecutionPointcut
  : ::T_DYNAMIC:: ::T_LEFT_PAREN::
      MethodDefinition()
    ::T_RIGHT_PAREN::
  ;

MatchInheritedPointcut
  : ::T_MATCH:: ::T_LEFT_PAREN:: ::T_RIGHT_PAREN::
  ;

PointcutReference
  : PointcutReferenceContext() ::T_OBJECT_ACCESS:: PropertyDefinitionBody()
  ;

PointcutReferenceContext
  : NamespacePattern()
  | <T_THIS>
  ;
```

--------------------------------

### Parse Expressions

Source: https://phplrt.org/docs/guide/creating-project

Execute the parser on an input string to generate an Abstract Syntax Tree or token list.

```php
$ast = $math->parse('2 + 2 + 4');

var_dump($ast);
```

--------------------------------

### Define Property and Method Reference Grammar

Source: https://phplrt.org/docs/examples/go-aop

Grammar rules for property and method definitions including modifiers and class filters.

```text
PropertyDefinition
  : PropertyModifiers()?
    ClassFilter() AccessType() PropertyDefinitionBody()
  ;

PropertyDefinitionBody
  : NamePattern()
  ;

MethodDefinition
  : MethodModifiers()?
    ClassFilter() AccessType() FunctionDefinition()
  ;
```

--------------------------------

### Map Grammar Rules to AST Nodes

Source: https://phplrt.org/docs/guide/creating-project

Define grammar rules to construct AST nodes. The `expression` rule creates an `Addition` node if there are two children, otherwise returns the child. The `number` rule creates a `Number` node from the token value.

```php
$compiler->load('
    // ... token definitions ...

    /**
     * BEFORE:
     * 
     *  expression : number() (::plus:: expression())?
     * 
     *  number : <number>
     * 
     */

    expression -> {
        // in case of "$children" sequence is a "number()" and "expression()"
        if (count($children) === 2) {
            return new \Calculator\Node\Addition($children[0], $children[1]);
        }

        // otherwise (only "number()")
        return $children;
    }
      : number() (::plus:: expression())?

    number -> { return new \Calculator\Node\Number((int)$token->getValue()); }
      : <number>

');

```

--------------------------------

### Define Grammar Rules

Source: https://phplrt.org/docs/guide/creating-project

Load grammar definitions into the compiler to specify tokens, skip rules, and expression structures.

```php
// Source grammar loading.
$compiler->load('

    // All digits sequence should be recognized as "number"
    %token number \d+
    
    // All "+" chars should be recognized as "plus"
    %token plus \+

    // All whitespace chars should be ignored
    %skip whitespace \s+

    // This means that each "expression" matches the sequence:
    //  - "number" (required)
    //  - then optional (from 0 to 1):
    //      - "plus"
    //      - then another "expression"
    expression : number() (::plus:: expression())?

    number : <number>

');
```

--------------------------------

### Implement Parse Method

Source: https://phplrt.org/docs/guide/creating-project

Add a public method to the Parser class to execute the parsing process.

```php
final class Parser
{
    /* ... constructor ... */ 

    public function parse(string $code): iterable
    {
        return $this->parser->parse($code);
    }
}
```

--------------------------------

### Compile and parse grammar

Source: https://phplrt.org/docs/compiler/grammar

Use the Phplrt Compiler to load a grammar string and parse input.

```php
use Phplrt\Compiler\Compiler;

$compiler = new Compiler();
$compiler->load('
    /**
     * Grammar sources
     */
    %token T_DIGIT      \d
    %token T_PLUS       \+
    %skip  T_WHITESPACE \s+
    
    #Sum
      : <T_DIGIT> ::T_PLUS:: <T_DIGIT> 
      ;
');

echo $compiler->parse('2 + 2');
```

--------------------------------

### Prepend a New Token to Phplrt Lexer

Source: https://phplrt.org/docs/lexer/modification

Use `prepend` to add a new token definition to the beginning of the lexer's token list. This ensures the new token is prioritized during lexing.

```php
<?php

$lexer = new \Phplrt\Lexer\Lexer(['T_INT' => '\d+']);

$lexer->prepend('T_FLOAT', '\d+\.\d+');

// Expected Token Definitions:
// [
//     'T_FLOAT' => '\d+\.\d+',
//     'T_INT' => '\d+',
// ]

```

--------------------------------

### Prepend Multiple New Tokens to Phplrt Lexer (Group)

Source: https://phplrt.org/docs/lexer/modification

Use `prependMany` to add multiple token definitions to the beginning of the lexer's token list as a group. The entire list is added to the front.

```php
$lexer = new \Phplrt\Lexer\Lexer(['T_WHITESPACE' => '\s+']);

$lexer->prependMany([
    'T_INT'   => '\d+',
    'T_FLOAT' => '\d+\.\d+',
]);

// Expected Token Definitions:
// [
//     'T_INT' => '\d+',
//     'T_FLOAT' => '\d+\.\d+',
//     'T_WHITESPACE' => '\s+',
// ]

```

--------------------------------

### Define Function Reference Grammar

Source: https://phplrt.org/docs/examples/go-aop

Grammar rules for defining function patterns and argument requirements.

```text
FunctionDefinition
  : NamePattern() FunctionDefinitionArguments()
  ;

FunctionDefinitionArguments
  : FunctionAnyArguments()
  | FunctionNoArguments()
  ;

FunctionNoArguments
  : ::T_LEFT_PAREN:: ::T_RIGHT_PAREN::
  ;

FunctionAnyArguments
  : ::T_LEFT_PAREN:: ::T_ASTERISK:: ::T_RIGHT_PAREN::
  ;
```

--------------------------------

### Delegate rule to PHP code block

Source: https://phplrt.org/docs/compiler/code

Execute custom PHP logic within curly braces to process rules, returning any non-null value.

```text
#Digit -> {
    var_dump($children);

    return new ExampleAstNode($children->getName());
}
  : <T_DIGIT> 
  ;
```

--------------------------------

### Define an Optional Rule

Source: https://phplrt.org/docs/parser/rules

Creates an Optional rule, which matches a sub-rule zero or one time. This is a shorthand for `new Repetition(rule, 0, 1)` and is generally faster.

```php
<?php

use Phplrt\Parser\Grammar\Optional;

//
// EBNF:
//  optional = some? ; 
//

$optional = new Optional(<ID_1>);

// Same as "new Repetition(<ID_1>, 0, 1)", but faster
```

--------------------------------

### Renaming Unknown Tokens with the 'unknown' Argument

Source: https://phplrt.org/docs/lexer/unknown-token

The `unknown` constructor argument provides a simple way to rename any unrecognized tokens to a specified string, such as 'WTF_IS_THAT'.

```php
$lexer = new Phplrt\Lexer\Lexer(
    tokens: ['T_DIGIT' => '\d+'],
    onUnknownToken: new \Phplrt\Lexer\Config\PassthroughHandler(),
    unknown: 'WTF_IS_THAT',
);

foreach ($lexer->lex('42 unknown 23') as $token) {
    echo $token->getName() . ' with value (' . $token->getValue() . ")\n";
}

// T_DIGIT with value (42)
// WTF_IS_THAT with value ( unknown )
// T_DIGIT with value (23)
// T_EOI with value (\0)
```

--------------------------------

### PP2 Grammar for Calculator Operations

Source: https://phplrt.org/docs/compiler/grammar

Defines tokens for digits and operators, and rules for arithmetic expressions including addition, division, subtraction, and multiplication. Use this to define a parser for arithmetic expressions.

```pp2
%skip  T_WHITESPACE     \s+

%token T_DIGIT          \-?\d+
%token T_PLUS           \+
%token T_MINUS          \-
%token T_DIV            /
%token T_MUL            \*

#Expression
  : Operation() 
  ;
    
Operation
  : <T_DIGIT> (
      Addition() | 
      Division() | 
      Subtraction() | 
      Multiplication()
    )? 
  ;

#Addition
  : ::T_PLUS:: Operation() 
  ;

#Division
  : ::T_DIV:: Operation() 
  ;

#Subtraction
  : ::T_MINUS:: Operation() 
  ;

#Multiplication
  : ::T_MUL:: Operation() 
  ;
```

--------------------------------

### Handle Syntax Errors

Source: https://phplrt.org/docs/parser

The parser throws an exception with location details when encountering invalid input.

```php
$result = $parser->parse('2 + + 2');
// Syntax error, unexpected "+" (T_PLUS)
//   1. | 2 + + 2
//      |     ^ in .../Parser/src/Exception/UnexpectedTokenException.php:37
```

--------------------------------

### phplrt Runtime and Compiler-Compiler Composer Requirements

Source: https://phplrt.org/docs/guide/introduction

Include these lines in your composer.json to add the phplrt runtime and compiler-compiler.

```json
{
    "require": {
        "phplrt/runtime": "^3.6"
    },
    "require-dev": {
        "phplrt/phplrt": "^3.6"
    }
}
```

--------------------------------

### Define Repetition Rules

Source: https://phplrt.org/docs/parser/rules

Creates Repetition rules for various quantifiers. These rules specify how many times a sub-rule must or can repeat.

```php
<?php

use Phplrt\Parser\Grammar\Repetition;

//
// EBNF:
//  repeat_zero_or_more = some* ; 
//
new Repetition(<ID_1>, 0, \INF); // repeat rule #1 from 0 to inf

//
// EBNF: 
//  repeat_one_or_more = some+ ; 
//
new Repetition(<ID_2>, 1, \INF); // repeat rule #2 from 1 to inf

//
// EBNF: 
//  repeat_1_2_or_3_times = some{1,3} ; 
//
new Repetition(<ID_3>, 1, 3); // repeat rule #3 from 1 to 3
```

--------------------------------

### AST for Simple Expression Parsing

Source: https://phplrt.org/docs/compiler/grammar

Represents the Abstract Syntax Tree generated from parsing the expression '4 + 8 - 15 * 16 / 23 + -42' using the defined PP2 grammar. This shows the hierarchical structure of the parsed expression.

```xml
<Ast>
  <Expression offset="0">
    <T_DIGIT offset="0">4</T_DIGIT>
    <Addition offset="2">
      <T_DIGIT offset="4">8</T_DIGIT>
      <Subtraction offset="6">
        <T_DIGIT offset="8">15</T_DIGIT>
        <Multiplication offset="11">
          <T_DIGIT offset="13">16</T_DIGIT>
          <Division offset="16">
            <T_DIGIT offset="18">23</T_DIGIT>
            <Addition offset="21">
              <T_DIGIT offset="23">-42</T_DIGIT>
            </Addition>
          </Division>
        </Multiplication>
      </Subtraction>
    </Addition>
  </Expression>
</Ast>
```

--------------------------------

### Define Addition Node Class

Source: https://phplrt.org/docs/guide/creating-project

Create an `Addition` class that implements `NodeInterface` to represent addition operations in the AST. It holds the left-hand side (`a`) and right-hand side (`b`) operands.

```php
namespace Calculator\Node;

use Phplrt\Contracts\Ast\NodeInterface;

final class Addition implements NodeInterface
{
    public Number $a;
    public object $b;

    /** @param Number|Addition $b */
    public function __construct(Number $a, object $b)
    {
        $this->a = $a;
        $this->b = $b;
    }

    public function getIterator(): \Traversable
    {
        return new \EmptyIterator();
    }
}
```

--------------------------------

### Define Class Reference Grammar

Source: https://phplrt.org/docs/examples/go-aop

Grammar rules for class definitions and instance filtering.

```text
ClassDefinition
  : NamespacePattern()
  ;

ClassInstanceOfDefinition
  : NamespacePattern() ::T_SUBNAMESPACE_SIGN::
  ;

ClassFilter
  : ClassInstanceOfDefinition()
  | ClassDefinition()
  ;
```

--------------------------------

### Define the math expression grammar

Source: https://phplrt.org/docs/examples/simple-math

Defines tokens and production rules for parsing simple math expressions.

```text
%token  T_NUMBER        \d+
%token  T_PLUS          \+
%token  T_MINUS         \-
%skip   T_WHITESPACE    \s+

%pragma root Expression

Expression
  : <T_NUMBER> Operation() (Expression() | <T_NUMBER>)
  ;

Operation
  : <T_PLUS>
  | <T_MINUS>
  ;
```

--------------------------------

### Define Whitespace and Block Comment Skipping

Source: https://phplrt.org/docs/examples/phpdoc-types

Defines rules to skip whitespace and block comments during parsing.

```grammar
%skip   T_WHITESPACE            \s+
```

```grammar
%skip   T_BLOCK_COMMENT         \h*/\*.*?\*/\h*
```

--------------------------------

### Define Name Part Rule

Source: https://phplrt.org/docs/examples/phpdoc-types

Defines 'NamePart' as a single T_NAME token.

```grammar
NamePart
  : <T_NAME>
  ;
```

--------------------------------

### Define Statement Rule

Source: https://phplrt.org/docs/examples/phpdoc-types

Defines the top-level 'Statement' rule as a 'BinaryStatement'.

```grammar
Statement
  : BinaryStatement()
  ;
```

--------------------------------

### Expected AST Output

Source: https://phplrt.org/docs/guide/creating-project

The expected output after parsing '2 + 2 + 4' shows a nested `Addition` node structure, with `Number` nodes representing the literal values.

```text
// Expected Output:

Calculator\Node\Addition {
  +a: Calculator\Node\Number {
    +value: 2
  }
  +b: Calculator\Node\Addition {
    +a: Calculator\Node\Number {
      +value: 2
    }
    +b: Calculator\Node\Number {
      +value: 4
    }
  }
}

```

--------------------------------

### Tokens Exclusion

Source: https://phplrt.org/docs/lexer

Explains how to configure the Lexer to skip specific tokens, such as whitespace, during the lexing process.

```APIDOC
## Tokens Exclusion

### Description
This section demonstrates how to exclude specific tokens from the lexer's output by providing a list of token names to skip.

### Usage Example
```php
$lexer = new Phplrt\Lexer\Lexer([
    'T_WHITESPACE'  => '\s+',
    'T_PLUS'        => '\+',
    'T_DIGIT'       => '\d+'
], skip: [ 'T_WHITESPACE' ]);

foreach ($lexer->lex('23 + 42') as $token) {
    echo $token . "\n";
}

// Expected output:
// "23" (T_DIGIT)
// "+" (T_PLUS)
// "42" (T_DIGIT)
// \0
```

### Method
`Phplrt\Lexer\Lexer::__construct(array $definitions, array $skip = [], array $except = [])`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
None

### Response
#### Success Response (200)
An iterator of `Phplrt\Contracts\Lexer\TokenInterface` objects, excluding the skipped tokens.

#### Response Example
```json
{
  "example": "Iterator<TokenInterface> (excluding skipped tokens)"
}
```
```

--------------------------------

### Define Number Node Class

Source: https://phplrt.org/docs/guide/creating-project

Create a `Number` class that implements `NodeInterface` to represent literal number nodes in the AST. It stores the integer value of the number.

```php
<?php

declare(strict_types=1);

namespace Calculator\Node;

use Phplrt\Contracts\Ast\NodeInterface;

final class Number implements NodeInterface
{
    public int $value;

    public function __construct(int $value)
    {
        $this->value = $value;
    }

    public function getIterator(): \Traversable
    {
        return new \EmptyIterator();
    }
}
```

--------------------------------

### Implementing a Custom Unknown Token Handler

Source: https://phplrt.org/docs/lexer/unknown-token

A custom handler can be implemented as an anonymous class to define specific logic for processing unknown tokens, such as identifying PHP language injections.

```php
use Phplrt\Contracts\Lexer\TokenInterface;
use Phplrt\Contracts\Source\ReadableInterface;
use Phplrt\Lexer\Config\HandlerInterface;

$lexer = new Phplrt\Lexer\Lexer(
    tokens: ['T_DIGIT' => '\d+'],
    onUnknownToken: new class implements HandlerInterface {
        public function handle(ReadableInterface $source, TokenInterface $token): ?TokenInterface
        {
            $content = $token->getValue();

            if (\str_starts_with($content, '<?php')) {
                return new \Phplrt\Lexer\Token\Composite('T_PHP_LANGUAGE_INJECTION', ...);
            }

            return $token;
        }
    }
);

foreach ($lexer->lex('...') as $token) { ... }
```

--------------------------------

### Lexing with Appended Token (Incorrect Order)

Source: https://phplrt.org/docs/lexer/modification

Demonstrates incorrect lexing behavior when a more specific token (T_FLOAT) is appended after a general token (T_INT), causing numbers to be tokenized as T_INT.

```php
foreach ($lexer->lex('42.0') as $i => $token) {
    echo $i . ' => ' . $token . "\n";
}

//
// Expected Output:
//
// 0 => "42" (T_INT)
// 1 => Error

 Syntax error, unrecognized "."
  1. | 42.0
     |   ^ in .../Lexer/src/Exception/UnrecognizedTokenException.php:40

```

--------------------------------

### Lexer with Token Exclusion

Source: https://phplrt.org/docs/lexer

Configure the Lexer to skip specific token types, such as whitespace, by providing a list of token names to the 'skip' parameter.

```php
$lexer = new Phplrt\Lexer\Lexer([
    'T_WHITESPACE'  => '\s+',
    'T_PLUS'        => '\+',
    'T_DIGIT'       => '\d+'
], skip: [ 'T_WHITESPACE' ]);

foreach ($lexer->lex('23 + 42') as $token) {
    echo $token . "\n";
}

//
// Expected output:
//
// > "23" (T_DIGIT)
// > "+" (T_PLUS)
// > "42" (T_DIGIT)
// > \0
//

```

--------------------------------

### Implement Custom AST Builder in PHPLRT

Source: https://phplrt.org/docs/parser/ast

Implement the `BuilderInterface` to define custom logic for constructing AST nodes based on parser states. Ensure your builder returns appropriate node objects for each state.

```php
use Phplrt\Parser\Builder\BuilderInterface;
use Phplrt\Parser\Context;

class MyBuilder implements BuilderInterface
{
    public function build(Context $ctx, $children)
    {
        switch ($ctx->getState()) {
            case 0: return new MyExampleNode($children);
            case 1: return new MyAnotherExampleNode($children);
        }

        return null;
    }
}
```

--------------------------------

### JSON5 Byte Order Mark (BOM) Skip Tokens

Source: https://phplrt.org/docs/examples/json5

Defines rules for skipping Byte Order Marks (BOM) for UTF-32 Big-Endian, UTF-32 Little-Endian, and UTF-16 Big-Endian encodings. These are typically at the beginning of a file.

```json5
%skip T_UTF32BE_BOM            ^\x00\x00\xFE\xFF
```

```json5
%skip T_UTF32LE_BOM            ^\xFE\xFF\x00\x00
```

```json5
%skip T_UTF16BE_BOM            ^\xFE\xFF
```

--------------------------------

### Define Access Modifier Grammar

Source: https://phplrt.org/docs/examples/go-aop

Grammar rules for object and static access modifiers.

```text
AccessType
  : ObjectAccess()
  | StaticAccess()
  ;

ObjectAccess
  : <T_OBJECT_ACCESS>
  ;

StaticAccess
  : <T_STATIC_ACCESS>
  ;
```

--------------------------------

### Implement Custom EOI Handler

Source: https://phplrt.org/docs/lexer/end-of-input

Defines a custom handler to perform specific actions or throw exceptions when the input stream ends.

```php
use Phplrt\Contracts\Lexer\TokenInterface;
use Phplrt\Contracts\Source\ReadableInterface;
use Phplrt\Lexer\Config\HandlerInterface;

$lexer = new Phplrt\Lexer\Lexer(
    tokens: ['T_DIGIT' => '\d+'],
    onEndOfInput: new class implements HandlerInterface {
        public function handle(ReadableInterface $source, TokenInterface $token): ?TokenInterface
        {
            throw new \RuntimeException('Input must not never ends!');
        }
    },
    eoi: 'CUSTOM_EOI_NAME',
);

foreach ($lexer->lex('42') as $token) {
    echo $token->getName() . "\n";
}

// T_DIGIT
// Uncaught RuntimeException: Input must not never ends!
```

--------------------------------

### Mathematical Expression Grammar

Source: https://phplrt.org/docs/examples/advanced-math

Define tokens for numbers, operators, and parentheses. This grammar handles operator precedence for arithmetic expressions.

```grammar
%token  T_FLOAT         \d+\.\d+
%token  T_INT           \d+

%token  T_PLUS          \+
%token  T_MINUS         \-
%token  T_MUL           \*
%token  T_DIV           /

%token  T_BRACE_OPEN    \(
%token  T_BRACE_CLOSE   \)

%skip   T_WHITESPACE    \s+

%pragma root Expression

Expression
  : BinaryExpression()
  ;

BinaryExpression
  : AdditiveExpression()
  ;

AdditiveExpression
  : (MultiplicativeExpression() (<T_PLUS>|<T_MINUS>))*
    MultiplicativeExpression()
  ;

MultiplicativeExpression
  : (UnaryExpression() (<T_DIV>|<T_MUL>))*
    UnaryExpression()
  ;

UnaryExpression
  : ::T_BRACE_OPEN:: Expression() ::T_BRACE_CLOSE::
  | Literal()
  ;

Literal
  : <T_FLOAT>
  | <T_INT>
  ;

```

--------------------------------

### phplrt Parser with Custom Lexer Composer Requirements

Source: https://phplrt.org/docs/guide/introduction

Use this composer.json configuration if you want to use phplrt solely as a parser with a custom lexer.

```json
{
    "require": {
        "phplrt/parser": "^3.6",
        "custom/lexer": "*"
    }
}
```

--------------------------------

### Define a simple math expression

Source: https://phplrt.org/docs/examples/simple-math

Represents a basic arithmetic string involving addition and subtraction.

```text
2 + 2 - 42
```

--------------------------------

### Define Template Parameter Rule

Source: https://phplrt.org/docs/examples/phpdoc-types

Defines a 'TemplateParameter' as a 'Statement'.

```grammar
TemplateParameter
  : Statement()
  ;
```

--------------------------------

### Define a Lexeme Rule

Source: https://phplrt.org/docs/parser/rules

Creates a Lexeme rule for a specific token name. This is used to define terminal rules in the grammar.

```php
<?php

use Phplrt\Parser\Grammar\Lexeme;

$kept = new Lexeme('T_NUMBER');
```

--------------------------------

### Define an Alternation Rule

Source: https://phplrt.org/docs/parser/rules

Creates an Alternation rule, which represents a choice between several grammar rules. It takes an array of rules, and one of them must match.

```php
<?php

use Phplrt\Parser\Grammar\Alternation;

//
// EBNF: 
//  choice = some1 | any2 ;
//
new Alternation([<ID_1>, <ID_1>]);
```

--------------------------------

### Set Root Rule

Source: https://phplrt.org/docs/examples/phpdoc-types

Sets the root rule for the grammar to 'Statement'.

```grammar
%pragma root Statement
```
Tokens:
Raw