### CharacterTypeLexer Implementation

Source: https://github.com/doctrine/lexer/blob/3.0.x/docs/en/simple-parser-example.rst

This PHP code defines a custom lexer by extending Doctrine's AbstractLexer. It specifies patterns for uppercase, lowercase, and numeric characters and implements logic to identify the type of each character.

```php
<?php

use Doctrine\Common\Lexer\AbstractLexer;

/**
 * @extends AbstractLexer<CharacterTypeLexer::T_*, string>
 */
class CharacterTypeLexer extends AbstractLexer
{
    const T_UPPER =  1;
    const T_LOWER =  2;
    const T_NUMBER = 3;

    protected function getCatchablePatterns(): array
    {
        return [
            '[a-bA-Z0-9]',
        ];
    }

    protected function getNonCatchablePatterns(): array
    {
        return [];
    }

    protected function getType(&$value): int
    {
        if (is_numeric($value)) {
            return self::T_NUMBER;
        }

        if (strtoupper($value) === $value) {
            return self::T_UPPER;
        }

        if (strtolower($value) === $value) {
            return self::T_LOWER;
        }
    }
}
```

--------------------------------

### Doctrine DQL Parser Initialization and AST Generation

Source: https://github.com/doctrine/lexer/blob/3.0.x/docs/en/dql-parser.rst

This code demonstrates how to initialize the Doctrine Parser with a DQL string and retrieve the Abstract Syntax Tree (AST). The parser uses the Lexer to tokenize the input and then constructs the AST based on the DQL grammar.

```php
<?php

use Doctrine\ORM\Query\Parser;
use Doctrine\ORM\Query\AST;

// Example DQL query
$dql = 'SELECT u FROM User u';

// Initialize the Parser with the DQL string
$parser = new Parser($dql);

// Get the Abstract Syntax Tree (AST)
$AST = $parser->getAST(); 

// The $AST variable will hold an AST node, e.g., AST\SelectStatement
var_dump($AST);

?>

```

--------------------------------

### Extracting Uppercase Characters with Custom Lexer

Source: https://github.com/doctrine/lexer/blob/3.0.x/docs/en/simple-parser-example.rst

This PHP code demonstrates how to use the custom CharacterTypeLexer to process a string and extract all uppercase characters. It initializes the lexer, iterates through the tokens, and collects characters identified as uppercase.

```php
<?php

class UpperCaseCharacterExtracter
{
    public function __construct(private CharacterTypeLexer $lexer)
    {
    }

    /** @return list<string> */
    public function getUpperCaseCharacters(string $string): array
    {
        $this->lexer->setInput($string);
        $this->lexer->moveNext();

        $upperCaseChars = [];
        while (true) {
            if (!$this->lexer->lookahead) {
                break;
            }

            $this->lexer->moveNext();

            if ($this->lexer->token->isA(CharacterTypeLexer::T_UPPER)) {
                $upperCaseChars[] = $this->lexer->token->value;
            }
        }

        return $upperCaseChars;
    }
}

$upperCaseCharacterExtractor = new UpperCaseCharacterExtracter(new CharacterTypeLexer());
$upperCaseCharacters = $upperCaseCharacterExtractor->getUpperCaseCharacters('1aBcdEfgHiJ12');

print_r($upperCaseCharacters);
```

--------------------------------

### Output of Uppercase Character Extraction

Source: https://github.com/doctrine/lexer/blob/3.0.x/docs/en/simple-parser-example.rst

This is the expected output when running the UpperCaseCharacterExtracter with the input string '1aBcdEfgHiJ12'. It shows an array containing only the uppercase characters found in the input.

```php
Array
(
    [0] => B
    [1] => E
    [2] => H
    [3] => J
)
```

--------------------------------

### Doctrine DQL Lexer Implementation

Source: https://github.com/doctrine/lexer/blob/3.0.x/docs/en/dql-parser.rst

This snippet defines a PHP class 'Lexer' that extends 'AbstractLexer' from the Doctrine Common library. It sets up various token constants (T_NONE, T_INTEGER, T_STRING, keywords like T_SELECT, T_FROM, etc.) and implements methods to define regular expressions for parsing DQL queries and identifying token types.

```php
<?php

use Doctrine\Common\Lexer\AbstractLexer;

class Lexer extends AbstractLexer
{
    // All tokens that are not valid identifiers must be < 100
    public const T_NONE              = 1;
    public const T_INTEGER           = 2;
    public const T_STRING            = 3;
    public const T_INPUT_PARAMETER   = 4;
    public const T_FLOAT             = 5;
    public const T_CLOSE_PARENTHESIS = 6;
    public const T_OPEN_PARENTHESIS  = 7;
    public const T_COMMA             = 8;
    public const T_DIVIDE            = 9;
    public const T_DOT               = 10;
    public const T_EQUALS            = 11;
    public const T_GREATER_THAN      = 12;
    public const T_LOWER_THAN        = 13;
    public const T_MINUS             = 14;
    public const T_MULTIPLY          = 15;
    public const T_NEGATE            = 16;
    public const T_PLUS              = 17;
    public const T_OPEN_CURLY_BRACE  = 18;
    public const T_CLOSE_CURLY_BRACE = 19;

    // All tokens that are identifiers or keywords that could be considered as identifiers should be >= 100
    public const T_ALIASED_NAME         = 100;
    public const T_FULLY_QUALIFIED_NAME = 101;
    public const T_IDENTIFIER           = 102;

    // All keyword tokens should be >= 200
    public const T_ALL      = 200;
    public const T_AND      = 201;
    public const T_ANY      = 202;
    public const T_AS       = 203;
    public const T_ASC      = 204;
    public const T_AVG      = 205;
    public const T_BETWEEN  = 206;
    public const T_BOTH     = 207;
    public const T_BY       = 208;
    public const T_CASE     = 209;
    public const T_COALESCE = 210;
    public const T_COUNT    = 211;
    public const T_DELETE   = 212;
    public const T_DESC     = 213;
    public const T_DISTINCT = 214;
    public const T_ELSE     = 215;
    public const T_EMPTY    = 216;
    public const T_END      = 217;
    public const T_ESCAPE   = 218;
    public const T_EXISTS   = 219;
    public const T_FALSE    = 220;
    public const T_FROM     = 221;
    public const T_GROUP    = 222;
    public const T_HAVING   = 223;
    public const T_HIDDEN   = 224;
    public const T_IN       = 225;
    public const T_INDEX    = 226;
    public const T_INNER    = 227;
    public const T_INSTANCE = 228;
    public const T_IS       = 229;
    public const T_JOIN     = 230;
    public const T_LEADING  = 231;
    public const T_LEFT     = 232;
    public const T_LIKE     = 233;
    public const T_MAX      = 234;
    public const T_MEMBER   = 235;
    public const T_MIN      = 236;
    public const T_NEW      = 237;
    public const T_NOT      = 238;
    public const T_NULL     = 239;
    public const T_NULLIF   = 240;
    public const T_OF       = 241;
    public const T_OR       = 242;
    public const T_ORDER    = 243;
    public const T_OUTER    = 244;
    public const T_PARTIAL  = 245;
    public const T_SELECT   = 246;
    public const T_SET      = 247;
    public const T_SOME     = 248;
    public const T_SUM      = 249;
    public const T_THEN     = 250;
    public const T_TRAILING = 251;
    public const T_TRUE     = 252;
    public const T_UPDATE   = 253;
    public const T_WHEN     = 254;
    public const T_WHERE    = 255;
    public const T_WITH     = 256;

    /**
     * Creates a new query scanner object.
     *
     * @param string $input A query string.
     */
    public function __construct(string $input)
    {
        $this->setInput($input);
    }

    /**
     * {@inheritdoc}
     */
    protected function getCatchablePatterns(): array
    {
        return [
            '[a-z_][a-z0-9_]*\:[a-z_][a-z0-9_]*(?:\\[a-z_][a-z0-9_]*)*', // aliased name
            '[a-z_\\][a-z0-9_]*(?:\\[a-z_][a-z0-9_]*)*', // identifier or qualified name
            '(?:[0-9]+(?:[\.][0-9]+)*)(?:e[+-]?[0-9]+)?', // numbers
            "'(?:[^']|'')*'", // quoted strings
            '\?[0-9]*|:[a-z_][a-z0-9_]*', // parameters
        ];
    }

    /**
     * {@inheritdoc}
     */
    protected function getNonCatchablePatterns(): array
    {
        return ['\s+', '(.)'];
    }

    /**
     * {@inheritdoc}
     */
    protected function getType(&$value): int
    {
        $type = self::T_NONE;

        switch (true) {
            // Recognize numeric values
            case (is_numeric($value)):

```

--------------------------------

### Doctrine Parser QueryLanguage Method

Source: https://github.com/doctrine/lexer/blob/3.0.x/docs/en/dql-parser.rst

This snippet shows the `QueryLanguage` method within the Doctrine Parser, which is responsible for parsing the main DQL query structure. It delegates to specific methods like `SelectStatement`, `UpdateStatement`, or `DeleteStatement` based on the initial token.

```php
<?php

// ... inside Doctrine\ORM\Query\Parser class

public function QueryLanguage(): AST\SelectStatement|AST\UpdateStatement|AST\DeleteStatement
{
    $this->lexer->moveNext();

    switch ($this->lexer->lookahead->type) {
        case Lexer::T_SELECT:
            $statement = $this->SelectStatement();
            break;
        case Lexer::T_UPDATE:
            $statement = $this->UpdateStatement();
            break;
        case Lexer::T_DELETE:
            $statement = $this->DeleteStatement();
            break;
        default:
            $this->syntaxError('SELECT, UPDATE or DELETE');
    }

    // Check for end of string
    if ($this->lexer->lookahead !== null) {
        $this->syntaxError('end of string');
    }

    return $statement;
}

// ...

?>
```

--------------------------------

### Doctrine Lexer Tokenization Logic

Source: https://github.com/doctrine/lexer/blob/3.0.x/docs/en/dql-parser.rst

This code snippet demonstrates the core logic within the Doctrine Lexer for identifying and categorizing different types of tokens in a DQL query. It handles numbers, quoted strings, identifiers, input parameters, and various symbols.

```php
<?php

// ... inside Doctrine\ORM\Query\Lexer class

switch (true) {
    // Recognize numbers
    case (strpos($value, '.') !== false || stripos($value, 'e') !== false):
        return self::T_FLOAT;
    default:
        return self::T_INTEGER;
}

// Recognize quoted strings
case ($value[0] === "'"):
    $value = str_replace("''", "'", substr($value, 1, strlen($value) - 2));
    return self::T_STRING;

// Recognize identifiers, aliased or qualified names
case (ctype_alpha($value[0]) || $value[0] === '_' || $value[0] === '\\'):
    $name = 'Doctrine\ORM\Query\Lexer::T_' . strtoupper($value);
    if (defined($name)) {
        $type = constant($name);
        if ($type > 100) {
            return $type;
        }
    }
    if (strpos($value, ':') !== false) {
        return self::T_ALIASED_NAME;
    }
    if (strpos($value, '\\') !== false) {
        return self::T_FULLY_QUALIFIED_NAME;
    }
    return self::T_IDENTIFIER;

// Recognize input parameters
case ($value[0] === '?' || $value[0] === ':'):
    return self::T_INPUT_PARAMETER;

// Recognize symbols
case ($value === '.'):
    return self::T_DOT;
case ($value === ','):
    return self::T_COMMA;
case ($value === '('):
    return self::T_OPEN_PARENTHESIS;
case ($value === ')'):
    return self::T_CLOSE_PARENTHESIS;
case ($value === '='):
    return self::T_EQUALS;
case ($value === '>'):
    return self::T_GREATER_THAN;
case ($value === '<'):
    return self::T_LOWER_THAN;
case ($value === '+'):
    return self::T_PLUS;
case ($value === '-'):
    return self::T_MINUS;
case ($value === '*'):
    return self::T_MULTIPLY;
case ($value === '/'):
    return self::T_DIVIDE;
case ($value === '!'):
    return self::T_NEGATE;
case ($value === '{'):
    return self::T_OPEN_CURLY_BRACE;
case ($value === '}'):
    return self::T_CLOSE_CURLY_BRACE;

// Default
default:
    // Do nothing
}

return $type;

?>
```

--------------------------------

### Doctrine Lexer API

Source: https://github.com/doctrine/lexer/blob/3.0.x/docs/en/index.rst

The low-level API provided by the Doctrine Lexer for tokenizing and navigating input strings. It includes methods for setting input, resetting the lexer, peeking at tokens, and moving through the input.

```APIDOC
setInput(string $input)
  - Sets the input data to be tokenized. Resets and tokenizes the new input.

reset()
  - Resets the lexer.

resetPeek()
  - Resets the peek pointer to 0.

resetPosition(int $position = 0)
  - Resets the lexer position on the input to the given position.

isNextToken(mixed $token)
  - Checks whether a given token matches the current lookahead.

isNextTokenAny(array $tokens)
  - Checks whether any of the given tokens matches the current lookahead.

moveNext()
  - Moves to the next token in the input string.

skipUntil(int $type)
  - Tells the lexer to skip input tokens until it sees a token with the given value.

isA(string $value, mixed $token)
  - Checks if given value is identical to the given token.

peek()
  - Moves the lookahead token forward.

glimpse()
  - Peeks at the next token, returns it and immediately resets the peek.
```

--------------------------------

### Abstract Lexer Methods

Source: https://github.com/doctrine/lexer/blob/3.0.x/docs/en/index.rst

Abstract methods that must be implemented when extending Doctrine\Common\Lexer\AbstractLexer. These methods define the lexical patterns and token processing logic.

```php
/**
 * Lexical catchable patterns.
 *
 * @return string[]
 */
abstract protected function getCatchablePatterns();

/**
 * Lexical non-catchable patterns.
 *
 * @return string[]
 */
abstract protected function getNonCatchablePatterns();

/** Retrieve token type. Also processes the token value if necessary. */
abstract protected function getType(string &$value): int;
```

--------------------------------

### Doctrine Lexer 2.0.0 Upgrade Notes

Source: https://github.com/doctrine/lexer/blob/3.0.x/UPGRADE.md

In Doctrine Lexer 2.0.0, `AbstractLexer::glimpse()` and `AbstractLexer::peek()` now return `Doctrine\Common\Lexer\Token` instances. Using these instances as arrays is deprecated in favor of accessing properties directly. The use of `count()` on these instances is also deprecated with no direct replacement.

```php
namespace Doctrine\Common\Lexer;

// Deprecated usage:
// $token = $lexer->glimpse();
// $value = $token['value']; // Deprecated
// $count = count($token); // Deprecated

// Recommended usage:
$token = $lexer->glimpse();
$value = $token->getValue(); // Recommended
```

--------------------------------

### Doctrine Lexer 3.0.0 Upgrade Notes

Source: https://github.com/doctrine/lexer/blob/3.0.x/UPGRADE.md

Upgrade to Doctrine Lexer 3.0.0 involves changes to the `Doctrine\Common\Lexer\Token` class, which no longer implements `ArrayAccess`. Additionally, parameter type declarations have been added to `Doctrine\Common\Lexer\AbstractLexer` and `Doctrine\Common\Lexer\Token`. Developers should ensure their lexers include corresponding parameter and return type declarations.

```php
namespace Doctrine\Common\Lexer;

// AbstractLexer and Token classes now have parameter type declarations.
// Ensure your lexer implementations adhere to these or add return type declarations based on phpdoc @return tags.
```

--------------------------------

### Doctrine Lexer Deprecation Practices

Source: https://github.com/doctrine/lexer/blob/3.0.x/UPGRADE.md

Doctrine Lexer utilizes two primary mechanisms for deprecation awareness: static analysis via `@deprecated` docblocks and a runtime deprecation API for low-overhead detection. This helps developers identify and address deprecated code usage.

```php
/**
 * @deprecated Use new_method() instead.
 */
public function old_method() {
    // ...
}
```

```php
use Doctrine\Deprecations\Deprecation;

// ...

Deprecation::trigger('doctrine/lexer', '2.0.0', 'Using count() on Token is deprecated.');
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.