RE/flex

https://www.genivia.com/doc/reflex/html/
Admin
RE/flex is a high-performance C++ regular expression library and lexical analyzer generator that...
Tokens:84,078
Snippets:924
Trust Score:9.5
Update:3 months ago
Show doc for...
Tokens:
Raw
### Declare and Use Start Conditions

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of declaring inclusive and exclusive start conditions and applying them to rules.

```text
%s A
%x X
%%
<A,X>pattern1 action1 // rule for states A and X
<A>pattern2 action2 // rule for state A
<X>pattern3 action3 // rule for state X
<*>pattern4 action4 // rule for states INITIAL, A and X
pattern5 action5 // rule for states INITIAL and A
<<EOF>> action6 // rule for states INITIAL, A and X
%%
```

--------------------------------

### Nest Start Condition Scopes

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of nesting start condition scopes to combine active states.

```text
%s A
%x X
%%
pattern action // rule for states INITIAL and A
<A>{
pattern action // rule for state A
<X>{
pattern action // rule for states A and X
}
}
<X>{
pattern action // rule for state X
<A>{
pattern action // rule for states A and X
}
}
%%
```

--------------------------------

### Flex-compatible BEGIN macro

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Sets a start condition.

```cpp
#define BEGIN YY_SCANNER_DOT_ start_ =
```

--------------------------------

### Define YYSTATE macro

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Get the current start condition.

```cpp
#define YYSTATE YY_SCANNER_DOT_ start()
```

--------------------------------

### Switch Start Condition States

Source: https://www.genivia.com/doc/reflex/html/index.html

Using start() to transition between states, such as entering a comment block.

```text
%x COMMENT
%%
"/*" start(COMMENT); // with −−flex use: BEGIN COMMENT;
.|\n echo(); // with −−flex use: ECHO;
<COMMENT>[^*]* // eat anything that is not a '*'
<COMMENT>"*"+[^*/]* // eat '*'s not followed by a '/'
<COMMENT>"*"+"/" start(INITIAL);
<COMMENT><<EOF>> std::cerr << "EOF in comment\n"; return 1;
%%
```

--------------------------------

### Initializing the Bison Parser

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of how to instantiate the lexer and parser in a C++ main program.

```cpp
FILE *f = fopen(filename, "r");
if (f != NULL)
{
yy::Lexer lexer(f); // read file, decode UTF-8/16/32 format
lexer.filename = filename; // the filename to display with error locations
yy::parser parser(lexer);
if (parser.parse() != 0)
... // error
```

--------------------------------

### Use Start Condition Scopes

Source: https://www.genivia.com/doc/reflex/html/index.html

Grouping multiple rules under a single start condition scope.

```text
<COMMENT>{
[^*]* // eat anything that is not a '*'
"*"+[^*/]* // eat '*'s not followed by a '/'
"*"+"/" start(INITIAL);
<<EOF>> std::cerr << "EOF in comment\n"; return 1;
}
```

--------------------------------

### Configure Bison Parser for Combined Bridge and Locations

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of a Bison grammar file setup combining both --bison-bridge and --bison-locations options.

```c
/* yacc grammar (.y file) for C or C++ */
%{
#include "lex.yy.h"
void yyerror(YYLTYPE*, yyscan_t, const char*);
#define YYPARSE_PARAM scanner
#define YYLEX_PARAM scanner
%}
%locations
%pure-parser
%lex-param { void *scanner }
%parse-param { void *scanner }
%union { // YYSTYPE yylval is a union:
int num; // yylval.num
char* str; // yylval.str
}
%token <num> CONST_NUMBER
%token <str> CONST_STRING
%%
... // grammar rules
%%
void yyerror(YYLTYPE *yylloc, yyscan_t scanner, const char *msg)
{
fprintf(stderr, "%s at %d,%d to line %d,%d\n",
msg,
yylloc->first_line,
yylloc->first_column,
yylloc->last_line,
yylloc->last_column);
}
```

--------------------------------

### Define Start Condition Rules

Source: https://www.genivia.com/doc/reflex/html/index.html

Basic syntax for prefixing patterns with start condition states.

```text
<A,B>pattern1 action1
<A>pattern2 action2
<B>pattern3 action3
```

--------------------------------

### Manage Start Condition Stack

Source: https://www.genivia.com/doc/reflex/html/index.html

Using push_state and pop_state to handle nested or temporary state transitions.

```text
%x COMMENT
%%
"/*" push_state(COMMENT);
.|\n echo();
<COMMENT>[^*]* // eat anything that is not a '*'
<COMMENT>"*"+[^*/]* // eat '*'s not followed by a '/'
<COMMENT>"*"+"/" pop_state();
<COMMENT><<EOF>> std::cerr << "EOF in comment\n"; return 1;
%%
```

--------------------------------

### Switching Input Sources

Source: https://www.genivia.com/doc/reflex/html/index.html

Examples of switching input sources using files, streams, strings, and memory segments.

```cpp
// read from a file, this also decodes UTF-8/16/32 encodings automatically
FILE *fd = fopen("cow.txt", "r");
if (fd == NULL)
... // error, bail out
lexer.in(fd);
lexer.lex();
// read from a stream (ASCII or UTF-8)
std::istream i = std::ifstream("file", std::ios::in);
lexer.in(i);
lexer.lex();
// read from a string (0-terminated, ASCII or UTF-8)
lexer.in("How now brown cow.");
lexer.lex();
// read from a memory segment (raw bytes, ASCII, or UTF-8)
const char *ptr = ...; // points to segment
size_t len = ...; // length of the memory segment
lexer.in(ptr, len);
lexer.lex();
// read from a wide string, 0-terminated, encoding it to UTF-8 for matching
lexer.in(L"How now brown cow.");
lexer.lex();
```

--------------------------------

### Flex-compatible state stack macros

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Manages the start condition state stack.

```cpp
#define YY_NULL (0)
```

```cpp
#define yy_pop_state | ( | | ) |  YY_SCANNER_DOT_ pop_state()
```

```cpp
#define yy_push_state | ( |  | n| ) |  YY_SCANNER_DOT_ push_state(n)
```

--------------------------------

### Configure Bison Pure Parser for Bison-bridge

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of a Bison grammar file setup for use with the --bison-bridge option.

```c
/* yacc grammar (.y file) for C or C++ */
%{
#include "lex.yy.h"
#define YYPARSE_PARAM scanner
#define YYLEX_PARAM scanner
%}
%pure-parser
%%
... // grammar rules
%%
```

--------------------------------

### Configure Bison Parser for Bison-locations

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of a Bison grammar file setup for use with the --bison-locations option, including error reporting with line and column numbers.

```c
/* yacc grammar (.y file) for C or C++ */
%{
#include "lex.yy.h"
void yyerror(const char*);
%}
%locations
%union { // YYSTYPE yylval is a union:
int num; // type of yylval.num is int
char* str; // type of yylval.str is char*
}
%{
/* reflex option −−bison-locations makes yylex() take yylval and yylloc */
extern int yylex(YYSTYPE*, YYLTYPE*);
#define YYLEX_PARAM &yylval, &yylloc
%}
/* add &yylval and &yyloc parameters to yylex() with a trick: use YYLEX_PARAM */
%lex-param { void *YYLEX_PARAM }
%token <num> CONST_NUMBER
%token <str> CONST_STRING
%%
... // grammar rules
%%
void yyerror(const char *msg)
{
fprintf(stderr, "%s at %d,%d to line %d,%d\n",
msg,
yylloc.first_line,
yylloc.first_column,
yylloc.last_line,
yylloc.last_column);
}
```

--------------------------------

### Defining Start Condition Scopes

Source: https://www.genivia.com/doc/reflex/html/index.html

Demonstrates the use of start condition scopes and the ^ operator to remove states within nested and global contexts.

```reflex
%s A B C
%%
<B,C>{
pattern action // rule for states B and C
<A,^B>{
pattern action // rule for states A and C
}
}
<*,^A,^C>pattern action // rule for states INITIAL and B (all states except A and C)
%%
```

--------------------------------

### Debugging Example with Reflex Macros

Source: https://www.genivia.com/doc/reflex/html/debug_8h.html

Demonstrates the use of DBGLOG, DBGLOGA, DBGCHK, and DBGSTR macros within a C++ program.

```cpp
#include <reflex/debug.h>
int main(int argc, char *argv[])
{
FILE *fd;
DBGLOG("Program start");
if ((fd = fopen("foo.bar", "r")) == NULL)
{
DBGLOG("Error %d: %s ", errno, DBGSTR(strerror(errno)));
for (int i = 1; i < argc; ++i)
DBGLOGA(" %s", argv[1]);
}
else
{
DBGCHK(fd != NULL);
// OK, so go ahead to read foo.bar ...
// ...
fclose(fd);
}
DBGLOG("Program end");
}
```

--------------------------------

### Switching Input Sources for Standard Scanners

Source: https://www.genivia.com/doc/reflex/html/index.html

Examples of configuring input sources for standard lexers using files, streams, strings, and memory buffers.

```cpp
// read from a file, this also decodes UTF-8/16/32 encodings automatically
FILE *fd = fopen("cow.txt", "r");
if (fd == NULL)
... // error, bail out
yyin = fd;
yylex();
// read from a stream (ASCII or UTF-8)
std::istream i = std::ifstream("file", std::ios::in);
yyin = &i;
yylex();
// read from a string (0-terminated, ASCII or UTF-8)
yy_delete_buffer(YY_CURRENT_BUFFER);
yy_scan_string("How now brown cow."); // new buffer to scan a string
// yyin = "How now brown cow."; // alternative, does not create a new buffer
yylex();
// read from a memory segment (raw bytes, ASCII, or UTF-8)
const char *ptr = ...; // points to memory segment
size_t len = ...; // length of the memory segment
yy_delete_buffer(YY_CURRENT_BUFFER);
yy_scan_bytes(ptr, len); // new buffer to scan memory
yylex();
// read from a wide string, 0-terminated, encoding it to UTF-8 for matching
yy_delete_buffer(YY_CURRENT_BUFFER);
yy_scan_wstring(L"How now brown cow."); // new buffer to scan a wide string
// yyin = L"How now brown cow."; // alternative, does not create a new buffer
yylex();
```

```cpp
// read a 0-terminated buffer in place, buffer content is changed!!
char *base = ...; // points to 0-terminated buffer
size_t size = ...; // length of the buffer including two final \0 bytes
yy_delete_buffer(YY_CURRENT_BUFFER);
yy_scan_buffer(base, size);
yylex();
```

--------------------------------

### State and Matcher Control Macros

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Macros for managing start condition states and manipulating matched text.

```cpp
#define yy_push_state(n) YY_SCANNER_DOT_ push_state(n)
```

```cpp
#define yy_pop_state() YY_SCANNER_DOT_ pop_state()
```

```cpp
#define yy_top_state() YY_SCANNER_DOT_ top_state()
```

```cpp
#define yymore() YY_SCANNER_DOT_ matcher().more()
```

```cpp
#define yyless(n) YY_SCANNER_DOT_ matcher().less(n)
```

```cpp
#define yy_flex_debug YY_SCANNER_DOT_ debug_;
```

--------------------------------

### reflex::timer_start

Source: https://www.genivia.com/doc/reflex/html/timer_8h.html

Starts the timer by initializing the provided timer_type object.

```APIDOC
## void reflex::timer_start(timer_type &t)

### Description
Starts the timer.

### Parameters
- **t** (timer_type &) - The timer object to start.
```

--------------------------------

### Declare start condition states

Source: https://www.genivia.com/doc/reflex/html/index.html

Define inclusive and exclusive start condition states using %state and %xstate.

```reflex
%s INCLUSIVE
%x EXCLUSIVE
```

--------------------------------

### Batch Input Scanner Configuration

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of specifying a buffer size for the batch scanner option.

```text
--batch=1024
```

--------------------------------

### Define yylook macro

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Get the next action index.

```cpp
#define yylook | ( | | ) |  YY_SCANNER_DOT_ matcher().scan()
```

--------------------------------

### State Management

Source: https://www.genivia.com/doc/reflex/html/classreflex_1_1_flex_lexer.html

Methods for managing lexer start condition states.

```APIDOC
## AbstractLexer & start(int state)

### Description
Transition to the given start condition state.

## int start() const

### Description
Returns the current start condition state.

## void push_state(int state)

### Description
Push the current start condition state on the stack and transition to the given start condition state.

## void pop_state()

### Description
Pop the stack start condition state and transition to that state.
```

--------------------------------

### Match Property Output

Source: https://www.genivia.com/doc/reflex/html/index.html

The console output generated by the match property example.

```text
accept: 1 text: How size: 3 line: 1 column: 0 first: 0 last: 3
accept: 1 text: now size: 3 line: 1 column: 4 first: 4 last: 7
accept: 1 text: brown size: 5 line: 1 column: 8 first: 8 last: 13
accept: 1 text: cow size: 3 line: 1 column: 14 first: 14 last: 17
```

--------------------------------

### POSIX Matcher Example

Source: https://www.genivia.com/doc/reflex/html/index.html

Demonstrates the default POSIX matcher behavior where the longest match is selected.

```bash
reflex -m reflex −−main lexer.l
c++ -o lexer lex.yy.cpp -lreflex
echo "integer" | ./lexer
=> identifier
```

--------------------------------

### Basic Indent/Nodent/Dedent Rules

Source: https://www.genivia.com/doc/reflex/html/index.html

Demonstrates basic indentation tracking using \i and \j anchors with line start anchors.

```RE/flex
^\h+ out() << "| "; // nodent: text is aligned to current indent
^\h+\i out() << "> "; // indent: matched and added with \i
^\h*\j out() << "< "; // dedent: matched with \j
\j out() << "< "; // dedent: for each extra level dedented
.|\n echo();
```

--------------------------------

### Lexer State Management

Source: https://www.genivia.com/doc/reflex/html/classreflex_1_1_abstract_lexer.html

Methods for managing the lexer's start condition states and stack.

```APIDOC
## AbstractLexer State Methods

### start(int state)
Transition to the given start condition state.

### start() const
Returns the current start condition state.

### push_state(int state)
Push the current start condition state on the stack and transition to the given start condition state.

### pop_state()
Pop the stack start condition state and transition to that state.

### top_state() const
Returns the stack top start condition state or 0 (INITIAL) if the stack is empty.

### states_empty() const
Returns true if the condition state stack is empty.
```

--------------------------------

### Define lexer rules for line input

Source: https://www.genivia.com/doc/reflex/html/index.html

Example rules for matching lines when using interactive input.

```text
%%
.+ echo(); // ECHO the entire line
\n echo(); // ECHO end of the line
%%
```

--------------------------------

### RE/flex Performance Report Output

Source: https://www.genivia.com/doc/reflex/html/index.html

Example output generated by the -p option showing rule match counts, byte counts, and execution times.

```text
reflex x.y.z json.l performance report:
  INITIAL rules matched:
    rule at line 51 accepted 58 times matching 255 bytes total in 0.009 ms
    rule at line 52 accepted 58 times matching 58 bytes total in 0.824 ms
    rule at line 53 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 54 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 55 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 56 accepted 5 times matching 23 bytes total in 0.007 ms
    rule at line 57 accepted 38 times matching 38 bytes total in 0.006 ms
    rule at line 72 accepted 0 times matching 0 bytes total in 0 ms
    default rule accepted 0 times
  STRING rules matched:
    rule at line 60 accepted 38 times matching 38 bytes total in 0.021 ms
    rule at line 61 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 62 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 63 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 64 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 65 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 66 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 67 accepted 0 times matching 0 bytes total in 0 ms
    rule at line 68 accepted 314 times matching 314 bytes total in 0.04 ms
    rule at line 69 accepted 8 times matching 25 bytes total in 0.006 ms
    rule at line 72 accepted 0 times matching 0 bytes total in 0 ms
    default rule accepted 0 times
  WARNING: execution times are relative:
    1) includes caller's execution time between matches when yylex() returns
    2) perf-report instrumentation adds overhead and increases execution times
```

--------------------------------

### Define Lexer Patterns and Actions

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of a lexer specification defining patterns for keywords, operators, identifiers, and constants.

```lex
%%
/* PATTERN */ /* ACTION */
"if" return KEYWORD_IF;
"*" return OP_MUL;
"=" return OP_ASSIGN;
[a-zA-Z_][a-zA-Z0-9_]* return ASCII_IDENTIFIER;
[0-9]+ return CONST_NUMBER;
\"([^\\\"]|\\\")*\" return CONST_STRING;
[ \t\r\n] /* no action: ignore all white space */
%%
```

--------------------------------

### Printing Match Properties

Source: https://www.genivia.com/doc/reflex/html/index.html

Example usage of BoostMatcher to iterate through matches and print metadata including accept index, text, size, line number, column, and byte offsets.

```cpp
#include <reflex/boostmatcher.h> // reflex::BoostMatcher, reflex::Input, boost::regex
reflex::BoostMatcher matcher("\\w+", "How now brown cow.");
while (matcher.find() != 0)
std::cout <<
"accept: " << matcher.accept() <<
"text: " << matcher.text() <<
"size: " << matcher.size() <<
"line: " << matcher.lineno() <<
"column: " << matcher.columno() <<
"first: " << matcher.first() <<
"last: " << matcher.last() << std::endl;
```

--------------------------------

### C++ Class Inheritance Example

Source: https://www.genivia.com/doc/reflex/html/graph_legend.html

A collection of class definitions demonstrating public, protected, and private inheritance, as well as template usage and containment for graph generation.

```cpp
1 /*! Invisible class because of truncation */
2 class Invisible { };
3
4 /*! Truncated class, inheritance relation is hidden */
5 class Truncated : public Invisible { };
6
7 /* Class not documented with doxygen comments */
8 class Undocumented { };
9
10 /*! Class that is inherited using public inheritance */
11 class PublicBase : public Truncated { };
12
13 /*! A template class */
14 template<class T> class Templ { };
15
16 /*! Class that is inherited using protected inheritance */
17 class ProtectedBase { };
18
19 /*! Class that is inherited using private inheritance */
20 class PrivateBase { };
21
22 /*! Class that is used by the Inherited class */
23 class Used { };
24
25 /*! Super class that inherits a number of other classes */
26 class Inherited : public PublicBase,
27 protected ProtectedBase,
28 private PrivateBase,
29 public Undocumented,
30 public Templ<int>
31 {
32 private:
33 Used *m_usedClass;
34 };
```

--------------------------------

### Initialize and use a reflex::Matcher

Source: https://www.genivia.com/doc/reflex/html/index.html

Demonstrates initializing a matcher with a string or file input and iterating through matches.

```cpp
#include <reflex/pcre2matcher.h> // reflex::PCRE2Matcher, reflex::Input
// set the input source to a string (or a stream or a FILE*)
reflex::Input source = "How now brown cow.";
reflex::PCRE2Matcher matcher("\\w+", source);
while (matcher.find() != 0)
std::cout << "Found " << matcher.text() << std::endl;
// use the same matcher with a new source (an Input object):
std::ifstream ifs("cows.txt", std::ios::in | std::ios::binary);
source = ifs; // Input source is reassignable
matcher.input(source); // can use ifs as parameter also
while (matcher.find() != 0)
std::cout << "Found " << matcher.text() << std::endl;
ifs.close();
```

--------------------------------

### Configure Lexer Matcher with Initial Code

Source: https://www.genivia.com/doc/reflex/html/index.html

Use %class and %init blocks to manage lexer state and configure the matcher behavior.

```reflex
%class{
bool init_matcher;
}
%init{
init_matcher = true;
}
%%
%{
if (init_matcher) // init the new matcher?
{
init_matcher = false; // init only once
if (BUFFER_ALL) // buffer all input at once?
matcher().buffer(); // same as %option batch
else if (INTERACTIVE) // console-based (TTY) "interactive" input?
matcher().interactive(); // same as %option interactive 
matcher().tabs(4); // same as %option tabs=4
}
%}
```

--------------------------------

### Switching Input Sources with Matchers

Source: https://www.genivia.com/doc/reflex/html/index.html

Demonstrates switching to a new input source and restoring the original one using RE/flex matcher methods.

```cpp
... // scanning etc.
Matcher *oldmatcher = matcher();
reflex::Input input(...);
Matcher *newmatcher = new_matcher(input);
matcher(newmatcher);
... // scan the new input
del_matcher(newmatcher);
matcher(oldmatcher);
... // continue scanning the old input
```

--------------------------------

### Switch input sources

Source: https://www.genivia.com/doc/reflex/html/classreflex_1_1_input.html

Demonstrates switching the input source of a reflex::Input object during runtime.

```cpp
reflex::Input input = "Hello";
std::string message;
char c;
while (input.get(&c, 1))
message.append(c);
input = L" world! To ∞ and beyond."; // switch input to a wide string
while (input.get(&c, 1))
message.append(c);
```

--------------------------------

### Compile an application with RE/flex and Boost

Source: https://www.genivia.com/doc/reflex/html/index.html

Link the application against the libreflex and boost_regex libraries during compilation.

```bash
c++ myapp.cpp -lreflex -lboost_regex
```

--------------------------------

### Define yytext macro

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Get the matched text.

```cpp
#define yytext const_cast<char*>(YY_SCANNER_DOT_ YYText())
```

--------------------------------

### Initialize a BoostMatcher engine

Source: https://www.genivia.com/doc/reflex/html/index.html

Create a BoostMatcher instance for Perl-style matching using a regex object or string and an input source.

```cpp
#include <reflex/boostmatcher.h>
reflex::BoostMatcher matcher( boost::regex or string, reflex::Input [, "options"] )
```

--------------------------------

### Define yylineno macro

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Get the current line number.

```cpp
#define yylineno static_cast<int>(YY_SCANNER_DOT_ matcher().lineno())
```

--------------------------------

### Define yyleng macro

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Get the length of the matched text.

```cpp
#define yyleng static_cast<yy_size_t>(YY_SCANNER_DOT_ YYLeng())
```

--------------------------------

### Configure Lexer with file input and custom output stream

Source: https://www.genivia.com/doc/reflex/html/index.html

Demonstrates instantiating the Lexer class to read from a file descriptor and write to a specific output file. This approach supports automatic character encoding decoding.

```cpp
int main(int argc, char **argv)
{
FILE *fd = stdin;
if (argc > 1 && (fd = fopen(argv[1], "r")) == NULL)
exit(EXIT_FAILURE);
std::ofstream of("output.txt", std::ofstream::out);
if (!of)
exit(EXIT_FAILURE);
Lexer(fd, of).lex();
of.close();
if (fd != stdin)
fclose(fd);
return 0;
}
```

--------------------------------

### Capture group output

Source: https://www.genivia.com/doc/reflex/html/index.html

The expected output from the PCRE2Matcher capture example.

```text
name: cow, number: 123
```

--------------------------------

### Initialize a RE/flex Matcher

Source: https://www.genivia.com/doc/reflex/html/index.html

Basic initialization of a matcher with a pattern and input source.

```cpp
reflex::Matcher matcher(pattern, input);
```

--------------------------------

### Switching Input Sources with --flex Option

Source: https://www.genivia.com/doc/reflex/html/index.html

Shows the equivalent input switching logic when using the --flex compatibility option.

```cpp
... // scanning etc.
YY_BUFFER_STATE oldbuf = YY_CURRENT_BUFFER;
reflex::Input input(...);
YY_BUFFER_STATE newbuf = yy_create_buffer(input, YY_BUF_SIZE);
yy_switch_to_buffer(newbuf);
... // scan the new input
yy_delete_buffer(newbuf);
... // the specified input is no longer needed here
yy_switch_to_buffer(oldbuf);
... // continue scanning the old input
```

--------------------------------

### Constructing reflex::Pattern and reflex::Matcher

Source: https://www.genivia.com/doc/reflex/html/index.html

Basic syntax for initializing a pattern and associating it with a matcher.

```cpp
#include <reflex/matcher.h>
[static] [const] reflex:Pattern pattern(string [, "options"] )
reflex::Matcher matcher(pattern, reflex::Input [, "options"] )
```

--------------------------------

### Flex-compatible YY_AT_BOL macro

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Performs an at-begin-of-line check; use only when the scanner has started.

```cpp
#define YY_AT_BOL | ( | | ) |  YY_SCANNER_DOT_ matcher().at_bol()
```

--------------------------------

### Flex-compatible scanner state macros

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Accesses the current matcher buffer and start condition.

```cpp
#define YY_CURRENT_BUFFER YY_SCANNER_DOT_ ptr_matcher()
```

```cpp
#define yy_current_state YY_SCANNER_DOT_ start()
```

--------------------------------

### Flex-compatible yy_act macro

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Returns the matched action index; use only when the scanner has started.

```cpp
#define yy_act YY_SCANNER_DOT_ matcher().accept()
```

--------------------------------

### Implement nested include directives with RE/flex

Source: https://www.genivia.com/doc/reflex/html/index.html

Uses a stack of matchers to handle nested #include directives up to a depth of 99 files.

```C
%top{
#include <stdio.h>
}
%class{
int depth;
void include_file()
{
depth++;
if (depth > 99)
exit(EXIT_FAILURE); // max include depth exceeded
char *q = strchr(text(), '"'); // get ..."filename"
char *file = strdup(q + 1); // get filename"
file[strlen(file) - 1] = '\0'; // get filename
FILE *fd = fopen(file, "r");
free(file);
if (!fd)
exit(EXIT_FAILURE); // cannot open file
push_matcher(new_matcher(fd)); // push current matcher, use new matcher
}
bool end_of_file()
{
if (depth == 0)
return true; // return true: no more input to read
fclose(in()); // close current input in() (a FILE*)
pop_matcher(); // delete current matcher, pop matcher
depth--;
return false; // return false: continue reading
}
}
%init{
depth = 0;
}
%%
^\h*#include\h*\".*?\" include_file();
.|\n echo();
<<EOF>> if (end_of_file()) return 0;
%%
```

--------------------------------

### Initialize Lexer with Custom Input

Source: https://www.genivia.com/doc/reflex/html/index.html

Instantiate a Lexer class with a specific input source and output stream.

```cpp
Lexer lexer(input, std::cout);
```

```cpp
yyFlexLexer lexer(input, std::cout);
```

--------------------------------

### Compile RE/flex from source

Source: https://www.genivia.com/doc/reflex/html/index.html

Manual compilation commands for cross-compilation or applying specific SIMD optimizations.

```bash
c++ -I. -Iinclude lex.yy.cpp lib/debug.cpp lib/error.cpp \
    lib/input.cpp lib/matcher.cpp lib/pattern.cpp lib/utf8.cpp lib/simd.cpp
```

```bash
c++ -DHAVE_NEON -mfpu=neon -I. -Iinclude lex.yy.cpp lib/debug.cpp lib/error.cpp \
    lib/input.cpp lib/matcher.cpp lib/pattern.cpp lib/utf8.cpp lib/simd.cpp
```

```bash
c++ -DHAVE_SSE2 -msse2 -I. -Iinclude lex.yy.cpp lib/debug.cpp lib/error.cpp \
    lib/input.cpp lib/matcher.cpp lib/pattern.cpp lib/utf8.cpp lib/simd.cpp
```

```bash
c++ -DHAVE_AVX2 -mavx2 -I. -Iinclude lex.yy.cpp lib/debug.cpp lib/error.cpp \
    lib/input.cpp lib/matcher.cpp lib/pattern.cpp lib/utf8.cpp lib/simd.cpp \
    lib/matcher_avx2.cpp lib/simd_avx2.cpp
```

```bash
c++ -DHAVE_AVX512BW -mavx512bw -I. -Iinclude lex.yy.cpp lib/debug.cpp lib/error.cpp \
    lib/input.cpp lib/matcher.cpp lib/pattern.cpp lib/utf8.cpp lib/simd.cpp \
    lib/matcher_avx2.cpp lib/matcher_avx512bw.cpp lib/simd_avx2.cpp lib/simd_avx512bw.cpp
```

--------------------------------

### Using External Buffers

Source: https://www.genivia.com/doc/reflex/html/index.html

Demonstrates using an external buffer for zero-copy input assignment.

```cpp
// read a 0-terminated buffer in place, buffer content is changed!!
char *base = ...; // points to the base address of the data
size_t size = ...; // length of the data including one extra byte for a final \0
lexer.buffer(base, size);
lexer.lex();
```

--------------------------------

### Initialize Pattern with Options

Source: https://www.genivia.com/doc/reflex/html/index.html

Configures a regex pattern with specific flags for case-insensitivity, dot-all mode, and exception handling.

```cpp
reflex::Pattern pattern(pattern, "isr")
```

--------------------------------

### Generated FSM Opcode Table

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of an FSM opcode table generated by the f=machine.cpp option.

```cpp
REFLEX_CODE_DECL reflex_code_FSM[11] =
{
0x617A0005, // 0: GOTO 5 ON a-z
0x5F5F0005, // 1: GOTO 5 ON _
0x415A0005, // 2: GOTO 5 ON A-Z
0x30390005, // 3: GOTO 5 ON 0-9
0x00FFFFFF, // 4: HALT
0xFF000001, // 5: TAKE 1
0x617A0005, // 6: GOTO 5 ON a-z
0x5F5F0005, // 7: GOTO 5 ON _
0x415A0005, // 8: GOTO 5 ON A-Z
0x30390005, // 9: GOTO 5 ON 0-9
0x00FFFFFF, // 10: HALT
};
```

--------------------------------

### Search and Split Text with Matcher and BoostMatcher

Source: https://www.genivia.com/doc/reflex/html/index.html

Demonstrates using reflex::Matcher and reflex::BoostMatcher for finding patterns, counting words, and splitting text using iterators and range-based loops.

```cpp
#include <reflex/matcher.h>
#include <reflex/boostmatcher.h>
using namespace reflex;
// some random text (with extra spacing)
std::string example("Monty\n Python's Flying Circus");
// construct a fixed pattern that is case insensitive
static const Pattern pattern("monty|python", "i");
// construct a matcher to search the example text
Matcher matcher(pattern, example);
while (matcher.find() != 0)
std::cout << matcher.text()
<< " at " << matcher.lineno() << "," << matcher.columno()
<< " spans " << matcher.first() << ".." << matcher.last()
<< std::endl;
// construct a Boost.Regex matcher to count the words (all non-spaces "\S+")
BoostMatcher boostmatcher("\\\S+", example);
boostmatcher.buffer(); // because Boost.Regex partial_match is broken!
std::cout << std::distance(boostmatcher.find.begin(), boostmatcher.find.end())
<< " words"
<< std::endl;
// reuse the BoostMatcher with a C++11 range-based loop to split the example text on space "\s+":
for (auto& split : boostmatcher.pattern("\\\s+").input(example).split)
std::cout << split.text() << " ";
std::cout << std::endl;
// reuse the BoostMatcher to split on whitespace using an iterator, sort into a set, and print
boostmatcher.input(example).buffer();
std::set<std::string> words(boostmatcher.split.begin(), boostmatcher.split.end());
std::copy(words.begin(), words.end(), std::ostream_iterator<std::string>(std::cout, " "));
std::cout << std::endl;
```

--------------------------------

### Scan nested structures

Source: https://www.genivia.com/doc/reflex/html/index.html

Uses a start condition state to scan nested parentheses as a single token.

```text
<NESTINGS>{
.*?[()] yymore();
if (yytext[yyleng-1] == '(') ++param;
else if (param-- <= 1) { do_something(yytext); yy_pop_state(); }
}
```

--------------------------------

### void compile()

Source: https://www.genivia.com/doc/reflex/html/classreflex_1_1_p_c_r_e2_u_t_f_matcher.html

Compiles the pattern for JIT partial matching and allocates necessary match data.

```APIDOC
## void compile()

### Description
Compiles the pattern for JIT partial matching and allocates match data.
```

--------------------------------

### matcher().tabs()

Source: https://www.genivia.com/doc/reflex/html/index.html

Gets or sets the current tab expansion value (1, 2, 4, or 8).

```APIDOC
## matcher().tabs()

### Description
Returns or sets the current tab expansion value used by the matcher.

### Parameters
- **n** (int) - Optional - The tab expansion value to set (1, 2, 4, or 8).

### Response
- **value** (int) - The current tab expansion value.
```

--------------------------------

### Compiling with libreflex

Source: https://www.genivia.com/doc/reflex/html/index.html

Link the application against the libreflex library during compilation.

```bash
c++ myapp.cpp -lreflex
```

--------------------------------

### Matcher Configuration Methods

Source: https://www.genivia.com/doc/reflex/html/index.html

Methods used to initialize a matcher, assign input sources, and configure patterns or buffering.

```APIDOC
## Matcher Configuration

### Methods
- `input(i)`: Set input to `reflex::Input i` (string, stream, or `FILE*`).
- `pattern(p)`: Set pattern `reflex::Pattern`, `boost::regex`, or a string `p`.
- `has_pattern()`: Returns true if the matcher has a pattern assigned.
- `own_pattern()`: Returns true if the matcher has a pattern to manage and delete.
- `pattern()`: Returns a reference to the pattern object.
- `buffer()`: Buffer all input at once.
- `buffer(n)`: Set the buffer window size to `n` bytes.
- `buffer(b, n)`: Use buffer of `n` bytes at address `b` (zero copy).
- `interactive()`: Set buffer size to 1 for TTY input.
- `flush()`: Flush remaining input from internal buffer.
- `reset()`: Resets the matcher.
- `reset(o)`: Resets the matcher with options string `o`.
```

--------------------------------

### matcher().first() and matcher().last()

Source: https://www.genivia.com/doc/reflex/html/index.html

Returns the position in the input stream of the match, counting in bytes from the start of the input.

```APIDOC
## matcher().first() / matcher().last()

### Description
Returns the position in the input stream of the match, counting in bytes from the start of the input at position 0.
```

--------------------------------

### Compile application with libreflex

Source: https://www.genivia.com/doc/reflex/html/index.html

Link the application against libreflex and ensure C++11 standard is enabled.

```bash
c++ -std=c++11 myapp.cpp -lreflex
```

--------------------------------

### reflex::AbstractLexer::top_state

Source: https://www.genivia.com/doc/reflex/html/classreflex_1_1_abstract_lexer.html

Returns the stack top start condition state or 0 (INITIAL) if the stack is empty.

```APIDOC
## int reflex::AbstractLexer< M >::top_state() const

### Description
Returns the stack top start condition state or 0 (INITIAL) if the stack is empty.

### Returns
- **int** - start condition (integer)
```

--------------------------------

### Bison/Yacc Grammar Definition

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of a Yacc grammar file defining the union type and tokens used by the scanner.

```yacc
/* yacc grammar (.y file) */
%{
extern int yylex(void);
%}
%union { // YYSTYPE yylval is a union:
int num; // yylval.num
char* str; // yylval.str
}
%token <num> CONST_NUMBER
%token <str> CONST_STRING
%%
... // grammar rules
%%
```

--------------------------------

### Define yyrestart macro

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Restart with a new input source.

```cpp
#define yyrestart | ( |  | i| ) |  YY_SCANNER_DOT_ in(i)
```

--------------------------------

### Flex-compatible buffer creation macros

Source: https://www.genivia.com/doc/reflex/html/flexlexer_8h.html

Creates and returns a new buffer.

```cpp
#define yy_create_buffer | ( |  | i,   
---|---|---|---  
|  |  | _   
| ) | |  YY_SCANNER_DOT_ new_matcher(i)
```

```cpp
#define YY_NEW_BUFFER | ( |  | i,   
---|---|---|---  
|  |  | _   
| ) | |  YY_SCANNER_DOT_ new_matcher(i)
```

--------------------------------

### Validate a date string using PCRE2

Source: https://www.genivia.com/doc/reflex/html/index.html

Uses the PCRE2Matcher class to perform the same date validation as the Boost example.

```cpp
#include <reflex/pcre2matcher.h> // reflex::PCRE2Matcher, reflex::Input
// use a PCRE2Matcher to check if the birthdate string is a valid date
if (reflex::PCRE2Matcher("\\d{4}-\\d{2}-\\d{2}", birthdate).matches())
std::cout << "Valid date!" << std::endl;
```

--------------------------------

### Implementing wrap() for Input Continuation

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of overriding the wrap() method to continue reading from std::cin after reaching EOF.

```cpp
%class{
virtual int wrap() // note: yywrap() with option −−flex
{
in(std::cin);
return in().good() ? 0 : 1;
}
}
```

--------------------------------

### Initialize a BoostPosixMatcher engine

Source: https://www.genivia.com/doc/reflex/html/index.html

Create a BoostPosixMatcher instance for POSIX-style matching, which does not support lazy quantifiers and uses the leftmost longest rule.

```cpp
#include <reflex/boostmatcher.h>
reflex::BoostPosixMatcher matcher( boost::regex or string, reflex::Input [, "options"] )
```

--------------------------------

### Lexer Specification with Defined Names

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of using defined names within a lexer specification to build character classes.

```reflex
lower [a-z]
upper [A-Z]
letter [||{lower}||{upper}]
alnum [0-9||{letter}]
name {letter}{alnum}*
%%
{name} std::cout << "name: " << text() << std::endl;
.|
 // skip everything else
%%
```

--------------------------------

### Generated Native C++ FSM Code

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of optimized native C++ code generated for an FSM using the o option.

```cpp
void reflex_code_FSM(reflex::Matcher& m)
{
int c = 0;
m.FSM_INIT(c);
S0:
c = m.FSM_CHAR();
if (97 <= c && c <= 122) goto S5;
if (c == 95) goto S5;
if (65 <= c && c <= 90) goto S5;
if (48 <= c && c <= 57) goto S5;
return m.FSM_HALT(c);
S5:
m.FSM_TAKE(1);
c = m.FSM_CHAR();
if (97 <= c && c <= 122) goto S5;
if (c == 95) goto S5;
if (65 <= c && c <= 90) goto S5;
if (48 <= c && c <= 57) goto S5;
return m.FSM_HALT(c);
}
```

--------------------------------

### Initialize Input with Encoding

Source: https://www.genivia.com/doc/reflex/html/index.html

Sets the input stream encoding to ISO-8859-1 for the matcher.

```cpp
reflex::Input input(stdin, reflex::Input::file_encoding::latin);
reflex::Matcher matcher(pattern, input);
```

--------------------------------

### Compile an application with RE/flex

Source: https://www.genivia.com/doc/reflex/html/index.html

Standard compilation command for linking an application against the libreflex library.

```bash
c++ myapp.cpp -lreflex

```

```bash
c++ myapp.cpp -lreflex -lpcre2-8

```

```bash
c++ myapp.cpp -lreflex -lboost_regex

```

--------------------------------

### State-based Indent Stop Management

Source: https://www.genivia.com/doc/reflex/html/index.html

Uses start conditions and stop management to handle nested comments while preserving indentation state.

```RE/flex
%{
int level; // a variable to track the /*-comment nesting level
std::vector<size_t> stops; // a variable to save the stop positions after indent
%}
%o tabs=8
%x COMMENT
%%
^\h+ out() << "| "; // nodent, text is aligned to current margin column
^\h+\i out() << "> "; // indent
stops = matcher().stops(); // save the stop positions
^"/*"\j level = 1; // do not count dedent(s) to the first line that has a /*-comment
start(COMMENT); // skip comment
^\h*\j out() << "< "; // dedent
\j out() << "< "; // dedent, triggered for each extra level dedented
(?^^\h*\n) // a negative pattern to eat empty lines without affecting indent stops
(?^^\h+/"/*") // a negative pattern to eat white space before /*-comments without affecting indent stops
"/*" level = 1;
start(COMMENT); // continue w/o indent matching
(?^\\\\\n\h*) // lines ending in \ continue on the next line
.|\n echo(); // ECHO character
<COMMENT>{
"/*" ++level; // allow nested /*-comments 
"*/" if (--level == 0)
{
matcher().stops() = stops; // restore the indent margin/tab stops
start(INITIAL); // back to initial state
}
.|\n // ignore all content in comments
<<EOF>> out() << "/* not closed";
}
%%
```

--------------------------------

### Handling Regex Syntax Errors

Source: https://www.genivia.com/doc/reflex/html/index.html

Example of using a try-catch block to handle regex syntax errors when the 'r' option is enabled.

```cpp
try
{
reflex::Pattern pattern(argv[1], "r"); // "r" option throws syntax error exceptions
...
// code that uses the pattern object
...
}
catch (reflex::regex_error& e)
{
switch (e.code())
{
case reflex::regex_error::mismatched_parens: std::cerr << "mismatched ( )"; break;
case reflex::regex_error::mismatched_braces: std::cerr << "mismatched { }"; break;
case reflex::regex_error::mismatched_brackets: std::cerr << "mismatched [ ]"; break;
case reflex::regex_error::mismatched_quotation: std::cerr << "mismatched \\Q...\\E quotation"; break;
case reflex::regex_error::empty_expression: std::cerr << "regex (sub)expression should not be empty"; break;
case reflex::regex_error::empty_class: std::cerr << "character class [...] is empty, e.g. [a&&[b]]"; break;
case reflex::regex_error::invalid_class: std::cerr << "invalid character class name"; break;
case reflex::regex_error::invalid_class_range: std::cerr << "invalid character class range, e.g. [Z-A]"; break;
case reflex::regex_error::invalid_escape: std::cerr << "invalid escape character"; break;
case reflex::regex_error::invalid_anchor: std::cerr << "invalid anchor or boundary"; break;
case reflex::regex_error::invalid_repeat: std::cerr << "invalid repeat, e.g. {10,1}"; break;
case reflex::regex_error::invalid_quantifier: std::cerr << "invalid lazy or possessive quantifier"; break;
case reflex::regex_error::invalid_modifier: std::cerr << "invalid (?ismux:) modifier"; break;
case reflex::regex_error::invalid_collating: std::cerr << "invalid collating element"; break;
case reflex::regex_error::invalid_backreference: std::cerr << "invalid backreference"; break;
case reflex::regex_error::invalid_syntax: std::cerr << "invalid regex syntax"; break;
case reflex::regex_error::exceeds_length: std::cerr << "exceeds length limit, pattern is too long"; break;
case reflex::regex_error::exceeds_limits: std::cerr << "exceeds complexity limits, e.g. {n,m} range too large"; break;
}
std::cerr << std::endl << e.what();
}
```

--------------------------------

### Configure fuzzy matching flags

Source: https://www.genivia.com/doc/reflex/html/index.html

Examples of using bitwise OR to combine maximum edit distance with specific fuzzy matching constraints.

```cpp
reflex::FuzzyMatcher matcher(regex, 3 | reflex::FuzzyMatcher::INS, input);
```

```cpp
reflex::FuzzyMatcher matcher(regex, 3 | reflex::FuzzyMatcher::INS | reflex::FuzzyMatcher::DEL, input);
```

--------------------------------

### Constructing a reflex::Matcher

Source: https://www.genivia.com/doc/reflex/html/index.html

Initialize a matcher using a pattern or string and an input source.

```cpp
#include <reflex/matcher.h>
reflex::Matcher matcher( reflex::Pattern or string, reflex::Input [, "options"] )
```

--------------------------------

### Switching Input Sources for Reentrant Scanners

Source: https://www.genivia.com/doc/reflex/html/index.html

Examples of configuring input sources for reentrant lexers, which require an additional yyscan_t argument.

```cpp
// read from a file, this also decodes UTF-8/16/32 encodings automatically
FILE *fd = fopen("cow.txt", "r");
if (fd == NULL)
... // error, bail out
yyget_in(yyscanner) = fd;
yylex();
// read from a stream (ASCII or UTF-8)
std::istream i = std::ifstream("file", std::ios::in);
yyget_in(yyscanner) = &i;
yylex();
// read from a string (0-terminated, ASCII or UTF-8)
yy_delete_buffer(YY_CURRENT_BUFFER, yyscanner);
yy_scan_string("How now brown cow.", yyscanner); // new buffer to scan a string
yylex();
// read from a memory segment (raw bytes, ASCII, or UTF-8)
const char *ptr = ...; // points to memory segment
size_t len = ...; // length of the memory segment
yy_delete_buffer(YY_CURRENT_BUFFER, yyscanner);
yy_scan_bytes(ptr, len, yyscanner); // new buffer to scan memory
yylex();
// read from a wide string, 0-terminated, encoding it to UTF-8 for matching
yy_delete_buffer(YY_CURRENT_BUFFER, yyscanner);
yy_scan_wstring(L"How now brown cow.", yyscanner); // new buffer to scan a wide string
yylex();
// read a 0-terminated buffer in place, buffer content is changed!!
char *base = ...; // points to 0-terminated buffer
size_t size = ...; // length of the buffer including final 0 byte
yy_delete_buffer(YY_CURRENT_BUFFER, yyscanner);
yy_scan_buffer(base, size, yyscanner);
yylex();
```
Tokens:
Raw