### Example Go Code Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html A simple Go function to demonstrate syntax highlighting. ```go func increment(a int) int { return a + 1 } ``` -------------------------------- ### Install Tree-sitter CLI Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Install the Tree-sitter CLI globally on your system. This command makes the `tree-sitter` executable available in your PATH. ```bash cargo install --path crates/cli ``` -------------------------------- ### Install mdBook CLI Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Install the `mdbook` CLI tool, which is required to run and iterate on the documentation locally. Assumes `cargo` is installed. ```bash cargo install mdbook ``` -------------------------------- ### Install mdbook-admonish Preprocessor Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Install `mdbook-admonish`, a preprocessor for `mdBook` that enables fancy admonitions in the documentation. This is also a requirement for local development. ```bash cargo install mdbook-admonish ``` -------------------------------- ### Install Node.js dependencies for editor support Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html Install the Tree-sitter's TypeScript API from npm to enable documentation and type information in your code editor. This is typically done using `npm install`. ```bash npm install # or your package manager of choice ``` -------------------------------- ### Serve Documentation Locally Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Navigate to the `docs` directory and start a local server for the documentation using `mdbook`. This command includes a live-reload feature. ```bash cd docs mdbook serve --open ``` -------------------------------- ### Install Tree-sitter TypeScript API Source: https://tree-sitter.github.io/tree-sitter/print.html Install the Tree-sitter's TypeScript API from npm into a `node_modules` directory. This enables editor integration for documentation and type information within `grammar.js`. ```bash npm install # or your package manager of choice ``` -------------------------------- ### Install Tree-sitter CLI with Cargo Source: https://tree-sitter.github.io/tree-sitter/print.html Install the tree-sitter CLI from crates.io using Cargo, the Rust package manager. This method works on all platforms. ```bash cargo install tree-sitter-cli --locked ``` -------------------------------- ### Clone Tree-sitter Repository Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Clone the Tree-sitter repository and navigate into the directory. This is the first step to start contributing. ```bash git clone https://github.com/tree-sitter/tree-sitter cd tree-sitter ``` -------------------------------- ### Example Return Statement Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html A simple return statement in JavaScript. ```javascript return x + y; ``` -------------------------------- ### Run Tests for Specific Language and Example Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Filter tests to run only for a particular language and a specific example within that language's corpus. Use the `-l` flag for language and `-e` for example. ```bash cargo xtask test -l javascript -e Arrays ``` -------------------------------- ### Tree-sitter Playground Example Source: https://tree-sitter.github.io/tree-sitter/7-playground.html This is a placeholder for the actual playground code. The playground allows interactive testing of grammars and queries. ```placeholder xxxxxxxxxx ``` -------------------------------- ### Get Node Byte Positions Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html Obtain the starting and ending byte offsets of a syntax node within the source code. ```c uint32_t ts_node_start_byte(TSNode); uint32_t ts_node_end_byte(TSNode); ``` -------------------------------- ### Initialize New Grammar with tree-sitter init Source: https://tree-sitter.github.io/tree-sitter/print.html Use this command to start a new tree-sitter grammar project. It creates necessary configuration and boilerplate files. Recommended for use with Git version control. ```bash tree-sitter init [OPTIONS] # Aliases: i ``` -------------------------------- ### JavaScript Return Statement Example Source: https://tree-sitter.github.io/tree-sitter/print.html A basic JavaScript return statement used to illustrate grammar complexity. No special setup is required to use this snippet. ```javascript return x + y; ``` -------------------------------- ### Go Code Syntax Tree Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html The abstract syntax tree representation of the example Go code. ```tree-sitter (source_file (function_declaration name: (identifier) parameters: (parameter_list (parameter_declaration name: (identifier) type: (type_identifier))) result: (type_identifier) body: (block (return_statement (expression_list (binary_expression left: (identifier) right: (int_literal))))))) ``` -------------------------------- ### Ruby Code Example for Highlighting Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html This Ruby code demonstrates various types of names including methods, formal parameters, local variables, and method calls. ```ruby def process_list(list) context = current_context list.map do |item| process_item(item, context) end end item = 5 list = [item] ``` -------------------------------- ### Example `tree-sitter.json` Grammar Configuration Source: https://tree-sitter.github.io/tree-sitter/cli/init.html This snippet shows a basic configuration for a single grammar within the `tree-sitter.json` file, specifying scope, file types, and a first-line regex. ```json { "tree-sitter": [ { "scope": "source.ruby", "file-types": [ "rb", "gemspec", "Gemfile", "Rakefile" ], "first-line-regex": "#!.*\\bruby$" } ] } ``` -------------------------------- ### Start Tree-sitter Playground Source: https://tree-sitter.github.io/tree-sitter/cli/playground.html Use this command to launch a local playground for interactive parser testing. Ensure your parser is built as a Wasm module first using `tree-sitter build --wasm`. ```bash tree-sitter playground [OPTIONS] # Aliases: play, pg, web-ui ``` -------------------------------- ### Build Tree-sitter CLI with Release-Dev Profile Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Build the Tree-sitter CLI using the `release-dev` profile for faster build times during rapid iteration cycles. This can be used for both building and installing. ```bash cargo build --profile release-dev # or cargo install --path crates/cli --profile release-dev ``` -------------------------------- ### Basic grammar.js structure Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html The `grammar.js` file defines the syntax rules for your parser. This is a minimal example with a single rule. ```javascript /** * @file PARSER_DESCRIPTION * @author PARSER_AUTHOR_NAME PARSER_AUTHOR_EMAIL * @license PARSER_LICENSE */ /// // @ts-check export default grammar({ name: 'LOWER_PARSER_NAME', rules: { // TODO: add the actual grammar rules source_file: $ => 'hello' } }); ``` -------------------------------- ### Example ERB Document Source: https://tree-sitter.github.io/tree-sitter/print.html An example of an Embedded Ruby (ERB) document, which mixes HTML structure with Ruby code for dynamic content generation. ```erb ``` -------------------------------- ### Configure Parser Directories Source: https://tree-sitter.github.io/tree-sitter/cli/init-config.html Specify directories where the Tree-sitter CLI should look for grammar repositories. Folders starting with `tree-sitter-` within these paths are recognized as grammars. ```json { "parser-directories": [ "/Users/my-name/code", "~/other-code", "$HOME/another-code" ] } ``` -------------------------------- ### JavaScript Keyword Example Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html Demonstrates the use of a JavaScript keyword `instanceof` in a conditional statement. This highlights the distinction between a keyword and an identifier. ```javascript if (a instanceof Something) b(); ``` -------------------------------- ### Test Ruby Code with `tree-sitter tags` Source: https://tree-sitter.github.io/tree-sitter/print.html Use the `tree-sitter tags` command to test tag query files against source code. This example demonstrates tagging a Ruby file. ```ruby module Foo class Bar # won't be included # is adjacent, will be def baz end end end ``` ```bash tree-sitter tags test.rb ``` ```text test.rb Foo | module def (0, 7) - (0, 10) `module Foo` Bar | class def (1, 8) - (1, 11) `class Bar` baz | method def (2, 8) - (2, 11) `def baz` "is adjacent, will be" ``` -------------------------------- ### C Program to Parse JSON with Tree-sitter Source: https://tree-sitter.github.io/tree-sitter/using-parsers/1-getting-started.html An example C program demonstrating how to use the Tree-sitter C API to parse a JSON string, inspect the syntax tree, and verify node properties. ```c #include #include #include #include // Declare the `tree_sitter_json` function, which is // implemented by the `tree-sitter-json` library. const TSLanguage *tree_sitter_json(void); int main() { // Create a parser. TSParser *parser = ts_parser_new(); // Set the parser's language (JSON in this case). ts_parser_set_language(parser, tree_sitter_json()); // Build a syntax tree based on source code stored in a string. const char *source_code = "[1, null]"; TSTree *tree = ts_parser_parse_string( parser, NULL, source_code, strlen(source_code) ); // Get the root node of the syntax tree. TSNode root_node = ts_tree_root_node(tree); // Get some child nodes. TSNode array_node = ts_node_named_child(root_node, 0); TSNode number_node = ts_node_named_child(array_node, 0); // Check that the nodes have the expected types. assert(strcmp(ts_node_type(root_node), "document") == 0); assert(strcmp(ts_node_type(array_node), "array") == 0); assert(strcmp(ts_node_type(number_node), "number") == 0); // Check that the nodes have the expected child counts. assert(ts_node_child_count(root_node) == 1); assert(ts_node_child_count(array_node) == 5); assert(ts_node_named_child_count(array_node) == 2); assert(ts_node_child_count(number_node) == 0); // Print the syntax tree as an S-expression. char *string = ts_node_string(root_node); printf("Syntax tree: %s\n", string); // Free all of the heap-allocated memory. free(string); ts_tree_delete(tree); ts_parser_delete(parser); return 0; } ``` -------------------------------- ### Initialize Tree Cursor Source: https://tree-sitter.github.io/tree-sitter/using-parsers/4-walking-trees.html Initializes a tree cursor from a given TSNode. The cursor is stateful and starts at the provided node. ```c TSTreeCursor ts_tree_cursor_new(TSNode); ``` -------------------------------- ### Grammar Rule Example Source: https://tree-sitter.github.io/tree-sitter/print.html Illustrates a grammar rule definition for an if statement, showing how named and anonymous nodes are formed. ```javascript if_statement: $ => seq("if", "(", $._expression, ")", $._statement); ``` -------------------------------- ### JavaScript Syntax Highlighting Test Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html This example demonstrates how to write a syntax highlighting test for JavaScript. Comments with special markers like '<-' and '^' assert expected scopes at specific positions. ```javascript __ var abc = function(d) { // <- keyword // ^ keyword // ^ variable.parameter // ^ function if (a) { // <- keyword // ^ punctuation.bracket foo(`foo ${bar}`); // <- function // ^ string // ^ variable } baz(); // <- !variable }; ``` -------------------------------- ### Tree Cursor Initialization Source: https://tree-sitter.github.io/tree-sitter/print.html Initializes a tree cursor from a given node. The cursor can then be used to efficiently walk the syntax tree starting from this node. ```APIDOC ## ts_tree_cursor_new ### Description Initializes a new tree cursor starting from the provided node. The cursor can then be used to navigate the syntax tree efficiently. ### Function Signature ```c TSTreeCursor ts_tree_cursor_new(TSNode); ``` ### Parameters - **node** (TSNode) - The starting node for the cursor. ``` -------------------------------- ### Conditional Scanning Logic Example Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/4-external-scanners.html Demonstrates how to conditionally scan for tokens like INDENT and DEDENT based on the `valid_symbols` array. ```APIDOC ## Conditional Scanning Logic ### Description This example illustrates how to implement conditional logic within the `scan` function to recognize specific tokens only when they are expected by the parser. ### Example ```c if (valid_symbols[INDENT] || valid_symbols[DEDENT]) { // Logic common to both INDENT and DEDENT if (valid_symbols[INDENT]) { // Logic specific to INDENT lexer->result_symbol = INDENT; return true; } // Potentially logic for DEDENT here if not handled by INDENT block } ``` ### Behavior The code checks if either `INDENT` or `DEDENT` symbols are valid. If `INDENT` is valid, it proceeds with the specific logic for `INDENT`, assigns the `INDENT` symbol to `lexer->result_symbol`, and returns `true`. This pattern ensures that scanning efforts are focused only on tokens the parser currently requires. ``` -------------------------------- ### Parse Mixed Language Document with Tree-sitter Source: https://tree-sitter.github.io/tree-sitter/using-parsers/3-advanced-parsing.html This example demonstrates parsing an ERB document by first parsing it as ERB, then identifying HTML and Ruby code sections using ranges. It then re-parses these sections with their respective languages (HTML and Ruby) and prints all three syntax trees. ```c #include #include // These functions are each implemented in their own repo. const TSLanguage *tree_sitter_embedded_template(void); const TSLanguage *tree_sitter_html(void); const TSLanguage *tree_sitter_ruby(void); int main(int argc, const char **argv) { const char *text = argv[1]; unsigned len = strlen(text); // Parse the entire text as ERB. TSParser *parser = ts_parser_new(); ts_parser_set_language(parser, tree_sitter_embedded_template()); TSTree *erb_tree = ts_parser_parse_string(parser, NULL, text, len); TSNode erb_root_node = ts_tree_root_node(erb_tree); // In the ERB syntax tree, find the ranges of the `content` nodes, // which represent the underlying HTML, and the `code` nodes, which // represent the interpolated Ruby. TSRange html_ranges[10]; TSRange ruby_ranges[10]; unsigned html_range_count = 0; unsigned ruby_range_count = 0; unsigned child_count = ts_node_child_count(erb_root_node); for (unsigned i = 0; i < child_count; i++) { TSNode node = ts_node_child(erb_root_node, i); if (strcmp(ts_node_type(node), "content") == 0) { html_ranges[html_range_count++] = (TSRange) { ts_node_start_point(node), ts_node_end_point(node), ts_node_start_byte(node), ts_node_end_byte(node), }; } else { TSNode code_node = ts_node_named_child(node, 0); ruby_ranges[ruby_range_count++] = (TSRange) { ts_node_start_point(code_node), ts_node_end_point(code_node), ts_node_start_byte(code_node), ts_node_end_byte(code_node), }; } } // Use the HTML ranges to parse the HTML. ts_parser_set_language(parser, tree_sitter_html()); ts_parser_set_included_ranges(parser, html_ranges, html_range_count); TSTree *html_tree = ts_parser_parse_string(parser, NULL, text, len); TSNode html_root_node = ts_tree_root_node(html_tree); // Use the Ruby ranges to parse the Ruby. ts_parser_set_language(parser, tree_sitter_ruby()); ts_parser_set_included_ranges(parser, ruby_ranges, ruby_range_count); TSTree *ruby_tree = ts_parser_parse_string(parser, NULL, text, len); TSNode ruby_root_node = ts_tree_root_node(ruby_tree); // Print all three trees. char *erb_sexp = ts_node_string(erb_root_node); char *html_sexp = ts_node_string(html_root_node); char *ruby_sexp = ts_node_string(ruby_root_node); printf("ERB: %s\n", erb_sexp); printf("HTML: %s\n", html_sexp); printf("Ruby: %s\n", ruby_sexp); return 0; } ``` -------------------------------- ### Test parser with a simple file (Linux/macOS) Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html Create a file named `example-file` with the content 'hello' and then use `tree-sitter parse` to test your newly generated parser. ```bash echo 'hello' > example-file tree-sitter parse example-file ``` -------------------------------- ### Get Node Character Positions Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html Retrieve the starting and ending row and column coordinates of a syntax node. Rows and columns are zero-based. ```c typedef struct { uint32_t row; uint32_t column; } TSPoint; TSPoint ts_node_start_point(TSNode); TSPoint ts_node_end_point(TSNode); ``` -------------------------------- ### Alternation for Call Expressions Source: https://tree-sitter.github.io/tree-sitter/print.html Use square brackets for alternations to match different patterns. This example captures either a variable or an object property as a function or method. ```tree-sitter query ( call_expression function: [ (identifier) @function (member_expression property: (property_identifier) @method) ]) ``` -------------------------------- ### Test parser with a simple file (Windows PowerShell) Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html Create a file named `example-file` with the content 'hello' and then use `tree-sitter parse` to test your newly generated parser using PowerShell. ```powershell "hello" | Out-File example-file -Encoding utf8 tree-sitter parse example-file ``` -------------------------------- ### TSRange Structure Definition Source: https://tree-sitter.github.io/tree-sitter/print.html Defines the structure for specifying a range within a document, including start and end points and byte offsets. Used with `ts_parser_set_included_ranges`. ```c typedef struct { TSPoint start_point; TSPoint end_point; uint32_t start_byte; uint32_t end_byte; } TSRange; void ts_parser_set_included_ranges( TSParser *self, const TSRange *ranges, uint32_t range_count ); ``` -------------------------------- ### Test Tree-sitter parser with a sample file (Windows PowerShell) Source: https://tree-sitter.github.io/tree-sitter/print.html Create a sample file named `example-file` with the content 'hello' using PowerShell and then use `tree-sitter parse` to test the generated parser. This is the Windows equivalent of the Unix/macOS test. ```powershell __ "hello" | Out-File example-file -Encoding utf8 tree-sitter parse example-file ``` -------------------------------- ### Capture Nodes in Class Declaration Pattern Source: https://tree-sitter.github.io/tree-sitter/print.html This example demonstrates capturing multiple nodes within a class declaration, including the class name and method names, using distinct capture names for each. ```tree-sitter grammar (class_declaration name: (identifier) @the-class-name body: (class_body (method_definition name: (property_identifier) @the-method-name))) ``` -------------------------------- ### Initialize Tree-sitter parser project Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html Use the `tree-sitter init` command to set up the basic file structure for your parser project. This command will prompt you for necessary information. ```bash tree-sitter init ``` -------------------------------- ### Find Cgo comments in Go code using #eq? and #match? Source: https://tree-sitter.github.io/tree-sitter/using-parsers/queries/3-predicates-and-directives.html This query finds C code embedded in Go comments that appear just before a 'C' import statement. It uses `#eq?` to ensure the import path is exactly '"C"' and `#match?` to check if the comment starts with '//'. ```tree-sitter query ((comment)+ @injection.content . (import_declaration (import_spec path: (interpreted_string_literal) @_import_c)) (#eq? @_import_c "\"C\"") (#match? @injection.content "^//")) ``` -------------------------------- ### Initialize Tree-sitter Configuration Source: https://tree-sitter.github.io/tree-sitter/cli/init-config.html Run this command to create a default configuration file for the Tree-sitter CLI. It will print the location of the created file. ```bash tree-sitter init-config ``` -------------------------------- ### Set Injection Language with set! Directive Source: https://tree-sitter.github.io/tree-sitter/print.html Use the `set!` directive to associate a key-value pair with a pattern. This example sets the `injection.language` property to "doxygen" for Doxygen-style comments, enabling them to be parsed by a Doxygen parser. ```tree-sitter grammar __ ((comment) @injection.content (#match? @injection.content "/[*\/][!*\/] [ $.expression, ], rules: { expression: $ => choice( $.identifier, // ... ), // ... }, }); ``` -------------------------------- ### Use Alternative Configuration File Source: https://tree-sitter.github.io/tree-sitter/cli/test.html Specify a path to an alternative `config.json` file for test configuration. ```bash tree-sitter test --config-path ``` -------------------------------- ### Get Root Node Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html Access the root node of a given syntax tree. ```c TSNode ts_tree_root_node(const TSTree *); ``` -------------------------------- ### Specify Configuration Path Source: https://tree-sitter.github.io/tree-sitter/print.html Provide an alternative configuration file using the `--config-path` option. Refer to the `init-config` command for more details. ```bash tree-sitter highlight --config-path [PATHS]... ``` -------------------------------- ### Access Node Children Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html Get the number of children a node has and retrieve a specific child by its index. ```c uint32_t ts_node_child_count(TSNode); TSNode ts_node_child(TSNode, uint32_t); ``` -------------------------------- ### Get Node Type Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html Retrieve the string type of a syntax node, which corresponds to a grammar rule. ```c const char *ts_node_type(TSNode); ``` -------------------------------- ### Tree-sitter Highlight Command Usage Source: https://tree-sitter.github.io/tree-sitter/cli/highlight.html Basic command structure for running syntax highlighting on files. Supports options for HTML output and other configurations. ```bash tree-sitter highlight [OPTIONS] [PATHS]... ``` -------------------------------- ### Build Wasm Library Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Build the WebAssembly library for Tree-sitter. This step is optional but recommended to avoid internet dependency for the playground. It uses Node.js and npm. ```bash cd lib/binding_web npm install # or your JS package manager of choice npm run build ``` -------------------------------- ### Get Current Node and Field Info Source: https://tree-sitter.github.io/tree-sitter/using-parsers/4-walking-trees.html Retrieves the current node the cursor is on, along with its associated field name and ID. ```c TSNode ts_tree_cursor_current_node(const TSTreeCursor *); const char *ts_tree_cursor_current_field_name(const TSTreeCursor *); TSFieldId ts_tree_cursor_current_field_id(const TSTreeCursor *); ``` -------------------------------- ### Create a new parser directory Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html Initialize a new directory for your Tree-sitter parser project. Replace `${LOWER_PARSER_NAME}` with the lowercase name of the language you are creating a parser for. ```bash mkdir tree-sitter-${LOWER_PARSER_NAME} cd tree-sitter-${LOWER_PARSER_NAME} ``` -------------------------------- ### Capture Node in Assignment Expression Source: https://tree-sitter.github.io/tree-sitter/using-parsers/queries/2-operators.html Associates a name with a specific node within a pattern. Capture names start with '@'. ```tree-sitter query (assignment_expression left: (identifier) @the-function-name right: (function)) ``` -------------------------------- ### Create a new Tree-sitter parser project Source: https://tree-sitter.github.io/tree-sitter/print.html Initialize a new Tree-sitter parser project. The `LOWER_PARSER_NAME` should be replaced with the lowercase name of the language for which the parser is being developed. ```bash __ mkdir tree-sitter-${LOWER_PARSER_NAME} cd tree-sitter-${LOWER_PARSER_NAME} ``` -------------------------------- ### Specify Query Paths for Highlighting Source: https://tree-sitter.github.io/tree-sitter/print.html Indicate the paths to query files (ending in `highlights.scm`) to be used for syntax highlighting via the `--query-paths` option. ```bash tree-sitter highlight --query-paths [PATHS]... ``` -------------------------------- ### Conditional Token Recognition (INDENT/DEDENT) Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/4-external-scanners.html Example of checking `valid_symbols` to conditionally recognize INDENT or DEDENT tokens, prioritizing INDENT if both are valid. ```c if (valid_symbols[INDENT] || valid_symbols[DEDENT]) { // ... logic that is common to both `INDENT` and `DEDENT` if (valid_symbols[INDENT]) { // ... logic that is specific to `INDENT` lexer->result_symbol = INDENT; return true; } } ``` -------------------------------- ### Node Type and Position Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html Retrieve the type of a syntax node and its start and end positions in terms of bytes and character coordinates (row/column). ```APIDOC ## Node Type and Position ### Description Get the type of a syntax node and its byte/character positions. ### Functions - `ts_node_type(TSNode)`: Returns the string type of the node. - `ts_node_start_byte(TSNode)`: Returns the starting byte position of the node. - `ts_node_end_byte(TSNode)`: Returns the ending byte position of the node. - `ts_node_start_point(TSNode)`: Returns the starting point (row, column) of the node. - `ts_node_end_point(TSNode)`: Returns the ending point (row, column) of the node. ### Types - `TSNode`: Represents a node in the syntax tree. - `TSPoint`: A struct containing `row` and `column` (uint32_t) for character coordinates. ``` -------------------------------- ### Tree-sitter Language Field Utilities Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html Utilities to get the count of fields, the name of a field by its ID, and the ID of a field by its name from a TSLanguage object. ```c uint32_t ts_language_field_count(const TSLanguage *); const char *ts_language_field_name_for_id(const TSLanguage *, TSFieldId); TSFieldId ts_language_field_id_for_name(const TSLanguage *, const char *, uint32_t); ``` -------------------------------- ### Configuration for Highlight Name to Color Mapping Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html A JSON configuration object mapping highlight names to colors for syntax highlighting. ```json { "theme": { "keyword": "purple", "function": "blue", "type": "green", "number": "brown" } } ``` -------------------------------- ### Display Verbose Build Information Source: https://tree-sitter.github.io/tree-sitter/cli/build.html Use the `-v` or `--verbose` flag to see detailed build information, including the working directory, compiler, arguments, and environment variables used during the build process. ```bash tree-sitter build -v ``` -------------------------------- ### Get Child Node by Field Name Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html Access a child node of a given TSNode using its field name. Requires the field name and its length. ```c TSNode ts_node_child_by_field_name( TSNode self, const char *field_name, uint32_t field_name_length ); ``` -------------------------------- ### Build Wasm Standard Library Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Build the Tree-sitter Wasm standard library using `xtask`. This process requires the Wasi SDK and `wasm-opt` tool, specified via environment variables. ```bash cargo xtask build-wasm-stdlib ``` -------------------------------- ### Define `word` Token in JavaScript Grammar Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html This example shows how to define an `identifier` as the `word` token in a JavaScript grammar. Tree-sitter uses this to optimize keyword matching. ```javascript grammar({ name: "javascript", word: $ => $.identifier, rules: { expression: $ => choice( $.identifier, $.unary_expression, $.binary_expression, // ... ), binary_expression: $ => choice( prec.left(1, seq($.expression, "instanceof", $.expression)), // ... ), unary_expression: $ => choice( prec.left(2, seq("typeof", $.expression)), // ... ), identifier: $ => /[a-z_]+/ }, }); ``` -------------------------------- ### Compile and Run JSON Parser with Static Linking Source: https://tree-sitter.github.io/tree-sitter/print.html This command compiles the C program using static linking with the Tree-sitter library. It requires the Tree-sitter API headers, the JSON parser source, and the static Tree-sitter library. After compilation, the executable is run. ```bash clang \ -I tree-sitter/lib/include \ test-json-parser.c \ tree-sitter-json/src/parser.c \ tree-sitter/libtree-sitter.a \ -o test-json-parser ./test-json-parser ``` -------------------------------- ### Set Injection Language with Doxygen Directive Source: https://tree-sitter.github.io/tree-sitter/using-parsers/queries/3-predicates-and-directives.html Use the `set!` directive to associate key-value pairs with a pattern. This example sets the `injection.language` to 'doxygen' for Doxygen-style comments. ```scm ((comment) @injection.content (#match? @injection.content "/[*][/][!*][/]?[^a-zA-Z]") (#set! injection.language "doxygen")) ``` -------------------------------- ### Compile and Run JSON Parser with Dynamic Linking Source: https://tree-sitter.github.io/tree-sitter/print.html This command compiles the C program using dynamic linking with the Tree-sitter library. It requires the Tree-sitter API headers and the JSON parser source, linking against the shared Tree-sitter library. Ensure the shared library is discoverable via `LD_LIBRARY_PATH`. ```bash clang \ -I tree-sitter/lib/include \ test-json-parser.c \ tree-sitter-json/src/parser.c \ -ltree-sitter \ -o test-json-parser ./test-json-parser ``` -------------------------------- ### Tree-sitter Query Command Usage Source: https://tree-sitter.github.io/tree-sitter/cli/query.html The basic syntax for using the `tree-sitter query` command. It takes a query path and one or more file paths as arguments. ```bash tree-sitter query [OPTIONS] [PATHS]... ``` -------------------------------- ### Get Child Node by Field ID Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html Access a child node of a given TSNode using its numeric field ID. This can be more efficient than using field names directly. ```c TSNode ts_node_child_by_field_id(TSNode, TSFieldId); ``` -------------------------------- ### Initial Go Grammar Skeleton Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html This snippet shows an initial, high-level grammar structure for a language like Go, focusing on major groupings of symbols such as definitions, functions, types, blocks, statements, and expressions. It uses TODO comments to indicate areas for future expansion. ```javascript { // ... rules: { source_file: $ => repeat($._definition), _definition: $ => choice( $.function_definition // TODO: other kinds of definitions ), function_definition: $ => seq( 'func', $.identifier, $.parameter_list, $._type, $.block ), parameter_list: $ => seq( '(' // TODO: parameters ')' ), _type: $ => choice( 'bool' // TODO: other kinds of types ), block: $ => seq( '{', repeat($._statement), '}' ), _statement: $ => choice( $.return_statement // TODO: other kinds of statements ), return_statement: $ => seq( 'return', $.expression, ';' ), expression: $ => choice( $.identifier, $.number // TODO: other kinds of expressions ), identifier: $ => /[a-z]+/, number: $ => /\d+/ } } ``` -------------------------------- ### Compile C Program with Static Linking Source: https://tree-sitter.github.io/tree-sitter/using-parsers/1-getting-started.html Compile the C program using clang, including necessary header paths and linking against the static Tree-sitter library and the JSON parser source. ```bash clang \ -I tree-sitter/lib/include \ test-json-parser.c \ tree-sitter-json/src/parser.c \ tree-sitter/libtree-sitter.a \ -o test-json-parser ./test-json-parser ``` -------------------------------- ### Resolve Parsing Conflicts with Precedence Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html Uses the `prec` function to assign precedence levels to grammar rules, resolving ambiguities. This example gives higher precedence to the unary minus operator. ```javascript { // ... unary_expression: $ => prec( 2, choice( seq("-", $.expression), seq("!", $.expression), // ... ), ); } ``` -------------------------------- ### Build Rust Libraries and CLI Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html Compile the Rust libraries and the command-line interface (CLI) for Tree-sitter using Cargo. The release build is optimized for performance. ```bash cargo build --release ``` -------------------------------- ### Define Included Ranges for Parsing Source: https://tree-sitter.github.io/tree-sitter/using-parsers/3-advanced-parsing.html Use TSRange to specify the start and end points, and bytes, for sections of text to be parsed. This is crucial for handling embedded languages within a larger document. ```c typedef struct { TSPoint start_point; TSPoint end_point; uint32_t start_byte; uint32_t end_byte; } TSRange; void ts_parser_set_included_ranges( TSParser *self, const TSRange *ranges, uint32_t range_count ); ``` -------------------------------- ### Invalid JavaScript Identifier Example Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html Shows an invalid JavaScript syntax where a keyword is immediately followed by characters that would form an identifier. Tree-sitter would normally tokenize this as two separate tokens. ```javascript if (a instanceofSomething) b(); ```