### Example Go Code
Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html
A simple Go function to demonstrate syntax highlighting.
```go
func increment(a int) int {
return a + 1
}
```
--------------------------------
### Install Tree-sitter CLI
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Install the Tree-sitter CLI globally on your system. This command makes the `tree-sitter` executable available in your PATH.
```bash
cargo install --path crates/cli
```
--------------------------------
### Install mdBook CLI
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Install the `mdbook` CLI tool, which is required to run and iterate on the documentation locally. Assumes `cargo` is installed.
```bash
cargo install mdbook
```
--------------------------------
### Install mdbook-admonish Preprocessor
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Install `mdbook-admonish`, a preprocessor for `mdBook` that enables fancy admonitions in the documentation. This is also a requirement for local development.
```bash
cargo install mdbook-admonish
```
--------------------------------
### Install Node.js dependencies for editor support
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html
Install the Tree-sitter's TypeScript API from npm to enable documentation and type information in your code editor. This is typically done using `npm install`.
```bash
npm install # or your package manager of choice
```
--------------------------------
### Serve Documentation Locally
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Navigate to the `docs` directory and start a local server for the documentation using `mdbook`. This command includes a live-reload feature.
```bash
cd docs
mdbook serve --open
```
--------------------------------
### Install Tree-sitter TypeScript API
Source: https://tree-sitter.github.io/tree-sitter/print.html
Install the Tree-sitter's TypeScript API from npm into a `node_modules` directory. This enables editor integration for documentation and type information within `grammar.js`.
```bash
npm install # or your package manager of choice
```
--------------------------------
### Install Tree-sitter CLI with Cargo
Source: https://tree-sitter.github.io/tree-sitter/print.html
Install the tree-sitter CLI from crates.io using Cargo, the Rust package manager. This method works on all platforms.
```bash
cargo install tree-sitter-cli --locked
```
--------------------------------
### Clone Tree-sitter Repository
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Clone the Tree-sitter repository and navigate into the directory. This is the first step to start contributing.
```bash
git clone https://github.com/tree-sitter/tree-sitter
cd tree-sitter
```
--------------------------------
### Example Return Statement
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html
A simple return statement in JavaScript.
```javascript
return x + y;
```
--------------------------------
### Run Tests for Specific Language and Example
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Filter tests to run only for a particular language and a specific example within that language's corpus. Use the `-l` flag for language and `-e` for example.
```bash
cargo xtask test -l javascript -e Arrays
```
--------------------------------
### Tree-sitter Playground Example
Source: https://tree-sitter.github.io/tree-sitter/7-playground.html
This is a placeholder for the actual playground code. The playground allows interactive testing of grammars and queries.
```placeholder
xxxxxxxxxx
```
--------------------------------
### Get Node Byte Positions
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html
Obtain the starting and ending byte offsets of a syntax node within the source code.
```c
uint32_t ts_node_start_byte(TSNode);
uint32_t ts_node_end_byte(TSNode);
```
--------------------------------
### Initialize New Grammar with tree-sitter init
Source: https://tree-sitter.github.io/tree-sitter/print.html
Use this command to start a new tree-sitter grammar project. It creates necessary configuration and boilerplate files. Recommended for use with Git version control.
```bash
tree-sitter init [OPTIONS] # Aliases: i
```
--------------------------------
### JavaScript Return Statement Example
Source: https://tree-sitter.github.io/tree-sitter/print.html
A basic JavaScript return statement used to illustrate grammar complexity. No special setup is required to use this snippet.
```javascript
return x + y;
```
--------------------------------
### Go Code Syntax Tree
Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html
The abstract syntax tree representation of the example Go code.
```tree-sitter
(source_file
(function_declaration
name: (identifier)
parameters: (parameter_list
(parameter_declaration
name: (identifier)
type: (type_identifier)))
result: (type_identifier)
body: (block
(return_statement
(expression_list
(binary_expression
left: (identifier)
right: (int_literal)))))))
```
--------------------------------
### Ruby Code Example for Highlighting
Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html
This Ruby code demonstrates various types of names including methods, formal parameters, local variables, and method calls.
```ruby
def process_list(list)
context = current_context
list.map do |item|
process_item(item, context)
end
end
item = 5
list = [item]
```
--------------------------------
### Example `tree-sitter.json` Grammar Configuration
Source: https://tree-sitter.github.io/tree-sitter/cli/init.html
This snippet shows a basic configuration for a single grammar within the `tree-sitter.json` file, specifying scope, file types, and a first-line regex.
```json
{
"tree-sitter": [
{
"scope": "source.ruby",
"file-types": [
"rb",
"gemspec",
"Gemfile",
"Rakefile"
],
"first-line-regex": "#!.*\\bruby$"
}
]
}
```
--------------------------------
### Start Tree-sitter Playground
Source: https://tree-sitter.github.io/tree-sitter/cli/playground.html
Use this command to launch a local playground for interactive parser testing. Ensure your parser is built as a Wasm module first using `tree-sitter build --wasm`.
```bash
tree-sitter playground [OPTIONS] # Aliases: play, pg, web-ui
```
--------------------------------
### Build Tree-sitter CLI with Release-Dev Profile
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Build the Tree-sitter CLI using the `release-dev` profile for faster build times during rapid iteration cycles. This can be used for both building and installing.
```bash
cargo build --profile release-dev
# or
cargo install --path crates/cli --profile release-dev
```
--------------------------------
### Basic grammar.js structure
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html
The `grammar.js` file defines the syntax rules for your parser. This is a minimal example with a single rule.
```javascript
/**
* @file PARSER_DESCRIPTION
* @author PARSER_AUTHOR_NAME PARSER_AUTHOR_EMAIL
* @license PARSER_LICENSE
*/
///
// @ts-check
export default grammar({
name: 'LOWER_PARSER_NAME',
rules: {
// TODO: add the actual grammar rules
source_file: $ => 'hello'
}
});
```
--------------------------------
### Example ERB Document
Source: https://tree-sitter.github.io/tree-sitter/print.html
An example of an Embedded Ruby (ERB) document, which mixes HTML structure with Ruby code for dynamic content generation.
```erb
<% people.each do |person| %>
- <%= person.name %>
<% end %>
```
--------------------------------
### Configure Parser Directories
Source: https://tree-sitter.github.io/tree-sitter/cli/init-config.html
Specify directories where the Tree-sitter CLI should look for grammar repositories. Folders starting with `tree-sitter-` within these paths are recognized as grammars.
```json
{
"parser-directories": [
"/Users/my-name/code",
"~/other-code",
"$HOME/another-code"
]
}
```
--------------------------------
### JavaScript Keyword Example
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html
Demonstrates the use of a JavaScript keyword `instanceof` in a conditional statement. This highlights the distinction between a keyword and an identifier.
```javascript
if (a instanceof Something) b();
```
--------------------------------
### Test Ruby Code with `tree-sitter tags`
Source: https://tree-sitter.github.io/tree-sitter/print.html
Use the `tree-sitter tags` command to test tag query files against source code. This example demonstrates tagging a Ruby file.
```ruby
module Foo
class Bar
# won't be included
# is adjacent, will be
def baz
end
end
end
```
```bash
tree-sitter tags test.rb
```
```text
test.rb
Foo | module def (0, 7) - (0, 10) `module Foo`
Bar | class def (1, 8) - (1, 11) `class Bar`
baz | method def (2, 8) - (2, 11) `def baz` "is adjacent, will be"
```
--------------------------------
### C Program to Parse JSON with Tree-sitter
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/1-getting-started.html
An example C program demonstrating how to use the Tree-sitter C API to parse a JSON string, inspect the syntax tree, and verify node properties.
```c
#include
#include
#include
#include
// Declare the `tree_sitter_json` function, which is
// implemented by the `tree-sitter-json` library.
const TSLanguage *tree_sitter_json(void);
int main() {
// Create a parser.
TSParser *parser = ts_parser_new();
// Set the parser's language (JSON in this case).
ts_parser_set_language(parser, tree_sitter_json());
// Build a syntax tree based on source code stored in a string.
const char *source_code = "[1, null]";
TSTree *tree = ts_parser_parse_string(
parser,
NULL,
source_code,
strlen(source_code)
);
// Get the root node of the syntax tree.
TSNode root_node = ts_tree_root_node(tree);
// Get some child nodes.
TSNode array_node = ts_node_named_child(root_node, 0);
TSNode number_node = ts_node_named_child(array_node, 0);
// Check that the nodes have the expected types.
assert(strcmp(ts_node_type(root_node), "document") == 0);
assert(strcmp(ts_node_type(array_node), "array") == 0);
assert(strcmp(ts_node_type(number_node), "number") == 0);
// Check that the nodes have the expected child counts.
assert(ts_node_child_count(root_node) == 1);
assert(ts_node_child_count(array_node) == 5);
assert(ts_node_named_child_count(array_node) == 2);
assert(ts_node_child_count(number_node) == 0);
// Print the syntax tree as an S-expression.
char *string = ts_node_string(root_node);
printf("Syntax tree: %s\n", string);
// Free all of the heap-allocated memory.
free(string);
ts_tree_delete(tree);
ts_parser_delete(parser);
return 0;
}
```
--------------------------------
### Initialize Tree Cursor
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/4-walking-trees.html
Initializes a tree cursor from a given TSNode. The cursor is stateful and starts at the provided node.
```c
TSTreeCursor ts_tree_cursor_new(TSNode);
```
--------------------------------
### Grammar Rule Example
Source: https://tree-sitter.github.io/tree-sitter/print.html
Illustrates a grammar rule definition for an if statement, showing how named and anonymous nodes are formed.
```javascript
if_statement: $ => seq("if", "(", $._expression, ")", $._statement);
```
--------------------------------
### JavaScript Syntax Highlighting Test
Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html
This example demonstrates how to write a syntax highlighting test for JavaScript. Comments with special markers like '<-' and '^' assert expected scopes at specific positions.
```javascript
__
var abc = function(d) {
// <- keyword
// ^ keyword
// ^ variable.parameter
// ^ function
if (a) {
// <- keyword
// ^ punctuation.bracket
foo(`foo ${bar}`);
// <- function
// ^ string
// ^ variable
}
baz();
// <- !variable
};
```
--------------------------------
### Tree Cursor Initialization
Source: https://tree-sitter.github.io/tree-sitter/print.html
Initializes a tree cursor from a given node. The cursor can then be used to efficiently walk the syntax tree starting from this node.
```APIDOC
## ts_tree_cursor_new
### Description
Initializes a new tree cursor starting from the provided node. The cursor can then be used to navigate the syntax tree efficiently.
### Function Signature
```c
TSTreeCursor ts_tree_cursor_new(TSNode);
```
### Parameters
- **node** (TSNode) - The starting node for the cursor.
```
--------------------------------
### Conditional Scanning Logic Example
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/4-external-scanners.html
Demonstrates how to conditionally scan for tokens like INDENT and DEDENT based on the `valid_symbols` array.
```APIDOC
## Conditional Scanning Logic
### Description
This example illustrates how to implement conditional logic within the `scan` function to recognize specific tokens only when they are expected by the parser.
### Example
```c
if (valid_symbols[INDENT] || valid_symbols[DEDENT]) {
// Logic common to both INDENT and DEDENT
if (valid_symbols[INDENT]) {
// Logic specific to INDENT
lexer->result_symbol = INDENT;
return true;
}
// Potentially logic for DEDENT here if not handled by INDENT block
}
```
### Behavior
The code checks if either `INDENT` or `DEDENT` symbols are valid. If `INDENT` is valid, it proceeds with the specific logic for `INDENT`, assigns the `INDENT` symbol to `lexer->result_symbol`, and returns `true`. This pattern ensures that scanning efforts are focused only on tokens the parser currently requires.
```
--------------------------------
### Parse Mixed Language Document with Tree-sitter
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/3-advanced-parsing.html
This example demonstrates parsing an ERB document by first parsing it as ERB, then identifying HTML and Ruby code sections using ranges. It then re-parses these sections with their respective languages (HTML and Ruby) and prints all three syntax trees.
```c
#include
#include
// These functions are each implemented in their own repo.
const TSLanguage *tree_sitter_embedded_template(void);
const TSLanguage *tree_sitter_html(void);
const TSLanguage *tree_sitter_ruby(void);
int main(int argc, const char **argv) {
const char *text = argv[1];
unsigned len = strlen(text);
// Parse the entire text as ERB.
TSParser *parser = ts_parser_new();
ts_parser_set_language(parser, tree_sitter_embedded_template());
TSTree *erb_tree = ts_parser_parse_string(parser, NULL, text, len);
TSNode erb_root_node = ts_tree_root_node(erb_tree);
// In the ERB syntax tree, find the ranges of the `content` nodes,
// which represent the underlying HTML, and the `code` nodes, which
// represent the interpolated Ruby.
TSRange html_ranges[10];
TSRange ruby_ranges[10];
unsigned html_range_count = 0;
unsigned ruby_range_count = 0;
unsigned child_count = ts_node_child_count(erb_root_node);
for (unsigned i = 0; i < child_count; i++) {
TSNode node = ts_node_child(erb_root_node, i);
if (strcmp(ts_node_type(node), "content") == 0) {
html_ranges[html_range_count++] = (TSRange) {
ts_node_start_point(node),
ts_node_end_point(node),
ts_node_start_byte(node),
ts_node_end_byte(node),
};
} else {
TSNode code_node = ts_node_named_child(node, 0);
ruby_ranges[ruby_range_count++] = (TSRange) {
ts_node_start_point(code_node),
ts_node_end_point(code_node),
ts_node_start_byte(code_node),
ts_node_end_byte(code_node),
};
}
}
// Use the HTML ranges to parse the HTML.
ts_parser_set_language(parser, tree_sitter_html());
ts_parser_set_included_ranges(parser, html_ranges, html_range_count);
TSTree *html_tree = ts_parser_parse_string(parser, NULL, text, len);
TSNode html_root_node = ts_tree_root_node(html_tree);
// Use the Ruby ranges to parse the Ruby.
ts_parser_set_language(parser, tree_sitter_ruby());
ts_parser_set_included_ranges(parser, ruby_ranges, ruby_range_count);
TSTree *ruby_tree = ts_parser_parse_string(parser, NULL, text, len);
TSNode ruby_root_node = ts_tree_root_node(ruby_tree);
// Print all three trees.
char *erb_sexp = ts_node_string(erb_root_node);
char *html_sexp = ts_node_string(html_root_node);
char *ruby_sexp = ts_node_string(ruby_root_node);
printf("ERB: %s\n", erb_sexp);
printf("HTML: %s\n", html_sexp);
printf("Ruby: %s\n", ruby_sexp);
return 0;
}
```
--------------------------------
### Test parser with a simple file (Linux/macOS)
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html
Create a file named `example-file` with the content 'hello' and then use `tree-sitter parse` to test your newly generated parser.
```bash
echo 'hello' > example-file
tree-sitter parse example-file
```
--------------------------------
### Get Node Character Positions
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html
Retrieve the starting and ending row and column coordinates of a syntax node. Rows and columns are zero-based.
```c
typedef struct {
uint32_t row;
uint32_t column;
} TSPoint;
TSPoint ts_node_start_point(TSNode);
TSPoint ts_node_end_point(TSNode);
```
--------------------------------
### Alternation for Call Expressions
Source: https://tree-sitter.github.io/tree-sitter/print.html
Use square brackets for alternations to match different patterns. This example captures either a variable or an object property as a function or method.
```tree-sitter query
(
call_expression
function: [
(identifier) @function
(member_expression
property: (property_identifier) @method)
])
```
--------------------------------
### Test parser with a simple file (Windows PowerShell)
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html
Create a file named `example-file` with the content 'hello' and then use `tree-sitter parse` to test your newly generated parser using PowerShell.
```powershell
"hello" | Out-File example-file -Encoding utf8
tree-sitter parse example-file
```
--------------------------------
### TSRange Structure Definition
Source: https://tree-sitter.github.io/tree-sitter/print.html
Defines the structure for specifying a range within a document, including start and end points and byte offsets. Used with `ts_parser_set_included_ranges`.
```c
typedef struct {
TSPoint start_point;
TSPoint end_point;
uint32_t start_byte;
uint32_t end_byte;
} TSRange;
void ts_parser_set_included_ranges(
TSParser *self,
const TSRange *ranges,
uint32_t range_count
);
```
--------------------------------
### Test Tree-sitter parser with a sample file (Windows PowerShell)
Source: https://tree-sitter.github.io/tree-sitter/print.html
Create a sample file named `example-file` with the content 'hello' using PowerShell and then use `tree-sitter parse` to test the generated parser. This is the Windows equivalent of the Unix/macOS test.
```powershell
__
"hello" | Out-File example-file -Encoding utf8
tree-sitter parse example-file
```
--------------------------------
### Capture Nodes in Class Declaration Pattern
Source: https://tree-sitter.github.io/tree-sitter/print.html
This example demonstrates capturing multiple nodes within a class declaration, including the class name and method names, using distinct capture names for each.
```tree-sitter grammar
(class_declaration
name: (identifier) @the-class-name
body: (class_body
(method_definition
name: (property_identifier) @the-method-name)))
```
--------------------------------
### Initialize Tree-sitter parser project
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html
Use the `tree-sitter init` command to set up the basic file structure for your parser project. This command will prompt you for necessary information.
```bash
tree-sitter init
```
--------------------------------
### Find Cgo comments in Go code using #eq? and #match?
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/queries/3-predicates-and-directives.html
This query finds C code embedded in Go comments that appear just before a 'C' import statement. It uses `#eq?` to ensure the import path is exactly '"C"' and `#match?` to check if the comment starts with '//'.
```tree-sitter query
((comment)+ @injection.content
.
(import_declaration
(import_spec path: (interpreted_string_literal) @_import_c))
(#eq? @_import_c "\"C\"")
(#match? @injection.content "^//"))
```
--------------------------------
### Initialize Tree-sitter Configuration
Source: https://tree-sitter.github.io/tree-sitter/cli/init-config.html
Run this command to create a default configuration file for the Tree-sitter CLI. It will print the location of the created file.
```bash
tree-sitter init-config
```
--------------------------------
### Set Injection Language with set! Directive
Source: https://tree-sitter.github.io/tree-sitter/print.html
Use the `set!` directive to associate a key-value pair with a pattern. This example sets the `injection.language` property to "doxygen" for Doxygen-style comments, enabling them to be parsed by a Doxygen parser.
```tree-sitter grammar
__
((comment) @injection.content
(#match? @injection.content "/[*\/][!*\/][^a-zA-Z]")
(#set! injection.language "doxygen"))
```
--------------------------------
### Build Tree-sitter Library with Make
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/1-getting-started.html
Use the `make` command to build the static library `libtree-sitter.a` and dynamic libraries on POSIX systems.
```bash
make
```
--------------------------------
### Using Supertypes to Hide Abstract Nodes in Tree-sitter
Source: https://tree-sitter.github.io/tree-sitter/print.html
This example demonstrates how to configure a Tree-sitter grammar to treat the 'expression' rule as a supertype, preventing it from generating a visible node in the syntax tree. This is achieved by including '$.expression' in the grammar's 'supertypes' definition.
```javascript
module.exports = grammar({
name: "javascript",
supertypes: $ => [
$.expression,
],
rules: {
expression: $ => choice(
$.identifier,
// ...
),
// ...
},
});
```
--------------------------------
### Use Alternative Configuration File
Source: https://tree-sitter.github.io/tree-sitter/cli/test.html
Specify a path to an alternative `config.json` file for test configuration.
```bash
tree-sitter test --config-path
```
--------------------------------
### Get Root Node
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html
Access the root node of a given syntax tree.
```c
TSNode ts_tree_root_node(const TSTree *);
```
--------------------------------
### Specify Configuration Path
Source: https://tree-sitter.github.io/tree-sitter/print.html
Provide an alternative configuration file using the `--config-path` option. Refer to the `init-config` command for more details.
```bash
tree-sitter highlight --config-path [PATHS]...
```
--------------------------------
### Access Node Children
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html
Get the number of children a node has and retrieve a specific child by its index.
```c
uint32_t ts_node_child_count(TSNode);
TSNode ts_node_child(TSNode, uint32_t);
```
--------------------------------
### Get Node Type
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html
Retrieve the string type of a syntax node, which corresponds to a grammar rule.
```c
const char *ts_node_type(TSNode);
```
--------------------------------
### Tree-sitter Highlight Command Usage
Source: https://tree-sitter.github.io/tree-sitter/cli/highlight.html
Basic command structure for running syntax highlighting on files. Supports options for HTML output and other configurations.
```bash
tree-sitter highlight [OPTIONS] [PATHS]...
```
--------------------------------
### Build Wasm Library
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Build the WebAssembly library for Tree-sitter. This step is optional but recommended to avoid internet dependency for the playground. It uses Node.js and npm.
```bash
cd lib/binding_web
npm install # or your JS package manager of choice
npm run build
```
--------------------------------
### Get Current Node and Field Info
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/4-walking-trees.html
Retrieves the current node the cursor is on, along with its associated field name and ID.
```c
TSNode ts_tree_cursor_current_node(const TSTreeCursor *);
const char *ts_tree_cursor_current_field_name(const TSTreeCursor *);
TSFieldId ts_tree_cursor_current_field_id(const TSTreeCursor *);
```
--------------------------------
### Create a new parser directory
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/1-getting-started.html
Initialize a new directory for your Tree-sitter parser project. Replace `${LOWER_PARSER_NAME}` with the lowercase name of the language you are creating a parser for.
```bash
mkdir tree-sitter-${LOWER_PARSER_NAME}
cd tree-sitter-${LOWER_PARSER_NAME}
```
--------------------------------
### Capture Node in Assignment Expression
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/queries/2-operators.html
Associates a name with a specific node within a pattern. Capture names start with '@'.
```tree-sitter query
(assignment_expression
left: (identifier) @the-function-name
right: (function))
```
--------------------------------
### Create a new Tree-sitter parser project
Source: https://tree-sitter.github.io/tree-sitter/print.html
Initialize a new Tree-sitter parser project. The `LOWER_PARSER_NAME` should be replaced with the lowercase name of the language for which the parser is being developed.
```bash
__
mkdir tree-sitter-${LOWER_PARSER_NAME}
cd tree-sitter-${LOWER_PARSER_NAME}
```
--------------------------------
### Specify Query Paths for Highlighting
Source: https://tree-sitter.github.io/tree-sitter/print.html
Indicate the paths to query files (ending in `highlights.scm`) to be used for syntax highlighting via the `--query-paths` option.
```bash
tree-sitter highlight --query-paths [PATHS]...
```
--------------------------------
### Conditional Token Recognition (INDENT/DEDENT)
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/4-external-scanners.html
Example of checking `valid_symbols` to conditionally recognize INDENT or DEDENT tokens, prioritizing INDENT if both are valid.
```c
if (valid_symbols[INDENT] || valid_symbols[DEDENT]) {
// ... logic that is common to both `INDENT` and `DEDENT`
if (valid_symbols[INDENT]) {
// ... logic that is specific to `INDENT`
lexer->result_symbol = INDENT;
return true;
}
}
```
--------------------------------
### Node Type and Position
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html
Retrieve the type of a syntax node and its start and end positions in terms of bytes and character coordinates (row/column).
```APIDOC
## Node Type and Position
### Description
Get the type of a syntax node and its byte/character positions.
### Functions
- `ts_node_type(TSNode)`: Returns the string type of the node.
- `ts_node_start_byte(TSNode)`: Returns the starting byte position of the node.
- `ts_node_end_byte(TSNode)`: Returns the ending byte position of the node.
- `ts_node_start_point(TSNode)`: Returns the starting point (row, column) of the node.
- `ts_node_end_point(TSNode)`: Returns the ending point (row, column) of the node.
### Types
- `TSNode`: Represents a node in the syntax tree.
- `TSPoint`: A struct containing `row` and `column` (uint32_t) for character coordinates.
```
--------------------------------
### Tree-sitter Language Field Utilities
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html
Utilities to get the count of fields, the name of a field by its ID, and the ID of a field by its name from a TSLanguage object.
```c
uint32_t ts_language_field_count(const TSLanguage *);
const char *ts_language_field_name_for_id(const TSLanguage *, TSFieldId);
TSFieldId ts_language_field_id_for_name(const TSLanguage *, const char *, uint32_t);
```
--------------------------------
### Configuration for Highlight Name to Color Mapping
Source: https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html
A JSON configuration object mapping highlight names to colors for syntax highlighting.
```json
{
"theme": {
"keyword": "purple",
"function": "blue",
"type": "green",
"number": "brown"
}
}
```
--------------------------------
### Display Verbose Build Information
Source: https://tree-sitter.github.io/tree-sitter/cli/build.html
Use the `-v` or `--verbose` flag to see detailed build information, including the working directory, compiler, arguments, and environment variables used during the build process.
```bash
tree-sitter build -v
```
--------------------------------
### Get Child Node by Field Name
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html
Access a child node of a given TSNode using its field name. Requires the field name and its length.
```c
TSNode ts_node_child_by_field_name(
TSNode self,
const char *field_name,
uint32_t field_name_length
);
```
--------------------------------
### Build Wasm Standard Library
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Build the Tree-sitter Wasm standard library using `xtask`. This process requires the Wasi SDK and `wasm-opt` tool, specified via environment variables.
```bash
cargo xtask build-wasm-stdlib
```
--------------------------------
### Define `word` Token in JavaScript Grammar
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html
This example shows how to define an `identifier` as the `word` token in a JavaScript grammar. Tree-sitter uses this to optimize keyword matching.
```javascript
grammar({
name: "javascript",
word: $ => $.identifier,
rules: {
expression: $ =>
choice(
$.identifier,
$.unary_expression,
$.binary_expression,
// ...
),
binary_expression: $ =>
choice(
prec.left(1, seq($.expression, "instanceof", $.expression)),
// ...
),
unary_expression: $ =>
choice(
prec.left(2, seq("typeof", $.expression)),
// ...
),
identifier: $ => /[a-z_]+/
},
});
```
--------------------------------
### Compile and Run JSON Parser with Static Linking
Source: https://tree-sitter.github.io/tree-sitter/print.html
This command compiles the C program using static linking with the Tree-sitter library. It requires the Tree-sitter API headers, the JSON parser source, and the static Tree-sitter library. After compilation, the executable is run.
```bash
clang \
-I tree-sitter/lib/include \
test-json-parser.c \
tree-sitter-json/src/parser.c \
tree-sitter/libtree-sitter.a \
-o test-json-parser
./test-json-parser
```
--------------------------------
### Set Injection Language with Doxygen Directive
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/queries/3-predicates-and-directives.html
Use the `set!` directive to associate key-value pairs with a pattern. This example sets the `injection.language` to 'doxygen' for Doxygen-style comments.
```scm
((comment) @injection.content
(#match? @injection.content "/[*][/][!*][/]?[^a-zA-Z]")
(#set! injection.language "doxygen"))
```
--------------------------------
### Compile and Run JSON Parser with Dynamic Linking
Source: https://tree-sitter.github.io/tree-sitter/print.html
This command compiles the C program using dynamic linking with the Tree-sitter library. It requires the Tree-sitter API headers and the JSON parser source, linking against the shared Tree-sitter library. Ensure the shared library is discoverable via `LD_LIBRARY_PATH`.
```bash
clang \
-I tree-sitter/lib/include \
test-json-parser.c \
tree-sitter-json/src/parser.c \
-ltree-sitter \
-o test-json-parser
./test-json-parser
```
--------------------------------
### Tree-sitter Query Command Usage
Source: https://tree-sitter.github.io/tree-sitter/cli/query.html
The basic syntax for using the `tree-sitter query` command. It takes a query path and one or more file paths as arguments.
```bash
tree-sitter query [OPTIONS] [PATHS]...
```
--------------------------------
### Get Child Node by Field ID
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/2-basic-parsing.html
Access a child node of a given TSNode using its numeric field ID. This can be more efficient than using field names directly.
```c
TSNode ts_node_child_by_field_id(TSNode, TSFieldId);
```
--------------------------------
### Initial Go Grammar Skeleton
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html
This snippet shows an initial, high-level grammar structure for a language like Go, focusing on major groupings of symbols such as definitions, functions, types, blocks, statements, and expressions. It uses TODO comments to indicate areas for future expansion.
```javascript
{
// ...
rules: {
source_file: $ => repeat($._definition),
_definition: $ => choice(
$.function_definition
// TODO: other kinds of definitions
),
function_definition: $ => seq(
'func',
$.identifier,
$.parameter_list,
$._type,
$.block
),
parameter_list: $ => seq(
'('
// TODO: parameters
')'
),
_type: $ => choice(
'bool'
// TODO: other kinds of types
),
block: $ => seq(
'{',
repeat($._statement),
'}'
),
_statement: $ => choice(
$.return_statement
// TODO: other kinds of statements
),
return_statement: $ => seq(
'return',
$.expression,
';'
),
expression: $ => choice(
$.identifier,
$.number
// TODO: other kinds of expressions
),
identifier: $ => /[a-z]+/,
number: $ => /\d+/
}
}
```
--------------------------------
### Compile C Program with Static Linking
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/1-getting-started.html
Compile the C program using clang, including necessary header paths and linking against the static Tree-sitter library and the JSON parser source.
```bash
clang \
-I tree-sitter/lib/include \
test-json-parser.c \
tree-sitter-json/src/parser.c \
tree-sitter/libtree-sitter.a \
-o test-json-parser
./test-json-parser
```
--------------------------------
### Resolve Parsing Conflicts with Precedence
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html
Uses the `prec` function to assign precedence levels to grammar rules, resolving ambiguities. This example gives higher precedence to the unary minus operator.
```javascript
{
// ...
unary_expression: $ =>
prec(
2,
choice(
seq("-", $.expression),
seq("!", $.expression),
// ...
),
);
}
```
--------------------------------
### Build Rust Libraries and CLI
Source: https://tree-sitter.github.io/tree-sitter/6-contributing.html
Compile the Rust libraries and the command-line interface (CLI) for Tree-sitter using Cargo. The release build is optimized for performance.
```bash
cargo build --release
```
--------------------------------
### Define Included Ranges for Parsing
Source: https://tree-sitter.github.io/tree-sitter/using-parsers/3-advanced-parsing.html
Use TSRange to specify the start and end points, and bytes, for sections of text to be parsed. This is crucial for handling embedded languages within a larger document.
```c
typedef struct {
TSPoint start_point;
TSPoint end_point;
uint32_t start_byte;
uint32_t end_byte;
} TSRange;
void ts_parser_set_included_ranges(
TSParser *self,
const TSRange *ranges,
uint32_t range_count
);
```
--------------------------------
### Invalid JavaScript Identifier Example
Source: https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html
Shows an invalid JavaScript syntax where a keyword is immediately followed by characters that would form an identifier. Tree-sitter would normally tokenize this as two separate tokens.
```javascript
if (a instanceofSomething) b();
```