### Install Dependencies and Build WASM Source: https://github.com/tree-sitter/node-tree-sitter/wiki/Native-is-twice-as-fast-as-Wasm This shell script sets up a directory, initializes a Node.js project with ES module support, installs tree-sitter and related packages, and builds the WebAssembly target for tree-sitter-python. It's a prerequisite for running the benchmark. ```shell mkdir /tmp/tree-sitter-vs-node-tree-sitter cd /tmp/tree-sitter-vs-node-tree-sitter npm init -y # add "type": "module", ``` ```shell npm install tree-sitter tree-sitter-cli web-tree-sitter tree-sitter-python npx tree-sitter build-wasm node_modules/tree-sitter-python ``` -------------------------------- ### Install Node Tree-sitter Source: https://github.com/tree-sitter/node-tree-sitter/blob/master/README.md Installs the core Node.js package for the tree-sitter parsing library using npm. This is the first step to using tree-sitter in a Node.js project. ```sh npm install tree-sitter ``` -------------------------------- ### Install Tree-sitter Grammar Source: https://github.com/tree-sitter/node-tree-sitter/blob/master/README.md Installs a specific Tree-sitter grammar for a language, such as JavaScript, using npm. This grammar is required by the parser to understand the syntax of the target language. ```sh npm install tree-sitter-javascript ``` -------------------------------- ### Simple Usage of tree-sitter Parser Source: https://github.com/tree-sitter/node-tree-sitter/wiki/Starter Demonstrates the basic steps to initialize the tree-sitter parser, set the language, and parse source code. ```javascript const Parser = require('tree-sitter'); const JavaScript = require('tree-sitter-javascript'); const parser = new Parser(); parser.setLanguage(JavaScript); const sourceCode = 'let x = 1; console.log(x);'; const tree = parser.parse(sourceCode); ``` -------------------------------- ### Prepare Benchmark Data Source: https://github.com/tree-sitter/node-tree-sitter/wiki/Native-is-twice-as-fast-as-Wasm This shell script clones the Python CPython repository and concatenates all Python files (`*.py`) from the Lib directory into a single file named `sample_python_code.py`. This file will be used as the input for parsing benchmarks. ```shell git clone https://github.com/python/cpython.git #cd cpython && checkout f508800 && cd .. find cpython/Lib/ -type f -name "*.py" -exec cat {} + > sample_python_code.py ``` -------------------------------- ### Initialize Parser with Language Source: https://github.com/tree-sitter/node-tree-sitter/blob/master/README.md Demonstrates how to create a new Tree-sitter parser instance and set the language grammar to be used for parsing. This prepares the parser for processing source code. ```javascript const Parser = require('tree-sitter'); const JavaScript = require('tree-sitter-javascript'); const parser = new Parser(); parser.setLanguage(JavaScript); ``` -------------------------------- ### Web-Tree-Sitter (WASM) Parsing Benchmark Source: https://github.com/tree-sitter/node-tree-sitter/wiki/Native-is-twice-as-fast-as-Wasm This JavaScript code utilizes `web-tree-sitter` to parse the `sample_python_code.py` file using WebAssembly. It initializes the WASM parser, loads the Python language grammar, and measures the parsing time, comparing it against the native Node.js version. ```javascript import Parser from "web-tree-sitter"; import fs from "fs"; await Parser.init(); const Python = await Parser.Language.load("./tree-sitter-python.wasm"); const parser = new Parser(); parser.setLanguage(Python); const file = fs.readFileSync("./sample_python_code.py", "utf8"); const start = process.hrtime.bigint(); parser.parse(file); const end = process.hrtime.bigint(); console.log(`web-tree-sitter parse time: ${(end - start) / 1000000n}ms`); ``` -------------------------------- ### Inspect Syntax Tree Source: https://github.com/tree-sitter/node-tree-sitter/blob/master/README.md Shows how to inspect the generated syntax tree, including printing its string representation and accessing specific nodes. This helps in understanding the parsed code structure. ```javascript console.log(tree.rootNode.toString()); // Example output: // (program // (lexical_declaration // (variable_declarator (identifier) (number))) // (expression_statement // (call_expression // (member_expression (identifier) (property_identifier)) // (arguments (identifier))))) const callExpression = tree.rootNode.child(1).firstChild; console.log(callExpression); // Example output: // { // type: 'call_expression', // startPosition: {row: 0, column: 16}, // endPosition: {row: 0, column: 30}, // startIndex: 0, // endIndex: 30 // } ``` -------------------------------- ### Tree Navigation API Source: https://github.com/tree-sitter/node-tree-sitter/wiki/Starter Provides details on accessing and inspecting the parsed syntax tree structure, including the root node and its children. ```APIDOC tree.rootNode - Accesses the root node of the parsed syntax tree. - Returns: An object representing the root node, typically a 'ProgramNode'. - Properties: - type: string (e.g., 'program') - startPosition: {row: number, column: number} - endPosition: {row: number, column: number} - childCount: number - Example: ProgramNode { type: program, startPosition: {row: 0, column: 0}, endPosition: {row: 0, column: 26}, childCount: 2, } tree.rootNode.children - Accesses an array containing the direct child nodes of the root node. - Returns: An array of node objects. - Example: [ LexicalDeclarationNode { type: lexical_declaration, startPosition: {row: 0, column: 0}, endPosition: {row: 0, column: 10}, childCount: 3, }, ExpressionStatementNode { type: expression_statement, startPosition: {row: 0, column: 11}, endPosition: {row: 0, column: 26}, childCount: 2, } ] ``` -------------------------------- ### Update Syntax Tree Efficiently Source: https://github.com/tree-sitter/node-tree-sitter/blob/master/README.md Demonstrates how to efficiently update an existing syntax tree when the source code changes, rather than re-parsing the entire code from scratch. This involves providing edit details like indices and positions. ```javascript // In the code, we replaced 'let' with 'const'. // So, we set our old end index to 3, and our new end index to 5. // Note that the end index is exclusive. const newSourceCode = 'const x = 1; console.log(x);'; // ^ ^ // indices: 3 5 // points: (0,3) (0,5) tree.edit({ startIndex: 0, oldEndIndex: 3, newEndIndex: 5, startPosition: {row: 0, column: 0}, oldEndPosition: {row: 0, column: 3}, newEndPosition: {row: 0, column: 5}, }); const newTree = parser.parse(newSourceCode, tree); ``` -------------------------------- ### Node-Tree-Sitter Parsing Benchmark Source: https://github.com/tree-sitter/node-tree-sitter/wiki/Native-is-twice-as-fast-as-Wasm This JavaScript code uses the native `tree-sitter` library in Node.js to parse the `sample_python_code.py` file. It measures and logs the time taken for the parsing operation, demonstrating the performance of the native implementation. ```javascript import Parser from "tree-sitter"; import Python from "tree-sitter-python"; import fs from "fs"; const parser = new Parser(); parser.setLanguage(Python); const file = fs.readFileSync("./sample_python_code.py", "utf8"); const start = process.hrtime.bigint(); parser.parse(file); const end = process.hrtime.bigint(); console.log(`node-tree-sitter parse time: ${(end - start) / 1000000n}ms`); ``` -------------------------------- ### Parse Source Code Source: https://github.com/tree-sitter/node-tree-sitter/blob/master/README.md Parses a given string of source code using the configured Tree-sitter parser. The result is a syntax tree that represents the structure of the code. ```javascript const sourceCode = 'let x = 1; console.log(x);'; const tree = parser.parse(sourceCode); ``` -------------------------------- ### Parse from Custom Data Structure Source: https://github.com/tree-sitter/node-tree-sitter/blob/master/README.md Explains how to parse source code when it's stored in a data structure other than a single string, such as an array of lines. This is achieved by providing a callback function to the `parse` method. ```javascript const sourceLines = [ 'let x = 1;', 'console.log(x);' ]; const tree = parser.parse((index, position) => { let line = sourceLines[position.row]; if (line) { return line.slice(position.column); } }); ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.