### Starlark Dictionary Comprehensions Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Provides examples of dictionary comprehensions in Starlark, demonstrating how to create dictionaries from iterables with optional filtering. ```Starlark {a: b for a, b in items} ``` ```Starlark {a: b for c in d for e in items} ``` -------------------------------- ### Starlark Set Comprehensions Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Illustrates Starlark set comprehensions, which create sets dynamically. Examples cover comprehensions with nested loops and complex element expressions. ```Starlark {a[b][c] for a, b, c in items} ``` ```Starlark {r for s in qs for n in ms} ``` ```APIDOC (module (expression_statement (set_comprehension (subscript (subscript (identifier) (identifier)) (identifier)) (for_in_clause (pattern_list (identifier) (identifier) (identifier)) (identifier)))) (expression_statement (set_comprehension (identifier) (for_in_clause (identifier) (identifier)) (for_in_clause (identifier) (identifier))))) ``` -------------------------------- ### Starlark Simple Tuples Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Provides examples of Starlark tuple syntax, including empty tuples, tuples with elements, and tuples with trailing commas. The AST shows how these are parsed. ```Starlark () ``` ```Starlark (a, b) ``` ```Starlark (a, b, c,) ``` ```Starlark (print, exec) ``` ```APIDOC (module (expression_statement (tuple)) (expression_statement (tuple (identifier) (identifier))) (expression_statement (tuple (identifier) (identifier) (identifier))) (expression_statement (tuple (identifier) (identifier)))) ``` -------------------------------- ### Starlark Assignments with Type Annotations and AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Shows an example of Starlark assignment with type annotations, demonstrating how a variable is declared with its type and initialized. The AST captures this structure. ```starlark tail_leaves: List[Leaf] = [] ``` ```tree-sitter-grammar (module (expression_statement (assignment (identifier) (type (identifier) (type_arguments (identifier)))) (list)))) ``` -------------------------------- ### Starlark Lambda Expressions Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Provides examples of lambda (anonymous function) expressions in Starlark, including variations with parameters, default arguments, and tuple unpacking. ```Python lambda b, c: d("e" % f) lambda: True lambda a, b = c, *d, **e: a lambda (a, b): (a, b) ``` -------------------------------- ### Starlark Dictionaries Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Shows different ways to define dictionaries in Starlark, including empty dictionaries, literal key-value pairs, and dictionary splats. ```Starlark {a: 1, b: 2} ``` ```Starlark {} ``` ```Starlark {**{}} ``` ```Starlark {**a} ``` ```Starlark {**a.b} ``` ```Starlark {**a[b].c} ``` ```Starlark {**a()} ``` -------------------------------- ### Starlark Lists Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Demonstrates various ways to define and use lists in Starlark, including nested lists and list splats for unpacking elements. ```Starlark [a, b, [c, d]] ``` ```Starlark [*()] ``` ```Starlark [*[]] ``` ```Starlark [*a] ``` ```Starlark [*a.b] ``` ```Starlark [*a[b].c] ``` ```Starlark [*a()] ``` -------------------------------- ### Starlark List Comprehensions Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Illustrates the syntax and usage of list comprehensions in Starlark, including nested loops and conditional filtering. ```Starlark [a + b for (a, b) in items] ``` ```Starlark [a for b in c for a in b] ``` ```Starlark [(x,y) for x in [1,2,3] for y in [1,2,3] if True] ``` ```Starlark [a for a in lambda: True, lambda: False if a()] ``` -------------------------------- ### Starlark Sets Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Demonstrates the syntax for creating sets in Starlark, including sets with elements and empty sets. The AST representation shows the structure of these set literals. ```Starlark {a, b, c,} ``` ```Starlark {*{}} ``` ```APIDOC (module (expression_statement (set (identifier) (identifier) (identifier))) (expression_statement (set (list_splat (dictionary))))) ``` -------------------------------- ### Starlark Expression Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Provides examples of Starlark expression statements, including identifiers, binary operations, and comma-separated lists. ```starlark a b + c 1, 2, 3 1, 2, 3, ``` -------------------------------- ### Starlark Multi-line Strings Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Covers the creation of multi-line strings in Starlark using triple double quotes (`"""`) and triple single quotes (`'''`). Includes examples with escaped newlines and embedded quotes. ```starlark """ A double quote hello, without double or single quotes. """ ``` ```starlark """ A double quote \"hello\", with double quotes. """ ``` ```starlark """ A double quote 'hello', with single quotes. """ ``` ```starlark ''' A single quote hello, without double or single quotes. ''' ``` ```starlark ''' A single quote 'hello', with single quotes. ''' ``` ```starlark ''' A single quote "hello", with double quotes. ''' ``` ```starlark """ A double quote hello\n\ with an escaped newline\n\ and another escaped newline\n\ """ ``` -------------------------------- ### Starlark 'exec' Identifier Usage Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Demonstrates the use of the 'exec' keyword in Starlark, including examples with string formatting and nested exec calls. The grammar captures the structure of these statements and their arguments. ```starlark exec("print '%s' has %i characters" % (public_function(), len(public_function()))", {"__builtins__" : None}, safe_dict) exec("\"exec _code_ in _globs_, _locs_\"") ``` ```tree-sitter-sexp (module (expression_statement (call (identifier) (argument_list (string (string_start) (string_content (escape_sequence) (escape_sequence)) (string_end)) (dictionary (pair (string (string_start) (string_content) (string_end)) (none))) (identifier)))) (expression_statement (call (identifier) (argument_list (string (string_start) (string_content) (string_end)))))) ``` -------------------------------- ### Starlark Print Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates basic Starlark print statements with single and multiple arguments, including boolean operations. ```starlark print a print b, c print 0 or 1, 1 or 0, print 0 or 1 ``` -------------------------------- ### Starlark Print Statements with Redirection Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates Starlark print statements that redirect output using the '>>' operator, supporting multiple arguments. ```starlark print >> a print >> a, "b", "c" ``` -------------------------------- ### Starlark While Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates Starlark while loops, including those with and without an optional 'else' block. Provides Python code and its AST. ```python while a: b ``` ```python while c: d else: e f ``` ```tree-sitter-grammar (module (while_statement condition: (identifier) body: (block (expression_statement (identifier)))) (while_statement condition: (identifier) body: (block (expression_statement (identifier))) alternative: (else_clause body: (block (expression_statement (identifier)) (expression_statement (identifier)))))) ``` -------------------------------- ### Starlark Async Function Definitions Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Covers various Starlark async function definitions, including parameters, type hints, and return types. Provides Python code and its AST. ```python async def a(): b ``` ```python async def c(d): e ``` ```python async def g(g, h,): i ``` ```python async def c(a: str): a ``` ```python async def c(a: b.c): a ``` ```python async def d(a: Sequence[T]) -> T: a ``` ```python async def i(a, b=c, *c, **d): a ``` ```python async def d(a: str) -> None: return None ``` ```tree-sitter-grammar (module (expression_statement (call (identifier) (argument_list (identifier)))) (expression_statement (call (identifier) (argument_list (identifier)))) (expression_statement (call (identifier) (argument_list (identifier) (identifier)))) (expression_statement (call (identifier) (argument_list (assignment_expression (identifier) (type_annotation (identifier))))))) (expression_statement (call (identifier) (argument_list (assignment_expression (identifier) (type_annotation (attribute (identifier) (identifier)))))))) (expression_statement (call (identifier) (argument_list (assignment_expression (identifier) (type_annotation (generic_type (identifier) (type_arguments (identifier)))))))) (expression_statement (call (identifier) (argument_list (identifier) (default_argument (identifier) (identifier)) (unpack_arguments (identifier)) (unpack_arguments (identifier))))) (expression_statement (call (identifier) (argument_list (assignment_expression (identifier) (type_annotation (identifier)))) (return_statement (none)))))) ``` -------------------------------- ### Starlark If-Else Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Provides examples of Starlark conditional logic, including if-elif-else structures, simple if-else, and inline if statements. ```starlark if a: b elif c: d else: f if a: b else: f if a: b if a: b; c ``` -------------------------------- ### Starlark With Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Details Starlark 'with' statements for context management, including multiple context managers and 'as' assignments. Provides Python code and its AST. ```python with a as b: c ``` ```python with (open('d') as d, open('e') as e): f ``` ```python with e as f, g as h,: i ``` ```tree-sitter-grammar (module (with_statement (with_clause (with_item (as_pattern (identifier) (as_pattern_target (identifier))))) (block (expression_statement (identifier)))) (with_statement (with_clause (with_item (as_pattern (call (identifier) (argument_list (string (string_start) (string_content) (string_end)))) (as_pattern_target (identifier)))) (with_item (as_pattern (call (identifier) (argument_list (string (string_start) (string_content) (string_end)))) (as_pattern_target (identifier))))) (block (expression_statement (identifier)))) (with_statement (with_clause (with_item (as_pattern (identifier) (as_pattern_target (identifier)))) (with_item (as_pattern (identifier) (as_pattern_target (identifier))))) (block (expression_statement (identifier))))) ``` -------------------------------- ### Starlark If Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates a basic Starlark if statement with a condition and a consequence block containing multiple statements. ```starlark if a: b c ``` -------------------------------- ### Starlark Lambda Expressions Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Shows the tree-sitter grammar structure for Starlark lambda expressions, detailing how parameters and function bodies are represented. ```APIDOC (module (expression_statement (lambda (lambda_parameters (identifier) (identifier)) (call (identifier) (argument_list (binary_operator (string (string_start) (string_content) (string_end)) (identifier)))))) (expression_statement (lambda (true))) (expression_statement (lambda (lambda_parameters (identifier) (default_parameter (identifier) (identifier)) (list_splat_pattern (identifier)) (dictionary_splat_pattern (identifier))) (identifier))) (expression_statement (lambda (lambda_parameters (tuple_pattern (identifier) (identifier))) (tuple (identifier) (identifier))))) ``` -------------------------------- ### Starlark For Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows Starlark for loops, including nested loops and loops with an 'else' clause. Includes Python code and its AST representation. ```python for line, i in lines: print line for character, j in line: print character else: print x ``` ```python for x, in [(1,), (2,), (3,)]: x ``` ```tree-sitter-grammar (module (for_statement left: (pattern_list (identifier) (identifier)) right: (identifier) body: (block (print_statement argument: (identifier)) (for_statement left: (pattern_list (identifier) (identifier)) right: (identifier) body: (block (print_statement argument: (identifier))))) alternative: (else_clause body: (block (print_statement argument: (identifier))))) (for_statement left: (pattern_list (identifier)) right: (list (tuple (integer)) (tuple (integer)) (tuple (integer))) body: (block (expression_statement (identifier))))) ``` -------------------------------- ### Define Function with Variadic Keyword Arguments (**kwargs) Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates a Python function accepting an arbitrary number of keyword arguments using the **kwargs syntax, with type hinting. ```python def e(**list: str): pass ``` -------------------------------- ### Starlark Augmented Assignments Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Represents the tree-sitter grammar structure for augmented assignment statements in Starlark, showing how these operations are parsed. ```APIDOC (module (expression_statement (augmented_assignment (identifier) (integer))) (expression_statement (augmented_assignment (identifier) (integer))) (expression_statement (augmented_assignment (identifier) (integer)))) ``` -------------------------------- ### Define Function with Variadic Positional Arguments (*args) Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates a Python function accepting an arbitrary number of positional arguments using the *args syntax, with type hinting. ```python def e(*list: str): pass ``` -------------------------------- ### Starlark Assert Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows Starlark assert statements, used for debugging and validating conditions, with single or multiple arguments. ```starlark assert a assert b, c ``` -------------------------------- ### Starlark Raw Strings Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Demonstrates Starlark raw string literals, which treat backslashes literally, useful for regular expressions or file paths. ```starlark 'ab\x00cd' "\n" ``` -------------------------------- ### Starlark String Literals Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Provides examples of Starlark string literals, including single quotes, double quotes, raw strings (UR''), bytes strings (b''), f-strings, multiline strings, and strings with various escape sequences. ```starlark "I'm ok" '"ok"' UR'bye' b'sup' B"sup" `1` "\\" "/" "multiline \ string" b"\x12\u12\U12\x13\N{WINKING FACE}" "\xab\123'\"\a\b\f\r\n\t\v\\" "\xgh\o123\p\q\c\d\e\u12\U1234" ``` -------------------------------- ### Define Async Function with Type Hints and Defaults Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates an asynchronous function definition in Starlark with type hints for parameters and a default value, returning None. ```python async def d(a:str="default", b=c) -> None: return None ``` -------------------------------- ### Starlark Return Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows various forms of Starlark return statements, including returning nothing, multiple values, or the result of an expression. ```starlark return return a + b, c return not b ``` -------------------------------- ### tree-sitter-starlark CMake Build Configuration Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/CMakeLists.txt This CMakeLists.txt file configures the build process for the tree-sitter-starlark grammar. It sets up the project, defines build options like shared libraries and allocator reuse, finds the tree-sitter CLI, generates the parser.c file from grammar.json, defines the library target, sets compile definitions and target properties, configures installation paths, and adds a custom target for running tests. ```CMake cmake_minimum_required(VERSION 3.13) project(tree-sitter-starlark VERSION "1.3.0" DESCRIPTION "Starlark grammar for tree-sitter" HOMEPAGE_URL "https://github.com/tree-sitter-grammars/tree-sitter-starlark" LANGUAGES C) option(BUILD_SHARED_LIBS "Build using shared libraries" ON) option(TREE_SITTER_REUSE_ALLOCATOR "Reuse the library allocator" OFF) set(TREE_SITTER_ABI_VERSION 14 CACHE STRING "Tree-sitter ABI version") if(NOT ${TREE_SITTER_ABI_VERSION} MATCHES "^[0-9]+$") unset(TREE_SITTER_ABI_VERSION CACHE) message(FATAL_ERROR "TREE_SITTER_ABI_VERSION must be an integer") endif() find_program(TREE_SITTER_CLI tree-sitter DOC "Tree-sitter CLI") add_custom_command(OUTPUT "${CMAKE_CURRENT_SOURCE_DIR}/src/parser.c" DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/grammar.json" COMMAND "${TREE_SITTER_CLI}" generate src/grammar.json --abi=${TREE_SITTER_ABI_VERSION} WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}" COMMENT "Generating parser.c") add_library(tree-sitter-starlark src/parser.c) if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/src/scanner.c) target_sources(tree-sitter-starlark PRIVATE src/scanner.c) endif() target_include_directories(tree-sitter-starlark PRIVATE src) target_compile_definitions(tree-sitter-starlark PRIVATE $<$:TREE_SITTER_REUSE_ALLOCATOR> $<$:TREE_SITTER_DEBUG>) set_target_properties(tree-sitter-starlark PROPERTIES C_STANDARD 11 POSITION_INDEPENDENT_CODE ON SOVERSION "${TREE_SITTER_ABI_VERSION}.${PROJECT_VERSION_MAJOR}" DEFINE_SYMBOL "") configure_file(bindings/c/tree-sitter-starlark.pc.in "${CMAKE_CURRENT_BINARY_DIR}/tree-sitter-starlark.pc" @ONLY) include(GNUInstallDirs) install(FILES bindings/c/tree-sitter-starlark.h DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}/tree_sitter") install(FILES "${CMAKE_CURRENT_BINARY_DIR}/tree-sitter-starlark.pc" DESTINATION "${CMAKE_INSTALL_DATAROOTDIR}/pkgconfig") install(TARGETS tree-sitter-starlark LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}") add_custom_target(ts-test "${TREE_SITTER_CLI}" test WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}" COMMENT "tree-sitter test") ``` -------------------------------- ### Starlark Boolean Operators and AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Covers Starlark's boolean logic operators (and, or, not) and their parsing structure in the tree-sitter grammar. Examples include combinations of these operators and their precedence. ```starlark a or b and c not d not a and b or c a and not b and c ``` ```tree-sitter-grammar (module (expression_statement (boolean_operator (identifier) (boolean_operator (identifier) (identifier)))) (expression_statement (not_operator (identifier))) (expression_statement (boolean_operator (boolean_operator (not_operator (identifier)) (identifier)) (identifier))) (expression_statement (boolean_operator (boolean_operator (identifier) (not_operator (identifier))) (identifier)))) ``` -------------------------------- ### Starlark Struct Initialization Example Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/struct.txt Demonstrates initializing a Starlark struct with various data types including integers, booleans, and strings. This is a common pattern for defining structured data in Starlark. ```starlark a = struct(x=123, y=false, z="hello") ``` -------------------------------- ### Starlark Named Expressions Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Details the tree-sitter grammar for Starlark's named expressions (`:=`), showing how assignments within expressions are structured. ```APIDOC (module (expression_statement (named_expression (identifier) (identifier))) (expression_statement (parenthesized_expression (named_expression (identifier) (call (identifier) (argument_list (identifier)))))) (expression_statement (call (identifier) (argument_list (keyword_argument (identifier) (parenthesized_expression (named_expression (identifier) (call (identifier) (argument_list (identifier))))))))) (expression_statement (assignment (identifier) (parenthesized_expression (named_expression (identifier) (call (identifier) (argument_list (identifier))))))) (function_definition (identifier) (parameters (default_parameter (identifier) (parenthesized_expression (named_expression (identifier) (integer))))) (block (return_statement (identifier)))) (function_definition (identifier) (parameters (typed_default_parameter (identifier) (type (parenthesized_expression (named_expression (identifier) (integer)))) (integer))) (block (return_statement (identifier)))) (expression_statement (call (identifier) (argument_list (named_expression (identifier) (integer)) (keyword_argument (identifier) (string (string_start) (string_content) (string_end)))))) (expression_statement (parenthesized_expression (named_expression (identifier) (parenthesized_expression (named_expression (identifier) (parenthesized_expression (named_expression (identifier) (integer))))))))) ``` -------------------------------- ### Starlark Tuples with Splats Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Illustrates the tree-sitter grammar for Starlark tuples containing splat expressions, showing how unpacked elements are parsed. ```APIDOC (module (expression_statement (tuple (identifier) (list_splat (identifier)) (list_splat (identifier))))) ``` -------------------------------- ### Starlark Assignments and AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Covers various Starlark assignment patterns, including simple assignments, tuple unpacking, starred assignments, and assignments to subscripts or attributes, along with their tree-sitter grammar structures. ```starlark a = 1 a, b = 1, 2 a, *c = 1, 2, 3 a, = 1, 2 a[b] = c = d a, *b.c = d ``` ```tree-sitter-grammar (module (expression_statement (assignment (identifier) (integer))) (expression_statement (assignment (pattern_list (identifier) (identifier)) (expression_list (integer) (integer)))) (expression_statement (assignment (pattern_list (identifier) (list_splat_pattern (identifier))) (expression_list (integer) (integer) (integer)))) (expression_statement (assignment (pattern_list (identifier)) (expression_list (integer) (integer)))) (expression_statement (assignment (subscript (identifier) (identifier)) (assignment (identifier) (identifier)))) (expression_statement (assignment (pattern_list (identifier) (list_splat_pattern (attribute (identifier) (identifier)))) (identifier)))) ``` -------------------------------- ### Starlark: Call Expressions AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Covers various Starlark function call syntaxes, including calls with no arguments, positional arguments, and keyword arguments. The AST shows the structure of these calls. ```starlark __a__() b(1) c(e, f=g) i(j, 5,) ``` ```ast (module (expression_statement (call (identifier) (argument_list))) (expression_statement (call (identifier) (argument_list (integer)))) (expression_statement (call (identifier) (argument_list (identifier) (keyword_argument (identifier) (identifier))))) (expression_statement (call (identifier) (argument_list (identifier) (integer))))) ``` -------------------------------- ### Define Function with *args and Nested Calls Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates a Python function that accepts *args and then makes calls using the unpacked arguments, potentially nested. ```python def h(*a): i((*a)) j(((*a))) ``` -------------------------------- ### Define Function with Typed Parameter and None Return Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows a Python function definition with a typed parameter and explicitly returning None. ```python def function_name(param: type_name): return None ``` -------------------------------- ### Starlark Async Context Managers and Iterators Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Illustrates the use of asynchronous context managers (`async with`) and asynchronous iterators (`async for`) within Starlark, including list comprehensions with async iteration. The grammar details the parsing of these constructs. ```starlark async with a as b: async for c in d: [e async for f in g] ``` ```APIDOC Async With Statement: syntax: async with expression [as target]: block description: Manages asynchronous context, ensuring setup and teardown operations are performed. Async For Statement: syntax: async for target in expression: block description: Iterates over an asynchronous iterable. Async List Comprehension: syntax: [expression async for target in expression] description: Creates a list by iterating over an asynchronous iterable. ``` -------------------------------- ### Starlark Delete Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates Starlark delete statements for removing elements from sequences or mappings, supporting multiple targets. ```starlark del a[1], b[2] ``` -------------------------------- ### Define Function with Mixed Parameters and Splats Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows a Python function definition with a mix of regular parameters, default parameters, list splat (*), and dictionary splat (**). ```python def function_name(p1, default_param=value, *args_list, **kwargs_dict): pass ``` -------------------------------- ### Starlark Augmented Assignments Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Demonstrates augmented assignment operations in Starlark, such as addition, bitwise shift, and floor division. These operations modify a variable in place. ```Python a += 1 b >>= 2 c //= 1 ``` -------------------------------- ### Starlark Primitive Types Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Shows the representation of fundamental Starlark primitive types: boolean values `True` and `False`, and the null value `None`. ```starlark True ``` ```starlark False ``` ```starlark None ``` -------------------------------- ### Starlark Nested If Statements Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates nested conditional logic in Starlark. Includes Python code and its abstract syntax tree (AST) representation. ```python if a: if b: c else: if e: f g ``` ```tree-sitter-grammar (module (if_statement condition: (identifier) consequence: (block (if_statement condition: (identifier) consequence: (block (expression_statement (identifier))) alternative: (else_clause body: (block (if_statement condition: (identifier) consequence: (block (expression_statement (identifier))))))))) (expression_statement (identifier))) ``` -------------------------------- ### Define Function with Tuple Unpacking in Parameters Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows a Python function definition where parameters are unpacked from a tuple. The function returns the unpacked tuple. ```python def e((a,b)): return (a,b) ``` -------------------------------- ### Starlark Function and If Statement Syntax Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates basic Starlark function definitions, if statements, and while loops. Includes the corresponding tree-sitter parse tree structure for these constructs, showing how the grammar defines elements like identifiers, parameters, and blocks. ```Starlark # These are not actually valid python; blocks # must contain at least one statement. But we # allow them because error recovery for empty # blocks doesn't work very well otherwise. def a(b, c): if d: print e while f(): ``` ```APIDOC (module (comment) (comment) (comment) (comment) (function_definition name: (identifier) parameters: (parameters (identifier) (identifier)) body: (block)) (if_statement condition: (identifier) consequence: (block (print_statement argument: (identifier)) (while_statement condition: (call function: (identifier) arguments: (argument_list))) body: (block))))) ``` -------------------------------- ### Define Function with Typed Parameter and Attribute Access Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates a Python function with a typed parameter where the type is an attribute access (e.g., module.Class). ```python def function_name(param: module.Attribute): pass ``` -------------------------------- ### Starlark: Await Expressions AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Demonstrates the Starlark `await` expression syntax, used for asynchronous operations, and its AST representation. This includes `await` used within statements and return statements. ```starlark await i(j, 5) return await i(j, 5) ``` ```ast (module (expression_statement (call (identifier) (ERROR (identifier)) (argument_list (identifier) (integer)))) (return_statement (call (identifier) (ERROR (identifier)) (argument_list (identifier) (integer))))) ``` -------------------------------- ### Starlark Float Literals Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Showcases Starlark float literal syntax, including decimal points, optional signs, digit separators, and complex number notation. ```starlark -.6_6 +.1_1 123.4123 123.123J 1_1.3_1 1_1. 1e+3_4j .3e1_4 ``` -------------------------------- ### Starlark: Print Function Usage AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Illustrates how the `print` function is handled in Starlark, both as a regular function call and when used in contexts like arguments or parameters. The AST captures these different usages. ```starlark print() print(a) print(a, b=c) print(d, e) print(d, *e) print(*f, **g,) a(print) ``` ```ast (module (expression_statement (call (identifier) (argument_list))) (expression_statement (call (identifier) (argument_list (identifier)))) (expression_statement (call (identifier) (argument_list (identifier) (keyword_argument (identifier) (identifier))))) (expression_statement (call (identifier) (argument_list (identifier) (identifier)))) (expression_statement (call (identifier) (argument_list (identifier) (list_splat (identifier))))) (expression_statement (call (identifier) (argument_list (list_splat (identifier)) (dictionary_splat (identifier))))) (expression_statement (call (identifier) (argument_list (identifier))))) ``` -------------------------------- ### Define Function with Generic Type Parameter and Return Type Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates a Python function definition using generic types for parameters and specifying a return type. ```python def function_name(param: identifier[type_identifier]): return return_type_identifier ``` -------------------------------- ### Define Function with Positional-Only, Keyword-Only, and **kwargs Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows a complex Python function signature including positional-only arguments (before /), keyword-only arguments (after *), a default value, and **kwargs. ```python def g(h, i, /, j, *, k=100, **kwarg): return h,i,j,k,kwarg ``` -------------------------------- ### Starlark If Statement Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates the tree-sitter grammar for a basic Starlark if statement. It captures the structure of an if condition, a block of code, and comments within the block. ```starlark (module (if_statement (identifier) (block (expression_statement (identifier)) (comment) (comment))) (if_statement (identifier) (block (expression_statement (identifier)) (comment) (comment))) (comment)) ``` -------------------------------- ### Starlark Conditional Expressions Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Shows the syntax for conditional (ternary) if expressions in Starlark, which allow for concise conditional assignment or value selection. ```Python x if y else z ``` -------------------------------- ### Starlark Arithmetic Operators and AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Demonstrates Starlark's arithmetic operations, including exponentiation and negation, and their representation in the tree-sitter grammar. Shows how binary and unary operators are parsed. ```starlark 2**2*3 -2**2 ``` ```tree-sitter-grammar (module (expression_statement (binary_operator (integer) (binary_operator (integer) (integer)))) (expression_statement (unary_operator (binary_operator (integer) (integer))))) ``` -------------------------------- ### Starlark Control-Flow: While Loop Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates the Starlark while loop structure, including a condition, a block of statements, and control flow keywords like pass, break, and continue. ```starlark while true: pass break continue ``` -------------------------------- ### Starlark Semicolon Statements Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates the tree-sitter grammar for Starlark statements that are terminated by semicolons, including single statements and sequences of statements. ```starlark foo; foo; bar foo; bar; (module (expression_statement (identifier)) (expression_statement (identifier)) (expression_statement (identifier)) (expression_statement (identifier)) (expression_statement (identifier))) ``` -------------------------------- ### Starlark Integer Literals Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Demonstrates various integer literal formats supported in Starlark, including decimal, hexadecimal, octal, binary, and complex numbers, as well as digit separators. ```starlark -1 0xDEAD 0XDEAD 1j -1j 0o123 0O123 0b001 0B001 1_1 0B1_1 0O1_1 0L ``` -------------------------------- ### Starlark Conditional Expressions Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Demonstrates the syntax and structure of conditional expressions (ternary operators) in Starlark, including their use with function calls and assignments. The grammar shows how these are parsed. ```starlark a = b if c else d something() if a else d slice(1,1,1) if a else d ``` ```APIDOC Conditional Expression: syntax: value_if_true if condition else value_if_false description: Evaluates a condition and returns one of two values based on the result. example: x = 10 if y > 5 else 0 Grammar Structure: (conditional_expression (expression_if_true) (condition) (expression_if_false)) description: Represents the abstract syntax tree for a conditional expression. ``` -------------------------------- ### Starlark Tuples with Splats Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Demonstrates the usage of splat operators (`*`) within tuple definitions in Starlark, allowing for unpacking of iterables into tuple elements. ```Python (foo, *bar, *baz) ``` -------------------------------- ### Python Raw and Escaped Strings Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Demonstrates the usage of Python raw strings and strings with escape sequences. Raw strings treat backslashes literally, while standard strings interpret escape sequences like newline characters. ```python r'ab\x00cd' ur"\n" ``` -------------------------------- ### Python Regex Compilation with Raw Strings Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Shows how to compile a regular expression in Python using concatenated raw strings. This pattern is designed to find include directives in source files, capturing the filename. ```python re.compile(r"(\n|\A)#include\s*['"]" + r"(?P[\w\d./\\]+[.]src)['"]" ``` -------------------------------- ### Define Function with Typed Default Parameter and String Value Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates a Python function with a typed parameter that also has a default value, which is a string literal. ```python def function_name(param: type_name = "default_string", other_param=value): return None ``` -------------------------------- ### Starlark Async/Await Identifier Usage Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Illustrates how 'async' and 'await' can be used as identifiers in Starlark, demonstrating their assignment and subsequent use in a print statement. The grammar recognizes these as simple identifiers. ```starlark async = 4 await = 5 print async, await ``` ```tree-sitter-sexp (module (expression_statement (assignment (identifier) (integer))) (expression_statement (assignment (identifier) (integer))) (print_statement (identifier) (identifier))) ``` -------------------------------- ### Starlark Calls with Splats Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Shows various ways to use argument unpacking (splats) in Starlark function calls, including list splats (*args) and dictionary splats (**kwargs). The grammar captures these unpacking mechanisms within argument lists. ```starlark a(*()) a(**{}) a(*b) c(d, *e, **g) ``` ```tree-sitter-sexp (module (expression_statement (call (identifier) (argument_list (list_splat (tuple))))) (expression_statement (call (identifier) (argument_list (dictionary_splat (dictionary))))) (expression_statement (call (identifier) (argument_list (list_splat (identifier))))) (expression_statement (call (identifier) (argument_list (identifier) (list_splat (identifier)) (dictionary_splat (identifier)))))) ``` -------------------------------- ### Starlark Newline and Comment Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates how the tree-sitter parser handles newline tokens preceding comments when there is no subsequent newline. It shows a print statement followed by a comment. ```starlark print "a" # We need to recognize the newline *preceding* this comment, because there's no newline after it (module (print_statement (string (string_start) (string_content) (string_end))) (comment)) ``` -------------------------------- ### Starlark Bitwise Operators and AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Details Starlark's bitwise operators (left shift, right shift, bitwise AND, bitwise OR) and their hierarchical representation within the tree-sitter grammar, demonstrating parsing order. ```starlark a << b | c >> d & e ``` ```tree-sitter-grammar (module (expression_statement (binary_operator (binary_operator (identifier) (identifier)) (binary_operator (binary_operator (identifier) (identifier)) (identifier))))) ``` -------------------------------- ### Starlark Binary Addition/Subtraction with Floats Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Demonstrates binary arithmetic operations in Starlark involving floating-point numbers and integers. The grammar recognizes these operations and correctly parses mixed-type operands. ```starlark .1-.0 .1+.0 .1-0 .1+0 1-.0 1+.0 ``` ```tree-sitter-sexp (module (expression_statement (binary_operator (float) (float))) (expression_statement (binary_operator (float) (float))) (expression_statement (binary_operator (float) (integer))) (expression_statement (binary_operator (float) (integer))) (expression_statement (binary_operator (integer) (float))) (expression_statement (binary_operator (integer) (float)))) ``` -------------------------------- ### Starlark Match Specific Values Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/pattern_matching.txt Demonstrates using Starlark's `match` statement to handle specific command patterns like 'quit', 'look', 'get', and 'go'. It shows how to destructure arguments like 'obj' and 'direction' within the `case` patterns. ```Starlark match command.split(): case ["quit"]: print("Goodbye!") quit_game() case ["look"]: current_room.describe() case ["get", obj]: character.get(obj, current_room) case ["go", direction]: current_room = current_room.neighbor(direction) # The rest of your commands go here ``` -------------------------------- ### Starlark: Subscript Expressions AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Illustrates Starlark subscript expressions, including single and multiple index access, and their corresponding AST structures. This covers accessing elements within sequences or mappings. ```starlark a[1] b[2, 3] c[4, 5,] ``` ```ast (module (expression_statement (subscript (identifier) (integer))) (expression_statement (subscript (identifier) (integer) (integer))) (expression_statement (subscript (identifier) (integer) (integer)))) ``` -------------------------------- ### Starlark Conditional Expressions Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Details the tree-sitter grammar for Starlark's conditional if expressions, illustrating the structure of `condition if true_value else false_value`. ```APIDOC (module (expression_statement (conditional_expression (identifier) (identifier) (identifier)))) ``` -------------------------------- ### Starlark Scientific Notation Floats Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Illustrates Starlark float literals using scientific notation (e.g., 1e322, 1e-3, 1e+3), including variations with decimal points and signs. ```starlark 1e322 1e-3 1e+3 1.8e10 1.e10 -1e10 ``` -------------------------------- ### Starlark F-strings with Format Specifiers Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Illustrates Starlark f-strings utilizing format specifiers for controlling output, such as width, precision, alignment, and type conversion. Supports complex expressions within format specifiers. ```starlark f"a {b:2} {c:34.5}" ``` ```starlark f"{b:{c.d}.{d.e}}" ``` ```starlark f"{a:#06x}" ``` ```starlark f"{a}=" ``` ```starlark f"{a=:.2f}" ``` -------------------------------- ### Decorated Starlark Definitions Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates Starlark syntax for decorated function definitions, including various forms of decorators (attribute access, function calls, subscripting). The APIDOC section shows the tree-sitter representation of these complex decorator structures and async function definitions. ```Starlark @a.b @d(1) @e[2].f.g def f(): g @f() async def f(): g ``` ```APIDOC (module (decorated_definition (decorator (attribute (identifier) (identifier))) (decorator (call (identifier) (argument_list (integer)))) (decorator (attribute (attribute (subscript (identifier) (integer)) (identifier)) (identifier))) (function_definition (identifier) (parameters) (block))) (expression_statement (identifier)) (decorated_definition (decorator (call (identifier) (argument_list))) (function_definition (identifier) (parameters) (block))) (expression_statement (identifier))) ``` -------------------------------- ### Starlark Operator Precedence and AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Illustrates Starlark's operator precedence rules, showing how function calls, subscripts, and attribute access are combined with multiplication and addition. The AST reflects the parsing order. ```starlark a() + b[c] * c.d.e ``` ```tree-sitter-grammar (module (expression_statement (binary_operator left: (call function: (identifier) arguments: (argument_list)) right: (binary_operator left: (subscript value: (identifier) subscript: (identifier)) right: (attribute object: (attribute object: (identifier) attribute: (identifier)) attribute: (identifier)))))) ``` -------------------------------- ### Starlark: Attribute References AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Shows the parsing of Starlark attribute references (e.g., `object.attribute`) and their nested forms (e.g., `a.b.c`) within the AST. This is fundamental for accessing members of objects. ```starlark a.b.c ``` ```ast (module (expression_statement (attribute (attribute (identifier) (identifier)) (identifier)))) ``` -------------------------------- ### Starlark Exec Statement Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Details the tree-sitter grammar for Starlark's exec statement, including variations with string literals, 'in' clauses with None, and 'in' clauses with identifiers. ```starlark exec '1+1' exec 'x+=1' in None exec 'x+=1' in a, b (module (exec_statement (string (string_start) (string_content) (string_end))) (exec_statement (string (string_start) (string_content) (string_end)) (none)) (exec_statement (string (string_start) (string_content) (string_end)) (identifier) (identifier))) ``` -------------------------------- ### Starlark Comments Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows how comments are handled within Starlark code, including single-line comments and comments at the end of lines. The APIDOC section details the tree-sitter grammar's recognition of comment nodes. ```Starlark print a # hi print b # bye print c ``` ```APIDOC (module (print_statement (identifier)) (comment) (print_statement (identifier)) (comment) (print_statement (identifier))) ``` -------------------------------- ### Starlark Arbitrary Indentation Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Shows how Starlark handles arbitrary indentation within code blocks, specifically demonstrating its flexibility in function calls and list definitions where newlines and indentation are used liberally. The grammar reflects this parsing capability. ```starlark def a(): b( 1, 2 ) c = [ 3 ] ``` ```APIDOC Function Definition: syntax: def function_name(parameters): block description: Defines a reusable block of code. Call Expression: syntax: function_name(argument1, argument2, ...) description: Invokes a function with specified arguments. List Definition: syntax: [element1, element2, ...] description: Creates a list literal. note: Starlark allows flexible indentation and newlines within argument lists and list definitions. ``` -------------------------------- ### Starlark: Print as Parameter AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Details Starlark function definitions where 'print' is used as a parameter name, including default values and splat arguments. The AST shows how these parameter declarations are parsed. ```starlark def a(print): b def a(printer=print): c def a(*print): b def a(**print): b def print(): a ``` ```ast (module (function_definition (identifier) (parameters (parameter (identifier))) (block (expression_statement (identifier)))) (function_definition (identifier) (parameters (parameter (identifier) (default (identifier))))) (block (expression_statement (identifier)))) (function_definition (identifier) (parameters (parameter (list_splat (identifier))))) (block (expression_statement (identifier)))) (function_definition (identifier) (parameters (parameter (dictionary_splat (identifier))))) (block (expression_statement (identifier)))) (function_definition (identifier) (parameters) (block (expression_statement (identifier))))) ``` -------------------------------- ### Starlark Nested F-strings Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Demonstrates Starlark's support for nested f-strings, including complex nesting with function calls and various quote styles. Handles different combinations of single, double, and triple quotes within interpolations. ```starlark f"a {b(f'c {e} d')} e" ``` ```starlark f"""a\"{b}c\""" ``` ```starlark f"""a\"\"{b}c\"\""" ``` ```starlark f"a {{}} e" ``` ```starlark f"a {b}}}" ``` ```starlark f"a {{{b}" ``` ```starlark f"a {{b}}" ``` ```starlark f"a {{{b}}}" ``` -------------------------------- ### Starlark Unicode Escape Sequences Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/literals.txt Demonstrates the use of various Unicode escape sequences within Starlark strings, including hexadecimal (`\x`), octal (`\123`), and general Unicode (`\u1234`) escapes. ```starlark "\x12 \123 \u1234" ``` -------------------------------- ### Starlark Comparison Operators and AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Explains Starlark's comparison operators (less than, less than or equal to, equal to, greater than or equal to, greater than, not in) and their AST representation, including chained comparisons and boolean combinations. ```starlark a < b <= c == d >= e > f not a == b or c == d a not in b ``` ```tree-sitter-grammar (module (expression_statement (comparison_operator (identifier) (identifier) (identifier) (identifier) (identifier) (identifier))) (expression_statement (boolean_operator (not_operator (comparison_operator (identifier) (identifier))) (comparison_operator (identifier) (identifier)))) (expression_statement (comparison_operator (identifier) (identifier)))) ``` -------------------------------- ### Starlark Math Operators and Precedence Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Illustrates the use of various arithmetic operators in Starlark, including addition, subtraction, multiplication, division, exponentiation, and unary negation/complement. The grammar defines the structure and precedence of these binary and unary operations. ```starlark a + b * c ** d - e / 5 -5 +x ~x ``` ```tree-sitter-sexp (module (expression_statement (binary_operator (binary_operator (identifier) (binary_operator (identifier) (binary_operator (identifier) (identifier)))) (binary_operator (identifier) (integer)))) (expression_statement (unary_operator (integer))) (expression_statement (unary_operator (identifier))) (expression_statement (unary_operator (identifier)))) ``` -------------------------------- ### Starlark Escaped Newline Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates the tree-sitter grammar for Starlark code that uses a backslash to escape a newline, allowing a single logical statement to span multiple physical lines. ```starlark len("a") \ or len("aa") (module (expression_statement (boolean_operator (call (identifier) (argument_list (string (string_start) (string_content) (string_end)))) (line_continuation) (call (identifier) (argument_list (string (string_start) (string_content) (string_end))))))) ``` -------------------------------- ### Starlark Extra Newlines Grammar Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows the tree-sitter grammar for handling multiple consecutive newlines within Starlark code, specifically within an if statement block containing function definitions and expressions. ```starlark if a: b() c() def d(): e() f() (module (if_statement (identifier) (block (expression_statement (call (identifier) (argument_list))) (expression_statement (call (identifier) (argument_list))) (function_definition (identifier) (parameters) (block (expression_statement (call (identifier) (argument_list))))) (expression_statement (call (identifier) (argument_list)))))) ``` -------------------------------- ### Starlark Comments at End of Indented Blocks Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Shows Starlark code with comments placed at the end of indented blocks. The APIDOC illustrates how the tree-sitter grammar correctly associates these comments with their respective blocks. ```Starlark if a: b # one # two if c: d # three # four ``` ```APIDOC (module (if_statement (identifier) (block (expression_statement (identifier)) (comment) (comment))) (if_statement (identifier) (block (expression_statement (identifier)) (comment) (comment)))) ``` -------------------------------- ### Starlark Function Definition Structures Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Defines the various syntactical structures for function definitions in Starlark, including parameters with default values, list splats, and dictionary splats. This grammar definition illustrates how these constructs are recognized. ```tree-sitter-sexp (module (function_definition (identifier) (parameters (identifier)) (block (expression_statement (identifier)))) (function_definition (identifier) (parameters (default_parameter (identifier) (identifier))) (block (expression_statement (identifier)))) (function_definition (identifier) (parameters (list_splat_pattern (identifier))) (block (expression_statement (identifier)))) (function_definition (identifier) (parameters (dictionary_splat_pattern (identifier))) (block (expression_statement (identifier)))) (function_definition (identifier) (parameters) (block (expression_statement (identifier))))) ``` -------------------------------- ### Starlark Match Statement with Walrus Operator Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/pattern_matching.txt Demonstrates the use of the walrus operator (:=) within a Starlark 'match' statement's 'if' clause for conditional assignment and matching. This example uses a regular expression match. ```starlark if match := re.fullmatch(r"(-)?(\d+:)?\d?\d:\d\d(\.\d*)?", time, flags=re.ASCII): return 42 ``` ```tree-sitter-grammar (module (if_statement (named_expression (identifier) (call (attribute (identifier) (identifier)) (argument_list (string (string_start) (string_content) (string_end)) (identifier) (keyword_argument (identifier) (attribute (identifier) (identifier)))))) (block (return_statement (integer))))) ``` -------------------------------- ### Starlark: Subscript Slice Expressions AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Details Starlark subscript slicing syntax, including full slices, partial slices, and slices with ellipsis. The AST representation shows how these slicing operations are structured. ```starlark a[:] b[5:] b[5:6, ...] c[::] ``` ```ast (module (expression_statement (subscript (identifier) (slice))) (expression_statement (subscript (identifier) (slice (integer)))) (expression_statement (subscript (identifier) (slice (integer) (integer)) (ellipsis))) (expression_statement (subscript (identifier) (slice)))) ``` -------------------------------- ### Starlark: Identifiers with Greek Letters AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Demonstrates the parsing of Starlark identifiers, including those using Greek letters, and their representation in the Abstract Syntax Tree (AST). This section shows how the grammar handles variable names and their assignments. ```starlark ψ1 = β_γ + Ψ_5 ``` ```ast (module (expression_statement (assignment left: (identifier) right: (binary_operator left: (identifier) right: (identifier))))) ``` -------------------------------- ### Starlark Comments at Different Indentation Levels Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Demonstrates the handling of comments interspersed with code at varying indentation levels within Starlark. The APIDOC shows how the tree-sitter grammar parses these comments relative to their surrounding code blocks and statements. ```Starlark if a: # one # two # three b # four c ``` ```APIDOC (module (if_statement (identifier) (comment) (comment) (comment) (block (expression_statement (identifier)) (comment) (expression_statement (identifier))))) ``` -------------------------------- ### Starlark Comments After Dedents Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/statements.txt Illustrates Starlark code structure where comments appear after dedented blocks. The APIDOC section details the tree-sitter grammar's parsing of comments that follow the end of an indented code sequence. ```Starlark if a: b # one c ``` ```APIDOC (module (if_statement (identifier) (block (expression_statement (identifier)))) (comment) (expression_statement (identifier))) ``` -------------------------------- ### Starlark Named Expressions (Walrus Operator) Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/expressions.txt Illustrates the use of named expressions, also known as the walrus operator `:=`, for assignment within expressions in Starlark. This includes assignments within function calls and nested structures. ```Python a := x (y := f(x)) foo(x=(y := f(x))) y0 = (y1 := f(x)) def foo(answer=(p := 42)): return answer; def foo(answer: (p := 42) = 5): return answer; foo(x := 3, cat='vector') (z := (y := (x := 0))) ``` -------------------------------- ### Python Keyword Argument with 'match' AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/pattern_matching.txt Illustrates the AST for a Python function call using 'match' as a keyword argument, showing the parsing of keyword arguments with string values. ```starlark (module (expression_statement (assignment (identifier) (call (identifier) (argument_list (keyword_argument (identifier) (string (string_start) (string_content) (string_end)))))))) ``` -------------------------------- ### Python Mixed Arguments with 'match' Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/pattern_matching.txt Demonstrates a Python function call using 'match' as both a keyword argument and a positional argument, showcasing mixed argument usage. ```python field = match(match=match, match) ``` -------------------------------- ### Python 'case' as Identifier and Keyword AST Source: https://github.com/tree-sitter-grammars/tree-sitter-starlark/blob/master/test/corpus/pattern_matching.txt Provides the AST for Python code using 'case' as an identifier and a keyword argument, demonstrating the parsing of these constructs. ```starlark (module (expression_statement (assignment (identifier) (list (identifier)))) (expression_statement (assignment (identifier) (list (identifier)))) (expression_statement (assignment (identifier) (call (identifier) (argument_list (keyword_argument (identifier) (true))))))) ```