### Run common grep-ast search examples Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/README.md Practical examples demonstrating pattern searching, case-insensitivity, line numbering, and encoding specification. ```bash # Search for TODO comments grep-ast "TODO" . # Case-insensitive class search grep-ast -i "class" src/ # With line numbers and color grep-ast -n --color "def " myfile.py # Specific encoding grep-ast --encoding latin-1 "pattern" oldfile.txt ``` -------------------------------- ### Install grep-ast for development Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Clone the repository and install in editable mode for local development. ```bash git clone https://github.com/paul-gauthier/grep-ast.git cd grep-ast pip install -e . ``` -------------------------------- ### Install grep-ast Source: https://github.com/aider-ai/grep-ast/blob/main/README.md Install the package directly from the GitHub repository using pip. ```bash python -m pip install git+https://github.com/paul-gauthier/grep-ast.git ``` -------------------------------- ### CLI Color Output Examples Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/configuration.md Examples demonstrating how to use color flags with piped output. ```bash # Auto-detected (color if interactive) grep-ast "pattern" file.py # Piped to less (auto-detect disables color, but flag overrides) grep-ast "pattern" file.py | less # No color grep-ast --color "pattern" file.py | less # Force color ``` -------------------------------- ### Run grep-ast examples Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Perform a search within the repository and list supported languages. ```bash # Search this repository cd grep-ast grep-ast "def " grep_ast/ # List languages grep-ast --languages ``` -------------------------------- ### Test grep-ast installation Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md List supported file extensions to confirm the tool is working correctly. ```bash grep-ast --languages | head ``` -------------------------------- ### Example .gitignore configuration Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Standard ignore patterns used by grep-ast to filter files. ```text # Python *.pyc __pycache__/ .venv/ dist/ build/ # Node node_modules/ .next/ # IDE .vscode/ .idea/ *.swp # Git .git/ ``` -------------------------------- ### Gitignore Rules Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md Example patterns for .gitignore files that are respected by the tool. ```text *.pyc # Ignores .pyc files __pycache__/ # Ignores directories .git/ # Ignores .git directory node_modules/ # Ignores node_modules ``` -------------------------------- ### Initialize and Use TreeContext Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-treecontext.md Example showing how to load a file, initialize TreeContext, perform a grep search, and format the output. ```python from grep_ast import TreeContext # Load a Python file with open('example.py', 'r') as f: code = f.read() # Create a TreeContext tc = TreeContext( 'example.py', code, color=True, verbose=False, line_number=True ) # Search for pattern and get results with context matches = tc.grep('def ', ignore_case=False) tc.add_lines_of_interest(matches) tc.add_context() print(tc.format()) ``` -------------------------------- ### Pattern Matching Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Examples of using regex patterns to search for code content. ```bash grep-ast "def " myfile.py # Search for "def " grep-ast "class\s+\w+" . # Regex pattern grep-ast "TODO|FIXME" src/ # Alternation ``` -------------------------------- ### Example of header range Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Shows how a function's header range is stored. ```python # For a 10-line function starting at line 5: header[5] = (5, 10) # Show all 10 lines as header (or truncate to header_max) ``` -------------------------------- ### Usage examples for dump Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-dump.md Demonstrates printing simple variables, multiple arguments, and complex objects. ```python from grep_ast.dump import dump x = 42 y = "hello" z = [1, 2, 3] dump(x) # Output: x: 42 dump(y) # Output: y: hello dump(z) # Output: z: [1, 2, 3] dump(x, y, z) # Output: x, y, z: 42, hello, [1, 2, 3] # With complex objects data = { "name": "test", "values": [1, 2, 3], } dump(data) # Output: data: # { # "name": "test", # "values": [1, 2, 3] # } ``` -------------------------------- ### Check Supported Languages Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/00-START-HERE.md Displays the list of languages supported by the current installation. ```bash grep-ast --languages ``` -------------------------------- ### Check grep-ast version Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Verify the installed version of the package using pip. ```bash pip show grep-ast ``` -------------------------------- ### Integrate with Make and Shell scripts Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Examples for using grep-ast in CI/CD pipelines or automated build tasks. ```bash # Find TODO before deployment grep-ast --no-color "TODO\|FIXME" src/ && echo "Found issues" # CI/CD integration if grep-ast --no-color "deprecated" src/; then echo "Found deprecated code" exit 1 fi ``` -------------------------------- ### Iterating and Checking Supported Extensions Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-parsers.md Example usage for iterating through supported extensions and verifying specific language support. ```python from grep_ast.parsers import PARSERS # Print all supported extensions for ext, lang in sorted(PARSERS.items()): print(f"{ext} -> {lang}") # Check if a language is supported if ".py" in PARSERS: print(f"Python: {PARSERS['.py']}") ``` -------------------------------- ### Integrate with Git Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Examples for searching committed files, specific commits, or branches using git commands piped to grep-ast. ```bash # Search only committed files git ls-files | xargs grep-ast "pattern" # Search files changed in a commit git show --name-only COMMIT | tail -n +5 | xargs grep-ast "pattern" # Search in a branch git ls-tree -r --name-only BRANCH | xargs grep-ast "pattern" ``` -------------------------------- ### Debugging collections Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-dump.md Usage example for inspecting lists and dictionaries. ```python from grep_ast.dump import dump items = [1, 2, 3, 4, 5] counts = {"a": 5, "b": 10, "c": 3} dump(items, counts) ``` -------------------------------- ### Custom File Encoding Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md Examples of specifying file encodings when searching through files. ```bash # Search in files with Latin-1 encoding grep-ast --encoding latin-1 "pattern" oldfiles/ # Custom encoding grep-ast --encoding cp1252 "pattern" windowsfiles/ ``` -------------------------------- ### Example of scope membership Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Demonstrates how nested scopes are tracked across line numbers. ```python # For a nested structure: # 0: def outer(): # 1: def inner(): # 2: pass scopes[0] = {0} # Line 0 is only in scope starting at 0 scopes[1] = {0, 1} # Line 1 is in scopes starting at 0 and 1 scopes[2] = {0, 1} # Line 2 is in scopes starting at 0 and 1 ``` -------------------------------- ### Common grep-ast command combinations Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Examples of standard usage patterns for interactive searching, piping, CI/CD environments, and debugging. ```bash # Interactive search with all features grep-ast -i -n --color "TODO" src/ # Output suitable for piping grep-ast --no-color "pattern" file.py | grep "something" # Safe search (ignore .gitignore, custom encoding) grep-ast --no-gitignore --encoding latin-1 "pattern" . # Debugging grep-ast --verbose -n "pattern" file.py # CI/CD environment (no color, line numbers) grep-ast --no-color -n "pattern" src/ ``` -------------------------------- ### Output format with line numbers Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Example of the output structure when the -n flag is enabled. ```text 42│def function(): 43│ """Docstring.""" 44█ TODO: fix this 45│ pass ``` -------------------------------- ### Initialize parser and parse code Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-tsl.md Example usage of get_parser to parse a Python string into a tree-sitter tree. ```python from grep_ast.tsl import get_parser parser = get_parser("python") code = "def hello(): pass" tree = parser.parse(bytes(code, "utf8")) print(tree.root_node) ``` -------------------------------- ### Check Supported Languages Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/errors.md Verify if a specific file extension is supported by the current grep-ast installation. ```bash # Check if language is supported grep-ast --languages | grep myextension # If not supported, consider: # 1. Renaming the file with a recognized extension # 2. Using a language pack that supports it ``` -------------------------------- ### Exit code verification Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Examples of checking exit codes after successful and failed executions. ```bash grep-ast "pattern" file.py echo $? # Prints 0 grep-ast echo $? # Prints 1 (no pattern) ``` -------------------------------- ### Interpret grep-ast Output Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Example of how grep-ast displays matched lines with context and scope information. ```bash $ grep-ast "TODO" main.py main.py: │ 42 def process_data(): │ 43 """Process input data.""" █ 44 TODO: optimize this function │ 45 result = [] │ 46 for item in data: ⋮ │ 52 return result ``` -------------------------------- ### Specify encoding for grep-ast CLI Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/errors.md Provides examples of using the --encoding flag to resolve UnicodeDecodeError issues. ```bash # Try different encoding if default UTF-8 fails grep-ast --encoding latin-1 "pattern" file.txt grep-ast --encoding cp1252 "pattern" file.txt ``` -------------------------------- ### CLI Help Output Source: https://github.com/aider-ai/grep-ast/blob/main/README.md View the full list of command-line options and arguments. ```text usage: grep_ast.py [-h] [-i] [--color] [--no-color] [--encoding ENCODING] [--languages] [--verbose] [pat] [filenames ...] positional arguments: pat the pattern to search for filenames the files to display options: -h, --help show this help message and exit -i, --ignore-case ignore case distinctions --color force color printing --no-color disable color printing --encoding ENCODING file encoding --languages print the parsers table --verbose enable verbose output ``` -------------------------------- ### Define console script entry points in setup.py Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md These entry points allow the grep-ast and gast commands to be used from the terminal. ```python "grep-ast=grep_ast.main:main" "gast=grep_ast.main:main" ``` -------------------------------- ### Initialize Tree-Sitter Parser Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-tsl.md Demonstrates how to obtain a language-specific parser and generate an AST root node from source code. ```python from grep_ast.tsl import get_parser from grep_ast.parsers import filename_to_lang # In TreeContext.__init__: lang = filename_to_lang(filename) parser = get_parser(lang) tree = parser.parse(bytes(code, "utf8")) root_node = tree.root_node ``` -------------------------------- ### Initialize TreeContext with configuration Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/configuration.md Demonstrates both minimal and full configuration options for the TreeContext constructor. ```python from grep_ast import TreeContext # Minimal configuration tc = TreeContext('main.py', code) # Full configuration tc = TreeContext( 'main.py', code, color=True, verbose=False, line_number=True, parent_context=True, child_context=True, last_line=True, margin=3, mark_lois=True, header_max=10, show_top_of_file_parent_scope=True, loi_pad=1, ) ``` -------------------------------- ### Help Command Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Displaying the help message. ```bash grep-ast --help grep-ast -h ``` -------------------------------- ### Import the main function Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md Import the entry point function from the main module. ```python from grep_ast.main import main ``` -------------------------------- ### List Project Documentation Files Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/MANIFEST.md Displays the list of generated markdown documentation files and their respective line counts. ```text api-reference-dump.md 163 lines api-reference-tsl.md 222 lines api-reference-parsers.md 232 lines api-reference-treecontext.md 308 lines errors.md 309 lines types.md 309 lines api-reference-main.md 316 lines configuration.md 374 lines module-index.md 390 lines quick-start.md 470 lines README.md 494 lines architecture.md 576 lines cli-reference.md 749 lines MANIFEST.md This file --- Total: 4,912+ lines ``` -------------------------------- ### Access TreeContext.nodes invariant Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/types.md Demonstrates accessing the list of AST nodes starting on a specific line. ```python nodes[15] = [Node(type='function_definition', ...)] ``` -------------------------------- ### Basic Syntax Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md The primary command structure for executing searches. ```bash grep-ast [OPTIONS] [PATTERN] [FILENAMES...] gast [OPTIONS] [PATTERN] [FILENAMES...] ``` -------------------------------- ### CLI Entry Points Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/module-index.md The grep-ast package provides command-line tools for searching source code files. ```APIDOC ## grep-ast / gast ### Description Console scripts to execute the main grep-ast search functionality. ### Usage `grep-ast [options] [pattern] [filenames...]` `gast [options] [pattern] [filenames...]` ``` -------------------------------- ### Define header range structure Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Maps a scope start line to the range of lines that constitute its header. ```python header[scope_start] = (head_start, head_end) ``` -------------------------------- ### Define scope membership structure Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Represents the mapping of line numbers to their active scope start indices. ```python scopes[line_num] = {scope_start_1, scope_start_2, ...} ``` -------------------------------- ### Perform a typical TreeContext workflow Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-treecontext.md Load a file, search for patterns using regex, add context to matches, and print the formatted result. ```python from grep_ast import TreeContext # Step 1: Load source and create context with open('myfile.py') as f: code = f.read() tc = TreeContext('myfile.py', code, color=True, line_number=True) # Step 2: Search for pattern matches = tc.grep(r'class\s+\w+', ignore_case=False) if not matches: print("No matches") exit() # Step 3: Mark matches and add context tc.add_lines_of_interest(matches) tc.add_context() # Step 4: Output formatted results print(f"{filename}:") print(tc.format()) ``` -------------------------------- ### Access TreeContext.header invariant Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/types.md Demonstrates accessing the header range tuple for a scope starting at a specific line. ```python header[5] = (5, 10) # Scope starting at line 5 has header spanning lines 5-10 ``` -------------------------------- ### Access TreeContext.scopes invariant Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/types.md Demonstrates accessing the set of scope start lines for a specific line index. ```python scopes[10] = {0, 5, 8} # Line 10 is inside scopes starting at lines 0, 5, and 8 ``` -------------------------------- ### Basic Usage Source: https://github.com/aider-ai/grep-ast/blob/main/README.md Perform a search using a regex pattern and optional filenames. ```bash grep-ast [pattern] [filenames...] ``` -------------------------------- ### Basic grep-ast usage commands Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Common command-line patterns for searching, line numbering, and color output. ```bash grep-ast "TODO" . grep-ast -n "pattern" . grep-ast --color "pattern" . ``` -------------------------------- ### Force search everything Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Use the --no-gitignore flag to bypass ignore rules. ```bash grep-ast --no-gitignore "pattern" . ``` -------------------------------- ### Define AST node tracking structure Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Maps line numbers to a list of AST nodes starting on that line. ```python nodes[line_num] = [node1, node2, ...] ``` -------------------------------- ### Track scopes during AST traversal Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Maintains a mapping of line numbers to their respective scope start lines for O(1) lookup. ```python # In walk_tree(): for i in range(start_line, end_line + 1): self.scopes[i].add(start_line) # Mark line i as part of scope starting at start_line # Later, to find parent scopes: for scope_start in self.scopes[line_num]: # scope_start is a parent scope ``` -------------------------------- ### Compare grep and grep-ast output Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/00-START-HERE.md Demonstration of how grep-ast provides structural context compared to standard grep. ```bash $ grep "def " main.py def process_data(): def helper(): def inner(): ``` ```bash $ grep-ast "def " main.py main.py: │ 1 import sys │ 2 █ 3 def process_data(): │ 4 """Main function.""" │ 5 result = [] │ 6 for item in data: │ 7 def helper(): │ 8 return item * 2 │ 9 return result ``` -------------------------------- ### Perform command-line searches Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Execute various search patterns and configurations using the grep-ast CLI. ```bash grep-ast "pattern" file.py ``` ```bash grep-ast "TODO" . ``` ```bash grep-ast -i "class" . ``` ```bash grep-ast -n --color "def " src/ ``` ```bash grep-ast --languages ``` -------------------------------- ### Development workflow comparison Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-dump.md Comparison between standard print statements and the dump utility. ```python # Instead of: print(f"x: {x}, y: {y}") # Just write: dump(x, y) ``` -------------------------------- ### Execute grep-ast via command line Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md Common command-line patterns for searching files, handling encodings, and managing gitignore rules. ```bash # Basic search grep-ast "def " myfile.py # Case-insensitive search grep-ast -i "class" . # With line numbers and color grep-ast -n --color "TODO" src/ # Ignore .gitignore and search everything grep-ast --no-gitignore "deprecated" . # Show all supported languages grep-ast --languages # Custom encoding grep-ast --encoding latin-1 "pattern" oldfile.txt ``` -------------------------------- ### Supported Languages Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Listing supported file extensions and languages. ```bash grep-ast --languages ``` ```text .bash: bash .c: c .css: css .go: go .java: java .js: javascript .md: markdown .py: python .rb: ruby .rs: rust ... ``` -------------------------------- ### Importing Parser Utilities Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-parsers.md Import the filename mapping function and the PARSERS dictionary. ```python from grep_ast.parsers import filename_to_lang, PARSERS ``` -------------------------------- ### Invoke main entry point Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md Directly call the main function to execute the CLI logic programmatically. ```python # Direct invocation (not typical) from grep_ast.main import main exit_code = main() ``` -------------------------------- ### Output with Color Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Enable colorized output for search results. ```bash grep-ast --color "pattern" file.py ``` -------------------------------- ### Format output for display Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-treecontext.md Generate a string representation of the lines of interest and their context. ```python output = tc.format() print(output) # Output example: # ├ 42 class Example: # ├ 43 def method(self): # █ 44 TODO: fix this # ├ 45 pass # ⋮ # ├ 50 def other(self): ``` -------------------------------- ### Troubleshoot pattern matching issues Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Commands to diagnose why patterns are not returning results, including ignoring gitignore files and checking language support. ```bash # Check with --no-gitignore grep-ast --no-gitignore "pattern" . # Check with case-insensitive grep-ast -i "pattern" . # Check supported languages grep-ast --languages | grep extension ``` -------------------------------- ### grep-ast CLI Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md The main entry point for the grep-ast command-line tool, used for searching patterns within files and directories. ```APIDOC ## CLI: grep-ast ### Description Entry point for the command-line interface. Parses arguments and processes all matching files. ### Parameters - **pattern** (positional) - Optional - Regex pattern to search for - **filenames** (positional) - Optional - Files or directories to search (default: ".") - **-h, --help** (flag) - Optional - Show help message and exit - **-i, --ignore-case** (flag) - Optional - Ignore case distinctions in pattern matching - **--color** (flag) - Optional - Force color output - **--no-color** (flag) - Optional - Disable color output - **--encoding** (string) - Optional - File encoding to use (default: "utf8") - **--languages** (flag) - Optional - Print supported language/extension table and exit - **--no-gitignore** (flag) - Optional - Ignore .gitignore file - **--verbose** (flag) - Optional - Enable verbose debugging output - **-n, --line-number** (flag) - Optional - Display line numbers in output ### Example ```bash grep-ast -n --color "TODO" src/ ``` ``` -------------------------------- ### Search Code from CLI Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/module-index.md Execute pattern matching directly from the command line with support for case-insensitivity, color, and line numbers. ```bash grep-ast "pattern" file.py grep-ast -i "class" src/ grep-ast --color --line-number "TODO" . ``` -------------------------------- ### Execute basic grep-ast search Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/README.md Standard syntax for running grep-ast with optional patterns and file targets. ```bash grep-ast [OPTIONS] [PATTERN] [FILES...] ``` -------------------------------- ### Troubleshoot file search issues Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Commands to verify file accessibility and bypass gitignore patterns. ```bash # Ignore .gitignore grep-ast --no-gitignore "pattern" . # Search specific file grep-ast "pattern" /full/path/file.py # Check permissions ls -la yourfile.py ``` -------------------------------- ### Working Directory Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Specify the directory scope for the search. ```bash cd /path/to/project grep-ast "pattern" . # Searches current directory grep-ast "pattern" src/ # Searches src/ subdirectory ``` -------------------------------- ### Use specific search patterns Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Use anchored or specific patterns to increase search speed. ```bash grep-ast "^def " . # Better than "def" ``` -------------------------------- ### File and Directory Arguments Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Specifying target files or directories for the search operation. ```bash grep-ast "pattern" myfile.py # Single file grep-ast "pattern" file1.py file2.py # Multiple files grep-ast "pattern" . # Current directory grep-ast "pattern" src/ lib/ # Multiple directories ``` -------------------------------- ### List supported languages and extensions Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/errors.md Display all languages currently supported by the grep-ast tool. ```bash # List all supported languages and extensions grep-ast --languages | head -20 ``` -------------------------------- ### Combined Options Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Use multiple flags simultaneously for refined output. ```bash grep-ast -i -n --color "TODO" src/ ``` -------------------------------- ### Search files using the command-line interface Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/README.md Execute searches directly from the terminal with support for regex patterns, case-insensitivity, and line numbering. ```bash # Basic search grep-ast "pattern" file.py # With options grep-ast -i -n --color "TODO" src/ # List languages grep-ast --languages ``` -------------------------------- ### List Supported Languages Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-parsers.md Iterates through the PARSERS dictionary to display all supported file extensions and their associated languages. ```python from grep_ast.parsers import PARSERS # List all supported file extensions print("Supported languages:") for ext, lang in sorted(PARSERS.items()): print(f" {ext:15} {lang}") ``` -------------------------------- ### Add and format context Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-treecontext.md Populate contextual information around marked lines and generate the final formatted output. ```python tc.add_lines_of_interest(matches) tc.add_context() # Populate show_lines with contextual information output = tc.format() ``` -------------------------------- ### Verbose Debugging Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Enabling detailed AST and scope analysis output. ```bash grep-ast --verbose "pattern" file.py ``` ```text {0, 5, 10} 0 import sys {0, 5, 10} 1 {0, 5, 10} 2 def function(): {0, 5, 10, 15} 3 return value ``` -------------------------------- ### Optimize Performance for Large Projects Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/errors.md Mitigate performance issues by narrowing search scope or excluding heavy directories. ```bash # Search specific files/directories instead of entire project grep-ast "pattern" src/ # Exclude large directories echo "build/" >> .gitignore echo "vendor/" >> .gitignore ``` -------------------------------- ### format() -> str Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-treecontext.md Generates a formatted multi-line string showing lines of interest with their discovered context. ```APIDOC ## format() -> str ### Description Generate formatted output showing lines of interest with context. ### Returns - **str** - A multi-line string ready for printing ### Example ```python output = tc.format() print(output) ``` ``` -------------------------------- ### Importing grep-ast modules Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/module-index.md Standard import patterns for accessing the library's public API, parsers, tree-sitter utilities, and debugging tools. ```python # Public exports from package root from grep_ast import TreeContext, filename_to_lang # Parsers module from grep_ast.parsers import PARSERS, filename_to_lang # Tree-sitter compatibility from grep_ast.tsl import get_parser, get_language, USING_TSL_PACK # Debug utility from grep_ast.dump import dump, cvt # CLI (typically not imported, use entry points instead) from grep_ast.main import main, enumerate_files, process_filename ``` -------------------------------- ### Command-Line Interface Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/README.md The grep-ast command-line tool provides a simple interface for searching files and listing supported languages. ```APIDOC ## grep-ast CLI ### Usage - **grep-ast "pattern" file.py**: Basic search for a pattern in a file. - **grep-ast -i -n --color "pattern" path/**: Search with options (ignore case, line numbers, color). - **grep-ast --languages**: List all supported programming languages. ``` -------------------------------- ### Project Documentation Structure Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/MANIFEST.md Visual representation of the project's documentation file hierarchy. ```text README.md (master index) ├── quick-start.md (entry point) ├── api-reference-* (5 files) ├── cli-reference.md (complete CLI) ├── module-index.md (structure) ├── types.md (type info) ├── configuration.md (options) ├── errors.md (error handling) └── architecture.md (internals) ``` -------------------------------- ### Execute grep-ast via Command Line Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/module-index.md Run the tool from the terminal using either the grep-ast or gast command. ```bash grep-ast [options] [pattern] [filenames...] gast [options] [pattern] [filenames...] ``` -------------------------------- ### Performance: Search specific directories Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Target specific directories instead of the entire filesystem to improve performance. ```bash grep-ast "pattern" src/ # Good grep-ast "pattern" / # Slow ``` -------------------------------- ### Tree-Sitter Import Fallback Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Implements import-time detection to choose between different tree-sitter language packages. ```python try: from tree_sitter_language_pack import get_parser, get_language USING_TSL_PACK = True except ImportError: from tree_sitter_languages import get_parser, get_language USING_TSL_PACK = False ``` -------------------------------- ### Combine with Git Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Integrate with git commands to search only tracked or recently changed files. ```bash # Search only committed files git ls-files | xargs grep-ast "pattern" # Search files changed in latest commit git diff HEAD~1 --name-only | xargs grep-ast "pattern" ``` -------------------------------- ### Import grep-ast Library Components Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/module-index.md Use these imports to access the core TreeContext class, language mapping utilities, and debugging tools. ```python from grep_ast import TreeContext, filename_to_lang from grep_ast.parsers import PARSERS from grep_ast.dump import dump ``` ```python from grep_ast import TreeContext, filename_to_lang ``` -------------------------------- ### Search multiple directories Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Specify multiple directories to narrow the search scope in large projects. ```bash grep-ast "pattern" src/ lib/ # Specific directories ``` -------------------------------- ### Performance: Limit scope Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Search specific files to reduce overhead on large repositories. ```bash grep-ast "pattern" file1.py file2.py # Good grep-ast "pattern" . # Slow on large repos ``` -------------------------------- ### Import TreeContext Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-treecontext.md Import the TreeContext class from the grep_ast package. ```python from grep_ast import TreeContext ``` -------------------------------- ### Find Import Statements Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Search for import or require statements in Python and JavaScript. ```bash # Python grep-ast "^import|^from" . # JavaScript grep-ast "^import|^require" . ``` -------------------------------- ### Find Specific Code Patterns Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Search for common programming patterns such as exception handling, assignments, and string formatting. ```bash # Exception handling grep-ast "try:|except" . # Variable assignments grep-ast "^\s*\w+\s*=" . # String formatting grep-ast "f'|f\"" . ``` -------------------------------- ### Find patterns using CLI Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/00-START-HERE.md Search for strings or regex patterns within files or directories using the command line. ```bash grep-ast "TODO" . grep-ast -i "TODO\|FIXME" src/ ``` ```bash grep-ast "def " myfile.py # Python grep-ast "function|const.*=" myfile.js # JavaScript ``` ```bash grep-ast -n "class " $(find . -name "*.py") ``` ```bash grep-ast --color "pattern" . | less -R ``` -------------------------------- ### Module Dependency Graph Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Mapping of internal module imports and external library dependencies for the grep-ast package. ```text grep_ast/__init__.py ├── grep_ast.TreeContext └── parsers.filename_to_lang grep_ast/grep_ast.py ├── parsers.filename_to_lang ├── tsl.get_parser └── tree_sitter (external) grep_ast/main.py ├── grep_ast.TreeContext ├── parsers.PARSERS ├── pathspec (external) └── argparse (stdlib) grep_ast/parsers.py ├── tsl.USING_TSL_PACK └── os (stdlib) grep_ast/tsl.py ├── tree_sitter_language_pack (optional) └── tree_sitter_languages (fallback) grep_ast/dump.py ├── traceback (stdlib) └── json (stdlib) ``` -------------------------------- ### Search specific directories Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Target specific directories to avoid scanning the entire project. ```bash grep-ast "pattern" src/ # Instead of entire project ``` -------------------------------- ### Configure Tree-Sitter parser fallback Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Attempts to import the language pack with a fallback to the lighter tree-sitter-languages package. ```python try: from tree_sitter_language_pack import get_parser USING_TSL_PACK = True except ImportError: from tree_sitter_languages import get_parser USING_TSL_PACK = False ``` -------------------------------- ### Output with Line Numbers Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Display line numbers for search results. ```bash grep-ast -n "pattern" file.py ``` -------------------------------- ### get_parser().parse() Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-tsl.md Parses source code into a Tree-Sitter AST. ```APIDOC ## get_parser().parse() ### Description Parses the provided source code and returns a `tree_sitter.Tree` object containing the AST. ### Parameters - **source_code** (bytes) - Required - UTF-8 encoded source code to be parsed. ### Returns - `tree_sitter.Tree`: An object containing a `root_node` attribute representing the root of the AST. ``` -------------------------------- ### Find Class Definitions Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Search for class definitions within files or directories. ```bash # Python grep-ast "class " . # Java grep-ast "class\s+\w+" src/ ``` -------------------------------- ### Process multiple targets Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Specify multiple files and directories in a single command. ```bash grep-ast "pattern" file1.py file2.py src/ lib/ # Processes all ``` -------------------------------- ### Performance: Exclude artifacts Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Exclude build artifacts and dependencies from search paths. ```text node_modules/ venv/ build/ ``` -------------------------------- ### Configure Ignore Patterns Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md Exclude directories from search by defining them in a .gitignore file. ```text node_modules/ build/ dist/ .venv/ ``` -------------------------------- ### Gitignore Handling Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Controlling whether .gitignore patterns are respected. ```bash # Respect .gitignore (default) grep-ast "pattern" . # Skips files matching .gitignore # Ignore .gitignore (search everything) grep-ast --no-gitignore "pattern" . # Searches all files ``` -------------------------------- ### Run grep-ast with Ignore Patterns Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md Execute a search after configuring ignore patterns. ```bash grep-ast "pattern" . ``` -------------------------------- ### Enumerate files with gitignore support Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md Recursively list files while applying gitignore rules via a pathspec object. ```python from pathlib import Path import pathspec # Create a pathspec from .gitignore with open('.gitignore') as f: spec = pathspec.PathSpec.from_lines('gitwildmatch', f) # Enumerate files from grep_ast.main import enumerate_files for filepath in enumerate_files(['.'], spec): print(filepath) ``` -------------------------------- ### Integrate grep-ast in Python Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/00-START-HERE.md Use the TreeContext class to programmatically search code and extract context-aware results. ```python from grep_ast import TreeContext with open('file.py') as f: code = f.read() tc = TreeContext('file.py', code) results = tc.grep(r'def \w+', ignore_case=False) tc.add_lines_of_interest(results) tc.add_context() print(tc.format()) ``` -------------------------------- ### Search code via CLI Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/00-START-HERE.md Perform searches on files or directories using standard grep-like flags. ```bash grep-ast "pattern" file.py grep-ast -i -n --color "TODO" . ``` -------------------------------- ### Piping Output Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Pipe search results to other command-line utilities. ```bash # Pipe to less grep-ast "pattern" . | less -R # Pipe to grep (second filter) grep-ast "pattern" . | grep "filter" # Pipe to wc (count) grep-ast "pattern" . | wc -l # Save to file grep-ast "pattern" . > results.txt ``` -------------------------------- ### Define show lines set Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Tracks the set of line numbers to be included in the final output. ```python show_lines = {0, 1, 5, 6, 7, 20} # These lines will be shown ``` -------------------------------- ### Configure color output precedence Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/configuration.md Demonstrates how command-line flags override default color detection behavior. ```bash # Color detection precedence grep-ast pattern # Auto-detect (TTY?) grep-ast --color pattern # Force color grep-ast --no-color pattern # Force no color ``` -------------------------------- ### Implement lazy context discovery Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Separates the marking of lines of interest from the expensive context computation phase. ```python # Phase 1: Simple self.lines_of_interest.update(line_nums) # Phase 2: Expensive self.add_context() # Compute what to show ``` -------------------------------- ### Import the dump utility Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-dump.md Required import statement to access the dump function. ```python from grep_ast.dump import dump ``` -------------------------------- ### Find Function Definitions Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Search for function definitions in Python and JavaScript files. ```bash # Python grep-ast "def " myfile.py # JavaScript grep-ast "function|const.*=" myfile.js ``` -------------------------------- ### Process a single file Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-main.md Search for a pattern within a specific file using an argparse namespace for configuration. ```python import argparse from grep_ast.main import process_filename args = argparse.Namespace( pattern='def ', encoding='utf8', color=True, verbose=False, line_number=True, ignore_case=False ) process_filename('myfile.py', args) ``` -------------------------------- ### Count matches Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Pipe the output to wc -l to count the number of occurrences. ```bash grep-ast "TODO" . | wc -l ``` -------------------------------- ### Create PathSpec from lines Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/types.md Initializes a PathSpec object using gitwildmatch patterns to filter files. ```python pathspec.PathSpec.from_lines("gitwildmatch", lines) ``` -------------------------------- ### walk_tree(node, depth=0) Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-treecontext.md Recursively traverses the AST tree to populate internal tracking structures for scopes, headers, and nodes. ```APIDOC ## walk_tree(node, depth=0) ### Description Recursively walk the AST tree and populate scope, header, and node tracking structures. This method is called during construction. ### Parameters - **node** (tree_sitter.Node) - Required - A tree-sitter Node (initially the root node). - **depth** (int) - Optional - Default: 0. Current recursion depth (used for verbose output indentation). ### Returns - **tuple[int, int]** - A tuple of (start_line, end_line). ``` -------------------------------- ### Find Function Definitions Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/cli-reference.md Search for function definitions across different programming languages. ```bash # Python grep-ast "def " myfile.py # JavaScript grep-ast "function\s+\w+|const\s+\w+\s*=" main.js # Multiple languages grep-ast "def |function " . ``` -------------------------------- ### Configure TreeContext Parameters Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/configuration.md Adjust TreeContext settings to control output formatting, such as color, line numbers, and context depth. ```python # Minimal output tc = TreeContext( filename, code, color=False, line_number=False, margin=0, loi_pad=0, parent_context=False, child_context=False ) # Maximum context tc = TreeContext( filename, code, color=True, line_number=True, margin=5, loi_pad=3, parent_context=True, child_context=True, header_max=20 ) ``` -------------------------------- ### TreeContext Constructor Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/configuration.md Initializes a new TreeContext instance with specific configuration for source code analysis and output formatting. ```APIDOC ## TreeContext(filename, code, ...) ### Description Initializes the TreeContext object to process source code and generate context-aware output based on the provided configuration parameters. ### Parameters - **filename** (str) - Required - Source code filename used to determine the language. - **code** (str) - Required - Complete source code as a string. - **color** (bool) - Optional - Enable ANSI color codes in output for highlighting matched text. Default: False. - **verbose** (bool) - Optional - Print debugging output showing scope and AST structure to stdout. Default: False. - **line_number** (bool) - Optional - Include line numbers in formatted output. Default: False. - **parent_context** (bool) - Optional - Include parent scope headers when showing lines of interest. Default: True. - **child_context** (bool) - Optional - Include relevant child nodes when showing lines of interest. Default: True. - **last_line** (bool) - Optional - Include the final line of the file with its parent context. Default: True. - **margin** (int) - Optional - Number of lines from top of file to include as margin context. Default: 3. - **mark_lois** (bool) - Optional - Mark lines of interest with a special character (█) in output. Default: True. - **header_max** (int) - Optional - Maximum lines to show as scope header. Default: 10. - **show_top_of_file_parent_scope** (bool) - Optional - Include file-level scope headers even without a body. Default: True. - **loi_pad** (int) - Optional - Number of padding lines around each line of interest to include. Default: 1. ``` -------------------------------- ### Cache scope headers Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/architecture.md Precomputes scope headers during the AST walk to optimize output generation speed. ```python # During walk_tree: if size > 0: # Multi-line scope self.header[start_line].append((size, start_line, end_line)) # Later, truncate to header_max: if len(header) > header_max: head_end = head_start + header_max ``` -------------------------------- ### Parse source code with TSL Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-tsl.md Use get_parser to retrieve a parser instance and parse bytes into an AST. ```python def parse(source_code: bytes) -> tree_sitter.Tree ``` ```python from grep_ast.tsl import get_parser parser = get_parser("python") code = b"def hello(): pass" tree = parser.parse(code) root = tree.root_node # Inspect the AST print(f"Root type: {root.type}") print(f"Root start: {root.start_point}") print(f"Root end: {root.end_point}") ``` -------------------------------- ### Find Function Calls Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/quick-start.md Search for specific function calls like print or console.log. ```bash grep-ast "print\(|console\.log\(" . ``` -------------------------------- ### Walk the AST Tree Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-treecontext.md Recursively traverses the AST to populate internal tracking structures for scopes, headers, and nodes. ```python def walk_tree(self, node, depth: int = 0) -> tuple[int, int] ``` -------------------------------- ### get_parser() Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/module-index.md Factory function to retrieve a tree-sitter parser instance. ```APIDOC ## function get_parser() ### Description Factory function for creating or retrieving a tree-sitter parser. ### Import `from grep_ast.tsl import get_parser` ``` -------------------------------- ### Check TSL availability Source: https://github.com/aider-ai/grep-ast/blob/main/_autodocs/api-reference-tsl.md Use the USING_TSL_PACK flag to determine which language provider is currently active. ```python USING_TSL_PACK: bool ``` ```python from grep_ast.tsl import USING_TSL_PACK if USING_TSL_PACK: print("Comprehensive language pack is available") else: print("Using fallback tree-sitter-languages") ```