### Install Project Dependencies Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/docs/README.md Installs the necessary Node.js dependencies for the project. Run this after cloning the repository. ```bash npm install ``` -------------------------------- ### RAGgrep Indexing Command Examples Source: https://github.com/conradkoh/raggrep/blob/master/README.md Provides examples of how to initiate the RAGgrep indexing process, including specifying a directory, enabling watch mode, and setting concurrency. ```bash raggrep index # Index current directory raggrep index --dir ../other-repo # Index another path without cd raggrep index --watch # Watch mode - re-index on file changes raggrep index --verbose # Show detailed progress raggrep index --concurrency 8 # Set parallel workers (default: auto) raggrep index --model bge-small-en-v1.5 # Use specific embedding model ``` -------------------------------- ### Install Raggrep From Source Source: https://github.com/conradkoh/raggrep/blob/master/docs/getting-started.md Install raggrep by cloning the repository and building from source. This method is useful for development or if you need the latest unreleased changes. ```bash git clone https://github.com/conradkoh/raggrep.git cd raggrep bun install bun run build npm link ``` -------------------------------- ### Basic raggrep query example Source: https://github.com/conradkoh/raggrep/blob/master/docs/cli-reference.md A simple example of performing a natural language search for 'user authentication'. ```bash # Basic search raggrep query "user authentication" ``` -------------------------------- ### Force Raggrep Skill Installation Source: https://github.com/conradkoh/raggrep/blob/master/README.md Use this command to force the installation of Raggrep as a skill. This installs it to `~/.config/opencode/skill/raggrep/SKILL.md` for skill-based integration. ```bash raggrep opencode install --skill ``` -------------------------------- ### Configuration Search Examples Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/README.md Queries specifically targeting configuration files. These examples show how Raggrep can locate settings and parameters within JSON files. ```text "rate limit" → `config/default.json` ``` ```text "JWT expiry" → `config/default.json` ``` -------------------------------- ### Install Raggrep Globally with Bun Source: https://github.com/conradkoh/raggrep/blob/master/docs/getting-started.md Install raggrep globally using Bun. This provides an alternative to npm for global installation. ```bash # Or with Bun bun install -g raggrep ``` -------------------------------- ### Install PostgreSQL Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/docs/database.md Installs PostgreSQL version 14 on macOS using Homebrew or Ubuntu using apt. ```bash # macOS brew install postgresql@14 # Ubuntu sudo apt install postgresql-14 ``` -------------------------------- ### Force Raggrep Tool Installation Source: https://github.com/conradkoh/raggrep/blob/master/README.md Use this command to force the installation of Raggrep as a tool. This installs it to `~/.config/opencode/tool/raggrep.ts` and enables direct tool execution with full functionality. ```bash raggrep opencode install --tool ``` -------------------------------- ### Quick Start: Index and Search Directory Source: https://github.com/conradkoh/raggrep/blob/master/docs/sdk.md Demonstrates the basic workflow of indexing a directory and performing a search using the RAGgrep SDK. ```typescript import raggrep from "raggrep"; // Index a directory await raggrep.index("./my-project"); // Search const results = await raggrep.search("./my-project", "user authentication"); console.log(raggrep.formatSearchResults(results)); ``` -------------------------------- ### Run Development Server Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/docs/README.md Starts the project server in development mode. The server will typically run on http://localhost:3000. ```bash npm run dev ``` -------------------------------- ### Copy Environment Configuration Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/docs/README.md Copies the example environment file to create a new configuration file. This file should then be populated with actual environment variables. ```bash cp .env.example .env ``` -------------------------------- ### Install Raggrep for OpenCode Source: https://github.com/conradkoh/raggrep/blob/master/README.md Install raggrep as a tool for OpenCode integration. This is the recommended method for optimal activation rates within the AI coding assistant. ```bash raggrep opencode install ``` -------------------------------- ### Mutual Exclusivity: Install Tool (Prompts Skill Removal) Source: https://github.com/conradkoh/raggrep/blob/master/README.md Installing Raggrep as a tool will prompt the user to remove an existing skill installation. This ensures only one installation type is active. ```bash # Installing tool will prompt to remove existing skill raggrep opencode install --tool ``` -------------------------------- ### Example: Adding a New Feature - Code Structure Source: https://github.com/conradkoh/raggrep/blob/master/AGENTS.md Provides a directory structure example for adding a new feature ('code complexity analysis'), showing the placement of entities, ports, services, use cases, infrastructure adapters, and app coordinators. ```treeview src/ ├── domain/ │ ├── entities/ │ │ └── complexity.ts # ComplexityScore type │ ├── ports/ │ │ └── complexity.ts # IComplexityAnalyzer interface │ ├── services/ │ │ └── complexity.ts # calculateComplexity() pure function │ └── usecases/ │ └── analyzeComplexity.ts # Orchestrates analysis │ ├── infrastructure/ │ └── complexity/ │ └── tsComplexity.ts # TypeScript AST-based implementation │ └── app/ └── complexity/ └── index.ts # Coordinates analysis workflow ``` -------------------------------- ### Install RAGgrep Globally Source: https://github.com/conradkoh/raggrep/blob/master/docs/README.md Install the RAGgrep CLI tool globally using npm. This command is typically run once to set up the tool on your system. ```bash npm install -g raggrep ``` -------------------------------- ### Install RAGgrep with npm or Bun Source: https://github.com/conradkoh/raggrep/blob/master/docs/sdk.md Install the RAGgrep package using either npm or Bun package managers. ```bash # With npm npm install raggrep # With Bun bun add raggrep ``` -------------------------------- ### Mutual Exclusivity: Install Skill (Prompts Tool Removal) Source: https://github.com/conradkoh/raggrep/blob/master/README.md Installing Raggrep as a skill will prompt the user to remove an existing tool installation. This ensures only one installation type is active. ```bash # Installing skill will prompt to remove existing tool raggrep opencode install --skill ``` -------------------------------- ### Exact Symbol Matching Examples Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/README.md Examples of queries designed to find specific function or class names within the codebase. These queries rely on precise symbol matching. ```text authenticateUser → src/auth/login.ts ``` ```text validateSession → src/auth/session.ts ``` ```text sendPasswordResetEmail → src/services/email.ts ``` -------------------------------- ### Basic RAGgrep Query Examples Source: https://github.com/conradkoh/raggrep/blob/master/README.md Demonstrates fundamental query operations, including natural language searches, searching specific projects, exact matching, and limiting results. ```bash raggrep query "user login" # Natural language query raggrep query -C ~/projects/my-app "login" # Search a project without cd raggrep query "AUTH_SERVICE_URL" # Exact identifier (auto-triggers exact match) raggrep query "\`AuthService\`" # Backticks force exact match raggrep query "error handling" --top 5 # Limit results raggrep query "database" --min-score 0.2 # Set minimum score threshold raggrep query "login flow" --rank-by semantic # Order by semantic similarity first raggrep query "auth" --rank-by combined # Order by fused score only raggrep query "debug" --timing # Print timing breakdown raggrep query "interface" --type ts # Filter by file extension raggrep query "auth" --filter src/auth # Filter by path raggrep query "api" -f src/api -f src/routes # Multiple path filters ``` -------------------------------- ### Example SSE Query Flow Source: https://github.com/conradkoh/raggrep/blob/master/docs/design/structured-semantic-expansion.md Illustrates the step-by-step process of expanding a query using the SSE lexicon. This flow includes tokenization, term expansion with grades, weighted query construction, and search execution. ```plaintext Query: "find auth function" 1. Term tokenization: ["find", "auth", "function"] 2. Term expansion: - find → search, locate (moderate) - auth → authentication, authorization (strong), login, security (weak) - function → method (strong), handler, callback (moderate) 3. Weighted query construction: Original terms: weight = 1.0 Strong synonyms: weight = 0.9 Moderate synonyms: weight = 0.6 Weak synonyms: weight = 0.3 4. Search execution: Semantic search with expanded, weighted terms 5. Result scoring: score = Σ(term_match_score * term_weight) ``` -------------------------------- ### Raggrep Skill Usage Steps Source: https://github.com/conradkoh/raggrep/blob/master/README.md After loading the Raggrep skill, follow these steps: install the Raggrep CLI globally, index your codebase, and then use the `query` command for semantic search. ```bash npm install -g raggrep raggrep index raggrep query "your search term" ``` -------------------------------- ### CLI Display of Search Results Source: https://github.com/conradkoh/raggrep/blob/master/docs/design/introspection.md Example of how search results are presented in the command-line interface, indicating the file, line range, contributing module, and score. ```text 1. src/auth/login.ts:10-25 (handleLogin) Score: 85.2% | Type: function | via TypeScript | exported ``` -------------------------------- ### Cross-Type Search Examples Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/README.md Queries demonstrating the ability to find related content across different file types, such as code and documentation. These examples highlight Raggrep's unified search. ```text "authentication" → Code in `src/auth/` AND `docs/authentication.md` ``` ```text "database setup" → Code in `src/database/` AND `docs/database.md` ``` -------------------------------- ### raggrep query output format example Source: https://github.com/conradkoh/raggrep/blob/master/docs/cli-reference.md An example of the output format for raggrep query results, showing file path, line numbers, score, code type, and a code preview. ```text Found 3 results: 1. src/auth/authService.ts:24-55 (login) Score: 34.4% | Type: function | via TypeScript | exported export async function login(credentials: LoginCredentials): Promise { 2. src/users/types.ts:3-12 (User) Score: 26.0% | Type: interface | via TypeScript | exported export interface User { ``` -------------------------------- ### Use Raggrep Without Installing Source: https://github.com/conradkoh/raggrep/blob/master/docs/getting-started.md Execute raggrep commands directly using npx without a global installation. This is useful for quick, one-off searches. ```bash npx raggrep query "user login" ``` -------------------------------- ### JSON Path Extraction: Simple Object Source: https://github.com/conradkoh/raggrep/blob/master/codemaps/json-key-literal-indexing.codemap.md Example of extracting JSON paths from a simple object configuration file. ```json // config.json { "port": 3000, "host": "localhost" } ``` -------------------------------- ### Example RAGgrep Configuration File Source: https://github.com/conradkoh/raggrep/blob/master/docs/configuration.md This JSON file demonstrates how to configure RAGgrep, including specifying file extensions, paths to ignore, and module-specific options like the embedding model for TypeScript. ```json { "extensions": [".ts", ".tsx"], "ignorePaths": ["node_modules", ".git", "dist", "build", "__tests__"], "modules": [ { "id": "core", "enabled": true }, { "id": "language/typescript", "enabled": true, "options": { "embeddingModel": "bge-small-en-v1.5" } } ] } ``` -------------------------------- ### Example Queries for Literal Boosting Source: https://github.com/conradkoh/raggrep/blob/master/docs/design/literal-boosting.md Users can explicitly mark literals in their queries using backticks or quotes to ensure exact matching. ```text find the `AuthService` class where is "handleLogin" used ``` -------------------------------- ### RAGgrep Exact Match Search Examples Source: https://github.com/conradkoh/raggrep/blob/master/README.md Demonstrates how RAGgrep automatically performs exact matching for identifier-like queries and how to force exact matching using backticks. ```bash # Finds AUTH_SERVICE_URL in ALL file types (YAML, .env, config, etc.) raggrep query "AUTH_SERVICE_URL" # Finds the function by exact name raggrep query "getUserById" # Use backticks for explicit exact matching (even natural words) raggrep query "\`configuration\`" ``` -------------------------------- ### Query Scenario with Raggrep CLI Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/README.md Perform a semantic search query using the Raggrep CLI. This example searches for 'user authentication' and returns the top 5 results. ```bash # Run a query bun run ../../src/app/cli/main.ts query "user authentication" --top 5 ``` -------------------------------- ### Literal Index Example Source: https://github.com/conradkoh/raggrep/blob/master/docs/architecture.md Illustrates the mapping of an identifier name to its chunk location and vocabulary within the literal index. ```json literals/\n└── _index.json (literal → chunkId + filepath + vocabulary)\n\n Example:\n "getUserById" → {\n chunkId: "...",\n vocabulary: ["get", "user", "by", "id"]\n } ``` -------------------------------- ### Hybrid Scoring Configuration Example Source: https://github.com/conradkoh/raggrep/blob/master/docs/architecture.md Details the scoring weights for the semantic track in RAGgrep, showing how different matching strategies are combined. ```typescript TypeScript: tw.semantic×Sem + tw.bm25×BM25 + tw.vocab×Vocab\n (defaults ≈ 0.43 / 0.42 / 0.15; tune via RankingWeightsConfig) ``` -------------------------------- ### Semantic/Conceptual Search Examples Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/README.md Examples of natural language queries that test Raggrep's ability to understand concepts rather than just keywords. These queries should find relevant code based on meaning. ```text "how to verify user password" → Should find auth code ``` ```text "database transaction handling" → Should find connection.ts ``` -------------------------------- ### Raggrep Command Examples Source: https://github.com/conradkoh/raggrep/blob/master/README.md Common raggrep commands for managing the index and showing information. Use these to check index status, reset it, or view the tool's version. ```bash raggrep status # Show index status and statistics raggrep status --dir ./packages/api raggrep reset # Clear the index for the current directory raggrep reset -C ~/projects/my-app raggrep --version # Show version ``` -------------------------------- ### Introspection Index Example Source: https://github.com/conradkoh/raggrep/blob/master/docs/architecture.md Shows the structure of file metadata stored in the introspection index, including nearest README, path context, and conventions. ```json introspection/\n└── files/\n └── src/auth/session.json\n {\n "nearestReadme": "src/auth/README.md",\n "pathContext": {...},\n "conventions": [...]\n } ``` -------------------------------- ### Vocabulary Index Structure Example Source: https://github.com/conradkoh/raggrep/blob/master/docs/plans/vocabulary-scoring.md Demonstrates the structure of the vocabulary index, showing how identifiers are decomposed into their constituent vocabulary words. ```text "validateUserSession" → vocabulary: ["validate", "user", "session"] "authenticateUser" → vocabulary: ["authenticate", "user"] "SessionValidator" → vocabulary: ["session", "validator"] ``` -------------------------------- ### Query Vocabulary Matching Example Source: https://github.com/conradkoh/raggrep/blob/master/docs/architecture.md Illustrates how query vocabulary terms are matched against indexed identifiers. This shows which identifiers would be considered matches for specific query terms. ```text Query "user" matches getUserById, UserService, fetchUserData Query "where is user session validated" matches validateUserSession (overlap: user, session, validate*) ``` -------------------------------- ### Example Query and Expected Outcome Source: https://github.com/conradkoh/raggrep/blob/master/docs/plans/vocabulary-scoring.md Illustrates a long query with a phrase match and the desired outcome where a semantically relevant chunk ranks highly. ```text Query: "where is the user authentication session validated" Expected: A chunk containing `validateUserSession()` or text about "user authentication session" should rank highly. ``` -------------------------------- ### Configuration Override for Introspection Source: https://github.com/conradkoh/raggrep/blob/master/docs/design/introspection.md Example of overriding auto-detected project settings like scope and framework in the config.json file. Useful for custom monorepo structures. ```json { "introspection": { "projects": { "apps/webapp": { "scope": "frontend", "framework": "nextjs" }, "apps/api": { "scope": "backend", "framework": "express" } } } } ``` -------------------------------- ### Configure Indexed File Extensions Source: https://github.com/conradkoh/raggrep/blob/master/docs/advanced.md Customize the `extensions` array in `config.json` to specify which file types RAGgrep should index. This example limits indexing to TypeScript files. ```json { "extensions": [".ts", ".tsx"] } ``` -------------------------------- ### Example TypeScript Module Index Layout Source: https://github.com/conradkoh/raggrep/blob/master/docs/architecture.md Shows the specific directory structure for a TypeScript module within the RAGgrep index. This includes manifest, symbolic, literals, and source code chunk information. ```tree .raggrep/index/language/typescript/ ├── manifest.json ├── symbolic/ │ ├── _meta.json │ └── src/auth/authService.json # File summary + keywords ├── literals/ │ └── _index.json └── src/auth/ └── authService.json # Chunks + embeddings ``` -------------------------------- ### Navigate to Project Directory Source: https://github.com/conradkoh/raggrep/blob/master/docs/getting-started.md Before running a search, navigate to your project's root directory using the 'cd' command. ```bash cd your-project ``` -------------------------------- ### Create PostgreSQL Databases Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/docs/database.md Creates development and testing databases for the application. ```bash createdb myapp_dev createdb myapp_test ``` -------------------------------- ### Creating a New Scenario Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/README.md Steps to create a new test scenario, including making a directory, adding documentation, test files, and an optional sanity check script. ```bash mkdir scenarios/my-scenario ``` -------------------------------- ### extractJsonPaths Function Signature and Example Source: https://github.com/conradkoh/raggrep/blob/master/codemaps/json-key-literal-indexing.codemap.md Extracts all key paths from a JSON object as literals, prefixing them with the filename. This function is used to generate literals for JSON file indexing. The example demonstrates its usage with a nested object. ```typescript /** * Extract all key paths from a JSON object as literals. * Prefixes all paths with the filename (without extension). * * @param obj - Parsed JSON object * @param fileBasename - Filename without extension (e.g., "user" from "user.json") * @returns Array of literals representing all dot-notation paths * * @example * // user.json: { name: { first: "john" } } * extractJsonPaths({ name: { first: "john" } }, "user") * // Returns: [ * // { value: "user.name", type: "identifier", matchType: "definition" }, * // { value: "user.name.first", type: "identifier", matchType: "definition" } * // ] */ export function extractJsonPaths( obj: unknown, fileBasename: string ): ExtractedLiteral[]; ``` -------------------------------- ### Show Raggrep Help and Commands Source: https://github.com/conradkoh/raggrep/blob/master/docs/cli-reference.md Displays the help message and lists all available commands for the Raggrep CLI. ```bash raggrep --help ``` ```bash raggrep -h ``` -------------------------------- ### Directory Structure of Test Scenarios Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/README.md Illustrates the organization of the test scenarios folder, including the README, basic project structure, and placeholders for future scenarios. ```bash scenarios/ ├── README.md # This file ├── basic/ # Basic multi-file TypeScript/JS project │ ├── README.md # Scenario-specific documentation │ ├── run-sanity-checks.sh │ ├── test-queries.md │ └── src/... # Test source files └── [future-scenario]/ # Add new scenarios here ``` -------------------------------- ### Indexing Flow Steps Source: https://github.com/conradkoh/raggrep/blob/master/docs/architecture.md Outlines the sequential steps involved in the indexing process, from parsing arguments to building and persisting various indexes. ```plaintext 1. CLI parses arguments 2. Load config from index directory (or use defaults) 3. Find all matching files (respecting ignore patterns) 4. For each file (in parallel): a. Parse into chunks via AST (functions, classes, etc.) b. Generate embeddings for each chunk c. Extract keywords for symbolic index d. Extract literals from chunk names (for literal index) e. Write per-file index to index// 5. Build and persist BM25 index 6. Build and persist literal index 7. Update manifests ``` -------------------------------- ### Manage Database Migrations Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/docs/database.md Commands to run pending database migrations and create new migration files using npm scripts. ```bash # Run pending migrations npm run db:migrate # Create new migration npm run db:migrate:create add_phone_to_users ``` -------------------------------- ### Index Scenario with Raggrep CLI Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/README.md Use the Raggrep command-line interface to index the files in the current scenario. This prepares the data for searching. ```bash # Index the scenario bun run ../../src/app/cli/main.ts index ``` -------------------------------- ### Build a Code Search API with Express Source: https://github.com/conradkoh/raggrep/blob/master/docs/sdk.md Sets up an Express.js server to index a project directory and provide a search endpoint. The API indexes the project on startup and handles search queries with optional path filtering. ```typescript import express from "express"; import raggrep from "raggrep"; const app = express(); const PROJECT_DIR = "./my-project"; // Index on startup await raggrep.index(PROJECT_DIR); app.get("/search", async (req, res) => { const query = req.query.q as string; const path = req.query.path as string | undefined; const results = await raggrep.search(PROJECT_DIR, query, { topK: 20, pathFilter: path ? [path] : undefined, }); res.json(results); }); app.listen(3000); ``` -------------------------------- ### Load Raggrep Skill in OpenCode Source: https://github.com/conradkoh/raggrep/blob/master/README.md To use Raggrep as a skill within OpenCode, load it using the `skill` function with its name. This is the first step before installing and indexing. ```javascript skill({ name: "raggrep" }) ``` -------------------------------- ### Running Tests on a Scenario Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/README.md Commands to navigate to a scenario directory, index the codebase, and run search queries using RAGgrep's CLI. ```bash # Navigate to the scenario folder cd scenarios/basic # Index the scenario bun run ../../src/app/cli/main.ts index # Run queries bun run ../../src/app/cli/main.ts query "your search query" --top 5 # Or use the sanity check script (if available) ./run-sanity-checks.sh ``` -------------------------------- ### Configure Ignored Paths Source: https://github.com/conradkoh/raggrep/blob/master/docs/advanced.md Modify the `ignorePaths` array in `config.json` to exclude specific directories from indexing. This example adds custom paths like `__tests__` and `generated`. ```json { "ignorePaths": [ "node_modules", ".git", "dist", "build", "__tests__", // Add your own "generated" // Add your own ] } ``` -------------------------------- ### Perform Your First Code Search Source: https://github.com/conradkoh/raggrep/blob/master/docs/getting-started.md Execute a search query using raggrep. The index is created automatically on the first query. The embedding model will be downloaded and cached if it's the first run. ```bash raggrep query "handle user login" ``` -------------------------------- ### Integrating RAGgrep Tests with CI Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/README.md Command to run RAGgrep integration tests, which utilize files from the 'scenarios/basic/' directory. ```bash bun test src/tests/integration.test.ts ``` -------------------------------- ### Vocabulary Extraction Example Source: https://github.com/conradkoh/raggrep/blob/master/docs/architecture.md Illustrates how RAGgrep extracts vocabulary from identifiers at index time. This process breaks down camelCase and snake_case identifiers into individual words for broader matching. ```text getUserById → ["get", "user", "by", "id"] AuthService → ["auth", "service"] validate_session → ["validate", "session"] ``` -------------------------------- ### Index Directory with Options Source: https://github.com/conradkoh/raggrep/blob/master/docs/sdk.md Index a directory with custom options for embedding model, verbosity, and concurrency. Shows how to access results per module. ```typescript const results = await raggrep.index("./my-project", { model: "bge-small-en-v1.5", // Optional: embedding model verbose: true, // Optional: show progress concurrency: 8, // Optional: parallel workers }); // Results per module for (const result of results) { console.log(`${result.moduleId}: ${result.indexed} indexed`); } ``` -------------------------------- ### Limit RAGgrep Query Results Source: https://github.com/conradkoh/raggrep/blob/master/docs/advanced.md Control the number of search results returned by `raggrep query` using the `--top` flag. This example limits the results to 5. ```bash raggrep query "user" --top 5 # Return 5 results (default: 10) ``` -------------------------------- ### Run Project Tests Source: https://github.com/conradkoh/raggrep/blob/master/scenarios/basic/docs/README.md Executes the test suite for the project. Ensure all tests pass to verify the project's integrity. ```bash npm test ``` -------------------------------- ### Index with Verbose Output Source: https://github.com/conradkoh/raggrep/blob/master/docs/cli-reference.md Indexes the current directory and displays detailed progress for each file processed. ```bash raggrep index --verbose ``` -------------------------------- ### Integrating Semantic Expansion into Search Queries Source: https://github.com/conradkoh/raggrep/blob/master/codemaps/structured-semantic-expansion.codemap.md Illustrates how to integrate the default lexicon into the search process. It shows query expansion using synonyms, generating embeddings from the expanded query, and tracking used synonyms in the search results context. ```typescript // In search() method: // 1. Expand query with synonyms const expandedQuery = expandQuery(query, DEFAULT_LEXICON, { maxDepth: 1, includeWeak: true, }); // 2. Use expanded query for embedding const queryEmbedding = await getEmbedding(expandedQuery.expandedQueryString); // 3. Track expansion in result context results.push({ // ... context: { // existing fields... synonymsUsed: expandedQuery.expandedTerms .filter((t) => t.source !== "original") .map((t) => t.term), }, }); ``` -------------------------------- ### Start Raggrep Indexing with Watch Mode and Custom Concurrency Source: https://github.com/conradkoh/raggrep/blob/master/docs/cli-reference.md Initiates the Raggrep indexing process in watch mode, enabling automatic re-indexing upon file changes. It also specifies the number of parallel workers for indexing. ```bash raggrep index --watch --concurrency 8 ``` -------------------------------- ### Index Time: JsonModule.indexFile Source: https://github.com/conradkoh/raggrep/blob/master/codemaps/json-key-literal-indexing.codemap.md Demonstrates parsing JSON content, extracting file paths, and creating module data without embeddings. It stores pending literals for the finalize step. ```typescript // Parse JSON and extract paths const parsed = JSON.parse(content); const fileBasename = path.basename(filepath, path.extname(filepath)); const jsonPaths = extractJsonPaths(parsed, fileBasename); // Create single chunk for the file (simplified) const chunks: Chunk[] = [ { id: generateChunkId(filepath, 1, lineCount), content: content, startLine: 1, endLine: lineCount, type: "file", }, ]; // Store literals for finalize (NO embeddings generated) const literals: ExtractedLiteral[] = jsonPaths; this.pendingLiterals.set(chunk.id, { filepath, literals }); // Module data without embeddings const moduleData: JsonModuleData = { jsonPaths: jsonPaths.map((l) => l.value), }; ``` -------------------------------- ### Index and Search with a Higher-Quality Model Source: https://github.com/conradkoh/raggrep/blob/master/docs/sdk.md Demonstrates how to index a project using a specific, higher-quality embedding model ('nomic-embed-text-v1.5') and then perform searches. This can improve search relevance at the cost of increased model size and indexing time. ```typescript import raggrep from "raggrep"; // Index with nomic-embed-text-v1.5 (768 dimensions) await raggrep.index("./my-project", { model: "nomic-embed-text-v1.5", verbose: true, }); // Search works the same const results = await raggrep.search("./my-project", "authentication"); ``` -------------------------------- ### Search with Literal Boosting Source: https://github.com/conradkoh/raggrep/blob/master/codemaps/literal-boosting.codemap.md This snippet demonstrates the core search logic, including parsing query literals, executing searches, building a match map, and applying multiplicative boosting to scores based on literal matches. It also covers adding literal-only matches. ```typescript // 1. Parse query for literals const { literals: queryLiterals, remainingQuery } = parseQueryLiterals(query); // 2. Execute three searches const literalMatches = this.literalIndex.findMatches(queryLiterals); // semanticSearch uses remainingQuery // bm25Search uses full query // 3. Build match map for scoring const literalMatchMap = new Map(); for (const match of literalMatches) { const matches = literalMatchMap.get(match.chunkId) || []; matches.push(match); literalMatchMap.set(match.chunkId, matches); } // 4. Score with multiplicative boosting for (const { filepath, chunk, embedding } of allChunksData) { const semanticScore = cosineSimilarity(queryEmbedding, embedding); const bm25Score = bm25Scores.get(chunk.id) || 0; // Base hybrid score let score = SEMANTIC_WEIGHT * semanticScore + BM25_WEIGHT * bm25Score; // Apply literal multiplier const matches = literalMatchMap.get(chunk.id); let literalMultiplier = 1.0; if (matches && matches.length > 0) { // Use highest multiplier from all matches literalMultiplier = Math.max( ...matches.map((m) => calculateLiteralMultiplier( m.indexedLiteral.matchType, m.queryLiteral.confidence ) ) ); score *= literalMultiplier; } // Add other boosts score += pathBoost + fileTypeBoost + chunkTypeBoost + exportBoost; results.push({ filepath, chunk, score, moduleId: this.id, context: { semanticScore, bm25Score, literalMultiplier, // ... other fields }, }); } // 5. Add literal-only matches (not found by semantic/BM25) for (const [chunkId, matches] of literalMatchMap) { if (!resultsHaveChunk(chunkId)) { // Load chunk and add with base score const chunk = await loadChunk(chunkId); results.push({ filepath: chunk.filepath, chunk, score: LITERAL_BASE_SCORE, moduleId: this.id, context: { literalOnly: true }, }); } } ``` -------------------------------- ### Show Raggrep Version Source: https://github.com/conradkoh/raggrep/blob/master/docs/cli-reference.md Displays the current version of the Raggrep CLI. ```bash raggrep --version ``` ```bash raggrep -v ``` -------------------------------- ### Structured Semantic Expansion Flow Diagram Source: https://github.com/conradkoh/raggrep/blob/master/codemaps/structured-semantic-expansion.codemap.md Illustrates the flow of Structured Semantic Expansion at query time, from query expansion to semantic search. ```plantuml @startuml title Structured Semantic Expansion Flow == Query Time == participant "TypeScriptModule" as TSM participant "QueryExpander" as QE participant "Lexicon" as LEX participant "SemanticSearch" as SS TSM -> QE: expandQuery(query: string): ExpandedQuery QE -> LEX: getSynonyms(term: string): SynonymMatch[] LEX --> QE: synonyms with grades QE --> TSM: { originalTerms, expandedTerms, weights } note over TSM: Build weighted query for embedding TSM -> SS: search with expanded query SS --> TSM: results note over TSM: Score includes synonym contribution @enduml ``` -------------------------------- ### Correct Import Patterns in TypeScript Source: https://github.com/conradkoh/raggrep/blob/master/AGENTS.md Illustrates valid import patterns for different layers (Infrastructure, App, Domain) in TypeScript projects, emphasizing dependency rules. ```typescript // ✅ CORRECT: Infrastructure implements domain port // src/infrastructure/embeddings/xenovaEmbeddingProvider.ts import type { IEmbeddingProvider } from "../../domain/ports"; // ✅ CORRECT: App uses domain and infrastructure // src/app/indexer/index.ts import type { Config } from "../../domain/entities"; import { loadConfig } from "../../infrastructure/config"; // ✅ CORRECT: Use case accepts injected dependencies // src/domain/usecases/indexDirectory.ts import type { FileSystem } from "../ports"; ``` -------------------------------- ### Search Directory with Options Source: https://github.com/conradkoh/raggrep/blob/master/docs/sdk.md Perform a search with options to control the number of results, minimum score, file patterns, and path filters. Displays formatted output. ```typescript const results = await raggrep.search("./my-project", "user login", { topK: 5, // Number of results minScore: 0.2, // Minimum similarity (0-1) filePatterns: ["*.ts", "*.tsx"], // Filter by file type pathFilter: ["src/auth"], // Filter by path prefix }); for (const result of results) { console.log(`${result.filepath}:${result.chunk.startLine}`); console.log(` Score: ${(result.score * 100).toFixed(1)}%`); console.log(` ${result.chunk.content.slice(0, 100)}...`); } ``` -------------------------------- ### raggrep index --watch with verbose output Source: https://github.com/conradkoh/raggrep/blob/master/docs/cli-reference.md Enable verbose output for the 'raggrep index --watch' command to see detailed progress for each file being processed during the watch mode. ```bash # Watch with verbose output raggrep index --watch --verbose ``` -------------------------------- ### Configure Indexed File Extensions Source: https://github.com/conradkoh/raggrep/blob/master/docs/configuration.md Specify which file extensions RAGgrep should index by providing a list of strings in the `extensions` field of the configuration file. ```json { "extensions": [".ts", ".tsx"] } ``` -------------------------------- ### Literal Boosting Flow Sequence Diagram Source: https://github.com/conradkoh/raggrep/blob/master/codemaps/literal-boosting.codemap.md Illustrates the flow of Literal Boosting during indexing and search operations, detailing interactions between different components. ```plantuml @startuml title Literal Boosting Flow == Indexing == participant "TypeScriptModule" as TSM participant "LiteralExtractor" as LE participant "LiteralIndex" as LI TSM -> LE: extractLiterals(chunk: Chunk): ExtractedLiteral[] LE --> TSM: literals with matchType (definition/reference/import) TSM -> LI: addLiterals(chunkId: string, literals: ExtractedLiteral[]): void note right: During finalize() TSM -> LI: save(): Promise == Search == participant "QueryLiteralParser" as QLP participant "LiteralScorer" as LS TSM -> QLP: parseQueryLiterals(query: string): QueryLiteralParseResult QLP --> TSM: { literals, remainingQuery } note over TSM: Execute 3 searches in parallel TSM -> TSM: literalMatches = literalIndex.findMatches(literals) TSM -> TSM: semanticResults = semanticSearch(remainingQuery) TSM -> TSM: bm25Results = bm25Search(query) TSM -> LS: mergeAndScore(literalMatches, semanticResults, bm25Results): ScoredResult[] note right: Multiplicative boosting by match type LS --> TSM: sorted results @enduml ``` -------------------------------- ### Parser Architecture Diagram Source: https://github.com/conradkoh/raggrep/blob/master/docs/architecture.md Illustrates the parser interface and the concrete implementations for TypeScript and Tree-sitter based parsers. ```text ┌──────────────────────────────────────────────────┐ │ IParser Interface (Domain Port) │ │ - parse(content, filepath, config) │ │ - canParse(filepath) │ └──────────────────────────────────────────────────┘ │ ┌─────────────┴──────────────┐ ▼ ▼ ┌──────────────────┐ ┌──────────────────────┐ │ TypeScriptParser │ │ TreeSitterParser │ │ (TS Compiler) │ │ (web-tree-sitter) │ │ │ │ │ │ - Rich types │ │ - Multi-language │ │ - JSDoc │ │ - WASM-based │ │ - Proven │ │ - Regex fallback │ └──────────────────┘ └──────────────────────┘ ```