### Wire Up and Process with SyntaxTreePreprocessor in C# Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/README.md Illustrates the setup and execution of the SyntaxTreePreprocessor. It involves creating a resource store, adding parsed SyntaxTrees as resources, instantiating the preprocessor with a reference extractor, and processing a main resource. ```csharp using TinyAst.Preprocessor.Bridge; using TinyAst.Preprocessor.Bridge.Resources; using TinyPreprocessor.Core; // Create resource store and add your resources var store = new InMemorySyntaxTreeResourceStore(); var mainTree = SyntaxTree.ParseAndBind(mainSource, schema); var mainResource = new Resource(new ResourceId("main"), mainTree); store.Add(mainResource); // Add other resources that can be imported var libTree = SyntaxTree.ParseAndBind(libSource, schema); store.Add(new Resource(new ResourceId("lib"), libTree)); // Create the preprocessor (defaults wired for SyntaxTree content) var preprocessor = new SyntaxTreePreprocessor( store, getReference: n => n.Reference); // Process var result = await preprocessor.ProcessAsync( mainResource, context: null!, PreprocessorOptions.Default); if (result.Success) { // result.Content is the merged SyntaxTree Console.WriteLine(result.Content.ToText()); } else { foreach (var diagnostic in result.Diagnostics) { Console.WriteLine(diagnostic); } } ``` -------------------------------- ### Diagnose Resolution Failures for Missing Imports in C# Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt Explains how the TinyAST preprocessor provides detailed diagnostic information when it fails to resolve imported resources. This example shows how to set up a resource store, index import locations, and process resources to capture `ResolutionFailedDiagnostic` with source locations. ```csharp using TinyAst.Preprocessor.Bridge; using TinyAst.Preprocessor.Bridge.Resources; using TinyAst.Preprocessor.Bridge.Imports; using TinyPreprocessor; using TinyPreprocessor.Core; using TinyPreprocessor.Diagnostics; var store = new InMemorySyntaxTreeResourceStore(); var mainSource = "import \"nonexistent\"\nimport \"also-missing\"\nlet x = 1"; var mainResource = CreateResource("main", mainSource, schema); store.Add(mainResource); // Build location index for precise error locations var locationIndex = new ImportDirectiveLocationIndex(); var parser = new ImportDirectiveParser(n => n.Reference); foreach (var directive in parser.Parse(mainResource.Content, mainResource.Id)) { locationIndex.Add(mainResource.Id, directive.Reference, directive.Location); } var resolver = new InMemorySyntaxTreeResourceResolver(store, locationIndex); var preprocessor = new SyntaxTreePreprocessor(resolver, n => n.Reference); var options = new PreprocessorOptions { ContinueOnError = true }; var result = await preprocessor.ProcessAsync(mainResource, null, options); foreach (var diagnostic in result.Diagnostics.OfType()) { Console.WriteLine($"Failed to resolve: {diagnostic.Reference}"); Console.WriteLine($" In resource: {diagnostic.Resource?.Path}"); Console.WriteLine($" At location: {diagnostic.Location}"); } ``` -------------------------------- ### IContentModel Interface Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md Abstracts content manipulation operations, allowing TinyPreprocessor to work with various content types. It defines methods for getting content length and slicing. ```csharp interface IContentModel { int GetLength(TContent content); TContent Slice(TContent content, int start, int length); } ``` -------------------------------- ### Manage Syntax Tree Content Length and Slicing (C#) Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt Illustrates the use of SyntaxTreeContentModel.Instance to get the total length of a SyntaxTree and to slice it into sub-trees. The length is measured in characters, including whitespace. Slicing re-parses the specified text segment, and a zero-length slice correctly returns an empty SyntaxTree. ```csharp using TinyAst.Preprocessor.Bridge.Content; using TinyTokenizer.Ast; var model = SyntaxTreeContentModel.Instance; var tree = SyntaxTree.ParseAndBind("let x = 1\nlet y = 2", schema); // Get total content length int length = model.GetLength(tree); Console.WriteLine($"Tree length: {length}"); // Character count including whitespace // Slice content (re-parses the sliced text) SyntaxTree slice = model.Slice(tree, start: 0, length: 9); Console.WriteLine(slice.ToText()); // "let x = 1" // Zero-length slice returns empty tree SyntaxTree empty = model.Slice(tree, 0, 0); Console.WriteLine(empty == SyntaxTree.Empty); // true ``` -------------------------------- ### Represent and Create ImportDirective in C# Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt The ImportDirective record struct holds information about a parsed import directive, including its reference string, source location, and an optional resource identifier. While typically generated by a parser, it can be manually constructed for testing purposes. Example usage demonstrates accessing its properties. ```csharp using TinyAst.Preprocessor.Bridge.Imports; using TinyPreprocessor.Core; // ImportDirective is typically created by ImportDirectiveParser // but can be constructed manually for testing var directive = new ImportDirective( Reference: "path/to/module", Location: 0..0, // Zero-length range at node start position Resource: new ResourceId("main.txt")); Console.WriteLine($"Import: {directive.Reference}"); Console.WriteLine($"Location: {directive.Location}"); Console.WriteLine($"Resource: {directive.Resource?.Path}"); ``` -------------------------------- ### Manage SyntaxTree Resources with InMemorySyntaxTreeResourceStore in C# Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt The InMemorySyntaxTreeResourceStore is a thread-safe in-memory storage for SyntaxTree resources, suitable for testing and sample scenarios. It allows adding new resources, retrieving existing ones by ResourceId, and replacing existing resources with updated SyntaxTrees. The Add method handles both initial additions and updates. ```csharp using TinyAst.Preprocessor.Bridge.Resources; using TinyPreprocessor.Core; using TinyTokenizer.Ast; var store = new InMemorySyntaxTreeResourceStore(); // Add resources var tree = SyntaxTree.ParseAndBind(source, schema); var resource = new Resource(new ResourceId("utils/helpers"), tree); store.Add(resource); // Retrieve resources if (store.TryGet(new ResourceId("utils/helpers"), out var retrieved)) { Console.WriteLine($"Found resource: {retrieved.Id.Path}"); Console.WriteLine($"Content length: {retrieved.Content.TextLength}"); } // Replace existing resource var updatedTree = SyntaxTree.ParseAndBind(newSource, schema); store.Add(new Resource(new ResourceId("utils/helpers"), updatedTree)); ``` -------------------------------- ### Resolve Syntax Tree Resources from In-Memory Store (C#) Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt Demonstrates using InMemorySyntaxTreeResourceResolver to find syntax tree resources stored in an InMemorySyntaxTreeResourceStore. It supports path-like resolution and uses an optional ImportDirectiveLocationIndex for diagnostics. The process involves adding resources to the store, creating the resolver, and then calling ResolveAsync with a reference and context. ```csharp using TinyAst.Preprocessor.Bridge.Resources; using TinyAst.Preprocessor.Bridge.Imports; using TinyPreprocessor.Core; using TinyTokenizer.Ast; var store = new InMemorySyntaxTreeResourceStore(); store.Add(CreateResource("lib/math", mathSource, schema)); store.Add(CreateResource("lib/utils", utilsSource, schema)); // Create resolver with optional location index for diagnostics var locationIndex = new ImportDirectiveLocationIndex(); var resolver = new InMemorySyntaxTreeResourceResolver(store, locationIndex); // Resolve a reference var result = await resolver.ResolveAsync( reference: "lib/math", context: mainResource, // Requesting resource ct: CancellationToken.None); if (result.Success) { Console.WriteLine($"Resolved to: {result.Resource!.Id.Path}"); } else { Console.WriteLine($"Resolution failed: {result.Diagnostic}"); } ``` -------------------------------- ### C# SyntaxTreePreprocessor with Custom Import Node Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt Demonstrates using SyntaxTreePreprocessor with a custom import node type (MyImportNode) and a schema-bound import pattern. It sets up an in-memory resource store, parses and binds syntax trees, and processes imports, outputting the merged content or diagnostics. ```csharp using TinyAst.Preprocessor.Bridge; using TinyAst.Preprocessor.Bridge.Resources; using TinyPreprocessor; using TinyPreprocessor.Core; using TinyTokenizer.Ast; // Define your import node type public sealed class MyImportNode : SyntaxNode { public MyImportNode(CreationContext context) : base(context) { } public string Reference { get { var stringToken = Children .OfType() .FirstOrDefault(t => t.Kind == NodeKind.String); if (stringToken is null) return string.Empty; var text = stringToken.Text; return text.Length >= 2 && text[0] == '"' && text[^1] == '"' ? text[1..^1] : text; } } } // Create schema with import pattern var importPattern = new PatternBuilder() .Ident("import") .String() .BuildQuery(); var schema = Schema.Create() .DefineSyntax("Import", b => b.Match(importPattern)) .Build(); // Set up resources var store = new InMemorySyntaxTreeResourceStore(); var libSource = "let lib = 42"; var mainSource = "import \"lib\"\nlet main = lib"; var libTree = SyntaxTree.ParseAndBind(libSource, schema); store.Add(new Resource(new ResourceId("lib"), libTree)); var mainTree = SyntaxTree.ParseAndBind(mainSource, schema); var mainResource = new Resource(new ResourceId("main"), mainTree); store.Add(mainResource); // Create preprocessor and process var preprocessor = new SyntaxTreePreprocessor( store, getReference: n => n.Reference); var result = await preprocessor.ProcessAsync( mainResource, context: null, PreprocessorOptions.Default); if (result.Success) { Console.WriteLine(result.Content.ToText()); // Output: let lib = 42 // let main = lib } else { foreach (var diagnostic in result.Diagnostics) { Console.WriteLine($"Error: {diagnostic}"); } } ``` -------------------------------- ### Handle Diamond Dependencies with Deduplication in C# Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt Demonstrates how the TinyAST preprocessor resolves diamond dependency patterns by deduplicating imported resources. This ensures that shared dependencies are only processed once, even when imported through multiple paths. ```csharp using TinyAst.Preprocessor.Bridge; using TinyAst.Preprocessor.Bridge.Resources; using TinyPreprocessor; using TinyPreprocessor.Core; using TinyTokenizer.Ast; var store = new InMemorySyntaxTreeResourceStore(); // Diamond: main -> a, b; a -> shared; b -> shared var sharedSource = "let shared = 0"; var aSource = "import \"shared\"\nlet a = shared"; var bSource = "import \"shared\"\nlet b = shared"; var mainSource = "import \"a\"\nimport \"b\"\nlet main = a + b"; store.Add(CreateResource("shared", sharedSource, schema)); store.Add(CreateResource("a", aSource, schema)); store.Add(CreateResource("b", bSource, schema)); var mainResource = CreateResource("main", mainSource, schema); store.Add(mainResource); var preprocessor = new SyntaxTreePreprocessor(store, n => n.Reference); var options = new PreprocessorOptions { DeduplicateIncludes = true }; var result = await preprocessor.ProcessAsync(mainResource, null, options); // shared content appears once, not twice Console.WriteLine(result.Content.ToText()); // let shared = 0 // let a = shared // let b = shared // let main = a + b ``` -------------------------------- ### Configure Preprocessor Behavior with Options (C#) Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt Details how to configure the TinyPreprocessor with PreprocessorOptions, controlling aspects like maximum include depth, deduplication of includes, and error handling behavior. Custom options can be set to override defaults, impacting how the preprocessor processes files, including its response to errors or deeply nested imports. ```csharp using TinyPreprocessor; using TinyPreprocessor.Core; // Default options var defaultOptions = PreprocessorOptions.Default; // Custom options var options = new PreprocessorOptions { MaxIncludeDepth = 10, // Limit nesting depth (prevents stack overflow) DeduplicateIncludes = true, // Handle diamond dependencies ContinueOnError = true // Collect all errors vs fail-fast }; var result = await preprocessor.ProcessAsync( mainResource, context: null, options); // Check for depth exceeded errors if (result.Diagnostics.OfType().Any()) { Console.WriteLine("Import chain too deep!"); } ``` -------------------------------- ### Register Import Node in Schema using C# Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/README.md Demonstrates how to register the custom 'MyImportNode' with a TinyAst schema. It defines a pattern for 'import' followed by a string and associates this pattern with the 'MyImportNode' type. ```csharp var importPattern = new PatternBuilder() .Ident("import") .String() .BuildQuery(); var schema = Schema.Create() .DefineSyntax("Import", b => b.Match(importPattern)) .Build(); ``` -------------------------------- ### C# SyntaxTreePreprocessor with Custom Context Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt Illustrates using the generic SyntaxTreePreprocessor with a custom context type (PreprocessContext) for passing state through the preprocessing pipeline. This is useful for propagating configuration or metadata to custom merge strategies. ```csharp using TinyAst.Preprocessor.Bridge; using TinyPreprocessor.Core; using TinyTokenizer.Ast; // Custom context for tracking processing metadata public class PreprocessContext { public List ProcessedResources { get; } = new(); public Dictionary Metadata { get; } = new(); } // Create preprocessor with custom context type var preprocessor = new SyntaxTreePreprocessor( resolver, getReference: n => n.Reference); var context = new PreprocessContext(); var result = await preprocessor.ProcessAsync( mainResource, context, new PreprocessorOptions { MaxIncludeDepth = 10 }); // Access context data after processing Console.WriteLine($"Processed {context.ProcessedResources.Count} resources"); ``` -------------------------------- ### IContentBoundaryResolverProvider Interface Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md Provides access to specific IContentBoundaryResolver implementations for given content and boundary types. ```csharp interface IContentBoundaryResolverProvider { bool TryGet(out IContentBoundaryResolver resolver); } ``` -------------------------------- ### IMergeStrategy Interface Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md Defines the strategy for merging multiple resolved resources into a single content output. It takes a list of ordered resources and user context. ```csharp interface IMergeStrategy { TContent Merge(IReadOnlyList> orderedResources, TContext userContext, MergeContext mergeContext); } ``` -------------------------------- ### Merge Resolved Syntax Trees with Source Maps (C#) Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt Shows how SyntaxTreeMergeStrategy is used to combine multiple resolved syntax trees, replacing import nodes with their content and constructing source maps. This strategy is typically invoked by the preprocessor but can be used directly for custom merging tasks. It requires a list of ordered resolved resources and a merge context. ```csharp using TinyAst.Preprocessor.Bridge.Merging; using TinyPreprocessor.Core; using TinyPreprocessor.Merging; using TinyTokenizer.Ast; var mergeStrategy = new SyntaxTreeMergeStrategy( getReference: n => n.Reference); // Typically called by the preprocessor, but can be used directly // for custom merge scenarios: // Build ordered resources (dependencies first, root last) var orderedResources = new List> { new(helperResourceId, helperTree, helperDirectives), new(libResourceId, libTree, libDirectives), new(mainResourceId, mainTree, mainDirectives) }; var mergeContext = new MergeContext( resolvedReferences, resolvedCache, sourceMapBuilder, diagnostics); SyntaxTree merged = mergeStrategy.Merge( orderedResources, userContext: null!, mergeContext); Console.WriteLine(merged.ToText()); ``` -------------------------------- ### Create and Use SyntaxTreeBridge in C# Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt The SyntaxTreeBridge provides a wrapper for accessing directive parser and merge strategy components, allowing fine-grained control over the preprocessing pipeline. It requires a type for import nodes and context, and a delegate to extract references from nodes. The bridge exposes parser and merge strategy instances that can be used to configure a custom Preprocessor. ```csharp using TinyAst.Preprocessor.Bridge; using TinyAst.Preprocessor.Bridge.Content; using TinyAst.Preprocessor.Bridge.Imports; using TinyPreprocessor; using TinyPreprocessor.Core; using TinyTokenizer.Ast; // Create bridge with reference extractor var bridge = new SyntaxTreeBridge( getReference: n => n.Reference); // Access individual components ImportDirectiveParser parser = bridge.Parser; SyntaxTreeMergeStrategy mergeStrategy = bridge.MergeStrategy; // Use components in custom configuration var config = new PreprocessorConfiguration( parser, ImportDirectiveModel.Instance, customResolver, mergeStrategy, SyntaxTreeContentModel.Instance, SyntaxTreeContentBoundaryResolverProvider.Instance); var preprocessor = new Preprocessor(config); ``` -------------------------------- ### Preprocessor Class and ProcessAsync Method Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md The main class for performing preprocessing. The ProcessAsync method orchestrates the preprocessing pipeline for a given root resource and context. ```csharp class Preprocessor { Task> ProcessAsync(IResource root, TContext context, PreprocessorOptions? options = null, CancellationToken ct = default); } ``` -------------------------------- ### IContentBoundaryResolver Interface Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md Resolves logical boundaries within content, useful for interpreting locations in more complex scenarios. It returns a collection of offset integers. ```csharp interface IContentBoundaryResolver { IEnumerable ResolveOffsets(TContent content, ResourceId resourceId, int startOffset, int endOffset); } ``` -------------------------------- ### Define Custom Import Node in C# Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/README.md Defines a custom SyntaxNode type 'MyImportNode' to represent import directives. It includes logic to extract the import reference string from its child tokens, stripping surrounding quotes. ```csharp using TinyTokenizer.Ast; public sealed class MyImportNode : SyntaxNode { public MyImportNode(CreationContext context) : base(context) { } public string Reference { get { // Extract the import path from your node's children // Example: import "path/to/file" var stringToken = Children .OfType() .FirstOrDefault(t => t.Kind == NodeKind.String); if (stringToken is null) return string.Empty; var text = stringToken.Text; // Strip quotes return text.Length >= 2 && text[0] == '"' && text[^1] == '"' ? text[1..^1] : text; } } } ``` -------------------------------- ### Detect Circular Dependencies in C# Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt Illustrates the TinyAST preprocessor's capability to automatically detect and report circular dependencies between code modules. It shows how to process resources and check for diagnostics, specifically identifying `CircularDependencyDiagnostic`. ```csharp using TinyAst.Preprocessor.Bridge; using TinyAst.Preprocessor.Bridge.Resources; using TinyPreprocessor; using TinyPreprocessor.Core; using TinyPreprocessor.Diagnostics; var store = new InMemorySyntaxTreeResourceStore(); // Circular: a -> b -> a var aSource = "import \"b\"\nlet a = 1"; var bSource = "import \"a\"\nlet b = 2"; store.Add(CreateResource("a", aSource, schema)); store.Add(CreateResource("b", bSource, schema)); var aResource = CreateResource("a", aSource, schema); var preprocessor = new SyntaxTreePreprocessor(store, n => n.Reference); var result = await preprocessor.ProcessAsync(aResource, null, PreprocessorOptions.Default); if (!result.Success) { var cycleDiagnostic = result.Diagnostics .OfType() .FirstOrDefault(); if (cycleDiagnostic != null) { Console.WriteLine($"Circular dependency detected: {cycleDiagnostic}"); } } ``` -------------------------------- ### IDirectiveModel Interface Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md Provides methods to interact with parsed directives, including retrieving their location and checking for references. ```csharp interface IDirectiveModel { Range GetLocation(TDirective directive); bool TryGetReference(TDirective directive, out string reference); } ``` -------------------------------- ### IResource Interface Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md Represents a resource with content, an ID, and associated metadata. Used to abstract different types of content that TinyPreprocessor can process. ```csharp interface IResource { ResourceId Id { get; } TContent Content { get; } IReadOnlyDictionary Metadata { get; } } ``` -------------------------------- ### IResourceResolver Interface Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md Handles the resolution of resource references asynchronously. It takes a reference string and optional context to return a resolved resource. ```csharp interface IResourceResolver { ValueTask> ResolveAsync(string reference, IResource? context, CancellationToken ct); } ``` -------------------------------- ### Parse ImportDirectives with ImportDirectiveParser in C# Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt The ImportDirectiveParser class is responsible for parsing ImportDirective instances from schema-bound syntax trees. It requires a delegate to extract the reference from TImportNode. The Parse method takes the syntax tree and a resource ID, returning an enumerable of ImportDirective objects. It throws an InvalidOperationException if the input SyntaxTree is not schema-bound. ```csharp using TinyAst.Preprocessor.Bridge.Imports; using TinyPreprocessor.Core; using TinyTokenizer.Ast; var parser = new ImportDirectiveParser( getReference: node => node.Reference); // Parse directives from a schema-bound tree var tree = SyntaxTree.ParseAndBind(source, schema); var resourceId = new ResourceId("main"); IEnumerable directives = parser.Parse(tree, resourceId); foreach (var directive in directives) { Console.WriteLine($"Found import: {directive.Reference} at position {directive.Location.Start}"); } // Note: Throws InvalidOperationException if tree.HasSchema == false try { var unboundTree = SyntaxTree.Parse(source); parser.Parse(unboundTree, resourceId); // Throws! } catch (InvalidOperationException ex) { Console.WriteLine($"Error: {ex.Message}"); // "Directive parsing requires a schema-bound SyntaxTree (HasSchema == true)." } ``` -------------------------------- ### PreprocessResult Structure Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md Holds the result of a preprocessing operation, including the processed content, source map, and any diagnostics generated. ```csharp struct PreprocessResult { TContent Content { get; } SourceMap SourceMap { get; } DiagnosticCollection Diagnostics { get; } } ``` -------------------------------- ### IDirectiveParser Interface Source: https://github.com/dsisco11/tinyast.preprocessor/blob/master/docs/bridge/03-generic-preprocessor-contracts.md Defines a contract for parsing directives from content. It takes content and a resource ID to produce a collection of directives. ```csharp interface IDirectiveParser { IEnumerable Parse(TContent content, ResourceId resourceId); } ``` -------------------------------- ### Utilize ImportDirectiveModel Singleton in C# Source: https://context7.com/dsisco11/tinyast.preprocessor/llms.txt The ImportDirectiveModel is a singleton class implementing IDirectiveModel, providing location extraction and reference validation for import directives. The GetLocation method returns the directive's range, while TryGetReference attempts to extract a non-empty reference string. Empty or whitespace references are correctly handled as non-dependencies. ```csharp using TinyAst.Preprocessor.Bridge.Imports; var model = ImportDirectiveModel.Instance; var directive = new ImportDirective("module", 42..42, null); // Get directive location Range location = model.GetLocation(directive); Console.WriteLine($"Location: {location}"); // 42..42 // Try to get reference (validates non-empty) if (model.TryGetReference(directive, out string reference)) { Console.WriteLine($"Reference: {reference}"); // "module" } // Empty/whitespace references return false var emptyDirective = new ImportDirective(" ", 0..0, null); if (!model.TryGetReference(emptyDirective, out _)) { Console.WriteLine("Empty reference treated as non-dependency"); } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.