### V2 Pipeline Setup Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Illustrates the V2 approach to setting up a pipeline, including configuration and service registration. ```csharp // Program.cs await PipelineHostBuilder.Create() .ConfigureAppConfiguration((ctx, builder) => { builder.AddJsonFile("appsettings.json"); }) .ConfigureServices((ctx, services) => { services.AddModule() .AddModule() .AddModule(); }) .ExecutePipelineAsync(); // BuildModule.cs public class BuildModule : Module { protected internal override TimeSpan Timeout => TimeSpan.FromMinutes(10); protected override async Task ExecuteAsync( IPipelineContext context, CancellationToken cancellationToken) { var result = await context.DotNet().Build(new DotNetBuildOptions()); return new BuildOutput(result.StandardOutput); } } // DeployModule.cs [DependsOn] [DependsOn] public class DeployModule : Module { protected internal override Task ShouldSkip(IPipelineContext context) { if (context.Git().Information.BranchName != "main") return Task.FromResult(SkipDecision.Skip("Not main branch")); return Task.FromResult(SkipDecision.DoNotSkip); } protected override async Task ExecuteAsync( IPipelineContext context, CancellationToken cancellationToken) { var buildResult = await GetModule(); if (buildResult.ModuleResultType != ModuleResultType.Success) return false; // Deploy using buildResult.Value return true; } } ``` -------------------------------- ### V2 Pipeline Entry Point Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Illustrates the V2 pipeline setup using PipelineHostBuilder.Create(). This pattern is replaced in V3. ```csharp await PipelineHostBuilder.Create() .ConfigureAppConfiguration((context, builder) => { builder.AddJsonFile("appsettings.json") .AddUserSecrets() .AddEnvironmentVariables(); }) .ConfigureServices((context, collection) => { collection.Configure(context.Configuration.GetSection("MySettings")); if (context.HostingEnvironment.IsDevelopment()) { collection.AddModule(); } collection.AddModule(); }) .ConfigurePipelineOptions((context, options) => { options.ExecutionMode = ExecutionMode.StopOnFirstException; }) .AddModule() .AddModule() .ExecutePipelineAsync(); ``` -------------------------------- ### V3 Pipeline Setup Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Illustrates the V3 approach to setting up a pipeline, including configuration and service registration, using the new `Pipeline.CreateBuilder` and `ModuleConfiguration`. ```csharp // Program.cs var builder = Pipeline.CreateBuilder(args); builder.Configuration.AddJsonFile("appsettings.json"); builder.Services .AddModule() .AddModule() .AddModule(); await builder.Build().RunAsync(); // BuildModule.cs public class BuildModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithTimeout(TimeSpan.FromMinutes(10)) .Build(); protected override async Task ExecuteAsync( IModuleContext context, CancellationToken cancellationToken) { var result = await context.DotNet().Build(new DotNetBuildOptions()); return new BuildOutput(result.StandardOutput); } } // DeployModule.cs [DependsOn] [DependsOn] public class DeployModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithSkipWhen(ctx => ctx.Git().Information.BranchName != "main" ? SkipDecision.Skip("Not main branch") : SkipDecision.DoNotSkip) .Build(); protected override async Task ExecuteAsync( IModuleContext context, CancellationToken cancellationToken) { var buildResult = await context.GetModule(); if (buildResult is not ModuleResult.Success { Value: var output }) return false; // Deploy using output return true; } } ``` -------------------------------- ### Create and Run a Basic Pipeline Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/pipeline-host Use Pipeline.CreateBuilder() to start building your pipeline. Add modules and then build and run the pipeline asynchronously. ```csharp var builder = Pipeline.CreateBuilder(args); builder.Services .AddModule() .AddModule() .AddModule(); await builder.Build().RunAsync(); ``` -------------------------------- ### Implement Windows Requirement Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/requirements Implement `IPipelineRequirement` to define a startup check. This example checks if the operating system is Windows. ```csharp public class WindowsRequirement : IPipelineRequirement { public Task MustAsync(IPipelineHookContext context) { return (context.Environment.OperatingSystem == OSPlatform.Windows).AsTask(); } } ``` -------------------------------- ### Mock File System for Reading Configuration Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/testing This example demonstrates how to mock IFileSystemProvider to simulate reading a configuration file. Ensure the mock provider is registered before the pipeline runs. ```csharp using Moq; using ModularPipelines; using ModularPipelines.FileSystem; using ModularPipelines.Extensions; [Test] public async Task MyModule_ReadsConfigFile() { // Create a mock provider var mockProvider = new Mock(); mockProvider.Setup(p => p.ReadAllTextAsync( It.IsAny(), It.IsAny())) .ReturnsAsync("{\"setting\": \"value\"}"); // Run pipeline with mock var builder = Pipeline.CreateBuilder(args); builder.Services.AddSingleton(mockProvider.Object); builder.Services.AddModule(); var result = await builder.Build().RunAsync(); // Assert results Assert.That(result.Status, Is.EqualTo(PipelineStatus.Success)); } ``` -------------------------------- ### Utilize Preset Logging Options Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 CommandLoggingOptions provides preset configurations such as Silent, Diagnostic, and Default for convenient setup. ```csharp CommandLoggingOptions.Silent / .Diagnostic / .Default presets ``` -------------------------------- ### Command Logging Configuration (V2) Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Example of V2 command logging configuration, where logging settings were part of tool options and offered limited control. ```csharp // Logging settings were on tool options with limited control await context.DotNet().Build(new DotNetBuildOptions { LogInput = true, LogOutput = false, // Limited options available }); ``` -------------------------------- ### Single File C# Pipeline Example Source: https://thomhurst.github.io/ModularPipelines/docs/examples/single-file-csharp A complete single-file C# script using ModularPipelines to update .NET workloads and check the .NET SDK. Requires the ModularPipelines.DotNet package. ```csharp #!/usr/bin/dotnet run #:package ModularPipelines.DotNet@3.* using ModularPipelines; using ModularPipelines.Attributes; using ModularPipelines.Context; using ModularPipelines.DotNet.Extensions; using ModularPipelines.Extensions; using ModularPipelines.Models; using ModularPipelines.Modules; var builder = Pipeline.CreateBuilder(args); builder.Services .AddModule() .AddModule(); await builder.Build().RunAsync(); public class UpdateDotnetWorkloads : Module { protected override async Task ExecuteAsync(IModuleContext context, CancellationToken cancellationToken) { return await context.DotNet().DotNetWorkload.Update(token: cancellationToken); } } [DependsOn] public class CheckDotnetSdkModule : Module { protected override async Task ExecuteAsync(IModuleContext context, CancellationToken cancellationToken) { return await context.DotNet().Sdk.Check(token: cancellationToken); } } ``` -------------------------------- ### Mock File System for Verifying File Writes Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/testing This example shows how to mock IFileSystemProvider to verify that a file write operation occurred with specific content. Remember to mock all methods your module uses. ```csharp [Test] public async Task MyModule_WritesOutputFile() { var mockProvider = new Mock(); var builder = Pipeline.CreateBuilder(args); builder.Services.AddSingleton(mockProvider.Object); builder.Services.AddModule(); await builder.Build().RunAsync(); // Verify the write occurred with expected content mockProvider.Verify(p => p.WriteAllTextAsync( It.Is(path => path.Contains("output")), It.Is(content => content.Contains("result")), It.IsAny())); } ``` -------------------------------- ### Mocking Path Operations Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/testing If your code uses path operations, you must also mock them. This example shows how to mock the Combine method. ```csharp mockProvider.Setup(p => p.Combine(It.IsAny())) .Returns((string[] paths) => Path.Combine(paths)); ``` -------------------------------- ### Custom Time Provider Implementation Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/time-estimator Implement IModuleEstimatedTimeProvider to manage module time estimations. This example saves and retrieves times from a local directory. It handles both main module times and sub-module times. ```csharp public class MyEstimatedTimeProvider : IModuleEstimatedTimeProvider { private readonly string _directory = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "ModularPipelines", "EstimatedTimes"); public async Task GetModuleEstimatedTimeAsync(Type moduleType) { var fileName = $"{moduleType.FullName}.txt"; return await GetEstimatedTimeAsync(fileName); } public async Task SaveModuleTimeAsync(Type moduleType, TimeSpan duration) { var fileName = $"{moduleType.FullName}.txt"; await SaveModuleTimeAsync(duration, fileName); } public async Task> GetSubModuleEstimatedTimesAsync(Type moduleType) { var directoryInfo = new DirectoryInfo(_directory); if (!directoryInfo.Exists) { directoryInfo.Create(); } directoryInfo.Create(); var paths = directoryInfo .EnumerateFiles("*.txt", SearchOption.TopDirectoryOnly) .Where(x => x.Name.StartsWith($"Mod-{moduleType.FullName}")) .ToList(); var subModuleEstimations = await paths.ToAsyncProcessorBuilder() .SelectAsync(async file => { try { var name = Path.GetFileNameWithoutExtension(file.FullName).Split("-Sub-")[1]; var time = await GetEstimatedTimeAsync(file.FullName); return new SubModuleEstimation(name, time); } catch { File.Delete(file.FullName); return null; } }) .ProcessInParallel(); return subModuleEstimations.OfType(); } public async Task SaveSubModuleTimeAsync(Type moduleType, SubModuleEstimation subModuleEstimation) { var fileName = $"Mod-{moduleType.FullName}-Sub-{subModuleEstimation.SubModuleName}.txt"; await SaveModuleTimeAsync(subModuleEstimation.EstimatedDuration, fileName); } private async Task GetEstimatedTimeAsync(string fileName) { var path = Path.Combine(_directory, fileName); if (File.Exists(path)) { var contents = await File.ReadAllTextAsync(path); return TimeSpan.Parse(contents); } // Some default fallback. We can't estimate for now so we'll estimate next time. return TimeSpan.FromMinutes(2); } private async Task SaveModuleTimeAsync(TimeSpan duration, string fileName) { Directory.CreateDirectory(_directory); var path = Path.Combine(_directory, fileName); await File.WriteAllTextAsync(path, duration.ToString()); } } ``` -------------------------------- ### Run Single File C# Script Source: https://thomhurst.github.io/ModularPipelines/docs/examples/single-file-csharp Command to execute a single-file C# script using the dotnet CLI. Ensure the .NET SDK is installed and in your PATH. ```bash dotnet run example.cs ``` -------------------------------- ### Combine Always Run with Other Behaviors Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/always-run Combine WithAlwaysRun() with other behaviors like WithIgnoreFailures() and WithTimeout() for comprehensive control. This example ensures cleanup runs even if dependencies fail and that the pipeline doesn't fail if the cleanup itself encounters an error. ```csharp [DependsOn] [DependsOn] public class CleanupModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithAlwaysRun() .WithIgnoreFailures() // Don't fail the pipeline if cleanup fails .WithTimeout(TimeSpan.FromMinutes(5)) .Build(); protected override async Task ExecuteAsync(IModuleContext context, CancellationToken cancellationToken) { // Clean up resources - runs even if Build or Test failed // Won't fail the pipeline even if cleanup itself fails } } ``` -------------------------------- ### Implement Module Hooks Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/hooks Define a class that implements IPipelineModuleHooks to log information when a module starts and finishes. The OnBeforeModuleStartAsync method logs the module name, and OnBeforeModuleEndAsync logs the module name and its duration. ```csharp public class MyModuleHooks : IPipelineModuleHooks { public Task OnBeforeModuleStartAsync(IPipelineHookContext context, IModule module) { context.Logger.LogInformation("{Module} is starting", module.GetType().Name); return Task.CompletedTask; } public Task OnBeforeModuleEndAsync(IPipelineHookContext context, IModule module) { context.Logger.LogInformation("{Module} finished after {Elapsed}", module.GetType().Name, module.Duration); return Task.CompletedTask; } } ``` -------------------------------- ### Combine Skip Condition with Other Behaviors Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/skipping Combine module skipping with other behaviors like 'AlwaysRun' and 'Timeout'. This example skips the module if not on the 'main' branch, but will always run if not skipped, and has a 5-minute timeout. ```csharp public class CleanupModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithSkipWhen(ctx => ctx.Git().Information.BranchName != "main") .WithAlwaysRun() // Run even if dependencies fail (when not skipped) .WithTimeout(TimeSpan.FromMinutes(5)) .Build(); } ``` -------------------------------- ### Set Global Default Command Logging Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/logging Configure the default logging behavior for all commands in a pipeline. This example sets all commands to use Silent logging, meaning no command output will be logged unless explicitly overridden. ```csharp var builder = Pipeline.CreateBuilder(args); // All commands will use Silent logging unless overridden builder.Options.DefaultLoggingOptions = CommandLoggingOptions.Silent; // Or use Diagnostic for debugging builder.Options.DefaultLoggingOptions = CommandLoggingOptions.Diagnostic; await builder.Build().RunAsync(); ``` -------------------------------- ### Implement Global Hooks Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/hooks Define a class that implements IPipelineGlobalHooks to log messages when the pipeline starts and ends. The OnStartAsync method logs that the pipeline is starting, and OnEndAsync logs that the pipeline is ending. ```csharp public class MyGlobalHooks : IPipelineGlobalHooks { public Task OnStartAsync(IPipelineHookContext pipelineContext) { pipelineContext.Logger.LogInformation("Pipeline is starting"); return Task.CompletedTask; } public Task OnEndAsync(IPipelineHookContext pipelineContext, PipelineSummary pipelineSummary) { pipelineContext.Logger.LogInformation("Pipeline is ending"); return Task.CompletedTask; } } ``` -------------------------------- ### Register Module Hooks Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/hooks Register a custom class that implements IPipelineModuleHooks to execute actions before and after each module starts and ends. ```csharp collection.AddPipelineModuleHooks(); ``` -------------------------------- ### Configuration, Services, and Options: V2 vs V3 Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 V3 provides direct access to `builder.Configuration`, `builder.Services`, and `builder.Options` for configuration, service collection, and pipeline options, respectively. This replaces V2's `.ConfigureAppConfiguration(...)`, `.ConfigureServices(...)`, and `.ConfigurePipelineOptions(...)` methods. ```csharp builder.Configuration.Add...() ``` ```csharp .ConfigureAppConfiguration((context, builder) => { ... }) ``` ```csharp builder.Services.Add...() ``` ```csharp .ConfigureServices((context, collection) => { ... }) ``` ```csharp builder.Options.PropertyName = value ``` ```csharp .ConfigurePipelineOptions((context, options) => { ... }) ``` -------------------------------- ### Complete Pipeline Configuration and Execution Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/pipeline-host Use this snippet to set up a comprehensive pipeline, including configuration sources, execution options, service registrations, conditional module loading, and final execution. ```csharp using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using ModularPipelines; using ModularPipelines.Extensions; using ModularPipelines.Options; var builder = Pipeline.CreateBuilder(args); // Configuration builder.Configuration .AddJsonFile("appsettings.json") .AddUserSecrets() .AddEnvironmentVariables(); // Options builder.Options.ExecutionMode = ExecutionMode.StopOnFirstException; builder.Options.IgnoreCategories = ["Experimental"]; // Services builder.Services.Configure(builder.Configuration.GetSection("NuGet")); builder.Services.Configure(builder.Configuration.GetSection("Publish")); // Conditional modules if (builder.Environment.IsDevelopment()) { builder.Services .AddModule() .AddModule(); } else { builder.Services .AddModule() .AddModule() .AddModule(); } // Always-registered modules builder.Services .AddModule() .AddModule(); // Run await builder.Build().RunAsync(); ``` -------------------------------- ### Implementing a Plugin System Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Shows how to create reusable pipeline extensions by implementing the `IModularPipelinesPlugin` interface and registering them via attributes. ```csharp public class MyPlugin : IModularPipelinesPlugin { public string Name => "MyPlugin"; public int Priority => 0; // Lower runs first public void ConfigureServices(IServiceCollection services) { services.AddSingleton(); } public void ConfigurePipeline(PipelineBuilder builder) { builder.Services.AddModule(); builder.Options.PrintLogo = false; } } // Register plugin via attribute on assembly [assembly: ModularPipelinesPlugin(typeof(MyPlugin))] ``` -------------------------------- ### Build and Run Pipeline Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/pipeline-host The pipeline creation process involves two steps: building the pipeline configuration and then running it asynchronously. Check the summary status for success or failure. ```csharp var builder = Pipeline.CreateBuilder(args); builder.Services.AddModule(); // Step 1: Build the pipeline var pipeline = builder.Build(); // Step 2: Run it var summary = await pipeline.RunAsync(); // Check results if (summary.Status == PipelineStatus.Failed) { Environment.Exit(1); } ``` -------------------------------- ### Pipeline Creation: V2 vs V3 Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Use `Pipeline.CreateBuilder(args)` in V3 instead of `PipelineHostBuilder.Create()` in V2. V3's method accepts command-line arguments. ```csharp Pipeline.CreateBuilder(args) ``` ```csharp PipelineHostBuilder.Create() ``` -------------------------------- ### Skip Module Using Pipeline Context Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/skipping Conditionally skip a module based on pipeline context, such as the current Git branch name. This example skips if the branch is not 'main'. ```csharp public class MyModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithSkipWhen(ctx => ctx.Git().Information.BranchName != "main") .Build(); protected override async Task ExecuteAsync(IModuleContext context, CancellationToken cancellationToken) { // This only runs on the main branch } } ``` -------------------------------- ### V3 Module Execution with Pattern Matching Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Demonstrates how to execute a module and access its result using pattern matching in V3. This is the recommended approach for handling different result types. ```csharp [DependsOn] public class DeployModule : Module { protected override async Task ExecuteAsync( IModuleContext context, CancellationToken cancellationToken) { // Method moved to context var buildResult = await context.GetModule(); // Option 1: Pattern matching (recommended) return buildResult switch { ModuleResult.Success { Value: var output } => await Deploy(output.ArtifactPath), ModuleResult.Skipped { Decision: var skip } => null, ModuleResult.Failure { Exception: var ex } => throw new InvalidOperationException("Build failed", ex), _ => throw new InvalidOperationException("Unexpected result type") }; } } ``` -------------------------------- ### Pipeline Execution: V2 vs V3 Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 V3 introduces a two-step execution process with `.Build().RunAsync()`, replacing V2's single `.ExecutePipelineAsync()`. Extension methods may offer alternative single-step execution. ```csharp .Build().RunAsync() ``` ```csharp .ExecutePipelineAsync() ``` -------------------------------- ### Create Custom Mandatory Run Condition Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/run-conditions Inherit from MandatoryRunConditionAttribute to create custom conditions that must all return true for the module to run. This example shows two such conditions. ```csharp public class MyMandatoryCustomRunConditionAttribute : MandatoryRunConditionAttribute { public override async Task Condition(IPipelineHookContext pipelineContext) { var response = await pipelineContext.Http.SendAsync("https://www.example.com/service1/mustbeup/ping"); return response.StatusCode == HttpStatusCode.OK; } } public class MyMandatoryCustomRunCondition2Attribute : MandatoryRunConditionAttribute { public override async Task Condition(IPipelineHookContext pipelineContext) { var response = await pipelineContext.Http.SendAsync("https://www.example.com/service2/mustbeup/ping"); return response.StatusCode == HttpStatusCode.OK; } } ``` -------------------------------- ### Create Pipeline Builder and Add Modules Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/storing-and-retrieving-results Initializes a pipeline builder, adds modules, and configures a results repository. ```csharp var builder = Pipeline.CreateBuilder(args); builder.Services .AddModule() .AddModule() .AddModule(); builder.AddResultsRepository(); await builder.Build().RunAsync(); ``` -------------------------------- ### V2 Module Configuration (Before V3) Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Illustrates the V2 method of configuring module behavior using virtual property and method overrides. ```csharp public class MyModule : Module { // Timeout override protected internal override TimeSpan Timeout => TimeSpan.FromMinutes(5); // Retry policy override protected override AsyncRetryPolicy RetryPolicy => Policy.Handle() .WaitAndRetryAsync(3, i => TimeSpan.FromSeconds(i * i)); // Skip logic override protected internal override Task ShouldSkip(IPipelineContext context) { if (context.Git().Information.BranchName != "main") return Task.FromResult(SkipDecision.Skip("Only runs on main branch")); return Task.FromResult(SkipDecision.DoNotSkip); } // Ignore failures override protected internal override Task ShouldIgnoreFailures( IPipelineContext context, Exception exception) => Task.FromResult(true); // Always run override public override ModuleRunType ModuleRunType => ModuleRunType.AlwaysRun; // Lifecycle hooks protected internal override Task OnBeforeExecute(IPipelineContext context) { // Pre-execution logic return Task.CompletedTask; } protected internal override Task OnAfterExecute(IPipelineContext context) { // Post-execution logic return Task.CompletedTask; } protected override async Task ExecuteAsync( IPipelineContext context, CancellationToken cancellationToken) { // Module logic return "result"; } } ``` -------------------------------- ### V3 Pipeline Builder Configuration Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Demonstrates the V3 approach to configuring application settings, user secrets, and environment variables using direct property access on the builder. ```csharp var builder = Pipeline.CreateBuilder(args); // Direct property access instead of callbacks builder.Configuration .AddJsonFile("appsettings.json") .AddUserSecrets() .AddEnvironmentVariables(); // Configure services directly builder.Services.Configure(builder.Configuration.GetSection("MySettings")); if (builder.Environment.IsDevelopment()) { builder.Services.AddModule(); } builder.Services .AddModule() .AddModule() .AddModule(); // Configure options directly builder.Options.ExecutionMode = ExecutionMode.StopOnFirstException; // Two-step build and run await builder.Build().RunAsync(); ``` -------------------------------- ### Disable Parallel Execution with NotInParallel Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/parallelization Use the `[NotInParallel]` attribute to ensure a module runs before any other modules that can be executed in parallel. This is useful for operations like installing applications that cannot run concurrently. ```csharp [NotInParallel] public class MyModule : Module { protected override Task?> ExecuteAsync(IModuleContext context, CancellationToken cancellationToken) { // Do something } } ``` -------------------------------- ### Create and Run a Pipeline Source: https://thomhurst.github.io/ModularPipelines/docs/fundamentals Use Pipeline.CreateBuilder() to initialize a pipeline, add services, and then build and run it. This pattern is similar to ASP.NET Core minimal APIs. ```csharp var builder = Pipeline.CreateBuilder(args); builder.Services.AddModule(); await builder.Build().RunAsync(); ``` -------------------------------- ### Tagging Modules with Attributes and Properties Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Demonstrates how to tag modules for organization and dependency management using attributes or property overrides. ```csharp // Via attributes [ModuleTag("critical")] [ModuleTag("deployment")] [ModuleCategory("Infrastructure")] public class DeployModule : Module { } // Via property override public class MyModule : Module { public override IReadOnlySet Tags => new HashSet { "critical", "fast" }; public override string? Category => "Build"; } ``` -------------------------------- ### V3 Module Configuration Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Demonstrates the V3 fluent `Configure()` builder for setting module timeouts, retry policies, skip conditions, and lifecycle hooks. ```csharp public class MyModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithTimeout(TimeSpan.FromMinutes(5)) .WithRetryCount(3) .WithSkipWhen(ctx => ctx.Git().Information.BranchName != "main" ? SkipDecision.Skip("Only runs on main branch") : SkipDecision.DoNotSkip) .WithIgnoreFailures() .WithAlwaysRun() .WithBeforeExecute(ctx => LogStartAsync(ctx)) .WithAfterExecute(ctx => LogEndAsync(ctx)) .Build(); protected override async Task ExecuteAsync( IModuleContext context, CancellationToken cancellationToken) { // Module logic return "result"; } } ``` -------------------------------- ### Skip Module with Async Condition Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/skipping Implement asynchronous skip conditions, for example, by making an HTTP request to determine if the module should run. The module skips if the HTTP response status code is not successful. ```csharp public class MyModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithSkipWhen(async () => { var response = await HttpClient.GetAsync("https://api.example.com/should-run"); return !response.IsSuccessStatusCode; }) .Build(); } ``` -------------------------------- ### Create and Configure Pipeline with Time Estimator Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/time-estimator This snippet demonstrates how to create a pipeline builder, add modules, register a custom time provider, and run the pipeline. Ensure your custom time provider implements IModuleEstimatedTimeProvider. ```csharp var builder = Pipeline.CreateBuilder(args); builder.Services .AddModule() .AddModule() .AddModule(); builder.AddModuleEstimatedTimeProvider(); await builder.Build().RunAsync(); ``` -------------------------------- ### Manipulate Command Log Output Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/logging Transform logged command output using manipulators. This example demonstrates truncating long input strings and redacting sensitive data like API keys from the output. ```csharp new CommandExecutionOptions { LogSettings = new CommandLoggingOptions { Verbosity = CommandLogVerbosity.Normal }, InputLoggingManipulator = input => input.Length > 500 ? input.Substring(0, 500) + "... (truncated)" : input, OutputLoggingManipulator = output => output.Replace("api-key-value", "***") } ``` -------------------------------- ### V3 Sub-Module Execution Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Demonstrates the V3 approach where sub-module execution is handled via the `context.SubModule` method. ```csharp public class PackageModule : Module { protected override async Task ExecuteAsync( IModuleContext context, CancellationToken cancellationToken) { var packages = new[] { "Package1", "Package2", "Package3" }; foreach (var package in packages) { // Method moved to context await context.SubModule(package, async () => { await context.DotNet().Pack(new DotNetPackOptions { Project = package }); }); } return new PackageResult(packages.Length); } } ``` -------------------------------- ### Use Command Logging Presets Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/logging Apply predefined logging configurations for commands. This shows how to use the Silent preset for no command logging and the Diagnostic preset for maximum verbosity. ```csharp // Silent - no command logging new CommandExecutionOptions { LogSettings = CommandLoggingOptions.Silent } ``` ```csharp // Diagnostic - maximum verbosity new CommandExecutionOptions { LogSettings = CommandLoggingOptions.Diagnostic } ``` ```csharp // Default - normal verbosity new CommandExecutionOptions { LogSettings = CommandLoggingOptions.Default } ``` -------------------------------- ### Convert Single File to Project Source: https://thomhurst.github.io/ModularPipelines/docs/examples/single-file-csharp Command to convert a single-file C# application into a traditional C# project structure. ```bash dotnet project convert example.cs ``` -------------------------------- ### Fluent API for Pipeline Configuration Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/pipeline-host Utilize extension methods for a more concise and fluent way to configure services, pipeline options, and execute the pipeline. ```csharp var builder = Pipeline.CreateBuilder(args); await builder .ConfigureServices(services => { services.AddModule(); services.AddModule(); }) .ConfigurePipelineOptions(options => { options.ExecutionMode = ExecutionMode.StopOnFirstException; }) .ExecutePipelineAsync(); ``` -------------------------------- ### V3 Sub-Module Execution with Return Value Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Shows how to execute a sub-module in V3 and capture its return value. ```csharp // V2: await SubModule(name, func) // V3: await context.SubModule(name, func) var result = await context.SubModule("ProcessItem", async () => { // Process and return value return "processed"; }); ``` -------------------------------- ### Add Configuration Sources to Pipeline Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/pipeline-host Configure the pipeline host by adding various configuration sources like JSON files, user secrets, and environment variables. ```csharp var builder = Pipeline.CreateBuilder(args); builder.Configuration .AddJsonFile("appsettings.json", optional: false) .AddJsonFile($"appsettings.{builder.Environment.EnvironmentName}.json", optional: true) .AddUserSecrets() .AddEnvironmentVariables(); // Use configuration in services builder.Services.Configure(builder.Configuration.GetSection("MySettings")); ``` -------------------------------- ### Simplify Async Configuration Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Separate 'Async' methods for configuration have been removed. The existing methods now accept both synchronous and asynchronous lambdas. ```csharp WithSkipWhenAsync(async () => ...) ``` ```csharp WithSkipWhen(async () => ...) ``` ```csharp WithIgnoreFailuresWhenAsync(...) ``` ```csharp WithIgnoreFailuresWhen(...) ``` -------------------------------- ### Use Property Initializers for Tool Options Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 All tool options constructors have been removed in V3. Use property initializers instead for creating option objects. ```csharp new DotNetNewOptions("console") ``` ```csharp new DotNetNewOptions { TemplateShortName = "console" } ``` ```csharp new DotNetPackOptions(projectPath) ``` ```csharp new DotNetPackOptions { TargetPath = projectPath } ``` ```csharp new GitTagOptions("v1.0.0") ``` ```csharp new GitTagOptions { TagName = "v1.0.0" } ``` -------------------------------- ### V3 Compatibility: ExecutePipelineAsync Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Shows that the `ExecutePipelineAsync()` extension method is still available in V3 for simpler migration scenarios. ```csharp // This still works in V3 await builder.ExecutePipelineAsync(); ``` -------------------------------- ### Module Configuration: Fluent Builder in V3 Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 V3 replaces property overrides for module configuration with a fluent builder pattern using `Configure()`. This includes methods like `WithTimeout`, `WithRetryCount`, `WithSkipWhen`, `WithIgnoreFailures`, `WithAlwaysRun`, `WithBeforeExecute`, and `WithAfterExecute`. ```csharp Configure().WithTimeout(TimeSpan) ``` ```csharp protected internal override TimeSpan Timeout => ... ``` ```csharp Configure().WithRetryCount(int) ``` ```csharp protected override AsyncRetryPolicy RetryPolicy => ... ``` ```csharp Configure().WithSkipWhen(Func) ``` ```csharp protected internal override Task ShouldSkip(...) ``` ```csharp Configure().WithIgnoreFailures() ``` ```csharp protected internal override Task ShouldIgnoreFailures(...) ``` ```csharp Configure().WithAlwaysRun() ``` ```csharp public override ModuleRunType ModuleRunType => ModuleRunType.AlwaysRun ``` ```csharp Configure().WithBeforeExecute(...) or OnBeforeExecuteAsync(...) ``` ```csharp protected internal override Task OnBeforeExecute(IPipelineContext context) ``` ```csharp Configure().WithAfterExecute(...) or OnAfterExecuteAsync(...) ``` ```csharp protected internal override Task OnAfterExecute(IPipelineContext context) ``` -------------------------------- ### Execute Custom CLI Command Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/custom-commands Use this method to run any command-line tool not directly supported by strong objects. Ensure all arguments are passed as individual strings in an array for correct parsing. ```csharp await context.Shell.Command.ExecuteCommandLineTool(new CommandLineToolOptions("dotnet") { Arguments = new[] { "tool", "install", "--global", "dotnet-coverage" }, }, cancellationToken); ``` -------------------------------- ### Alternative Result Access Patterns in V3 Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Illustrates various ways to access module results in V3, including pattern matching, a functional style `Match` helper, safe accessors, and quick status checks. ```csharp var buildResult = await context.GetModule(); // Option 1: Pattern matching (recommended - handles all cases) return buildResult switch { ModuleResult.Success { Value: var output } => Process(output), ModuleResult.Skipped => null, ModuleResult.Failure { Exception: var ex } => throw ex, _ => null }; // Option 2: Match helper method (functional style) return buildResult.Match( onSuccess: output => Process(output), onFailure: ex => throw new InvalidOperationException("Failed", ex), onSkipped: skip => null ); // Option 3: Safe accessor (simplest migration path) var artifact = buildResult.ValueOrDefault?.ArtifactPath; if (artifact == null) return null; return await Deploy(artifact); // Option 4: Quick status checks if (buildResult.IsSuccess) { var value = buildResult.ValueOrDefault; } if (buildResult.IsFailure) { var ex = buildResult.ExceptionOrDefault; } if (buildResult.IsSkipped) { var reason = buildResult.SkipDecisionOrDefault?.Reason; } ``` -------------------------------- ### V2 Command Execution API Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 In V2, command execution was handled directly via `context.Command`. ```csharp await context.Command.ExecuteCommandLineTool(new CommandLineToolOptions("mytool") { Arguments = new[] { "arg1", "arg2" } }); ``` -------------------------------- ### Configure Module to Always Run Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/always-run Use WithAlwaysRun() within the Configure() method to ensure this module executes regardless of dependency status. This is ideal for cleanup operations. ```csharp public class CleanupModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithAlwaysRun() .Build(); protected override async Task ExecuteAsync(IModuleContext context, CancellationToken cancellationToken) { // Clean up resources - runs even if other modules failed } } ``` -------------------------------- ### Registering Pipeline Hooks and Requirements Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/pipeline-host Configure global and module-specific hooks, and define pipeline requirements such as SDKs or version control. ```csharp var builder = Pipeline.CreateBuilder(args); // Global hooks (run before/after all modules) builder.AddPipelineGlobalHooks(); // Module hooks (run before/after each module) builder.AddPipelineModuleHooks(); // Requirements (validated before pipeline starts) builder.AddRequirement(); builder.AddRequirement(); ``` -------------------------------- ### Migrate Command Logging Options Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 The logging system has been updated. LogInput and LogOutput on DotNetBuildOptions are replaced by CommandLoggingOptions with specific settings like ShowCommandArguments and ShowStandardOutput. ```csharp new DotNetBuildOptions { LogInput = true, LogOutput = false } ``` ```csharp new CommandExecutionOptions { LogSettings = new CommandLoggingOptions { ShowCommandArguments = true, ShowStandardOutput = false } } ``` -------------------------------- ### V3 Command Execution Pattern with DotNet Build Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Use this pattern to execute tool-specific commands with separate execution options. It allows for detailed configuration of the command's environment and behavior. ```csharp await context.DotNet().Build( new DotNetBuildOptions { ProjectSolution = "MySolution.sln", Configuration = "Release", }, new CommandExecutionOptions { WorkingDirectory = "/path/to/project", EnvironmentVariables = new Dictionary { ["CI"] = "true" }, ThrowOnNonZeroExitCode = false, ExecutionTimeout = TimeSpan.FromMinutes(10) }); ``` -------------------------------- ### Implement IAlwaysRun and ISkippable Source: https://thomhurst.github.io/ModularPipelines/docs/architecture/module-execution-lifecycle Modules implementing both IAlwaysRun and ISkippable will have their skipping logic evaluated independently. The IAlwaysRun behavior ensures the module is not cancelled by other failures, while ISkippable allows it to be conditionally skipped. ```csharp public class CleanupModule : Module, IAlwaysRun, ISkippable { public Task ShouldSkip(IPipelineContext context) { // Skip logic still applies to AlwaysRun modules if (context.Environment.IsDevelopment()) { return SkipDecision.Skip("No cleanup in dev").AsTask(); } return SkipDecision.DoNotSkip.AsTask(); } } ``` -------------------------------- ### Configure Module Hooks with ModuleConfiguration Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/hooks Use ModuleConfiguration.Create() to define simple before/after execution hooks. This approach is useful for combining hooks with other configurations like timeouts or retry policies. ```csharp public class MyModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithBeforeExecute(ctx => { ctx.Logger.LogInformation("MyModule starting!"); return Task.CompletedTask; }) .WithAfterExecute(ctx => { ctx.Logger.LogInformation("MyModule ended!"); return Task.CompletedTask; }) .Build(); public override async Task ExecuteAsync(IModuleContext context, CancellationToken cancellationToken) { return "Done"; } } ``` ```csharp protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithBeforeExecute(ctx => LogStartAsync(ctx)) .WithAfterExecute(ctx => LogEndAsync(ctx)) .WithTimeout(TimeSpan.FromMinutes(5)) .WithRetryCount(3) .Build(); ``` -------------------------------- ### V3 Async Module with Return Value Template Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Template for creating an asynchronous module in v3 that returns a value. Configure timeouts and other settings using ModuleConfiguration.Create(). ```csharp // Async module WITH return value public class MyModule : Module { protected override ModuleConfiguration Configure() => ModuleConfiguration.Create() .WithTimeout(TimeSpan.FromMinutes(5)) // Add other configuration as needed .Build(); protected override async Task ExecuteAsync( IModuleContext context, CancellationToken cancellationToken) { // Implementation return new MyResult(); } } ``` -------------------------------- ### Pattern Matching Module Results Source: https://thomhurst.github.io/ModularPipelines/docs/fundamentals Use pattern matching to handle `Success`, `Skipped`, and `Failure` outcomes of a module. This is the recommended approach for its clarity and exhaustiveness. ```csharp var myModule = await context.GetModule(); // Pattern matching (recommended) return myModule switch { ModuleResult.Success { Value: var result } => await ProcessResult(result), ModuleResult.Skipped { Decision: var skip } => null, // Module was skipped ModuleResult.Failure { Exception: var ex } => throw new Exception("Dependency failed", ex), _ => null }; ``` -------------------------------- ### Configure Command Logging Verbosity Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Use predefined verbosity levels like Silent, Minimal, Normal, Detailed, or Diagnostic for quick logging configuration. ```csharp CommandLogVerbosity.Silent/Minimal/Normal/Detailed/Diagnostic ``` -------------------------------- ### Async Configuration Methods in V3 Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 The `WithSkipWhen` and `WithIgnoreFailuresWhen` methods in V3 accept both synchronous and asynchronous lambdas using the same method name. No separate `Async` versions are needed. ```csharp // Sync lambda .WithSkipWhen(() => someCondition) // Async lambda - same method name .WithSkipWhen(async () => await CheckConditionAsync()) // With context - sync .WithSkipWhen(ctx => ctx.Git().Information.BranchName != "main") // With context - async .WithSkipWhen(async ctx => await ctx.SomeAsyncCheck()) // Returning SkipDecision .WithSkipWhen(ctx => ctx.Git().Information.BranchName != "main" ? SkipDecision.Skip("Not main branch") : SkipDecision.DoNotSkip) ``` ```csharp // Sync .WithIgnoreFailuresWhen((ctx, ex) => ex is TimeoutException) // Async .WithIgnoreFailuresWhen(async (ctx, ex) => await ShouldIgnoreAsync(ex)) ``` -------------------------------- ### Using the Match Helper for Exhaustive Handling Source: https://thomhurst.github.io/ModularPipelines/docs/fundamentals Employ the `Match` helper method for exhaustive handling of all module result states. This provides a concise way to define actions for each outcome. ```csharp var myModule = await context.GetModule(); return await myModule.Match( onSuccess: result => ProcessResultAsync(result), onFailure: ex => HandleFailureAsync(ex), onSkipped: skip => Task.FromResult(null) ); ``` -------------------------------- ### V3 DotNetNewOptions Property Initializers Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 In V3, options classes no longer accept constructor arguments. Use property initializers instead. ```csharp var newOptions = new DotNetNewOptions { TemplateShortName = "console" }; var packOptions = new DotNetPackOptions { TargetPath = projectPath }; var pushOptions = new DotNetNugetPushOptions { PackagePath = packagePath }; ``` -------------------------------- ### Use Logging Presets in V3 CommandExecutionOptions Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Utilize predefined logging presets like Silent, Diagnostic, or Default for CommandExecutionOptions. These presets simplify the configuration of common logging levels. ```csharp // Silent - no command logging at all new CommandExecutionOptions { LogSettings = CommandLoggingOptions.Silent } // Diagnostic - maximum verbosity for debugging new CommandExecutionOptions { LogSettings = CommandLoggingOptions.Diagnostic } // Default - normal verbosity new CommandExecutionOptions { LogSettings = CommandLoggingOptions.Default } ``` -------------------------------- ### Regex for PipelineHostBuilder Migration Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Use this regex to replace PipelineHostBuilder.Create() with Pipeline.CreateBuilder(args). ```regex # Entry point s/PipelineHostBuilder\.Create\(\)/Pipeline.CreateBuilder(args)/g ``` -------------------------------- ### V3 Result Handling: Match Helper Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 Alternative pattern for handling module results in v3 using the Match helper method. This provides a concise way to define actions for success, failure, and skipped states. ```csharp // Pattern 2: Match helper var result = await context.GetModule(); return result.Match( onSuccess: output => Process(output), onFailure: ex => throw ex, onSkipped: skip => null ); ``` -------------------------------- ### Add ModularPipelines Package Reference Source: https://thomhurst.github.io/ModularPipelines/docs/examples/single-file-csharp Use this directive at the top of your .cs file to include the ModularPipelines package. Specify a version range or a specific version. ```csharp #:package ModularPipelines@3.* ``` ```csharp #:package ModularPipelines@3.0.0 ``` -------------------------------- ### V2 DotNetNewOptions Constructor Source: https://thomhurst.github.io/ModularPipelines/docs/migrating-to-v3 In V2, options classes accepted constructor arguments. This is no longer supported in V3. ```csharp var newOptions = new DotNetNewOptions("console"); var packOptions = new DotNetPackOptions(projectPath); var pushOptions = new DotNetNugetPushOptions(packagePath); ``` -------------------------------- ### Define a Module with a Return Value Source: https://thomhurst.github.io/ModularPipelines/docs/how-to/defining-modules Create a module that returns a `FileInfo` object by inheriting from `Module` and overriding `ExecuteAsync`. ```csharp public class FindAFileModule : Module { protected override async Task ExecuteAsync( IModuleContext context, CancellationToken cancellationToken) { return context.Files .Glob("C:\\**\\MyJsonFile.json") .Single(); } } ```