### Install retrypool Go Library Source: https://github.com/davidroman0o/retrypool/blob/main/readme.md Instructions to install the retrypool Go library using the `go get` command. ```bash go get github.com/davidroman0O/retrypool ``` -------------------------------- ### IndependentDependencyPool Usage Example Source: https://github.com/davidroman0o/retrypool/blob/main/expecting behaviour.md Demonstrates the use case for an IndependentDependencyPool, ideal for sequential data processing pipelines where all tasks and their dependencies are known upfront. ```APIDOC IndependentPool Example: - Used for data processing pipeline where tasks have clear dependencies - Tasks are all submitted upfront - Each task runs independently after its dependencies complete - Execution order: Load Data -> Clean Data -> Transform Data -> Save Results - Perfect for ETL workflows, build systems, or any sequential processing ``` -------------------------------- ### BlockingDependencyPool Usage Example Source: https://github.com/davidroman0o/retrypool/blob/main/expecting behaviour.md Demonstrates the use case for a BlockingDependencyPool, suitable for dynamic workflows where main tasks create and wait for subtasks, such as document processing. ```APIDOC BlockingPool Example: - Used for dynamic workflow where tasks create subtasks - Main task creates and waits for subtasks to complete - Subtasks can run in parallel if no dependencies between them - Shows document processing workflow: * Main task creates subtasks * Text extraction must complete before analysis * Sentiment analysis and keyword extraction can run in parallel - Great for recursive tasks, document processing, or any workflow where tasks need to spawn and wait for subtasks ``` -------------------------------- ### Attempt to Activate Pending Group in GroupPool Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This helper function tries to activate a pending task group when resources become available. It prioritizes groups that have already completed their 'pending' state and then looks for other pending groups with start instructions, allocating a free or newly created pool to them. ```Go func (gp *GroupPool[T, GID]) tryActivatePendingGroup() error { // - find free pool if any exists // - if no free pool and can create new pool (check MaxActivePools): // - create new pool // - add to pools map // - if no pool available, return (groups stay pending) // Important: handle completed pending groups first // - check completedPendingGroups list: // - if any exist, take first // - get its pending group // - remove from pending groups // - activate it with available pool // - return // If no completed pending groups: // - check other pending groups: // - find one that has start instruction // - validate group can be activated // - create active group with: // - pool ID // - group ID // - pending tasks list // - empty fails list // - update pool state: // - assign group // - mark busy // - update mappings: // - remove from pending groups // - add to active groups // - initialize metrics if needed // - trigger OnGroupStarts // - submit first task if any exist // - break after first group activated } ``` -------------------------------- ### Get GroupPool Metrics Snapshot Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This method provides a snapshot of the current metrics for the `GroupPool`. It acquires a read lock to ensure data consistency and returns a copy of the internal metrics snapshot. ```Go // GetSnapshot returns current metrics snapshot func (gp *GroupPool[T, GID]) GetSnapshot() GroupMetricsSnapshot[T, GID] { // - acquire read lock // - return copy of snapshot } ``` -------------------------------- ### Go: Implementing Bounce Retry for Tasks with Unique Worker Credentials Source: https://github.com/davidroman0o/retrypool/blob/main/readme.md This Go example demonstrates how `retrypool`'s 'bounce retry' feature works when workers are associated with unique credentials (simulated by `ID` here). It defines an `APIWorker` that can fail based on its ID. When a task fails, `retrypool` automatically re-submits it, potentially to a different worker, improving the success rate for tasks that might be sensitive to specific worker conditions or resources. The `WithBounceRetry` option is crucial for enabling this behavior. ```Go type APIWorker struct { ID int } func (w *APIWorker) OnStart(ctx context.Context) { fmt.Printf("Worker with id %d started\n", w.ID) } func (w *APIWorker) Run(ctx context.Context, task *retrypool.RequestResponse[string, error]) error { fmt.Printf("Task '%s' started with id %d\n", task.Request, w.ID) if w.ID == 1 { fmt.Printf("\tTask '%s' failed with id %d\n", task.Request, w.ID) return fmt.Errorf("failed with id %d", w.ID) } task.Complete(nil) fmt.Printf("Task '%s' succeeded with id %d\n", task.Request, w.ID) return nil } func main() { ctx := context.Background() workers := []retrypool.Worker[*retrypool.RequestResponse[string, error]]{ &APIWorker{}, &APIWorker{}, } pool := retrypool.New(ctx, workers, retrypool.WithAttempts[*retrypool.RequestResponse[string, error]](5), ) for i := 0; i < 20; i++ { req := retrypool.NewRequestResponse[string, error](fmt.Sprintf("Task #%d", i)) if err := pool.Submit(req, retrypool.WithBounceRetry[*retrypool.RequestResponse[string, error]]()); err != nil { fmt.Printf("Failed to submit task: %v\n", err) } } pool.WaitWithCallback(ctx, func(q, p, d int) bool { return q > 0 && p > 0 && d > 0 }, 500*time.Millisecond) pool.Close() pool.RangeWorkerQueues(func(workerID int, queueSize int64) bool { fmt.Println("WorkerID:", workerID, "QueueSize:", queueSize) return true }) pool.RangeTaskQueues(func(workerID int, queue retrypool.TaskQueue[*retrypool.RequestResponse[string, error]]) bool { fmt.Println("WorkerID:", workerID, "Size:", queue.Length()) return true }) } ``` -------------------------------- ### Basic Usage of retrypool in Go Source: https://github.com/davidroman0o/retrypool/blob/main/readme.md Demonstrates how to define a custom worker, implement the `Worker` interface, initialize the retrypool with basic options like attempts, delay, and rate limit, and submit tasks for processing. It also shows how to wait for task completion and close the pool. ```go package main import ( "context" "fmt" "time" "github.com/davidroman0O/retrypool" ) // Define your worker type MyWorker struct { ID int } // Implement the Worker interface func (w *MyWorker) Run(ctx context.Context, data int) error { fmt.Printf("Worker %d processing: %d\n", w.ID, data) return nil } func main() { ctx := context.Background() // Create workers workers := []retrypool.Worker[int]{ &MyWorker{}, &MyWorker{}, } // Initialize pool with options pool := retrypool.New(ctx, workers, retrypool.WithAttempts[int](3), retrypool.WithDelay[int](time.Second), retrypool.WithRateLimit[int](10.0), // 10 tasks per second ) // Submit tasks for i := 0; i < 5; i++ { if err := pool.Submit(i); err != nil { fmt.Printf("Failed to submit task: %v\n", err) } } // Wait for completion pool.WaitWithCallback(ctx, func(queueSize, processingCount, deadTaskCount int) bool { return queueSize > 0 || processingCount > 0 }, time.Second) pool.Close() } ``` -------------------------------- ### Define and Execute a Dependency Group in Go Source: https://github.com/davidroman0o/retrypool/blob/main/idea.md This Go code snippet demonstrates how to define a `Group` with multiple dependent tasks. It shows the process of adding tasks, specifying input/output mappings, and defining dependencies between tasks. The group is then built with an initial input and submitted for execution using a dependency pooler (dp). ```Go group, err := NewGroup("group1") .Add( "taskA", func(ctx context.Context, input int) (int, error) { return input * 2, nil }, ). Require("input"). Map("int", "output") .Add( "taskB", func(ctx context.Context, input int) (int, error) { return input * 2, nil }, ). DependsOn("taskA"). Match("output").To("input"). Build(42) if err != nil { panic(err) } if err := group.Submit(dp); err != nil { panic(err) } ``` -------------------------------- ### Go GroupPool Constructor Function Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md The `NewGroupPool` function creates and initializes a new `GroupPool` instance. It sets up default configurations, applies provided options, and initializes internal data structures. ```go // NewGroupPool creates a new GroupPool instance with provided options func NewGroupPool[T GroupTask[GID], GID comparable]( ctx context.Context, opts ...GroupPoolOption[T, GID], ) (*GroupPool[T, GID], error) { // - validate context // - create cancel context // - create default config with: // - logger set to debug level but disabled // - MaxActivePools = -1 (unlimited) // - MaxWorkersPerPool = -1 (unlimited) // - UseFreeWorkerOnly = false // - GroupMustSucceed = false // - apply options // - validate worker factory exists // - initialize maps: // - pools // - activeGroups // - pendingGroups // - poolMetrics // - snapshotGroups // - initialize empty completedPendingGroups slice // - set nextPoolID to 0 // - return pool instance } ``` -------------------------------- ### Apply Specific Task Options in retrypool Source: https://github.com/davidroman0o/retrypool/blob/main/readme.md Demonstrates how to submit tasks with specific options such as immediate retry, per-task timeout, and bounce retry (retrying on different workers), providing fine-grained control over task execution behavior. ```go // Submit with immediate retry pool.Submit(data, retrypool.WithImmediateRetry[int]()) // Submit with timeout pool.Submit(data, retrypool.WithTimeout[int](time.Minute)) // Submit with bounce retry (retry on different workers) pool.Submit(data, retrypool.WithBounceRetry[int]()) ``` -------------------------------- ### Manage retrypool Worker Lifecycle Source: https://github.com/davidroman0o/retrypool/blob/main/readme.md Illustrates how to dynamically add, pause, resume, and remove workers from the retrypool after its initialization, allowing for flexible worker management based on changing resource availability. ```go // Add a new worker newWorker := &MyWorker{} pool.Add(newWorker, nil) // Pause a worker pool.Pause(workerID) // Resume a worker pool.Resume(workerID) // Remove a worker pool.Remove(workerID) ``` -------------------------------- ### Configure Custom Retry Policy for retrypool Source: https://github.com/davidroman0o/retrypool/blob/main/readme.md Shows how to configure a custom retry policy, specifically an exponential backoff policy, when initializing the retrypool to manage task retries with increasing delays. ```go pool := retrypool.New(ctx, workers, retrypool.WithRetryPolicy[int](&retrypool.ExponentialBackoffRetryPolicy[int]{ BaseDelay: time.Second, MaxDelay: time.Minute, MaxJitter: time.Second, }), ) ``` -------------------------------- ### Go GroupPool Main Structure Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md Defines the main `GroupPool` struct, which orchestrates task grouping and execution. It contains the pool's configuration, maps for active and pending groups, and metrics tracking. ```go type GroupPool[T GroupTask[GID], GID comparable] struct { mu deadlock.RWMutex ctx context.Context cancel context.CancelFunc config GroupPoolConfig[T, GID] pools map[uint]*poolGroupPoolItem[T, GID] activeGroups map[GID]*poolGroupActive pendingGroups map[GID]*poolGroupPending[T, GID] completedPendingGroups []GID // Keep accumulative group metrics for each active/pending group: snapshotMu deadlock.RWMutex poolMetrics map[uint]MetricsSnapshot[T] // Track metrics per pool snapshotGroups map[GID]MetricsSnapshot[T] // The final snapshot snapshot GroupMetricsSnapshot[T, GID] nextPoolID uint } ``` -------------------------------- ### Blocking Dependency Pool Execution Flow Source: https://github.com/davidroman0o/retrypool/blob/main/expecting behaviour.md Illustrates the execution order of tasks in a blocking dependency pool, where parent tasks wait for child tasks to complete, demonstrating a reversed completion order. ```APIDOC Execution Order (forward): 1. groupA-1 (start) 2. groupA-2 (start) 3. groupA-3 (start) 4. groupA-3 (complete) 5. groupA-2 (complete) 6. groupA-1 (complete) Execution Order (reversed): 1. groupB-3 (start) 2. groupB-2 (start) 3. groupB-1 (start) 4. groupB-1 (complete) 5. groupB-2 (complete) 6. groupB-3 (complete) ``` -------------------------------- ### DependencyPool Core Logic: Submit and Completion Events Source: https://github.com/davidroman0o/retrypool/blob/main/expecting behaviour.md Describes the internal mechanisms of the DependencyPool when a task is submitted and when a task completes, detailing state management and dependency resolution. ```APIDOC 1. On Submit event: - Store task state - Check if task can run (based on mode and dependencies) - If yes -> submit to pool - If no -> keep in pending state 2. On Task Completion event: - Update task state as completed - Check all pending tasks that depend on completed one - Submit any that can now run ``` -------------------------------- ### Build Internal Pool for GroupPool Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This internal method constructs a new pool instance, configuring it with a worker, unique pool ID, and various options. It sets up logging, snapshotting with callbacks for metric updates, and defines success and failure handlers for task execution. These handlers extract group metadata and invoke `onTaskExecuted` accordingly, with the failure handler also returning `TaskActionRemove`. It returns the wrapped pool item or an error if creation fails. ```Go // buildPool creates a new internal pool with proper configuration and event handling. // Returns the wrapped pool item or error if creation failed. func (gp *GroupPool[T, GID]) buildPool() (*poolGroupPoolItem[T, GID], error) { // - create worker and validate // - generate pool ID // - add pool metadata containing poolID // - create pool options: // - logger from config // - metadata // - snapshots enabled with interval // - snapshot callback that: // - updates internal metrics // - triggers user snapshot callback if configured // - wrap and add success handler that: // - extracts group metadata ($group containing $groupID and $poolID) // - calls onTaskExecuted with success // - wrap and add failure handler that: // - extracts group metadata ($group containing $groupID and $poolID) // - calls onTaskExecuted with error // - returns TaskActionRemove // - create pool with worker and options // - wrap in poolGroupPoolItem and return } ``` -------------------------------- ### Go GroupPool Configuration Structure Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md Defines the `GroupPoolConfig` struct, holding all configurable options for the GroupPool. This includes logger, worker factory, pool limits, and various callbacks for task and group lifecycle events. ```go // GroupPoolConfig holds configuration options for the GroupPool. type GroupPoolConfig[T GroupTask[GID], GID comparable] struct { Logger Logger WorkerFactory WorkerFactory[T] MaxActivePools int MaxWorkersPerPool int UseFreeWorkerOnly bool GroupMustSucceed bool // by default a group can have deadtask/failure, if true the pending tasks will be discarded OnTaskSuccess func(gid GID, poolID uint, data T, metadata map[string]any) OnTaskFailure func(gid GID, poolID uint, data T, metadata map[string]any, err error) TaskAction // GroupPool specific callback handled either OnTaskSuccess or OnTaskFailure OnTaskExecuted func(gid GID, poolID uint, data T, metadata map[string]any, err error) OnSnapshot func(snapshot GroupMetricsSnapshot[T, GID]) OnGroupStarts func(gid GID, poolID uint) OnGroupEnds func(gid GID, poolID uint) OnGroupFails func(gid GID, poolID uint, pendingTasks []T) // only if `GroupMustSucceed` is enabled } ``` -------------------------------- ### Non-Blocking Dependency Pool Execution Flow Source: https://github.com/davidroman0o/retrypool/blob/main/expecting behaviour.md Illustrates the execution order of tasks in a non-blocking dependency pool, where tasks complete independently without waiting for others, demonstrating a sequential completion order. ```APIDOC Execution Order (forward): 1. groupA-1 (start) 2. groupA-1 (complete) 3. groupA-2 (start) 4. groupA-2 (complete) 5. groupA-3 (start) 6. groupA-3 (complete) Execution Order (reversed): 1. groupB-3 (start) 2. groupB-3 (complete) 3. groupB-2 (start) 4. groupB-2 (complete) 5. groupB-1 (start) 6. groupB-1 (complete) ``` -------------------------------- ### Go GroupPool Functional Option Type Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md Defines `GroupPoolOption` as a functional option type, used for configuring a `GroupPool` instance. This pattern allows for flexible and extensible pool initialization. ```go // GroupPoolOption is a functional option for configuring a GroupPool. type GroupPoolOption[T any, GID comparable] func(cfg *GroupPoolConfig[T, GID]) ``` -------------------------------- ### Submit Task to a Group Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This method submits a task to a specified group within the `GroupPool`. It validates the group and task IDs, then acquires a lock to manage state. The method handles tasks differently based on whether the group is active or pending, checking for group termination or failure conditions. It attempts to activate pending groups if resources are available and appends tasks to active groups or pending queues as appropriate, adding essential group metadata for tracking. ```Go // Submit task to a group func (gp *GroupPool[T, GID]) Submit(gid GID, task T, opts ...GroupTaskOption[T, GID]) error { // - validate group ID not zero value // - validate task's group ID matches submitted gid // - acquire lock for state access // - if check if group is active or not // - if active then: // - CHECK IF GROUP IS ENDED -> return error if true // - CHECK IF GROUP FAILED (GroupMustSucceed=true) -> return error // - check if we have tasks pending before that task, we just append our task to that tasks array or our active group // - else then, if we enabled `UseFreeWorkerOnly` we have to check if we have free workers // - if not, then we append the task to the `tasks` of our active group // - else, we can use the private submit function that will handle our task and opts // - if not active then: // - CHECK IF GROUP WAS FAILED -> return error // - check if pending group exists (if not create it) and assign the task to that pending group // - ATTEMPT TO ACTIVATE GROUP (might succeed right away if resources available) // - will add extra metadata: $groupID $poolID which will be helpful when onTaskExecuted is executed } ``` -------------------------------- ### Release Pool for Reuse in GroupPool Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This helper function marks a specific pool as available for reuse. After clearing its assignment and marking it as not busy, it attempts to activate any pending groups, leveraging the newly freed resource. ```Go func (gp *GroupPool[T, GID]) releasePool(poolID uint) { // - clear pool assignment // - mark as not busy // - attempt to activate pending group } ``` -------------------------------- ### Define Group Task Configuration Options Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md Defines the `groupSubmitConfig` struct used to configure task submissions within a group, including notifications and metadata. The `GroupTaskOption` type is a functional option pattern for modifying this configuration. The `WithTaskGroupQueueNotification`, `WithTaskGroupProcessedNotification`, and `WithTaskGroupMetadata` functions are constructors for these options, allowing users to specify callbacks for queueing/processing events and attach arbitrary metadata to tasks. ```Go type groupSubmitConfig struct { queueNotification *QueuedNotification processedNotification *ProcessedNotification metadata map[string]any } type GroupTaskOption[T any, GID comparable] func(*groupSubmitConfig) func WithTaskGroupQueueNotification[T any, GID comparable](notification *QueuedNotification) GroupTaskOption[T, GID] { return func(c *groupSubmitConfig) { c.queueNotification = notification } } func WithTaskGroupProcessedNotification[T any, GID comparable](notification *ProcessedNotification) GroupTaskOption[T, GID] { return func(c *groupSubmitConfig) { c.processedNotification = notification } } func WithTaskGroupMetadata[T any, GID comparable](metadata map[string]any) GroupTaskOption[T, GID] { return func(c *groupSubmitConfig) { c.metadata = metadata } } ``` -------------------------------- ### Go GroupTask Interface Definition Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md Defines the `GroupTask` interface, which tasks must implement to provide a group identifier. This allows the GroupPool to categorize and manage tasks based on their associated group. ```go // GroupTask is the interface for tasks that have a group ID. type GroupTask[GID comparable] interface { GetGroupID() GID } ``` -------------------------------- ### Submit Task to Specific Internal Pool Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This internal method manages the submission of a task to a specific `poolGroupPoolItem`. It processes task options, incorporating queue and processed notifications, and combines user-provided metadata with internal group tracking metadata. Depending on the `UseFreeWorkerOnly` configuration, it dispatches the task using either `SubmitToFreeWorker` or the normal `Submit` method of the underlying pool. ```Go // submitToPool handles submitting a task to a specific pool with proper configuration // Manages notifications, metadata and submission method based on pool config func (gp *GroupPool[T, GID]) submitToPool(pi *poolGroupPoolItem[T, GID], t T, opts ...GroupTaskOption[T, GID]) error { // - process task options into config // - build task options array: // - add queue notification if configured // - add processed notification if configured // - build metadata combining: // - user provided metadata // - group tracking metadata ($group with $groupID and $poolID) // - if UseFreeWorkerOnly enabled: // - use SubmitToFreeWorker // - else: // - use normal Submit } ``` -------------------------------- ### Go Helper Function to Check Zero Value of Group ID Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md A utility function `isZeroValueOfGroup` that checks if a given group ID is its zero value. This is typically used for validation purposes within the GroupPool. ```go func isZeroValueOfGroup[GID comparable](gid GID) bool { var zero GID return reflect.DeepEqual(gid, zero) } ``` -------------------------------- ### Go GroupPool Shutdown Method Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md The `Close` method is responsible for gracefully shutting down all managed pools and releasing associated resources. This ensures a clean termination of the GroupPool. ```go // Close shuts down all pools and cleans up resources ``` -------------------------------- ### Go Internal Active Group Structure Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md Defines `poolGroupActive`, an internal struct representing a group currently being processed by an active pool. It tracks the pool ID, group ID, and the state of tasks (pending, failed). ```go // Own by a pool and active type poolGroupActive[T GroupTask[GID], GID comparable] struct { poolID uint groupID GID isEnded bool // will refuse new tasks isFailed bool // track group failure state tasks []T // tasks not yet sent fails []T // failed tasks } ``` -------------------------------- ### Handle Task Execution Completion in GroupPool Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This function is the central driver for task flow within the GroupPool. It processes the outcome of a task execution, handling success, failure (especially for `GroupMustSucceed` groups), and determining the next steps for the associated task group, including submitting subsequent tasks or triggering group completion callbacks. ```Go func (gp *GroupPool[T, GID]) onTaskExecuted(gid GID, poolID uint, data T, metadata map[string]any, err error) { // - get active group // - handle error if GroupMustSucceed: // - mark group as failed // - clean up pool assignment // - mark pool as not busy // - trigger OnGroupFails with remaining tasks // - clear metrics // - ATTEMPT TO ACTIVATE PENDING GROUP since we freed a pool // - return early // - if group has pending tasks: // - CHECK IF GROUP IS ENDED before submitting next task // - submit next task immediately // - if submit fails: // - add to fails list // - if GroupMustSucceed, handle failure as above // - if no more tasks and group marked ended: // - trigger OnGroupEnds // - cleanup group // - clean up pool assignment // - mark pool as not busy // - clear metrics // - attempt to activate another pending group // - trigger OnTaskExecuted callback } ``` -------------------------------- ### Go GroupPool Error Definitions Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md Defines common error types used within the GroupPool system, indicating conditions such as a group not being found, already ended, or invalid task-group ID mismatches. ```go var ( ErrGroupNotFound = errors.New("group not found") ErrGroupEnded = errors.New("group already ended") ErrGroupFailed = errors.New("group has failed") ErrInvalidGroupID = errors.New("invalid group ID") ErrTaskGroupMismatch = errors.New("task group ID does not match submit group ID") ) ``` -------------------------------- ### Go Wait for Group Completion Method Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md The `WaitGroup` method allows callers to block until all tasks within a specified group have completed processing. It polls the group's status, checking for pending or active tasks. ```go // WaitGroup waits for all tasks in a group to complete func (gp *GroupPool[T, GID]) WaitGroup(ctx context.Context, gid GID) error { // - validate group ID not zero // - create ticker for polling (200ms default) // - loop: // - check context cancellation // - check if group is pending: // - if pending and not ended, continue waiting // - check if group is active: // - get pool // - check queue size and processing count // - if both zero and group ended, return // - if group not found and not pending, return (completed) } ``` -------------------------------- ### Go Internal Pool Group Item Structure Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md Defines `poolGroupPoolItem`, an internal struct representing a managed pool within the GroupPool. It tracks the pool's ID, the underlying worker pool, its assigned group, and its busy status. ```go // Managed pool by group pool type poolGroupPoolItem[T any, GID comparable] struct { mu deadlock.RWMutex PoolID uint Pool Pooler[T] assignedGroup GID busy bool } ``` -------------------------------- ### Cleanup Group Resources in GroupPool Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This helper function is responsible for releasing all resources associated with a completed or failed task group. It removes the group from active lists, cleans up pool assignments, clears metrics, and marks the pool as available. ```Go func (gp *GroupPool[T, GID]) cleanupGroup(gid GID, poolID uint) { // - remove from active groups // - cleanup pool assignment // - clear metrics // - mark pool as not busy } ``` -------------------------------- ### Go Internal Pending Group Structure Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md Defines `poolGroupPending`, an internal struct for groups whose tasks are not yet assigned to an active pool. It holds the group ID and the list of pending tasks. ```go // Not yet ready to be sent // - no pool own that group yet type poolGroupPending[T GroupTask[GID], GID comparable] struct { groupID GID tasks []T } ``` -------------------------------- ### Go Check for Free Workers in Pool Group Item Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md The `HasFreeWorkers` method checks if a specific `poolGroupPoolItem` has any available workers. This is crucial for the GroupPool to determine if a pool can accept new tasks. ```go // HasFreeWorkers checks if pool has any available workers func (pi *poolGroupPoolItem[T, GID]) HasFreeWorkers() bool { // - check if pool exists // - get free workers list // - return true if list not empty } ``` -------------------------------- ### Close GroupPool Resources Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This method closes the `GroupPool`, releasing resources and clearing internal states. It acquires a lock, cancels the context, closes all individual pools, collects any errors, and clears all internal maps and pending group lists. It returns the first error encountered during the closing process. ```Go func (gp *GroupPool[T, GID]) Close() error { // - acquire lock // - cancel context // - for each pool: // - close pool // - collect first error // - clear maps: // - pools // - activeGroups // - pendingGroups // - poolMetrics // - snapshotGroups // - clear completedPendingGroups // - return first error encountered if any } ``` -------------------------------- ### End a Task Group Source: https://github.com/davidroman0o/retrypool/blob/main/grouppool.md This method signals the end of a specific group, preventing it from accepting new tasks. It validates the group ID and acquires a lock for state management. If the group is active, it marks it as `isEnded`. If the group is pending and not already ended, it adds it to `completedPendingGroups`. The group remains active until all its pending tasks are fully processed. ```Go // Signal we ended the group // Once a group is ended, it will no longer accept new tasks // Group remains active until all pending tasks are processed func (gp *GroupPool[T, GID]) EndGroup(gid GID) error { // - validate group ID not zero value // - acquire lock for state access // - check if group is active: // - if active: // - mark isEnded = true // - return success // - check if group is pending: // - if pending and not already ended: // - add to completedPendingGroups if not already there } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.