### SQL Query for Fetching Records After Interruption

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/iteration-how-it-works.md

Illustrates the SQL query executed by Sidekiq Iteration to fetch the next batch of records, starting after the last successfully processed primary key.

```sql
SELECT "users".* FROM "users" WHERE "users"."id" > 2 ORDER BY "products"."id" LIMIT 100
```

--------------------------------

### Sidekiq Iteration: ActiveRecord Batch vs Records Enumerator

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/best-practices.md

Compares two approaches for iterating over ActiveRecord records in Sidekiq jobs. The 'bad' example processes a batch of comments, while the 'good' example processes individual comments. Both use `active_record_batches_enumerator` and `active_record_records_enumerator` respectively, with a batch size of 5. This difference impacts job interruption frequency and processing granularity.

```ruby
class BatchesJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(product_id, cursor:)
    active_record_batches_enumerator(
      Comment.where(product_id: product_id),
      cursor: cursor,
      batch_size: 5,
    )
  end

  def each_iteration(batch_of_comments, product_id)
    batch_of_comments.each(&:destroy)
  end
end
```

```ruby
class RecordsJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(product_id, cursor:)
    active_record_records_enumerator(
      Comment.where(product_id: product_id),
      cursor: cursor,
      batch_size: 5,
    )
  end

  def each_iteration(comment, product_id)
    comment.destroy
  end
end
```

--------------------------------

### Basic Sidekiq Job Example

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

A standard Sidekiq job that iterates over User records using `find_each`. This serves as a baseline to illustrate the problem Sidekiq Iteration solves, where long processing times can lead to lost progress.

```ruby
class SimpleJob
  include Sidekiq::Job

  def perform
    User.find_each do |user|
      user.notify_about_something
    end
  end
end
```

--------------------------------

### Job with positional arguments for sidekiq-iteration

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/argument-semantics.md

Shows a Sidekiq job utilizing sidekiq-iteration where positional arguments are passed to both `build_enumerator` and `each_iteration` methods. The example includes how to enqueue such a job.

```ruby
class ArgumentativeJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(arg1, arg2, arg3, cursor:)
    # ...
  end

  def each_iteration(single_object_yielded_from_enumerator, arg1, arg2, arg3)
    # ...
  end
end
```

--------------------------------

### CSV Iteration

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

Example of iterating over CSV data using Sidekiq Iteration.

```APIDOC
## Ruby Class: CsvJob

### Description
This class shows how to iterate over CSV data in a Sidekiq background job using Sidekiq Iteration.

### Method
`build_enumerator` fetches an import record and creates a CSV enumerator.
`each_iteration` is called for each row in the CSV.

### Endpoint
N/A (This is a background job implementation, not a web API endpoint)

### Parameters
**build_enumerator:**
- `import_id` (Integer) - The ID of the import record.
- `cursor` (String) - Required - The cursor for resuming iteration.

**each_iteration:**
- `csv_row` (Hash/Array) - Represents a row from the CSV file.
- `import_id` (Integer) - The ID of the import record associated with the CSV row.

### Request Example
```ruby
class CsvJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(import_id, cursor:)
    import = Import.find(import_id)
    csv_enumerator(import.csv, cursor: cursor)
  end

  def each_iteration(csv_row, import_id)
    # insert csv_row to database
  end
end
```

### Response
N/A (This is a background job implementation)

```

--------------------------------

### Array Iteration

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

Example of iterating over an arbitrary array using Sidekiq Iteration.

```APIDOC
## Ruby Class: ArrayJob

### Description
This class demonstrates how to use Sidekiq Iteration to process elements of an array in a background job.

### Method
`build_enumerator` is used to create an enumerator for the array.
`each_iteration` is called for each element in the array.

### Endpoint
N/A (This is a background job implementation, not a web API endpoint)

### Parameters
**build_enumerator:**
- `cursor` (String) - Required - The cursor for resuming iteration.

**each_iteration:**
- `array_element` (String) - Description of the element being processed.

### Request Example
```ruby
class ArrayJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(cursor:)
    array_enumerator(['build', 'enumerator', 'from', 'any', 'array'], cursor: cursor)
  end

  def each_iteration(array_element)
    # process array_element
  end
end
```

### Response
N/A (This is a background job implementation)

```

--------------------------------

### Nested Iteration

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

Example of performing nested iterations within a Sidekiq background job.

```APIDOC
## Ruby Class: NestedIterationJob

### Description
This class demonstrates how to handle nested iterations, processing records from multiple levels of relationships (e.g., shops, products, variants) using Sidekiq Iteration.

### Method
`build_enumerator` chains multiple enumerators using `nested_enumerator` to traverse through different data relationships.
`each_iteration` processes the innermost elements (product variants in this case).

### Endpoint
N/A (This is a background job implementation, not a web API endpoint)

### Parameters
**build_enumerator:**
- `cursor` (String) - Required - The cursor for resuming iteration.

**each_iteration:**
- `product_variants_relation` (Object) - Represents the product variants being processed.

### Request Example
```ruby
class NestedIterationJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(cursor:)
    nested_enumerator(
      [
        ->(cursor) { active_record_records_enumerator(Shop.all, cursor: cursor) },
        ->(shop, cursor) { active_record_records_enumerator(shop.products, cursor: cursor) },
        ->(_shop, product, cursor) { active_record_relations_enumerator(product.product_variants, cursor: cursor) }
      ],
      cursor: cursor
    )
  end

  def each_iteration(product_variants_relation)
    # do something
  end
end
```

### Response
N/A (This is a background job implementation)

```

--------------------------------

### Throttle Job with Dynamic Backoff (Ruby)

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/throttling.md

This example shows how to throttle a Sidekiq job using a dynamic backoff period defined by a Proc. The job pauses execution if the database is unhealthy, with the backoff duration determined by `RandomBackoffGenerator.generate_duration`. This provides flexibility in handling retry intervals.

```ruby
class DeleteAccountsThrottledJob
  throttle_on(backoff: -> { RandomBackoffGenerator.generate_duration } ) do
    DatabaseStatus.unhealthy?
  end
  # ...
end
```

--------------------------------

### Define Interruptible Job with ActiveRecord

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/iteration-how-it-works.md

Example of a Sidekiq job using SidekiqIteration to process ActiveRecord records interruptibly. It defines `build_enumerator` to create an enumerator from an ActiveRecord relation and `each_iteration` for processing individual records.

```ruby
class NotifyUsers
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(cursor:)
    active_record_records_enumerator(User.all, cursor: cursor)
  end

  def each_iteration(user)
    user.notify_about_something
  end
end
```

--------------------------------

### Custom enumerator yielding object and cursor

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/argument-semantics.md

Provides an example of a custom enumerator for sidekiq-iteration. It demonstrates yielding both the object for the current iteration and a cursor value to track progress. The yielded cursor is an Integer in this case.

```ruby
Enumerator.new do |yielder|
  # In this case `cursor` is an Integer
  cursor.upto(99999) do |offset|
    yielder.yield(fetch_record_at(offset), offset)
  end
end
```

--------------------------------

### Iterate over CSV with Sidekiq

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

This example demonstrates iterating over rows of a CSV file within a Sidekiq job. It uses `csv_enumerator` to process each row, assuming an `Import` model with a CSV attribute.

```ruby
class CsvJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(import_id, cursor:)
    import = Import.find(import_id)
    csv_enumerator(import.csv, cursor: cursor)
  end

  def each_iteration(csv_row, import_id)
    # insert csv_row to database
  end
end
```

--------------------------------

### SQL Query for Initial Record Fetching

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/iteration-how-it-works.md

Shows the SQL query used by Sidekiq Iteration to fetch the initial batch of records for processing.

```sql
SELECT "users".* FROM "users" ORDER BY "users"."id" LIMIT 100
```

--------------------------------

### Job with keyword arguments for sidekiq-iteration

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/argument-semantics.md

Illustrates a Sidekiq job using sidekiq-iteration with keyword arguments. These arguments are passed as a Hash to `build_enumerator` and `each_iteration`, requiring fetching specific parameters.

```ruby
class ParameterizedJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(kwargs, cursor:)
    name = kwargs.fetch("name")
    email = kwargs.fetch("email")
    # ...
  end

  def each_iteration(object_yielded_from_enumerator, kwargs)
    name = kwargs.fetch("name")
    email = kwargs.fetch("email")
    # ...
  end
end
```

--------------------------------

### Sidekiq Iteration: Basic Usage

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

Demonstrates the fundamental usage of Sidekiq Iteration by including the `SidekiqIteration::Iteration` module and implementing `build_enumerator` and `each_iteration`. It uses `active_record_records_enumerator` to process `User` models.

```ruby
class NotifyUsersJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(cursor:)
    active_record_records_enumerator(User.all, cursor: cursor)
  end

  def each_iteration(user)
    user.notify_about_something
  end
end
```

--------------------------------

### Sidekiq Iteration: Custom Lifecycle Callbacks

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

Illustrates the use of various lifecycle callbacks provided by Sidekiq Iteration. These callbacks (`on_start`, `around_iteration`, `on_resume`, `on_shutdown`, `on_complete`) allow for custom logic execution at different stages of the job's lifecycle.

```ruby
class NotifyUsersJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def on_start
    # Will be called when the job starts iterating. Called only once, for the first time.
  end

  def around_iteration
    # Will be called around each iteration.
    # Can be useful for some metrics collection, performance tracking etc.
    yield
  end

  def on_resume
    # Called when the job resumes iterating.
  end

  def on_shutdown
    # Called each time the job is interrupted.
    # This can be due to throttling, `max_job_runtime` configuration, or sidekiq restarting.
  end

  def on_complete
    # Called when the job finished iterating.
  end

  # ...
end
```

--------------------------------

### Job with mixed arguments for sidekiq-iteration

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/argument-semantics.md

Demonstrates a Sidekiq job that accepts both positional and keyword arguments, managed by sidekiq-iteration. Keyword arguments are provided as a Hash, while positional arguments are passed directly.

```ruby
class HighlyConfigurableGreetingJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(subject_line, kwargs, cursor:)
    name = kwargs.fetch("sender_name")
    email = kwargs.fetch("sender_email")
    # ...
  end

  def each_iteration(object_yielded_from_enumerator, subject_line, kwargs)
    name = kwargs.fetch("sender_name")
    email = kwargs.fetch("sender_email")
    # ...
  end
end
```

--------------------------------

### Job without arguments for sidekiq-iteration

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/argument-semantics.md

Demonstrates a Sidekiq job that includes the SidekiqIteration::Iteration module and does not accept any arguments. The `build_enumerator` and `each_iteration` methods are defined, with `cursor` being managed internally.

```ruby
class ArglessJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(cursor:)
    # ...
  end

  def each_iteration(single_object_yielded_from_enumerator)
    # ...
  end
end
```

--------------------------------

### Sidekiq Iteration: Per-Job Max Job Runtime Configuration

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/best-practices.md

Shows how to configure the maximum job runtime on a per-job class basis within Sidekiq Iteration. This provides more granular control, allowing different jobs to have distinct runtime limits. The setting can be overridden in subclasses.

```ruby
class MyJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  self.max_job_runtime = 3.minutes

  # ...
end
```

--------------------------------

### Define Interruptible Job with Custom Enumerator

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/iteration-how-it-works.md

Demonstrates how to create a Sidekiq job with SidekiqIteration using a custom Enumerator, such as processing elements from Redis.

```ruby
class MyJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(cursor:)
    Enumerator.new do
      Redis.lpop("mylist") # or: Kafka.poll(timeout: 10.seconds)
    end
  end

  def each_iteration(element_from_redis)
    # ...
  end
end
```

--------------------------------

### Sidekiq Iteration: Job with Custom Arguments

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

Shows how to pass custom arguments to a Sidekiq Iteration job. The `build_enumerator` and `each_iteration` methods can accept additional parameters beyond the `cursor`, enabling more flexible job configurations.

```ruby
class ArgumentsJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(arg1, arg2, cursor:)
    active_record_records_enumerator(User.all, cursor: cursor)
  end

  def each_iteration(user, arg1, arg2)
    user.notify_about_something
  end
end

ArgumentsJob.perform_async(arg1, arg2)
```

--------------------------------

### Nested Iteration with Sidekiq

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

This snippet illustrates how to handle nested iterations in Sidekiq. It uses `nested_enumerator` to chain multiple enumerators, allowing iteration through levels of data, such as shops, their products, and product variants.

```ruby
class NestedIterationJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(cursor:)
    nested_enumerator(
      [
        ->(cursor) { active_record_records_enumerator(Shop.all, cursor: cursor) },
        ->(shop, cursor) { active_record_records_enumerator(shop.products, cursor: cursor) },
        ->(_shop, product, cursor) { active_record_relations_enumerator(product.product_variants, cursor: cursor) }
      ],
      cursor: cursor
    )
  end

  def each_iteration(product_variants_relation)
    # do something
  end
end
```

--------------------------------

### Throttle Job on Database Unhealthy Condition (Ruby)

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/throttling.md

This snippet demonstrates how to throttle a Sidekiq job when the database is unhealthy. It uses `throttle_on` with a 1-minute backoff period. The job iterates through inactive accounts and deletes them. It includes dependencies on `Sidekiq::Job`, `SidekiqIteration::Iteration`, and a custom `DatabaseStatus` class.

```ruby
class DeleteAccountsThrottledJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  throttle_on(backoff: 1.minute) do
    DatabaseStatus.unhealthy?
  end

  def build_enumerator(cursor:)
    active_record_relations_enumerator(Account.inactive, cursor: cursor)
  end

  def each_iteration(accounts)
    accounts.delete_all
  end
end
```

--------------------------------

### Sidekiq Iteration: Iterating Over Batches of Active Record Objects

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

Demonstrates iterating over batches of Active Record objects using `active_record_batches_enumerator`. This method is useful for processing records in chunks, specified by `batch_size`, which can be more efficient for certain operations.

```ruby
class BatchesJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(product_id, cursor:)
    active_record_batches_enumerator(
      Comment.where(product_id: product_id).select(:id),
      cursor: cursor,
      batch_size: 100,
    )
  end

  def each_iteration(batch_of_comments, product_id)
    comment_ids = batch_of_comments.map(&:id)
    CommentService.call(comment_ids: comment_ids)
  end
end
```

--------------------------------

### Sidekiq Job with Stripe Enumerator in Ruby

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/custom-enumerator.md

Illustrates a Sidekiq job that integrates the `StripeListEnumerator` to process paginated Stripe refund data. It defines the `build_enumerator` method to instantiate the custom enumerator with specific Stripe parameters and cursor handling.

```ruby
class LoadRefundsForChargeJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(charge_id, cursor:)
    StripeListEnumerator.new(
      Stripe::Refund,
      params: { charge: charge_id }, # "charge_id" will be a prefixed Stripe ID such as "chrg_123"
      options: { api_key: "sk_test_123", stripe_version: "2018-01-18" },
      cursor: cursor
    ).to_enumerator
  end

  # Note that in this case `each_iteration` will only receive one positional argument per iteration.
  # If what your enumerator yields is a composite object you will need to unpack it yourself
  # inside the `each_iteration`.
  def each_iteration(stripe_refund, charge_id)
    # ...
  end
end

LoadRefundsForChargeJob.perform_later(_charge_id = "chrg_345")
```

--------------------------------

### Sidekiq Iteration: Global Max Job Runtime Configuration

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/best-practices.md

Demonstrates how to set a global maximum runtime for all Sidekiq Iteration jobs. This setting controls how frequently jobs will interrupt themselves, even if no external interruption signals are present. The default value is nil, meaning no self-interruption based on runtime.

```ruby
SidekiqIteration.max_job_runtime = 5.minutes # nil by default
```

--------------------------------

### Sidekiq Iteration: Iterating Over Active Record Relations

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

Shows how to iterate over Active Record relations using `active_record_relations_enumerator`. This is suitable for operations that can be performed directly on a relation, like bulk updates using `update_all`.

```ruby
class RelationsJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(product_id, cursor:)
    active_record_relations_enumerator(
      Product.find(product_id).comments,
      cursor: cursor,
      batch_size: 100,
    )
  end

  def each_iteration(comments_relation, product_id)
    # comments_relation will be a Comment::ActiveRecord_Relation
    comments_relation.update_all(deleted: true)
  end
end
```

--------------------------------

### Iterate over Arrays with Sidekiq

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/README.md

This snippet shows how to iterate over an arbitrary array using Sidekiq. It defines a job that uses `array_enumerator` to process each element of a predefined array.

```ruby
class ArrayJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(cursor:)
    array_enumerator(['build', 'enumerator', 'from', 'any', 'array'], cursor: cursor)
  end

  def each_iteration(array_element)
    # use array_element
  end
end
```

--------------------------------

### Cursorless Redis List Enumerator in Ruby

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/custom-enumerator.md

Demonstrates a Sidekiq job that iterates over items in a Redis list using an Enumerator. It ignores the cursor as Redis lists are treated as queues. The enumerator yields each item from the list and `nil` as the cursor.

```ruby
class ListJob
  include Sidekiq::Job
  include SidekiqIteration::Iteration

  def build_enumerator(*)
    @redis = Redis.new
    Enumerator.new do |yielder|
      loop do
        item = @redis.lpop(key)
        break unless item

        yielder.yield(item, nil)
      end
    end
  end

  def each_iteration(item_from_redis)
    # ...
  end
end
```

--------------------------------

### Stripe Paginated API Enumerator in Ruby

Source: https://github.com/fatkodima/sidekiq-iteration/blob/master/guides/custom-enumerator.md

Shows a Ruby class that creates an Enumerator for paginated results from the Stripe API. It uses the item's ID as a cursor to handle pagination and resuming from the last processed item. The `to_enumerator` method prepares it for use with Sidekiq Iteration.

```ruby
class StripeListEnumerator
  # @param resource [Stripe::APIResource] The type of Stripe object to request
  # @param params [Hash] Query parameters for the request
  # @param options [Hash] Request options, such as API key or version
  # @param cursor [nil, String] The Stripe ID of the last item iterated over
  def initialize(resource, params: {}, options: {}, cursor:)
    pagination_params = {}
    pagination_params[:starting_after] = cursor unless cursor.nil?

    # The following line makes a request, consider adding your rate limiter here.
    @list = resource.public_send(:list, params.merge(pagination_params), options)
  end

  def to_enumerator
    to_enum(:each).lazy
  end

  private

  # We yield our enumerator with the object id as the index so it is persisted
  # as the cursor on the job. This allows us to properly set the
  # `starting_after` parameter for the API request when resuming.
  def each
    loop do
      @list.each do |item, _index|
        # The first argument is what gets passed to `each_iteration`.
        # The second argument (item.id) is going to be persisted as the cursor, 
        # it doesn't get passed to `each_iteration`.
        yield item, item.id
      end

      # The following line makes a request, consider adding your rate limiter here.
      @list = @list.next_page

      break if @list.empty?
    end
  end
end
```