### Install ExqLimit and Configure Queues in Elixir Source: https://context7.com/ananthakumaran/exq_limit/llms.txt Provides instructions for adding ExqLimit to your Elixir project's dependencies and configuring various rate limiters for your Exq queues in `config/config.exs`. It includes examples for Global, GCRA, and combined limiters. ```elixir # mix.exs def deps do [ {:exq, "~> 0.15"}, {:exq_limit, "~> 0.1.3"}, {:redix, ">= 0.9.0"} ] end # config/config.exs config :exq, name: Exq, host: "127.0.0.1", port: 6379, namespace: "exq", queues: [ {"default", {ExqLimit.Global, limit: 10}}, {"emails", {ExqLimit.GCRA, period: :hour, rate: 1000, burst: 50}}, {"critical", {ExqLimit.And, [ {ExqLimit.Local, limit: 5}, {ExqLimit.Global, limit: 20} ]}} ] ``` -------------------------------- ### Configure Global Concurrency Limit for Exq Queue Source: https://context7.com/ananthakumaran/exq_limit/llms.txt Configure a queue in Exq to use ExqLimit.Global for limiting job concurrency across all worker nodes. This example shows basic and advanced configuration options, including heartbeat intervals and custom node IDs. ```elixir config :exq, queues: [{"default", {ExqLimit.Global, limit: 10}}] ``` ```elixir config :exq, queues: [ {"api_calls", {ExqLimit.Global, limit: 100, # Maximum 100 concurrent jobs across all nodes interval: 20_000, # Heartbeat interval in milliseconds (default: 20_000) missed_heartbeats_allowed: 10, # Node considered dead after 10 missed heartbeats node_id: "worker_1" # Custom node identifier (defaults to Exq node identifier) }} ] ``` -------------------------------- ### Programmatic Global Concurrency Limit with ExqLimit.Global Source: https://context7.com/ananthakumaran/exq_limit/llms.txt Demonstrates programmatic initialization, checking availability, dispatching, and processing jobs using ExqLimit.Global. This is useful for testing or custom implementations where configuration files are not used. ```elixir queue_info = %{queue: "hard"} options = [limit: 10, interval: 50, missed_heartbeats_allowed: 5, redis: MyRedis] # Initialize two nodes with the global limiter {:ok, node1_state} = ExqLimit.Global.init(queue_info, options ++ [node_id: "n1"]) # node1_state.current == 10 (all capacity goes to first node) # node1_state.allowed == 10 {:ok, node2_state} = ExqLimit.Global.init(queue_info, options ++ [node_id: "n2"]) # node2_state.current == 0 (waits for rebalance) # node2_state.allowed == 5 (will get 5 after rebalance) # Check availability and dispatch jobs {:ok, true, node1_state} = ExqLimit.Global.available?(node1_state) {:ok, node1_state} = ExqLimit.Global.dispatched(node1_state) # Mark job as processed or failed {:ok, node1_state} = ExqLimit.Global.processed(node1_state) # or {:ok, node1_state} = ExqLimit.Global.failed(node1_state) # Clean shutdown removes node from pool :ok = ExqLimit.Global.stop(node2_state) ``` -------------------------------- ### Programmatic Local Concurrency Limit with ExqLimit.Local Source: https://context7.com/ananthakumaran/exq_limit/llms.txt Demonstrates programmatic usage of ExqLimit.Local for initializing, checking job availability, dispatching, and processing jobs. This allows for dynamic control over local concurrency. ```elixir queue_info = %{queue: "processing"} options = [limit: 5] {:ok, state} = ExqLimit.Local.init(queue_info, options) # state.limit == 5 # state.current == 0 # Check if a job slot is available {:ok, true, state} = ExqLimit.Local.available?(state) # Dispatch a job (increments current counter) {:ok, state} = ExqLimit.Local.dispatched(state) # state.current == 1 # After processing 5 jobs, available? returns false state = Enum.reduce(1..4, state, fn _, s -> {:ok, s} = ExqLimit.Local.dispatched(s) s end) # state.current == 5 {:ok, false, state} = ExqLimit.Local.available?(state) # Mark jobs as completed {:ok, state} = ExqLimit.Local.processed(state) # state.current == 4 {:ok, true, state} = ExqLimit.Local.available?(state) ``` -------------------------------- ### Programmatic GCRA Rate Limiter Usage in Elixir Source: https://context7.com/ananthakumaran/exq_limit/llms.txt Demonstrates the programmatic initialization and usage of the GCRA rate limiter in Elixir. It covers initializing the limiter, checking job availability, dispatching jobs, and marking them as processed, all while respecting the defined rate limits. ```elixir # Programmatic usage queue_info = %{queue: "rate_limited"} options = [period: :second, rate: 10, burst: 5, redis: MyRedis] {:ok, state} = ExqLimit.GCRA.init(queue_info, options) # Check availability (respects rate limit timing) {:ok, available?, state} = ExqLimit.GCRA.available?(state) if available? do {:ok, state} = ExqLimit.GCRA.dispatched(state) # ... process job ... {:ok, state} = ExqLimit.GCRA.processed(state) end ``` -------------------------------- ### Combine Rate Limiters with ExqLimit.And in Elixir Source: https://context7.com/ananthakumaran/exq_limit/llms.txt Illustrates how to combine multiple rate limiters (Local, Global, GCRA) using ExqLimit.And for sophisticated rate limiting strategies. Jobs are only dequeued when all combined limiters permit. The `short_circuit` option is explained. ```elixir # config/config.exs - Combine local, global, and GCRA limiters config :exq, queues: [ {"critical", {ExqLimit.And, [ # Local limit: max 20 concurrent jobs per node {ExqLimit.Local, limit: 20}, # Global limit: max 100 concurrent jobs across all nodes {ExqLimit.Global, limit: 100}, # Rate limit: max 60 per second with short_circuit for efficiency {ExqLimit.GCRA, [period: :second, rate: 60, burst: 0], short_circuit: true} ]}} ] # Programmatic usage for testing specs = [ {ExqLimit.Local, limit: 3}, {ExqLimit.Global, [limit: 6, redis: TestRedis, interval: 50, missed_heartbeats_allowed: 20]} ] queue_info = %{queue: "combined"} {:ok, state} = ExqLimit.And.init(queue_info, specs) # Check availability (all limiters must allow) {:ok, available?, state} = ExqLimit.And.available?(state) # Dispatch (notifies all limiters) {:ok, state} = ExqLimit.And.dispatched(state) # Process/fail (notifies all limiters) {:ok, state} = ExqLimit.And.processed(state) {:ok, state} = ExqLimit.And.failed(state) # Clean shutdown (stops all limiters) :ok = ExqLimit.And.stop(state) ``` -------------------------------- ### Handle ExqLimit Telemetry Events in Elixir Source: https://context7.com/ananthakumaran/exq_limit/llms.txt This code demonstrates how to attach telemetry handlers to monitor ExqLimit's Global, Local, and GCRA rate limiter events. It shows how to access measurements (like running jobs and quota) and metadata (like queue name) for logging and monitoring purposes. ```elixir # Attach handlers for ExqLimit telemetry events :telemetry.attach( "exq-limit-global-handler", [:exq_limit, :global], fn _event, measurements, metadata, _config -> # measurements: %{running: integer, quota: integer} # metadata: %{queue: string} Logger.info("Global limiter - Running: #{measurements.running}, Quota: #{measurements.quota}, Queue: #{metadata.queue}") end, nil ) :telemetry.attach( "exq-limit-local-handler", [:exq_limit, :local], fn _event, measurements, metadata, _config -> # measurements: %{running: integer} # metadata: %{queue: string} Logger.info("Local limiter - Running: #{measurements.running}, Queue: #{metadata.queue}") end, nil ) :telemetry.attach( "exq-limit-gcra-handler", [:exq_limit, :gcra], fn _event, measurements, metadata, _config -> # measurements: %{running: integer} # metadata: %{queue: string} Logger.info("GCRA limiter - Running: #{measurements.running}, Queue: #{metadata.queue}") end, nil ) ``` -------------------------------- ### Configure Global Rate Limiter for Exq Source: https://github.com/ananthakumaran/exq_limit/blob/master/README.md This snippet shows how to configure the Exq queue to use ExqLimit.Global for a global concurrency limit. It sets the limit to 10 jobs across all worker nodes. ```elixir config :exq, queues: [{"default", {ExqLimit.Global, limit: 10}}] ``` -------------------------------- ### Configure GCRA Rate Limiter in Exq Source: https://context7.com/ananthakumaran/exq_limit/llms.txt This snippet shows how to configure the GCRA (Generalized Rate Control Algorithm) rate limiter for an Exq queue. It specifies the period (e.g., 5 minutes) and the rate (e.g., 100 jobs per period). ```elixir config :exq, queues: [ {"custom", {ExqLimit.GCRA, period: 300, rate: 100}} # 100 jobs per 5 minutes ] ``` -------------------------------- ### Configure Local Concurrency Limit for Exq Queue Source: https://context7.com/ananthakumaran/exq_limit/llms.txt Configure an Exq queue to use ExqLimit.Local for limiting the number of concurrent jobs on a single worker node. This provides per-node concurrency control. ```elixir config :exq, queues: [{"default", {ExqLimit.Local, limit: 5}}] ``` -------------------------------- ### Configure Time-Based Rate Limiting with ExqLimit.GCRA Source: https://context7.com/ananthakumaran/exq_limit/llms.txt Configure an Exq queue with ExqLimit.GCRA to implement time-based rate limiting using the Generic Cell Rate Algorithm. Options include specifying the rate per period and an optional burst capacity. ```elixir config :exq, queues: [{"emails", {ExqLimit.GCRA, period: :hour, rate: 60}}] ``` ```elixir config :exq, queues: [ {"api_sync", {ExqLimit.GCRA, period: :hour, rate: 60, burst: 5 # Can process 5 jobs immediately when "battery" is full }} ] ``` ```elixir config :exq, queues: [ {"local_api", {ExqLimit.GCRA, period: :second, rate: 10, burst: 0, local: true # Rate limit applies per node, not globally }} ] ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.