### Install and Start Redis on macOS Source: https://github.com/akira/exq/blob/master/README.md This snippet provides commands to install and start a Redis instance on macOS using Homebrew. Redis is a prerequisite for using the Exq library, as it serves as the data store for background jobs. After installation, `redis-server` command starts the Redis server. ```bash > brew install redis > redis-server ``` -------------------------------- ### Start Exq in IEx Console Source: https://github.com/akira/exq/blob/master/README.md Steps to get Exq dependencies and start the Elixir interactive shell (IEx) for testing. This allows for interactive development and experimentation with Exq. ```bash > mix deps.get > iex -S mix ``` -------------------------------- ### Start Exq Manually Source: https://github.com/akira/exq/blob/master/README.md Demonstrates how to start Exq manually using `Exq.start_link`. This can be used for basic startup or with custom configurations for multiple instances. ```elixir Exq.start_link ``` ```elixir Exq.start_link([host: "127.0.0.1", port: 6379, namespace: "x"]) ``` ```elixir Exq.start_link(name: Exq.Custom) ``` -------------------------------- ### Run Exq Standalone Source: https://github.com/akira/exq/blob/master/README.md Command to start Exq as a standalone application from the command line. This involves fetching dependencies, starting the application, and running Exq. ```bash > mix do app.start, exq.run ``` -------------------------------- ### Start Multiple Named Exq Instances Source: https://context7.com/akira/exq/llms.txt Shows how to configure and start multiple Exq instances with distinct names, queue configurations, and concurrency settings. This allows for specialized handling of different job types. ```elixir # Start multiple instances with different names {:ok, _} = Exq.start_link( name: CriticalExq, queues: ["critical"], concurrency: 50 ) {:ok, _} = Exq.start_link( name: BulkExq, queues: ["bulk", "reports"], concurrency: 10 ) ``` -------------------------------- ### Start and Enqueue Exq Job in IEx Source: https://context7.com/akira/exq/llms.txt Demonstrates how to start Exq within an IEx session and manually enqueue a job. This is helpful for interactive debugging and testing Exq's functionality. ```elixir # Or in IEx for interactive debugging iex -S mix # Then manually start and test iex> {:ok, sup} = Exq.start_link() iex> {:ok, jid} = Exq.enqueue(Exq, "default", MyWorker, ["test"]) iex> Exq.Api.queue_size(Exq.Api, "default") ``` -------------------------------- ### Custom Exq Middleware for Logging Source: https://context7.com/akira/exq/llms.txt Illustrates how to create and configure custom middleware for Exq to add specific behaviors like logging. The example defines a `CustomLogger` module that implements `Exq.Middleware.Behaviour` and logs job start and completion times. It's then added to the Exq middleware chain in `config/config.exs`. ```elixir defmodule MyApp.Middleware.CustomLogger do @behaviour Exq.Middleware.Behaviour alias Exq.Middleware.Pipeline def before_work(%Pipeline{assigns: assigns} = pipeline) do IO.puts("Starting job #{assigns.job.jid} - #{assigns.job.class}") start_time = System.monotonic_time(:millisecond) Pipeline.assign(pipeline, :start_time, start_time) end def after_processed_work(%Pipeline{assigns: assigns} = pipeline) do duration = System.monotonic_time(:millisecond) - assigns.start_time IO.puts("Completed job #{assigns.job.jid} in #{duration}ms") pipeline end def after_failed_work(%Pipeline{assigns: assigns} = pipeline) do duration = System.monotonic_time(:millisecond) - assigns.start_time IO.puts("Failed job #{assigns.job.jid} after #{duration}ms - #{inspect(assigns.error)}") pipeline end end # config/config.exs - add to middleware chain config :exq, middleware: [ Exq.Middleware.Stats, Exq.Middleware.Job, Exq.Middleware.Manager, Exq.Middleware.Unique, MyApp.Middleware.CustomLogger, Exq.Middleware.Logger ] ``` -------------------------------- ### Configure Exq for Testing with Mock Adapter Source: https://github.com/akira/exq/blob/master/README.md Shows how to configure Exq to use the `Exq.Adapters.Queue.Mock` for testing purposes. This involves updating the `config/test.exs` file and starting the mock server in `test_helper.exs`. ```elixir config :exq, queue_adapter: Exq.Adapters.Queue.Mock Exq.Mock.start_link(mode: :redis) ``` -------------------------------- ### Direct Exq Enqueuer Usage Source: https://context7.com/akira/exq/llms.txt Demonstrates how to directly use Exq.Enqueuer to start the enqueuer process and enqueue jobs. This involves starting the enqueuer with a specified port and then using functions like `enqueue`, `enqueue_in`, and `enqueue_at` to schedule tasks. ```elixir {:ok, _} = Exq.Enqueuer.start_link(port: 6379) {:ok, jid} = Exq.Enqueuer.enqueue(Exq.Enqueuer, "default", MyWorker, [arg1, arg2]) {:ok, jid} = Exq.Enqueuer.enqueue_in(Exq.Enqueuer, "default", 300, MyWorker, [arg1]) {:ok, jid} = Exq.Enqueuer.enqueue_at(Exq.Enqueuer, "default", datetime, MyWorker, [arg1]) ``` -------------------------------- ### Manually Start Exq in Supervision Tree Source: https://github.com/akira/exq/blob/master/README.md Demonstrates how to manually add Exq to your application's supervision tree when `start_on_application` is set to `false`. This provides explicit control over Exq's lifecycle. ```elixir def start(_type, _args) do children = [ # Start the Ecto repository MyApp.Repo, # Start the endpoint when the application starts MyApp.Endpoint, # Start the EXQ supervisor Exq, ] end ``` -------------------------------- ### Add Exq to OTP Application Source: https://github.com/akira/exq/blob/master/README.md Includes Exq in the list of applications to start when your OTP application boots. Exq will use configurations from `config.exs`. ```elixir def application do [ applications: [:logger, :exq], #other stuff... ] end ``` -------------------------------- ### Exq Installation and Configuration Source: https://context7.com/akira/exq/llms.txt Defines Exq dependencies in mix.exs and configures Redis connection and queue options in config.exs. Supports direct Redis options, Redis URL, and environment variables. ```elixir defp deps do [ {:exq, "~> 0.23.0"}, {:jason, "~> 1.0"} # or {:poison, ">= 1.2.0"} ] end # config/config.exs config :exq, name: Exq, host: "127.0.0.1", port: 6379, password: "optional_redis_auth", namespace: "exq", concurrency: :infinite, queues: ["default"], poll_timeout: 50, scheduler_poll_timeout: 200, scheduler_enable: true, max_retries: 25, mode: :default, shutdown_timeout: 5000, dead_max_jobs: 10_000, dead_timeout_in_seconds: 180 * 24 * 60 * 60, json_library: Jason, middleware: [ Exq.Middleware.Stats, Exq.Middleware.Job, Exq.Middleware.Manager, Exq.Middleware.Unique, Exq.Middleware.Logger ] # Using Redis URL instead of individual options config :exq, url: "redis://user:password@host:6379/0", namespace: "exq" # Using environment variables config :exq, host: {:system, "REDIS_HOST"}, port: {:system, "REDIS_PORT"} ``` -------------------------------- ### Disable Automatic Exq Startup Source: https://github.com/akira/exq/blob/master/README.md Configures Exq to not start automatically on application boot. This is useful for manual integration into the supervision tree, especially in frameworks like Phoenix. ```elixir config :exq, start_on_application: false ``` -------------------------------- ### Configure Exq Telemetry Handlers in Elixir Source: https://context7.com/akira/exq/llms.txt Attaches telemetry handlers to Exq job events (start, stop, exception) to log job status and metrics. It requires a Telemetry module that implements the handle_event callback. This setup is crucial for monitoring job execution. ```elixir defmodule MyApp.Application do use Application def start(_type, _args) do # Attach telemetry handlers :telemetry.attach_many( "exq-metrics", [ [:exq, :job, :start], [:exq, :job, :stop], [:exq, :job, :exception] ], &MyApp.Telemetry.handle_event/4, nil ) children = [ MyApp.Repo, Exq ] Supervisor.start_link(children, strategy: :one_for_one) end end defmodule MyApp.Telemetry do require Logger def handle_event([:exq, :job, :start], measurements, metadata, _config) do Logger.info("Job started: #{metadata.class} (#{metadata.jid}) on queue #{metadata.queue}") end def handle_event([:exq, :job, :stop], measurements, metadata, _config) do duration_ms = System.convert_time_unit(measurements.duration, :native, :millisecond) Logger.info("Job completed: #{metadata.class} (#{metadata.jid}) in #{duration_ms}ms") # Send to your metrics system # MyApp.Metrics.histogram("exq.job.duration", duration_ms, tags: [queue: metadata.queue]) end def handle_event([:exq, :job, :exception], measurements, metadata, _config) do duration_ms = System.convert_time_unit(measurements.duration, :native, :millisecond) Logger.error("Job failed: #{metadata.class} (#{metadata.jid}) - #{metadata.kind}: #{inspect(metadata.reason)}") # MyApp.Metrics.increment("exq.job.failed", tags: [queue: metadata.queue, class: metadata.class]) end end ``` -------------------------------- ### Configure Exq Running Modes Source: https://context7.com/akira/exq/llms.txt Examples of configuring Exq to run in different modes tailored to specific use cases. Modes include default (worker, enqueuer, API), enqueuer-only, API-only, and a combination of enqueuer and API without workers. ```elixir # Default mode - starts worker, enqueuer, and API {:ok, _} = Exq.start_link(mode: :default) # Enqueuer only - for web servers that only enqueue jobs {:ok, _} = Exq.start_link(mode: :enqueuer, name: ExqEnqueuer) {:ok, _} = Exq.Enqueuer.enqueue(ExqEnqueuer.Enqueuer, "default", MyWorker, []) # API only - for monitoring dashboards {:ok, _} = Exq.start_link(mode: :api, name: ExqApi) {:ok, queues} = Exq.Api.queues(ExqApi.Api) # Combined enqueuer and API (no workers) {:ok, _} = Exq.start_link(mode: [:enqueuer, :api], name: ExqClient) {:ok, _} = Exq.Enqueuer.enqueue(ExqClient.Enqueuer, "default", MyWorker, []) {:ok, queues} = Exq.Api.queues(ExqClient.Api) ``` -------------------------------- ### Configure Exq in Phoenix Application Source: https://github.com/akira/exq/blob/master/README.md This snippet shows how to add Exq to your Phoenix application's mix.exs file. It ensures that Exq is included in the list of applications started with your Phoenix project. ```elixir def application do [ mod: {Chat, []}, applications: [:phoenix, :phoenix_html, :cowboy, :logger, :exq] ] end ``` -------------------------------- ### Enqueue Jobs without Workers Source: https://github.com/akira/exq/blob/master/README.md Shows how to start an Exq enqueuer process independently and enqueue jobs without running full worker processes. Useful for scenarios where only job submission is needed. ```elixir {:ok, sup} = Exq.Enqueuer.start_link([port: 6379]) {:ok, ack} = Exq.Enqueuer.enqueue(Exq.Enqueuer, "default", MyWorker, []) ``` -------------------------------- ### Dynamically Subscribe/Unsubscribe Queues Source: https://github.com/akira/exq/blob/master/README.md Allows for dynamic management of queue subscriptions after Exq has started. You can subscribe to new queues with an optional concurrency limit or unsubscribe from existing ones. ```elixir # To subscribe to a new queue: :ok = Exq.subscribe(Exq, "new_queue_name", 10) # To unsubscribe from a queue: :ok = Exq.unsubscribe(Exq, "queue_to_unsubscribe") # To unsubscribe from all queues: :ok = Exq.unsubscribe_all(Exq) ``` -------------------------------- ### Phoenix Integration with Exq Source: https://context7.com/akira/exq/llms.txt Shows how to configure and integrate Exq within a Phoenix application. This includes disabling auto-start in `config/config.exs`, starting Exq as a child process in `application.ex`, defining a worker that uses Ecto, and enqueuing jobs from a Phoenix controller. ```elixir # config/config.exs - disable auto-start config :exq, start_on_application: false # lib/my_app/application.ex defmodule MyApp.Application do use Application def start(_type, _args) do children = [ MyApp.Repo, MyAppWeb.Endpoint, Exq # Start Exq after Repo is available ] opts = [strategy: :one_for_one, name: MyApp.Supervisor] Supervisor.start_link(children, opts) end end # Worker using Ecto defmodule MyApp.Workers.UserNotifier do alias MyApp.{Repo, User, Mailer} def perform(user_id, notification_type) do user = Repo.get!(User, user_id) case notification_type do "welcome" -> Mailer.send_welcome(user) "reminder" -> Mailer.send_reminder(user) end end end # Enqueueing from a Phoenix controller defmodule MyAppWeb.UserController do use MyAppWeb, :controller def create(conn, %{"user" => user_params}) do case MyApp.Accounts.create_user(user_params) do {:ok, user} -> Exq.enqueue(Exq, "default", MyApp.Workers.UserNotifier, [user.id, "welcome"]) conn |> put_flash(:info, "User created!") |> redirect(to: ~p"/users/#{user}") {:error, changeset} -> render(conn, :new, changeset: changeset) end end end ``` -------------------------------- ### Exq Worker Performing Database Operations in Phoenix Source: https://github.com/akira/exq/blob/master/README.md An example of an Exq worker performing a database insert operation using Ecto's Repo within a Phoenix application. This demonstrates how to interact with your database from within an Exq worker. ```elixir defmodule Worker do def perform do HelloPhoenix.Repo.insert!(%HelloPhoenix.User{name: "Hello", email: "world@yours.com"}) end end ``` -------------------------------- ### Snooze Middleware for Rescheduling Jobs Source: https://context7.com/akira/exq/llms.txt Explains how to enable and use the Snooze middleware in Exq to reschedule jobs from within a worker. The middleware is enabled in `config/config.exs`, and the example worker demonstrates returning `{:snooze, seconds}` to indicate a job should be retried after a specified delay. ```elixir # config/config.exs - add Snooze middleware config :exq, middleware: [ Exq.Middleware.Stats, Exq.Middleware.Job, Exq.Middleware.Manager, Exq.Middleware.Unique, Exq.Middleware.Snooze, # Enable snooze support Exq.Middleware.Logger ] # Worker that snoozes defmodule MyApp.Workers.RateLimitedWorker do def perform(api_call_params) do case MyApp.ExternalApi.call(api_call_params) do {:ok, result} -> process_result(result) {:error, :rate_limited} -> # Reschedule job to run in 60 seconds {:snooze, 60} {:error, :temporary_failure} -> # Retry in 5 seconds {:snooze, 5} end end defp process_result(result) do # Process the successful result :ok end end ``` -------------------------------- ### API: Managing Jobs in Queues Source: https://context7.com/akira/exq/llms.txt This API allows you to query, find, and remove jobs from your Exq queues. You can list all jobs, jobs in a specific queue with pagination, or get raw JSON representations of jobs. It also supports finding specific jobs by JID and removing enqueued jobs, with an option for unique token cleanup. Additionally, you can clear an entire queue. ```APIDOC ## GET /jobs ### Description Lists all jobs across all queues. ### Method GET ### Endpoint /jobs ### Parameters #### Query Parameters - **queue** (string) - Optional - The name of the queue to list jobs from. - **size** (integer) - Optional - The number of jobs to return per page. - **offset** (integer) - Optional - The starting offset for pagination. - **raw** (boolean) - Optional - If true, returns raw JSON (not deserialized). ### Request Example ```json { "size": 10, "offset": 0, "raw": true } ``` ### Response #### Success Response (200) - **queue_name** (string) - The name of the queue. - **jobs** (array) - A list of jobs in the queue. #### Response Example ```json { "default": [ { "jid": "job-jid-1", ... }, { "jid": "job-jid-2", ... } ], "critical": [ { "jid": "job-jid-3", ... } ] } ``` ## GET /jobs/{queue_name} ### Description Lists jobs in a specific queue with pagination. ### Method GET ### Endpoint /jobs/{queue_name} ### Parameters #### Path Parameters - **queue_name** (string) - Required - The name of the queue. #### Query Parameters - **size** (integer) - Optional - The number of jobs to return per page. - **offset** (integer) - Optional - The starting offset for pagination. - **raw** (boolean) - Optional - If true, returns raw JSON (not deserialized). ### Request Example ```json { "size": 10, "offset": 0, "raw": true } ``` ### Response #### Success Response (200) - **jobs** (array) - A list of jobs in the specified queue. #### Response Example ```json [ { "jid": "job-jid-1", ... }, { "jid": "job-jid-2", ... } ] ``` ## GET /jobs/{queue_name}/{jid} ### Description Finds a specific job by its JID within a given queue. ### Method GET ### Endpoint /jobs/{queue_name}/{jid} ### Parameters #### Path Parameters - **queue_name** (string) - Required - The name of the queue. - **jid** (string) - Required - The Job ID of the job to find. ### Response #### Success Response (200) - **job** (object) - The found job object. #### Response Example ```json { "jid": "job-jid-here", ... } ``` ## DELETE /jobs/{queue_name} ### Description Removes jobs from a specific queue. This can be used to remove enqueued jobs, with an option for unique token cleanup. ### Method DELETE ### Endpoint /jobs/{queue_name} ### Parameters #### Path Parameters - **queue_name** (string) - Required - The name of the queue. #### Query Parameters - **clear_unique_tokens** (boolean) - Optional - If true, clears associated unique tokens. #### Request Body - **jobs** (array) - Required - A list of job objects to remove. ### Request Example ```json [ { "jid": "job-jid-1", ... }, { "jid": "job-jid-2", ... } ] ``` ### Response #### Success Response (200) - **message** (string) - Indicates successful removal. #### Response Example ```json { "message": "Jobs removed successfully." } ``` ## DELETE /queues/{queue_name} ### Description Clears an entire queue, removing all jobs within it. ### Method DELETE ### Endpoint /queues/{queue_name} ### Parameters #### Path Parameters - **queue_name** (string) - Required - The name of the queue to clear. ### Response #### Success Response (200) - **message** (string) - Indicates successful clearing of the queue. #### Response Example ```json { "message": "Queue cleared successfully." } ``` ``` -------------------------------- ### Run Exq Tests Source: https://github.com/akira/exq/blob/master/README.md Provides commands to run Exq tests. `mix test --no-start` runs the basic test suite, while `mix test --trace --include failure_scenarios:true --no-start` runs the full suite including failure scenarios. ```bash mix test --no-start ``` ```bash mix test --trace --include failure_scenarios:true --no-start ``` -------------------------------- ### Execute Exq Job Inline with Mock Source: https://context7.com/akira/exq/llms.txt Shows how to configure Exq to execute jobs inline using the mock adapter. This allows for immediate execution and assertion of job side effects within the same process, useful for integration-like tests. ```elixir test "executes job inline" do Exq.Mock.set_mode(:inline) # Job executes immediately in the same process {:ok, _jid} = Exq.enqueue(Exq, "default", MyApp.Workers.SimpleWorker, ["arg"]) # Assertions about side effects end ``` -------------------------------- ### Integrate Exq with Redis using Mock Source: https://context7.com/akira/exq/llms.txt Illustrates setting Exq's mock mode to 'redis' for integration testing. This enqueues jobs to a real Redis instance, allowing for testing the interaction with the Redis backend. ```elixir test "integration test with redis" do Exq.Mock.set_mode(:redis) # Actually enqueues to Redis {:ok, jid} = Exq.enqueue(Exq, "default", MyApp.Workers.EmailWorker, [123, "test"]) # Job will be processed by running Exq workers end ``` -------------------------------- ### Get Exq Job Statistics Source: https://context7.com/akira/exq/llms.txt Retrieve statistics for processed jobs on a specific date or real-time statistics. These functions query the Exq API to provide counts of jobs processed. ```elixir Exq.Api.stats(Exq.Api, "processed", ~D[2024-01-15]) Exq.Api.realtime_stats(Exq.Api) ``` -------------------------------- ### Enqueue and Test Email Job with Exq Mock Source: https://context7.com/akira/exq/llms.txt Demonstrates enqueuing a welcome email job using Exq's mock mode and asserting job details. This is useful for unit testing job enqueuing without actual Redis interaction. ```elixir defmodule MyApp.Workers.EmailWorkerTest do use ExUnit.Case, async: true setup do Exq.Mock.set_mode(:fake) :ok end test "enqueues welcome email job" do user_id = 123 {:ok, jid} = Exq.enqueue(Exq, "default", MyApp.Workers.EmailWorker, [user_id, "welcome"]) jobs = Exq.Mock.jobs() assert length(jobs) == 1 [job] = jobs assert job.class == MyApp.Workers.EmailWorker assert job.args == [user_id, "welcome"] assert job.queue == "default" end end ``` -------------------------------- ### Enqueue Jobs with Exq Source: https://github.com/akira/exq/blob/master/README.md Demonstrates how to enqueue jobs to Exq. Covers basic enqueuing with arguments, specifying worker modules by name, and overriding max retries. Jobs are processed by workers. ```elixir {:ok, ack} = Exq.enqueue(Exq, "default", MyWorker, ["arg1", "arg2"]) {:ok, ack} = Exq.enqueue(Exq, "default", "MyWorker", ["arg1", "arg2"]) ## Don't retry job in per worker {:ok, ack} = Exq.enqueue(Exq, "default", MyWorker, ["arg1", "arg2"], max_retries: 0) ## max_retries = 10, it will override :max_retries in config {:ok, ack} = Exq.enqueue(Exq, "default", MyWorker, ["arg1", "arg2"], max_retries: 10) ``` -------------------------------- ### Snooze a Job by Returning {:snooze, seconds} in Elixir Source: https://github.com/akira/exq/blob/master/README.md Provides an example of how a worker can return `{:snooze, time_to_sleep_in_seconds}` from its `perform` method to temporarily pause its execution. This feature requires the `Exq.Middleware.Snooze` to be added to the middleware configuration. ```elixir defmodule MyWorker do def perform do {:snooze, 10} end end ``` -------------------------------- ### Schedule Jobs Atomically with Exq in Elixir Source: https://github.com/akira/exq/blob/master/README.md Shows how to use `enqueue_all` to schedule multiple jobs with different timing options. It demonstrates scheduling jobs immediately, after a delay (`:in`), and at a specific time (`:at`), ensuring atomicity across all scheduled jobs. ```elixir {:ok, [{:ok, jid_1}, {:ok, jid_2}, {:ok, jid_3}]} = Exq.enqueue_all(Exq, [ [job_1_queue, job_1_worker, job_1_args, [schedule: {:in, 60 * 60}]], [job_2_queue, job_2_worker, job_2_args, [schedule: {:at, midnight}]], [job_3_queue, job_3_worker, job_3_args, []] # no schedule key is present, it is enqueued immediately ]) ``` -------------------------------- ### Query Exq Job Statistics via API Source: https://context7.com/akira/exq/llms.txt Utilizes the Exq.Api module to query job statistics, including queue sizes, job counts, and worker statistics. This API can be used independently or with a full Exq worker setup. ```elixir # Start API-only mode (no workers) {:ok, _} = Exq.Api.start_link() # Or with full Exq {:ok, _} = Exq.start_link() # List all queues with jobs {:ok, queues} = Exq.Api.queues(Exq.Api) # Returns: {:ok, ["default", "critical", "reports"]} # Get queue sizes for all queues {:ok, sizes} = Exq.Api.queue_size(Exq.Api) # Returns: {:ok, [{"default", 42}, {"critical", 5}]} # Get size of specific queue {:ok, count} = Exq.Api.queue_size(Exq.Api, "default") # Returns: {:ok, 42} # Get number of busy workers {:ok, busy} = Exq.Api.busy(Exq.Api) # Returns: {:ok, 15} # Get processing statistics {:ok, processed} = Exq.Api.stats(Exq.Api, "processed") {:ok, failed} = Exq.Api.stats(Exq.Api, "failed") ``` -------------------------------- ### Schedule Jobs with Exq Source: https://github.com/akira/exq/blob/master/README.md Illustrates scheduling jobs to run at a future time or a specific timestamp. Requires `scheduler_enable` to be set to true in the configuration. Supports both relative (in X minutes) and absolute times. ```elixir ## Schedule a job to start in 5 mins: {:ok, ack} = Exq.enqueue_in(Exq, "default", 300, MyWorker, ["arg1", "arg2"]) # If using `mode: [:enqueuer]` {:ok, ack} = Exq.Enqueuer.enqueue_in(Exq.Enqueuer, "default", 300, MyWorker, ["arg1", "arg2"]) ## Schedule a job to start at 8am 2015-12-25 UTC: time = Timex.now() |> Timex.shift(days: 8) {:ok, ack} = Exq.enqueue_at(Exq, "default", time, MyWorker, ["arg1", "arg2"]) # If using `mode: [:enqueuer]` {:ok, ack} = Exq.Enqueuer.enqueue_at(Exq.Enqueuer, "default", time, MyWorker, ["arg1", "arg2"]) ``` -------------------------------- ### Running Modes Source: https://context7.com/akira/exq/llms.txt Configure Exq to run in different modes tailored to specific use cases. This includes the default mode (worker, enqueuer, API), enqueuer-only (for web servers), API-only (for monitoring), and combined enqueuer and API modes (without workers). ```APIDOC ## POST /start ### Description Starts the Exq process with a specified mode. ### Method POST ### Endpoint /start ### Parameters #### Request Body - **mode** (string or array) - Required - The running mode(s) for Exq. Can be a single string (e.g., `:default`, `:enqueuer`, `:api`) or an array of strings (e.g., `[:enqueuer, :api]`). - **name** (string) - Optional - A name to register the Exq process under. ### Request Example ```json { "mode": ":default" } ``` ### Response #### Success Response (200) - **message** (string) - Indicates successful startup. #### Response Example ```json { "message": "Exq started successfully." } ``` ## POST /enqueue ### Description Enqueues a job when Exq is running in `:enqueuer` or combined modes. ### Method POST ### Endpoint /enqueue ### Parameters #### Request Body - **name** (string) - Required - The registered name of the Exq enqueuer process. - **queue** (string) - Required - The name of the queue to add the job to. - **worker** (string) - Required - The module name of the worker to execute. - **args** (array) - Required - Arguments to pass to the worker. ### Request Example ```json { "name": "ExqEnqueuer", "queue": "default", "worker": "MyWorker", "args": [] } ``` ### Response #### Success Response (200) - **message** (string) - Indicates the job was enqueued successfully. #### Response Example ```json { "message": "Job enqueued successfully." } ``` ## GET /queues ### Description Retrieves a list of all queues when Exq is running in `:api` or combined modes. ### Method GET ### Endpoint /queues ### Parameters #### Query Parameters - **name** (string) - Required - The registered name of the Exq API process. ### Response #### Success Response (200) - **queues** (array) - A list of queue names. #### Response Example ```json { "queues": ["default", "critical", "high"] } ``` ``` -------------------------------- ### Enqueue Unique Jobs for Deduplication Source: https://context7.com/akira/exq/llms.txt Enqueues jobs with unique constraints to prevent duplicates. This feature supports configurable lock durations and release conditions, including unlocking on job start or expiry, and custom unique tokens for cross-queue uniqueness. ```elixir # Basic unique job - prevents duplicates for 1 hour {:ok, jid} = Exq.enqueue(Exq, "default", MyApp.Workers.SyncWorker, [user_id], unique_for: 3600) # Attempting to enqueue duplicate returns conflict {:conflict, existing_jid} = Exq.enqueue(Exq, "default", MyApp.Workers.SyncWorker, [user_id], unique_for: 3600) # Custom unique token (for cross-queue uniqueness) {:ok, jid} = Exq.enqueue(Exq, "queue1", MyApp.Workers.SyncWorker, [data], unique_for: 3600, unique_token: "sync-user-#{user_id}" ) {:conflict, ^jid} = Exq.enqueue(Exq, "queue2", MyApp.Workers.SyncWorker, [data], unique_for: 3600, unique_token: "sync-user-#{user_id}" ) # Unlock on job start (debounce pattern) {:ok, jid} = Exq.enqueue(Exq, "default", MyApp.Workers.IndexWorker, [doc_id], unique_for: 300, unique_until: :start ) # Keep lock until expiry (idempotency pattern) {:ok, jid} = Exq.enqueue(Exq, "default", MyApp.Workers.WelcomeEmail, [user_id], unique_for: 86400, # 24 hours unique_until: :expiry ) # Keep lock until success (batch pattern) {:ok, jid} = Exq.enqueue_in(Exq, "default", 3600, MyApp.Workers.BatchNotifier, [user_id], unique_for: 7200, unique_until: :success ) ``` -------------------------------- ### Enqueue and Query Specific Exq Instances Source: https://context7.com/akira/exq/llms.txt Demonstrates how to enqueue jobs to and query the status of specific named Exq instances. This is essential when running multiple Exq processes with different configurations. ```elixir # Enqueue to specific instance {:ok, _} = Exq.enqueue(CriticalExq, "critical", PaymentWorker, [order_id]) {:ok, _} = Exq.enqueue(BulkExq, "bulk", ReportWorker, [report_id]) # Query specific instance {:ok, size} = Exq.Api.queue_size(CriticalExq.Api, "critical") {:ok, processes} = Exq.Api.processes(BulkExq.Api) ``` -------------------------------- ### Create Exq Worker Module Source: https://github.com/akira/exq/blob/master/README.md Defines a basic Elixir module that can be used as an Exq worker. The `perform` function is the entry point for job execution. It can accept arguments that are passed during job enqueuing. ```elixir defmodule MyWorker do def perform do end end ## Enqueue a job to this worker: {:ok, jid} = Exq.enqueue(Exq, "default", MyWorker, []) ## Example with arguments: {:ok, jid} = Exq.enqueue(Exq, "default", "MyWorker", [arg1, arg2]) ## Matching perform method signature: defmodule MyWorker do def perform(arg1, arg2) do end end ``` -------------------------------- ### Manage Exq Retry Queue Source: https://context7.com/akira/exq/llms.txt Functions for managing jobs in the retry queue, which holds jobs that failed and are waiting for another attempt. Supports listing jobs with pagination and scores, getting queue size, finding specific jobs, removing them, re-enqueuing them, and clearing the entire retry queue. ```elixir # List jobs in retry queue {:ok, retries} = Exq.Api.retries(Exq.Api) # With pagination and score {:ok, retries} = Exq.Api.retries(Exq.Api, size: 50, offset: 0, score: true) # Get retry queue size {:ok, count} = Exq.Api.retry_size(Exq.Api) # Find specific job in retry queue {:ok, job} = Exq.Api.find_retry(Exq.Api, score, jid) # Remove jobs from retry queue :ok = Exq.Api.remove_retry_jobs(Exq.Api, raw_jobs) # Re-enqueue retry jobs immediately {:ok, num_enqueued} = Exq.Api.dequeue_retry_jobs(Exq.Api, raw_jobs) # Clear all retries :ok = Exq.Api.clear_retries(Exq.Api) ``` -------------------------------- ### Manage Exq Dead (Failed) Jobs Source: https://context7.com/akira/exq/llms.txt Functions for managing jobs that have exceeded their retry count and are considered dead or failed. This includes listing failed jobs with pagination and scores, getting the failed queue size, finding specific jobs, removing them, re-enqueuing them for another retry, and clearing all failed jobs. ```elixir # List dead/failed jobs {:ok, failed} = Exq.Api.failed(Exq.Api) # With pagination {:ok, failed} = Exq.Api.failed(Exq.Api, size: 100, offset: 0, score: true) # Get failed queue size {:ok, count} = Exq.Api.failed_size(Exq.Api) # Find specific failed job {:ok, job} = Exq.Api.find_failed(Exq.Api, score, jid) # Remove failed jobs :ok = Exq.Api.remove_failed_jobs(Exq.Api, raw_jobs) # Re-enqueue failed jobs (retry them) {:ok, num_enqueued} = Exq.Api.dequeue_failed_jobs(Exq.Api, raw_jobs) # Clear all failed jobs :ok = Exq.Api.clear_failed(Exq.Api) ``` -------------------------------- ### API: Retry Queue Management Source: https://context7.com/akira/exq/llms.txt Manage jobs that have failed and are waiting to be retried. This API allows you to list jobs in the retry queue, with options for pagination and including their scores. You can also get the size of the retry queue, find specific jobs by score and JID, remove jobs from the retry queue, re-enqueue them immediately, and clear all retries. ```APIDOC ## GET /retries ### Description Lists all jobs in the retry queue. ### Method GET ### Endpoint /retries ### Parameters #### Query Parameters - **size** (integer) - Optional - The number of jobs to return per page. - **offset** (integer) - Optional - The starting offset for pagination. - **score** (boolean) - Optional - If true, includes the score (retry timestamp) for each job. ### Response #### Success Response (200) - **retry_jobs** (array) - A list of jobs in the retry queue. #### Response Example ```json [ { "jid": "job-jid-1", ..., "retry_at": 1678886400 }, { "jid": "job-jid-2", ..., "retry_at": 1678886460 } ] ``` ## GET /retries/size ### Description Gets the total count of jobs in the retry queue. ### Method GET ### Endpoint /retries/size ### Response #### Success Response (200) - **count** (integer) - The number of jobs in the retry queue. #### Response Example ```json { "count": 50 } ``` ## GET /retries/{score}/{jid} ### Description Finds a specific job in the retry queue using its score (retry time) and JID. ### Method GET ### Endpoint /retries/{score}/{jid} ### Parameters #### Path Parameters - **score** (integer) - Required - The retry time (Unix timestamp) of the job. - **jid** (string) - Required - The Job ID of the job to find. ### Response #### Success Response (200) - **job** (object) - The found retry job object. #### Response Example ```json { "jid": "job-jid-here", "retry_at": 1678886400, ... } ``` ## DELETE /retries ### Description Removes specified jobs from the retry queue. ### Method DELETE ### Endpoint /retries ### Request Body - **jobs** (array) - Required - A list of job objects to remove. ### Request Example ```json [ { "jid": "job-jid-1", ... }, { "jid": "job-jid-2", ... } ] ``` ### Response #### Success Response (200) - **message** (string) - Indicates successful removal. #### Response Example ```json { "message": "Retry jobs removed successfully." } ``` ## POST /retries/dequeue ### Description Re-enqueues specified jobs from the retry queue immediately. ### Method POST ### Endpoint /retries/dequeue ### Request Body - **jobs** (array) - Required - A list of job objects to dequeue. ### Request Example ```json [ { "jid": "job-jid-1", ... }, { "jid": "job-jid-2", ... } ] ``` ### Response #### Success Response (200) - **num_enqueued** (integer) - The number of jobs successfully re-enqueued. #### Response Example ```json { "num_enqueued": 2 } ``` ## DELETE /retries/clear ### Description Clears all jobs from the retry queue. ### Method DELETE ### Endpoint /retries/clear ### Response #### Success Response (200) - **message** (string) - Indicates successful clearing of the retry queue. #### Response Example ```json { "message": "Retry queue cleared successfully." } ``` ``` -------------------------------- ### Queue Subscription Management Source: https://context7.com/akira/exq/llms.txt Dynamically subscribe to and unsubscribe from queues at runtime to manage job processing. ```APIDOC ## Queue Management API ### Description APIs for managing queue subscriptions, allowing dynamic control over which queues Exq processes. ### Endpoints #### GET /subscriptions ##### Description Retrieves a list of all currently subscribed queues. ##### Response - **queues** (list) - A list of queue names. #### POST /subscribe ##### Description Subscribes to a new queue or updates the concurrency for an existing one. ##### Parameters - **queue** (string) - Required - The name of the queue to subscribe to. - **concurrency** (integer) - Optional - The maximum number of concurrent jobs for this queue. Defaults to the global concurrency setting. #### POST /unsubscribe ##### Description Unsubscribes from a specific queue, stopping job processing for it. ##### Parameters - **queue** (string) - Required - The name of the queue to unsubscribe from. #### POST /unsubscribe/all ##### Description Unsubscribes from all queues, effectively pausing all job processing. ### Response Examples #### GET /subscriptions Response ```json { "queues": ["default", "critical", "reports"] } ``` #### POST /subscribe Response ```json { "status": "ok" } ``` #### POST /unsubscribe Response ```json { "status": "ok" } ``` #### POST /unsubscribe/all Response ```json { "status": "ok" } ``` ``` -------------------------------- ### API: Dead Jobs (Failed) Management Source: https://context7.com/akira/exq/llms.txt Manage jobs that have failed and exceeded their retry count. This API allows you to list dead/failed jobs, with options for pagination and including their scores. You can get the size of the failed queue, find specific failed jobs by score and JID, remove failed jobs, re-enqueue them (retry them), and clear all failed jobs. ```APIDOC ## GET /failed ### Description Lists all dead/failed jobs. ### Method GET ### Endpoint /failed ### Parameters #### Query Parameters - **size** (integer) - Optional - The number of jobs to return per page. - **offset** (integer) - Optional - The starting offset for pagination. - **score** (boolean) - Optional - If true, includes the score (failure timestamp) for each job. ### Response #### Success Response (200) - **failed_jobs** (array) - A list of dead/failed jobs. #### Response Example ```json [ { "jid": "job-jid-1", ..., "failed_at": 1678886400 }, { "jid": "job-jid-2", ..., "failed_at": 1678886460 } ] ``` ## GET /failed/size ### Description Gets the total count of dead/failed jobs. ### Method GET ### Endpoint /failed/size ### Response #### Success Response (200) - **count** (integer) - The number of dead/failed jobs. #### Response Example ```json { "count": 25 } ``` ## GET /failed/{score}/{jid} ### Description Finds a specific failed job using its score (failure time) and JID. ### Method GET ### Endpoint /failed/{score}/{jid} ### Parameters #### Path Parameters - **score** (integer) - Required - The failure time (Unix timestamp) of the job. - **jid** (string) - Required - The Job ID of the job to find. ### Response #### Success Response (200) - **job** (object) - The found failed job object. #### Response Example ```json { "jid": "job-jid-here", "failed_at": 1678886400, ... } ``` ## DELETE /failed ### Description Removes specified jobs from the failed queue. ### Method DELETE ### Endpoint /failed ### Request Body - **jobs** (array) - Required - A list of job objects to remove. ### Request Example ```json [ { "jid": "job-jid-1", ... }, { "jid": "job-jid-2", ... } ] ``` ### Response #### Success Response (200) - **message** (string) - Indicates successful removal. #### Response Example ```json { "message": "Failed jobs removed successfully." } ``` ## POST /failed/dequeue ### Description Re-enqueues specified failed jobs immediately, allowing them to be retried. ### Method POST ### Endpoint /failed/dequeue ### Request Body - **jobs** (array) - Required - A list of job objects to dequeue. ### Request Example ```json [ { "jid": "job-jid-1", ... }, { "jid": "job-jid-2", ... } ] ``` ### Response #### Success Response (200) - **num_enqueued** (integer) - The number of jobs successfully re-enqueued. #### Response Example ```json { "num_enqueued": 2 } ``` ## DELETE /failed/clear ### Description Clears all jobs from the failed queue. ### Method DELETE ### Endpoint /failed/clear ### Response #### Success Response (200) - **message** (string) - Indicates successful clearing of the failed queue. #### Response Example ```json { "message": "Failed queue cleared successfully." } ``` ```