### Starting Honeydew Queue and Workers Source: https://github.com/koudelka/honeydew/blob/master/examples/initialized_worker/README.md Start the Honeydew queue and workers using `start_queue/1` and `start_workers/3`. The second argument to `start_workers/3` is a tuple containing the worker module and its initialization arguments. ```elixir defmodule App do def start do Honeydew.start_queue(:my_queue) Honeydew.start_workers(:my_queue, {Worker, ['127.0.0.1', 8087]}) end end ``` -------------------------------- ### Start Global Honeydew Queue Source: https://github.com/koudelka/honeydew/blob/master/examples/global/README.md Initiates a global Honeydew queue on a specific node. Use `{:global, name}` to make the queue accessible cluster-wide. This example uses the Mnesia queue. ```elixir defmodule QueueApp do def start do :ok = Honeydew.start_queue({:global, :my_queue}, queue: {Honeydew.Queue.Mnesia, [disc_copies: nodes]}) end end iex(queue@dax)1> QueueApp.start :ok ``` -------------------------------- ### Start a Honeydew Queue Source: https://context7.com/koudelka/honeydew/llms.txt Starts a queue process under Honeydew's supervision tree. Accepts an atom name for local queues or a {:global, name} tuple for cluster-wide queues. Various queue backends, failure modes, success modes, and dispatchers can be configured. ```elixir :ok = Honeydew.start_queue(:my_queue) ``` ```elixir :ok = Honeydew.start_queue(:my_queue, queue: {Honeydew.Queue.Mnesia, [disc_copies: [node()]]} ) ``` ```elixir :ok = Honeydew.start_queue(:my_queue, queue: Honeydew.Queue.ErlangQueue, failure_mode: {Honeydew.FailureMode.Retry, [times: 3, finally: {Honeydew.FailureMode.Move, [queue: :dead_letters]}]}, success_mode: Honeydew.SuccessMode.Log, dispatcher: Honeydew.Dispatcher.MRU ) ``` ```elixir :ok = Honeydew.start_queue({:global, :my_queue}, queue: {Honeydew.Queue.Mnesia, [disc_copies: [node()]]} ) ``` -------------------------------- ### Start Honeydew Queue and Workers Source: https://github.com/koudelka/honeydew/blob/master/examples/local/README.md Starts the Honeydew queue and registers workers within the application's supervision tree. Ensure Honeydew is configured in your application's supervision tree. ```elixir defmodule App do def start do :ok = Honeydew.start_queue(:my_queue) :ok = Honeydew.start_workers(:my_queue, Worker) end end ``` -------------------------------- ### Start Honeydew Queue and Workers Source: https://github.com/koudelka/honeydew/blob/master/README.md Defines a worker module and starts a Honeydew queue and associated workers. Enqueues a job asynchronously. ```elixir defmodule MyWorker do def do_a_thing do IO.puts "doing a thing!" end end :ok = Honeydew.start_queue(:my_queue) :ok = Honeydew.start_workers(:my_queue, MyWorker) :do_a_thing |> Honeydew.async(:my_queue) # => "doing a thing!" ``` -------------------------------- ### Start Honeydew Workers for Global Queue Source: https://github.com/koudelka/honeydew/blob/master/examples/global/README.md Starts a pool of workers that connect to a global queue. Specify the number of workers and the nodes where they should run. Honeydew attempts to maintain connections if nodes go down. ```elixir defmodule WorkerApp do def start do :ok = Honeydew.start_workers({:global, :my_queue}, HeavyTask, num: 10, nodes: [:clientfacing@dax, :queue@dax]) end end iex(background@dax)1> WorkerApp.start :ok ``` -------------------------------- ### Honeydew.start_queue/2 Source: https://context7.com/koudelka/honeydew/llms.txt Starts a queue process under Honeydew's supervision tree. It can be a local or cluster-wide queue and supports various backend queues, failure modes, success modes, and dispatchers. ```APIDOC ## Honeydew.start_queue/2 — Start a Queue Starts a queue process under Honeydew's supervision tree. Accepts an atom name for local queues or a `{:global, name}` tuple for cluster-wide queues. The `queue:` option selects the backend, `failure_mode:` configures what happens on worker crash, `success_mode:` registers a callback on job success, and `dispatcher:` selects the job dispatch strategy. ```elixir # Default in-memory Mnesia queue (simplest case) :ok = Honeydew.start_queue(:my_queue) # Mnesia queue with disk persistence :ok = Honeydew.start_queue(:my_queue, queue: {Honeydew.Queue.Mnesia, [disc_copies: [node()]]} ) # Fast in-memory ErlangQueue with Retry failure mode and Move fallback :ok = Honeydew.start_queue(:my_queue, queue: Honeydew.Queue.ErlangQueue, failure_mode: {Honeydew.FailureMode.Retry, [times: 3, finally: {Honeydew.FailureMode.Move, [queue: :dead_letters]}]}, success_mode: Honeydew.SuccessMode.Log, dispatcher: Honeydew.Dispatcher.MRU ) # Global (cluster-wide) queue :ok = Honeydew.start_queue({:global, :my_queue}, queue: {Honeydew.Queue.Mnesia, [disc_copies: [node()]]} ) ``` ``` -------------------------------- ### Start Honeydew Workers Source: https://context7.com/koudelka/honeydew/llms.txt Starts a worker pool for a named queue. The second argument can be a bare module (stateless) or a {Module, args} tuple (stateful). The 'num:' option sets the pool size, and 'nodes:' can specify nodes for cluster healing. ```elixir :ok = Honeydew.start_workers(:my_queue, MyWorker, num: 5) ``` ```elixir :ok = Honeydew.start_workers(:my_queue, {MyWorker, ["db.host", 5432]}, num: 10) ``` ```elixir :ok = Honeydew.start_workers({:global, :my_queue}, HeavyTask, num: 10, nodes: [:clientfacing@dax, :queue@dax] ) ``` -------------------------------- ### Configure CockroachDB Database Type Source: https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/README.md Specify `:cockroachdb` as the database type when starting the queue if you are using CockroachDB. ```elixir :ok = Honeydew.start_queue(:classify_photos, queue: {EctoPollQueue, [schema: Photo, repo: Repo, database: :cockroachdb]}) ``` -------------------------------- ### Start Global Distributed Queue Node Source: https://context7.com/koudelka/honeydew/llms.txt Configure a node to host a global Honeydew queue using `{:global, name}`. The `queue: {Honeydew.Queue.Mnesia, [disc_copies: nodes]}` option specifies Mnesia for distributed storage. ```elixir # On queue node (queue@dax) defmodule QueueApp do def start do nodes = [node(), :"worker@dax"] :ok = Honeydew.start_queue({:global, :transcoding}, queue: {Honeydew.Queue.Mnesia, [disc_copies: nodes]} ) end end ``` -------------------------------- ### Configure Application with Honeydew Queues and Workers Source: https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/README.md Start Honeydew queues and workers within your Elixir application's supervisor. ```elixir defmodule MyApp.Application do use Application alias Honeydew.EctoPollQueue alias EctoPollQueue.Repo alias EctoPollQueue.Photo alias EctoPollQueue.ClassifyPhoto def start(_type, _args) do children = [Repo] opts = [strategy: :one_for_one, name: MyApp.Supervisor] {:ok, supervisor} = Supervisor.start_link(children, opts) :ok = Honeydew.start_queue(:classify_photos, queue: {EctoPollQueue, [schema: Photo, repo: Repo]}) :ok = Honeydew.start_workers(:classify_photos, ClassifyPhoto) {:ok, supervisor} end end ``` -------------------------------- ### Start Workers for Global Distributed Queue Source: https://context7.com/koudelka/honeydew/llms.txt Start workers for a global queue on a specific node. The `nodes:` option is crucial for cluster healing, ensuring workers reconnect after node disconnections. ```elixir defmodule WorkerApp do def start do :ok = Honeydew.start_workers({:global, :transcoding}, TranscodeWorker, num: 8, nodes: [:"clientfacing@dax", :"queue@dax"] # for cluster healing ) end end ``` -------------------------------- ### Honeydew.start_workers/3 Source: https://context7.com/koudelka/honeydew/llms.txt Starts a worker pool for a named queue. Workers can be stateless (bare module) or stateful ({Module, args}). The `num:` option sets the pool size, and `nodes:` can specify nodes for cluster healing. ```APIDOC ## Honeydew.start_workers/3 — Start Workers Starts a worker pool for a named queue. The second argument is either a bare module (stateless workers) or a `{Module, args}` tuple (stateful workers, where `args` are passed to `init/1`). The `num:` option sets the pool size. ```elixir # Stateless worker pool with 5 workers :ok = Honeydew.start_workers(:my_queue, MyWorker, num: 5) # Stateful workers initialized with a database connection :ok = Honeydew.start_workers(:my_queue, {MyWorker, ["db.host", 5432]}, num: 10) # Workers on a global queue watching specific nodes for cluster healing :ok = Honeydew.start_workers({:global, :my_queue}, HeavyTask, num: 10, nodes: [:clientfacing@dax, :queue@dax] ) ``` ``` -------------------------------- ### Filter Job Execution with `:run_if` Option Source: https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/README.md Use the `:run_if` option when starting a queue to specify a SQL fragment that determines if a job should be executed. ```elixir # run if the User's name is NULL or the name isn't "dont run me" :ok = Honeydew.start_queue(:classify_photos, queue: {EctoPollQueue, [schema: User, repo: Repo, run_if: ~s{NAME IS NULL OR NAME != 'dont run me'}]}) ``` -------------------------------- ### Move Job Source: https://github.com/koudelka/honeydew/blob/master/README/api.md Moves a job from one queue to another, provided it has not yet been started. ```APIDOC ## Move Job ### Description Move a job from one queue to another if it has not been started yet. ### Functions - `Honeydew.move/2`: Moves a job between queues. ``` -------------------------------- ### Honeydew.filter/2 and Honeydew.cancel/2 Source: https://context7.com/koudelka/honeydew/llms.txt Allows searching for jobs within a queue using a pattern and canceling jobs that have not yet started. `filter/2` returns a list of matching `%Honeydew.Job{}` structs, and `cancel/2` returns `:ok` or `{:error, :in_progress}`. ```APIDOC ## `Honeydew.filter/2` and `Honeydew.cancel/2` — Find and Cancel Jobs `filter/2` searches a queue for jobs matching a pattern (a map with fields like `:task`). Returns a list of matching `%Honeydew.Job{}` structs. `cancel/2` removes a job from the queue if it has not yet started; returns `:ok` or `{:error, :in_progress}`. ```elixir :ok = Honeydew.start_queue(:work_queue) :ok = Honeydew.start_workers(:work_queue, MyWorker, num: 2) # Enqueue 10 jobs Enum.each(1..10, fn i -> {:process, [i]} |> Honeydew.async(:work_queue) end) # Find jobs with a specific argument jobs = Honeydew.filter(:work_queue, %{task: {:process, [7]}}) # Cancel the matching job case List.first(jobs) do nil -> IO.puts("Job not found (may have already run)") job -> case Honeydew.cancel(job) do :ok -> IO.puts("Job cancelled successfully") {:error, :in_progress} -> IO.puts("Job already running, cannot cancel") end end # Verify updated queue count Honeydew.status(:work_queue) |> Map.get(:queue) |> IO.inspect # => %{count: 8, in_progress: 1} ``` ``` -------------------------------- ### Get Queue and Worker Status Source: https://context7.com/koudelka/honeydew/llms.txt Use `Honeydew.status/1` to retrieve a map containing queue statistics (pending/in-progress jobs) and worker PIDs with their current job status. Useful for monitoring. ```elixir import Honeydew.Progress defmodule HeavyTask do import Honeydew.Progress @behaviour Honeydew.Worker def crunch(secs) do progress("Starting heavy computation...") Enum.each(1..secs, fn i -> Process.sleep(1_000) progress("Completed step #{i}/#{secs}") end) :done end end :ok = Honeydew.start_queue(:crunch_queue) :ok = Honeydew.start_workers(:crunch_queue, HeavyTask, num: 4) {:crunch, [10]} |> Honeydew.async(:crunch_queue) Process.sleep(500) status = Honeydew.status(:crunch_queue) # Inspect queue stats IO.inspect(status[:queue]) # => %{count: 0, in_progress: 1} # Inspect worker activity Enum.each(status[:workers], fn {pid, nil} -> IO.puts("#{inspect pid} -> idle") {pid, {_job, progress}} -> IO.puts("#{inspect pid} -> #{inspect progress}") end) # => #PID<0.123.0> -> "Starting heavy computation..." ``` -------------------------------- ### Move Job Between Queues Source: https://context7.com/koudelka/honeydew/llms.txt Use `Honeydew.move/2` to transfer a pending job to a different queue. Returns `{:error, :in_progress}` if the job has already started. ```elixir :ok = Honeydew.start_queue(:high_priority) :ok = Honeydew.start_queue(:low_priority) :ok = Honeydew.start_workers(:high_priority, MyWorker) :ok = Honeydew.start_workers(:low_priority, MyWorker) job = {:process, ["important_item"]} |> Honeydew.async(:low_priority) # Promote to high priority queue case Honeydew.move(job, :high_priority) do :ok -> IO.puts("Job promoted to high priority") {:error, :in_progress} -> IO.puts("Job already executing") end ``` -------------------------------- ### Find and Cancel Jobs Source: https://context7.com/koudelka/honeydew/llms.txt Use `Honeydew.filter/2` to find jobs matching a pattern and `Honeydew.cancel/2` to remove a pending job. `cancel/2` returns `{:error, :in_progress}` if the job has already started. ```elixir :ok = Honeydew.start_queue(:work_queue) :ok = Honeydew.start_workers(:work_queue, MyWorker, num: 2) # Enqueue 10 jobs Enum.each(1..10, fn i -> {:process, [i]} |> Honeydew.async(:work_queue) end) # Find jobs with a specific argument jobs = Honeydew.filter(:work_queue, %{task: {:process, [7]}}) # Cancel the matching job case List.first(jobs) do nil -> IO.puts("Job not found (may have already run)") job -> case Honeydew.cancel(job) do :ok -> IO.puts("Job cancelled successfully") {:error, :in_progress} -> IO.puts("Job already running, cannot cancel") end end # Verify updated queue count Honeydew.status(:work_queue) |> Map.get(:queue) |> IO.inspect # => %{count: 8, in_progress: 1} ``` -------------------------------- ### Cancel Job Source: https://github.com/koudelka/honeydew/blob/master/README/api.md Cancels a job that has not yet started processing. ```APIDOC ## Cancel Job ### Description Cancel a job that has not yet run. ### Functions - `Honeydew.cancel/1` - `Honeydew.cancel/2` ``` -------------------------------- ### Ecto Schema for Poll Queue Source: https://context7.com/koudelka/honeydew/llms.txt Define an Ecto schema with `honeydew_fields` to enable Honeydew to poll the database for new jobs. This requires corresponding migration setup. ```elixir defmodule MyApp.Photo do use Ecto.Schema import Honeydew.EctoPollQueue.Schema schema "photos" do field :tag, :string honeydew_fields(:classify_photos) end end ``` -------------------------------- ### Honeydew.move/2 Source: https://context7.com/koudelka/honeydew/llms.txt Moves a pending job from one queue to another. This operation is useful for reprioritizing jobs. It returns `:ok` on success or `{:error, :in_progress}` if the job has already started processing. ```APIDOC ## `Honeydew.move/2` — Move a Job Between Queues Moves a pending (not-yet-started) job from one queue to another. Returns `:ok` on success or `{:error, :in_progress}` if the job has already been picked up by a worker. ```elixir :ok = Honeydew.start_queue(:high_priority) :ok = Honeydew.start_queue(:low_priority) :ok = Honeydew.start_workers(:high_priority, MyWorker) :ok = Honeydew.start_workers(:low_priority, MyWorker) job = {:process, ["important_item"]} |> Honeydew.async(:low_priority) # Promote to high priority queue case Honeydew.move(job, :high_priority) do :ok -> IO.puts("Job promoted to high priority") {:error, :in_progress} -> IO.puts("Job already executing") end ``` ``` -------------------------------- ### Application Startup for Ecto Poll Queue Source: https://context7.com/koudelka/honeydew/llms.txt Configure the application's `start/2` function to initialize the Ecto Poll Queue and its workers. Specify schema, repo, poll interval, and the job module. ```elixir defmodule MyApp.Application do use Application alias Honeydew.EctoPollQueue def start(_type, _args) do children = [MyApp.Repo] {:ok, sup} = Supervisor.start_link(children, strategy: :one_for_one) :ok = Honeydew.start_queue(:classify_photos, queue: {EctoPollQueue, [ schema: MyApp.Photo, repo: MyApp.Repo, poll_interval: 5 ]} # seconds ) :ok = Honeydew.start_workers(:classify_photos, MyApp.ClassifyPhoto) {:ok, sup} end end ``` -------------------------------- ### Create a Job Module Source: https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/README.md Define the `run/1` function that Honeydew will execute for each job. It receives the primary key of the inserted row. ```elixir defmodule MyApp.ClassifyPhoto do alias MyApp.Photo alias MyApp.Repo # By default, Honeydew will call the `run/1` function with the primary key of your newly inserted row. # # If your table uses compound keys, a keyword list suitable for passing to `Repo.get_by/2` will be given # as an arguement. For exaxmple: `[first_name: "Darwin", last_name: "Shapiro"]` def run(id) do photo = Repo.get(Photo, id) tag = Enum.random(["newt", "ripley", "jonesey", "xenomorph"]) IO.puts "Photo contained a #{tag}!" photo |> Ecto.Changeset.change(%{tag: tag}) |> Repo.update!() end end ``` -------------------------------- ### Worker Module with Initialization Source: https://github.com/koudelka/honeydew/blob/master/examples/initialized_worker/README.md Implement the `init/1` callback to establish initial state for your worker. The `{:ok, state}` return value is stored and passed to subsequent tasks. Honeydew retries `init/1` if it fails. ```elixir defmodule Worker do def init([ip, port]) do {:ok, db} = Database.connect(ip, port) {:ok, db} end def send_email(id, db) do %{name: name, email: email} = Database.find(id, db) IO.puts "sending email to #{email}" IO.inspect "hello #{name}, want to enlarge ur keyboard by 500%??" end end ``` -------------------------------- ### Re-initialize Worker on Failed Init Source: https://github.com/koudelka/honeydew/blob/master/README/workers.md Implement `failed_init/0` to call `Honeydew.reinitialize_worker/0` when the `init/1` callback fails. This allows the worker to attempt re-initialization. ```elixir defmodule FailedInitWorker do def init(_) do raise "init failed" end def failed_init do Honeydew.reinitialize_worker() end end ``` -------------------------------- ### Enqueue Job with Delay Source: https://context7.com/koudelka/honeydew/llms.txt Use `Honeydew.async/3` with the `delay_secs` option to schedule a job for future execution. Requires `iex --erl "+C multi_time_warp"`. ```elixir {:send_reminder, [user_id]} |> Honeydew.async(:email_queue, delay_secs: 3600) ``` -------------------------------- ### Configure Honeydew Failure Modes Source: https://context7.com/koudelka/honeydew/llms.txt Configure failure modes on `start_queue/2` to determine job handling upon worker crashes. Modes include Abandon, Retry, Move, and ExponentialRetry, which can be chained with `finally:`. ```elixir alias Honeydew.FailureMode.{Abandon, Move, Retry, ExponentialRetry} # Simple retry 3 times, then abandon :ok = Honeydew.start_queue(:q1, failure_mode: {Retry, [times: 3]}) # Retry 3 times, then move to a dead-letter queue :ok = Honeydew.start_queue(:q2, failure_mode: {Retry, [times: 3, finally: {Move, [queue: :dead_letters]}]} ) # Exponential backoff: 0s, 1s, 3s, 7s, 15s delays (base^n - 1), then abandon :ok = Honeydew.start_queue(:q3, failure_mode: {ExponentialRetry, [times: 5, base: 2]} ) # Exponential retry with a dead-letter queue fallback :ok = Honeydew.start_queue(:q4, failure_mode: { ExponentialRetry, [times: 4, base: 3, finally: {Move, [queue: :failed_jobs]}] } ) # Corresponding workers :ok = Honeydew.start_workers(:q3, MyWorker) Logger.configure(level: :error) {:do_work, [DateTime.utc_now()]} |> Honeydew.async(:q3, delay_secs: 1) ``` -------------------------------- ### Configure Mnesia Application Inclusion (Honeydew Managed) Source: https://github.com/koudelka/honeydew/blob/master/README.md Configures the application to include :mnesia, allowing Honeydew to manage it when it's the sole user. ```elixir def application do [ included_applications: [:mnesia] ] end ``` -------------------------------- ### Submit a Task to the Queue Source: https://github.com/koudelka/honeydew/blob/master/examples/local/README.md Submits a task to the specified Honeydew queue using `async/3`. The task is represented as a tuple containing the function name and its arguments. The function executes immediately and prints output. ```elixir iex(1)> {:hello, ["World"]} |> Honeydew.async(:my_queue) Hello World! ``` -------------------------------- ### Ecto Migration for Poll Queue Source: https://context7.com/koudelka/honeydew/llms.txt Create a database migration using `honeydew_fields` and `honeydew_indexes` to prepare tables for the Ecto Poll Queue. Supports PostgreSQL and CockroachDB. ```elixir defmodule MyApp.Repo.Migrations.CreatePhotos do use Ecto.Migration import Honeydew.EctoPollQueue.Migration def change do create table(:photos) do add :tag, :string honeydew_fields(:classify_photos) end honeydew_indexes(:photos, :classify_photos) end end ``` -------------------------------- ### Insert Schema Instance to Trigger Job Source: https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/README.md Insert a new record into your schema table to initiate a Honeydew job. ```elixir iex(1)> {:ok, _photo} = %MyApp.Photo{} |> MyApp.Repo.insert #=> "Photo contained a xenomorph!" ``` -------------------------------- ### Add Migration Fields and Indexes Source: https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/README.md Include Honeydew's database columns and indexes in your Ecto migration. ```elixir defmodule MyApp.Repo.Migrations.CreatePhotos do use Ecto.Migration import Honeydew.EctoPollQueue.Migration def change do create table(:photos) do add :tag, :string # You can have as many queues as you'd like, they just need unique names. honeydew_fields(:classify_photos) end honeydew_indexes(:photos, :classify_photos) end end ``` -------------------------------- ### Ecto Job Module Source: https://context7.com/koudelka/honeydew/llms.txt Implement a job module where the `run/1` function receives the primary key of a new row from the Ecto Poll Queue. It fetches the record, processes it, and updates the database. ```elixir defmodule MyApp.ClassifyPhoto do def run(photo_id) do photo = MyApp.Repo.get(MyApp.Photo, photo_id) tag = classify(photo) photo |> Ecto.Changeset.change(%{tag: tag}) |> MyApp.Repo.update!() IO.puts("Photo #{photo_id} classified as: #{tag}") end defp classify(_photo), do: Enum.random(["cat", "dog", "bird"]) end ``` -------------------------------- ### List and Filter Jobs Source: https://github.com/koudelka/honeydew/blob/master/README/api.md Lists all jobs in a queue and allows for optional filtering. ```APIDOC ## List and Filter Jobs ### Description List jobs in a queue and optionally filter the list. ### Functions - `Honeydew.filter/2`: Lists and filters jobs in a queue. ``` -------------------------------- ### Configure Mnesia Application Inclusion Source: https://github.com/koudelka/honeydew/blob/master/README.md Configures the application to include the :mnesia application, necessary when Mnesia is used outside of Honeydew. ```elixir def application do [ extra_applications: [:mnesia] ] end ``` -------------------------------- ### Enqueue Job to Global Queue Source: https://context7.com/koudelka/honeydew/llms.txt Enqueue a job to a global Honeydew queue from any node in the cluster. The job tuple `{:transcode, ["/uploads/video.mp4"]}` is sent to the globally named queue. ```elixir # From any node (clientfacing@dax) {:transcode, ["/uploads/video.mp4"]} |> Honeydew.async({:global, :transcoding}) # => "Transcoding /uploads/video.mp4" ``` -------------------------------- ### Specify CockroachDB in Migration Source: https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/README.md Pass the `:database` option set to `:cockroachdb` to `honeydew_fields/2` in your migration when using CockroachDB. ```elixir honeydew_fields(:classify_photos, database: :cockroachdb) ``` -------------------------------- ### Define a Honeydew Worker Module Source: https://context7.com/koudelka/honeydew/llms.txt A worker module implements @behaviour Honeydew.Worker. Public functions become callable job names. 'init/1' can be implemented for stateful workers, and 'failed_init/0' for custom failure behavior. ```elixir defmodule EmailWorker do @behaviour Honeydew.Worker def init([db_host, db_port]) do {:ok, conn} = MyDB.connect(db_host, db_port) {:ok, conn} # state is loaned to each job end def send_welcome_email(user_id, conn) do user = MyDB.get_user(conn, user_id) MyMailer.send(user.email, "Welcome!") :ok end def failed_init do Honeydew.reinitialize_worker() end end :ok = Honeydew.start_queue(:email_queue) :ok = Honeydew.start_workers(:email_queue, {EmailWorker, ["db.host", 5432]}) {:send_welcome_email, [42]} |> Honeydew.async(:email_queue) ``` -------------------------------- ### Worker Module — Defining Jobs Source: https://context7.com/koudelka/honeydew/llms.txt Worker modules implement `@behaviour Honeydew.Worker`. Public functions become callable job names. `init/1` can be implemented for stateful workers, and `failed_init/0` for custom failure handling. ```APIDOC ## Worker Module — Defining Jobs A worker module implements `@behaviour Honeydew.Worker`. Public functions become callable job names. Optionally implement `init/1` to return `{:ok, state}` for stateful workers — the state is appended as the last argument to every job call. Implement `failed_init/0` to customize behavior when `init/1` fails. ```elixir defmodule EmailWorker do @behaviour Honeydew.Worker # init/1 receives the args passed to start_workers/3 def init([db_host, db_port]) do {:ok, conn} = MyDB.connect(db_host, db_port) {:ok, conn} # state is loaned to each job end # conn is automatically appended by Honeydew def send_welcome_email(user_id, conn) do user = MyDB.get_user(conn, user_id) MyMailer.send(user.email, "Welcome!") :ok end def failed_init do # Optionally re-initialize immediately instead of waiting 5 seconds Honeydew.reinitialize_worker() end end # Start and enqueue :ok = Honeydew.start_queue(:email_queue) :ok = Honeydew.start_workers(:email_queue, {EmailWorker, ["db.host", 5432]}) {:send_welcome_email, [42]} |> Honeydew.async(:email_queue) ``` ``` -------------------------------- ### Add Honeydew Dependency Source: https://github.com/koudelka/honeydew/blob/master/README.md Specifies the Honeydew dependency in the mix.exs file for a project. ```elixir defp deps do [{:honeydew, "~> 1.5.0"}] end ``` -------------------------------- ### Enqueue Job with Anonymous Function Source: https://context7.com/koudelka/honeydew/llms.txt Enqueue a job that executes an anonymous function using `Honeydew.async/2`. ```elixir fn -> IO.puts("hello from a lambda!") end |> Honeydew.async(:my_queue) ``` -------------------------------- ### Honeydew.async/3 Source: https://context7.com/koudelka/honeydew/llms.txt Enqueues a job on the named queue. Jobs can be defined as `{function_name, [args]}` or anonymous functions. Supports `reply: true` for receiving results via `yield/2` and `delay_secs:` for deferred execution. ```APIDOC ## Honeydew.async/3 — Enqueue a Job Enqueues a job on the named queue. The task is either a `{function_name, [args]}` tuple referencing a public function on the worker module, or an anonymous function. Supports `reply: true` to receive the return value via `yield/2`, and `delay_secs:` to schedule deferred execution. ```elixir # Fire-and-forget {:send_welcome_email, [42]} |> Honeydew.async(:email_queue) # With reply — receive result via yield/2 job = {:compute_result, [100]} |> Honeydew.async(:my_queue, reply: true) {:ok, result} = Honeydew.yield(job, 5_000) # wait up to 5 seconds IO.inspect(result) # => 101 ``` ``` -------------------------------- ### Honeydew.status/1 Source: https://context7.com/koudelka/honeydew/llms.txt Returns a map containing the status of a queue and its workers. The `:queue` key provides counts of pending and in-progress jobs, while the `:workers` key maps worker PIDs to their current job status. ```APIDOC ## `Honeydew.status/1` — Queue and Worker Status Returns a map with `:queue` and `:workers` keys. `:queue` contains counts of pending and in-progress jobs. `:workers` maps each worker PID to its current `{job, job_status}` or `nil` if idle. Useful for monitoring dashboards and health checks. ```elixir import Honeydew.Progress defmodule HeavyTask do import Honeydew.Progress @behaviour Honeydew.Worker def crunch(secs) do progress("Starting heavy computation...") Enum.each(1..secs, fn i -> Process.sleep(1_000) progress("Completed step #{i}/#{secs}") end) :done end end :ok = Honeydew.start_queue(:crunch_queue) :ok = Honeydew.start_workers(:crunch_queue, HeavyTask, num: 4) {:crunch, [10]} |> Honeydew.async(:crunch_queue) Process.sleep(500) status = Honeydew.status(:crunch_queue) # Inspect queue stats IO.inspect(status[:queue]) # => %{count: 0, in_progress: 1} # Inspect worker activity Enum.each(status[:workers], fn {pid, nil} -> IO.puts("#{inspect pid} -> idle") {pid, {_job, progress}} -> IO.puts("#{inspect pid} -> #{inspect progress}") end) # => #PID<0.123.0> -> "Starting heavy computation..." ``` ``` -------------------------------- ### Honeydew.suspend/1 and Honeydew.resume/1 Source: https://context7.com/koudelka/honeydew/llms.txt Provides mechanisms to halt and re-enable job dispatch to workers. `suspend/1` stops new jobs from being processed, and `resume/1` restarts the dispatch process, useful for maintenance or controlled draining. ```APIDOC ## `Honeydew.suspend/1` and `Honeydew.resume/1` — Queue Flow Control `suspend/1` halts job dispatch to workers without removing pending jobs from the queue. `resume/1` re-enables dispatch. Useful for maintenance windows, deployment rollouts, or controlled draining. ```elixir :ok = Honeydew.start_queue(:batch_queue) :ok = Honeydew.start_workers(:batch_queue, BatchWorker) # Enqueue some work Enum.each(1..50, fn i -> {:process_record, [i]} |> Honeydew.async(:batch_queue) end) # Pause processing :ok = Honeydew.suspend(:batch_queue) IO.puts("Queue suspended — #{Honeydew.status(:batch_queue)[:queue][:count]} jobs waiting") # Perform maintenance, deploy updated worker code, etc. # ... # Resume when ready :ok = Honeydew.resume(:batch_queue) IO.puts("Queue resumed") ``` ``` -------------------------------- ### Define a Stateless Honeydew Worker Source: https://github.com/koudelka/honeydew/blob/master/examples/global/README.md Defines a worker module for handling heavy tasks. This worker is stateless, so the `init/1` function is omitted. The `work_really_hard/1` function simulates a time-consuming operation. ```elixir defmodule HeavyTask do @behaviour Honeydew.Worker # note that in this case, our worker is stateless, so we left out `init/1` def work_really_hard(secs) do :timer.sleep(1_000 * secs) IO.puts "I worked really hard for #{secs} secs!" end end ``` -------------------------------- ### Define Worker for Global Queue Source: https://context7.com/koudelka/honeydew/llms.txt Define a worker module that implements the `Honeydew.Worker` behavior for a global queue. The `transcode/1` function represents the job processing logic. ```elixir defmodule TranscodeWorker do @behaviour Honeydew.Worker def transcode(file_path) do # heavy CPU work here IO.puts("Transcoding #{file_path}") end end ``` -------------------------------- ### Job Progress Tracking Source: https://github.com/koudelka/honeydew/blob/master/README/api.md Allows jobs to emit their current status updates. ```APIDOC ## Job Progress Tracking ### Description Jobs can emit their current status updates, such as progress indicators. ### Functions - `use Honeydew.Progress`: Enables progress reporting within a job module. - `progress/1`: Function provided by `Honeydew.Progress` to report status. ``` -------------------------------- ### Suspend and Resume Job Dispatch Source: https://context7.com/koudelka/honeydew/llms.txt Use `Honeydew.suspend/1` to halt job dispatch and `Honeydew.resume/1` to re-enable it. Useful for maintenance or controlled draining without removing pending jobs. ```elixir :ok = Honeydew.start_queue(:batch_queue) :ok = Honeydew.start_workers(:batch_queue, BatchWorker) # Enqueue some work Enum.each(1..50, fn i -> {:process_record, [i]} |> Honeydew.async(:batch_queue) end) # Pause processing :ok = Honeydew.suspend(:batch_queue) IO.puts("Queue suspended — #{Honeydew.status(:batch_queue)[:queue][:count]} jobs waiting") # Perform maintenance, deploy updated worker code, etc. # ... # Resume when ready :ok = Honeydew.resume(:batch_queue) IO.puts("Queue resumed") ``` -------------------------------- ### Adding a Task to the Queue Source: https://github.com/koudelka/honeydew/blob/master/examples/initialized_worker/README.md Add a task to the Honeydew queue using `async/3`. The worker's initialized state is automatically appended to the task arguments when the worker processes the task. ```elixir iex(1)> {:run, [123]} |> Honeydew.async(:my_queue) Sending email to koudelka+honeydew@ryoukai.org! "hello koudelka, want to enlarge ur keyboard by 500%??" ``` -------------------------------- ### Job Replies Source: https://github.com/koudelka/honeydew/blob/master/README/api.md Enables receiving replies from jobs by setting `reply: true` when initiating an asynchronous job. ```APIDOC ## Job Replies ### Description Receive replies from jobs by passing `reply: true` to `async/3`. ### Functions - `Honeydew.async/3`: Initiates an asynchronous job with the option to receive replies. - `Honeydew.yield/2`: Retrieves replies from jobs. ``` -------------------------------- ### Suspend and Resume Queue Source: https://github.com/koudelka/honeydew/blob/master/README/api.md Allows for halting the distribution of new jobs to workers and then resuming it. ```APIDOC ## Suspend and Resume Queue ### Description Suspend a queue to halt the distribution of new jobs to workers. Resume the queue to resume job distribution. ### Functions - `Honeydew.suspend/1`: Suspends the queue. - `Honeydew.resume/1`: Resumes the queue. ``` -------------------------------- ### Define Schema with Honeydew Fields Source: https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/README.md Add Honeydew's required fields to your Ecto schema definition. ```elixir defmodule MyApp.Photo do use Ecto.Schema import Honeydew.EctoPollQueue.Schema schema "photos" do field(:tag) honeydew_fields(:classify_photos) end end ``` -------------------------------- ### Enqueue a Honeydew Job Source: https://context7.com/koudelka/honeydew/llms.txt Enqueues a job on the named queue. Jobs can be {function_name, [args]} tuples or anonymous functions. Supports 'reply: true' for receiving results via 'yield/2' and 'delay_secs:' for deferred execution. ```elixir {:send_welcome_email, [42]} |> Honeydew.async(:email_queue) ``` ```elixir job = {:compute_result, [100]} |> Honeydew.async(:my_queue, reply: true) {:ok, result} = Honeydew.yield(job, 5_000) # wait up to 5 seconds IO.inspect(result) # => 101 ``` -------------------------------- ### Await Job Result with Timeout Source: https://context7.com/koudelka/honeydew/llms.txt Use `Honeydew.yield/2` to wait for a job's result when enqueued with `reply: true`. Handles success, failure, or timeout. Always call `yield/2` after `reply: true` to prevent mailbox buildup. ```elixir defmodule MathWorker do @behaviour Honeydew.Worker def add(a, b), do: a + b end :ok = Honeydew.start_queue(:math_queue) :ok = Honeydew.start_workers(:math_queue, MathWorker) job = {:add, [3, 4]} |> Honeydew.async(:math_queue, reply: true) case Honeydew.yield(job, 3_000) do {:ok, result} -> IO.puts("Result: #{result}") # => Result: 7 {:error, reason} -> IO.puts("Failed: #{inspect reason}") nil -> IO.puts("Timeout waiting for result") end ``` -------------------------------- ### Define a Honeydew Worker Module Source: https://github.com/koudelka/honeydew/blob/master/examples/local/README.md Defines a worker module that implements the Honeydew.Worker behaviour. This module contains the functions that workers will execute. ```elixir defmodule Worker do @behaviour Honeydew.Worker def hello(thing) do IO.puts "Hello #{thing}!" end end ``` -------------------------------- ### Ecto Poll Queue with Conditional Execution Source: https://context7.com/koudelka/honeydew/llms.txt Optionally configure the Ecto Poll Queue to run jobs only if a specific condition is met, using the `run_if` option with a SQL-like string. ```elixir :ok = Honeydew.start_queue(:classify_photos, queue: {EctoPollQueue, [ schema: MyApp.Photo, repo: MyApp.Repo, run_if: ~s{tag IS NULL} ]} ) ``` -------------------------------- ### Enqueue a Job to a Global Queue Source: https://github.com/koudelka/honeydew/blob/master/examples/global/README.md Submits a job to a global queue for asynchronous processing. The job is defined as a tuple containing the function name and its arguments. The `{:global, :my_queue}` tuple ensures the job is sent to the globally accessible queue. ```elixir iex(clientfacing@dax)1> {:work_really_hard, [5]} |> Honeydew.async({:global, :my_queue}) %Honeydew.Job{by: nil, failure_private: nil, from: nil, monitor: nil, private: {false, -576460752303423485}, queue: {:global, :my_queue}, result: nil, task: {:work_really_hard, [5]}} ``` -------------------------------- ### Honeydew.yield/2 Source: https://context7.com/koudelka/honeydew/llms.txt Waits for the result of a job that was enqueued with `reply: true`. It takes the job identifier and an optional timeout in milliseconds. Returns `{:ok, result}` on success, `{:error, reason}` on failure, or `nil` on timeout. ```APIDOC ## `Honeydew.yield/2` — Await a Job Result Waits for the result of a job that was enqueued with `reply: true`. The second argument is the timeout in milliseconds (defaults to 5000). Returns `{:ok, result}` on success or `{:error, reason}` on failure. Always call `yield/2` after using `reply: true` to avoid mailbox buildup. ```elixir defmodule MathWorker do @behaviour Honeydew.Worker def add(a, b), do: a + b end :ok = Honeydew.start_queue(:math_queue) :ok = Honeydew.start_workers(:math_queue, MathWorker) job = {:add, [3, 4]} |> Honeydew.async(:math_queue, reply: true) case Honeydew.yield(job, 3_000) do {:ok, result} -> IO.puts("Result: #{result}") # => Result: 7 {:error, reason} -> IO.puts("Failed: #{inspect reason}") nil -> IO.puts("Timeout waiting for result") end ``` ``` -------------------------------- ### Configure Queue with Custom Poll Interval Source: https://github.com/koudelka/honeydew/blob/master/examples/ecto_poll_queue/README.md Customize the poll interval for the Ecto Poll Queue to adjust how frequently it checks for new jobs. ```elixir :ok = Honeydew.start_queue(:classify_photos, queue: {EctoPollQueue, queue_args(Photo, Repo)}) defp queue_args(schema, repo) do # Note that the interval is in seconds to poll the database for new jobs [schema: schema, repo: repo, poll_interval: Application.get_env(:ecto_poll_queue, :interval, 2)] end ``` -------------------------------- ### Inserting Row Triggers Ecto Poll Queue Job Source: https://context7.com/koudelka/honeydew/llms.txt Inserting a new row into the configured Ecto schema automatically triggers the associated Honeydew job. This demonstrates the self-managing aspect of the Ecto Poll Queue. ```elixir {:ok, _photo} = %MyApp.Photo{} |> MyApp.Repo.insert() # => "Photo 1 classified as: cat" ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.