### Install Oban with Igniter Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Use the igniter.install command to install Oban and configure it automatically. ```bash mix igniter.install oban ``` -------------------------------- ### Run Oban Install Task Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Execute the mix oban.install task to automate the installation steps. ```bash mix oban.install ``` -------------------------------- ### Install Oban with Igniter and Specify Repo Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Use the igniter.install command with the --repo flag to specify an alternate Ecto repository. ```bash mix igniter.install oban --repo MyApp.LiteRepo ``` -------------------------------- ### Start Dynamic Repo and Oban Instance Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md Starts a separate Oban instance and an Ecto dynamic repository instance, bundling them under the same supervisor. The `get_dynamic_repo` option in Oban is configured to use the dynamic repo's PID. ```elixir def start_repo_and_oban(instance_id) do children = [ {MyDynamicRepo, name: nil, url: repo_url(instance_id)}, {Oban, name: instance_id, get_dynamic_repo: fn -> repo_pid(instance_id) end} ] Supervisor.start_link(children, strategy: :one_for_one) end ``` -------------------------------- ### Start Isolated Oban Instances with Names Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md Configure and start multiple Oban supervisors with distinct names and prefixes within the application's supervision tree. ```elixir def start(_type, _args) do children = [ MyApp.Repo, {Oban, name: ObanA, repo: MyApp.Repo}, {Oban, name: ObanB, repo: MyApp.Repo, prefix: "special"}, {Oban, name: ObanC, repo: MyApp.Repo, prefix: "private"} ] Supervisor.start_link(children, strategy: :one_for_one, name: MyApp.Supervisor) end ``` -------------------------------- ### Start Oban Facades in Supervision Tree Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md Include Oban facades as children in your application's supervision tree to ensure they are started and managed. ```elixir @impl true def start(_type, _args) do children = [ MyApp.Repo, MyApp.ObanA, MyApp.ObanB ] Supervisor.start_link(children, strategy: :one_for_one, name: MyApp.Supervisor) end ``` -------------------------------- ### Setup Oban Test Database Source: https://github.com/oban-bg/oban/blob/main/README.md Run this command to set up the necessary PostgreSQL and MySQL databases for running the Oban test suite. Ensure PostgreSQL 14+ and MySQL 8+ are running. ```bash mix test.setup ``` -------------------------------- ### Run Oban queues via environment variables Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/splitting-queues.md Examples of setting the OBAN_QUEUES environment variable to control which queues are active at startup. ```bash OBAN_QUEUES="default,10 media,5" mix phx.server # default: 10, media: 5 ``` ```bash OBAN_QUEUES="*" mix phx.server # default: 15, media: 10, events: 25 ``` ```bash mix phx.server # default: 15, media: 10, events: 25 ``` -------------------------------- ### Add Oban to Dependencies (Manual) Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Add Oban to your project's dependencies in mix.exs for manual installation. ```elixir {:oban, "~> 2.23"} ``` -------------------------------- ### Ecto Multi-tenancy Query Preparation Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md An example of an Ecto Repo's `prepare_query/3` callback that handles multi-tenancy by conditionally adding an `org_id` to queries. It includes an exception for Oban queries (identified by the `:oban` option) and other specific cases like schema migrations. ```elixir defmodule MyApp.Repo do use Ecto.Repo, otp_app: :my_app require Ecto.Query @impl true def prepare_query(_operation, query, opts) do cond do opts[:skip_org_id] || opts[:schema_migration] || opts[:oban] -> {query, opts} org_id = opts[:org_id] -> {Ecto.Query.where(query, org_id: ^org_id), opts} true -> raise "expected org_id or skip_org_id to be set" end end end ``` -------------------------------- ### Example Plugin Configuration Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/writing_plugins.md Configure the `Breakdown` plugin within your application's Oban configuration, specifying the desired polling interval. Oban automatically injects `:name` and `:conf` options. ```elixir config :my_app, Oban, plugins: [ {MyApp.Plugins.Breakdown, interval: :timer.seconds(10)} ] ... ``` -------------------------------- ### Scheduling Jobs in the Near Future Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/reliable-scheduling.md Examples demonstrating how to schedule jobs to run in the very near future using `scheduled_in` and `scheduled_at`. Be aware of the `:stage_interval` configuration's impact. ```elixir # 1 second from now %{} |> new(scheduled_in: 1) |> Oban.insert() # 500 milliseconds from now very_soon = DateTime.utc_now() |> DateTime.add(500, :millisecond) %{} |> new(scheduled_at: very_soon) |> Oban.insert() ``` -------------------------------- ### Add Oban and Igniter to Dependencies Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Add Oban and Igniter to your project's dependencies in mix.exs for semi-automatic installation. ```elixir {:oban, "~> 2.23"}, {:igniter, "~> 0.5", only: [:dev]}, ``` -------------------------------- ### JSON Configuration for Oban Queues Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/release_configuration.md Define your Oban queue configurations in a JSON file. This example shows how to specify concurrency for 'special', 'default', and 'events' queues. ```json {"queues": {"special": 1, "default": 10, "events": 20}} ``` -------------------------------- ### Example of Calling Plugin Interface Function Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/writing_plugins.md Shows how to invoke the `pause` interface function for the `Breakdown` plugin, either with the default Oban instance or a specific Oban instance. ```elixir MyApp.Plugins.Breakdown.pause() # or MyApp.Plugins.Breakdown.pause(MyApp.OtherOban) ``` -------------------------------- ### Example Oban Plugin: Breakdown Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/writing_plugins.md This plugin periodically generates and prints a table of counts for each queue/state combination in the `oban_jobs` table. It demonstrates handling options, using the `Oban.Config` struct, and interacting with the database. ```elixir defmodule MyApp.Plugins.Breakdown do @behaviour Oban.Plugin use GenServer import Ecto.Query, only: [group_by: 3, select: 3] @impl Oban.Plugin def start_link(opts) do name = Keyword.get(opts, :name, __MODULE__) GenServer.start_link(__MODULE__, opts, name: name) end @impl Oban.Plugin def validate(opts) do Oban.Validation.validate_schema(opts, conf: :any, name: :any, interval: :pos_integer ) end @impl GenServer def init(opts) do state = Map.new(opts) {:ok, schedule_poll(state)} end @impl GenServer def handle_info(:poll, %{conf: conf} = state) do breakdown = Oban.Repo.all( conf, Oban.Job |> group_by([j], [j.queue, j.state]) |> select([j], {j.queue, j.state, count(j.id)}) ) IO.inspect(breakdown) {:noreply, schedule_poll(state)} end defp schedule_poll(%{interval: interval} = state) do Process.send_after(self(), :poll, interval) state end end ``` -------------------------------- ### Enable Periodic Index Rebuilding with Reindexer Plugin Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/scaling.md Add the Oban.Plugins.Reindexer plugin to rebuild indexes periodically and free up disk space. Configure the schedule, with '@weekly' as an example. ```diff config :my_app, Oban, plugins: [ + {Oban.Plugins.Reindexer, schedule: "@weekly"}, … ] ``` -------------------------------- ### Attach Default Telemetry Logger Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/ready_for_production.md Configure the default Oban telemetry logger within the application start function. ```elixir defmodule MyApp.Application do use Application @impl Application def start(_type, _args) do Oban.Telemetry.attach_default_logger() children = [ ... ] end end ``` ```elixir Oban.Telemetry.attach_default_logger(encode: false, level: :debug) ``` -------------------------------- ### Starting a Recursive Backfill Job in IEx Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/recursive-jobs.md Initiates the recursive backfilling process for user timezone data by enqueuing the first job. The recursion continues until no more users without timezone information are found. ```elixir iex> %{id: 1, backfill: true} |> MyApp.Workers.TimezoneWorker.new() |> Oban.insert() ``` -------------------------------- ### Attach Custom Handler in Application Source: https://github.com/oban-bg/oban/blob/main/guides/learning/instrumentation.md Registers the custom log handler within the application's start callback to monitor job lifecycle events. ```elixir def start(_type, _args) do events = [ [:oban, :job, :start], [:oban, :job, :stop], [:oban, :job, :exception] ] :telemetry.attach_many("oban-logger", events, &MyApp.ObanLogger.handle_event/4, []) Supervisor.start_link(...) end ``` -------------------------------- ### Run Database Migrations Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Execute the database migrations to create the Oban jobs table. ```bash mix ecto.migrate ``` -------------------------------- ### Calling Plugin Interface Functions with Oban.Registry Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/writing_plugins.md Demonstrates how to define and call an interface function (e.g., `pause`) for a plugin using `Oban.Registry` for process discovery. This allows interaction with the plugin process even when it's named dynamically. ```elixir alias Oban.Registry def pause(oban_name \ Oban) do oban_name |> Registry.whereis({:plugin, __MODULE__}) |> GenServer.call(:pause) end ``` -------------------------------- ### Basic Queue Configuration Source: https://github.com/oban-bg/oban/blob/main/guides/learning/defining_queues.md Configure multiple queues with a simple map of queue names to their maximum concurrency limits. Ensure the Oban repo is specified. ```elixir config :my_app, Oban, queues: [default: 10, mailers: 20, events: 50, media: 5], repo: MyApp.Repo ``` -------------------------------- ### Configure Oban for Postgres Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Configure Oban to use the Basic engine with Postgres in config.exs. ```elixir config :my_app, Oban, engine: Oban.Engines.Basic, queues: [default: 10], repo: MyApp.Repo ``` -------------------------------- ### Custom Error Reporter with Reportable Check Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/expected-failures.md An error reporter that checks the Reportable protocol before sending notifications to Honeybadger. It attempts to get a worker struct to pass to the protocol. ```elixir defmodule MyApp.ErrorReporter do alias MyApp.Reportable def handle_event(_, _, meta, _) do worker_struct = maybe_get_worker_struct(meta.job.worker) if Reportable.reportable?(worker_struct, meta.job.attempt) do context = Map.take(meta.job, [:id, :args, :queue, :worker]) Honeybadger.notify(meta.reason, context, meta.stacktrace) end end def maybe_get_worker_struct(worker) do try do {:ok, module} = Oban.Worker.from_string(worker) struct(module) rescue UndefinedFunctionError -> worker end end end ``` -------------------------------- ### Configure Oban Facades Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md Configure facades in the application environment, specifying the repository and a unique prefix for each facade. ```elixir config :my_app, MyApp.ObanA, repo: MyAppo.Repo, prefix: "special" config :my_app, MyApp.ObanB, repo: MyAppo.Repo, prefix: "private" ``` -------------------------------- ### Run CI Checks Locally Source: https://github.com/oban-bg/oban/blob/main/README.md Execute this command to replicate the CI environment locally. It checks formatting, dependencies, linting, tests, and Dialyzer. ```bash mix test.ci ``` -------------------------------- ### Configure Oban for SQLite3 Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Configure Oban to use the Lite engine with SQLite3 in config.exs. ```elixir config :my_app, Oban, engine: Oban.Engines.Lite, queues: [default: 10], repo: MyApp.Repo ``` -------------------------------- ### Configure Oban Global Peer in Development Source: https://github.com/oban-bg/oban/blob/main/guides/learning/clustering.md Use Oban.Peers.Global for faster leadership transitions in development environments. This requires Distributed Erlang. ```elixir config :my_app, Oban, peer: Oban.Peers.Global, ... ``` -------------------------------- ### Configure Oban Testing with Prefixes Source: https://github.com/oban-bg/oban/blob/main/guides/testing/testing.md When using PostgreSQL schemas for isolation, specify the 'prefix' option when using Oban.Testing. ```elixir use Oban.Testing, repo: MyApp.Repo, prefix: "private" ``` -------------------------------- ### Configure Oban queues in config.exs Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/splitting-queues.md Define the default set of queues and their concurrency limits in the application configuration. ```elixir config :my_app, Oban, repo: MyApp.Repo, queues: [default: 15, media: 10, events: 25] ``` -------------------------------- ### Configure Oban Queues and Repo Source: https://github.com/oban-bg/oban/blob/main/README.md Configure Oban to use a specific Ecto repository and define the queues it should manage. This is typically done in your application's configuration file. ```elixir config :my_app, Oban, repo: MyApp.Repo, queues: [mailers: 20] ``` -------------------------------- ### Asynchronous Zip Building Task Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/reporting-progress.md Initiates an asynchronous task to build the zip file, sending progress updates back to the job's process. ```elixir defp build_zip(paths) do job_pid = self() Task.async(fn -> zip_path = Zipper.new() paths |> Enum.with_index(1) |> Enum.each(fn {path, index} -> :ok = Zipper.add_file(zip_path, path) send(job_pid, {:progress, trunc(index / length(paths) * 100)}) end) send(job_pid, {:complete, zip_path}) end) end ``` -------------------------------- ### Configure Oban for MySQL Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Configure Oban to use the Dolphin engine with MySQL in config.exs. ```elixir config :my_app, Oban, engine: Oban.Engines.Dolphin, queues: [default: 10], repo: MyApp.Repo ``` -------------------------------- ### Implement dynamic queue parsing in Application module Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/splitting-queues.md Use an environment variable to override queue settings at runtime within the application supervisor. ```elixir defmodule MyApp.Application do @moduledoc false use Application def start(_type, _args) do children = [ MyApp.Repo, MyApp.Endpoint, {Oban, oban_opts()} ] Supervisor.start_link(children, strategy: :one_for_one, name: MyApp.Supervisor) end defp oban_opts do env_queues = System.get_env("OBAN_QUEUES") :my_app |> Application.get_env(Oban) |> Keyword.update(:queues, [], &queues(env_queues, &1)) end defp queues("*", defaults), do: defaults defp queues(nil, defaults), do: defaults defp queues(_, false), do: false defp queues(values, _defaults) when is_binary(values) do values |> String.split(" ", trim: true) |> Enum.map(&String.split(&1, ",", trim: true)) |> Keyword.new(fn [queue, limit] -> {String.to_existing_atom(queue), String.to_integer(limit)} end) end end ``` -------------------------------- ### Visualize Cron Expression Fields Source: https://github.com/oban-bg/oban/blob/main/guides/learning/periodic_jobs.md A visual representation of the five fields required for a standard cron expression. ```text ┌───────────── minute (0 - 59) │ ┌───────────── hour (0 - 23) │ │ ┌───────────── day of the month (1 - 31) │ │ │ ┌───────────── month (1 - 12 or JAN-DEC) │ │ │ │ ┌───────────── day of the week (0 - 6 or SUN-SAT) │ │ │ │ │ │ │ │ │ │ * * * * * ``` -------------------------------- ### Insert Job Using Oban Facade Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md Use the specific Oban facade module to insert jobs, ensuring they are processed by the intended isolated instance. ```elixir MyApp.ObanA.insert(MyApp.Worker.new(%{})) ``` -------------------------------- ### Define Oban Facades Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md Create distinct Oban modules by using Oban with unique names. This allows for separate configurations and isolation. ```elixir defmodule MyApp.ObanA do use Oban, otp_app: :my_app end defmodule MyApp.ObanB do use Oban, otp_app: :my_app end ``` -------------------------------- ### Advanced Queue Configuration with Overrides Source: https://github.com/oban-bg/oban/blob/main/guides/learning/defining_queues.md Utilize an expanded form for queues to set individual overrides like dispatch cooldown, paused state, or global limits. Ensure the Oban repo is specified. ```elixir config :my_app, Oban, queues: [ default: 10, mailers: [limit: 20, dispatch_cooldown: 50], events: [limit: 50, paused: true], media: [limit: 1, global_limit: 10] ], repo: MyApp.Repo ``` -------------------------------- ### Verify Oban Configuration in IEx Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Check Oban's configuration in an IEx session to confirm it's set up correctly. ```elixir iex(1)> Oban.config() #=> %Oban.Config{repo: MyApp.Repo} ``` -------------------------------- ### Use Global Peer for Development Cron Jobs Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/troubleshooting.md In development, use the `Global` peer for more graceful handling of restarts, which can resolve issues with `@reboot` cron expressions not running. ```elixir # In config/dev.exs config :my_app, Oban, peer: Oban.Peers.Global, ... ``` -------------------------------- ### Define Migration for Prefixed Oban Jobs Table Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md Create a migration to set up Oban's job tables within a specific PostgreSQL schema (prefix). ```elixir defmodule MyApp.Repo.Migrations.AddPrefixedObanJobsTable do use Ecto.Migration def up do Oban.Migrations.up(prefix: "private") end def down do Oban.Migrations.down(prefix: "private") end end ``` -------------------------------- ### Switch Oban to Use Dedicated Repo and Dynamic Repo Function Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/scaling.md Update your Oban configuration to use the newly defined `MyApp.ObanRepo` and specify the `get_dynamic_repo` function. This ensures Oban utilizes its dedicated connection pool. ```diff config :my_app, Oban, - repo: MyApp.Repo, + repo: MyApp.ObanRepo, + get_dynamic_repo: {MyApp.Repo, :oban_repo, []} ... ``` -------------------------------- ### Enqueue a New Oban Job Source: https://github.com/oban-bg/oban/blob/main/README.md Create a new job instance for a worker and insert it into the Oban queue. The job's arguments are passed as a map. ```elixir %{email: %{to: "foo@example.com", body: "Hello from Oban!"}} |> MyApp.MailerWorker.new() |> Oban.insert() ``` -------------------------------- ### Configure Oban with Database Prefix Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md Specify the database prefix in the Oban configuration to ensure jobs are managed within the designated schema. ```elixir config :my_app, Oban, prefix: "private", repo: MyApp.Repo, queues: [default: 10] ``` -------------------------------- ### Configure Release with Custom Provider Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/release_configuration.md Integrate the custom Oban Config Provider into your Elixir Release configuration in `mix.exs`. Specify the path to the JSON configuration file. ```elixir releases: umbrella_app: [ version: "0.0.1", applications: child_app: :permanent ], config_providers: [{Path.To.ConfigProvider, "/etc/config.json"}] ] ``` -------------------------------- ### Insert Job as Available Source: https://github.com/oban-bg/oban/blob/main/guides/learning/job_lifecycle.md Insert a new job into Oban that will be immediately available for execution. This is the default behavior when no scheduling is specified. ```elixir %{id: 123} |> MyApp.Worker.new() |> Oban.insert() ``` -------------------------------- ### Configure Lifeline Plugin Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/ready_for_production.md Use the Lifeline plugin to automatically rescue orphaned jobs after a specified duration. ```elixir config :my_app, Oban, plugins: [ {Oban.Plugins.Lifeline, rescue_after: :timer.minutes(30)}, ... ``` -------------------------------- ### Configure Lifeline Plugin and Shutdown Grace Period Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/troubleshooting.md Use the Lifeline plugin and increase the shutdown grace period to mitigate jobs stuck in an executing state during deployments or restarts. ```elixir config :my_app, Oban, plugins: [Oban.Plugins.Lifeline], shutdown_grace_period: :timer.seconds(60), ... ``` -------------------------------- ### Batch insert jobs from Rails Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/migrating-from-other-languages.md Use insert_all to perform a bulk insertion of multiple jobs into the queue. ```ruby Oban::Job.insert_all([ {worker: "MyWorker", args: {id: 1}, queue: "default"}, {worker: "MyWorker", args: {id: 2}, queue: "default"}, {worker: "MyWorker", args: {id: 3}, queue: "default"}, ]) ``` -------------------------------- ### Configure Oban for Testing Mode Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Enable manual testing mode for Oban in config/test.exs to prevent job execution during tests. ```elixir config :my_app, Oban, testing: :manual ``` -------------------------------- ### Configure Dynamic Repo for Oban Transactions Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/scaling.md Modify your main Ecto Repo to include a function that returns the appropriate repository (application's or Oban's) based on whether a transaction is active. This ensures Oban uses its dedicated repo unless within an existing application transaction. ```diff defmodule MyApp.Repo do use Ecto.Repo, adapter: Ecto.Adapters.Postgres, otp_app: :my_app + def oban_repo do + if in_transaction?() do + MyApp.Repo + else + MyApp.ObanRepo + end + end end ``` -------------------------------- ### Use Oban Testing Helpers in Test Cases Source: https://github.com/oban-bg/oban/blob/main/guides/testing/testing.md Include Oban.Testing in your test module to gain access to assertion helpers. Requires specifying the Ecto repository. ```elixir defmodule MyApp.Case do use ExUnit.CaseTemplate using do quote do use Oban.Testing, repo: MyApp.Repo end end end ``` -------------------------------- ### Configure Oban for Inline Testing Source: https://github.com/oban-bg/oban/blob/main/guides/testing/testing.md Set Oban to :inline testing mode in your application configuration. Jobs execute immediately without database interaction. ```elixir config :my_app, Oban, testing: :inline ``` -------------------------------- ### Validate dynamic Oban configuration Source: https://github.com/oban-bg/oban/blob/main/guides/testing/testing_config.md Use Oban.Config.validate/1 to verify that runtime configuration options are correctly structured. ```elixir test "production oban config is valid" do config = "config/config.exs" |> Config.Reader.read!(env: :prod) |> get_in([:my_app, Oban]) assert :ok = Oban.Config.validate(config) end ``` ```elixir {:error, "expected :engine to be an Oban.Queue.Engine, got: MyApp.Repo"} ``` -------------------------------- ### Define Oban Migration Functions Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/installation.md Implement the up and down functions in the generated migration file using Oban.Migration. ```elixir defmodule MyApp.Repo.Migrations.AddObanJobsTable do use Ecto.Migration def up do Oban.Migration.up(version: 14) end # We specify `version: 1` in `down`, ensuring that we'll roll all the way back down if # necessary, regardless of which version we've migrated `up` to. def down do Oban.Migration.down(version: 1) end end ``` -------------------------------- ### Custom Oban Config Provider Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/release_configuration.md Implement a custom Elixir Config Provider to load Oban queue configurations from a JSON file. This provider parses a JSON file and merges the queue settings into the Oban configuration. ```elixir defmodule MyApp.ConfigProvider do @moduledoc """ Provide release configuration for Oban Queue Concurrency """ @behaviour Config.Provider def init(path) when is_binary(path), do: path def load(config, path) do case parse_json(path) do nil -> config queues -> Config.Reader.merge(config, ingestion: [{Oban, [queues: queues]}]) end end defp parse_json(path) do if File.exists?(path) do path |> File.read!() |> JSON.decode!() |> Map.fetch!("queues") |> Keyword.new(fn {key, value} -> {String.to_atom(key), value} end) end end end ``` -------------------------------- ### Implement Custom Oban Log Handler Source: https://github.com/oban-bg/oban/blob/main/guides/learning/instrumentation.md Defines a module to handle specific Oban job events with custom logging logic. ```elixir defmodule MyApp.ObanLogger do require Logger def handle_event([:oban, :job, :start], measure, meta, _) do Logger.warning("[Oban] :started #{meta.worker} at #{measure.system_time}") end def handle_event([:oban, :job, event], measure, meta, _) do Logger.warning("[Oban] #{event} #{meta.worker} ran in #{measure.duration}") end end ``` -------------------------------- ### Insert Job as Scheduled Source: https://github.com/oban-bg/oban/blob/main/guides/learning/job_lifecycle.md Insert a new job into Oban with a specified delay, making it initially scheduled rather than immediately available. The job will become available after the delay. ```elixir %{id: 123} |> MyApp.Worker.new(scheduled_in: 60) |> Oban.insert() ``` -------------------------------- ### Configure Job Uniqueness with Infinity Period and Keys Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/scaling.md Optimize job uniqueness by setting the period to :infinity and defining specific keys. This avoids relying on the full args or meta for uniqueness checks. ```diff use Oban.Worker, unique: [ - period: {1, :hour}, + period: :infinity, + keys: [:some_key] ] ``` -------------------------------- ### Schedule a job for the future from Rails Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/migrating-from-other-languages.md Pass a scheduled_at timestamp to the insert method to queue a job for future execution. ```ruby Oban::Job.insert(worker: "MyWorker", args: {id: 1}, scheduled_at: 1.minute.from_now.utc) ``` -------------------------------- ### Enable Pruner and Reindexer Plugins Source: https://github.com/oban-bg/oban/blob/main/guides/learning/operational_maintenance.md Enable both the Pruner plugin for job cleanup and the Reindexer plugin for index maintenance in your Oban configuration. This ensures efficient job processing and database health. ```elixir config :my_app, Oban, plugins: [Oban.Plugins.Pruner, Oban.Plugins.Reindexer], # ... ``` -------------------------------- ### Insert a single job from Rails Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/migrating-from-other-languages.md Use the custom insert method to enqueue a job for immediate execution. ```ruby Oban::Job.insert(worker: "MyWorker", args: {id: 1}, queue: "default") ``` -------------------------------- ### Test worker backoff and timeout callbacks Source: https://github.com/oban-bg/oban/blob/main/guides/testing/testing_workers.md Manually construct Oban.Job structs to test worker-specific callbacks like backoff and timeout directly. ```elixir test "calculating custom backoff as a multiple of job attempts" do assert 2 == MyWorker.backoff(%Oban.Job{attempt: 1}) assert 4 == MyWorker.backoff(%Oban.Job{attempt: 2}) assert 6 == MyWorker.backoff(%Oban.Job{attempt: 3}) end ``` ```elixir test "allowing a multiple of the attempt as job timeout" do assert 1000 == MyWorker.timeout(%Oban.Job{attempt: 1}) assert 2000 == MyWorker.timeout(%Oban.Job{attempt: 2}) end ``` -------------------------------- ### Implement Custom Error Reporter Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/ready_for_production.md Create a custom telemetry handler to report job exceptions to external monitoring services like Sentry. ```elixir defmodule MyApp.ObanReporter do def attach do :telemetry.attach("oban-errors", [:oban, :job, :exception], &__MODULE__.handle_event/4, []) end def handle_event([:oban, :job, :exception], measure, meta, _) do extra = meta.job |> Map.take([:id, :args, :meta, :queue, :worker]) |> Map.merge(measure) Sentry.capture_exception(meta.reason, stacktrace: meta.stacktrace, extra: extra) end end ``` ```elixir # application.ex @impl Application def start(_type, _args) do MyApp.ObanReporter.attach() end ``` -------------------------------- ### Use Oban Testing Helpers in Individual Tests Source: https://github.com/oban-bg/oban/blob/main/guides/testing/testing.md Alternatively, use Oban.Testing within specific tests for more granular control. Requires specifying the Ecto repository. ```elixir defmodule MyApp.WorkerTest do use MyApp.Case, async: true use Oban.Testing, repo: MyApp.Repo end ``` -------------------------------- ### Define an ActiveRecord model for Oban jobs Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/migrating-from-other-languages.md Create a skeletal ActiveRecord model to interface with the oban_jobs table, including a helper method for inserting jobs. ```ruby class Oban::Job < ApplicationRecord # This column is in use, but not used for the insert workflow. self.ignored_columns = %w[errors] # A simple wrapper around `create` that ensures the job is scheduled immediately. def self.insert(worker:, args: {}, queue: "default", scheduled_at: nil) create( worker: worker, queue: queue, args: args, scheduled_at: scheduled_at || Time.now.utc, state: scheduled_at ? "scheduled" : "available" ) end end ``` -------------------------------- ### Switch to PG Notifier for Clustered Environments Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/scaling.md Switch to Oban.Notifiers.PG for clustered applications to reduce database load and allow larger messages. This moves notifications out of the database. ```diff config :my_app, Oban, + notifier: Oban.Notifiers.PG, ``` -------------------------------- ### Receive Loop for Progress and Completion Source: https://github.com/oban-bg/oban/blob/main/guides/recipes/reporting-progress.md A custom receive loop that waits for progress or completion messages from the asynchronous task, broadcasting updates via Phoenix Channels. Includes a timeout mechanism. ```elixir defp await_zip(channel) do receive do {:progress, percent} -> Endpoint.broadcast(channel, "zip:progress", percent) await_zip(channel) {:complete, zip_path} -> Endpoint.broadcast(channel, "zip:complete", zip_path) after 30_000 -> Endpoint.broadcast(channel, "zip:failed", "zipping failed") raise RuntimeError, "no progress after 30s" end end ``` -------------------------------- ### Validate dynamic plugin configuration Source: https://github.com/oban-bg/oban/blob/main/guides/testing/testing_config.md Test complex plugin configurations in isolation using the Oban.Plugin.validate/1 callback. ```elixir defmodule MyApp.Oban do def cron_config do [crontab: [{"0 24 * * *", MyApp.Worker}]] end end ``` ```elixir test "testing cron plugin configuration" do config = MyApp.Oban.cron_config() assert :ok = Oban.Plugins.Cron.validate(config) end ``` ```elixir {:error, %ArgumentError{message: "expression field 24 is out of range 0..23"}} ``` -------------------------------- ### Schedule Job at a Specific Time Source: https://github.com/oban-bg/oban/blob/main/guides/learning/scheduling_jobs.md Schedule a job to run at a precise timestamp using the `scheduled_at` option. Ensure the timestamp is in UTC. ```elixir %{id: 1} |> MyApp.SomeWorker.new(scheduled_at: ~U[2020-12-25 19:00:00Z]) |> Oban.insert() ``` -------------------------------- ### Specifying Keys within Args for Uniqueness Source: https://github.com/oban-bg/oban/blob/main/guides/learning/unique_jobs.md Use the `:keys` option to specify particular keys within the `:args` map for uniqueness comparison, providing more granular control than comparing the entire map. ```elixir # This compares only the :url key within the args map use Oban.Worker, unique: [keys: [:url], fields: [:worker, :queue, :args]] ``` -------------------------------- ### Configure Pruner Plugin with Custom Max Age Source: https://github.com/oban-bg/oban/blob/main/guides/learning/operational_maintenance.md Configure the Pruner plugin to retain completed, cancelled, or discarded jobs for a custom duration. Specify the retention period in seconds using the `:max_age` option. ```elixir config :my_app, Oban, plugins: [{Oban.Plugins.Pruner, max_age: _5_minutes_in_seconds = 300}], # ... ``` -------------------------------- ### Configure Oban Pruner Plugin Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/scaling.md Add the Pruner plugin to your Oban configuration to periodically delete completed, cancelled, and discarded jobs. Set `max_age` to limit the retention period of historic jobs. ```diff config :my_app, Oban, plugins: [ + {Oban.Plugins.Pruner, max_age: 1_day_in_seconds} … ] ``` -------------------------------- ### Configure Job Pruning Source: https://github.com/oban-bg/oban/blob/main/guides/introduction/ready_for_production.md Add the Pruner plugin to the Oban configuration to automatically remove old jobs from the database. ```elixir config :my_app, Oban, plugins: [ {Oban.Plugins.Pruner, max_age: 60 * 60 * 24 * 7}, ... ``` -------------------------------- ### Basic Unique Job Configuration Source: https://github.com/oban-bg/oban/blob/main/guides/learning/unique_jobs.md Configure a worker to prevent duplicate jobs from being enqueued for as long as a matching job exists in the database, regardless of its state. ```elixir use Oban.Worker, unique: true ``` -------------------------------- ### Convert Local Time to UTC for Scheduling Source: https://github.com/oban-bg/oban/blob/main/guides/learning/scheduling_jobs.md Convert a local datetime to UTC before scheduling a job to ensure consistent execution across time zones and avoid daylight saving issues. ```elixir # Convert a datetime in a local timezone to UTC for scheduling utc_datetime = DateTime.shift_zone!(local_datetime, "Etc/UTC") %{id: 1} |> MyApp.SomeWorker.new(scheduled_at: utc_datetime) |> Oban.insert() ``` -------------------------------- ### Configure Oban for Umbrella Apps Source: https://github.com/oban-bg/oban/blob/main/guides/learning/isolation.md Set custom names for Oban configurations in different child applications of an umbrella project to manage interactions. ```elixir config :my_app_a, Oban, name: MyAppA.Oban, repo: MyApp.Repo config :my_app_b, Oban, name: MyAppB.Oban, repo: MyApp.Repo, queues: [default: 10] ``` -------------------------------- ### Schedule Job in Relative Time Source: https://github.com/oban-bg/oban/blob/main/guides/learning/scheduling_jobs.md Schedule a job to run after a specified number of seconds using the `scheduled_in` option. ```elixir %{id: 1} |> MyApp.SomeWorker.new(scheduled_in: _seconds = 5) |> Oban.insert() ``` -------------------------------- ### Define a Dedicated Oban Ecto Repo Source: https://github.com/oban-bg/oban/blob/main/guides/advanced/scaling.md Create a separate Ecto repository specifically for Oban's database interactions. This isolates Oban's queries from your application's main Ecto repository, preventing connection pool contention. ```elixir defmodule MyApp.ObanRepo do use Ecto.Repo, adapter: Ecto.Adapters.Postgres, otp_app: :my_app end ``` -------------------------------- ### Report Oban Job Exceptions with Honeybadger Source: https://github.com/oban-bg/oban/blob/main/guides/learning/error_handling.md Integrate with Honeybadger to report job failures by attaching a telemetry event handler to the `:oban, :job, :exception` event. This handler sends the job's reason and stacktrace to Honeybadger for monitoring. ```elixir defmodule MyApp.ErrorReporter do def attach do :telemetry.attach( "oban-errors", [:oban, :job, :exception], &__MODULE__.handle_event/4, [] ) end def handle_event([:oban, :job, :exception], measure, meta, _) do Honeybadger.notify(meta.reason, stacktrace: meta.stacktrace) end end # Attach it with: MyApp.ErrorReporter.attach() ``` -------------------------------- ### Integration Testing with Queue Draining Source: https://github.com/oban-bg/oban/blob/main/guides/testing/testing_queues.md Use Oban.drain_queue to execute pending jobs in a specific queue during integration tests. ```elixir defmodule MyApp.BusinessTest do use MyApp.DataCase, async: true alias MyApp.{Business, Worker} test "we stay in the business of doing business" do :ok = Business.schedule_a_meeting(%{email: "monty@brewster.com"}) assert %{success: 1, failure: 0} = Oban.drain_queue(queue: :mailer) # Now, make an assertion about the email delivery end end ```