===============
LIBRARY RULES
===============
From library maintainers:
- Requires Python 3.12+ and Django 5.2+. Install with pip install django-ox; on Django 5.2 LTS use django-ox[backport] for the django-tasks backport. Django 6.0+ ships the Tasks framework in core.
- Add django_ox to INSTALLED_APPS; set TASKS = {"default": {"BACKEND": "django_ox.backend.OxBackend"}}. Run python manage.py migrate django_ox before starting workers, never from a worker.
- Import task from django.tasks on Django 6.0+, or django_tasks on 5.2. Use @task and .enqueue(...). Task modules must be importable in workers running the producer's code. async def tasks work.
- OxBackend accepts provisional max_attempts, backoff and timeout kwargs on Django 6.1 or Django 5.2 with django-tasks 0.12+. Django 6.0 rejects these kwargs at import; bare @task works. Task.using() cannot set policy fields.
- Run python manage.py ox_worker separately from the web server; tasks need a running worker. --queues a,b restricts its queues. The worker polls; --interval defaults to 1.0 seconds. There is no LISTEN/NOTIFY.
- enqueue() is one INSERT on the OxTask write alias. For a joint commit, open transaction.atomic() there and write the caller's rows there too; bare atomic() uses default. Do not wrap enqueue in transaction.on_commit().
- Execution is at-least-once: write idempotent tasks using database state, not memory flags. Task functions run outside a transaction; open transaction.atomic() inside a task for atomic writes or select_for_update().
- django_ox.bulk.enqueue_many(task, [(args, kwargs), ...]) returns results in input order: one INSERT per 1,000 rows, one transaction. Set queue, priority and run_after once with task.using(...), not per call.
- QUEUES sits beside OPTIONS in TASKS, not inside OPTIONS. It defaults to ["default"]; [] permits any queue name. Inside OPTIONS it is silently ignored. Schedule settings also live under OPTIONS.
- OPTIONS defaults: MAX_ATTEMPTS=3, LOCK_TIMEOUT=300, BACKOFF_INITIAL=5, BACKOFF_MAX=600, TASK_TIMEOUT=None, TASK_TIMEOUTS={}, TASK_TIMEOUT_GRACE=30, SCHEDULES={}. Durations are seconds. Use a non-bool integer 1..32767 for MAX_ATTEMPTS.
- PostgreSQL claims use one UPDATE with a SKIP LOCKED subquery and RETURNING; MySQL 8+ uses SELECT FOR UPDATE SKIP LOCKED. SQLite and databases without SKIP LOCKED use an atomic compare-and-set UPDATE.
- An attempt is consumed at claim, including when a worker dies. max_attempts is stored at enqueue; later declarations do not alter that budget. Backoff and timeout use the worker's live task declaration on each attempt.
- Default retry delay after attempt n is min(BACKOFF_INITIAL * 2 ** (n - 1), BACKOFF_MAX), without jitter. A callback may return an uncapped delay or None to stop. Invalid results or callback errors use the default formula.
- LOCK_TIMEOUT limits missed lease renewal, not task runtime. Renewal is scheduled every LOCK_TIMEOUT / 3 seconds and needs database access. A live worker can lose its lease if renewal cannot reach the database in time.
- Timeout precedence: task timeout, TASK_TIMEOUTS[queue], TASK_TIMEOUT. A task timeout applies even on a queue exempted with None; a task cannot disable an inherited limit. Each attempt gets a fresh timeout.
- At the deadline, sync tasks receive django_ox.exceptions.TaskTimeout, subject to tracing exceptions; async tasks are cancelled. django_ox.deadline() and django_ox.remaining() expose the attempt deadline inside a task.
- A thread still running TASK_TIMEOUT_GRACE seconds after its deadline is recorded as failed; the worker exits 75 for supervisor restart. This stuck-thread path uses worker backoff without calling user callbacks.
- With sys.settrace or active sys.monitoring, timeouts_backstop_only logs: no sync exception is injected. A sync return within grace keeps its outcome; otherwise the task fails and worker recycles. Async cancellation still occurs at deadline.
- Use django_ox.testing.ImmediateBackend or DummyBackend for policy-bearing tasks; stock test backends can reject or ignore policy. Both validate but enforce no retries, backoff or timeouts. Test enforcement with a real worker.
- ox_worker --concurrency N runs a thread pool in one process; --processes N runs N workers under one supervisor. Each process has its own database connections. Use --processes N --concurrency 1 for CPU-bound work.
- A dead worker process restarts after one second, doubling to 30 seconds. More than five deaths of one slot within a minute stops the supervisor with exit 1. Allow for replacement backoff in heartbeat probes.
- PostgreSQL pool: at least concurrency + 1 pooled connections per worker; add a spare for fallback under full load. Budget up to max_size + 2 server connections per process/alias normally, or max_size + 3 during stop-time recovery.
- With Django's PostgreSQL pool, use psycopg_pool 3.3.0+ for the pool-drain fix; 3.3.3 is recommended. Review WARNING connection_pool_cannot_drain.
- OPTIONS["SCHEDULES"] uses cron or every, never both. every is seconds or timedelta with optional phase and fixed-instant ticks. Every worker dispatches; a unique constraint deduplicates tick enqueue. No scheduler process is needed.
- After an outage, enqueue only the most recent missed tick, skipping older ones. New schedules never fire before creation. Retiming stored schedules starts from the change; re-enabling does not replay missed ticks. Tick execution remains at-least-once.
- For admin-editable schedules, set OPTIONS["SCHEDULE_SOURCE"] to "django_ox.stored.DatabaseScheduleSource". Rows name keys exposed by @schedulable or OPTIONS["SCHEDULABLE_TASKS"], never arbitrary import paths.
- @schedulable requires its module to be imported; installed apps' tasks modules are imported. Write with django_ox.stored.create_schedule, create_schedules or update_schedule; objects.create() skips validation and leaves activation stale.
- django_ox.stored.create_schedules(rows, *, user=None) is public and stable. It creates many stored schedules all-or-nothing, returning a list of OxSchedule objects or [] for empty input. Names must be unique within the input.
- create_schedules: one ValidationError lists validation failures by row index, name, field and message. A non-mapping row or unsupported key raises TypeError immediately.
- create_schedules: PermissionDenied separately names denied rows. Database refusals at check or write raise for one row; a concurrent name conflict raises IntegrityError and rolls back the entire call.
- ox_import_beat_schedules reads django-celery-beat's database scheduler tables: PeriodicTask, CrontabSchedule and IntervalSchedule.
- The importer prints one all-or-nothing create_schedules call for review and application, reads Celery crontab grammar and lists untranslatable rows with named reasons.
- For ox_import_beat_schedules, --database selects the django-celery-beat source alias, defaulting to the OxSchedule read alias. --beat-timezone ZONE supplies beat's timezone. Review output and named reasons before applying it.
- 1.9.0 folds in optional, free BSD-3-Clause `oxpull`: unique tasks, batches, rate limiting, provisional workflows; Django 6.0+ (django-ox: 5.2 via django-tasks). Earlier `oxpull` installs: uninstall that distribution, upgrade django-ox, then migrate.
- 1.9.0 fixes Low-severity schedule permission checks in 1.2.0-1.8.0; access requires an API exposing update_schedule with user= or staff schedule permissions. https://github.com/oxpull/django-ox/security/advisories/GHSA-mw49-53pm-wrvf
- Existing django-ox: no new migration in 1.9.0. Before upgrade, psycopg2/mysqlclient: check boundaries/tick history via Compatibility queries. Upgrade workers/app processes; monitor schedule_row_skipped, schedule_tick_unreadable, schedule_dispatch_failed.
- path("ox/", include("django_ox.urls")) exposes GET /ox/metrics without authentication; protect it or restrict network access. django_ox.metrics.collector() registers the same gauges with a prometheus_client registry.
- The /ox/metrics gauges are django_ox_tasks{queue,status}, django_ox_ready_tasks, django_ox_oldest_ready_age_seconds, django_ox_last_claim_age_seconds, django_ox_throughput_per_minute and django_ox_failure_rate.
- Use ox_health database and queue thresholds for dependency checks and fleet alerts, not per-container liveness. --worker-timeout measures recent claims across workers; an idle queue can fail and another worker can make it pass.
- For local liveness, pair ox_worker --heartbeat-file PATH with ox_health --heartbeat-file PATH and matching --processes N. Use a private absolute path per container. A pass proves loop advancement, not task progress or renewal.
- Size heartbeat freshness for startup, poll interval, reap/dispatch, one claim and margin; include pool timeout (default 30 s). There is no overall poll-pass deadline. A database hang can trigger fleet-wide liveness restart storms.
- ox_prune --older-than 7d deletes old SUCCESSFUL and DISCARDED rows. FAILED and LOST stay unless --include-failed. READY, RUNNING and WAITING are never deleted.
- django_ox.actions.retry(result_id) requeues FAILED or LOST for one more attempt. discard(result_id) closes READY, WAITING, FAILED or LOST without running it. Both return bool; neither touches RUNNING tasks.
- Workflows in the optional oxpull app write and release OxTask.Status.WAITING. Workers do not claim it, ox_prune does not delete it and retry skips it. It reads as READY through django.tasks. Workflows are provisional.
- ox_worker, ox_prune and database-mode ox_health take --database, defaulting to the OxTask write alias; overrides are not checked against routing. Keep all django-ox tables together. ox_health --heartbeat-file refuses --database.
- oxpull is included and requires Django 6.0+. Add the app, migrate, and use OxpullBackend. Follow https://oxpull.com/django-ox/features/ for worker, option and reconciler requirements.
- Stored schedule start_time is its activation boundary. starting_deadline_seconds drops overly late ticks and logs schedule_tick_dropped with lateness; the default is no deadline.
- Alert on schedule_dispatch_error and schedule_dispatch_failed; inspect failures and suppressed, not just log counts. manage.py check cannot establish database acceptance. ox_worker --batch exit 0 does not prove every schedule enqueued.
- SIGTERM/SIGINT request drain then exit 0; a second signal forces exit 130. A main loop hung in a database call cannot begin draining until it returns. If the process cannot act on signals, SIGKILL is needed.
- For queued tests, keep OxBackend and use provisional django_ox.testing.run_tasks(). It runs due attempts with retries/backoff, not automatic timeouts, renewal, schedules or reconcilers. Use TransactionTestCase for worker autocommit behaviour.

### Define and enqueue a task with django-ox

Source: https://github.com/oxpull/django-ox/blob/main/README.md

Quickstart example for defining and enqueueing a task. Requires Django 6.0+ or the backport on Django 5.2. The result object can be refreshed to inspect status, return_value, and errors.

```python
from django.tasks import task  # Django 6.0+
# On Django 5.2 the Tasks framework comes from the backport:
# from django_tasks import task


@task
def send_welcome_email(user_id): ...


result = send_welcome_email.enqueue(user_id=42)
result.refresh()  # later: status, return_value, errors
```

--------------------------------

### Install django-ox on Django 6.0+

Source: https://github.com/oxpull/django-ox/blob/main/docs/wagtail.md

Install django-ox on Django 6.0 and later.

```bash
pip install django-ox
```

--------------------------------

### Run development server

Source: https://github.com/oxpull/django-ox/blob/main/docs/background-tasks.md

Command to start the Django development server.

```bash
python manage.py runserver
```

--------------------------------

### Install django-ox with uv

Source: https://github.com/oxpull/django-ox/blob/main/docs/agents.md

Install django-ox using uv. On Django 5.2 LTS, use the backport extra.

```bash
uv add django-ox
```

--------------------------------

### Install django-ox with pip

Source: https://github.com/oxpull/django-ox/blob/main/docs/index.md

Install the base package, or use the backport extra on Django 5.2 LTS.

```bash
pip install django-ox

# on Django 5.2 LTS
pip install "django-ox[backport]"
```

--------------------------------

### Start a worker

Source: https://github.com/oxpull/django-ox/blob/main/docs/agents.md

Start a worker in its own process, next to the web server, under the same supervisor.

```bash
python manage.py ox_worker
```

--------------------------------

### Run migrations and check installation

Source: https://github.com/oxpull/django-ox/blob/main/docs/migrating.md

Run these commands to apply migrations and verify the installation. For the verified oxpull 1.8.0 schema, the migration run reports 'No migrations to apply.' Earlier schemas may have migrations to apply.

```console
python manage.py migrate
python -m pip check
```

--------------------------------

### Check dependencies with pip check

Source: https://github.com/oxpull/django-ox/blob/main/docs/wagtail.md

Verify dependencies after installation or lockfile refresh.

```bash
pip check
```

--------------------------------

### Install django-ox with Django 5.2 backport

Source: https://github.com/oxpull/django-ox/blob/main/docs/background-tasks.md

On Django 5.2 LTS, install the backport extra to get compatibility.

```bash
pip install "django-ox[backport]"
```

--------------------------------

### ox_worker startup log output

Source: https://github.com/oxpull/django-ox/blob/main/docs/background-tasks.md

Example log lines from the worker supervisor and two worker processes, showing the expected startup sequence with the configured process and thread counts.

```text
2026-09-19 08:00:39,114 INFO django_ox Supervisor 24325 starting 2 worker process(es)
2026-09-19 08:00:39,299 INFO django_ox Worker myhost-24327-glmGVHgP-0 starting: queues=['default'] concurrency=4 poll=1.0s schedules=1
2026-09-19 08:00:39,299 INFO django_ox Worker myhost-24328-d7O05mDr-1 starting: queues=['default'] concurrency=4 poll=1.0s schedules=1
```

--------------------------------

### Install django-ox with pip

Source: https://github.com/oxpull/django-ox/blob/main/README.md

Install django-ox. On Django 5.2 LTS, install the backport extra to use the django-tasks backport.

```bash
pip install django-ox

# on Django 5.2 LTS
pip install "django-ox[backport]"
```

--------------------------------

### Install django-ox

Source: https://github.com/oxpull/django-ox/blob/main/docs/background-tasks.md

Installs the django-ox package via pip.

```bash
pip install django-ox
```

--------------------------------

### Run migrations

Source: https://github.com/oxpull/django-ox/blob/main/docs/background-tasks.md

Creates the database tables for django-ox and other apps.

```bash
python manage.py migrate
```

--------------------------------

### Create a batch and add tasks

Source: https://github.com/oxpull/django-ox/blob/main/docs/features/batches.md

Shows how to create a batch with an on_complete callback, add tasks, seal it, and check progress. The batch completes only when all members are settled or gone.

```python
from oxpull import batches


batch = batches.create("nightly-invoices", on_complete=summarize_run)
batch.add(send_invoice, 42)
batch.add(send_invoice, 43)
batch.seal()

batch.progress()  # BatchProgress(total=2, pending=1, succeeded=1, failed=0)
batches.get(batch.id)
```

--------------------------------

### Create and configure a workflow with dependencies

Source: https://github.com/oxpull/django-ox/blob/main/docs/llms-full.txt

Shows how to create a workflow, add nodes with dependencies, seal it, and check progress. Requires backends to declare WORKFLOWS and run OxpullWorker, and the reconciler must be scheduled.

```python
from oxpull import workflows

flow = workflows.create("nightly", on_complete=summarize)
flow.add("extract", extract)
flow.add("transform-a", transform, "a", depends_on="extract")
flow.add("transform-b", transform, "b", depends_on="extract")
flow.add("load", load, depends_on=["transform-a", "transform-b"])
flow.seal()

flow.progress()  # WorkflowProgress(total=4, pending=4, blocked=3, ...)
```

--------------------------------

### Worker startup and schedule dispatch logs

Source: https://github.com/oxpull/django-ox/blob/main/docs/background-tasks.md

Shows the worker startup line indicating schedules=1 and the log output when the schedule fires at the next minute boundary, including the dispatched task and its success.

```text
2026-09-19 07:59:12,044 INFO django_ox Worker myhost-23929-gJ7A6QmA starting: queues=['default'] concurrency=1 poll=1.0s schedules=1
2026-09-19 08:00:00,399 INFO django_ox Dispatched schedule signup-report tick 2026-09-19T08:00:00+00:00 (task id=fe1388c3-bb4a-4f11-9e21-85449a4d50e0)
Task id=fe1388c3-bb4a-4f11-9e21-85449a4d50e0 path=accounts.tasks.signup_report state=RUNNING
2026-09-19 08:00:00,403 INFO django_ox Task id=fe1388c3-bb4a-4f11-9e21-85449a4d50e0 path=accounts.tasks.signup_report succeeded in 0ms
Task id=fe1388c3-bb4a-4f11-9e21-85449a4d50e0 path=accounts.tasks.signup_report state=SUCCESSFUL

```

--------------------------------

### Enable and start ox-worker service

Source: https://github.com/oxpull/django-ox/blob/main/docs/production.md

Commands to enable and start the ox-worker systemd service and follow its logs.

```bash
sudo systemctl enable --now ox-worker
journalctl -u ox-worker -f
```

--------------------------------

### Create Django project and app

Source: https://github.com/oxpull/django-ox/blob/main/docs/background-tasks.md

Commands to create a new Django project and app in the current directory.

```bash
django-admin startproject myproject .
python manage.py startapp accounts
```

--------------------------------

### Install django-ox with backport extra using pip

Source: https://github.com/oxpull/django-ox/blob/main/docs/agents.md

Install django-ox with the backport extra on Django 5.2 LTS.

```bash
pip install "django-ox[backport]"
```

--------------------------------

### Run migrations

Source: https://github.com/oxpull/django-ox/blob/main/docs/wagtail.md

Run migrations to set up the database tables for django-ox.

```bash
python manage.py migrate
```

--------------------------------

### Example text success output

Source: https://github.com/oxpull/django-ox/blob/main/docs/monitoring.md

An example of the text success output with three heartbeat files and ages in seconds.

```text
OK: heartbeat_files=3 oldest_heartbeat_age=0.1s max_heartbeat_age=60s
```

--------------------------------

### Create the database table

Source: https://github.com/oxpull/django-ox/blob/main/docs/agents.md

Run the migration to create the required table for django-ox.

```bash
python manage.py migrate django_ox
```