Procrastinate

https://procrastinate.readthedocs.io/en/stable/
Admin
Procrastinate is an open-source Python distributed task processing library that leverages PostgreSQL...
Tokens:88,857
Snippets:911
Trust Score:9.6
Update:2 weeks ago
Show doc for...
Tokens:
Raw
### Install PostgreSQL client and create database on MacOS

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Installs libpq on MacOS and creates the development database using the createdb command from the libpq installation path.

```bash
$ # MacOS
$ brew install libpq
$ /usr/local/opt/libpq/bin/createdb

```

--------------------------------

### Install PostgreSQL client and create database on Ubuntu

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Installs the PostgreSQL client on Ubuntu and creates the development database using the createdb command.

```bash
$ # Ubuntu
$ sudo apt install postgresql-client
$ createdb

```

--------------------------------

### Install nox and dependencies with uv sync

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Install nox and dependencies for running acceptance tests. Use the minimal command for just the migration test group, or the maximal command for all extras and groups.

```bash
$ # Minimally:
$ uv sync --group migration_test
$ # Maximally:
$ uv sync --all-extras --all-groups

```

--------------------------------

### Set up development environment with dev-env script

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Run this one-liner to set up the development environment. It assumes macOS or Linux with bash or zsh, python3, and either uv/pre-commit installed or permission to install them, plus a PostgreSQL database or Docker Compose with port 5432 free. The script adds the scripts folder to your PATH for the current shell.

```bash
$ source ./dev-env

```

--------------------------------

### Bootstrap development environment

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Installs uv and pre-commit if necessary, using the scripts/bootstrap script.

```bash
$ scripts/bootstrap

```

--------------------------------

### Set up virtual environment and install dependencies

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Creates a virtual environment with uv, syncs project dependencies, and activates the environment. Replace {x} with the desired Python version.

```bash
$ uv venv --python=3.{x}  # Select the Python version you want to use (replace {x})
$ uv sync  # Install the project and its dependencies
$ uv run $SHELL  # Activate the virtual environment
$ exit # Quit the virtual environment

```

--------------------------------

### Install procrastinate[sphinx]

Source: https://procrastinate.readthedocs.io/en/stable/howto/advanced/sphinx.html

Install Procrastinate with the sphinx extra in the environment where you build your documentation. This is not mandatory, but it adds sphinx itself as a dependency and ensures any future dependencies of the extension are installed.

```bash
$ pip install procrastinate[sphinx]

```

--------------------------------

### PROCRASTINATE_ON_APP_READY settings.py example

Source: https://procrastinate.readthedocs.io/en/stable/howto/django/settings.html

Example of setting PROCRASTINATE_ON_APP_READY in settings.py to point to a function that modifies the app after creation, such as loading additional tasks from blueprints.

```python
# settings.py
PROCRASTINATE_ON_APP_READY = "myapp.procrastinate.on_app_ready"
```

--------------------------------

### Install Procrastinate with pip

Source: https://procrastinate.readthedocs.io/en/stable/quickstart.html

Installs Procrastinate within a virtual environment.

```bash
pip install procrastinate
```

--------------------------------

### Run Django demo migrations and server

Source: https://procrastinate.readthedocs.io/en/stable/demos.html

Runs the Django migrations and starts the Django development server for the Django demo. Run these in the first terminal.

```shell
$ procrastinate/demos/demo_django/manage.py migrate
$ procrastinate/demos/demo_django/manage.py runserver

```

--------------------------------

### Install procrastinate with Django extra

Source: https://procrastinate.readthedocs.io/en/stable/howto/django/configuration.html

Install Procrastinate with the Django extra using pip. This is the first step to use Procrastinate with Django.

```bash
(venv) $ pip install 'procrastinate[django]'

```

--------------------------------

### Install pre-commit hooks

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Installs pre-commit hooks to automatically check code before committing, preventing commits with linting problems.

```bash
$ pre-commit install

```

--------------------------------

### Launch PostgreSQL database with Docker

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Starts the PostgreSQL database container using Docker Compose. The database is a fresh standard out-of-the-box instance on the latest stable version.

```bash
$ docker compose up -d postgres

```

--------------------------------

### PROCRASTINATE_ON_APP_READY callback implementation

Source: https://procrastinate.readthedocs.io/en/stable/howto/django/settings.html

Example implementation of the on_app_ready callback in myapp/procrastinate.py, which receives the Procrastinate app instance and can add tasks from a blueprint.

```python
# myapp/procrastinate.py
import procrastinate


def on_app_ready(app: procrastinate.App):
    app.add_tasks_from(some_blueprint)
```

--------------------------------

### Define tasks with default parameters (omit parentheses)

Source: https://procrastinate.readthedocs.io/en/stable/howto/basics/tasks.html

Shows how to define sync and async tasks without parentheses after @app.task when using all default parameters. Both examples are shown together as alternatives.

```python
@app.task
def mytask(argument, other_argument): ...
```

```python
@app.task
async def mytask(argument, other_argument): ...
```

--------------------------------

### Run sync demo worker

Source: https://procrastinate.readthedocs.io/en/stable/demos.html

Starts the Procrastinate worker for the sync demo. Set the PROCRASTINATE_APP environment variable to the sync demo's app before running the CLI.

```shell
$ PROCRASTINATE_APP=procrastinate.demos.demo_sync.app.app procrastinate worker

```

--------------------------------

### Start Procrastinate worker in detached mode

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Starts a Procrastinate worker container in detached mode.

```bash
$ docker compose up -d procrastinate

```

--------------------------------

### Defer multiple jobs with tutorial.py

Source: https://procrastinate.readthedocs.io/en/stable/quickstart.html

Defers several jobs by running the tutorial script multiple times with different arguments. This is done before launching a worker to see the worker in action.

```bash
(venv) $ python tutorial.py 5 7
(venv) $ python tutorial.py 3 9
(venv) $ python tutorial.py 1 2

```

--------------------------------

### Build and browse documentation

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Builds the HTML documentation and opens it in the browser. Use scripts/docs to build and scripts/htmldoc to browse.

```bash
$ scripts/docs  # build the html doc
$ scripts/htmldoc  # browse the doc in you browser

```

--------------------------------

### Example migration script

Source: https://procrastinate.readthedocs.io/en/stable/howto/production/migrations.html

Example of a Procrastinate migration script. It is a pure-SQL script that can be applied to the database using any PostgreSQL client, such as psql or PGAdmin.

```sql
ALTER TABLE procrastinate_jobs ADD COLUMN extra TEXT;

```

--------------------------------

### App constructor

Source: https://procrastinate.readthedocs.io/en/stable/reference.html

Instantiate the main App object with a connector and optional settings.

```APIDOC
## App

### Description
Instantiate the main entry point for Procrastinate integration. Use it to decorate tasks and run workers.

### Constructor
`App(*, connector: BaseConnector, import_paths: Iterable[str] | None = None, worker_defaults: WorkerOptions | None = None, periodic_defaults: dict | None = None)`

### Parameters
- **connector** (BaseConnector) - Required - Typically an AiopgConnector. Responsible for all database communications.
- **import_paths** (Iterable[str] | None) - Optional - List of python dotted paths of modules to import, to ensure workers know about all possible tasks.
- **worker_defaults** (WorkerOptions | None) - Optional - Overrides default values sent when launching a worker.
- **periodic_defaults** (dict | None) - Optional - Parameters for fine tuning periodic tasks deferrer. Available parameters: `max_delay` (float, in seconds, defaults to 10 minutes).
```

--------------------------------

### Start the procrastinate shell

Source: https://procrastinate.readthedocs.io/en/stable/howto/production/monitoring.html

Start the interactive procrastinate shell with the `procrastinate shell` command. The shell is an experimental feature and provides commands like `help`, `list_jobs`, `list_tasks`, `list_queues`, `list_locks`, `retry`, and `cancel`.

```bash
$ procrastinate shell
Welcome to the procrastinate shell.   Type help or ? to list commands.

procrastinate> help

Documented commands (type help <topic>):
========================================
EOF  cancel  exit  help  list_locks  list_jobs  list_queues  list_tasks  retry

```

--------------------------------

### Run sync demo application

Source: https://procrastinate.readthedocs.io/en/stable/demos.html

Runs the sync demo application main entrypoint. Defer jobs by sending commands as indicated by the application.

```shell
$ python -m procrastinate.demos.demo_sync

```

--------------------------------

### Run the demo main file

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Runs the demo main file inside the container.

```bash
$ docker compose run --rm procrastinate python -m procrastinate_demo

```

--------------------------------

### Run a worker with procrastinate --verbose --app=tutorial.app worker

Source: https://procrastinate.readthedocs.io/en/stable/quickstart.html

Starts a Procrastinate worker on all queues. The output shows the worker starting, processing a job, and the job succeeding with a result of 5. Stop the worker with ctrl+c.

```shell
(venv) $ procrastinate --verbose --app=tutorial.app worker
Launching a worker on all queues
INFO:procrastinate.worker.worker:Starting worker on all queues
INFO:procrastinate.worker.worker:Starting job sum[1](a=2, b=3)
INFO:procrastinate.worker.worker:Job sum[1](a=2, b=3) ended with status: Success, lasted 1.822 s - Result: 5

```

--------------------------------

### Run tutorial.py with arguments 2 and 3

Source: https://procrastinate.readthedocs.io/en/stable/quickstart.html

Runs the tutorial script with arguments 2 and 3. Procrastinate warns that the app is instantiated in the main module, but this is not a problem for the tutorial. The script prints the scheduling message.

```bash
(venv) $ python tutorial.py 2 3

App is instantiated in the main Python module (tutorial.py). See https://procrastinate.readthedocs.io/en/stable/discussions.html#top-level-app
Scheduling computation of 2 + 3

```

--------------------------------

### Run async demo application

Source: https://procrastinate.readthedocs.io/en/stable/demos.html

Runs the async demo application main entrypoint. Defer jobs by sending commands as indicated by the application.

```shell
$ python -m procrastinate.demos.demo_async

```

--------------------------------

### register_worker

Source: https://procrastinate.readthedocs.io/en/stable/reference.html

Asynchronously registers a newly started worker in the database and returns the worker ID.

```APIDOC
## register_worker

### Description
Register a newly started worker (with an initial heartbeat) in the database.

### Method
async function

### Signature
`async register_worker() -> int`

### Parameters
None

### Returns
The ID of the registered worker

### Return Type
int
```

--------------------------------

### Create a job pattern and launch multiple jobs

Source: https://procrastinate.readthedocs.io/en/stable/howto/basics/defer.html

Creates a job pattern with fixed task_kwargs and then defers multiple jobs with different b values. Shows both sync and async defer methods.

```python
pattern = my_task.configure(task_kwargs={"a": 1})

pattern.defer(b=2)
pattern.defer(b=3)
pattern.defer(b=4)
# or
await pattern.defer_async(b=2)
await pattern.defer_async(b=3)
await pattern.defer_async(b=4)

```

--------------------------------

### Run bash in Procrastinate worker container

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Runs a bash shell in the Procrastinate worker container that was just started.

```bash
$ docker compose exec procrastinate bash

```

--------------------------------

### Set up PostgresConnector with DSN or aiopg pool

Source: https://procrastinate.readthedocs.io/en/stable/changelog.html

Shows the new way to set up a PostgresConnector in version 0.7.0. You can either pass a DSN directly or, if you already have an aiopg pool, create the connector and call set_pool() with your pool.

```python
connector = procrastinate.PostgresConnector(
    dsn="postgres://user:password@host:port/dbname"
)
# or if you already have an aiopg pool:
connector = procrastinate.PostgresConnector()
connector.set_pool(my_pool)
```

--------------------------------

### Create and register a Blueprint with a task

Source: https://procrastinate.readthedocs.io/en/stable/reference.html

Shows how to create a Blueprint, declare a task on it with a lock, and register the blueprint on an app with a namespace. The task will be registered as 'cat:path.to.feed_cat'.

```python
# Create blueprint for all tasks related to the cat
cat_blueprint = Blueprint()

# Declare tasks
@cat_blueprint.task(lock="...")
def feed_cat():
    ...

# Register blueprint (will register ``cat:path.to.feed_cat``)
app.add_tasks_from(cat_blueprint, namespace="cat")

```

--------------------------------

### Current function signature

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Current SQL function signature before a change. Used as an example in the backward-compatibility section.

```sql
procrastinate_func(arg1 integer, arg2 text, arg3 timestamp)

```

--------------------------------

### New function signature

Source: https://procrastinate.readthedocs.io/en/stable/contributing.html

Desired new SQL function signature after removing the third argument. Used as an example in the backward-compatibility section.

```sql
procrastinate_func(arg1 integer, arg2 text)

```

--------------------------------

### Configure PsycopgConnector via libpq environment variables

Source: https://procrastinate.readthedocs.io/en/stable/howto/basics/connector.html

Set libpq environment variables (e.g., PGHOST, PGPORT) to configure the connection, then instantiate a PsycopgConnector with no arguments so it reads those variables. The first block shows exporting variables in the shell or defining them just for the process.

```bash
$ export PGHOST=my.database.com  # Either export the variables in your shell
$ PGPORT=5433 python -m myapp  # Or define the variables just for your process
```

```python
import procrastinate
app = procrastinate.App(connector=procrastinate.PsycopgConnector())
```

--------------------------------

### Run Django demo worker

Source: https://procrastinate.readthedocs.io/en/stable/demos.html

Starts the Procrastinate worker for the Django demo using the Django management command. Run this in the second terminal.

```shell
$ procrastinate/demos/demo_django/manage.py procrastinate worker

```

--------------------------------

### Launch a job with sum.defer

Source: https://procrastinate.readthedocs.io/en/stable/quickstart.html

Launches a job by calling the task's defer method with keyword arguments. Opens the app connection, reads two integers from the command line, and schedules the sum task.

```python
import sys

...


def main():
    with app.open():
        a = int(sys.argv[1])
        b = int(sys.argv[2])
        print(f"Scheduling computation of {a} + {b}")
        # Only keyword arguments can be used with defer
        sum.defer(a=a, b=b)  # This is the line that launches a job


if __name__ == "__main__":
    main()

```

--------------------------------

### App.open

Source: https://procrastinate.readthedocs.io/en/stable/reference.html

Open the app synchronously.

```APIDOC
## open

### Description
Open the app synchronously.

### Method
`open(pool_or_engine: Any | None = None) -> App`

### Parameters
- **pool_or_engine** (Any | None) - Optional - Optional pool. Procrastinate can use an existing pool. Connection parameters passed in the constructor will be ignored. In case the SQLAlchemy connector is used, this can be an engine.

### Return Type
App
```

--------------------------------

### Run PostgreSQL container for Procrastinate

Source: https://procrastinate.readthedocs.io/en/stable/quickstart.html

Starts a PostgreSQL container named pg-procrastinate on port 5432 with password 'password'. Use `docker stop pg-procrastinate` to stop it.

```bash
docker run --name pg-procrastinate --detach --rm -p 5432:5432 -e POSTGRES_PASSWORD=password postgres
```

--------------------------------

### App.from_path

Source: https://procrastinate.readthedocs.io/en/stable/reference.html

Create an App object by dynamically loading the object at the given path.

```APIDOC
## from_path

### Description
Create an `App` object by dynamically loading the object at the given path.

### Method
`classmethod from_path(dotted_path: str) -> App`

### Parameters
- **dotted_path** (str) - Required - Dotted path to the object to load (e.g. `mymodule.submodule.procrastinate_app`)

### Return Type
App
```

--------------------------------

### Final tutorial app code

Source: https://procrastinate.readthedocs.io/en/stable/quickstart.html

The complete Python script for the tutorial. It defines a Procrastinate app with a PsycopgConnector, a sum task that sleeps randomly, and a main function that schedules the task via sum.defer().

```python
import random
import sys
import time

from procrastinate import App, PsycopgConnector

app = App(
    connector=PsycopgConnector(
        kwargs={
            "host": "localhost",
            "user": "postgres",
            "password": "password",
        }
    )
)


@app.task(name="sum")
def sum(a, b):
    time.sleep(random.random() * 5)  # Sleep up to 5 seconds
    return a + b


def main():
    with app.open():
        a = int(sys.argv[1])
        b = int(sys.argv[2])
        print(f"Scheduling computation of {a} + {b}")
        sum.defer(a=a, b=b)  # This is the line that launches a job


if __name__ == "__main__":
    main()

```

--------------------------------

### Run async demo worker

Source: https://procrastinate.readthedocs.io/en/stable/demos.html

Starts the Procrastinate worker for the async demo. Set the PROCRASTINATE_APP environment variable to the async demo's app before running the CLI.

```shell
$ PROCRASTINATE_APP=procrastinate.demos.demo_async.app.app procrastinate worker

```

--------------------------------

### Launch a worker with the CLI

Source: https://procrastinate.readthedocs.io/en/stable/howto/basics/worker.html

Launch a worker from the command line. Use --verbose for debug logging, --app to specify the dotted path to your Procrastinate app, and optionally --name to name the worker and list queues to listen to. If no queues are specified, the worker listens to every queue.

```bash
procrastinate --verbose --app=dotted.path.to.app worker [--name=worker-name] [queue [...]]
```
Tokens:
Raw