### Install Dependencies with uv Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Install project dependencies using the uv package manager. Use the 'migration_test' group for migration testing or '--all-extras --all-groups' for a full setup. ```console $ # Minimally: $ uv sync --group migration_test $ # Maximally: $ uv sync --all-extras --all-groups ``` -------------------------------- ### Bootstrap Development Tools Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Install uv and pre-commit if they are not already present in the environment. ```console $ scripts/bootstrap ``` -------------------------------- ### Install PostgreSQL Client Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Install the PostgreSQL client tools required for database management commands. ```console $ # Ubuntu $ sudo apt install postgresql-client $ createdb ``` ```console $ # MacOS $ brew install libpq $ /usr/local/opt/libpq/bin/createdb ``` -------------------------------- ### Start PostgreSQL with Docker Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md Use this command to spin up a PostgreSQL instance for development. ```console $ docker run --name pg-procrastinate --detach --rm -p 5432:5432 -e POSTGRES_PASSWORD=password postgres ``` -------------------------------- ### Setup Virtual Environment and Dependencies Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Create a virtual environment and sync project dependencies using uv. ```console $ uv venv --python=3.{x} # Select the Python version you want to use (replace {x}) $ uv sync # Install the project and its dependencies $ uv run $SHELL # Activate the virtual environment $ exit # Quit the virtual environment ``` -------------------------------- ### Install Procrastinate Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md Install the package within your virtual environment. ```console (venv) $ pip install procrastinate ``` -------------------------------- ### Launch Development Database Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Start the PostgreSQL database container using Docker Compose. ```console $ docker compose up -d postgres ``` -------------------------------- ### Run Sync Demo Application Source: https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/demos/README.md Starts the synchronous demo application. Use this in a separate terminal from the worker. ```console $ python -m procrastinate.demos.demo_sync ``` -------------------------------- ### Setup SyncPsycopgConnector Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/sync_defer.md Instantiate `SyncPsycopgConnector` to enable classic synchronous I/O for deferring tasks. Requires specifying connection details like the host. ```python import procrastinate app = procrastinate.App( connector=procrastinate.SyncPsycopgConnector( host="somehost", ), ) ``` -------------------------------- ### Example SQL Migration Script Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/migrations.md This is an example of a pure SQL migration script that can be applied to the Procrastinate database. It demonstrates adding a new column to an existing table. ```sql ALTER TABLE procrastinate_jobs ADD COLUMN extra TEXT; ``` -------------------------------- ### Run Django Migrations and Server Source: https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/demos/README.md Applies database migrations and starts the Django development server for the Django demo. ```console $ procrastinate/demos/demo_django/manage.py migrate $ procrastinate/demos/demo_django/manage.py runserver ``` -------------------------------- ### Install Procrastinate Sphinx support Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/sphinx.md Install the package with the sphinx extra to ensure necessary dependencies are available. ```bash $ pip install procrastinate[sphinx] ``` -------------------------------- ### Run Async Demo Application Source: https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/demos/README.md Starts the asynchronous demo application. Use this in a separate terminal from the worker. ```console $ python -m procrastinate.demos.demo_async ``` -------------------------------- ### Define and Run Synchronous Tasks with Procrastinate Source: https://github.com/procrastinate-org/procrastinate/blob/main/README.md Example of defining a synchronous task, deferring its execution, and running a worker. Ensure the `procrastinate` library is installed and a PostgreSQL database is configured. ```python import procrastinate # Make an app in your code app = procrastinate.App(connector=procrastinate.SyncPsycopgConnector()) # Then define tasks @app.task(queue="sums") def sum(a, b): with open("myfile", "w") as f: f.write(str(a + b)) with app.open(): # Launch a job sum.defer(a=3, b=5) # Somewhere in your program, run a worker (actually, it's usually a # different program than the one deferring jobs for execution) app.run_worker(queues=["sums"]) ``` -------------------------------- ### Integration Test with TransactionTestCase and Worker Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/tests.md This example shows how to run integration tests using Django's TransactionTestCase. It defers a task, starts a worker with specific configurations, and then asserts the task's successful execution. ```python from procrastinate.contrib.django import app from django.test import TransactionTestCase from mypackage.procrastinate import my_task class TestingTaskClass(TransactionTestCase): def test_task(self): # Run tasks my_task.defer(a=1, b=2) # Start worker with app.replace_connector(app.connector.get_worker_connector()): app.run_worker(wait=False, install_signal_handlers=False, listen_notify=False) # Check task has been executed assert ProcrastinateJob.objects.filter(task_name="my_task").status == "succeeded" ``` -------------------------------- ### Install Pre-commit Hooks Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Configure pre-commit to run automated linting checks before commits. ```console $ pre-commit install ``` -------------------------------- ### SQLAlchemy example Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/external_connection.md Use this to defer jobs atomically with other database writes when using SQLAlchemy with psycopg2. ```python from procrastinate import App from procrastinate.contrib.sqlalchemy import SQLAlchemyPsycopg2Connector connector = SQLAlchemyPsycopg2Connector(dsn="postgresql+psycopg2:///mydb") app = App(connector=connector) app.open() @app.task def process_order(order_id): ... with connector.engine.connect() as conn: conn.exec_driver_sql("INSERT INTO orders (id) VALUES (%s)", [42]) process_order.configure(connection=conn).defer(order_id=42) conn.commit() ``` -------------------------------- ### Sync example with psycopg Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/external_connection.md Use this when you need to defer a job atomically with other database writes in a synchronous transaction using psycopg. ```python import psycopg from procrastinate import App, SyncPsycopgConnector app = App(connector=SyncPsycopgConnector(conninfo="...")) app.open() @app.task def process_order(order_id): ... with psycopg.connect("...") as conn: conn.autocommit = False conn.execute("INSERT INTO orders (id, ...) VALUES (%s, ...)", [42]) process_order.configure(connection=conn).defer(order_id=42) conn.commit() # both the order row and the job are committed together ``` -------------------------------- ### Run a Worker Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md Start the worker to consume jobs from the queue. ```console (venv) $ procrastinate --verbose --app=tutorial.app worker Launching a worker on all queues INFO:procrastinate.worker.worker:Starting worker on all queues INFO:procrastinate.worker.worker:Starting job sum[1](a=2, b=3) INFO:procrastinate.worker.worker:Job sum[1](a=2, b=3) ended with status: Success, lasted 1.822 s - Result: 5 ``` -------------------------------- ### Setup SQLAlchemyPsycopg2Connector Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/sync_defer.md Use `SQLAlchemyPsycopg2Connector` when integrating Procrastinate with an existing SQLAlchemy application. This allows Procrastinate to share the SQLAlchemy engine and connection pool, minimizing database connections. Ensure SQLAlchemy is installed and an engine is created before initializing the connector and opening the app. ```python from sqlalchemy import create_engine from procrastinate import App from procrastinate.contrib.sqlalchemy import SQLAlchemyPsycopg2Connector engine = create_engine("postgresql+psycopg2://", echo=True) app = App(connector=SQLAlchemyPsycopg2Connector()) app.open(engine) ``` -------------------------------- ### Async example with psycopg Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/external_connection.md Use this for deferring jobs atomically with other database writes in an asynchronous transaction using psycopg. ```python import psycopg from procrastinate import App, PsycopgConnector app = App(connector=PsycopgConnector(conninfo="...")) await app.open_async() @app.task def process_order(order_id): ... async with await psycopg.AsyncConnection.connect("...") as conn: conn.autocommit = False await conn.execute("INSERT INTO orders (id, ...) VALUES (%s, ...)", [42]) await process_order.configure(connection=conn).defer_async(order_id=42) await conn.commit() ``` -------------------------------- ### Install Procrastinate with Django support Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/configuration.md Install Procrastinate along with its Django integration package using pip. This command ensures all necessary dependencies for Django support are included. ```console (venv) $ pip install 'procrastinate[django]' ``` -------------------------------- ### Pytest Integration Test with Worker Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/tests.md This example uses pytest with the django_db marker to run an integration test. It defers a task, starts a worker, and verifies the task's status. The `transaction=True` argument is crucial for `SELECT FOR UPDATE`. ```python from procrastinate.contrib.django import app from mypackage.procrastinate import my_task @pytest.mark.django_db(transaction=True) def test_task(): # Run tasks my_task.defer(a=1, b=2) # Start worker with app.replace_connector(app.connector.get_worker_connector()): app.run_worker(wait=False, install_signal_handlers=False, listen_notify=False) # Check task has been executed assert ProcrastinateJob.objects.filter(task_name="my_task").status == "succeeded" ``` -------------------------------- ### Launch worker via Python Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/worker.md Start a worker programmatically using synchronous or asynchronous methods. ```python app.run_worker(queues=["queue", ...], name="worker-name") # or await app.run_worker_async(queues=["queue", ...], name="worker-name") ``` -------------------------------- ### Launch worker via CLI Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/worker.md Use the command line interface to start a worker with optional name and queue filtering. ```console $ procrastinate --verbose --app=dotted.path.to.app worker [--name=worker-name] [queue [...]] ``` -------------------------------- ### Define a task Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/defer.md The base task definition used for subsequent deferral examples. ```python @app.task(queue="some_queue") def my_task(a: int, b:int): pass ``` -------------------------------- ### Apply Single Pre Migration Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/migrations.md Example of applying a single 'pre' migration script using psql. This is part of the safer method for applying migrations without service interruption. ```console $ MIGRATION_TO_APPLY="02.00.00_01_pre_some_migration.sql" $ cat $(procrastinate schema --migrations-path)/${MIGRATION_TO_APPLY} | psql ``` -------------------------------- ### Procrastinate App Task Example Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md A Python script demonstrating how to define a task and schedule it using Procrastinate. Ensure the database connection details are correct. ```python import random import sys import time from procrastinate import App, PsycopgConnector app = App( connector=PsycopgConnector( kwargs={ "host": "localhost", "user": "postgres", "password": "password", } ) ) @app.task(name="sum") def sum(a, b): time.sleep(random.random() * 5) # Sleep up to 5 seconds return a + b def main(): with app.open(): a = int(sys.argv[1]) b = int(sys.argv[2]) print(f"Scheduling computation of {a} + {b}") sum.defer(a=a, b=b) # This is the line that launches a job if __name__ == "__main__": main() ``` -------------------------------- ### List Migrations Command Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/migrations.md Use this command to list all available migration scripts within your Procrastinate installation. The output shows the path to the migration directory. ```console $ procrastinate schema --migrations-path /home/me/my_venv/lib/python3.x/site-packages/procrastinate/sql/migrations ``` -------------------------------- ### Apply Single Post Migration Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/migrations.md Example of applying a single 'post' migration script using psql. This is part of the safer method for applying migrations without service interruption. ```console $ MIGRATION_TO_APPLY="02.00.00_01_post_some_migration.sql" $ cat $(procrastinate schema --migrations-path)/${MIGRATION_TO_APPLY} | psql ``` -------------------------------- ### Integration Test with Database Job Check Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/tests.md This example demonstrates an integration test where you defer a task and then check its creation in the database using Django's ORM. It assumes Procrastinate models are available. ```python from procrastinate.contrib.django.models import ProcrastinateJob from mypackage.procrastinate import my_task def test_my_task(): # Run the task my_task.defer(a=1, b=2) # Check the job has been created assert ProcrastinateJob.objects.filter(task_name="my_task").count() == 1 ``` -------------------------------- ### Define task loading function Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/configuration.md Implement the function specified in PROCRASTINATE_ON_APP_READY to load tasks. This example shows how to add tasks from a blueprint using app.add_tasks_from(). ```python # myapp/procrastinate.py import procrastinate def on_app_ready(app: procrastinate.App): app.add_tasks_from(some_blueprint) ``` -------------------------------- ### Define and Run Asynchronous Tasks with Procrastinate Source: https://github.com/procrastinate-org/procrastinate/blob/main/README.md Example of defining an asynchronous task using coroutines, deferring its execution asynchronously, and running an asynchronous worker. This is the recommended approach for using Procrastinate. ```python import asyncio import procrastinate # Make an app in your code app = procrastinate.App(connector=procrastinate.PsycopgConnector()) # Define tasks using coroutine functions @app.task(queue="sums") async def sum(a, b): await asyncio.sleep(a + b) async with app.open_async(): # Launch a job await sum.defer_async(a=3, b=5) # Somewhere in your program, run a worker (actually, it's often a # different program than the one deferring jobs for execution) await app.run_worker_async(queues=["sums"]) ``` -------------------------------- ### Define Custom JSON Encoders and Decoders for Datetime Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/custom_json_encoder_decoder.md This example demonstrates how to define custom functions for encoding datetime objects to ISO format and decoding them back. It then integrates these functions into Procrastinate's App configuration using functools.partial to create custom json_dumps and json_loads arguments for the AiopgConnector. ```python import functools import json import datetime from procrastinate import App, AiopgConnector # Function used for encoding datetime objects def encode(obj): if isinstance(obj, datetime.datetime): return obj.isoformat() raise TypeError() # Function used for decoding datetime objects def decode(dict_): if "dt" in dict_: dict_["dt"] = datetime.datetime.fromisoformat(dict_["dt"]) return dict_ json_dumps = functools.partial(json.dumps, default=encode) json_loads = functools.partial(json.loads, object_hook=decode) app = App(connector=AiopgConnector(json_dumps=json_dumps, json_loads=json_loads)) ``` -------------------------------- ### Build and Browse Documentation Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Commands to build the project documentation and open it in a browser. ```console $ scripts/docs # build the html doc $ scripts/htmldoc # browse the doc in you browser ``` -------------------------------- ### Start Procrastinate Worker in Docker Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Start a Procrastinate worker service in detached mode using docker compose up. The '-d' flag runs the container in the background. ```console $ docker compose up -d procrastinate ``` -------------------------------- ### Apply Database Schema Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md Initialize the required database tables for Procrastinate. ```console (venv) $ export PYTHONPATH=. # required for procrastinate to find "tutorial.app" (venv) $ procrastinate --app=tutorial.app schema --apply ``` -------------------------------- ### Create a job pattern for multiple launches Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/defer.md Configure a pattern once and reuse it to launch multiple jobs with shared arguments. ```python pattern = my_task.configure(task_kwargs={"a": 1}) pattern.defer(b=2) pattern.defer(b=3) pattern.defer(b=4) # or await pattern.defer_async(b=2) await pattern.defer_async(b=3) await pattern.defer_async(b=4) ``` -------------------------------- ### Initialize Development Environment Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Use the dev-env script to automatically configure the shell environment and dependencies. ```console $ source ./dev-env ``` -------------------------------- ### Launch the administration shell Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/monitoring.md Opens an interactive shell for managing jobs, queues, and tasks. ```console $ procrastinate shell Welcome to the procrastinate shell. Type help or ? to list commands. procrastinate> help Documented commands (type help ): ======================================== EOF cancel exit help list_locks list_jobs list_queues list_tasks retry ``` -------------------------------- ### Launch a Job Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md Use the defer method to schedule a task execution. ```python import sys ... def main(): with app.open(): a = int(sys.argv[1]) b = int(sys.argv[2]) print(f"Scheduling computation of {a} + {b}") # Only keyword arguments can be used with defer sum.defer(a=a, b=b) # This is the line that launches a job if __name__ == "__main__": main() ``` -------------------------------- ### Configure Connector via Environment Variables Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/connector.md Use standard libpq environment variables for database connection configuration. ```console $ export PGHOST=my.database.com # Either export the variables in your shell $ PGPORT=5433 python -m myapp # Or define the variables just for your process ``` ```python import procrastinate app = procrastinate.App(connector=procrastinate.PsycopgConnector()) ``` -------------------------------- ### Delete jobs via CLI Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/delete_finished_jobs.md Use the CLI to set the delete-jobs flag when starting a worker. ```bash procrastinate worker --delete-jobs=always ``` -------------------------------- ### Configure Database Environment Variables Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Set standard libpq environment variables to interact with the local PostgreSQL database. ```console $ export PGDATABASE=procrastinate PGHOST=localhost PGUSER=postgres PGPASSWORD=password ``` -------------------------------- ### Initialize Procrastinate App Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md Create the application object with a PsycopgConnector. ```python from procrastinate import App, PsycopgConnector app = App( connector=PsycopgConnector( kwargs={ "host": "localhost", "user": "postgres", "password": "password", } ) ) ``` -------------------------------- ### Apply All Migrations with Service Interruption Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/migrations.md This console command demonstrates applying both 'pre' and 'post' migration scripts sequentially using psql. This method requires shutting down services that use Procrastinate. ```console $ MIGRATION_TO_APPLY="02.00.00_01_pre_some_migration.sql" $ cat $(procrastinate schema --migrations-path)/${MIGRATION_TO_APPLY} | psql $ MIGRATION_TO_APPLY="02.00.00_01_post_some_migration.sql" $ cat $(procrastinate schema --migrations-path)/${MIGRATION_TO_APPLY} | psql $ ... ``` -------------------------------- ### Run Sync Demo Worker Source: https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/demos/README.md Launches the Procrastinate worker for the synchronous demo. Ensure PROCRASTINATE_APP is set correctly. ```console $ PROCRASTINATE_APP=procrastinate.demos.demo_sync.app.app procrastinate worker ``` -------------------------------- ### Integrate worker in FastAPI Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/worker.md Example of running a worker within a FastAPI lifespan context, disabling signal handlers to prevent interference. ```python import asyncio import logging from contextlib import asynccontextmanager from fastapi import FastAPI from procrastinate import App, PsycopgConnector logging.basicConfig(level=logging.DEBUG) task_queue = App(connector=PsycopgConnector()) @task_queue.task async def sleep(length): await asyncio.sleep(length) @asynccontextmanager async def lifespan(app: FastAPI): async with task_queue.open_async(): worker = asyncio.create_task( task_queue.run_worker_async(install_signal_handlers=False) ) # Set to 100 to test the ungraceful shutdown await sleep.defer_async(length=5) print("STARTUP") yield print("SHUTDOWN") worker.cancel() try: await asyncio.wait_for(worker, timeout=10) except asyncio.TimeoutError: print("Ungraceful shutdown") except asyncio.CancelledError: print("Graceful shutdown") app = FastAPI(lifespan=lifespan) @app.get("/") async def root(): return {"Hello": "World"} ``` -------------------------------- ### Schedule a job at a specific time via CLI Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/schedule.md Uses the --at flag with an ISO 8601 formatted string to schedule a job. ```console $ procrastinate defer \ --at=2038-01-19T03:14:07Z \ path.to.create_bug '{"crash_everything": true}' ``` -------------------------------- ### Configure Connector via DSN Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/connector.md Pass a libpq connection string to the connector. ```python import procrastinate app = procrastinate.App( connector=procrastinate.PsycopgConnector( conninfo="postgres://user:password@host:port/dbname" ) ) ``` -------------------------------- ### Open and close the app with a context manager Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/open_connection.md Automatically handle connection termination by using a context manager. ```python with app.open(): pass ``` ```python async with app.open_async(): pass ``` -------------------------------- ### Remove Old Version of a Function Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md In post-migrations, remove the old SQL function to complete the schema update. This example shows dropping the original 'procrastinate_func'. ```sql -- xx_xx_xx_50_post_remove_old_version_procrastinate_func.sql DROP FUNCTION procrastinate_func(integer, text, timestamp); ... ``` -------------------------------- ### Verify Procrastinate configuration Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/basic_usage.md Run healthchecks to ensure the database connection, migrations, and app configuration are correct. ```console (venv) $ ./manage.py procrastinate healthchecks Database connection: OK Migrations: OK Default Django Procrastinate App: OK Worker App: OK ``` -------------------------------- ### Run Async Demo Worker Source: https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/demos/README.md Launches the Procrastinate worker for the asynchronous demo. Ensure PROCRASTINATE_APP is set correctly. ```console $ PROCRASTINATE_APP=procrastinate.demos.demo_async.app.app procrastinate worker ``` -------------------------------- ### Defer tasks via command line Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/defer.md Use the CLI to defer tasks, with support for unknown tasks via flags or environment variables. ```console $ procrastinate defer my_module.my_task '{"a": 1, "b": 2}' ``` ```console $ procrastinate defer --unknown my_module.my_task '{"a": 1, "b": 2}' $ # or $ export PROCRASTINATE_DEFER_UNKNOWN=1 $ procrastinate defer my_module.my_task '{"a": 1, "b": 2}' ``` -------------------------------- ### Create New Version of a Function Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md When modifying existing SQL functions, create a new version in pre-migrations to support older Python code. This example shows adding 'procrastinate_func_v4'. ```sql -- xx_xx_xx_01_pre_add_new_version_procrastinate_func.sql CREATE FUNCTION procrastinate_func_v4(arg1 integer, arg2 text) RETURNS INT ... ``` -------------------------------- ### Add a book link Source: https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/demos/demo_django/demo/templates/demo/book_list.html Provides a link to the create_book view. ```django-template [Add a Book]({% url 'create_book' %}) ``` -------------------------------- ### Mark a Running Job for Abortion Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/cancellation.md To mark a job that is currently running for abortion, use the `abort=True` option. This behaves like cancellation if the job hasn't started processing. Both sync and async methods are available. ```python app.job_manager.cancel_job_by_id(33, abort=True) ``` ```python await app.job_manager.cancel_job_by_id_async(33, abort=True) ``` -------------------------------- ### Configure Connector via Connection Arguments Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/connector.md Pass specific connection parameters as a dictionary to the connector. ```python import procrastinate procrastinate.PsycopgConnector( kwargs={ "dbname": "dbname", "user": "user", "password": "password", "host": "host", } ) ``` -------------------------------- ### Execute Script Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md Run the script to defer a job. ```console (venv) $ python tutorial.py 2 3 App is instantiated in the main Python module (tutorial.py). See https://procrastinate.readthedocs.io/en/stable/discussions.html#top-level-app Scheduling computation of 2 + 3 ``` -------------------------------- ### Set PROCRASTINATE_ON_APP_READY in settings.py Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/settings.md Configure the `PROCRASTINATE_ON_APP_READY` setting in your `settings.py` file to specify a dotted path to a function that will be called when the Procrastinate app is ready. This is useful for loading additional tasks. ```python # settings.py PROCRASTINATE_ON_APP_READY = "myapp.procrastinate.on_app_ready" ``` -------------------------------- ### Configure task parameters before deferral Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/defer.md Use configure to set task-specific options like locks, queues, or scheduling before deferring. ```python my_task.configure( lock="the name of my lock", schedule_in={"hours": 1}, queue="not_the_default_queue" ).defer(a=1, b=2) # or await my_task.configure( lock="the name of my lock", schedule_in={"hours": 1}, queue="not_the_default_queue" ).defer_async(a=1, b=2) ``` -------------------------------- ### Run health checks via CLI Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/monitoring.md Verifies application configuration, database connectivity, and table existence. ```console $ procrastinate healthchecks App configuration: OK DB connection: OK Found procrastinate_jobs table: OK ``` -------------------------------- ### Initialize Procrastinate App with import_paths Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/app.md Use the import_paths parameter to specify the dotted paths to modules containing task definitions when the app is in a dedicated module. ```python import procrastinate app = procrastinate.App( connector=connector, import_paths=["dotted.path.to", "all.the.modules", "that.define.tasks"] ) ``` -------------------------------- ### Configure basic Procrastinate logging Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/logs.md Sets up a standard logging configuration for the procrastinate logger using a StreamHandler. ```python LOGGING = { "version": 1, "formatters": { "procrastinate": { "format": "%(asctime)s %(levelname)-7s %(name)s %(message)s" }, }, "handlers": { "procrastinate": { "level": "DEBUG", "class": "logging.StreamHandler", "formatter": "procrastinate", }, }, "loggers": { "procrastinate": { "handlers": ["procrastinate"], "level": "DEBUG", "propagate": False, }, }, } ``` -------------------------------- ### Run Procrastinate migrations in Django Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/migrations.md Execute this command to apply Procrastinate migrations to the database. ```bash ./manage.py migrate ``` -------------------------------- ### Creating a Procrastinate Blueprint Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/blueprints.md Instantiate a Blueprint and define tasks using the standard @blueprint.task() decorator. This is the first step in modularizing your tasks. ```python from procrastinate import Blueprint my_blueprint = Blueprint() @my_blueprint.task() def mytask(argument, other_argument): ... ``` -------------------------------- ### Run Procrastinate Demo Main File in Docker Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Execute the main Python file for the Procrastinate demo application within a Docker container. ```console $ docker compose run --rm procrastinate python -m procrastinate_demo ``` -------------------------------- ### Run Django Demo Worker Source: https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/demos/README.md Launches the Procrastinate worker specifically for the Django demo application. ```console $ procrastinate/demos/demo_django/manage.py procrastinate worker ``` -------------------------------- ### Configure structlog for uniform logging Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/logging.md Set up structlog processors and formatters to ensure Procrastinate logs match the application's standard output format. ```python shared_processors = [ structlog.contextvars.merge_contextvars, structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.processors.StackInfoRenderer(), structlog.dev.set_exc_info, ] structlog.configure( processors=shared_processors + [structlog.stdlib.ProcessorFormatter.wrap_for_formatter], logger_factory=structlog.stdlib.LoggerFactory(), cache_logger_on_first_use=True, ) formatter = structlog.stdlib.ProcessorFormatter( foreign_pre_chain=shared_processors, processors=[ structlog.stdlib.ProcessorFormatter.remove_processors_meta, structlog.dev.ConsoleRenderer(event_key="message"), ], ) handler = logging.StreamHandler() handler.setFormatter(formatter) root = logging.getLogger() root.addHandler(handler) root.setLevel(log_level) ``` -------------------------------- ### Build Procrastinate Docker Image Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Build the Procrastinate Docker image using docker compose. Ensure UID and GID environment variables are set to match the host user. ```console $ export UID GID $ docker compose build procrastinate ``` -------------------------------- ### Lock Dependencies with uv Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Use this command to recompute the lockfile for development dependencies when needed. ```console $ uv lock ``` -------------------------------- ### Configure Log Format Style Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/command_line.md Use the --log-format-style or PROCRASTINATE_LOG_FORMAT_STYLE parameters to choose different styles for the log format, such as '{' or '$'. ```console --log-format-style= PROCRASTINATE_LOG_FORMAT_STYLE= ``` -------------------------------- ### Defer and Run Tasks via Procrastinate CLI Source: https://github.com/procrastinate-org/procrastinate/blob/main/README.md Demonstrates how to defer a task and run a worker using the Procrastinate command-line interface. Set the PROCRASTINATE_APP environment variable to point to your application instance. ```bash export PROCRASTINATE_APP="mycode.app" # Launch a job procrastinate defer mycode.sum '{"a": 3, "b": 5}' # Run a worker procrastinate worker -q sums ``` -------------------------------- ### Specify App with --app parameter Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/command_line.md When using the Procrastinate CLI, specify your application using the '--app' parameter followed by the dotted path to your app. ```console $ procrastinate --app=dotted.path.to.app worker ``` -------------------------------- ### Verify Healthchecks Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md Check the configuration and database connection status. ```console (venv) $ procrastinate --app=tutorial.app healthchecks App configuration: OK DB connection: OK Found procrastinate_jobs table: OK ``` -------------------------------- ### Explicitly open and close the app Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/open_connection.md Manually manage the connection pool lifecycle using open and close methods. ```python app.open() ... app.close() ``` ```python await app.open_async() ... await app.close_async() ``` -------------------------------- ### Configure Log Format Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/command_line.md Use the --log-format or PROCRASTINATE_LOG_FORMAT parameters to control how log lines are formatted. It uses %-style placeholders by default. ```console --log-format= PROCRASTINATE_LOG_FORMAT= ``` -------------------------------- ### Iterate and display book list Source: https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/demos/demo_django/demo/templates/demo/book_list.html Uses a for loop to render book titles, authors, and indexing status from the book_list context variable. ```django-template {% for book in book_list %}* {{ book.title }} by {{ book.author }} - Indexed: {{ book.indexed | yesno}} {% endfor %} ``` -------------------------------- ### Define on_app_ready function in myapp/procrastinate.py Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/settings.md Implement the `on_app_ready` function in your specified module. This function receives the Procrastinate app instance and can be used to add tasks from blueprints or perform other app modifications. ```python import procrastinate def on_app_ready(app: procrastinate.App): app.add_tasks_from(some_blueprint) ``` -------------------------------- ### Implement SET NULL Migration for Job Deletion Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/related.md Apply a custom migration to set the foreign key to null when a job is deleted externally. ```python class Migration(migrations.Migration): operations = [ # ... other migrations migrations.RunSQL( sql=""" ALTER TABLE app_name_mymodel ADD CONSTRAINT app_name_mymodel_job_id_key FOREIGN KEY (procrastinate_job_id) REFERENCES procrastinate_jobs(id) ON DELETE SET NULL; """ reverse_sql=""" ALTER TABLE app_name_mymodel DROP CONSTRAINT app_name_mymodel_job_id_key; """ ), ] ``` -------------------------------- ### Run Procrastinate worker manually in Django Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/scripts.md Requires manual django.setup() and replacing the default connector with one suitable for worker processes. ```python # myapp/worker.py import django from procrastinate.contrib.django import app def main(): django.setup() # By default, the app uses the Django database connection, which is unsuitable # for the worker. with app.replace_connector(app.connector.get_worker_connector()): app.run_worker() if __name__ == "__main__": main() ``` -------------------------------- ### Declare a Task Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/quickstart.md Define a task using the @app.task decorator. ```python # at the top of the file import random import time ... # at the bottom of the file @app.task(name="sum") def sum(a, b): time.sleep(random.random() * 5) # Sleep up to 5 seconds return a + b ``` -------------------------------- ### Schedule a job with a delay via CLI Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/schedule.md Uses the --in flag with a duration in seconds to schedule a job. ```console $ procrastinate defer --in=5400 path.to.clean ``` -------------------------------- ### Define Simple Retry Strategies Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/retry.md Configure tasks to retry a fixed number of times or indefinitely upon failure. ```python @app.task(retry=5) def flaky_task(): if random.random() > 0.9: raise Exception("Who could have seen this coming?") print("Hello world") ``` ```python @app.task(retry=True) def flaky_task(): if random.random() > 0.9: raise Exception("Who could have seen this coming?") print("Hello world") ``` -------------------------------- ### Run Migration Tests with Pytest Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Execute migration tests specifically using pytest. Ensure you are in the correct virtual environment. ```console (venv) $ pytest tests/migration ``` -------------------------------- ### Create Django Superuser Source: https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/demos/README.md Creates a superuser account for accessing the Django admin interface. ```console $ procrastinate/demos/demo_django/manage.py createsuperuser ``` -------------------------------- ### Batch defer with task configuration Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/batch_defer.md Apply task configuration like locks, schedules, or custom queues before batch deferring jobs. Avoid using locks with batch deferral to prevent AlreadyEnqueued exceptions. ```python my_task.configure( lock="the name of my lock", schedule_in={"hours": 1}, queue="not_the_default_queue" ).batch_defer( {"a": 1, "b": 2}, {"a": 3, "b": 4}, {"a": 5, "b": 6}, ) # or await my_task.configure( lock="the name of my lock", schedule_in={"hours": 1}, queue="not_the_default_queue" ).batch_defer_async( {"a": 1, "b": 2}, {"a": 3, "b": 4}, {"a": 5, "b": 6}, ) ``` -------------------------------- ### Implement CASCADE Migration for Job Deletion Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/django/related.md Apply a custom migration to cascade deletions to related records when a job is removed. ```python class Migration(migrations.Migration): operations = [ # ... other migrations migrations.RunSQL( sql=""" ALTER TABLE app_name_mymodel ADD CONSTRAINT app_name_mymodel_job_id_key FOREIGN KEY (procrastinate_job_id) REFERENCES procrastinate_jobs(id) ON DELETE CASCADE; ", reverse_sql=""" ALTER TABLE app_name_mymodel DROP CONSTRAINT app_name_mymodel_job_id_key; """ ), ] ``` -------------------------------- ### Configure Procrastinate with a Custom Schema Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/schema.md Use the `PsycopgConnector` to specify a custom PostgreSQL schema by setting the `search_path` option. This ensures Procrastinate creates and interacts with database objects in the designated schema. ```python app = procrastinate.App( connector=procrastinate.PsycopgConnector( kwargs={ "host": "localhost", "options": "-c search_path=myschema", }, ) ) ``` -------------------------------- ### Defer Task with Priority via CLI Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/priorities.md Defer a task with a specified priority using the command line interface. This is useful for scripting or manual task deferral. ```console $ procrastinate defer --priority=5 path.to.my_task ``` -------------------------------- ### View Test Coverage Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Generate and view the HTML test coverage report. ```console $ scripts/htmlcov ``` -------------------------------- ### Safer Migration Application Workflow Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/production/migrations.md This console command sequence illustrates the safer method for applying migrations without service interruption. It involves applying migrations incrementally and deploying code updates between migration steps. ```console $ MIGRATION_TO_APPLY="02.01.00_01_pre_some_migration.sql" $ cat $(procrastinate schema --migrations-path)/${MIGRATION_TO_APPLY} | psql $ yoursystem/deploy procrastinate 2.1.0 $ MIGRATION_TO_APPLY="02.01.00_01_post_some_migration.sql" $ cat $(procrastinate schema --migrations-path)/${MIGRATION_TO_APPLY} | psql $ MIGRATION_TO_APPLY="02.02.00_01_pre_some_migration.sql" $ cat $(procrastinate schema --migrations-path)/${MIGRATION_TO_APPLY} | psql $ yoursystem/deploy procrastinate 2.2.0 $ MIGRATION_TO_APPLY="02.02.00_01_post_some_migration.sql" $ cat $(procrastinate schema --migrations-path)/${MIGRATION_TO_APPLY} | psql ``` -------------------------------- ### Launch Procrastinate CLI Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/basics/command_line.md The command-line tool can be launched using either the 'procrastinate' command or by running it as a Python module. ```console $ procrastinate ``` ```console $ python -m procrastinate ``` -------------------------------- ### Reuse job configuration for multiple deferrals Source: https://github.com/procrastinate-org/procrastinate/blob/main/docs/howto/advanced/locks.md Store a configured job description to defer multiple tasks with the same lock settings. ```python job_description = my_task.configure(lock=customer.id) job_description.defer(a=1) job_description.defer(a=2) ``` -------------------------------- ### Apply Procrastinate Database Schema in Docker Source: https://github.com/procrastinate-org/procrastinate/blob/main/CONTRIBUTING.md Apply the Procrastinate database schema using the 'schema --apply' command within a Docker container. ```console $ docker compose run --rm procrastinate procrastinate schema --apply ```