### Typical Application User Setup Source: https://github.com/nikolays/pgque/blob/main/README.md Example SQL commands to set up an application user with the `pgque_writer` role, granting them necessary permissions to produce and consume messages. This includes installing PgQue and optionally starting the ticker. ```sql \i sql/pgque.sql select pgque.start(); -- optional pg_cron ticker + maint create user app_orders with password '...'; -- replace with a real password grant pgque_writer to app_orders; create user metrics with password '...'; -- replace with a real password grant pgque_reader to metrics; ``` -------------------------------- ### Install and Start PgQue Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Basic installation and startup commands for PgQue. Assumes pg_cron is installed; otherwise, manual calls to ticker() and maint() are required. ```sql \i sql/pgque.sql select pgque.start() ``` -------------------------------- ### Install pgque using a single SQL file Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md This demonstrates the simplified installation process for pgque, which involves including a single SQL file and optionally starting the pg_cron jobs. ```sql \i pgque.sql SELECT pgque.start(); -- optional: creates pg_cron jobs ``` -------------------------------- ### Install PgQue Source: https://context7.com/nikolays/pgque/llms.txt Install PgQue by running the provided SQL file within a transaction. After installation, verify it by checking the version. If pg_cron is available, start its ticker and maintenance jobs. ```sql begin; \i sql/pgque.sql commit; -- Verify the installation select pgque.version(); -- Returns: [[your version]] -- Start pg_cron ticker and maintenance jobs (if pg_cron is available) select pgque.start(); ``` -------------------------------- ### Start pgque Service Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Initiates the pgque service after installation. This command is used to begin queue operations. ```sql SELECT pgque.start() ``` -------------------------------- ### Go Producer and Consumer Setup Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Example of using `pgque-go` to send messages and set up a consumer. The consumer handles message acknowledgment implicitly on successful return of the handler function, and nacks on error. ```go package main import ( "context" "github.com/pgque/pgque-go" ) func main() { client, _ := pgque.Connect(ctx, "postgresql://localhost/mydb") // Producer client.Send(ctx, "orders", pgque.Event{ Type: "order.created", Payload: Order{ID: 42, Total: 99.95}, }) // Consumer consumer := client.NewConsumer("orders", "order_processor", pgque.WithPollInterval(30 * time.Second), pgque.WithMaxRetries(5), ) consumer.Handle("order.created", func(ctx context.Context, msg pgque.Message) error { var order Order msg.Decode(&order) return processOrder(ctx, order) // return nil = ack, return error = nack with retry }) consumer.Start(ctx) // blocks until context cancelled } ``` -------------------------------- ### Start, Stop, and Status of PgQue Source: https://github.com/nikolays/pgque/blob/main/blueprints/PHASES.md Use these functions to manage the lifecycle of the PgQue system. Ensure PgQue is installed before calling these functions. ```sql pgque.start() ``` ```sql pgque.stop() ``` ```sql pgque.status() ``` -------------------------------- ### Install PgQue SQL Script Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Use this command to install the PgQue SQL script. This is the first step in the tutorial. ```sql \i sql/pgque.sql ``` -------------------------------- ### Installation Source: https://context7.com/nikolays/pgque/llms.txt Instructions for installing PgQue using a single SQL file within a transaction. ```APIDOC ## Installation Install PgQue by running a single SQL file inside a transaction to ensure atomic deployment. ### Request Example ```sql -- Install PgQue in a single transaction begin; \i sql/pgque.sql commit; -- Verify the installation select pgque.version(); -- Returns: [[your version]] -- Start pg_cron ticker and maintenance jobs (if pg_cron is available) select pgque.start(); ``` ``` -------------------------------- ### Start PgQue Jobs Source: https://context7.com/nikolays/pgque/llms.txt Starts the pg_cron scheduled jobs for ticker and maintenance operations. Ensure pg_cron is installed and configured. ```sql -- Start pg_cron jobs (ticker every 1s, maint every 30s) select pgque.start(); ``` -------------------------------- ### Install pgque on Managed PostgreSQL Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Install pgque using a single SQL file and start it with pg_cron. This method avoids the need for DBA access or custom extensions on platforms like RDS, Cloud SQL, or Supabase. ```sql \i pgque.sql select pgque.start() ``` -------------------------------- ### Get PgQue version Source: https://github.com/nikolays/pgque/blob/main/docs/tutorial.md Verify the PgQue installation by querying its version. ```sql select pgque.version(); ``` -------------------------------- ### Install PL/pgSQL Components Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Installs the PL/pgSQL components of pgque. This is a single-file installation that does not require `make` or `CREATE EXTENSION`. ```sql \i structure/tables.sql \i structure/func_internal.sql \i lowlevel_pl/insert_event.sql \i structure/func_public.sql \i structure/triggers_pl.sql \i structure/grants.sql ``` -------------------------------- ### Install PgQue Schema from Shell Source: https://github.com/nikolays/pgque/blob/main/README.md Install the PgQue schema using psql from the command line, ensuring a single-transaction guarantee. ```bash PAGER=cat psql --no-psqlrc --single-transaction -d mydb -f sql/pgque.sql ``` -------------------------------- ### pgque Status Output Example Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md This example shows the typical output of the `pgque status` command, detailing system health, queue statistics, and consumer lag. It includes information about the pgque version, PostgreSQL version, and pg_cron status. ```text pgque v1.0.0 | PostgreSQL 16.2 | pg_cron 1.6 System: Ticker: running (job 42, every 2s, last run 1.2s ago) Maint: running (job 43, every 30s, last run 12s ago) Queues: NAME DEPTH RATE ROTATION CONSUMERS DLQ HEALTH orders 42 150/s 1h23m ago 3 0 ok notifications 831 45/s 0h47m ago 1 12 warning audit 0 0/s 1h59m ago 2 0 ok Consumers: QUEUE CONSUMER LAG PENDING STATUS orders order_processor 2.1s 42 ok orders analytics 15.3s 412 ok orders notifications 1m12s 1831 warning notifications email_sender 3m45s 831 warning audit compliance_log 0.5s 0 ok ``` -------------------------------- ### Verify pgque Installation Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Verify the installation by checking the status, performing basic queue operations, and ensuring the health checks pass. This also includes steps to stop and uninstall pgque. ```sql pgque.status() shows ticker and maint running create queue, send, ticker, receive, ack all work pgque.queue_health() returns 'ok' for all checks pgque.stop() removes pg_cron jobs pgque.uninstall() removes all objects cleanly ``` -------------------------------- ### Python Pgque-py Client Example Source: https://github.com/nikolays/pgque/blob/main/README.md Example of using the PgqueClient and Consumer classes in Python with psycopg 3. Set up a client, send a message, and define a consumer to handle messages for a specific event. ```python from pgque import PgqueClient, Consumer client = PgqueClient(conn) client.send("orders", {"order_id": 42}) consumer = Consumer(dsn, queue="orders", name="processor", poll_interval=30) @consumer.on("order.created") def handle_order(msg): process_order(msg.payload) consumer.start() ``` -------------------------------- ### Install PgQue Source: https://github.com/nikolays/pgque/blob/main/docs/tutorial.md Install PgQue within a transaction to ensure atomicity. This creates the pgque schema, roles, and functions. ```sql begin; \i sql/pgque.sql commit; ``` -------------------------------- ### Go pgque-go Client Example Source: https://github.com/nikolays/pgque/blob/main/README.md Demonstrates connecting to a PostgreSQL database and setting up a consumer using the pgque-go library with pgx/v5. Handles messages for a specific event type. ```go client, _ := pgque.Connect(ctx, "postgresql://localhost/mydb") consumer := client.NewConsumer("orders", "processor") consumer.Handle("order.created", func(ctx context.Context, msg pgque.Message) error { return processOrder(msg) }) consumer.Start(ctx) ``` -------------------------------- ### Create and Subscribe to a Queue Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Create a new queue named 'orders' and subscribe a worker to it. This is part of the initial setup in the tutorial. ```sql create_queue('orders'); subscribe('orders', 'worker'); ``` -------------------------------- ### Fan-Out Pattern Setup Source: https://context7.com/nikolays/pgque/llms.txt Configure a queue for fan-out by creating it and subscribing multiple consumers. All subscribed consumers will receive every message sent to the queue. ```sql -- Set up fan-out: subscribe multiple consumers BEFORE producing select pgque.create_queue('events'); select pgque.subscribe('events', 'audit_logger'); select pgque.subscribe('events', 'notification_sender'); select pgque.subscribe('events', 'analytics_pipeline'); ``` -------------------------------- ### TypeScript pgque-ts Client Example Source: https://github.com/nikolays/pgque/blob/main/README.md Example of using the PgqueClient with node-postgres in TypeScript. Connect to the database, send a message with an event type, subscribe a consumer, and receive messages. ```typescript const client = new PgqueClient('postgresql://localhost/mydb'); await client.connect(); await client.send('orders', { order_id: 42 }, 'order.created'); await client.subscribe('orders', 'processor'); const messages = await client.receive('orders', 'processor', 100); if (messages.length > 0) await client.ack(messages[0].batch_id); ``` -------------------------------- ### PgQue API: Start Ticker Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Starts the PgQue ticker process, which is responsible for moving events through the queue system. This can be driven by pg_cron. ```sql start(); ``` -------------------------------- ### Red/Green TDD: Nack DLQ Unit Test Example Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md This SQL example demonstrates a unit test written using the Red/Green TDD approach. It specifically tests the nack() function's behavior of moving an event to the DLQ after maximum retries are reached, by forging the retry_count to isolate the logic. ```sql -- Red: test that nack() moves event to DLQ after max retries -- (write this BEFORE implementing nack) do $$ declare v_msg pgque.message; v_dlq_count bigint; begin -- Setup: queue with max_retries=2 perform pgque.create_queue('test_dlq'); perform pgque.set_queue_config('test_dlq', 'max_retries', '2'); perform pgque.subscribe('test_dlq', 'c1'); perform pgque.send('test_dlq', '{"x":1}'::jsonb); perform pgque.ticker(); -- Simulate 2 prior retries (retry_count=2 >= max_retries=2) select * into v_msg from pgque.receive('test_dlq', 'c1', 1); -- Forge retry_count to simulate prior retries v_msg.retry_count := 2; perform pgque.nack(v_msg.batch_id, v_msg, '1 second', 'test failure'); perform pgque.ack(v_msg.batch_id); -- Assert: event is in DLQ select count(*) into v_dlq_count from pgque.dead_letter where dl_queue_id = (select queue_id from pgque.queue where queue_name = 'test_dlq'); assert v_dlq_count = 1, 'expected 1 DLQ entry, got ' || v_dlq_count; -- Cleanup perform pgque.unsubscribe('test_dlq', 'c1'); perform pgque.drop_queue('test_dlq'); end; $$; ``` -------------------------------- ### PgQ Consumer Loop Example Source: https://github.com/nikolays/pgque/blob/main/docs/pgq-concepts.md Illustrates the typical flow for a PgQ consumer to process events. It shows fetching a batch, processing events, and finishing the batch. ```sql batch_id = next_batch(queue, consumer) -- NULL → sleep, retry events = get_batch_events(batch_id) process(events) -- nack individual failures finish_batch(batch_id) commit ``` -------------------------------- ### Idempotent pgque Installation/Upgrade Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Safely re-run the install script to perform upgrades or accidental re-runs without breaking existing functionality. This ensures data and consumer positions are preserved. ```sql run \i pgque.sql again Verify: no errors existing queues and events are preserved (check depth matches) consumer positions are preserved (sub_last_tick unchanged) all functions work correctly after re-install send + ticker + receive + ack cycle works ``` -------------------------------- ### Python Client Usage Source: https://context7.com/nikolays/pgque/llms.txt Example of using the PgQue Python client library with psycopg 3 for type-safe message handling. ```APIDOC ## Python Client Usage Use the Python client library with psycopg 3 for type-safe message handling. ### Language Python ### Libraries - `pgque` - `psycopg` ### Description The `pgque` Python client library simplifies interaction with PgQue, providing convenient methods for sending and receiving messages. It integrates seamlessly with `psycopg` for database connections, enabling type-safe message handling and efficient producer operations. ### Request Example ```python from pgque import PgqueClient, Consumer import psycopg # Producer: send messages with psycopg.connect("postgresql://localhost/mydb") as conn: client = PgqueClient(conn) # Send single message event_id = client.send("orders", {"order_id": 42, "total": 99.95}) # Send with explicit type event_id = client.send("orders", {"order_id": 43}, type="order.created") # Batch send event_ids = client.send_batch("orders", "order.created", [ {"order_id": 1}, {"order_id": 2}, {"order_id": 3} ]) conn.commit() ``` ``` -------------------------------- ### Get All Consumer Info Source: https://context7.com/nikolays/pgque/llms.txt Retrieve status for all consumers across all queues. Monitor lag, last seen activity, and pending events. ```sql -- Get info for all consumers select queue_name, consumer_name, lag, last_seen, pending_events from pgque.get_consumer_info(); ``` -------------------------------- ### Start PgQue with pg_cron Source: https://github.com/nikolays/pgque/blob/main/docs/tutorial.md Schedules essential PgQue maintenance and ticker jobs using pg_cron. This function should be called once to set up automated operations. ```sql select pgque.start(); ``` -------------------------------- ### Get Queue Information Source: https://github.com/nikolays/pgque/blob/main/docs/reference.md Retrieves detailed information for each queue, including ticker configuration and live statistics. Requires 'pgque_reader' role. ```sql select * from pgque.get_queue_info(); ``` -------------------------------- ### PgQue SQL Quick Start Source: https://github.com/nikolays/pgque/blob/main/README.md Basic SQL commands for creating queues, subscribing consumers, sending messages, and receiving/acknowledging them. Ensure send, tick, and receive operations are in separate transactions. ```sql -- tx 1: create queue + consumer select pgque.create_queue('orders'); select pgque.subscribe('orders', 'processor'); -- tx 2: send a message select pgque.send('orders', '{"order_id": 42, "total": 99.95}'::jsonb); -- tx 3: advance the queue if you are not using pg_cron -- (force_tick bypasses lag/count thresholds — handy in demos/tests) select pgque.force_tick('orders'); select pgque.ticker(); -- tx 4: receive (batch_id is the same for every returned row) select * from pgque.receive('orders', 'processor', 100); -- tx 5: acknowledge select pgque.ack(:batch_id); ``` -------------------------------- ### Status Check Source: https://context7.com/nikolays/pgque/llms.txt Get a comprehensive diagnostic report of the PgQue installation and job status. ```APIDOC ## Status Check Get a comprehensive diagnostic report of the PgQue installation and job status. ### Method GET (or equivalent SQL query) ### Endpoint `pgque.status()` ### Response #### Success Response (200) - **component** (string) - The name of the component being checked. - **status** (string) - The status of the component (e.g., 'info', 'scheduled', 'unavailable'). - **detail** (string) - Additional details about the component's status. ### Request Example ```sql -- Run overall health check select * from pgque.status(); ``` ### Response Example ``` -- component | status | detail -- ------------+-------------+---------------------------------- -- postgresql | info | PostgreSQL 17.2 on ... -- pgque | info | v0.1.0 -- ticker | scheduled | job_id=1, every 1 second -- maintenance| scheduled | job_id=2, every 30 seconds -- queues | info | 1 queues configured -- consumers | info | 1 active subscriptions -- If pg_cron is not installed: -- pg_cron | unavailable | pg_cron not installed -- call ticker() manually ``` ``` -------------------------------- ### Create a Queue with Options Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Demonstrates creating a new queue with specific configuration options such as rotation period, maximum ticker count, and maximum retries. ```sql -- Create a queue with options SELECT pgque.create_queue('orders', '{ "rotation_period": "4 hours", "ticker_max_count": 1000, "ticker_max_lag": "5 seconds", "max_retries": 10 }'::jsonb); ``` -------------------------------- ### Stop PgQue Services Source: https://github.com/nikolays/pgque/blob/main/docs/reference.md Unschedules the pg_cron jobs previously set up by the start() function. This operation is safe even if pg_cron is not installed. ```sql select pgque.stop(); ``` -------------------------------- ### Get Status Report Source: https://context7.com/nikolays/pgque/llms.txt Run a comprehensive health check for the PgQue installation and its components. Provides status on PostgreSQL, PgQue version, and scheduled jobs. ```sql -- Run overall health check select * from pgque.status(); ``` -------------------------------- ### Get Batch Info Source: https://context7.com/nikolays/pgque/llms.txt Inspect details of an active batch, including start and end times, tick boundaries, and lag. Requires a batch ID. ```sql -- Get details about an active batch select * from pgque.get_batch_info(1); ``` -------------------------------- ### Idempotency Implementation Notes Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Implementation details for ensuring idempotent installation scripts, focusing on using `CREATE TABLE IF NOT EXISTS`, `CREATE OR REPLACE FUNCTION`, and similar constructs to preserve data across re-runs. ```sql CREATE TABLE IF NOT EXISTS CREATE OR REPLACE FUNCTION CREATE TYPE ... IF NOT EXISTS (or DO $$ BEGIN ... EXCEPTION WHEN duplicate_object ...) CREATE SEQUENCE IF NOT EXISTS ``` -------------------------------- ### Fan-out with multiple consumers Source: https://github.com/nikolays/pgque/blob/main/docs/examples.md Set up multiple subscribers to the same queue. Each subscriber maintains its own cursor and receives all events. Subscribe before producing to ensure no events are missed. ```sql select pgque.subscribe('orders', 'audit_logger'); select pgque.subscribe('orders', 'notification_sender'); select pgque.subscribe('orders', 'analytics_pipeline'); select pgque.send('orders', 'order.created', '{"order_id": 1}'::jsonb); select pgque.force_tick('orders'); select pgque.ticker(); select * from pgque.receive('orders', 'audit_logger', 100); select * from pgque.receive('orders', 'notification_sender', 100); ``` -------------------------------- ### PgQ Set Queue Configuration Source: https://github.com/nikolays/pgque/blob/main/docs/pgq-concepts.md Demonstrates how to set configuration parameters for a specific PgQ queue. These parameters influence ticker behavior, rotation, and retry limits. ```sql pgque.set_queue_config(queue, param, value) ``` -------------------------------- ### Create Queue Source: https://context7.com/nikolays/pgque/llms.txt How to create a new named queue or configure an existing one. ```APIDOC ## Create Queue Create a named queue to hold events. Queues are shared event logs where multiple producers can write concurrently and multiple consumers can subscribe independently. ### Request Example ```sql -- Create a new queue (returns 1 if created, 0 if already exists) select pgque.create_queue('orders'); -- Output: -- create_queue -- -------------- -- 1 -- Create queue with custom configuration select pgque.create_queue('high_priority'); select pgque.set_queue_config('high_priority', 'max_retries', '10'); select pgque.set_queue_config('high_priority', 'ticker_max_lag', '00:00:01'); ``` ``` -------------------------------- ### Get Overall Status Source: https://github.com/nikolays/pgque/blob/main/docs/tutorial.md Provides a comprehensive health check of all PgQue components and their status. ```sql select * from pgque.status(); ``` ```text component | status | detail ------------+-------------+---------------------------------- postgresql | info | PostgreSQL 17.2 on ... pgque | info | [[your version]] pg_cron | unavailable | pg_cron not installed -- call ... queues | info | 1 queues configured consumers | info | 1 active subscriptions ``` -------------------------------- ### Create and subscribe to a queue Source: https://github.com/nikolays/pgque/blob/main/docs/tutorial.md Create a new queue named 'orders' and subscribe a consumer named 'processor' to it. The create_queue call is idempotent. ```sql select pgque.create_queue('orders'); ``` ```sql select pgque.subscribe('orders', 'processor'); ``` -------------------------------- ### Lifecycle Management Source: https://github.com/nikolays/pgque/blob/main/docs/reference.md Functions to manage the lifecycle of PgQue, including starting, stopping, and uninstalling the extension. ```APIDOC ## pgque.start() ### Description Schedules three pg_cron jobs in the current database: `pgque_ticker` (every 1s), `pgque_maint` (every 30s), and `pgque_rotate_step2` (every 10s). Requires the `pg_cron` extension. This operation is idempotent as it calls `stop()` first. ### Method N/A (SQL Function) ### Endpoint N/A ### Parameters N/A ### Request Example ```sql select pgque.start(); ``` ### Response #### Success Response (200) - **void** - Indicates successful execution. #### Response Example ``` (no return value) ``` ``` ```APIDOC ## pgque.stop() ### Description Unschedules the pg_cron jobs that were set up by `pgque.start()` and clears the stored job IDs. This function is safe to call even if `pg_cron` is not present. ### Method N/A (SQL Function) ### Endpoint N/A ### Parameters N/A ### Request Example ```sql select pgque.stop(); ``` ### Response #### Success Response (200) - **void** - Indicates successful execution. #### Response Example ``` (no return value) ``` ``` ```APIDOC ## pgque.uninstall() ### Description Calls `pgque.stop()` (if pg_cron is present) and then drops the `pgque` schema and all its contents. Note that the roles `pgque_reader`, `pgque_writer`, and `pgque_admin` are not dropped and must be removed manually if desired. This function is restricted to superusers or the schema owner. ### Method N/A (SQL Function) ### Endpoint N/A ### Parameters N/A ### Request Example ```sql select pgque.uninstall(); ``` ### Response #### Success Response (200) - **void** - Indicates successful execution. #### Response Example ``` (no return value) ``` ``` -------------------------------- ### Get Queue Info Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Retrieves detailed information about a specific queue. Useful for monitoring and diagnostics. ```sql get_queue_info('my_queue'); ``` -------------------------------- ### Listen for Low-Latency Wakeups Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Use `LISTEN pgque_` for low-latency consumer wakeups, combined with a polling fallback to ensure notifications are not missed. Requires session-mode connection pooling. ```sql LISTEN pgque_ ``` -------------------------------- ### Lifecycle Management Source: https://context7.com/nikolays/pgque/llms.txt Control pg_cron scheduled jobs for ticker and maintenance operations, and manage PgQue installation. ```APIDOC ## Lifecycle Management Control pg_cron scheduled jobs for ticker and maintenance operations. ### Method POST (or equivalent SQL query) ### Endpoints - `pgque.start()` - `pgque.stop()` - `pgque.ticker()` - `pgque.maint()` - `pgque.uninstall()` ### Description - `pgque.start()`: Starts the pg_cron scheduled jobs for the ticker (every 1s) and maintenance (every 30s). - `pgque.stop()`: Stops all pg_cron scheduled jobs. - `pgque.ticker()`: Manually calls the ticker function (use if pg_cron is not available). - `pgque.maint()`: Manually calls the maintenance function (use if pg_cron is not available). - `pgque.uninstall()`: Uninstalls PgQue (superuser only, drops schema cascade). ### Request Example ```sql -- Start pg_cron jobs select pgque.start(); -- Stop all pg_cron jobs select pgque.stop(); -- Manual ticker call select pgque.ticker(); -- Manual maintenance call select pgque.maint(); -- Uninstall PgQue select pgque.uninstall(); ``` ``` -------------------------------- ### Uninstall PgQue (Superuser Only) Source: https://github.com/nikolays/pgque/blob/main/blueprints/PHASES.md This function removes the PgQue installation. It requires superuser privileges and should be used with caution. ```sql pgque.uninstall() ``` -------------------------------- ### Get Batch Info Source: https://context7.com/nikolays/pgque/llms.txt Inspect an active batch's details including tick boundaries and timing. ```APIDOC ## Get Batch Info Inspect an active batch's details including tick boundaries and timing. ### Method GET (or equivalent SQL query) ### Endpoint `pgque.get_batch_info(batch_id)` ### Parameters #### Path Parameters - **batch_id** (integer) - Required - The ID of the batch to inspect. ### Response #### Success Response (200) - **queue_name** (string) - The name of the queue. - **consumer_name** (string) - The name of the consumer. - **batch_start** (timestamp) - The start time of the batch. - **batch_end** (timestamp) - The end time of the batch. - **prev_tick_id** (integer) - The ID of the previous tick. - **tick_id** (integer) - The ID of the current tick. - **lag** (interval) - The lag associated with the batch. - **seq_start** (integer) - The starting sequence number of the batch. - **seq_end** (integer) - The ending sequence number of the batch. ### Request Example ```sql -- Get details about an active batch select * from pgque.get_batch_info(1); ``` ### Response Example ``` -- queue_name | consumer_name | batch_start | batch_end | prev_tick_id | tick_id | lag | seq_start | seq_end -- ------------+---------------+--------------------------+--------------------------+--------------+---------+--------------+-----------+--------- -- orders | processor | 2026-04-17 10:00:00+00 | 2026-04-17 10:00:01+00 | 0 | 1 | 00:00:02.123 | 1 | 42 ``` ``` -------------------------------- ### Get Consumer Info Source: https://context7.com/nikolays/pgque/llms.txt Retrieve consumer status including lag, last activity, and pending event count. ```APIDOC ## Get Consumer Info Retrieve consumer status including lag, last activity, and pending event count. ### Method GET (or equivalent SQL query) ### Endpoint `pgque.get_consumer_info()` ### Parameters #### Query Parameters - **queue_name** (string) - Optional - The name of the queue to filter by. - **consumer_name** (string) - Optional - The name of the consumer to filter by. ### Response #### Success Response (200) - **queue_name** (string) - The name of the queue. - **consumer_name** (string) - The name of the consumer. - **lag** (interval) - The current lag of the consumer. - **last_seen** (interval) - The time since the last activity of the consumer. - **pending_events** (integer) - The number of pending events for the consumer. ### Request Example ```sql -- Get info for all consumers select queue_name, consumer_name, lag, last_seen, pending_events from pgque.get_consumer_info(); -- Get info for a specific queue/consumer select * from pgque.get_consumer_info('orders', 'processor'); ``` ### Response Example ``` -- Output: -- -[ RECORD 1 ]--+-------------- -- queue_name | orders -- consumer_name | processor -- lag | 00:00:00.520 -- last_seen | 00:00:00.201 -- last_tick | 1247 -- current_batch | -- next_tick | -- pending_events | 0 ``` ### Red flags to monitor: - lag climbing into minutes = consumer not finishing batches - last_seen climbing = consumer stopped calling receive - pending_events growing = stuck consumer blocking rotation ``` -------------------------------- ### Run psql commands Source: https://github.com/nikolays/pgque/blob/main/docs/tutorial.md Execute psql commands with specific configurations for reproducible output. Ensure the pgque.sql file is resolvable. ```bash cd /path/to/pgque PAGER=cat psql --no-psqlrc -d mydb ``` -------------------------------- ### Get Batch Info Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Retrieves information about a specific event batch. Useful for debugging and understanding batch processing. ```sql get_batch_info(batch_id); ``` -------------------------------- ### Sprint 1: Repackaging Deliverables Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md This list outlines the key deliverables for Sprint 1, focusing on the mechanical transformation of PgQ code into the pgque schema without introducing new logic. ```text 1. Set up PgQ as a git submodule (`pgq/`, pinned to v3.5.1) 2. Create `build/transform.sh` that reads PL-only source files and applies all mechanical transformations (items 3-9 below) 3. Global rename: `pgq.` -> `pgque.` (schema prefix in ~40 files) 4. Replace `txid_*` with `pg_*` functions (8 distinct replacements) 5. Replace `bigint` with `xid8` for txid columns (schema + functions) 6. Replace `txid_snapshot` type with `pg_snapshot` 7. Add `SET search_path = pgque, pg_catalog` to all `SECURITY DEFINER` functions 8. Remove `queue_per_tx_limit` column and references 9. Remove `set default_with_oids = 'off' 10. Remove `maint_operations` pgq_node/Londiste hooks 11. Add `pgque.config` table 12. Build concatenated `pgque.sql` and `pgque-unpgque.sql` 13. Create roles: `pgque_reader`, `pgque_writer`, `pgque_admin` 14. Regression tests: run PgQ's existing test suite against pgque ``` -------------------------------- ### Get Consumer Info Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Retrieves detailed information about a specific consumer. Helps in understanding consumer status and lag. ```sql get_consumer_info('my_consumer'); ``` -------------------------------- ### API Summary: Modern vs. PgQ Primitives Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md A comparative overview of the modern PgQ API and the underlying PgQ primitives, highlighting their mappings and notes. ```APIDOC ## API Summary: Modern vs. PgQ Primitives This table summarizes the mapping between the modern PgQ API functions and their underlying PgQ primitives, along with relevant notes. | Modern API | PgQ primitive underneath | Notes | |---|---|---| | `pgque.send(queue, payload)` | `pgque.insert_event(queue, type, data)` | TEXT overload is default for untyped literals (fast path, opaque bytes); JSONB overload is opt-in via `::jsonb` cast (validation + canonicalization) | | `pgque.send_batch(queue, type, payloads[])` | Loop of `insert_event()` calls | Single TX; `text[]` default, `jsonb[]` opt-in via `::jsonb[]` cast | | `pgque.send_at(queue, type, payload, time)` | `delayed_events` table + `maint_deliver_delayed()` | New | | `pgque.receive(queue, consumer, n)` | `next_batch()` + `get_batch_events()` | Combined | | `pgque.ack(batch_id)` | `finish_batch(batch_id)` | Rename | | `pgque.nack(batch_id, msg, delay)` | `event_retry(batch_id, msg_id, seconds)` | + DLQ logic, reads max_retries from queue config | | `pgque.subscribe(queue, consumer)` | `register_consumer(queue, consumer)` | Rename | | `pgque.unsubscribe(queue, consumer)` | `unregister_consumer(queue, consumer)` | Rename | | `pgque.event_dead(batch, event_id, reason, ...)` | `dead_letter` table insert | New, accepts event fields from caller | | `pgque.dlq_replay(dl_id)` | `insert_event()` from dead_letter row | New | | `pgque.pause_queue(queue)` | `set_queue_config(queue, 'ticker_paused', 'true')` | Convenience | The PgQ-style API (`insert_event`, `next_batch`, `get_batch_events`, `finish_batch`, `event_retry`) remains fully available for users who need fine-grained control. ``` -------------------------------- ### Create pgque.config Singleton Table Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Initializes the `pgque.config` table, a singleton table used for tracking pg_cron job IDs for ticker and maintenance operations. It stores the job IDs and the timestamp of installation. ```sql CREATE TABLE pgque.config ( singleton bool PRIMARY KEY DEFAULT true CHECK (singleton), ticker_job_id bigint, maint_job_id bigint, installed_at timestamptz NOT NULL DEFAULT clock_timestamp() ); INSERT INTO pgque.config (singleton) VALUES (true); ``` -------------------------------- ### PgQue API: Get Status Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Retrieves the current status of the PgQue system, including information about queues and consumers. ```sql status(); ``` -------------------------------- ### PgQ Canonical Consumer Loop Source: https://github.com/nikolays/pgque/blob/main/docs/tutorial.md Illustrates the fundamental PgQ consumer loop, showing the relationship between `next_batch`, `get_batch_events`, `process`, `finish_batch`, and `commit`. PgQue's `receive` and `ack` abstract these steps. ```sql batch_id = next_batch(queue, consumer) -- NULL → sleep and retry events = get_batch_events(batch_id) process(events) -- event_retry per event on failure finish_batch(batch_id) commit ``` -------------------------------- ### Ruby Producer and Standalone Consumer Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md Shows how to produce messages and set up a standalone consumer in Ruby using `pgque-rb`. Handlers are blocks that process messages, with implicit acknowledgment on success. ```ruby require 'pgque' client = Pgque::Client.new("postgresql://localhost/mydb") # Producer client.send("orders", type: "order.created", payload: { order_id: 42, total: 99.95 }) # Consumer (standalone) consumer = Pgque::Consumer.new( client, queue: "orders", name: "order_processor", poll_interval: 30, max_retries: 5 ) consumer.on("order.created") do |msg| OrderService.process(msg.payload) end consumer.start # blocks # Rails integration (ActiveJob adapter) class OrderJob < ApplicationJob queue_as :orders def perform(order_id) Order.find(order_id).process! end end ``` -------------------------------- ### Get Next Batch Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Retrieves the next available batch of events from the queue. Use 'next_batch_custom' for custom batch timing. ```sql next_batch ``` ```sql next_batch_custom ``` -------------------------------- ### Get PgQue Queue Information Source: https://github.com/nikolays/pgque/blob/main/blueprints/PHASES.md Retrieve detailed information about one or all queues. This is useful for monitoring queue status and configuration. ```sql pgque.get_queue_info() ``` ```sql pgque.get_queue_info(queue) ``` -------------------------------- ### Send and Receive an Order Source: https://github.com/nikolays/pgque/blob/main/blueprints/BETTER_DOCS.md Send an order to the queue and attempt to receive it immediately. This demonstrates the initial state before the ticker runs. ```sql send('orders', 'order_payload'); receive('orders'); ``` -------------------------------- ### Get consumer statistics Source: https://github.com/nikolays/pgque/blob/main/docs/reference.md Returns statistics for consumers, including lag, pending events, and batch activity. This function is `security definer`. ```sql pgque.consumer_stats() → table(queue_name text, consumer_name text, lag interval, pending_events bigint, last_batch_start timestamptz, batch_active boolean, batch_id bigint) ``` -------------------------------- ### pgque CLI Usage and Commands Source: https://github.com/nikolays/pgque/blob/main/blueprints/SPECx.md The pgque CLI is a Go-based tool for administering pgque. It supports various commands for installation, upgrades, system status, queue management, and data manipulation. Connection details can be provided via a DSN string or environment variables. ```bash pgque -PgQ Extended administration tool Usage: pgque [flags] Connection: --dsn, -d PostgreSQL connection string (or PGQUE_DSN env var) --database Database name (or PGDATABASE) Commands: install Install pgque schema into database upgrade Upgrade pgque to latest version uninstall Remove pgque schema (requires --force) start Start pg_cron ticker and maintenance jobs stop Stop pg_cron jobs status Show system health, queue stats, consumer lag queues List all queues with depth and health consumers List all consumers with lag and status depth Show queue depth over time (sparkline) drain Wait until queue is empty or timeout replay-dlq Replay dead letter events back into queue purge-dlq Delete old dead letter events create-queue Create a new queue drop-queue Drop a queue (requires --force if consumers exist) pause Pause a queue's ticker resume Resume a queue's ticker Examples: pgque install -d "postgresql://localhost/mydb" pgque start pgque status pgque queues pgque depth orders --watch 5s pgque consumers orders pgque replay-dlq orders --limit 100 pgque drain orders --timeout 60s ```