### Install Psycopg 3 with Binary Packages Source: https://www.psycopg.org/psycopg3/docs/basic/install.html Use this command to install a self-contained Psycopg 3 package with all necessary libraries. This is the quickest way to start developing. ```bash pip install "psycopg[binary]" ``` -------------------------------- ### Install Psycopg with Binary Distribution Source: https://www.psycopg.org/psycopg3/docs/basic/install.html Recommended for most users. Installs precompiled C extensions and client libraries. Ensure pip is up-to-date before installation. ```bash pip install --upgrade pip pip install "psycopg[binary]" ``` -------------------------------- ### Install Psycopg 3 with Local Installation (C Extension) Source: https://www.psycopg.org/psycopg3/docs/basic/install.html Install Psycopg 3 with the C extension for a performing and maintainable library, linked to system libraries. Requires a C compiler and development headers. ```bash pip install "psycopg[c]" ``` -------------------------------- ### Install libpq on Debian/Ubuntu for Pure Python Installation Source: https://www.psycopg.org/psycopg3/docs/basic/install.html Install the libpq PostgreSQL client library on Debian-based systems, which is required for the pure Python installation of Psycopg 3. ```bash sudo apt install libpq5 ``` -------------------------------- ### Install Psycopg Pure Python Source: https://www.psycopg.org/psycopg3/docs/basic/install.html Installs only the pure Python implementation. This option is slower and requires local libpq client libraries. Use this if you cannot build C extensions. ```bash pip install psycopg ``` -------------------------------- ### Install Psycopg 3 Connection Pool Source: https://www.psycopg.org/psycopg3/docs/basic/install.html Install the Psycopg connection pools by installing the 'pool' extra or the 'psycopg_pool' package separately. ```bash pip install "psycopg[pool]" ``` -------------------------------- ### TypeInfo Fetch and Register Example Source: https://www.psycopg.org/psycopg3/docs/api/types.html Demonstrates how to fetch TypeInfo from a database and register it for custom type handling. ```APIDOC ## TypeInfo.fetch() ### Description Queries a system catalog to read information about a PostgreSQL data type. ### Method `classmethod fetch(conn, name)` `async classmethod fetch(aconn, name)` ### Parameters #### Path Parameters * **conn** (_Connection_ or _AsyncConnection_) - The connection to query. * **name** (`str` or `Identifier`) - The name of the type to query. It can include a schema name. ### Returns A `TypeInfo` object (or subclass) populated with the type information, or `None` if not found. ### Async Usage Example ```python t = await TypeInfo.fetch(aconn, "mytype") ``` ## TypeInfo.register() ### Description Registers the type information, either globally or within a specified context. ### Method `register(context: AdaptContext | None = None) -> None` ### Parameters #### Path Parameters * **context** (_Optional_[_AdaptContext_]) - The context where the type is registered, such as a `Connection` or `Cursor`. If `None`, the `TypeInfo` is registered globally. ### Usage Example ```python from psycopg.adapt import Loader from psycopg.types import TypeInfo t = TypeInfo.fetch(conn, "mytype") t.register(conn) # Example of custom loader usage class MyTypeLoader(Loader): def load(self, data): # parse the data and return a MyType instance pass conn.adapters.register_loader("mytype", MyTypeLoader) ``` ``` -------------------------------- ### Install Psycopg with Local C Extensions Source: https://www.psycopg.org/psycopg3/docs/basic/install.html Builds C extensions from source. Requires build tools and local libpq client libraries. Use this if the binary distribution is not suitable. ```bash pip install psycopg[c] ``` -------------------------------- ### open Source: https://www.psycopg.org/psycopg3/docs/api/pool.html Opens the pool, starting connections and accepting clients. It can optionally wait for a specified number of connections to be ready. ```APIDOC ## open ### Description Opens the pool by starting connections and accepting clients. If `wait` is `False`, it returns immediately. Otherwise, it waits up to `timeout` seconds for the requested number of connections to be ready. ### Method open ### Parameters - **_wait** (bool) - Optional - If `False`, return immediately. Defaults to `False`. - **_timeout** (float) - Optional - Seconds to wait for connections to be ready. Defaults to `30.0`. ### Notes It is safe to call `open()` multiple times on an already open pool. A closed pool cannot be re-opened. ``` -------------------------------- ### Asynchronous Connection Establishment Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Connect to a database server asynchronously and get a new AsyncConnection instance. Domain names are resolved asynchronously. ```python conn = await psycopg.AsyncConnection.connect("dbname=test") ``` -------------------------------- ### ClientCursor mogrify Method Example Source: https://www.psycopg.org/psycopg3/docs/api/cursors.html Demonstrates how to use the mogrify method of the ClientCursor class to merge a query with its parameters on the client side. This is useful for executing queries that cannot be parameterized on the server. ```python cursor.mogrify("SELECT %s, %s", ('a', 1)) ``` -------------------------------- ### Install dnspython package Source: https://www.psycopg.org/psycopg3/docs/api/dns.html The `_dns` module depends on the `dnspython` package, which must be installed manually. ```bash $ pip install "dnspython >= 2.1" ``` -------------------------------- ### AsyncConnection tpc_prepare Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Performs the first phase of a transaction started with tpc_begin(). ```APIDOC ## async tpc_prepare() ### Description Perform the first phase of a transaction started with `tpc_begin()`. ### Method ASYNC POST ### Endpoint /connections/tpc_prepare ### Parameters None ### Request Example None ### Response #### Success Response (200) None #### Response Example None ``` -------------------------------- ### open Source: https://www.psycopg.org/psycopg3/docs/api/pool.html Opens the connection pool, starting connections and accepting clients. It can optionally wait for a minimum number of connections to be ready. ```APIDOC ## open ### Description Open the pool by starting connecting and and accepting clients. If _wait_ is `False`, return immediately and let the background worker fill the pool if `min_size` > 0. Otherwise wait up to _timeout_ seconds for the requested number of connections to be ready. ### Method async ### Parameters - **_wait** (bool) - Optional - If `False`, return immediately. - **_timeout** (float) - Optional - Seconds to wait for connections if _wait_ is `True`. ### Response None ``` -------------------------------- ### Example: Custom String Range Type Source: https://www.psycopg.org/psycopg3/docs/basic/pgtypes.html This example demonstrates creating a custom string range type in PostgreSQL, fetching its information using `RangeInfo`, and registering it with Psycopg using `register_range`. It then shows how to query the database with a `Range` object and retrieve data of the custom type. ```python >>> from psycopg.types.range import Range, RangeInfo, register_range >>> conn.execute("CREATE TYPE strrange AS RANGE (SUBTYPE = text)") >>> info = RangeInfo.fetch(conn, "strrange") >>> register_range(info, conn) >>> conn.execute("SELECT pg_typeof(%s)", [Range("a", "z")]).fetchone()[0] 'strrange' >>> conn.execute("SELECT '[a,z]'::strrange").fetchone()[0] Range('a', 'z', '[]') ``` -------------------------------- ### Pipeline Synchronization Example Source: https://www.psycopg.org/psycopg3/docs/advanced/pipeline.html Demonstrates how to use a pipeline with autocommit enabled, including error handling for a non-existent table. Note that the exact statement raising the error can be arbitrary, and subsequent statements might be affected. ```python >>> with psycopg.connect(autocommit=True) as conn: ... with conn.pipeline() as p, conn.cursor() as cur: ... try: ... cur.execute("INSERT INTO mytable (data) VALUES (%s)", ["one"]) ... cur.execute("INSERT INTO no_such_table (data) VALUES (%s)", ["two"]) ... conn.execute("INSERT INTO mytable (data) VALUES (%s)", ["three"]) ... p.sync() ... except psycopg.errors.UndefinedTable: ... pass ... cur.execute("INSERT INTO mytable (data) VALUES (%s)", ["four"]) ``` -------------------------------- ### Creating a Cursor Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Shows how to obtain a cursor from a connection object. Cursors are used to send commands and queries to the database. This example illustrates creating a standard client-side cursor. ```python with conn.cursor() as cur: ... ``` -------------------------------- ### Starting a Two-Phase Commit Transaction Source: https://www.psycopg.org/psycopg3/docs/basic/transactions.html Initiate a distributed transaction using `Connection.tpc_begin()` with a previously created transaction ID. ```python # Start a distributed transaction conn.tpc_begin(xid) ``` -------------------------------- ### Opening a Connection Pool Source: https://www.psycopg.org/psycopg3/docs/api/pool.html Opens the pool by starting connections and accepting clients. If wait is False, it returns immediately. Otherwise, it waits for a specified timeout for connections to be ready. ```python pool.open(_wait = False_, _timeout = 30.0_) ``` -------------------------------- ### Starting a Transaction Block Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Initiate a transaction block using a context manager. This can be a new transaction or a nested transaction using savepoints. The transaction will be rolled back if an error occurs, unless `force_rollback` is set to `True`. ```python with conn.transaction(): ... with conn.transaction() as tx: ... ``` -------------------------------- ### Registering Custom Multirange Types Source: https://www.psycopg.org/psycopg3/docs/basic/pgtypes.html Example demonstrating how to create a custom text-based multirange type in PostgreSQL, fetch its information, register it with Psycopg, and then use it to insert and retrieve data. ```python from psycopg.types.multirange import \ Multirange, MultirangeInfo, register_multirange from psycopg.types.range import Range conn.execute("CREATE TYPE strrange AS RANGE (SUBTYPE = text)") info = MultirangeInfo.fetch(conn, "strmultirange") register_multirange(info, conn) rec = conn.execute( "SELECT pg_typeof(%(mr)s), %(mr)s", {"mr": Multirange([Range("a", "q"), Range("l", "z")])}).fetchone() rec[0] rec[1] ``` -------------------------------- ### Verify hstore Type Conversion Source: https://www.psycopg.org/psycopg3/docs/basic/pgtypes.html These Python examples show how to verify that Psycopg3 correctly identifies and handles the hstore data type. The first checks the PostgreSQL type of a dictionary, and the second demonstrates parsing an hstore literal into a Python dictionary. ```python >>> conn.execute("SELECT pg_typeof(%s)", [{"a": "b"}]).fetchone()[0] 'hstore' ``` ```python >>> conn.execute("SELECT 'foo => bar'::hstore").fetchone()[0] {'foo': 'bar'} ``` -------------------------------- ### Basic Async Connection and Cursor Usage Source: https://www.psycopg.org/psycopg3/docs/advanced/async.html Demonstrates how to establish an asynchronous connection, create a cursor, execute commands, and iterate over results using asyncio. ```python async with await psycopg.AsyncConnection.connect( "dbname=test user=postgres") as aconn: async with aconn.cursor() as acur: await acur.execute( "INSERT INTO test (num, data) VALUES (%s, %s)", (100, "abc'def")) await acur.execute("SELECT * FROM test") await acur.fetchone() # will return (1, 100, "abc'def") async for record in acur: print(record) ``` -------------------------------- ### Connection.isolation_level Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Gets or sets the isolation level for new transactions started on the connection. ```APIDOC ## Connection.isolation_level ### Description The isolation level of the new transactions started on the connection. ### Property Type Readable and writable. ### Notes `None` means use the default set in the server's configuration. ``` -------------------------------- ### Manual Connection Management (Equivalent to Context Manager) Source: https://www.psycopg.org/psycopg3/docs/basic/usage.html Demonstrates the explicit steps equivalent to using a connection as a context manager, including try-except-finally blocks for commit, rollback, and close operations. ```python conn = psycopg.connect() try: ... # use the connection except BaseException: conn.rollback() else: conn.commit() finally: conn.close() ``` -------------------------------- ### Registering and Fetching Extension Types Source: https://www.psycopg.org/psycopg3/docs/advanced/adapt.html Demonstrates how to fetch information about extension types registered on a connection and then register them globally for use. ```python >>> conn.adapters.types["integer"] ``` ```python >>> (t := psycopg.types.TypeInfo.fetch(conn, "hstore")) ``` ```python >>> t.register() # globally ``` ```python >>> psycopg.adapters.types["hstore"] ``` -------------------------------- ### Example Pool Operations Log Output Source: https://www.psycopg.org/psycopg3/docs/advanced/pool.html This is an example of the log output generated by the `psycopg.pool` logger when running the provided Python script. Log messages may change across versions. ```log 2023-09-20 11:02:39,718 INFO psycopg.pool: waiting for pool 'pool-1' initialization 2023-09-20 11:02:39,720 INFO psycopg.pool: adding new connection to the pool 2023-09-20 11:02:39,720 INFO psycopg.pool: adding new connection to the pool 2023-09-20 11:02:39,720 INFO psycopg.pool: pool 'pool-1' is ready to use 2023-09-20 11:02:39,720 INFO root: pool ready 2023-09-20 11:02:39,721 INFO psycopg.pool: connection requested from 'pool-1' 2023-09-20 11:02:39,721 INFO psycopg.pool: connection given by 'pool-1' 2023-09-20 11:02:39,721 INFO psycopg.pool: connection requested from 'pool-1' 2023-09-20 11:02:39,721 INFO psycopg.pool: connection given by 'pool-1' 2023-09-20 11:02:39,721 INFO psycopg.pool: connection requested from 'pool-1' 2023-09-20 11:02:39,722 INFO psycopg.pool: connection requested from 'pool-1' 2023-09-20 11:02:40,724 INFO root: The square of 0 is 0. 2023-09-20 11:02:40,724 INFO root: The square of 1 is 1. 2023-09-20 11:02:40,725 INFO psycopg.pool: returning connection to 'pool-1' 2023-09-20 11:02:40,725 INFO psycopg.pool: connection given by 'pool-1' 2023-09-20 11:02:40,725 INFO psycopg.pool: returning connection to 'pool-1' 2023-09-20 11:02:40,726 INFO psycopg.pool: connection given by 'pool-1' 2023-09-20 11:02:41,728 INFO root: The square of 3 is 9. 2023-09-20 11:02:41,729 INFO root: The square of 2 is 4. 2023-09-20 11:02:41,729 INFO psycopg.pool: returning connection to 'pool-1' 2023-09-20 11:02:41,730 INFO psycopg.pool: returning connection to 'pool-1' ``` -------------------------------- ### Dynamic Query Construction with sql.SQL Source: https://www.psycopg.org/psycopg3/docs/advanced/typing.html Demonstrates how to safely construct dynamic SQL queries for table and field names using `sql.SQL`. Parameters for `SQL()` must be literal strings. ```python def count_records(conn: psycopg.Connection[Any], table: str) -> int: query = "SELECT count(*) FROM %s" % table # BAD! return conn.execute(query).fetchone()[0] ``` -------------------------------- ### Connection.deferrable Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Controls whether new transactions started on the connection are deferrable. ```APIDOC ## Connection.deferrable ### Description The deferrable state of the new transactions started on the connection. ### Property Type Readable and writable. ### Notes `None` means use the default set in the server's configuration. ``` -------------------------------- ### Connection.read_only Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Controls whether new transactions started on the connection are read-only. ```APIDOC ## Connection.read_only ### Description The read-only state of the new transactions started on the connection. ### Property Type Readable and writable. ### Notes `None` means use the default set in the server's configuration. ``` -------------------------------- ### AsyncConnection.transaction Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Starts a context block with a new transaction or nested transaction asynchronously. ```APIDOC ## AsyncConnection.transaction ### Description Start a context block with a new transaction or nested transaction. ### Method `async with` ### Parameters - **savepoint_name** (str | None) - Optional - Name of the savepoint for nested transactions. - **force_rollback** (bool) - Optional - Force rollback at the end of the block. ### Request Example ```python async with conn.transaction(): await conn.execute('...') ``` ### Response #### Success Response - **AsyncTransaction** - An asynchronous transaction object. ``` -------------------------------- ### Creating a Binary Cursor Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Demonstrates how to create a cursor that handles binary data. This is useful when dealing with types that have binary loaders. ```python cur = conn.cursor(binary=True) ``` -------------------------------- ### Asynchronous Rollback Transaction Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Roll back to the start of any pending transaction asynchronously. ```python await conn.rollback() ``` -------------------------------- ### Using alternative functions for SET and NOTIFY Source: https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html Shows how to use PostgreSQL functions like set_config() and pg_notify() as alternatives to the SET and NOTIFY statements when server-side binding is not directly supported. ```python >>> conn.execute("SELECT set_config('TimeZone', %s, false)", ["UTC"]) >>> conn.execute("SELECT pg_notify(%s, %s)", ["chan", "42"]) ``` -------------------------------- ### Pool startup check with `wait()` Source: https://www.psycopg.org/psycopg3/docs/advanced/pool.html Use the `pool.wait()` method after opening the pool to ensure that a minimum number of connections (`min_size`) are established before proceeding. This helps catch configuration issues early. ```python with ConnectionPool(...) as pool: pool.wait() use_the(pool) ``` -------------------------------- ### Connection.rollback Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Rolls back to the start of any pending transaction, discarding any uncommitted changes. ```APIDOC ## Connection.rollback ### Description Roll back to the start of any pending transaction. ### Method Not specified (assumed to be a method call on a Connection object) ### Return Type None ``` -------------------------------- ### psycopg.IntegrityError Source: https://www.psycopg.org/psycopg3/docs/api/errors.html Raised when the relational integrity of the database is affected, for example, by a foreign key check failure. ```APIDOC exception psycopg.IntegrityError An error caused when the relational integrity of the database is affected. An example may be a foreign key check failed. ``` -------------------------------- ### Connection.transaction Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Starts a context block for a new transaction or a nested transaction, with options for savepoints and forced rollback. ```APIDOC ## Connection.transaction ### Description Start a context block with a new transaction or nested transaction. ### Method Context manager (used with `with` statement) ### Parameters * **savepoint_name** (str or None) – Name of the savepoint used to manage a nested transaction. If None, one will be chosen automatically. * **force_rollback** (bool) – Roll back the transaction at the end of the block even if there were no error. ### Return Type Transaction ### Usage Example ```python with conn.transaction(): ... with conn.transaction() as tx: ... ``` ### Notes Inside a transaction block, `commit()` or `rollback()` cannot be called. ``` -------------------------------- ### Composing SQL Queries with Named Placeholders Source: https://www.psycopg.org/psycopg3/docs/api/sql.html Illustrates how to build SQL queries with named placeholders and format them with SQL identifiers for fields, tables, and primary keys. ```python query = sql.SQL("SELECT {field} FROM {table} WHERE {pkey} = %s").format( field=sql.Identifier('my_name'), table=sql.Identifier('some_table'), pkey=sql.Identifier('id')) ``` -------------------------------- ### Accessing Connection Timezone Source: https://www.psycopg.org/psycopg3/docs/api/objects.html Retrieve the Python timezone information for the connection. This example shows how to access the timezone attribute. ```python >>> conn.info.timezone zoneinfo.ZoneInfo(key='Europe/Rome') ``` -------------------------------- ### Asynchronous Transaction Context Manager Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Start a context block with a new transaction or nested transaction using 'async with'. ```python async with conn.transaction() as tx: ... ``` -------------------------------- ### Basic Psycopg 3 Database Operations Source: https://www.psycopg.org/psycopg3/docs/basic/usage.html Demonstrates connecting to a database, creating a table, inserting data with parameter substitution, querying results, using executemany for batch operations, and iterating over fetched records. Ensure you have a database named 'test' and a user 'postgres' configured. ```python import psycopg # Connect to an existing database with psycopg.connect("dbname=test user=postgres") as conn: # Open a cursor to perform database operations with conn.cursor() as cur: # Execute a command: this creates a new table cur.execute(""" CREATE TABLE test ( id serial PRIMARY KEY, num integer, data text) """) # Pass data to fill a query placeholders and let Psycopg perform # the correct conversion (no SQL injections!) cur.execute( "INSERT INTO test (num, data) VALUES (%s, %s)", (100, "abc'def")) # Query the database and obtain data as Python objects. cur.execute("SELECT * FROM test") print(cur.fetchone()) # will print (1, 100, "abc'def") # You can use `cur.executemany()` to perform an operation in batch cur.executemany( "INSERT INTO test (num) values (%s)", [(33,), (66,), (99,)]) # You can use `cur.fetchmany()`, `cur.fetchall()` to return a list # of several records, or even iterate on the cursor cur.execute("SELECT id, num FROM test order by num") for record in cur: print(record) # Make the changes to the database persistent conn.commit() ``` -------------------------------- ### Using psycopg.sql for CREATE TABLE statement Source: https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html Illustrates how to use the psycopg.sql module to correctly format and execute a CREATE TABLE statement with a default value, ensuring proper quoting and parameter handling. ```python >>> from psycopg import sql >>> cur.execute(sql.SQL("CREATE TABLE foo (id int DEFAULT {})").format(42)) ``` -------------------------------- ### Setting Connection Options for DateStyle Source: https://www.psycopg.org/psycopg3/docs/basic/adapt.html Demonstrates how to set connection options to ensure a supported DateStyle for loading timestamp with time zone data in text format. ```python conn = psycopg.connect(options="-c datestyle=ISO,YMD") conn.execute("show datestyle").fetchone()[0] ``` -------------------------------- ### Initialize AsyncConnectionPool with open=False Source: https://www.psycopg.org/psycopg3/docs/api/pool.html To ensure compatibility with future versions, specify `open=False` during initialization and use an explicit `await pool.open()` call. This pattern is recommended over the default behavior of opening the pool immediately. ```python pool = AsyncConnectionPool(..., open=False) await pool.open() ``` -------------------------------- ### Connect, Execute, and Fetch in One Expression Source: https://www.psycopg.org/psycopg3/docs/basic/usage.html A concise way to connect to the database, execute a query, and fetch the first column of the first row in a single line. ```python print(psycopg.connect(DSN).execute("SELECT now()").fetchone()[0]) ``` -------------------------------- ### Getting Pool Statistics Source: https://www.psycopg.org/psycopg3/docs/api/pool.html Retrieves the current statistics about the pool's usage. This provides insights into connection activity and pool state. ```python pool.get_stats() ``` -------------------------------- ### Incorrect Usage of IN Operator with Lists Source: https://www.psycopg.org/psycopg3/docs/basic/adapt.html Shows an example of a SQL syntax error when attempting to use the IN operator with a Python list for array comparison. ```python conn.execute("SELECT * FROM mytable WHERE id IN %s", [[10,20,30]]) ``` -------------------------------- ### Async Connection Manual Management Source: https://www.psycopg.org/psycopg3/docs/advanced/async.html Shows how to manually manage an asynchronous connection and its cursor, separating connection establishment from context management. ```python aconn = await psycopg.AsyncConnection.connect() async with aconn: async with aconn.cursor() as cur: await cur.execute(...) ``` -------------------------------- ### Creating a Server-Side Cursor Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Illustrates the creation of a server-side cursor by providing a name. Server-side cursors offer different fetching capabilities compared to client-side cursors. ```python cur = conn.cursor(name='mycursor') ``` -------------------------------- ### cursor.nextset() Source: https://www.psycopg.org/psycopg3/docs/api/cursors.html Advances the cursor to the next result set. This is used when multiple result sets are returned, for example, by `executemany()` with `returning=True` or `execute()` with multiple commands. ```APIDOC ## cursor.nextset() ### Description Move to the result set of the next query executed through `executemany()` or to the next result set if `execute()` returned more than one. ### Return Value `bool | None`. Returns `True` if a new result is available, which will be the one methods `fetch*()` will operate on. Returns `None` otherwise. ``` -------------------------------- ### cursor.format Source: https://www.psycopg.org/psycopg3/docs/api/cursors.html Attribute to get or set the format of the data returned by queries. Can be set during cursor initialization or changed during its lifetime, and can also be overridden for single queries. ```APIDOC ## cursor.format ### Description The format of the data returned by the queries. It can be selected initially (e.g., specifying `Connection.cursor(binary=True)`) and changed during the cursor’s lifetime. It is also possible to override the value for single queries (e.g., specifying `execute(binary=True)`). ### Type `pq.Format` ### Default `TEXT` ``` -------------------------------- ### Executing and Stringifying Composed SQL Queries Source: https://www.psycopg.org/psycopg3/docs/api/sql.html Demonstrates how to execute a composed SQL query object and how to convert it into a raw SQL string for inspection or debugging. ```python cur.execute(query, (42,)) full_query = query.as_string(cur) ``` -------------------------------- ### Default Transaction Behavior in Psycopg Source: https://www.psycopg.org/psycopg3/docs/basic/transactions.html Demonstrates the default transaction behavior where operations start a new transaction. Changes are lost if not explicitly committed before closing the connection. ```python conn = psycopg.connect() # Creating a cursor doesn't start a transaction or affect the connection # in any way. cur = conn.cursor() cur.execute("SELECT count(*) FROM my_table") # This function call executes: # - BEGIN # - SELECT count(*) FROM my_table # So now a transaction has started. # If your program spends a long time in this state, the server will keep # a connection "idle in transaction", which is likely something undesired cur.execute("INSERT INTO data VALUES (%s)", ("Hello",)) # This statement is executed inside the transaction conn.close() # No COMMIT was sent: the INSERT was discarded. ``` -------------------------------- ### Fetch and Register TypeInfo Source: https://www.psycopg.org/psycopg3/docs/api/types.html Demonstrates how to fetch TypeInfo for a custom type and register it with the connection. This allows Psycopg to automatically manage arrays of this type. ```python from psycopg.adapt import Loader from psycopg.types import TypeInfo t = TypeInfo.fetch(conn, "mytype") t.register(conn) for record in conn.execute("SELECT mytypearray FROM mytable"): # records will return lists of "mytype" as string class MyTypeLoader(Loader): def load(self, data): # parse the data and return a MyType instance conn.adapters.register_loader("mytype", MyTypeLoader) for record in conn.execute("SELECT mytypearray FROM mytable"): # records will return lists of MyType instances ``` -------------------------------- ### psycopg.crdb.CrdbConnectionInfo Source: https://www.psycopg.org/psycopg3/docs/api/crdb.html `ConnectionInfo` subclass to get info about a CockroachDB database. The object is returned by the `info` attribute of `CrdbConnection` and `AsyncCrdbConnection`. The object behaves like `ConnectionInfo`, with the following differences: ```APIDOC ## CrdbConnectionInfo ### Description `ConnectionInfo` subclass to get info about a CockroachDB database. The object is returned by the `info` attribute of `CrdbConnection` and `AsyncCrdbConnection`. The object behaves like `ConnectionInfo`, with the following differences: ### Attributes * **vendor** - The `CockroachDB` string. * **server_version** - Return the CockroachDB server version connected. Return a number in the PostgreSQL format (e.g. 21.2.10 -> 210210). ``` -------------------------------- ### Create and Use a Global Connection Pool Source: https://www.psycopg.org/psycopg3/docs/advanced/pool.html Create a global connection pool that opens connections immediately upon import. Use it from other modules by importing the pool object. ```python # module db.py in your program from psycopg_pool import ConnectionPool pool = ConnectionPool(..., open=True, ...) # the pool starts connecting immediately. # in another module from .db import pool def my_function(): with pool.connection() as conn: conn.execute(...) ``` -------------------------------- ### Get CockroachDB Server Version Source: https://www.psycopg.org/psycopg3/docs/api/crdb.html The `server_version` attribute of a `CrdbConnectionInfo` object returns the CockroachDB server version as a number in PostgreSQL format (e.g., 21.2.10 becomes 210210). ```python server_version¶ ``` -------------------------------- ### Condensed Async Connection and Cursor Usage Source: https://www.psycopg.org/psycopg3/docs/advanced/async.html A more concise way to manage asynchronous connections and cursors using the 'async with await' syntax. ```python async with await psycopg.AsyncConnection.connect() as aconn: async with aconn.cursor() as cur: await cur.execute(...) ``` -------------------------------- ### Use Custom Row Factory with Connection Source: https://www.psycopg.org/psycopg3/docs/advanced/rows.html Demonstrates how to apply a custom row factory, such as DictRowFactory, when establishing a database connection. This ensures all subsequent cursors and results use the specified factory. ```python conn = psycopg.connect(row_factory=DictRowFactory) cur = conn.execute("SELECT first_name, last_name, age FROM persons") person = cur.fetchone() print(f"{person['first_name']} {person['last_name']}") ``` -------------------------------- ### `pq` – libpq wrapper module Source: https://www.psycopg.org/psycopg3/docs/api/index.html A low-level wrapper around the libpq C library. ```APIDOC ## `pq` – libpq wrapper module ### Description Provides a low-level interface to the PostgreSQL libpq C library, exposing its functionalities. ### Contents - `pq` module implementations: Core implementation details. - Module content: Overview of the exposed libpq functions and structures. - Objects wrapping libpq structures and functions: Python objects that interface with libpq. - Enumerations: Enumerated types used within the libpq wrapper. ``` -------------------------------- ### Using Connection as a Context Manager Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Demonstrates how to use the Connection class as a context manager to ensure the transaction is committed or rolled back and the connection is closed upon exiting the block. ```python with psycopg.connect() as conn: ... ``` -------------------------------- ### Timestamptz Conversion with Different Timezones Source: https://www.psycopg.org/psycopg3/docs/basic/adapt.html Demonstrates how PostgreSQL timestamptz, stored in UTC, is converted to the connection's TimeZone setting upon output. This example shows conversion to 'Europe/Rome'. ```python conn.execute("SET TIMEZONE to 'Europe/Rome'") # UTC+2 in summer conn.execute("SELECT '2042-07-01 12:00Z'::timestamptz").fetchone()[0] # UTC input datetime.datetime(2042, 7, 1, 14, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Rome')) ``` -------------------------------- ### Create a one-off ClientCursor Source: https://www.psycopg.org/psycopg3/docs/advanced/cursors.html Instantiate a `ClientCursor` directly by passing a connection object to its constructor for ad-hoc client-side binding. ```python conn = psycopg.connect(DSN) cur = psycopg.ClientCursor(conn) ``` -------------------------------- ### Server-side binding failure examples Source: https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html Demonstrates scenarios where server-side binding fails in Psycopg 3, such as with SET, NOTIFY, and CREATE TABLE statements. These typically result in SyntaxError or UndefinedParameter. ```python >>> conn.execute("SET TimeZone TO %s", ["UTC"]) Traceback (most recent call last): ... psycopg.errors.SyntaxError: syntax error at or near "$1" LINE 1: SET TimeZone TO $1 ^ >>> conn.execute("NOTIFY %s, %s", ["chan", 42]) Traceback (most recent call last): ... psycopg.errors.SyntaxError: syntax error at or near "$1" LINE 1: NOTIFY $1, $2 ^ >>> conn.execute("CREATE TABLE foo (id int DEFAULT %s)", [42]) Traceback (most recent call last): ... psycopg.errors.UndefinedParameter: there is no parameter $1 LINE 1: CREATE TABLE foo (id int DEFAULT $1) ^ ``` -------------------------------- ### Executing Many Commands Asynchronously Source: https://www.psycopg.org/psycopg3/docs/api/cursors.html Illustrates how to execute the same SQL command multiple times with different sets of parameters asynchronously. This is useful for bulk operations. ```python await cursor.executemany(query, params_seq) ``` -------------------------------- ### Get Type OID by Name Source: https://www.psycopg.org/psycopg3/docs/api/types.html Demonstrates how to obtain the Object Identifier (OID) for a PostgreSQL type given its name. This method also returns the array OID if the type name ends with '[]'. ```python >>> psycopg.adapters.types.get_oid("text[]") 1009 ``` -------------------------------- ### Enable Pool Operations Logging Source: https://www.psycopg.org/psycopg3/docs/advanced/pool.html Configure the `logging` module to capture `psycopg.pool` INFO level messages for debugging pool behavior. This example demonstrates pool usage within a multithreaded environment. ```python import time import logging from concurrent.futures import ThreadPoolExecutor, as_completed from psycopg_pool import ConnectionPool logging.basicConfig( level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s" ) logging.getLogger("psycopg.pool").setLevel(logging.INFO) pool = ConnectionPool(min_size=2) pool.wait() logging.info("pool ready") def square(n): with pool.connection() as conn: time.sleep(1) rec = conn.execute("SELECT %s * %s", (n, n)).fetchone() logging.info(f"The square of {n} is {rec[0]}.") with ThreadPoolExecutor(max_workers=4) as executor: futures = [executor.submit(square, n) for n in range(4)] for future in as_completed(futures): future.result() ``` -------------------------------- ### Synchronous Connection and Cursor Usage Source: https://www.psycopg.org/psycopg3/docs/advanced/async.html Illustrates the standard synchronous connection and cursor usage with context managers for transaction and connection management. ```python with psycopg.connect("dbname=test user=postgres") as conn: with conn.cursor() as cur: cur.execute(...) # the cursor is closed upon leaving the context # the transaction is committed, the connection closed ``` -------------------------------- ### Manage Async Pool Lifecycle with FastAPI Lifespan Source: https://www.psycopg.org/psycopg3/docs/advanced/pool.html Integrate an asynchronous connection pool with FastAPI using its lifespan function. This ensures the pool is opened before the application starts and closed upon shutdown. ```python from contextlib import asynccontextmanager from fastapi import FastAPI from psycopg_pool import AsyncConnectionPool pool = AsyncConnectionPool(..., open=False, ...) @asynccontextmanager async def lifespan(instance: FastAPI): await pool.open() yield await pool.close() app = FastAPI(lifespan=lifespan) ``` -------------------------------- ### Verify Unaffected Connection with Registered Adapters Source: https://www.psycopg.org/psycopg3/docs/basic/pgtypes.html This example shows that registering Shapely adapters on a specific connection does not affect other connections. The second connection retrieves a string representation of the geometry instead of a Shapely object. ```python >>> conn2 = psycopg.connect(CONN_STR) >>> conn2.execute(""" ... SELECT ST_GeomFromGeoJSON('{ ... "type":"Point", ... "coordinates":[-48.23456,20.12345]}') ... """,).fetchone()[0] '0101000020E61000009279E40F061E48C0F2B0506B9A1F3440' ``` -------------------------------- ### Basic SQL Query Composition with psycopg.sql Source: https://www.psycopg.org/psycopg3/docs/api/sql.html Shows the recommended way to dynamically insert SQL identifiers like table names into queries using psycopg.sql.SQL and sql.Identifier. ```python from psycopg import sql cur.execute( sql.SQL("INSERT INTO {} VALUES (%s, %s)") .format(sql.Identifier('my_table')), [10, 20]) ``` -------------------------------- ### Handling Nested Composite Types Source: https://www.psycopg.org/psycopg3/docs/basic/pgtypes.html Illustrates how Psycopg 3 handles nested composite types, provided that the component types are also registered. This example shows a 'card_back' type containing a 'card' type. ```python >>> conn.execute("CREATE TYPE card_back AS (face card, back text)") >>> info2 = CompositeInfo.fetch(conn, "card_back") >>> register_composite(info2, conn) >>> conn.execute("SELECT ((8, 'hearts'), 'blue')::card_back").fetchone()[0] card_back(face=card(value=8, suit='hearts'), back='blue') ``` -------------------------------- ### Asynchronous Pipeline Context Manager Source: https://www.psycopg.org/psycopg3/docs/api/connections.html Use 'async with' to switch the connection into pipeline mode for batch operations. ```python async with conn.pipeline() as p: ... ``` -------------------------------- ### Initiating a COPY Operation Asynchronously Source: https://www.psycopg.org/psycopg3/docs/api/cursors.html Demonstrates how to initiate an asynchronous COPY operation to transfer data between the database and the application. It requires using an async with statement to manage the copy operation. ```python async with cursor.copy() as copy: ... ``` -------------------------------- ### Force psycopg implementation using environment variable Source: https://www.psycopg.org/psycopg3/docs/api/pq.html You can force the use of a specific psycopg implementation (e.g., 'c') by setting the PSYCOPG_IMPL environment variable before importing the library. This example shows an ImportError if the requested implementation is not available. ```bash $ PSYCOPG_IMPL=c python -c "import psycopg" Traceback (most recent call last): ... ImportError: couldn't import requested psycopg 'c' implementation: No module named 'psycopg_c' ``` -------------------------------- ### Create a ClientCursor using cursor_factory Source: https://www.psycopg.org/psycopg3/docs/advanced/cursors.html Instantiate a connection with `ClientCursor` as the default cursor factory to use client-side binding for all cursors created from this connection. ```python from psycopg import connect, ClientCursor conn = psycopg.connect(DSN, cursor_factory=ClientCursor) cur = conn.cursor() # ``` -------------------------------- ### Classic Parameterized Query Equivalent Source: https://www.psycopg.org/psycopg3/docs/basic/tstrings.html This shows the equivalent of the template string INSERT statement using the traditional %s placeholder and a tuple of parameters. ```python cursor.execute( "INSERT INTO mytable (first_name, last_name) VALUES (%s, %s)", (first_name, last_name), ) ``` -------------------------------- ### Using Connection Context for Automatic Transactions Source: https://www.psycopg.org/psycopg3/docs/basic/transactions.html Illustrates how to use the connection context manager (`with psycopg.connect()`) to automatically commit transactions upon successful block exit or roll back on exceptions. ```python with psycopg.connect() as conn: cur = conn.cursor() cur.execute("SELECT count(*) FROM my_table") # This function call executes: # - BEGIN # - SELECT count(*) FROM my_table # So now a transaction has started. cur.execute("INSERT INTO data VALUES (%s)", ("Hello",)) # This statement is executed inside the transaction # No exception at the end of the block: # COMMIT is executed. ``` -------------------------------- ### SQLSTATE Exceptions Lookup Source: https://www.psycopg.org/psycopg3/docs/api/errors.html Psycopg provides specific exception classes for each SQLSTATE error code, allowing for idiomatic error handling. The `lookup` function can be used to dynamically get the exception class for a given SQLSTATE code or constant name. ```APIDOC ## Function: psycopg.errors.lookup(_sqlstate: str_) → type[Error] ### Description Lookup an error code or constant name and return its exception class. Raise `KeyError` if the code is not found. ### Parameters - **_sqlstate** (str) - The SQLSTATE code or error constant name to look up. ### Returns - type[Error] - The exception class corresponding to the SQLSTATE code. ### Raises - `KeyError` - If the SQLSTATE code is not found. ### Example ```python try: cur.execute("LOCK TABLE mytable IN ACCESS EXCLUSIVE MODE NOWAIT") except psycopg.errors.lookup("UNDEFINED_TABLE") as e: print("Table not found") except psycopg.errors.lookup("55P03") as e: print("Lock not available") ``` ``` -------------------------------- ### Format Named Placeholders Source: https://www.psycopg.org/psycopg3/docs/api/sql.html Shows how to generate SQL queries with named placeholders (e.g., %(name)s) by passing identifier names to sql.Placeholder. ```python >>> names = ['foo', 'bar', 'baz'] >>> q2 = sql.SQL("INSERT INTO my_table ({}) VALUES ({})").format( ... sql.SQL(', ').join(map(sql.Identifier, names)), ... sql.SQL(', ').join(map(sql.Placeholder, names))) >>> print(q2.as_string(conn)) INSERT INTO my_table ("foo", "bar", "baz") VALUES (%(foo)s, %(bar)s, %(baz)s) ```