RelStorage

https://relstorage.readthedocs.io/en/latest/
Admin
RelStorage is a storage implementation for ZODB that stores pickles in a relational database...
Tokens:307,923
Snippets:4,116
Trust Score:9.6
Update:3 months ago
Show doc for...
Tokens:
Raw
### RelStorage Configuration Example

Source: https://relstorage.readthedocs.io/en/latest/_sources/configure-application.rst.txt

Example configuration for setting up RelStorage with a MySQL database. Ensure the correct versions of RelStorage and database drivers are installed.

```ini
rel-storage =
    type mysql
    db plone
    user plone
    passwd PASSWORD
```

--------------------------------

### Install RelStorage with Database Adapters

Source: https://relstorage.readthedocs.io/en/latest/install.html

Install RelStorage with specific database adapter extras. This is the easiest way to get the recommended and tested database adapter for your platform and database. Ensure you have the necessary database client software and development libraries installed.

```bash
pip install "RelStorage[mysql]"

```

```bash
pip install "RelStorage[postgresql]"

```

```bash
pip install "RelStorage[oracle]"

```

```bash
pip install "RelStorage[sqlite3]"

```

--------------------------------

### Start SYS Session in Oracle

Source: https://relstorage.readthedocs.io/en/latest/developing/running.html

Initiates a SQL*Plus session with SYS privileges for Oracle database setup. Requires 'oracle' user privileges.

```bash
$ su - oracle
$ sqlplus / as sysdba

```

--------------------------------

### PostgreSQL and MySQL URI Examples

Source: https://relstorage.readthedocs.io/en/latest/zodburi.html

Examples of postgres and mysql URIs, including user, password, host, port, database name, and query string arguments like connection_cache_size.

```text
postgres://someuser:somepass@somehost:5432/somedb?connection_cache_size=20000
mysql://someuser:somepass@somehost:5432/somedb?connection_cache_size=20000

```

--------------------------------

### Install RelStorage with Oracle adapter

Source: https://relstorage.readthedocs.io/en/latest/_sources/install.rst.txt

Install RelStorage with the Oracle database adapter using pip extras. Ensure database client software and development libraries are installed.

```bash
pip install "RelStorage[oracle]"
```

--------------------------------

### PostgreSQL Test Database Setup (psql)

Source: https://relstorage.readthedocs.io/en/latest/_sources/developing/running.rst.txt

Shell commands to set up PostgreSQL databases and users for testing.

```shell
.. literalinclude:: ../../.travis/postgres.sh
   :language: shell
```

--------------------------------

### Install RelStorage with SQLite3 adapter

Source: https://relstorage.readthedocs.io/en/latest/_sources/install.rst.txt

Install RelStorage with the SQLite3 database adapter using pip extras. Ensure database client software and development libraries are installed.

```bash
pip install "RelStorage[sqlite3]"
```

--------------------------------

### Install RelStorage with Test and MySQL Dependencies

Source: https://relstorage.readthedocs.io/en/latest/_sources/developing/running.rst.txt

Install RelStorage in editable mode with necessary test and MySQL driver dependencies.

```bash
pip install -e ".[test,mysql]"
```

--------------------------------

### Install RelStorage with MySQL adapter

Source: https://relstorage.readthedocs.io/en/latest/_sources/install.rst.txt

Install RelStorage with the MySQL database adapter using pip extras. Ensure database client software and development libraries are installed.

```bash
pip install "RelStorage[mysql]"
```

--------------------------------

### PostgreSQL URI Example

Source: https://relstorage.readthedocs.io/en/latest/_sources/zodburi.rst.txt

Example of a PostgreSQL URI with user, password, host, port, database name, and connection cache size.

```sql
postgres://someuser:somepass@somehost:5432/somedb?connection_cache_size=20000
```

--------------------------------

### Example Zlib Storage Configuration

Source: https://relstorage.readthedocs.io/en/latest/zodbconvert.html

This configuration shows a zlib storage wrapper around a file storage.

```ini
<zlibstorage>
  <filestorage>
    path /zope/var/Data.fs
  </filestorage>
</zlibstorage>


```

--------------------------------

### MySQL URI Example

Source: https://relstorage.readthedocs.io/en/latest/_sources/zodburi.rst.txt

Example of a MySQL URI with user, password, host, port, database name, and connection cache size.

```sql
mysql://someuser:somepass@somehost:5432/somedb?connection_cache_size=20000
```

--------------------------------

### SQLite URI Example

Source: https://relstorage.readthedocs.io/en/latest/_sources/zodburi.rst.txt

Example of a SQLite URI, which only requires the path to the data directory.

```sql
sqlite://?data_dir=/the/path/to/database
```

--------------------------------

### MySQL Test Setup Script

Source: https://relstorage.readthedocs.io/en/latest/_sources/developing/running.rst.txt

Shell commands to set up MySQL databases and users for testing.

```shell
.. literalinclude:: ../../.travis/mysql.sh
   :language: shell
```

--------------------------------

### Install RelStorage using pip

Source: https://relstorage.readthedocs.io/en/latest/_sources/install.rst.txt

Install the core RelStorage package using pip. This command installs the latest compatible version.

```bash
pip install RelStorage
```

--------------------------------

### SQLite URI Example

Source: https://relstorage.readthedocs.io/en/latest/zodburi.html

Example of a SQLite URI, specifying the directory for database files using the data_dir query argument.

```text
sqlite://?data_dir=/the/path/to/database

```

--------------------------------

### Oracle URI Example

Source: https://relstorage.readthedocs.io/en/latest/_sources/zodburi.rst.txt

Example of an Oracle URI, which only requires the DSN.

```sql
oracle://?dsn="HERE GOES THE DSN"
```

--------------------------------

### Install RelStorage with PostgreSQL adapter

Source: https://relstorage.readthedocs.io/en/latest/_sources/install.rst.txt

Install RelStorage with the PostgreSQL database adapter using pip extras. This attempts to install from source and may require database client software and development libraries.

```bash
pip install "RelStorage[postgresql]"
```

--------------------------------

### Example zodbpack Configuration File

Source: https://relstorage.readthedocs.io/en/latest/zodbpack.html

An example configuration file in ZConfig format for the zodbpack script, specifying a RelStorage with MySQL backend and garbage collection enabled.

```zconfig
<relstorage>
  pack-gc true
  <mysql>
    db zodb
  </mysql>
</relstorage>

```

--------------------------------

### zodbconvert Configuration File Example

Source: https://relstorage.readthedocs.io/en/latest/_sources/zodbconvert.rst.txt

This configuration defines a FileStorage as the source and a MySQL-based RelStorage as the destination for data conversion using zodbconvert.

```ini
<filestorage source>
  path /zope/var/Data.fs
</filestorage>

<relstorage destination>
  <mysql>
    db zodb
  </mysql>
</relstorage>
```

--------------------------------

### Install PostgreSQL Procedures

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/postgresql/schema.html

Installs necessary stored procedures if they are not already installed. It first checks for existing procedures and then proceeds with installation if required. It also disables function body checking during installation to handle interdependencies.

```python
def create_procedures(self, cursor):
        if not self.__all_procedures_installed(cursor):
            self.__install_procedures(cursor)
            if not self.__all_procedures_installed(cursor):
                raise AssertionError(
                    "Could not get version information after "
                    "installing the stored procedures.")
```

```python
def __install_procedures(self, cursor):
        """Install the stored procedures"""
        self.__install_languages(cursor)

        # PostgreSQL procedures in the SQL language
        # do lots of validation at compile time; in particular,
        # they check that the functions they use in SELECT statements
        # actually exist. When we have procedures that call each other,
        # that means there's an order they have to be created in.
        # Rather than try to figure out what that order is, or encode it
        # in names somehow, we'll disable that validation for the duration
        # of this transaction (just like pg_dump.)
        cursor.execute('SET LOCAL check_function_bodies = off')
```

--------------------------------

### Instantiate RelStorage with Options

Source: https://relstorage.readthedocs.io/en/latest/relstorage.options.html

Demonstrates how to instantiate RelStorage with packing options. These parameters can be provided as keyword options in the RelStorage constructor.

```python
storage = RelStorage(adapter, pack_gc=True, pack_prepack_only=True)
```

--------------------------------

### Create MySQL User and Database

Source: https://relstorage.readthedocs.io/en/latest/_sources/mysql/setup.rst.txt

Sample SQL statements to create a dedicated user, database, and grant necessary privileges for RelStorage.

```sql
CREATE USER 'zodbuser'@'localhost' IDENTIFIED BY 'mypassword';
CREATE DATABASE zodb;
GRANT ALL ON zodb.* TO 'zodbuser'@'localhost';
FLUSH PRIVILEGES;
```

--------------------------------

### Check All Procedures Installed

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/postgresql/schema.html

Verifies if all required stored procedures are installed and up-to-date in the database. It compares the expected set of procedures with those currently installed.

```python
def __all_procedures_installed(self, cursor):
        """
        Check whether all required stored procedures are installed.

        Returns True only if all required procedures are installed and
        up to date.
        """

        expected = self.procedures

        installed = self.list_procedures(cursor)
        # If the database evolves over tiem, there could be
        # extra procs still there that we don't care about.
        installed = {k: v for k, v in installed.items() if k in expected}
        if installed != expected:
            logger.info(
                "Procedures incorrect, will reinstall. "
                "Expected: %s."
                "Actual: %s",
                expected, installed
            )
            return False
        return True
```

--------------------------------

### Install Oracle Package

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/oracle/schema.html

Installs or updates the 'relstorage_op' package. It checks the current version and installs the appropriate package code (history-preserving or history-free).

```python
def install_package(self, cursor):
        """Install the package containing stored procedures"""
        if self.keep_history:
            code = oracle_history_preserving_package
        else:
            code = oracle_history_free_package

        for stmt in code.split('\n/\n'):
            if stmt.strip():
                cursor.execute(stmt)
```

--------------------------------

### Show cache_simul.py Help

Source: https://relstorage.readthedocs.io/en/latest/_sources/cache-tracing.rst.txt

Displays the command-line syntax for the cache_simul.py tool. This tool simulates ZEO client cache behavior based on trace file events.

```bash
python -m ZEO.scripts.cache_simul --help
```

--------------------------------

### Set up MySQL Test Environment

Source: https://relstorage.readthedocs.io/en/latest/developing/running.html

Bash script to create a MySQL user, databases, and grant necessary privileges for RelStorage testing. It handles dynamic database names and connection details.

```bash
#!/bin/bash
# To be able to successfully test against a MySQL on a different host,
# it's best to set up a proxy: socat tcp-listen:3306,reuseaddr,fork tcp:192.168.99.100:3306
# This can arise when running mysql in docker on macOS, which doesn't
# correctly handle the `docker run` `-p` option to forward ports to the local host;
# specify the address of the virtual machine.
# docker run  --publish 3306:3306 --rm --name mysqld -e MYSQL_ALLOW_EMPTY_PASSWORD=yes mysql:8.0
HOST="-h ${RS_DB_HOST-localhost}"
DBNAME=${RELSTORAGETEST_DBNAME:-relstoragetest}
DBNAME2=${DBNAME}2
DBNAME_HF=${DBNAME}_hf
DBNAME2_HF=${DBNAME}2_hf
PW=$RELSTORAGETEST_MY_PW
echo $DBNAME_hf
echo $DBNAME2_hf
mysql -uroot $PW $HOST -e "CREATE USER 'relstoragetest' IDENTIFIED BY 'relstoragetest';"
mysql -uroot $PW $HOST -e "CREATE DATABASE $DBNAME;"
mysql -uroot $PW $HOST -e "GRANT ALL ON $DBNAME.* TO 'relstoragetest';"
mysql -uroot $PW $HOST -e "CREATE DATABASE $DBNAME2;"
mysql -uroot $PW $HOST -e "GRANT ALL ON $DBNAME2.* TO 'relstoragetest';"
mysql -uroot $PW $HOST -e "CREATE DATABASE $DBNAME_HF;"
mysql -uroot $PW $HOST -e "GRANT ALL ON $DBNAME_HF.* TO 'relstoragetest';"
mysql -uroot $PW $HOST -e "CREATE DATABASE $DBNAME2_HF;"
mysql -uroot $PW $HOST -e "GRANT ALL ON $DBNAME2_HF.* TO 'relstoragetest';"
mysql -uroot $PW $HOST -e "GRANT SELECT ON performance_schema.* TO 'relstoragetest'"
mysql -uroot $PW $HOST -e "GRANT SELECT ON sys.* TO 'relstoragetest'"
mysql -uroot $PW $HOST -e "GRANT PROCESS ON *.* TO 'relstoragetest'"
mysql -uroot $PW $HOST -e "SELECT version()"
mysql -uroot $PW $HOST -e "FLUSH PRIVILEGES;"

```

--------------------------------

### PostgreSQL URI with Query String Arguments

Source: https://relstorage.readthedocs.io/en/latest/zodburi.html

An example demonstrating a PostgreSQL URI that includes connection details and query string arguments for cache size.

```text
postgres://someuser:somepass@somehost:5432/somedb?connection_cache_size=20000

```

--------------------------------

### Create RelStorage Options Instance

Source: https://relstorage.readthedocs.io/en/latest/relstorage.options.html

Shows how to create an Options instance and pass it to the RelStorage constructor. This is an alternative to passing keyword options directly.

```python
options = Options(pack_gc=True, pack_prepack_only=True)
storage = RelStorage(adapter, options=options)
```

--------------------------------

### Schema Installer Interface: Prepare Schema

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/interfaces.html

Ensures the database schema exists and is up-to-date. It creates the schema if it doesn't exist, performs necessary migration steps, and then calls `verify`.

```python
def prepare():
    """
    Create the database schema if it does not already exist.

    Perform any migration steps needed, and call :meth:`verify`
    before returning.
    """
```

--------------------------------

### Handling Object References in Pre-Pack

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/packundo.html

Shows how to retrieve references from an object's state and insert them into the 'object_ref' table. Includes error handling for unpickling issues.

```Python
if state:
                    try:
                        to_oids = get_references(state)
                    except:
                        logger.exception(
                            "pre_pack: can't unpickle "
                            "object %d in transaction %d; state length = %d",
                            from_oid, tid, len(state)
                        )
                        raise

                    for to_oid in to_oids:
                        row = (from_oid, tid, to_oid)
                        num_refs_found += 1
                        store_batcher.insert_into(
                            'object_ref (zoid, tid, to_zoid)',
                            object_ref_schema,
                            row,
                            row,
                            size=3
                        )
```

--------------------------------

### Oracle URI Example

Source: https://relstorage.readthedocs.io/en/latest/zodburi.html

Example of an Oracle URI, which primarily uses a Data Source Name (DSN) for connection information.

```text
oracle://?dsn="HERE GOES THE DSN"

```

--------------------------------

### Sqlite3Adapter Initialization

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/sqlite/adapter.html

Illustrates the initialization of the Sqlite3Adapter, setting up essential components for SQLite database operations. This includes defining the data directory, pragmas, and optional components like oidallocator, locker, mover, and connmanager.

```python
# -*- coding: utf-8 -*-
##############################################################################
#
# Copyright (c) 2019 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""sqlite3 adapter for RelStorage."""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import os.path

from zope.interface import implementer

from ..adapter import AbstractAdapter
from ..dbiter import HistoryFreeDatabaseIterator
from ..dbiter import HistoryPreservingDatabaseIterator
from ..interfaces import IRelStorageAdapter
from ..packundo import HistoryFreePackUndo
from ..packundo import HistoryPreservingPackUndo

from . import drivers
from .batch import Sqlite3RowBatcher
from .connmanager import Sqlite3ConnectionManager
from .locker import Sqlite3Locker
from .mover import Sqlite3ObjectMover
from .oidallocator import Sqlite3OIDAllocator
from .schema import Sqlite3SchemaInstaller
from .stats import Sqlite3Stats
from .txncontrol import Sqlite3TransactionControl
from .poller import Sqlite3Poller
from .scriptrunner import Sqlite3ScriptRunner




@implementer(IRelStorageAdapter)
class Sqlite3Adapter(AbstractAdapter): # pylint:disable=too-many-instance-attributes
    """
    Adapter for sqlite.
    """
    driver_options = drivers
    WRITING_REQUIRES_EXCLUSIVE_LOCK = True

    def __init__(self, data_dir, pragmas,
                 options=None, oidallocator=None,
                 locker=None,
                 mover=None,
                 connmanager=None):
        self.data_dir = os.path.abspath(data_dir)
        self.pragmas = pragmas
        self.oidallocator = oidallocator
        self.locker = locker
        self.mover = mover
        self.connmanager = connmanager
        super().__init__(options)

    def _create(self):
        driver = self.driver
        options = self.options
        if self.connmanager is None:
            self.connmanager = Sqlite3ConnectionManager(
                driver,
                path=os.path.join(self.data_dir, 'main.sqlite3'),
                pragmas=self.pragmas,
                options=options
            )

        if self.mover is None:
            self.mover = Sqlite3ObjectMover(
                driver,
                options=options,
            )
        if self.locker is None:
            self.locker = Sqlite3Locker(
                options,
                driver,
                batcher_factory=Sqlite3RowBatcher)

        if not self.oidallocator:
            self.oidallocator = Sqlite3OIDAllocator(
                os.path.join(self.data_dir, 'oids.sqlite3'),
                # No switching during OID allocation. It holds an exclusive
                # lock anyway.
                driver=drivers.Sqlite3Driver()
            )

        self.runner = Sqlite3ScriptRunner()
        self.poller = Sqlite3Poller(
            self.driver,
            keep_history=self.keep_history,
            runner=self.runner,
            revert_when_stale=options.revert_when_stale,
            transactions_may_go_backwards=False
        )

        self.txncontrol = Sqlite3TransactionControl(
            connmanager=self.connmanager,
            poller=self.poller,
            keep_history=self.keep_history,
            Binary=driver.Binary,
        )


        self.schema = Sqlite3SchemaInstaller(
            driver=driver,
            oid_allocator=self.oidallocator,
            connmanager=self.connmanager,
            runner=self.runner,
            keep_history=self.keep_history
        )

        self.stats = Sqlite3Stats(
            self.connmanager,
            self.keep_history
        )

        if self.keep_history:
            self.packundo = HistoryPreservingPackUndo(
                driver,
                connmanager=self.connmanager,
                runner=self.runner,
                locker=self.locker,
                options=options,
            )
            self.dbiter = HistoryPreservingDatabaseIterator(
                driver,
            )
        else:
            self.packundo = HistoryFreePackUndo(
                driver,
                connmanager=self.connmanager,
                runner=self.runner,
                locker=self.locker,
                options=options,
            )
            self.dbiter = HistoryFreeDatabaseIterator(
                driver,
            )

```

--------------------------------

### Initialize PostgreSQL Adapter

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/postgresql/adapter.html

Initializes the PostgreSQL adapter, setting up packundo and dbiter based on whether history preservation is enabled. It also initializes statistics.

```python
        if self.keep_history:
            self.packundo = HistoryPreservingPackUndo(
                driver,
                connmanager=self.connmanager,
                runner=self.runner,
                locker=self.locker,
                options=options,
            )
            self.dbiter = HistoryPreservingDatabaseIterator(
                driver,
            )
        else:
            self.packundo = HistoryFreePackUndo(
                driver,
                connmanager=self.connmanager,
                runner=self.runner,
                locker=self.locker,
                options=options,
            )
            self.dbiter = HistoryFreeDatabaseIterator(
                driver,
            )

        self.stats = PostgreSQLStats(
            connmanager=self.connmanager,
            keep_history=self.keep_history
        )
```

--------------------------------

### Install PostgreSQL Triggers

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/postgresql/schema.html

Installs the 'blob_chunk_delete' trigger if it is not already present in the database. It first lists existing triggers to check for its presence.

```python
def create_triggers(self, cursor):
        triggers = self.list_triggers(cursor)
        __traceback_info__ = triggers, self.list_tables(cursor), self.get_database_name(cursor)
        if 'blob_chunk_delete' not in triggers:
            self.__install_triggers(cursor)
```

--------------------------------

### AbstractConnectionManager Initialization

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/connmanager.html

Initializes the AbstractConnectionManager with options and a driver, setting up replica selectors and ignored exceptions.

```python
def __init__(self, options, driver):
    """
    :param options: A :class:`relstorage.options.Options`.
    :param driver: A :class:`relstorage.adapters.interfaces.IDBDriver`,
        which we use for its exceptions.
    """
    self.keep_history = options.keep_history
    self.driver = driver
    self.options = options

    self._ignored_exceptions = tuple(set(
        driver.close_exceptions
        + driver.disconnected_exceptions
        + (ReplicaClosedException,)
    ))

    if options.replica_conf:
        self.replica_selector = ReplicaSelector(
            options.replica_conf, options.replica_timeout)

    if options.ro_replica_conf:
        self.ro_replica_selector = ReplicaSelector(
            options.ro_replica_conf, options.replica_timeout)
    else:
        self.ro_replica_selector = self.replica_selector
```

--------------------------------

### Create New Adapter Instance

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/postgresql/adapter.html

Creates and returns a new instance of the adapter, copying configuration from the current instance.

```python
    def new_instance(self):
        inst = type(self)(
            dsn=self._dsn,
            options=self.options,
            oidallocator=self.oidallocator.new_instance(),
            locker=self.locker,
            mover=self.mover,
            connmanager=self.connmanager,
        )
        return inst
```

--------------------------------

### Install PL/pgSQL Language

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/postgresql/schema.html

Installs the 'plpgsql' language if it is not already available in the PostgreSQL database. This is a prerequisite for using stored procedures written in PL/pgSQL.

```python
def __install_languages(self, cursor):
        if 'plpgsql' not in self.list_languages(cursor):
            cursor.execute("CREATE LANGUAGE plpgsql")
```

--------------------------------

### List Oracle Installed Packages

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/oracle/schema.html

Lists installed stored procedure packages and their versions. It queries user_objects and user_source to find package names and extract version information.

```python
def list_packages(self, cursor):
        """List installed stored procedure packages.

        Returns {package name: version}.  version may be None.
        """
        stmt = """
        SELECT object_name
        FROM user_objects
        WHERE object_type = 'PACKAGE'
        """
        cursor.execute(stmt)
        names = [name for (name,) in cursor.fetchall()]

        res = {}
        for name in names:
            version = None
            stmt = """
            SELECT TEXT FROM USER_SOURCE
            WHERE TYPE='PACKAGE BODY'
                AND NAME=:1
            """
            cursor.execute(stmt, (name,))
            for (text,) in cursor:
                match = re.search(r'Version:\s*([0-9a-zA-Z.]+)', text)
                if match is not None:
                    version = match.group(1)
                    break
            res[name.lower()] = version
        return res
```

--------------------------------

### Schema Installer Interface: Verify Schema

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/interfaces.html

Checks if the currently installed database schema is compatible with this RelStorage configuration. Raises an exception if there's a mismatch, such as a history-preserving schema with a history-free configuration.

```python
def verify():
    """
    Ensure that the schema that's installed can be used by this
    RelStorage.

    If it cannot, for example it's history-preserving and we were configured
    to be history-free, raise an exception.
    """
```

--------------------------------

### PostgreSQL Adapter Initialization

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/postgresql/adapter.html

Initializes the PostgreSQL adapter, setting up essential components like the connection manager, locker, schema installer, mover, and object allocator. This is the primary entry point for configuring the PostgreSQL backend.

```python
@implementer(IRelStorageAdapter)
class PostgreSQLAdapter(AbstractAdapter):
    """PostgreSQL adapter for RelStorage."""
    # pylint:disable=too-many-instance-attributes

    driver_options = drivers

    def __init__(self, dsn='', options=None, oidallocator=None,
                 locker=None,
                 mover=None,
                 connmanager=None
                 ):
        # options is a relstorage.options.Options or None
        self._dsn = dsn
        self.oidallocator = oidallocator
        self.locker = locker
        self.mover = mover
        self.connmanager = connmanager
        super().__init__(options)

    def _create(self):
        driver = self.driver
        options = self.options
        dsn = self._dsn

        self.version_detector = PostgreSQLVersionDetector()
        if self.connmanager is None:
            self.connmanager = Psycopg2ConnectionManager(
                driver,
                dsn=dsn,
                options=options,
            )
        self.runner = ScriptRunner()
        if self.locker is None:
            self.locker = PostgreSQLLocker(
                options,
                driver,
                PostgreSQLRowBatcher,
            )
        self.schema = PostgreSQLSchemaInstaller(
            options=options,
            connmanager=self.connmanager,
            runner=self.runner,
            locker=self.locker,
        )

        if self.mover is None:
            self.mover = PostgreSQLObjectMover(
                driver,
                options=options,
                runner=self.runner,
                version_detector=self.version_detector,
                batcher_factory=PostgreSQLRowBatcher,
            )
        if self.oidallocator is None:
            self.oidallocator = PostgreSQLOIDAllocator()

        self.poller = PGPoller(
            self.driver,
            keep_history=self.keep_history,
            runner=self.runner,
            revert_when_stale=options.revert_when_stale,
            transactions_may_go_backwards=(
                self.connmanager.replica_selector is not None
                or self.connmanager.ro_replica_selector is not None
            )
        )

        self.txncontrol = PostgreSQLTransactionControl(
            connmanager=self.connmanager,
            poller=self.poller,
            keep_history=self.keep_history,
            Binary=driver.Binary,
        )
```

--------------------------------

### ISchemaInstaller

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/interfaces.html

Interface for installing, verifying, clearing, and dropping the database schema.

```APIDOC
## prepare

### Description
Create the database schema if it does not already exist. Perform any migration steps needed, and call :meth:`verify` before returning.

## verify

### Description
Ensure that the schema that's installed can be used by this RelStorage. If it cannot, for example it's history-preserving and we were configured to be history-free, raise an exception.

## zap_all

### Description
Clear all data out of the database.

## drop_all

### Description
Drop all tables and sequences.
```

--------------------------------

### iter_transactions_range

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/dbiter.html

Iterates over a range of transactions specified by start and stop TID values.

```APIDOC
## iter_transactions_range

### Description
Iterates over a range of transactions, from `start` to `stop` TID. If `start` or `stop` are not provided, defaults are used (0 for start, MAX_TID for stop).

### Method
`iter_transactions_range(cursor, start=None, stop=None)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
None

### Response
#### Success Response (200)
Returns an iterable of transaction records within the specified range.

#### Response Example
None
```

--------------------------------

### StoreConnection Begin Transaction

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/connections.html

Initiates a transaction on the store connection, opening it if necessary.

```Python
    def begin(self):
        self.connmanager.begin(*self.open_if_needed())
```

--------------------------------

### Reset Object ID Sequence

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/interfaces.html

Resets the OID allocation sequence to begin from the start.

```python
def reset_oid(cursor):
    """
    Cause the sequence of OIDs to begin again from the beginning.
    """

```

--------------------------------

### Getting Pooled Connection Count

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/connections.html

Returns the current number of connections available in the pool.

```python
    @property
    def pooled_connection_count(self):
        return len(self._connections)
```

--------------------------------

### Configure PostgreSQL pg_hba.conf

Source: https://relstorage.readthedocs.io/en/latest/developing/running.html

Add these lines to the top of your pg_hba.conf file to allow local connections for RelStorage test databases. Ensure they are not overridden by parameters lower in the file.

```text
local   relstoragetest     relstoragetest   md5
local   relstoragetest2    relstoragetest   md5
local   relstoragetest_hf  relstoragetest   md5
local   relstoragetest2_hf relstoragetest   md5
host    relstoragetest     relstoragetest   127.0.0.1/32 md5
host    relstoragetest_hf  relstoragetest   127.0.0.1/32 md5

```

--------------------------------

### Schema Installer Interface: Zap All Data

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/interfaces.html

Clears all data from the database, leaving the schema intact.

```python
def zap_all():
    """Clear all data out of the database."""
```

--------------------------------

### List PostgreSQL Languages

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/postgresql/schema.html

Retrieves a list of all installed languages in the PostgreSQL database by querying 'pg_catalog.pg_language'.

```sql
SELECT lanname FROM pg_catalog.pg_language
```

```python
def list_languages(self, cursor):
        cursor.execute("SELECT lanname FROM pg_catalog.pg_language")
        return self.__native_names_only(cursor)
```

--------------------------------

### Initialize Temporary Tables on Store Open

Source: https://relstorage.readthedocs.io/en/latest/_modules/relstorage/adapters/sqlite/mover.html

The `on_store_opened` method initializes temporary tables required for blob chunk storage and general temporary data. It uses `Sqlite3ScriptRunner` to execute the setup script and ensures the connection is not in a transaction before execution.

```python
@metricmethod_sampled
    def on_store_opened(self, cursor, restart=False):
        """Create the temporary table for storing objects"""
        # Don't use cursor.executescript, it commits. Remember our store
        # connection is usually meant to be in auto-commit mode and shouldn't take
        # exclusive locks until tpc_vote time. Prior to Python 3.6, if we were
        # in a transaction this would commit it. but we shouldn't be in a transaction.
        if not restart:
            assert not cursor.connection.in_transaction, cursor
            Sqlite3ScriptRunner().run_script(cursor, self.__on_store_opened_script)
            assert not cursor.connection.in_transaction
            cursor.connection.register_before_commit_cleanup(self._before_commit)

        super().on_store_opened(cursor, restart)

```

--------------------------------

### Control Packing Process Options

Source: https://relstorage.readthedocs.io/en/latest/changelog.html

Use `--prepack` to analyze data for packing separately from the actual packing phase. Use `--use-prepack-state` to apply a previously generated pre-pack state.

```bash
zodbpack --prepack
zodbpack --use-prepack-state
```

--------------------------------

### Configure Oracle with RelStorage

Source: https://relstorage.readthedocs.io/en/latest/_sources/configure-application.rst.txt

Configure RelStorage for Oracle (10g XE example) using ZConfig.

```xml
<relstorage>
  <oracle>
    user username
    password pass
    dsn XE
  </oracle>
</relstorage>
```

--------------------------------

### Run zodbpack with prepack-only option

Source: https://relstorage.readthedocs.io/en/latest/_sources/zodbpack.rst.txt

The --prepack option instructs the storage to only perform the pre-pack phase without deleting any data. This is useful for analysis before actual packing.

```bash
zodbpack --prepack
```

--------------------------------

### SQLite Configuration with PRAGMAs

Source: https://relstorage.readthedocs.io/en/latest/_sources/sqlite3/options.rst.txt

Example of configuring SQLite with specific PRAGMAs for maximum performance, potentially at the expense of safety. This includes settings like synchronous, fullfsync, and foreign_keys.

```ini
<sqlite3>
    data-dir /dev/shm
    <pragmas>
        synchronous off
        checkpoint_fullfsysnc off
        defer_foreign_keys on
        fullfsync off
        ignore_check_constraints on
        foreign_keys off
    </pragmas>
</sqlite3>
```
Tokens:
Raw