### Start PGHoard and PostgreSQL Servers (systemd)

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

After restoring the basebackup, start the PGHoard server process first, followed by the PostgreSQL server. This ensures PostgreSQL can initiate its recovery process correctly. This example assumes a systemd-based system and PostgreSQL 9.5.

```bash
systemctl start pghoard
systemctl start postgresql-9.5
```

--------------------------------

### Enable and Start PGHoard Service with systemd

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

These commands enable the pghoard service to start on boot and then start the service immediately. This is the recommended way to run PGHoard on systems with systemd.

```bash
systemctl enable pghoard.service
systemctl start pghoard.service
```

--------------------------------

### PGHoard JSON Configuration Example

Source: https://github.com/aiven-open/pghoard/blob/main/docs/configuration.rst

This is a comprehensive example of the PGHoard JSON configuration file, illustrating the structure for defining backup sites and their associated settings.

```json
{
    "json_state_file_path": "/var/lib/pghoard/pghoard_state.json"
    "backup_sites": {
        "mycluster": {
            "nodes": [
                {
                    "host": "127.0.0.1",
                    "password": "secret",
                    "port": 5432,
                    "user": "backup",
                    "slot": "pghoard"
                }
            ],
            "basebackup_count": 5,
            "basebackup_mode": "delta",
            "object_storage": {
                "storage_type": "local",
                "directory": "/tmp/pghoard/backups"
            }
        }
    }
}
```

--------------------------------

### Install Python Egg Package

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

Install the PGHoard application by running this command with the path to the generated .egg file. This is for Python/Other installations.

```bash
easy_install dist/pghoard-1.7.0-py3.6.egg
```

--------------------------------

### Install PGHoard using pip

Source: https://github.com/aiven-open/pghoard/blob/main/docs/install.rst

Install PGHoard using pip. Ensure you are using an up-to-date version.

```bash
pip install pghoard
```

--------------------------------

### Start Vagrant Development Environment

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

Use this command to bring up the Vagrant development environment. After it's running, you can connect to the instance via SSH.

```bash
vagrant up
vagrant ssh
cd /vagrant
```

--------------------------------

### PGHoard Log Output Example

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

Example log output from PGHoard indicating a successful base backup and upload process. This shows the sequence of actions performed by PGHoard.

```log
2021-07-30 15:56:48,678 PGBaseBackup    Thread-23       INFO    Started: ['/usr/bin/pg_basebackup', '--format', 'tar', '--label', 'pghoard_base_backup', '--verbose', '--pgdata', '/mnt/pghoard_backup/state/my_test_cluster/basebackup_incoming/2021-07-30_13-56_0', '--wal-method=none', '--progress', '--dbname', "dbname='replication' host='127.0.0.1' replication='true' user='pghoard'"], running as PID: 3652881, basebackup_location: '/mnt/pghoard_backup/state/my_test_cluster/basebackup_incoming/2021-07-30_13-56_0/base.tar'
2021-07-30 15:56:48,805 PGBaseBackup    Thread-23       INFO    Ran: ['/usr/bin/pg_basebackup', '--format', 'tar', '--label', 'pghoard_base_backup', '--verbose', '--pgdata', '/mnt/pghoard_backup/state/my_test_cluster/basebackup_incoming/2021-07-30_13-56_0', '--wal-method=none', '--progress', '--dbname', "dbname='replication' host='127.0.0.1' replication='true' user='pghoard'"], took: 0.127s to run, returncode: 0
2021-07-30 15:56:48,922 Compressor      Thread-3        INFO    Compressed 33009152 byte open file '/mnt/pghoard_backup/state/my_test_cluster/basebackup_incoming/2021-07-30_13-56_0/base.tar' to 6797509 bytes (21%), took: 0.091s
2021-07-30 15:56:48,925 TransferAgent   Thread-12       INFO    Uploading file to object store: src='/mnt/pghoard_backup/state/my_test_cluster/basebackup/2021-07-30_13-56_0' dst='my_test_cluster/basebackup/2021-07-30_13-56_0'
2021-07-30 15:56:48,928 TransferAgent   Thread-12       INFO    Deleting file: '/mnt/pghoard_backup/state/my_test_cluster/basebackup/2021-07-30_13-56_0' since it has been uploaded

```

--------------------------------

### Install Debian Package

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

Use this command to install the previously built .deb package on a Debian-based system. Ensure you are in the correct directory or provide the correct path to the .deb file.

```bash
dpkg -i ../pghoard*.deb
```

--------------------------------

### Start Vagrant Instance and SSH

Source: https://github.com/aiven-open/pghoard/blob/main/docs/development.rst

Initiates a Vagrant development environment and connects to it via SSH. This is the first step to begin development or testing within the isolated environment.

```bash
vagrant up
vagrant ssh
vagrant@ubuntu2004:~$ cd /vagrant
```

--------------------------------

### Install RPM Package

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

Install the built .rpm package on a Fedora or RPM-based system using the dnf command. Adjust the path if your package is located elsewhere.

```bash
dnf install rpm/RPMS/noarch/*
```

--------------------------------

### PGHoard Configuration

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

Example configuration file for PGHoard. This sets up local archiving and specifies connection details for PostgreSQL.

```json
{
    "backup_location": "/mnt/pghoard_backup/state/",
    "backup_sites": {
      "my_test_cluster": {
        "nodes": [
          {
            "host": "127.0.0.1",
            "password": "secret",
            "user": "pghoard",
            "slot": "pghoard_slot"
          }
        ],
        "object_storage": {
            "storage_type": "local",
            "directory": "/mnt/pghoard_backup/"
        },
        "pg_data_directory": "/var/lib/postgres/data/",
        "pg_receivexlog_path": "/usr/bin/pg_receivewal",
        "pg_basebackup_path": "/usr/bin/pg_basebackup",
        "basebackup_interval_hours": 24,
        "active_backup_mode": "basic"

      }
    }
}

```

--------------------------------

### Configure PostgreSQL for WAL Archival

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

These are example settings for postgresql.conf to enable WAL archival. Changing wal_level or max_wal_senders requires a PostgreSQL restart.

```sql
wal_level = archive
max_wal_senders = 4
```

--------------------------------

### Test with Python 3.11 and Postgresql 17

Source: https://github.com/aiven-open/pghoard/blob/main/docs/development.rst

Activates a Python 3.11 virtual environment, sets the PostgreSQL version to 17, runs unit tests, and then deactivates the environment. This is an example of testing specific Python and PostgreSQL version combinations.

```bash
vagrant@ubuntu2004:~$ source ~/venv3.11/bin/activate
vagrant@ubuntu2004:~$ PG_VERSION=17 make unittest
vagrant@ubuntu2004:~$ deactivate
```

--------------------------------

### PGHoard Transfer Configuration Example

Source: https://github.com/aiven-open/pghoard/blob/main/docs/configuration.rst

This snippet shows the configuration for transfer parameters, including thread count for parallel uploads and the retry limit for upload attempts.

```json
{
  "transfer": {
    "thread_count": 4,
    "upload_retries_warning_limit": 3
  }
}
```

--------------------------------

### Build Debian Package

Source: https://github.com/aiven-open/pghoard/blob/main/docs/development.rst

Builds a Debian installation package (.deb) for PGHoard from the Git checkout. This command should be run from the root directory of the PGHoard Git repository.

```bash
make deb
```

--------------------------------

### Restore Latest Basebackup with pghoard_restore

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

Fetch the latest basebackup and prepare the target directory for restoration. The target directory must be empty or non-existent. PGHoard must be running before starting PostgreSQL.

```bash
pghoard_restore get-basebackup --config /var/lib/pghoard/pghoard.json \
    --target-dir /var/lib/pgsql/9.5/data --restore-to-primary
```

--------------------------------

### Build RPM Package

Source: https://github.com/aiven-open/pghoard/blob/main/docs/development.rst

Builds an RPM installation package for PGHoard from the Git checkout. This command is typically executed from the root directory of the PGHoard Git repository.

```bash
make rpm
```

--------------------------------

### Build Python Egg Package

Source: https://github.com/aiven-open/pghoard/blob/main/docs/development.rst

Creates a Python Egg installation file for PGHoard. This command is run from the root directory of the PGHoard Git checkout and produces an egg file in a 'dist' directory.

```bash
python setup.py bdist_egg
```

--------------------------------

### Configure pg_hba.conf for replication access

Source: https://github.com/aiven-open/pghoard/blob/main/docs/install.rst

Allow the pghoard user to connect using a replication connection by editing the pg_hba.conf file. This example uses md5 authentication for local connections.

```text
# TYPE  DATABASE     USER     ADDRESS       METHOD
host    replication  pghoard  127.0.0.1/32  md5
```

--------------------------------

### PGHoard Encryption Key Configuration Snippet

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

This is an example configuration snippet showing the structure for storing encryption keys within the pghoard configuration file. It includes both private and public keys.

```json
{
    "backup_sites": {
        "my_test_site": {
            "encryption_key_id": "1",
            "encryption_keys": {
                "1": {
                    "private": "-----BEGIN PRIVATE KEY-----<actual key>-----END PRIVATE KEY-----
",
                    "public": "-----BEGIN PUBLIC KEY-----<actual key>-----END PUBLIC KEY-----
"
                }
            }
        }
    }
}
```

--------------------------------

### Launch PGHoard

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

Command to launch PGHoard with the specified configuration file. Ensure the configuration file path is correct.

```bash
pghoard pghoard.json
```

--------------------------------

### Compression Configuration

Source: https://github.com/aiven-open/pghoard/blob/main/docs/configuration.rst

Configure the compression algorithm, level, and thread count for backups. Defaults are snappy, level 3, and cpu_count + 1 threads.

```json
{
  "compression": {
    "algorithm": "snappy",
    "level": 3,
    "thread_count": 4
  }
}
```

--------------------------------

### List Basebackups with pghoard_restore

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

Use the 'list-basebackups' command to view available basebackups stored by PGHoard. This command requires the path to the pghoard.json configuration file.

```bash
pghoard_restore list-basebackups --config /var/lib/pghoard/pghoard.json
```

--------------------------------

### List Available Basebackups

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

Command to list all available base backups for a specific cluster using the pghoard_restore tool. Requires the configuration file and verbose output.

```bash
pghoard_restore list-basebackups --config pghoard.json -v
```

--------------------------------

### Test with Python 3.12 and Postgresql 18

Source: https://github.com/aiven-open/pghoard/blob/main/docs/development.rst

Activates a Python 3.12 virtual environment, sets the PostgreSQL version to 18, runs unit tests, and then deactivates the environment. This demonstrates testing another specific Python and PostgreSQL version combination.

```bash
vagrant@ubuntu2004:~$ source ~/venv3.12/bin/activate
vagrant@ubuntu2004:~$ PG_VERSION=18 make unittest
vagrant@ubuntu2004:~$ deactivate
```

--------------------------------

### Configure pg_receivexlog for Archiving

Source: https://github.com/aiven-open/pghoard/blob/main/docs/configuration.rst

This snippet shows how to configure the 'pg_receivexlog' mode for archiving. It includes options for monitoring disk space and pausing WAL receiving when free space falls below a specified threshold.

```json
{
    "pg_receivexlog": {
        "min_free_bytes": 1073741824
    }
}
```

--------------------------------

### Generate RSA Keys for Encryption

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

Use this script to generate a public/private RSA key pair for encrypting your backups. The --site and --key-id arguments are required.

```bash
pghoard_create_keys --site my_test_site --key-id 1
```

--------------------------------

### Backup Site Node Configuration

Source: https://github.com/aiven-open/pghoard/blob/main/docs/configuration.rst

Define connection details for a PostgreSQL node within a backup site. Supports libpq parameters or connection URIs. Replication slots are recommended for WAL streaming.

```json
{
  "backup_sites": {
    "mysite": {
      "nodes": [
          {
              "host": "127.0.0.1",
              "password": "secret",
              "port": 5432,
              "user": "backup",
              "slot": "pghoard",
              "sslmode": "require"
          }
      ]
    }
  }
}
```

--------------------------------

### Restore Latest Basebackup

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

Use this command to fetch the latest basebackup for restoring your PostgreSQL database to the most recent point in time. Ensure the target directory is empty or non-existent, as PGHoard will create it.

```bash
pghoard_restore get-basebackup --config pghoard.json \
    --target-dir <destination> --restore-to-primary
```

--------------------------------

### Check Backup Storage Contents

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

Use the 'tree' command to verify the contents of the final storage location after PGHoard has completed its backup process.

```bash
tree  /mnt/pghoard_backup/my_test_cluster
```

--------------------------------

### pghoard_create_keys Usage

Source: https://github.com/aiven-open/pghoard/blob/main/docs/commands.rst

Generates and outputs encryption keys in pghoard configuration format. Used for encrypting backups.

```bash
usage: pghoard_create_keys [-h] [-D] [--version] [--site SITE] --key-id KEY_ID [--bits BITS] [--config CONFIG]

-h, --help       show this help message and exit
-D, --debug      Enable debug logging
--version        show program version
--site SITE      backup site
--key-id KEY_ID  key alias as used with encryption_key_id configuration directive
--bits BITS      length of the generated key in bits, default 3072
--config CONFIG  configuration file to store the keys in
```

--------------------------------

### Configure PGHoard for Direct WAL Streaming

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

Set active_backup_mode to 'pg_receivexlog' in pghoard.json to have PGHoard read the WAL stream directly from PostgreSQL. Ensure archive_mode is disabled in postgresql.conf.

```json
{
    "backup_sites": {
        "default": {
            "active_backup_mode": "pg_receivexlog",
            ...
         },
     },
     ...
 }
```

--------------------------------

### Create Replication Slot

Source: https://github.com/aiven-open/pghoard/blob/main/docs/quickstart.rst

Use pg_receivewal to create a replication slot. This prevents WAL files from being recycled before they are consumed by PGHoard.

```bash
pg_receivewal --create-slot -S pghoard_slot -U pghoard
```

--------------------------------

### Configure PostgreSQL WAL Archiving with PGHoard

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

Use this configuration in postgresql.conf to enable PostgreSQL's WAL-archive mechanism with PGHoard. The pghoard_postgres_command will be called for each new WAL segment.

```postgresql
archive_mode = on
archive_command = pghoard_postgres_command --mode archive --site default --xlog %f
```

--------------------------------

### Create PostgreSQL replication user

Source: https://github.com/aiven-open/pghoard/blob/main/docs/install.rst

Create a PostgreSQL user with the REPLICATION attribute and set a password for it. This user is required for replication connections.

```sql
-- create the user
CREATE USER pghoard REPLICATION;
-- Setup a password for the pghoard user
\password pghoard
```

--------------------------------

### Configure PGHoard for Standalone Hot Backups

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

Set active_backup_mode to 'standalone_hot_backup' in pghoard.json for standalone hot backups. Ensure archive_mode is disabled in postgresql.conf.

```json
{
    "backup_sites": {
        "default": {
            "active_backup_mode": "standalone_hot_backup",
            ...
         },
     },
     ...
 }
```

--------------------------------

### Statsd Configuration for Monitoring

Source: https://github.com/aiven-open/pghoard/blob/main/docs/configuration.rst

Configure PGHoard to send metrics to a statsd daemon. Supports Telegraf or DataDog formats with tags.

```json
{
    "host": "<statsd address>",
    "port": <statsd port>,
    "format": "<statsd message format>",
    "tags": {
        "<tag>": "<value>"
    }
}
```

--------------------------------

### Create PGHoard PostgreSQL User

Source: https://github.com/aiven-open/pghoard/blob/main/README.rst

This SQL command creates a PostgreSQL user named 'pghoard' with the specified password and grants it replication privileges. This user is required for PGHoard to connect and perform backups.

```sql
CREATE USER pghoard PASSWORD 'putyourpasswordhere' REPLICATION;
```

--------------------------------

### pghoard Daemon Usage

Source: https://github.com/aiven-open/pghoard/blob/main/docs/commands.rst

The main pghoard daemon process handles PostgreSQL backups. It should be run under a service manager like systemd or supervisord.

```bash
usage: pghoard [-h] [-D] [--version] [-s] [--config CONFIG] [config_file]

postgresql automatic backup daemon

positional arguments:
  config_file      configuration file path (for backward compatibility)

optional arguments:
  -h, --help       show this help message and exit
  -D, --debug      Enable debug logging
  --version        show program version
  -s, --short-log  use non-verbose logging format
  --config CONFIG  configuration file path
```

--------------------------------

### Prometheus Pushgateway Configuration

Source: https://github.com/aiven-open/pghoard/blob/main/docs/configuration.rst

Enable sending metrics to a Prometheus Pushgateway with custom tags. Requires specifying the endpoint address.

```json
{
    "endpoint": "<pushgateway address>",
    "tags": {
        "<tag>": "<value>"
    }
}
```

--------------------------------

### Configure max_wal_senders in PostgreSQL

Source: https://github.com/aiven-open/pghoard/blob/main/docs/install.rst

Set the max_wal_senders parameter in PostgreSQL to allow for replication connections. This change requires a PostgreSQL restart.

```sql
max_wal_senders = 4
```

--------------------------------

### pghoard_archive_sync Usage

Source: https://github.com/aiven-open/pghoard/blob/main/docs/commands.rst

Checks for missing or outdated WAL files in the archive. Can verify archive integrity and optionally request a new basebackup on failure.

```bash
usage: pghoard_archive_sync [-h] [-D] [--version] [--site SITE] [--config CONFIG]
                              [--max-hash-checks MAX_HASH_CHECKS] [--no-verify] [--create-new-backup-on-failure]

-h, --help            show this help message and exit
-D, --debug           Enable debug logging
--version             show program version
--site SITE           pghoard site
--config CONFIG       pghoard config file
--max-hash-checks MAX_HASH_CHECKS
                      Maximum number of files for which to validate hash in addition to basic existence check
--no-verify           do not verify archive integrity
--create-new-backup-on-failure
                      request a new basebackup if verification fails
```

--------------------------------

### pghoard_archive_cleanup Usage

Source: https://github.com/aiven-open/pghoard/blob/main/docs/commands.rst

Cleans up orphan WAL files from the object store. Use --dry-run to list files without deleting.

```bash
usage: pghoard_archive_cleanup [-h] [--version] [--site SITE] [--config CONFIG] [--dry-run]

-h, --help       show this help message and exit
--version        show program version
--site SITE      pghoard site
--config CONFIG  pghoard config file
--dry-run        only list redundant segments and calculate total file size but do not delete
```

--------------------------------

### pghoard_restore Command Usage

Source: https://github.com/aiven-open/pghoard/blob/main/docs/commands.rst

A tool to restore PostgreSQL backups from pghoard or object stores. It can also configure recovery.conf for WAL restore_command.

```bash
usage: pghoard_restore [-h] [-D] [--status-output-file STATUS_OUTPUT_FILE] [--version]
                       {list-basebackups-http,list-basebackups,get-basebackup} ...

positional arguments:
      list-basebackups-http
        List available basebackups from a HTTP source
      list-basebackups
        List basebackups from an object store
      get-basebackup
        Download a basebackup from an object store

-h, --help            show this help message and exit
-D, --debug           Enable debug logging
--status-output-file STATUS_OUTPUT_FILE
                      Filename for status output JSON
--version             show program version
```

--------------------------------

### Reload PostgreSQL configuration

Source: https://github.com/aiven-open/pghoard/blob/main/docs/install.rst

Reload the PostgreSQL configuration after editing pg_hba.conf or other configuration files. This can be done via SQL or system service manager.

```sql
SELECT pg_reload_conf();
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.