### Verify rclone Installation Source: https://github.com/kjnez/django-rclone/blob/master/docs/getting-started.md Checks if the rclone command-line tool is installed and accessible in the system's PATH. This is a prerequisite for using django-rclone. ```bash rclone version ``` -------------------------------- ### List Available Backups Source: https://github.com/kjnez/django-rclone/blob/master/docs/getting-started.md Displays a list of all available database backups stored on the configured rclone remote. Shows backup name, size, and modification time. ```bash python manage.py listbackups ``` -------------------------------- ### Development Workflow Commands Source: https://github.com/kjnez/django-rclone/blob/master/README.md This bash snippet outlines the commands used for development, including dependency installation, running tests with coverage, linting, formatting checks, and type checking using uv and other tools. ```bash uv sync uv run pytest --cov --cov-branch uv run ruff check . uv run ruff format --check . uv run ty check ``` -------------------------------- ### Configure Local rclone Remote Source: https://github.com/kjnez/django-rclone/blob/master/docs/getting-started.md Sets up a new rclone remote named 'localbackup' that stores files on the local filesystem. This is useful for local testing. ```bash rclone config create localbackup local ``` -------------------------------- ### Install django-rclone Source: https://github.com/kjnez/django-rclone/blob/master/docs/index.md Install the django-rclone package using pip. This is the first step to integrating rclone with your Django project for backup management. ```bash pip install django-rclone ``` -------------------------------- ### Create and Register Custom Database Connector Source: https://context7.com/kjnez/django-rclone/llms.txt Provides an example of creating a custom database connector for Oracle by subclassing `BaseConnector`. It demonstrates implementing the `extension`, `_env`, `dump`, and `restore` methods, and how to register the custom connector in Django settings using `CONNECTOR_MAPPING` or `CONNECTORS`. ```python # myapp/connectors.py import os import subprocess from django_rclone.db.base import BaseConnector class OracleConnector(BaseConnector): """Custom connector for Oracle databases using expdp/impdp.""" @property def extension(self) -> str: return "dmp" def _env(self) -> dict[str, str]: """Set Oracle-specific environment variables.""" env = os.environ.copy() env["ORACLE_SID"] = self.name return env def dump(self) -> subprocess.Popen[bytes]: """Export Oracle database using Data Pump.""" cmd = [ "expdp", f"{self.user}/{self.password}@{self.host}:{self.port}/{self.name}", "directory=BACKUP_DIR", "dumpfile=export.dmp", ] return subprocess.Popen( cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=self._env() ) def restore(self, stdin=None) -> subprocess.Popen[bytes]: """Import Oracle database using Data Pump.""" cmd = [ "impdp", f"{self.user}/{self.password}@{self.host}:{self.port}/{self.name}", "directory=BACKUP_DIR", "dumpfile=export.dmp", ] return subprocess.Popen( cmd, stdin=stdin, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=self._env() ) # settings.py - Register the custom connector DJANGO_RCLONE = { "REMOTE": "myremote:backups", "CONNECTOR_MAPPING": { "django.db.backends.oracle": "myapp.connectors.OracleConnector", }, # Or override for specific database alias "CONNECTORS": { "oracle_db": "myapp.connectors.OracleConnector", }, } ``` -------------------------------- ### Full django-rclone Configuration Example (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md This snippet shows the complete structure of the DJANGO_RCLONE settings dictionary in Python. It includes all available options with their default values and comments explaining their purpose. Users can customize these settings to tailor backup and sync behavior. ```python DJANGO_RCLONE = { # Required "REMOTE": "", # rclone remote:path for backups # rclone binary "RCLONE_BINARY": "rclone", # Path to the rclone binary "RCLONE_CONFIG": None, # Path to rclone.conf (None = rclone default) "RCLONE_FLAGS": [], # Extra global flags passed to every rclone call # Database backups "DB_BACKUP_DIR": "db", # Subdirectory under REMOTE for DB backups "DB_FILENAME_TEMPLATE": "{database}-{datetime}.{ext}", "DB_DATE_FORMAT": "%Y-%m-%d-%H%M%S", "DB_CLEANUP_KEEP": 10, # Number of most recent backups to keep per database # Media backups "MEDIA_BACKUP_DIR": "media", # Subdirectory under REMOTE for media backups # Connector overrides "CONNECTORS": {}, # Per-database connector class (dotted path) "CONNECTOR_MAPPING": {}, # Engine-to-connector class mapping overrides } ``` -------------------------------- ### Restore Database from Latest Backup Source: https://github.com/kjnez/django-rclone/blob/master/docs/getting-started.md Restores the database from the most recent backup found on the configured rclone remote. Prompts for confirmation by default. ```bash python manage.py dbrestore ``` -------------------------------- ### Implement Pre-Restore Validation for Production Database (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/signals.md This example illustrates how to implement a pre-restore validation check, specifically to prevent accidental restores of the production database. It uses the `pre_db_restore` signal and checks an environment variable to confirm the restore operation. ```python from django.dispatch import receiver from django_rclone.signals import pre_db_restore @receiver(pre_db_restore) def confirm_restore(sender, database, path, **kwargs): if database == "default": # Custom safeguard: could check an env var or flag import os if not os.environ.get("ALLOW_RESTORE"): raise RuntimeError("Set ALLOW_RESTORE=1 to restore the production database") ``` -------------------------------- ### Restore Database from Specific Backup Source: https://github.com/kjnez/django-rclone/blob/master/docs/getting-started.md Restores the database from a specific backup file identified by its path on the rclone remote. Use --noinput for non-interactive restores. ```bash python manage.py dbrestore --input-path default-2024-01-15-120000.dump ``` -------------------------------- ### Setting the Remote Backup Location (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Demonstrates how to configure the `REMOTE` setting in Python, specifying the rclone remote and path for storing backups. Examples cover various storage backends like S3, Google Cloud Storage, SFTP, and local filesystems. ```python # S3 bucket "REMOTE": "s3:my-backup-bucket/django-backups" # Google Cloud Storage "REMOTE": "gcs:my-bucket/backups" # SFTP server "REMOTE": "mysftp:backups/production" # Local filesystem "REMOTE": "local:/var/backups/myproject" ``` -------------------------------- ### Perform Database Backup Source: https://github.com/kjnez/django-rclone/blob/master/docs/getting-started.md Executes a database backup using the django-rclone management command. This command creates a dump of the default database and uploads it to the configured rclone remote. ```bash python manage.py dbbackup ``` -------------------------------- ### Sync Media Files to Remote Source: https://github.com/kjnez/django-rclone/blob/master/docs/getting-started.md Synchronizes media files from MEDIA_ROOT to the configured rclone remote. This command uses 'rclone sync', ensuring only changed files are transferred. ```bash python manage.py mediabackup ``` -------------------------------- ### Adding Global rclone Flags (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Provides an example of configuring `RCLONE_FLAGS` in Python, allowing you to pass additional global flags to every rclone command executed by django-rclone. This can be used for settings like verbosity or bandwidth limits. ```python "RCLONE_FLAGS": [ "--verbose", "--bwlimit", "10M", "--transfers", "4", ] ``` -------------------------------- ### Configure rclone remote for django-rclone Source: https://github.com/kjnez/django-rclone/blob/master/README.md Sets up the DJANGO_RCLONE setting in Django's settings.py to specify the rclone remote and path for backups. Requires rclone to be installed and configured separately. ```python DJANGO_RCLONE = { "REMOTE": "myremote:backups", } ``` -------------------------------- ### Send Slack Notification on Database Backup (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/signals.md This example shows how to send a Slack notification after a database backup is completed. It utilizes the `post_db_backup` signal and the `requests` library to send a POST request to a Slack webhook URL. ```python import requests from django.dispatch import receiver from django_rclone.signals import post_db_backup SLACK_WEBHOOK = "https://hooks.slack.com/services/..." @receiver(post_db_backup) def notify_slack(sender, database, path, **kwargs): requests.post(SLACK_WEBHOOK, json={ "text": f"Database `{database}` backed up to `{path}`", }) ``` -------------------------------- ### Import Signals in Django AppConfig (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/signals.md This snippet shows the standard Django way to ensure custom signal handlers are imported and connected when the application starts. It involves importing the signals module within the `ready()` method of your app's `AppConfig`. ```python # myapp/apps.py from django.apps import AppConfig class MyAppConfig(AppConfig): name = "myapp" def ready(self): import myapp.signals # noqa: F401 ``` -------------------------------- ### Ping Health Check Service After Database Backup (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/signals.md This code snippet demonstrates how to ping a health check service after a database backup operation. It uses the `post_db_backup` signal and the `requests` library to send a GET request to a specified health check URL. ```python import requests from django.dispatch import receiver from django_rclone.signals import post_db_backup @receiver(post_db_backup) def ping_healthcheck(sender, **kwargs): requests.get("https://hc-ping.com/your-uuid-here") ``` -------------------------------- ### Use Database Connectors for Dump and Restore Source: https://context7.com/kjnez/django-rclone/llms.txt Shows how to use built-in database connectors (PostgreSQL, MySQL, SQLite, MongoDB) to dump and restore databases. It covers retrieving a connector via registry lookup or direct instantiation, performing dump/restore operations using streaming subprocesses, and accessing the file extension for backups. ```python from django_rclone.db.base import BaseConnector from django_rclone.db.postgresql import PgDumpConnector from django_rclone.db.registry import get_connector from django.conf import settings import subprocess # Get connector for a database alias (uses registry lookup) connector = get_connector("default") # Create connector directly with database settings db_settings = settings.DATABASES["default"] connector = PgDumpConnector(db_settings) # Dump database - returns Popen with stdout pipe dump_proc = connector.dump() backup_data = dump_proc.stdout.read() dump_proc.wait() if dump_proc.returncode != 0: error = dump_proc.stderr.read().decode() raise Exception(f"Dump failed: {error}") # Restore database - accepts stdin pipe restore_proc = connector.restore(stdin=subprocess.PIPE) restore_proc.communicate(input=backup_data) if restore_proc.returncode != 0: error = restore_proc.stderr.read().decode() raise Exception(f"Restore failed: {error}") # Get file extension for backups ext = connector.extension # "dump" for PostgreSQL, "sql" for MySQL, "sqlite3" for SQLite ``` -------------------------------- ### Initialize and Use Rclone Class Source: https://context7.com/kjnez/django-rclone/llms.txt Demonstrates initializing the Rclone class with default or custom settings and performing common rclone operations like streaming data, syncing directories, listing files, deleting, and moving files. It also shows basic error handling for RcloneError. ```python from django_rclone.rclone import Rclone from django_rclone.exceptions import RcloneError # Initialize with default settings from DJANGO_RCLONE rclone = Rclone() # Or override specific settings rclone = Rclone( remote="s3:my-bucket/backups", config="/path/to/rclone.conf", binary="/usr/local/bin/rclone", flags=["--verbose", "--bwlimit", "5M"] ) # Stream data to remote file (used by dbbackup) with open("dump.sql", "rb") as f: rclone.rcat("db/backup.sql", stdin=f) # Stream data from remote file (used by dbrestore) proc = rclone.cat("db/backup.sql") data = proc.stdout.read() proc.wait() # Sync directories (used by mediabackup/mediarestore) rclone.sync("/var/media", "myremote:backups/media") rclone.sync("myremote:backups/media", "/var/media") # List files as JSON files = rclone.lsjson("db/", recursive=True) # Returns: [{"Name": "backup.dump", "Size": 1234567, "ModTime": "2024-01-15T12:00:00Z", "IsDir": False}, ...] # Delete a file rclone.delete("db/old-backup.dump") # Move/rename a file rclone.moveto("db/temp.dump", "db/final.dump") # Error handling try: rclone.sync("/missing", "myremote:dest") except RcloneError as e: print(f"Command failed: {e.cmd}") print(f"Exit code: {e.returncode}") print(f"Error output: {e.stderr}") ``` -------------------------------- ### List Database Backups with Django Rclone Source: https://github.com/kjnez/django-rclone/blob/master/docs/commands.md This command lists all available backups for a specified database. It requires the database name as an argument. The output shows the backup name, size, and modification time. ```python python manage.py listbackups --database default ``` -------------------------------- ### MySQL/MariaDB Dump and Restore using mysqldump and mysql Source: https://github.com/kjnez/django-rclone/blob/master/docs/connectors.md This snippet illustrates the command-line syntax for backing up MySQL/MariaDB databases with mysqldump and restoring them using the mysql client. It emphasizes secure password handling via the MYSQL_PWD environment variable. ```bash # Dump command mysqldump --quick [--host HOST] [--port PORT] [--user USER] DBNAME # Restore command mysql [--host HOST] [--port PORT] [--user USER] DBNAME ``` -------------------------------- ### Environment-Specific Rclone Configuration File Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Configures Django to use a specific rclone configuration file based on an environment variable. This is useful when different environments require entirely separate rclone setups. ```python DJANGO_RCLONE = { "REMOTE": "backups:", "RCLONE_CONFIG": f"/etc/rclone/{ENVIRONMENT}.conf", } ``` -------------------------------- ### List Media Files with Django Rclone Source: https://github.com/kjnez/django-rclone/blob/master/docs/commands.md This command lists all media files stored on the remote. It does not require any arguments. The output includes the total count of media files and their total size, along with individual file paths and sizes. ```python python manage.py listbackups --media ``` -------------------------------- ### PostGIS Backup and Restore Commands (Bash) Source: https://context7.com/kjnez/django-rclone/llms.txt These bash commands illustrate how to perform database backups and restores for a PostGIS database using Django's management commands. The restore command automatically handles the creation of the PostGIS extension if it doesn't exist. ```bash # PostGIS backup works identically to PostgreSQL python manage.py dbbackup # Restore automatically runs: CREATE EXTENSION IF NOT EXISTS postgis; python manage.py dbrestore --noinput ``` -------------------------------- ### PostgreSQL Dump and Restore using pg_dump and pg_restore Source: https://github.com/kjnez/django-rclone/blob/master/docs/connectors.md This snippet demonstrates the command-line usage for dumping and restoring PostgreSQL databases using pg_dump in custom binary format and pg_restore. It highlights security best practices by using the PGPASSWORD environment variable for credentials. ```bash # Dump command pg_dump --format=custom [-h HOST] [-p PORT] [-U USER] DBNAME # Restore command pg_restore --no-owner --no-acl -d DBNAME [-h HOST] [-p PORT] [-U USER] ``` -------------------------------- ### Configuring rclone Binary Path (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Shows how to specify the `RCLONE_BINARY` setting in Python if the rclone executable is not in the system's PATH. This ensures django-rclone can locate and execute the rclone command. ```python "RCLONE_BINARY": "/usr/local/bin/rclone" ``` -------------------------------- ### Development Commands for django-rclone Source: https://github.com/kjnez/django-rclone/blob/master/CLAUDE.md This snippet outlines essential development commands for managing dependencies, linting, formatting, type checking, and running tests for the django-rclone project. It utilizes `uv` for dependency management and `ruff` for linting and formatting. ```bash uv sync # Install dependencies uv run ruff check . # Lint uv run ruff format --check . # Check formatting uv run ty check # Type check uv run pytest # Run tests ``` -------------------------------- ### Media Backup and Restore Commands Source: https://github.com/kjnez/django-rclone/blob/master/README.md Provides bash commands for synchronizing Django's MEDIA_ROOT with a remote location using rclone. Supports both syncing from local to remote and remote to local. ```bash # Sync MEDIA_ROOT to remote (incremental -- only changed files transfer) python manage.py mediabackup # Sync remote back to MEDIA_ROOT python manage.py mediarestore ``` -------------------------------- ### SQLite Database Dump and Restore using sqlite3 Source: https://github.com/kjnez/django-rclone/blob/master/docs/connectors.md This snippet details the process of dumping and restoring SQLite databases using the sqlite3 command-line tool. It shows the command for creating a dump and how to pipe SQL statements for restoration. ```bash # Dump command sqlite3 DBNAME .dump # Restore command (reads SQL from stdin) sqlite3 DBNAME ``` -------------------------------- ### Setting Database Backup Date Format (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Demonstrates configuring the `DB_DATE_FORMAT` in Python, which specifies the `strftime` format used for the `{datetime}` variable in database backup filenames. ```python "DB_DATE_FORMAT": "%Y-%m-%d-%H%M%S" ``` -------------------------------- ### Backup Database with dbbackup Command Source: https://context7.com/kjnez/django-rclone/llms.txt This bash command initiates a database backup using django-rclone's `dbbackup` management command. It supports backing up the default or a specific database, with options for automatic cleanup of old backups and silent execution for cron jobs. The output confirms the backup process and completion. ```bash # Back up the default database python manage.py dbbackup # Back up a specific database python manage.py dbbackup --database analytics # Back up and remove old backups (keeps 10 most recent by default) python manage.py dbbackup --clean # Silent mode for cron jobs python manage.py dbbackup --verbosity 0 # Output: # Backing up database 'default' to db/default-2024-01-15-120000.dump # Backup completed: db/default-2024-01-15-120000.dump ``` -------------------------------- ### Custom Oracle Connector Implementation in Python Source: https://github.com/kjnez/django-rclone/blob/master/docs/connectors.md Demonstrates how to create a custom database connector for Oracle by subclassing `BaseConnector`. It defines the backup file extension and implements `dump` and `restore` methods using `expdp` and `impdp` commands. This requires the `subprocess` module and the `django_rclone.db.base` module. ```python import os import subprocess from django_rclone.db.base import BaseConnector class OracleConnector(BaseConnector): @property def extension(self) -> str: return "dmp" def dump(self) -> subprocess.Popen[bytes]: cmd = ["expdp", f"{self.user}/{self.password}@{self.host}:{self.port}/{self.name}"] return subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) def restore(self, stdin=None) -> subprocess.Popen[bytes]: cmd = ["impdp", f"{self.user}/{self.password}@{self.host}:{self.port}/{self.name}"] return subprocess.Popen(cmd, stdin=stdin, stdout=subprocess.PIPE, stderr=subprocess.PIPE) ``` -------------------------------- ### Customizing Database Backup Filename Template (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Shows how to set the `DB_FILENAME_TEMPLATE` in Python using a format string. This allows customization of the names for database backup files, including variables like database alias, formatted datetime, and file extension. ```python "DB_FILENAME_TEMPLATE": "{database}-{datetime}.{ext}" ``` -------------------------------- ### Handle Post Database Backup Signal Source: https://github.com/kjnez/django-rclone/blob/master/README.md This Python snippet illustrates how to use Django signals to react to database backup events. It shows how to import and register a receiver for the `post_db_backup` signal to log backup information. ```python from django_rclone.signals import post_db_backup from django.dispatch import receiver import logging logger = logging.getLogger(__name__) @receiver(post_db_backup) def notify_on_backup(sender, database, path, **kwargs): logger.info("Database %s backed up to %s", database, path) ``` -------------------------------- ### Combine Compression and Encryption with Rclone Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Demonstrates stacking rclone remotes to first compress data and then encrypt it. This provides both space savings and data security. The order of operations is important. ```bash # First compress, then encrypt rclone config create myremote-compressed compress remote=myremote:backups rclone config create myremote-secure crypt remote=myremote-compressed: ``` -------------------------------- ### Configure rclone Compress Remote Source: https://github.com/kjnez/django-rclone/blob/master/README.md This bash snippet shows how to set up a compressed remote using rclone's compress backend. Alternatively, compression can be enabled by passing flags via `RCLONE_FLAGS` in Django settings. ```bash rclone config create myremote-compressed compress remote=myremote:backups ``` -------------------------------- ### MongoDB Dump and Restore using mongodump and mongorestore Source: https://github.com/kjnez/django-rclone/blob/master/docs/connectors.md This snippet outlines the command-line arguments for performing streaming backups and restores of MongoDB databases using mongodump and mongorestore with the --archive option. It also shows how to configure the authentication source. ```bash # Dump command mongodump --db DBNAME --host HOST:PORT [--username USER] [--password PASS] [--authenticationDatabase SOURCE] --archive # Restore command mongorestore --host HOST:PORT [--username USER] [--password PASS] [--authenticationDatabase SOURCE] --drop --archive ``` ```python DATABASES = { "default": { "ENGINE": "djongo", "NAME": "mydb", "HOST": "localhost", "PORT": "27017", "USER": "admin", "PASSWORD": "...", "AUTH_SOURCE": "admin", # authentication database } } ``` -------------------------------- ### Rclone Remote Configuration for Encryption and Compression (Bash) Source: https://context7.com/kjnez/django-rclone/llms.txt These bash commands demonstrate how to configure rclone remotes for encryption and compression. They show creating an encrypted remote wrapping an S3 bucket, a compressed remote, and stacking both compression and encryption. ```bash # Create encrypted remote wrapping S3 bucket rclone config create myremote-encrypted crypt \ remote=s3:my-bucket/backups \ password=$(rclone obscure your-secret-password) # Create compressed remote rclone config create myremote-compressed compress \ remote=s3:my-bucket/backups # Stack compression + encryption (compress first, then encrypt) rclone config create myremote-compressed compress \ remote=s3:my-bucket/backups rclone config create myremote-secure crypt \ remote=myremote-compressed: \ password=$(rclone obscure your-secret-password) ``` -------------------------------- ### List Backups Management Commands Source: https://github.com/kjnez/django-rclone/blob/master/README.md Provides bash commands for listing available backups managed by django-rclone. Supports listing all database backups, filtering by database, or listing media files on the remote. ```bash # List all database backups python manage.py listbackups # Filter by database python manage.py listbackups --database default # List media files on remote python manage.py listbackups --media ``` -------------------------------- ### Configuring Number of Backups to Keep (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Illustrates setting the `DB_CLEANUP_KEEP` option in Python. This determines how many recent database backups are retained per database when the `dbbackup --clean` command is used. ```python "DB_CLEANUP_KEEP": 10 ``` -------------------------------- ### Migrate django-dbbackup Signals to django-rclone Source: https://github.com/kjnez/django-rclone/blob/master/docs/migration-from-dbbackup.md Demonstrates how to migrate signal handlers from django-dbbackup to django-rclone. This involves changing the import path for signals and using the appropriate signal from django_rclone.signals. It's essential for users who have existing custom logic tied to backup completion events. ```python from django.dispatch import receiver from django_rclone.signals import post_db_backup @receiver(post_db_backup) def on_backup_complete(sender, database, path, **kwargs): # Your custom logic here pass ``` -------------------------------- ### PostGIS Extension Creation and Database Operations Source: https://github.com/kjnez/django-rclone/blob/master/docs/connectors.md This snippet shows how to ensure the PostGIS extension is available in a PostgreSQL database before restoring data. It includes the Python configuration for specifying an admin user and the SQL command for creating the extension. ```python DATABASES = { "default": { "ENGINE": "django.contrib.gis.db.backends.postgis", "NAME": "geodb", "USER": "app_user", "PASSWORD": "...", "HOST": "localhost", "ADMIN_USER": "postgres", # used for CREATE EXTENSION } } ``` ```sql CREATE EXTENSION IF NOT EXISTS postgis; ``` -------------------------------- ### Database Backup Management Commands Source: https://github.com/kjnez/django-rclone/blob/master/README.md Provides bash commands for managing database backups using django-rclone. Supports backing up default or specific databases, cleaning old backups, and restoring from backups. ```bash # Backup the default database python manage.py dbbackup # Backup a specific database python manage.py dbbackup --database analytics # Backup and clean old backups beyond retention count python manage.py dbbackup --clean ``` -------------------------------- ### Sync Media Files with mediabackup Command Source: https://context7.com/kjnez/django-rclone/llms.txt This bash command uses the `mediabackup` management command to synchronize Django's `MEDIA_ROOT` directory with the configured rclone remote. It performs an incremental sync, transferring only new or modified files, ensuring efficient backups of media assets. ```bash # Sync media files to remote python manage.py mediabackup # Output: # Syncing media files to myremote:backups/media/ # Media backup completed ``` -------------------------------- ### Django Management Commands for Backup and Restore Source: https://github.com/kjnez/django-rclone/blob/master/docs/index.md Utilize the provided Django management commands for database backup and restore, media file synchronization, and listing available backups. These commands leverage rclone for efficient data transfer. ```bash python manage.py dbbackup # Backup database python manage.py dbrestore # Restore latest backup python manage.py mediabackup # Sync media files to remote python manage.py mediarestore # Sync media files from remote python manage.py listbackups # List all backups ``` -------------------------------- ### Overriding Database Connector Classes (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Shows how to use the `CONNECTORS` setting in Python to specify custom connector classes for particular database aliases. This provides fine-grained control over how specific databases are handled. ```python "CONNECTORS": { "default": "myapp.connectors.CustomPgConnector", "analytics": "myapp.connectors.CustomSqliteConnector", } ``` -------------------------------- ### Mapping Database Engines to Connector Classes (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Demonstrates configuring `CONNECTOR_MAPPING` in Python to override the default mapping between database engines and their corresponding connector classes. This allows using custom connectors for specific database backends. ```python "CONNECTOR_MAPPING": { "django.db.backends.postgresql": "myapp.connectors.CustomPgConnector", } ``` -------------------------------- ### Configure rclone for compressed storage Source: https://github.com/kjnez/django-rclone/blob/master/docs/migration-from-dbbackup.md This bash command shows how to create an rclone remote that automatically handles compression. django-rclone can leverage this by specifying the compressed remote in its settings, eliminating the need for django-dbbackup's `--compress` flag. ```bash rclone config create myremote-compress compress remote=myremote:backups ``` -------------------------------- ### Update INSTALLED_APPS for django-rclone Source: https://github.com/kjnez/django-rclone/blob/master/docs/migration-from-dbbackup.md This snippet shows the necessary change in the `INSTALLED_APPS` list within Django settings to replace `dbbackup` with `django_rclone`. ```diff INSTALLED_APPS = [ - "dbbackup", + "django_rclone", ] ``` -------------------------------- ### Database Restore Management Commands Source: https://github.com/kjnez/django-rclone/blob/master/README.md Provides bash commands for restoring databases using django-rclone. Supports restoring from the latest backup or a specific backup file, and non-interactive restore for automation. ```bash # Restore from the latest backup python manage.py dbrestore # Restore a specific backup python manage.py dbrestore --input-path default-2024-01-15-120000.dump # Non-interactive restore (for automation) python manage.py dbrestore --noinput --input-path default-2024-01-15-120000.dump ``` -------------------------------- ### Writing Custom Connectors Source: https://context7.com/kjnez/django-rclone/llms.txt Users can create custom database connectors by subclassing `BaseConnector` and implementing the necessary interface. These custom connectors can then be registered using the `CONNECTORS` or `CONNECTOR_MAPPING` settings. ```APIDOC ## Writing Custom Connectors ### Description Create custom connectors by subclassing `BaseConnector` and implementing the required interface. Register via `CONNECTORS` or `CONNECTOR_MAPPING` settings. ### Example: Oracle Connector ```python # myapp/connectors.py import os import subprocess from django_rclone.db.base import BaseConnector class OracleConnector(BaseConnector): """Custom connector for Oracle databases using expdp/impdp.""" @property def extension(self) -> str: return "dmp" def _env(self) -> dict[str, str]: """Set Oracle-specific environment variables.""" env = os.environ.copy() env["ORACLE_SID"] = self.name return env def dump(self) -> subprocess.Popen[bytes]: """Export Oracle database using Data Pump.""" cmd = [ "expdp", f"{self.user}/{self.password}@{self.host}:{self.port}/{self.name}", "directory=BACKUP_DIR", "dumpfile=export.dmp", ] return subprocess.Popen( cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=self._env() ) def restore(self, stdin=None) -> subprocess.Popen[bytes]: """Import Oracle database using Data Pump.""" cmd = [ "impdp", f"{self.user}/{self.password}@{self.host}:{self.port}/{self.name}", "directory=BACKUP_DIR", "dumpfile=export.dmp", ] return subprocess.Popen( cmd, stdin=stdin, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=self._env() ) # settings.py - Register the custom connector DJANGO_RCLONE = { "REMOTE": "myremote:backups", "CONNECTOR_MAPPING": { "django.db.backends.oracle": "myapp.connectors.OracleConnector", }, # Or override for specific database alias "CONNECTORS": { "oracle_db": "myapp.connectors.OracleConnector", }, } ``` ``` -------------------------------- ### List Backups with listbackups Command Source: https://context7.com/kjnez/django-rclone/llms.txt This bash command utilizes the `listbackups` management command to display backups stored on the rclone remote. It leverages rclone's `lsjson` for structured output and can filter backups by database or list media files. The output format varies depending on whether database or media files are listed. ```bash # List all database backups python manage.py listbackups # Output: # Name Size Modified # --------------------------------------------------------------------------------------- # default-2024-01-15-120000.dump 1.2 MB 2024-01-15T12:00:00Z # default-2024-01-14-120000.dump 1.1 MB 2024-01-14T12:00:00Z # Filter by database python manage.py listbackups --database analytics # List media files on remote python manage.py listbackups --media # Output: # Media files: 42, Total size: 156.3 MB # # Path Size # ------------------------------------------------------------------------ # uploads/photos/image001.jpg 2.4 MB # uploads/photos/image002.jpg 1.8 MB ``` -------------------------------- ### Configure rclone for encrypted storage Source: https://github.com/kjnez/django-rclone/blob/master/docs/migration-from-dbbackup.md This bash command demonstrates how to set up an encrypted remote using rclone's `crypt` backend, which is used by django-rclone for transparent encryption. It wraps an existing remote (e.g., 'myremote:backups') with encryption. ```bash # Create an encrypted remote wrapping your storage remote rclone config create myremote-crypt crypt \ remote=myremote:backups \ password=$(rclone obscure your-password) ``` -------------------------------- ### Database Connector API Source: https://context7.com/kjnez/django-rclone/llms.txt Database connectors facilitate database backups and restores by wrapping native dump and restore tools as streaming subprocesses. Built-in connectors support PostgreSQL, MySQL, SQLite, and MongoDB. ```APIDOC ## Database Connector API ### Description Database connectors wrap native dump/restore tools as streaming subprocesses. Built-in connectors support PostgreSQL, MySQL, SQLite, and MongoDB. ### Usage ```python from django_rclone.db.base import BaseConnector from django_rclone.db.postgresql import PgDumpConnector from django_rclone.db.registry import get_connector from django.conf import settings import subprocess # Get connector for a database alias (uses registry lookup) connector = get_connector("default") # Create connector directly with database settings db_settings = settings.DATABASES["default"] connector = PgDumpConnector(db_settings) # Dump database - returns Popen with stdout pipe dump_proc = connector.dump() backup_data = dump_proc.stdout.read() dump_proc.wait() if dump_proc.returncode != 0: error = dump_proc.stderr.read().decode() raise Exception(f"Dump failed: {error}") # Restore database - accepts stdin pipe restore_proc = connector.restore(stdin=subprocess.PIPE) restore_proc.communicate(input=backup_data) if restore_proc.returncode != 0: error = restore_proc.stderr.read().decode() raise Exception(f"Restore failed: {error}") # Get file extension for backups ext = connector.extension # "dump" for PostgreSQL, "sql" for MySQL, "sqlite3" for SQLite ``` ``` -------------------------------- ### Configure rclone Crypt Remote Source: https://github.com/kjnez/django-rclone/blob/master/README.md This bash snippet demonstrates how to set up an encrypted remote using rclone's crypt backend. After creating the crypt remote, you can reference it in your Django settings. ```bash rclone config create myremote-crypt crypt remote=myremote:backups password=your-password ``` -------------------------------- ### PostGIS Database Configuration in Django (Python) Source: https://context7.com/kjnez/django-rclone/llms.txt This Python code configures the `DATABASES` setting in Django's `settings.py` to use the PostGIS connector. It includes essential database credentials and specifies an `ADMIN_USER` with privileges to create extensions, ensuring PostGIS is enabled before restore. ```python # settings.py DATABASES = { "default": { "ENGINE": "django.contrib.gis.db.backends.postgis", "NAME": "geodb", "USER": "app_user", "PASSWORD": "app_password", "HOST": "localhost", "PORT": "5432", # Admin user with CREATE EXTENSION privileges "ADMIN_USER": "postgres", } } DJANGO_RCLONE = { "REMOTE": "s3:geo-backups", } ``` -------------------------------- ### Configure Django-Rclone Settings Source: https://github.com/kjnez/django-rclone/blob/master/README.md This snippet shows the structure for configuring django-rclone within your Django settings. It includes required and optional parameters for remote storage, binary paths, backup directories, filename templates, and database connector overrides. ```python DJANGO_RCLONE = { "REMOTE": "myremote:backups", "RCLONE_BINARY": "rclone", "RCLONE_CONFIG": None, "RCLONE_FLAGS": [], "DB_BACKUP_DIR": "db", "DB_FILENAME_TEMPLATE": "{database}-{datetime}.{ext}", "DB_DATE_FORMAT": "%Y-%m-%d-%H%M%S", "DB_CLEANUP_KEEP": 10, "MEDIA_BACKUP_DIR": "media", "CONNECTORS": {}, "CONNECTOR_MAPPING": {}, } ``` -------------------------------- ### Registering Custom Oracle Connector in Django Settings Source: https://github.com/kjnez/django-rclone/blob/master/docs/connectors.md Shows how to register a custom `OracleConnector` within Django's `settings.py`. This involves defining the `DJANGO_RCLONE` setting and using the `CONNECTOR_MAPPING` key to associate the Oracle database engine with the custom connector class path. This configuration is essential for django-rclone to utilize the custom connector. ```python # settings.py DJANGO_RCLONE = { "REMOTE": "myremote:backups", "CONNECTOR_MAPPING": { "django.db.backends.oracle": "myapp.connectors.OracleConnector", }, } ``` -------------------------------- ### Log Database Backup and Restore Events (Python) Source: https://github.com/kjnez/django-rclone/blob/master/docs/signals.md This snippet demonstrates how to use Django signals to log database backup and restore events. It imports the necessary signals from `django_rclone.signals` and uses the `@receiver` decorator to connect custom logging functions. ```python import logging from django.dispatch import receiver from django_rclone.signals import post_db_backup, post_db_restore logger = logging.getLogger("backups") @receiver(post_db_backup) def log_backup(sender, database, path, **kwargs): logger.info("Database '%s' backed up to %s", database, path) @receiver(post_db_restore) def log_restore(sender, database, **kwargs): logger.info("Database '%s' restored", database) ``` -------------------------------- ### Configure Compressed Rclone Remote Source: https://github.com/kjnez/django-rclone/blob/master/docs/configuration.md Creates a compress remote that wraps an existing storage remote. Data is compressed before storage and decompressed on read. This can be combined with encryption. ```bash rclone config create myremote-compressed compress \ remote=myremote:backups ``` -------------------------------- ### Configure INSTALLED_APPS for django-rclone Source: https://github.com/kjnez/django-rclone/blob/master/README.md Adds 'django_rclone' to the INSTALLED_APPS setting in Django's settings.py. This enables the django-rclone app and its associated management commands. ```python INSTALLED_APPS = [ # ... "django_rclone", ] ```