### Start MySQL and PostgreSQL databases with docker-compose

Source: https://github.com/erezsh/reladiff/blob/master/CONTRIBUTING.md

Starts the MySQL and PostgreSQL databases in the background using docker-compose. Requires docker-compose to be installed.

```shell-session
$ docker-compose up -d mysql postgres # run mysql and postgres dbs in background
```

--------------------------------

### Install dependencies for development

Source: https://github.com/erezsh/reladiff/blob/master/CONTRIBUTING.md

Install system dependencies and Python packages for development. For MacOS, use brew to install MySQL and PostgreSQL C bindings; for Debian, use apt-get. Then install poetry and project dependencies.

```bash
$ brew install mysql postgresql # MacOS dependencies for C bindings
$ apt-get install libpq-dev libmysqlclient-dev # Debian dependencies
$ pip install poetry # Python dependency isolation tool
$ poetry install # Install dependencies
```

--------------------------------

### Install each database driver individually

Source: https://github.com/erezsh/reladiff/blob/master/docs/install.md

Alternative method to install all drivers by running separate pip commands. You can omit drivers you don't plan to use.

```bash
pip install reladiff[duckdb]
```

```bash
pip install reladiff[mysql]
```

```bash
pip install reladiff[postgresql]
```

```bash
pip install reladiff[snowflake]
```

```bash
pip install reladiff[presto]
```

```bash
pip install reladiff[oracle]
```

```bash
pip install reladiff[trino]
```

```bash
pip install reladiff[clickhouse]
```

```bash
pip install reladiff[vertica]
```

--------------------------------

### Install Reladiff with all database drivers

Source: https://github.com/erezsh/reladiff/blob/master/docs/install.md

Installs Reladiff along with all supported database drivers using the 'all' extra. The second command shows the equivalent explicit list of drivers. It is recommended to install within a virtual environment due to many dependencies.

```sh
# Install all database drivers
pip install reladiff[all]

# The above line is equivalent to:
pip install reladiff[duckdb,mysql,postgresql,snowflake,presto,oracle,trino,clickhouse,vertica]
```

--------------------------------

### Install Reladiff with pip

Source: https://github.com/erezsh/reladiff/blob/master/docs/install.md

Installs the Reladiff library and CLI without any database drivers. Requires Python 3.8+ with pip.

```sh
pip install reladiff
```

--------------------------------

### Install Reladiff with specific drivers (Postgresql and DuckDB)

Source: https://github.com/erezsh/reladiff/blob/master/docs/install.md

Installs Reladiff with only the DuckDB and PostgreSQL drivers, as an example of removing drivers you don't plan to use.

```sh
pip install reladiff[duckdb,postgresql]
```

--------------------------------

### Example TOML configuration file

Source: https://github.com/erezsh/reladiff/blob/master/docs/how-to-use.md

Defines a test PostgreSQL database connection and run parameters. The run 'test_diff' overrides verbose to false and specifies source tables. Use with 'reladiff --conf myconfig.toml --run test_diff'.

```toml
# Specify the connection params to the test database.
[database.test_postgresql]
driver = "postgresql"
user = "postgres"
password = "Password1"

# Specify the default run params
[run.default]
update_column = "timestamp"
verbose = true

# Specify params for a run 'test_diff'.
[run.test_diff]
verbose = false
# Source 1 ("left")
1.database = "test_postgresql"                      # Use options from database.test_postgresql
1.table = "rating"
# Source 2 ("right")
2.database = "postgresql://postgres:Password1@/"    # Use URI like in the CLI
2.table = "rating_del1"
```

--------------------------------

### Run benchmarks with custom sample size

Source: https://github.com/erezsh/reladiff/blob/master/CONTRIBUTING.md

Runs the benchmark script with a custom number of rows by setting the N_SAMPLES environment variable. The example uses 100 million rows, which is the canonical target.

```shell-session
$ N_SAMPLES=100000000 dev/benchmark.sh #  100m which is our canonical target
```

--------------------------------

### Diff Events Table Between Postgres and Snowflake

Source: https://github.com/erezsh/reladiff/blob/master/README.md

A real-world example showing how to compare an events table across different database systems with specific keys, columns, and filters.

```bash
reladiff \
  postgresql:/// \
  events \
  "snowflake://<username>:<password>@<host>/<DATABASE>/<SCHEMA>?warehouse=<WAREHOUSE>&role=<ROLE>" \
  events \
  -k event_id \	# Identifier of event
  -c event_data \	# Extra column to compare
  -w "event_time < '2024-10-10'"    # Filter the rows on both dbs
```

--------------------------------

### Adding dependencies to pyproject.toml

Source: https://github.com/erezsh/reladiff/blob/master/docs/new-database-driver-guide.rst

Specify 3rd party database libraries in the [tool.poetry.extras] section to allow users to install only the drivers they need.

```toml
[tool.poetry.extras]
postgresql = ["psycopg2"]
```

--------------------------------

### Row Difference Output

Source: https://github.com/erezsh/reladiff/blob/master/docs/technical-explanation.md

Example output showing the identifier and timestamp for rows that were found to be different.

```text
(122001, 1653672821)
```

--------------------------------

### Python: Diff two tables with connect_to_table and diff_tables

Source: https://github.com/erezsh/reladiff/blob/master/docs/how-to-use.md

Connects to two tables (PostgreSQL and MySQL) and iterates over the differences, printing each sign and row. Requires the reladiff module and optional logging setup.

```python
# Optional: Set logging to display the progress of the diff
import logging
logging.basicConfig(level=logging.INFO)

from reladiff import connect_to_table, diff_tables

table1 = connect_to_table("postgresql:///", "table_name", "id")
table2 = connect_to_table("mysql:///", "table_name", "id")

sign: Literal['+' | '-']
row: tuple[str, ...]
for sign, row in diff_tables(table1, table2):
    print(sign, row)
```

--------------------------------

### Run benchmarks

Source: https://github.com/erezsh/reladiff/blob/master/CONTRIBUTING.md

Runs the benchmark script and generates graphs from the results. The benchmark script outputs results to benchmark_<sha>.csv, and the graph script creates graphs from benchmark_*.csv files.

```shell-session
$ dev/benchmark.sh #  runs benchmarks and puts results in benchmark_<sha>.csv
$ poetry run python3 dev/graph.py #  create graphs from benchmark_*.csv files
```

--------------------------------

### connect

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Establishes a connection to a database using a connection string or a database object.

```APIDOC
## connect

### Description
Connects to a database using a connection string or a database object.

### Parameters
- **conn** (str or Database) - Required - Connection string or database object.

### Returns
- **Database** - A database connection object.

```

--------------------------------

### Download seeding data CSVs

Source: https://github.com/erezsh/reladiff/blob/master/CONTRIBUTING.md

Downloads the 1 million row CSV for seeding the database. A larger 25 million row dataset is commented out because it takes 25x longer to import.

```shell-session
$ curl https://datafold-public.s3.us-west-2.amazonaws.com/1m.csv -o dev/ratings.csv
# For a larger data-set (but takes 25x longer to import):
# - curl https://datafold-public.s3.us-west-2.amazonaws.com/25m.csv -o dev/ratings.csv
```

--------------------------------

### Seed databases with preql

Source: https://github.com/erezsh/reladiff/blob/master/CONTRIBUTING.md

Inserts the downloaded CSV data into the testing databases using preql. It is optional to seed more than one database to run reladiff against. Includes commands for MySQL, PostgreSQL, and cloud databases (Snowflake, MSSQL, BigQuery).

```shell-session
# It's optional to seed more than one to run reladiff(1) against.
$ poetry run preql -f dev/prepare_db.pql mysql://mysql:Password1@127.0.0.1:3306/mysql
$ poetry run preql -f dev/prepare_db.pql postgresql://postgres:Password1@127.0.0.1:5432/postgres
# Cloud databases
$ poetry run preql -f dev/prepare_db.pql snowflake://<uri>
$ poetry run preql -f dev/prepare_db.pql mssql://<uri>
$ poetry run preql -f dev/prepare_db.pql bigquery:///<project>
```

--------------------------------

### Implementing _query() for cursor connections

Source: https://github.com/erezsh/reladiff/blob/master/docs/new-database-driver-guide.rst

For standard cursor-based connections, implement _query by calling base._query_conn.

```python
return _query_conn(self._conn, sql_code)
```

--------------------------------

### Configure DATABASE_TYPES in tests/test_database_types.py

Source: https://github.com/erezsh/reladiff/blob/master/docs/new-database-driver-guide.rst

Register your new database class and its supported type categories in the DATABASE_TYPES dictionary to enable testing.

```python
DATABASE_TYPES = {
    ...
    db.PostgreSQL: {
        "int": [ "int",  "bigint" ],
        "datetime": [
            "timestamp(6) without time zone",
            "timestamp(3) without time zone",
            "timestamp(0) without time zone",
            "timestamp with time zone",
        ],
        ...
    },
```

--------------------------------

### Run Cross-DB Diff via CLI

Source: https://github.com/erezsh/reladiff/blob/master/README.md

Perform a cross-database diff using hashes between two different database URIs.

```bash
reladiff  DB1_URI  TABLE1_NAME  DB2_URI  TABLE2_NAME  [OPTIONS]
```

--------------------------------

### Run Same-DB Diff via CLI

Source: https://github.com/erezsh/reladiff/blob/master/README.md

Perform a same-database diff using an outer join when both tables reside in the same database.

```bash
reladiff  DB1_URI  TABLE1_NAME  TABLE2_NAME  [OPTIONS]
```

--------------------------------

### Run unit tests with unittest-parallel

Source: https://github.com/erezsh/reladiff/blob/master/CONTRIBUTING.md

Run all unit tests in parallel using unittest-parallel, or run an individual test with the -k flag. There are over 1000 tests for various type and database combinations.

```shell-session
$ poetry run unittest-parallel -j 16 #  run all tests
$ poetry run python -m unittest -k <test> #  run individual test
```

--------------------------------

### Cross-DB diff using hashes

Source: https://github.com/erezsh/reladiff/blob/master/docs/how-to-use.md

Basic command-line syntax for comparing two tables in different databases. Uses hashes and accepts optional flags. Surround database URLs with quotes because they may contain special characters.

```bash
# Cross-DB diff, using hashes
reladiff  DB1_URI  TABLE1_NAME  DB2_URI  TABLE2_NAME  [OPTIONS]
```

--------------------------------

### Quoting for shell and powershell

Source: https://github.com/erezsh/reladiff/blob/master/docs/install.md

Shows how to quote the 'all' extra in bash and powershell to allow the [] syntax. Use single quotes for bash and double quotes for powershell.

```sh
pip install 'reladiff[all]'     # will work on bash
```

```sh
pip install "reladiff[all]"     # will work on powershell (Windows)
```

--------------------------------

### AbstractDatabase

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Abstract base class for database connections.

```APIDOC
## AbstractDatabase

### Description
Abstract base class for database connections.

### Methods
- (All methods are abstract and must be implemented by subclasses.)

```

--------------------------------

### Same-DB diff using outer join

Source: https://github.com/erezsh/reladiff/blob/master/docs/how-to-use.md

Shorter command-line syntax when both tables belong to the same database. Uses an outer join instead of hashes.

```bash
# Same-DB diff, using outer join
reladiff  DB_URI  TABLE1_NAME  TABLE2_NAME  [OPTIONS]
```

--------------------------------

### Use Reladiff in Python

Source: https://github.com/erezsh/reladiff/blob/master/README.md

Import and use the library to programmatically diff tables by connecting to them and iterating through the results.

```python
from reladiff import connect_to_table, diff_tables

table1 = connect_to_table("postgresql:///", "table_name", "id")
table2 = connect_to_table("mysql:///", "table_name", "id")

sign: Literal['+' | '-']
row: tuple[str, ...]
for sign, row in diff_tables(table1, table2):
    print(sign, row)
```

--------------------------------

### Diff Events Tables in Same Postgres DB with Materialization

Source: https://github.com/erezsh/reladiff/blob/master/README.md

Compare two tables in the same database and materialize the results into a new table with a timestamped name.

```bash
reladiff \
  postgresql:///  events  old_events \
  -k org_id \
  -c created_at -c is_internal \
  -w "org_id != 1 and org_id < 2000" \
  -m test_results_%t \
  --materialize-all-rows \
  --table-write-limit 10000
```

--------------------------------

### Signature of _query()

Source: https://github.com/erezsh/reladiff/blob/master/docs/new-database-driver-guide.rst

The _query method is the central point for all database queries, accepting SQL code and returning a list of rows.

```python
def _query(self, sql_code: str) -> list: ...
```

--------------------------------

### AbstractDialect

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Abstract base class for database dialects.

```APIDOC
## AbstractDialect

### Description
Abstract base class for database dialects.

### Methods
- (All methods are abstract and must be implemented by subclasses.)

```

--------------------------------

### Importing 3rd party libraries on demand

Source: https://github.com/erezsh/reladiff/blob/master/docs/new-database-driver-guide.rst

Avoid module-level imports of 3rd party libraries by using the @import_helper decorator to initialize them within a function.

```python
from .base import import_helper

@import_helper("postgresql")
def import_postgresql():
    import psycopg2
    import psycopg2.extras

    psycopg2.extensions.set_wait_callback(psycopg2.extras.wait_select)
    return psycopg2
```

--------------------------------

### connect_to_table

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Creates a TableSegment for a specific table, allowing operations like diffing and sampling.

```APIDOC
## connect_to_table

### Description
Connects to a specific table in a database.

### Parameters
- **conn** (str or Database) - Required - Connection string or database object.
- **table_name** (str) - Required - Name of the table to connect to.
- **schema** (str, optional) - Optional - Schema name.
- **key** (str, optional) - Optional - Primary key column.
- **extra_columns** (list, optional) - Optional - Additional columns to include.

### Returns
- **TableSegment** - A table segment object.

```

--------------------------------

### Checksum Mismatch Visualization

Source: https://github.com/erezsh/reladiff/blob/master/docs/technical-explanation.md

A visual representation of a checksum mismatch between PostgreSQL and Snowflake for a specific data segment.

```text
┌──────────────────────┐              ┌──────────────────────┐
│     PostgreSQL       │              │      Snowflake       │
├──────────────────────┤              ├──────────────────────┤
│    checksum=0102     │              │    checksum=0102     │
├──────────────────────┤   mismatch!  ├──────────────────────┤
│    checksum=ffff     ◀──────────────▶    checksum=aaab    │
├──────────────────────┤              ├──────────────────────┤
│    checksum=abab     │              │    checksum=abab     │
├──────────────────────┤              ├──────────────────────┤
│    checksum=f0f0     │              │    checksum=f0f0     │
├──────────────────────┤              ├──────────────────────┤
│         ...          │              │         ...          │
├──────────────────────┤              ├──────────────────────┤
│    checksum=9494     │              │    checksum=9494     │
└──────────────────────┘              └──────────────────────┘
```

--------------------------------

### DbPath

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Type alias for a database path.

```APIDOC
## DbPath

### Description
Type alias for a database path.

```

--------------------------------

### Bisection Segment Visualization

Source: https://github.com/erezsh/reladiff/blob/master/docs/technical-explanation.md

A visual representation of how Reladiff splits a mismatched segment into smaller sub-segments for row-level comparison.

```text
┌──────────────────────┐              ┌──────────────────────┐
│     PostgreSQL       │              │      Snowflake       │
├──────────────────────┤              ├──────────────────────┤
│    id=100k..110k     │              │    id=100k..110k     │
├──────────────────────┤              ├──────────────────────┤
│    id=110k..120k     │              │    id=110k..120k     │
├──────────────────────┤              ├──────────────────────┤
│    id=120k..130k     │              │    id=120k..130k     │
├──────────────────────┤              ├──────────────────────┤
│    id=130k..140k     │              │    id=130k..140k     │
├──────────────────────┤              ├──────────────────────┤
│         ...          │              │         ...          │
├──────────────────────┤              ├──────────────────────┤
│      190k..200k      │              │      190k..200k      │
└──────────────────────┘              └──────────────────────┘
```

--------------------------------

### HashDiffer

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

A differ that uses hashing to compare tables.

```APIDOC
## HashDiffer

### Description
A differ that uses hashing to compare tables.

### Constructor
- **__init__(self, algorithm='auto', ...)** - Initializes a HashDiffer with the given algorithm.

### Methods
- **diff_tables(self, table1, table2, ...)** - Compares two tables and returns a DiffResultWrapper.

```

--------------------------------

### diff_tables

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Compares two tables and returns a DiffResultWrapper containing the differences.

```APIDOC
## diff_tables

### Description
Compares two tables and returns a DiffResultWrapper containing the differences.

### Parameters
- **table1** (TableSegment) - Required - First table segment.
- **table2** (TableSegment) - Required - Second table segment.
- **on_error** (str, optional) - Optional - Error handling strategy.
- **extra_columns** (list, optional) - Optional - Additional columns to compare.
- **algorithm** (Algorithm, optional) - Optional - Algorithm to use for diffing.

### Returns
- **DiffResultWrapper** - A wrapper around the diff results.

```

--------------------------------

### DbKey

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Type alias for a database key.

```APIDOC
## DbKey

### Description
Type alias for a database key.

```

--------------------------------

### Algorithm

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Enum representing the diff algorithm to use.

```APIDOC
## Algorithm

### Description
Enum representing the diff algorithm to use.

### Values
- **auto** - Automatically choose the best algorithm.
- **hash** - Use hash-based diffing.
- **join** - Use join-based diffing.

```

--------------------------------

### Run Reladiff against seeded database

Source: https://github.com/erezsh/reladiff/blob/master/CONTRIBUTING.md

Runs Reladiff to compare two tables in a PostgreSQL database. The command uses the postgresql:// connection string and specifies the tables 'rating' and 'rating_del1' with the --verbose flag.

```bash
poetry run python3 -m reladiff postgresql://postgres:Password1@localhost/postgres rating postgresql://postgres:Password1@localhost/postgres rating_del1 --verbose
```

--------------------------------

### JoinDiffer

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

A differ that uses joins to compare tables.

```APIDOC
## JoinDiffer

### Description
A differ that uses joins to compare tables.

### Constructor
- **__init__(self, ...)** - Initializes a JoinDiffer.

### Methods
- **diff_tables(self, table1, table2, ...)** - Compares two tables and returns a DiffResultWrapper.

```

--------------------------------

### DbTime

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Type alias for a database time value.

```APIDOC
## DbTime

### Description
Type alias for a database time value.

```

--------------------------------

### Checksum query for a segment

Source: https://github.com/erezsh/reladiff/blob/master/docs/technical-explanation.md

Checksums a segment of rows by hashing the key and update columns, returning a count and a sum of hash values. Used by Reladiff to compare segments across databases; the WHERE clause bounds the segment by key range.

```sql
SELECT count(*),
    sum(cast(conv(substring(md5(concat(cast(id as char), cast(updated_at as char))), 18), 16, 10) as unsigned))
FROM `rating_del1`
WHERE (id >= 1) AND (id < 100000)
```

--------------------------------

### DiffResultWrapper

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Wraps the result of a diff operation, providing iteration and statistics.

```APIDOC
## DiffResultWrapper

### Description
Wraps the result of a diff operation, providing iteration and statistics.

### Methods
- **__iter__(self)** - Iterates over the diff results.
- **close(self)** - Closes the result set.
- **get_stats_dict(self)** - Returns statistics as a dictionary.
- **get_stats_string(self)** - Returns statistics as a string.

```

--------------------------------

### TableSegment

Source: https://github.com/erezsh/reladiff/blob/master/docs/python-api.rst

Represents a segment of a table, used for diffing and sampling.

```APIDOC
## TableSegment

### Description
Represents a segment of a table, used for diffing and sampling.

### Constructor
- **__init__(self, database, table_name, schema=None, key=None, extra_columns=None, ...)** - Initializes a TableSegment.

### Methods
- **get_values(self, columns, ...)** - Retrieves values from the segment.
- **choose_checkpoints(self, ...)** - Chooses checkpoints for segmentation.
- **segment_by_checkpoints(self, ...)** - Segments the table by checkpoints.
- **count(self, ...)** - Counts rows in the segment.
- **count_and_checksum(self, ...)** - Counts rows and computes a checksum.
- **is_bounded(self, ...)** - Checks if the segment is bounded.
- **new(self, ...)** - Creates a new segment.
- **with_schema(self, ...)** - Returns a segment with a different schema.

```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.