SQL Splitter

https://github.com/helgesverre/sql-splitter
Admin
High-performance CLI tool for splitting large SQL dump files into individual table files, offering...
Tokens:179,866
Snippets:2,417
Trust Score:9.8
License:MIT
Update:2 months ago
Show doc for...
Tokens:
Raw
### Install from source

Source: https://github.com/helgesverre/sql-splitter/blob/main/README.md

Clone the repository and use just to install the binary and documentation.

```bash
git clone https://github.com/helgesverre/sql-splitter
cd sql-splitter
just install  # Installs binary + shell completions + man pages
```

--------------------------------

### Install Fish Completions

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/completions.mdx

Installation steps for fish shell completions.

```bash
# Create directory if needed
mkdir -p ~/.config/fish/completions

# Generate completions (takes effect immediately in new shells)
sql-splitter completions fish > ~/.config/fish/completions/sql-splitter.fish
```

--------------------------------

### System-wide Completion Installation

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/completions.mdx

Install completions for all users using sudo.

```bash
sudo sql-splitter completions bash > /etc/bash_completion.d/sql-splitter
```

--------------------------------

### Install man pages manually

Source: https://github.com/helgesverre/sql-splitter/blob/main/README.md

Install documentation pages manually after cloning the source.

```bash
git clone https://github.com/helgesverre/sql-splitter
cd sql-splitter
just install-man
```

--------------------------------

### Install via PowerShell

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/getting-started/installation.mdx

Installs the tool on Windows using PowerShell.

```powershell
powershell -ExecutionPolicy Bypass -c "irm https://sql-splitter.dev/install.ps1 | iex"
```

--------------------------------

### Install and render Graphviz

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/graph.mdx

Instructions for installing Graphviz and rendering diagrams manually.

```bash
# Install Graphviz
brew install graphviz  # macOS
apt install graphviz   # Ubuntu/Debian

# Render DOT to PNG
sql-splitter graph dump.sql -o schema.dot
dot -Tpng schema.dot -o schema.png

# Or use --render (requires Graphviz installed)
sql-splitter graph dump.sql -o schema.png --render
```

--------------------------------

### Install C compiler for build

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/troubleshooting.mdx

Install build-essential or gcc to resolve linker errors during installation.

```bash
# Ubuntu/Debian
sudo apt-get install build-essential

# Fedora/RHEL
sudo dnf install gcc

# Then retry
cargo install sql-splitter
```

--------------------------------

### Install via Shell Script

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/getting-started/installation.mdx

Downloads and installs the pre-built binary for macOS and Linux to the cargo bin directory.

```bash
curl -fsSL https://sql-splitter.dev/install.sh | sh
```

--------------------------------

### Define example schema

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/multi-tenant.mdx

Example SQL schema showing global and tenant-specific tables.

```sql
-- Global tables (no tenant_id)
CREATE TABLE countries (id INT PRIMARY KEY, name VARCHAR(100));
CREATE TABLE currencies (id INT PRIMARY KEY, code VARCHAR(3));

-- Tenant tables
CREATE TABLE tenants (id INT PRIMARY KEY, name VARCHAR(100));
CREATE TABLE users (
  id INT PRIMARY KEY,
  tenant_id INT REFERENCES tenants(id),
  name VARCHAR(100)
);
CREATE TABLE orders (
  id INT PRIMARY KEY,
  tenant_id INT REFERENCES tenants(id),
  user_id INT REFERENCES users(id)
);
```

--------------------------------

### Setup model YAML fuzzing

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/contributing/testing.mdx

Install required dependencies for fuzz testing.

```bash
cargo install cargo-fuzz
rustup toolchain install nightly
```

--------------------------------

### Install PowerShell Completions

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/completions.mdx

Installation steps for PowerShell completions via the profile script.

```powershell
echo $PROFILE
# Typically: ~/.config/powershell/Microsoft.PowerShell_profile.ps1 (Linux/macOS)
# Or: ~/Documents/PowerShell/Microsoft.PowerShell_profile.ps1 (Windows)
```

```powershell
# Create profile if it doesn't exist
if (!(Test-Path -Path $PROFILE)) { New-Item -ItemType File -Path $PROFILE -Force }

# Append completions to profile
sql-splitter completions powershell >> $PROFILE

# Reload profile
. $PROFILE
```

--------------------------------

### Build from Source

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/getting-started/installation.mdx

Clones the repository and installs the binary, shell completions, and man pages using just.

```bash
git clone https://github.com/helgesverre/sql-splitter
cd sql-splitter
just install # Installs binary + shell completions + man pages
```

```bash
cargo install --git https://github.com/helgesverre/sql-splitter
```

--------------------------------

### Install Rust

Source: https://github.com/helgesverre/sql-splitter/blob/main/CONTRIBUTING.md

Command to install the Rust toolchain via rustup.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

--------------------------------

### Installation Command

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/og-image.html

Shell command to install sql-splitter via a script.

```bash
$ curl -fsSL https://sql-splitter.dev/install.sh | sh
```

--------------------------------

### Install Bash Completions

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/completions.mdx

Methods for installing bash completions system-wide or for the current user.

```bash
sudo sql-splitter completions bash > /etc/bash_completion.d/sql-splitter
```

```bash
# Create completions directory if needed
mkdir -p ~/.bash_completion.d

# Generate completions
sql-splitter completions bash > ~/.bash_completion.d/sql-splitter

# Add to .bashrc (one-time setup)
echo 'source ~/.bash_completion.d/sql-splitter' >> ~/.bashrc

# Reload current session
source ~/.bashrc
```

--------------------------------

### Executing Basic Queries

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/query.mdx

Examples of running standard SQL queries against a dump file.

```bash
# Count rows in a table
sql-splitter query dump.sql "SELECT COUNT(*) FROM users"

# Filter and inspect data
sql-splitter query dump.sql "SELECT * FROM orders WHERE total > 100"

# Join across tables
sql-splitter query dump.sql "SELECT u.name, COUNT(o.id) FROM users u JOIN orders o ON u.id = o.user_id GROUP BY u.name"
```

--------------------------------

### Install sql-splitter via Cargo

Source: https://github.com/helgesverre/sql-splitter/blob/main/README.md

Install the binary directly from crates.io.

```bash
cargo install sql-splitter
```

--------------------------------

### Verify Installation

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/getting-started/installation.mdx

Checks the installed version of the tool.

```bash
sql-splitter --version
```

--------------------------------

### PostgreSQL Example

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/reference/dialects.mdx

Example of a PostgreSQL plain text dump using COPY FROM stdin.

```sql
CREATE TABLE users (
id SERIAL PRIMARY KEY,
name VARCHAR(100) NOT NULL,
email VARCHAR(255)
);

COPY users (id, name, email) FROM stdin;
1 Alice alice@example.com
2 Bob bob@example.com
\.
```

--------------------------------

### Verify installation

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/troubleshooting.mdx

Check if the sql-splitter binary is correctly located in your PATH.

```bash
which sql-splitter
# Should output: /Users/you/.cargo/bin/sql-splitter
```

--------------------------------

### Input SQL Dump Example

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBT_INTEGRATION_DEEP_DIVE.md

Example source SQL dump containing table definitions for users and orders.

```sql
-- MySQL dump
CREATE TABLE users (
  id INT AUTO_INCREMENT PRIMARY KEY,
  email VARCHAR(255) NOT NULL UNIQUE,
  status ENUM('active', 'inactive') DEFAULT 'active',
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  COMMENT 'User accounts'
);

CREATE TABLE orders (
  id INT AUTO_INCREMENT PRIMARY KEY,
  user_id INT NOT NULL,
  total DECIMAL(10,2) NOT NULL CHECK (total > 0),
  status VARCHAR(20) CHECK (status IN ('pending', 'completed', 'cancelled')),
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
```

--------------------------------

### Visualize dependencies and sample

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/sample.mdx

Generate a dependency graph and perform sampling starting from specific root tables.

```bash
# Understand the FK graph before deciding on root tables
sql-splitter graph dump.sql --format mermaid -o schema.md
sql-splitter sample dump.sql --percent 10 --preserve-relations --root-tables users,orgs -o dev.sql
```

--------------------------------

### Install via Homebrew

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/getting-started/installation.mdx

Installs the tool using the Homebrew package manager on macOS.

```bash
brew install helgesverre/tap/sql-splitter
```

--------------------------------

### CI/CD Integration Example

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/glob-patterns.mdx

Example of using the tool within a GitHub Actions workflow.

```yaml
# GitHub Actions example
- name: Validate all SQL dumps
  run: |
    sql-splitter validate "migrations/*.sql" --strict --json > validation.json
    if jq -e '.failed > 0' validation.json; then
      echo "Validation failed"
      exit 1
    fi
```

--------------------------------

### Install Agent Skills

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/ai-integration.mdx

Commands to install sql-splitter skills into various development environments.

```bash
git clone https://github.com/helgesverre/sql-splitter.git /tmp/sql-splitter
cp -r /tmp/sql-splitter/skills/sql-splitter ~/.claude/skills/
```

```bash
amp skill add helgesverre/sql-splitter
```

```bash
git clone https://github.com/helgesverre/sql-splitter.git /tmp/sql-splitter
cp -r /tmp/sql-splitter/skills/sql-splitter .github/skills/
```

```bash
git clone https://github.com/helgesverre/sql-splitter.git /tmp/sql-splitter
cp -r /tmp/sql-splitter/skills/sql-splitter .cursor/skills/
```

--------------------------------

### Install Zsh Completions

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/completions.mdx

Various methods for installing zsh completions, including standard fpath, Oh-My-Zsh, and manual fpath configuration.

```bash
echo $fpath | tr ' ' '\n'
```

```bash
# Use the first writable fpath directory (usually ~/.zsh/completions or similar)
sql-splitter completions zsh > "${fpath[1]}/_sql-splitter"

# Rebuild completion cache and reload
autoload -Uz compinit && compinit
source ~/.zshrc
```

```bash
# Oh-My-Zsh includes ~/.oh-my-zsh/completions in fpath
mkdir -p ~/.oh-my-zsh/completions
sql-splitter completions zsh > ~/.oh-my-zsh/completions/_sql-splitter

# Reload
source ~/.zshrc
```

```bash
# Create one and add to fpath in .zshrc
mkdir -p ~/.zsh/completions
echo 'fpath=(~/.zsh/completions $fpath)' >> ~/.zshrc
sql-splitter completions zsh > ~/.zsh/completions/_sql-splitter
source ~/.zshrc
```

--------------------------------

### SQLite Example

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/reference/dialects.mdx

Example of an SQLite dump generated via the .dump command.

```sql
CREATE TABLE users (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
email TEXT
);

INSERT INTO users VALUES (1, 'Alice', 'alice@example.com');
```

--------------------------------

### Convert MySQL to PostgreSQL

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/ai-integration.mdx

Example agent prompt for converting database dialects.

```bash
Agent: I'll convert your MySQL dump to PostgreSQL.

sql-splitter validate mysql_dump.sql --strict
sql-splitter convert mysql_dump.sql --to postgres -o postgres_dump.sql --progress
sql-splitter validate postgres_dump.sql --dialect postgres --strict

Result: Conversion complete. 2 warnings about ENUM types
that were converted to VARCHAR.
```

--------------------------------

### MSSQL Example

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/reference/dialects.mdx

Example of an MSSQL script using square bracket quoting and GO batch separators.

```sql
CREATE TABLE [users] (
[id] INT IDENTITY(1,1) PRIMARY KEY,
[name] NVARCHAR(100) NOT NULL,
[email] NVARCHAR(255)
);
GO

INSERT INTO [users] ([name], [email]) VALUES (N'Alice', N'alice@example.com');
GO
```

--------------------------------

### Install Shell Completions

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/getting-started/installation.mdx

Generates and installs shell completion scripts for Bash, Zsh, and Fish.

```bash
# Bash (user)
sql-splitter completions bash >> ~/.bashrc

# Zsh (oh-my-zsh)
sql-splitter completions zsh > ~/.oh-my-zsh/completions/_sql-splitter

# Fish
sql-splitter completions fish > ~/.config/fish/completions/sql-splitter.fish
```

--------------------------------

### Execute SQL Splitter migration commands

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/enterprise/04-execution-design.md

Examples of CLI commands for generating a migration plan, executing a migration, and resuming a migration from a state file.

```text
sql-splitter migrate \
  --source-config source.toml \
  --target-config target.toml \
  --consistency consistent-snapshot \
  --plan-output plan.json

sql-splitter migrate \
  --plan-input plan.json \
  --source-config source.toml \
  --target-config target.toml \
  --approval-ref CHANGE-1234 \
  --execute \
  --strict-verification \
  --state-output migration.state

sql-splitter migrate \
  --resume migration.state \
  --source-config source.toml \
  --target-config target.toml
```

--------------------------------

### Create SQL test fixture

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/contributing/testing.mdx

Example SQL file for testing new features.

```sql
-- tests/fixtures/static/mysql/my_feature.sql
CREATE TABLE test (
    id INT AUTO_INCREMENT PRIMARY KEY,
    name VARCHAR(100)
);

INSERT INTO test VALUES (1, 'Alice');
INSERT INTO test VALUES (2, 'Bob');
```

--------------------------------

### Verify Completion Installation

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/completions.mdx

Test that the completion script is correctly installed by triggering the command list.

```bash
# Type and press Tab
sql-splitter <TAB>

# Should show all commands:
# analyze  completions  convert  diff  graph  merge  order  query  redact  sample  shard  split  validate
```

--------------------------------

### MySQL Example

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/reference/dialects.mdx

Example of a MySQL dump structure using backtick quoting and ENGINE definitions.

```sql
CREATE TABLE `users` (
`id` INT AUTO_INCREMENT PRIMARY KEY,
`name` VARCHAR(100) NOT NULL,
`email` VARCHAR(255),
INDEX `idx_email` (`email`)
) ENGINE=InnoDB;

INSERT INTO `users` VALUES (1, 'Alice', 'alice@example.com');
```

--------------------------------

### Install Xcode command line tools

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/troubleshooting.mdx

Required for building on Apple Silicon systems.

```bash
xcode-select --install
```

--------------------------------

### Basic Comparison Examples

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/diff.mdx

Common patterns for comparing schema and data between two SQL files.

```bash
# Compare two dumps (schema + data)
sql-splitter diff old.sql new.sql

# Schema-only comparison (faster, no data loading)
sql-splitter diff old.sql new.sql --schema-only

# Data-only comparison (assumes schema is identical)
sql-splitter diff old.sql new.sql --data-only
```

--------------------------------

### Compare SQL dump querying methods

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DUCKDB_INTEGRATION_DEEP_DIVE.md

Demonstrates the inefficiency of manual database setup or manual parsing compared to the integrated DuckDB approach.

```bash
# Option 1: Manual database setup (slow)
createdb tempdb
psql tempdb < dump.sql
psql tempdb -c "SELECT COUNT(*) FROM users"
dropdb tempdb

# Option 2: Parse manually (very slow)
grep "INSERT INTO users" dump.sql | wc -l
```

```bash
# One command, instant results
sql-splitter query dump.sql "SELECT COUNT(*) FROM users"
```

--------------------------------

### Command Interface Examples for sql-splitter migrate

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/MIGRATE_FEATURE.md

Various command-line usage patterns for generating migrations, rollbacks, and handling specific schema scenarios.

```bash
# Generate migration from schema diff
sql-splitter migrate old.sql new.sql -o migration.sql

# Generate with rollback script
sql-splitter migrate old.sql new.sql -o migration.sql --rollback rollback.sql

# Check for breaking changes only
sql-splitter migrate old.sql new.sql --breaking-changes

# Dry run (show migration without writing)
sql-splitter migrate old.sql new.sql --dry-run

# Generate for specific dialect
sql-splitter migrate old.sql new.sql -o migration.sql --dialect postgres

# Generate data migration helpers
sql-splitter migrate old.sql new.sql -o migration.sql --with-data

# Split into multiple migration files
sql-splitter migrate old.sql new.sql -o migrations/ --split

# Generate only for specific tables
sql-splitter migrate old.sql new.sql --tables "users,orders"
```

--------------------------------

### Advanced Glob Pattern Examples

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/glob-patterns.mdx

Additional examples for recursive matching and specific file naming patterns.

```bash
# All .sql files in dumps/
sql-splitter analyze "dumps/*.sql"

# All .sql and .sql.gz files recursively
sql-splitter validate "backups/**/*.sql*"

# Files starting with "prod_"
sql-splitter analyze "prod_*.sql"

# Files from 2024
sql-splitter validate "backup_2024*.sql"
```

--------------------------------

### Launch interactive schema browser

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/INTEGRATION_OPPORTUNITIES.md

Starts a local web server to provide an interactive, browser-based exploration of the SQL schema.

```bash
# Generate interactive schema browser
sql-splitter browse dump.sql

# Starts local web server on :8080
# Interactive schema exploration in browser
# - Click tables to see columns
# - Highlight FK relationships
# - Search across schema
```

--------------------------------

### Type Conversion Bottleneck Examples

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/PARALLEL_ALL_COMMANDS_ANALYSIS.md

Illustrative examples of computational type conversion operations that cause CPU bottlenecks.

```rust
// Each conversion is computational
"VARCHAR(255)" → "TEXT"               // String matching
"AUTO_INCREMENT" → "SERIAL"           // Pattern replacement
"DATETIME" → "TIMESTAMP"              // Type mapping
"COPY ... FROM stdin" → "INSERT ..."  // Complex parsing + rebuilding
```

--------------------------------

### Interactive REPL session

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/analytics-queries.mdx

Example commands and output within the interactive REPL environment.

```text
sql> .tables
users
orders
products

sql> .schema users
CREATE TABLE users (
  id INT PRIMARY KEY,
  name VARCHAR(100),
  email VARCHAR(255)
);

sql> SELECT COUNT(*) FROM users;
┌──────────────┐
│ count_star() │
├──────────────┤
│ 1500         │
└──────────────┘

sql> .sample orders 5
┌────┬─────────┬────────┐
│ id │ user_id │ total  │
├────┼─────────┼────────┤
│ 1  │ 42      │ 99.99  │
│ 2  │ 17      │ 149.50 │
...

sql> .exit
```

--------------------------------

### Sample and query

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/sample.mdx

Create a database sample and execute ad-hoc SQL queries against the resulting file.

```bash
# Create sample, then run ad-hoc queries against it
sql-splitter sample dump.sql --percent 5 --preserve-relations -o sample.sql
sql-splitter query sample.sql "SELECT COUNT(*) FROM users"
```

--------------------------------

### Benchmark Execution Examples

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/contributing/benchmarking.mdx

Common command-line patterns for generating test data, exporting results, and verifying tool compatibility.

```bash
# Generate 500MB test and run 5 iterations
./docker/run-benchmark.sh --generate 500 --runs 5

# Export results to markdown
./docker/run-benchmark.sh --generate 100 --export results.md

# List available tools
./docker/run-benchmark.sh --list

# Test which tools work with a specific file
./docker/run-benchmark.sh --test /data/dump.sql
```

--------------------------------

### Interactive REPL Mode

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DUCKDB_INTEGRATION_DEEP_DIVE.md

Example session demonstrating the interactive DuckDB shell for querying and inspecting SQL dumps.

```bash
$ sql-splitter query prod.sql --interactive

Importing dump.sql... Done (8.2s, 10 tables, 10M rows)
DuckDB ready. Type 'exit' to quit.

duckdb> SELECT COUNT(*) FROM users;
┌───────────┐
│ count(*) │
├───────────┤
│ 10,234    │
└───────────┘

duckdb> .tables
users
orders
products
...

duckdb> .schema users
CREATE TABLE users (
  id INTEGER PRIMARY KEY,
  email VARCHAR NOT NULL,
  created_at TIMESTAMP
);

duckdb> \timing on
duckdb> SELECT country, COUNT(*) FROM users GROUP BY country;
(0.03s)
...

duckdb> .export csv results.csv
duckdb> SELECT * FROM users LIMIT 100;
Exported to results.csv (100 rows)

duckdb> exit
```

--------------------------------

### Basic merge examples

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/merge.mdx

Perform a standard merge or preview the output without writing files.

```bash
# Merge all tables from a directory
sql-splitter merge tables/ -o restored.sql

# Preview what would be merged (no files written)
sql-splitter merge tables/ --dry-run
```

--------------------------------

### Common analytics queries

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/analytics-queries.mdx

Examples for checking table sizes, finding duplicates, and identifying orphan records.

```bash
sql-splitter query dump.sql "
  SELECT 'users' as tbl, COUNT(*) as rows FROM users
  UNION ALL
  SELECT 'orders', COUNT(*) FROM orders
  UNION ALL
  SELECT 'products', COUNT(*) FROM products
"
```

```bash
# Find duplicates
sql-splitter query dump.sql "
  SELECT email, COUNT(*) as cnt
  FROM users
  GROUP BY email
  HAVING cnt > 1
"

# Find orphan records
sql-splitter query dump.sql "
  SELECT o.*
  FROM orders o
  LEFT JOIN users u ON o.user_id = u.id
  WHERE u.id IS NULL
"
```

--------------------------------

### Autocomplete Usage Examples

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/completions.mdx

Demonstrates how tab completion works for commands, flags, and flag values in the terminal.

```bash
$ sql-splitter sp<TAB>
$ sql-splitter split

$ sql-splitter split dump.sql --dia<TAB>
$ sql-splitter split dump.sql --dialect

$ sql-splitter split dump.sql --dialect <TAB>
mysql    postgres    sqlite    mssql
```

--------------------------------

### CI/CD Integration Examples

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/diff.mdx

Automate diff checks in CI pipelines using JSON output and external tools like jq.

```bash
# Fail CI if any schema differences exist (exit code is 0 either way,
# so check the JSON summary instead)
sql-splitter diff production.sql staging.sql --schema-only --format json \
  | jq -e '.summary.tables_added + .summary.tables_removed + .summary.tables_modified == 0'

# Export diff as JSON for further processing
sql-splitter diff old.sql new.sql --format json -o diff.json
```

--------------------------------

### Clone and build the project

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/contributing/development.mdx

Initial steps to download the repository and compile the project using Cargo.

```bash
git clone https://github.com/helgesverre/sql-splitter
cd sql-splitter
cargo build
```

--------------------------------

### Configure Dockerfile with Shell for Piping

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/docker-usage.mdx

Installs the binary from GitHub releases and sets a shell entrypoint to support command piping.

```dockerfile
FROM debian:bookworm-slim
RUN apt-get update && \
    apt-get install -y ca-certificates curl && \
    rm -rf /var/lib/apt/lists/*

# Install from GitHub releases (replace VERSION with actual version)
ARG VERSION=1.13.7
RUN apt-get update && apt-get install -y xz-utils && rm -rf /var/lib/apt/lists/*
RUN curl -L https://github.com/helgesverre/sql-splitter/releases/download/v${VERSION}/sql-splitter-x86_64-unknown-linux-gnu.tar.xz | \
    tar -xJ --strip-components=1 -C /usr/local/bin/ \
    sql-splitter-x86_64-unknown-linux-gnu/sql-splitter

# Use shell entrypoint for piping support
ENTRYPOINT ["/bin/sh", "-c"]
```

--------------------------------

### Create Safe Dev Dataset

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/ai-integration.mdx

Example agent prompt for sampling and anonymizing production data.

```bash
Agent: I'll create an anonymized sample for development.

sql-splitter sample prod.sql --percent 10 --preserve-relations -o sampled.sql
sql-splitter redact sampled.sql --hash "*.email" --fake "*.name" --null "*.ssn" -o dev.sql

sql-splitter validate dev.sql --strict

Result: Created dev.sql with 10% of data, all PII anonymized.
```

--------------------------------

### Bootstrap a new dbt project

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBT_INTEGRATION_DEEP_DIVE.md

Initialize a dbt project from an existing production database dump to immediately generate staging models and documentation.

```bash
# Existing production database
pg_dump production > prod_dump.sql

# Generate dbt project
sql-splitter dbt-init prod_dump.sql -o dbt_project/

# Immediate value
cd dbt_project/
dbt run   # All staging models built
dbt test  # 100+ tests pass
dbt docs generate  # Full documentation

# Team can now focus on marts layer (business logic)
```

--------------------------------

### Query output formats

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/analytics-queries.mdx

Examples of querying data and formatting the output as table, JSON, CSV, or JSON Lines.

```bash
sql-splitter query dump.sql "SELECT * FROM users LIMIT 5"
```

```bash
sql-splitter query dump.sql "SELECT * FROM users" -f json
```

```json
[
  { "id": 1, "name": "Alice", "email": "alice@example.com" },
  { "id": 2, "name": "Bob", "email": "bob@example.com" }
]
```

```bash
sql-splitter query dump.sql "SELECT * FROM users" -f csv -o users.csv
```

```bash
sql-splitter query dump.sql "SELECT * FROM users" -f jsonl
```

--------------------------------

### CLI Configuration for Enum Conversion

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/ENUM_CONVERSION.md

Examples of CLI commands for converting between MySQL and PostgreSQL with specific enum handling flags.

```bash
# MySQL → PostgreSQL
sql-splitter convert dump.sql --from mysql --to postgres \
  --enum-naming per-column  # or: dedupe

# PostgreSQL → MySQL
sql-splitter convert dump.sql --from postgres --to mysql \
  --strict  # fail on unknown enum types instead of VARCHAR fallback
```

--------------------------------

### Example multi-schema YAML output

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBT_INTEGRATION_DEEP_DIVE.md

Represents the expected structure of a generated DBT sources configuration file.

```yaml
version: 2

sources:
  - name: production
    database: mydb
    schema: production
    tables:
      - name: users
      - name: orders

  - name: analytics
    database: mydb
    schema: analytics
    tables:
      - name: user_metrics
      - name: order_stats
```

--------------------------------

### Manage configuration and validation

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/redact.mdx

Commands and syntax tips for validating configurations and generating templates.

```bash
--validate
```

```bash
--generate-config
```

--------------------------------

### CLI Options for dbt-init

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBT_INTEGRATION_DEEP_DIVE.md

Configuration flags available for the dbt-init command.

```text
--output, -o          Output directory for dbt project
--name                Project name (default: inferred from dump filename)
--profile             dbt profile name (default: "postgres")
--schema              Source schema name (default: "raw")
--database            Source database name (default: inferred)
--tables              Filter to specific tables (comma-separated)
--with-marts          Generate example marts layer models
--staging-prefix      Prefix for staging models (default: "stg_")
--docs                Generate extended documentation
```

--------------------------------

### Integrate sql-splitter in CI/CD pipelines

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/docker-usage.mdx

Examples for running sql-splitter validation and diffing in GitHub Actions, GitLab CI, and CircleCI.

```yaml
name: Validate SQL Dumps

on:
  push:
    paths:
      - 'migrations/*.sql'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install sql-splitter
        run: cargo install sql-splitter

      - name: Validate migrations
        run: sql-splitter validate "migrations/*.sql" --strict

      - name: Check for breaking changes
        run: |
          sql-splitter diff migrations/prev.sql migrations/current.sql \
            --format json > diff.json
          if jq -e '.schema.tables_removed | length > 0' diff.json; then
            echo "Breaking change: tables removed"
            exit 1
          fi
```

```yaml
validate-sql:
  image: rust:slim
  before_script:
    - cargo install sql-splitter
  script:
    - sql-splitter validate "dumps/*.sql" --strict --json > validation.json
  artifacts:
    paths:
      - validation.json
    when: always
```

```yaml
version: 2.1

jobs:
  validate:
    docker:
      - image: rust:slim
    steps:
      - checkout
      - run:
          name: Install sql-splitter
          command: cargo install sql-splitter
      - run:
          name: Validate SQL dumps
          command: sql-splitter validate "*.sql" --strict
```

--------------------------------

### Define dbt models

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBT_INTEGRATION_DEEP_DIVE.md

Examples of staging and mart model definitions using Jinja templating for materialization and references.

```sql
-- models/staging/stg_users.sql
{{ config(materialized='view') }}

SELECT
    id AS user_id,
    email,
    created_at,
    -- Standardize boolean
    CASE status WHEN 'active' THEN TRUE ELSE FALSE END AS is_active
FROM {{ source('raw', 'users') }}
```

```sql
-- models/marts/fct_orders.sql
{{ config(materialized='table') }}

SELECT
    o.id AS order_id,
    o.user_id,
    u.email AS user_email,
    o.total,
    o.created_at AS order_date
FROM {{ ref('stg_orders') }} AS o
LEFT JOIN {{ ref('stg_users') }} AS u
    ON o.user_id = u.user_id
```

--------------------------------

### Clone the repository

Source: https://github.com/helgesverre/sql-splitter/blob/main/CONTRIBUTING.md

Initial steps to download the project source code and enter the directory.

```bash
git clone https://github.com/helgesverre/sql-splitter.git
cd sql-splitter
```

--------------------------------

### Build and test the project

Source: https://github.com/helgesverre/sql-splitter/blob/main/CONTRIBUTING.md

Commands to compile the project and execute the test suite.

```bash
cargo build --release
cargo nextest run
```

--------------------------------

### Configure Table Randomness

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/superpowers/specs/2026-07-16-synthetic-data-generation-design.md

Example YAML configuration demonstrating how to inherit, override, or disable seeds for specific tables.

```yaml
seed: 42

tables:
  inherits:
    # omitted table seed: inherit 42
  independent:
    seed: 9001
  always_random:
    seed: null
```

--------------------------------

### Format and test documentation navigation

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/superpowers/plans/2026-07-19-synthetic-data-sidebar.md

Uses Prettier to format configuration and documentation files, followed by a build and test cycle.

```bash
cd website
bunx prettier astro.config.mjs src/buildOutput.test.ts src/content/docs/commands/generate/index.mdx src/content/docs/commands/generate/model-reference.mdx --write
bun run build
bun test src/buildOutput.test.ts
```

--------------------------------

### Diagnostic Error Report

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/superpowers/specs/2026-07-16-synthetic-data-generation-design.md

Example of a structured diagnostic error message indicating a column ownership conflict during compilation.

```text
error[GEN-COLUMN-OWNER-CONFLICT] synthetic.yaml:84:7
  tables.orders.columns.total is produced by both:
    - columns.total.generator
    - planners[0] (commerce.order_family)
  help: remove the column generator or remove `total` from the planner mapping
```

--------------------------------

### Interactive REPL Exploration

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/query.mdx

Launch an interactive session to explore the SQL dump file.

```bash
sql-splitter query dump.sql --interactive
```

--------------------------------

### Basic usage of sql-splitter sample

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/sample.mdx

The base command structure for sampling SQL files.

```bash
sql-splitter sample <INPUT> [OPTIONS]
```

--------------------------------

### Configure temporal.interval planner

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/superpowers/specs/2026-07-16-synthetic-data-generation-design.md

Coordinates start, end, and duration columns to ensure temporal consistency, supporting both closed intervals and open-ended states.

```yaml
planners:
  - kind: temporal.interval
    columns:
      start: started_at
      end: ended_at
      duration: duration_seconds
      open: is_running
    start:
      kind: observed_range # observed_range | range | monotonic
      min: 2024-01-01T00:00:00Z
      max: 2026-01-01T00:00:00Z
    duration:
      kind: histogram # histogram | uniform | normal | fixed | observed
      unit: seconds
      min: 30
      max: 43200
    open_probability: 0.07
    end_inclusive: true
    timezone: preserve # preserve | utc | named IANA zone
```

--------------------------------

### Get Auto-Increment Syntax

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBML_SUPPORT.md

Returns the dialect-specific syntax for auto-incrementing columns.

```rust
    fn auto_increment_syntax(&self) -> String {
        match self.target_dialect {
            SqlDialect::MySQL => "AUTO_INCREMENT".to_string(),
            SqlDialect::PostgreSQL => "GENERATED ALWAYS AS IDENTITY".to_string(),
            SqlDialect::SQLite => "AUTOINCREMENT".to_string(), // Only with INTEGER PRIMARY KEY
            SqlDialect::MSSQL => "IDENTITY(1,1)".to_string(),
        }
    }
```

--------------------------------

### Redaction Configuration File

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/dev-data-seeding.mdx

Example YAML configuration for defining redaction strategies.

```yaml
seed: 12345
locale: en

rules:
  - column: "*.email"
    strategy: hash
  - column: "*.name"
    strategy: fake
    generator: name
  - column: "*.ssn"
    strategy: "null"
  - column: "*.phone"
    strategy: fake
    generator: phone
  - column: "*.credit_card"
    strategy: mask
    pattern: "****-****-****-XXXX"

skip_tables:
  - schema_migrations
  - ar_internal_metadata
```

--------------------------------

### Mermaid ER Diagram

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/graph.mdx

Example of the Mermaid ER diagram output format.

```mermaid
erDiagram
    users {
        INT id PK
        VARCHAR name
        VARCHAR email
    }
    orders {
        INT id PK
        INT user_id FK
    }
    orders }|--|| users : user_id
```

--------------------------------

### Build and run a custom sql-splitter image

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/docker-usage.mdx

Clone the repository and build an optimized Docker image using a multi-stage build process.

```bash
# Clone and build
git clone https://github.com/helgesverre/sql-splitter
cd sql-splitter

# Build optimized image
docker build -t sql-splitter -f - . <<'EOF'
FROM rust:slim AS builder
WORKDIR /app
COPY . .
RUN cargo build --release

FROM debian:bookworm-slim
COPY --from=builder /app/target/release/sql-splitter /usr/local/bin/
ENTRYPOINT ["sql-splitter"]
EOF

# Run
docker run --rm -v $(pwd):/data sql-splitter analyze /data/dump.sql
```

--------------------------------

### JSON Output Structure

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/analyze.mdx

Example of the JSON structure returned by the analyze command.

```json
{
  "input_file": "dump.sql",
  "dialect": "mysql",
  "size_mb": 0.15,
  "elapsed_secs": 0.025,
  "summary": {
    "total_tables": 4,
    "total_inserts": 20,
    "total_bytes": 158720
  },
  "tables": [
    {
      "name": "users",
      "inserts": 2,
      "creates": 1,
      "statements": 3,
      "bytes": 12595,
      "size_mb": 0.01
    },
    {
      "name": "orders",
      "inserts": 5,
      "creates": 1,
      "statements": 6,
      "bytes": 46284,
      "size_mb": 0.04
    }
  ]
}
```

--------------------------------

### Pipeline Parallelism Concept

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/PARALLEL_ALL_COMMANDS_ANALYSIS.md

Conceptual example of pipeline parallelism for sampling operations.

```rust
// While sampling orders, start pre-filtering order_items
// But gains are minimal for added complexity
```

--------------------------------

### Sample Development Datasets

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/getting-started/index.mdx

Create smaller datasets from production dumps while maintaining foreign key integrity.

```bash
sql-splitter sample prod.sql -o dev.sql --percent 10 --preserve-relations
```

--------------------------------

### Analyze Command Output

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/glob-patterns.mdx

Example of generating aggregate statistics in JSON format.

```bash
sql-splitter analyze "dumps/*.sql" --json
```

```json
{
  "total_files": 5,
  "succeeded": 5,
  "failed": 0,
  "results": [
    {
      "file": "dumps/users.sql",
      "tables_count": 1,
      "total_inserts": 1000,
      "status": "success"
    },
    {
      "file": "dumps/orders.sql",
      "tables_count": 1,
      "total_inserts": 50000,
      "status": "success"
    }
  ]
}
```

--------------------------------

### Reload Shell Configuration

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/completions.mdx

Commands to reload shell configuration files after installing completions.

```bash
# Bash
source ~/.bashrc

# Zsh
source ~/.zshrc

# Fish (usually automatic, but try)
source ~/.config/fish/config.fish

# PowerShell
. $PROFILE
```

--------------------------------

### Initialize dbt project with sql-splitter

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBT_INTEGRATION_DEEP_DIVE.md

Use the dbt-init command to generate a dbt project structure from a SQL dump file. Various flags allow for customization of the project name, profile, and table filtering.

```bash
# Basic usage
sql-splitter dbt-init dump.sql -o my_dbt_project/

# Customize project name
sql-splitter dbt-init dump.sql \
  --output my_dbt_project/ \
  --name analytics \
  --profile snowflake

# Filter tables (only include specific tables)
sql-splitter dbt-init dump.sql -o dbt/ --tables users,orders,products

# Include marts layer scaffolding
sql-splitter dbt-init dump.sql -o dbt/ --with-marts
```

--------------------------------

### Implement feature test

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/contributing/testing.mdx

Example test function validating a feature using a fixture.

```rust
#[test]
fn test_my_feature() {
    let result = my_function("tests/fixtures/static/mysql/my_feature.sql");
    assert!(result.is_ok());
    assert_eq!(result.unwrap().table_count, 1);
}
```

--------------------------------

### Initialize and Run dbt Project

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBT_INTEGRATION_DEEP_DIVE.md

Commands to generate a dbt project from a SQL dump and execute initial dbt operations.

```bash
# 1. Generate complete dbt project from dump
sql-splitter dbt-init dump.sql -o my_dbt_project/

# Creates:
# - dbt_project.yml
# - models/sources.yml (all tables configured)
# - models/schema.yml (tests auto-generated from constraints)
# - models/staging/*.sql (one model per table)
# - README.md

# 2. Run dbt immediately
cd my_dbt_project/
dbt run   # ✓ Works out of the box!
dbt test  # ✓ 50+ tests auto-generated
dbt docs generate  # ✓ Full documentation

# 3. Iterate and refine
# Developers focus on business logic, not YAML boilerplate
```

--------------------------------

### Generate a sample dump with JSON output

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/reference/json-output.mdx

Executes the sample command to create a subset of the database and returns the result as a JSON object.

```bash
sql-splitter sample dump.sql --percent 10 -o dev.sql --json
```

```json
{
  "input_file": "dump.sql",
  "output_file": "dev.sql",
  "dialect": "mysql",
  "dry_run": false,
  "mode": {
    "type": "percent",
    "value": 10
  },
  "statistics": {
    "tables_sampled": 15,
    "tables_skipped": 0,
    "rows_selected": 10500,
    "rows_total": 100000,
    "sample_rate_percent": 10.5,
    "fk_orphans_rejected": 0
  },
  "tables": [
    {
      "name": "users",
      "classification": "root",
      "rows_selected": 100,
      "rows_total": 1000,
      "sample_rate_percent": 10.0
    }
  ]
}
```

--------------------------------

### Generate Graphviz DOT schema

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/graph.mdx

Example of a DOT file structure for schema visualization.

```text
digraph schema {
  rankdir=LR;
  users [label="users|id INT PK\lname VARCHAR\lemail VARCHAR\l"];
  orders [label="orders|id INT PK\luser_id INT FK\l"];
  orders -> users [label="user_id"];
}
```

--------------------------------

### Execute sampling with a configuration file

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/sample.mdx

Run the sample command using a previously defined YAML configuration file to apply complex sampling rules.

```bash
sql-splitter sample dump.sql -o dev.sql --config sample.yaml --preserve-relations
```

--------------------------------

### Validate SQL Dumps

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/ai-integration.mdx

Example agent prompt for validating multiple SQL files.

```bash
Agent: I'll check your SQL dumps for issues.

sql-splitter validate "dumps/*.sql" --strict --json

Result: Found 3 issues in backup_2024.sql:
- Line 42: Syntax error near 'CREAT'
- Duplicate PK in users table
- FK violation: orders.user_id references non-existent user
```

--------------------------------

### Validate Command Output

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/glob-patterns.mdx

Example of generating validation status for each file in JSON format.

```bash
sql-splitter validate "*.sql" --json
```

```json
{
  "passed": 2,
  "failed": 1,
  "results": [
    {
      "file": "a.sql",
      "passed": true,
      "summary": { "summary": { "errors": 0, "warnings": 0 } }
    },
    {
      "file": "b.sql",
      "passed": true,
      "summary": { "summary": { "errors": 0, "warnings": 1 } }
    },
    {
      "file": "c.sql",
      "passed": false,
      "summary": { "summary": { "errors": 2, "warnings": 0 } }
    }
  ]
}
```

--------------------------------

### Shuffle Strategy SQL Example

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/PARALLEL_ALL_COMMANDS_ANALYSIS.md

Illustrates the effect of shuffling values within a column.

```sql
-- Before
INSERT INTO users (email) VALUES
('alice@example.com'),
('bob@example.com'),
('carol@example.com');

-- After shuffle
('carol@example.com'),  -- Shuffled order
('alice@example.com'),
('bob@example.com');
```

--------------------------------

### Initialize dbt project from SQL dump

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/INTEGRATION_ROADMAP_MASTER.md

Generates a complete dbt project structure including configuration files, models, and tests from a provided SQL dump file.

```bash
# Generate complete dbt project
sql-splitter dbt-init dump.sql -o my_dbt_project/

# Output:
# - dbt_project.yml
# - models/sources.yml (all tables configured)
# - models/staging/*.sql (one model per table)
# - models/staging/_models.yml (50+ tests auto-generated)
# - README.md (getting started guide)

cd my_dbt_project/
dbt run   # ✓ Works immediately
dbt test  # ✓ All tests pass
dbt docs generate  # ✓ Full documentation
```

--------------------------------

### Example dbt sources.yml Structure

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBT_INTEGRATION_DEEP_DIVE.md

Sample YAML structure for dbt source definitions.

```yaml
version: 2

sources:
  - name: raw
    database: mydb
    schema: public
    description: Raw tables from mydb database

    tables:
      - name: users
        description: User accounts
        columns:
          - name: id
            description: Primary key
          - name: email
            description: User email address
          - name: created_at
            description: Account creation timestamp

        freshness:
          warn_after:
            count: 24
            period: hour
          error_after:
            count: 48
            period: hour

      - name: orders
        description: Customer orders
        columns:
          - name: id
          - name: user_id
          - name: total
          - name: created_at
```

--------------------------------

### Generate code coverage report

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/contributing/testing.mdx

Install and run cargo-tarpaulin to generate an HTML coverage report.

```bash
cargo install cargo-tarpaulin
cargo tarpaulin --out Html
```

--------------------------------

### DuckDB SQL features

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/analytics-queries.mdx

Examples of advanced SQL features supported by the underlying DuckDB engine.

```sql
-- Window functions
SELECT
  id,
  name,
  SUM(total) OVER (PARTITION BY user_id ORDER BY created_at) as running_total
FROM orders;

-- CTEs
WITH active_users AS (
  SELECT * FROM users WHERE active = 1
)
SELECT * FROM orders WHERE user_id IN (SELECT id FROM active_users);

-- QUALIFY
SELECT * FROM orders
QUALIFY ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY created_at DESC) = 1;
```

--------------------------------

### Portable Column YAML Definition

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/generate/model-reference.mdx

Example configuration for a column definition using the portable schema.

```yaml
- name: email
  source_type: varchar(255)
  family: text
  nullable: false
  primary_key: false
  unique: true
  generated: false
  identity: false
```

--------------------------------

### Combine sql-splitter with sample

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DUCKDB_INTEGRATION_DEEP_DIVE.md

Pipe sampled data directly into a DuckDB query for analysis.

```bash
# Sample data, then analyze with DuckDB
sql-splitter sample prod.sql --percent 10 | \
  sql-splitter query - "
    SELECT
      DATE_TRUNC('month', created_at) as month,
      COUNT(*) as signups
    FROM users
    GROUP BY month
    ORDER BY month
  "
```

--------------------------------

### JSON output structure

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/generate/index.mdx

Example of the machine-readable report generated when using the --json flag.

```json
{
  "mode": "generate",
  "rows_written": 65,
  "effective_seed": 42,
  "diagnostics": [
    {
      "code": "GEN-SOURCE-VALUES",
      "severity": "advisory",
      "path": "tables",
      "message": "2 rule(s) replay hand-authored literal values; the output is synthetic, not anonymized source data",
      "documentation_url": "https://sql-splitter.dev/commands/generate/diagnostics/#GEN-SOURCE-VALUES",
      "related": [
        { "path": "tables.customers.columns.notes", "description": "constant" },
        {
          "path": "tables.orders.columns.status",
          "description": "source_default"
        }
      ]
    }
  ]
}
```

--------------------------------

### Generate and Review Migrations

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/MIGRATE_FEATURE.md

Perform a dry run to preview changes before generating actual upgrade and rollback SQL files.

```bash
sql-splitter migrate v1.sql v2.sql --dry-run
# Review changes

sql-splitter migrate v1.sql v2.sql -o upgrade.sql --rollback downgrade.sql
```

--------------------------------

### Generating Migration Scripts

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/commands/diff.mdx

Output the differences as a runnable SQL DDL file.

```bash
sql-splitter diff old.sql new.sql --format sql -o migration.sql
```

--------------------------------

### View Agent Detection Notes

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/advanced/ai-integration.mdx

Example output shown when sql-splitter detects a coding agent.

```text
Notes for AI agents:
  Documentation in LLM-readable form (read this before composing commands):
    https://sql-splitter.dev/llms.txt        index of the doc set
    ...
```

--------------------------------

### Sample SQL Dumps

Source: https://github.com/helgesverre/sql-splitter/blob/main/skills/sql-splitter/SKILL.md

Commands to create reduced datasets from a SQL dump while preserving foreign key relationships.

```bash
sql-splitter sample dump.sql --output sampled.sql --percent 10 --preserve-relations
sql-splitter sample dump.sql --output sampled.sql --rows 1000 --preserve-relations
sql-splitter sample dump.sql --output sampled.sql --percent 10 --tables users,orders
sql-splitter sample dump.sql --output sampled.sql --percent 10 --seed 42
```

--------------------------------

### Custom dbt Test Example

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DBT_INTEGRATION_DEEP_DIVE.md

A custom SQL test to ensure order totals are positive.

```sql
-- Test: total must be positive
SELECT *
FROM {{ ref('stg_orders') }}
WHERE NOT (total > 0)
```

--------------------------------

### Run sql-splitter with pre-built images

Source: https://github.com/helgesverre/sql-splitter/blob/main/website/src/content/docs/cookbook/docker-usage.mdx

Execute sql-splitter using official images from Docker Hub or GitHub Container Registry.

```bash
# Docker Hub
docker run --rm -v $(pwd):/data helgesverre/sql-splitter analyze /data/dump.sql

# GitHub Container Registry
docker run --rm -v $(pwd):/data ghcr.io/helgesverre/sql-splitter analyze /data/dump.sql
```

--------------------------------

### Define MySQL Enum Column

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/ENUM_CONVERSION.md

Example of a MySQL ENUM definition with standard and escaped quotes.

```sql
status ENUM('simple', 'with ''quote', 'with\\backslash')
```

--------------------------------

### CLI Usage for Selective Import

Source: https://github.com/helgesverre/sql-splitter/blob/main/docs/features/DUCKDB_INTEGRATION_DEEP_DIVE.md

Example command to import only the 'users' table from a dump file.

```bash
# Only import 'users' table, ignore rest
sql-splitter query dump.sql \
  --table users \
  --sql "SELECT COUNT(*) FROM users WHERE active = true"

# Much faster for large dumps!
```
Tokens:
Raw