### Django Project Setup Source: https://docs.paradedb.com/documentation/getting-started/environment Install Django, Psycopg, and django-paradedb, then create a new Django project and app. ```bash python3 -m venv .venv source .venv/bin/activate pip install django psycopg django-paradedb==0.8.0 python3 -m django startproject myproject . python3 manage.py startapp myapp ``` -------------------------------- ### Install Dependencies (Ubuntu) Source: https://docs.paradedb.com/deploy/byoc Installs Terraform, Kubectl, and PostgreSQL on Ubuntu using apt-get. ```bash sudo apt-get install -y terraform kubectl postgresql ``` -------------------------------- ### Setup Distributed Search with Citus Source: https://docs.paradedb.com/deploy/citus Create and distribute a table, then build a BM25 index for full-text search. This example demonstrates inserting data and performing a distributed search query. ```sql CREATE EXTENSION citus; CREATE EXTENSION pg_search; -- Create a table with a distribution key CREATE TABLE articles ( id SERIAL, author_id INT NOT NULL, title TEXT, body TEXT, PRIMARY KEY (author_id, id) -- Must include distribution column ); -- Distribute the table across shards SELECT create_distributed_table('articles', 'author_id'); -- Create a BM25 index on the distributed table CREATE INDEX articles_search_idx ON articles USING bm25 (id, title, body) WITH (key_field='id'); -- Insert some data INSERT INTO articles (author_id, title, body) VALUES (1, 'PostgreSQL Performance', 'Optimizing PostgreSQL queries for large datasets'), (1, 'Distributed Databases', 'Understanding sharding and replication strategies'), (2, 'Full-Text Search', 'Building search engines with PostgreSQL'); -- Search across shards SELECT id, title FROM articles WHERE body ||| 'PostgreSQL distributed' ORDER BY id; -- Results: -- id | title -- ----+------------------------ -- 1 | PostgreSQL Performance -- 3 | Full-Text Search ``` -------------------------------- ### Install Dependencies with Pip Source: https://docs.paradedb.com/documentation/getting-started/environment Install SQLAlchemy, Alembic, Psycopg, and sqlalchemy-paradedb using pip within a virtual environment. ```bash python3 -m venv .venv source .venv/bin/activate pip install sqlalchemy psycopg alembic sqlalchemy-paradedb==0.6.0 ``` -------------------------------- ### Install Docker on Droplet Source: https://docs.paradedb.com/deploy/cloud-platforms/digitalocean Installs Docker on your DigitalOcean Droplet. This script fetches and executes the official Docker installation script. ```bash curl -fsSL https://get.docker.com | sh ``` -------------------------------- ### Install Dependencies (macOS) Source: https://docs.paradedb.com/deploy/byoc Installs Terraform, Kubectl, and PostgreSQL on macOS using Homebrew. ```bash brew install terraform kubectl postgresql ``` -------------------------------- ### Example: Indexing and Searching with Search Tokenizer Source: https://docs.paradedb.com/documentation/tokenizers/search-tokenizer Demonstrates creating a table, inserting data, creating an index with a specific search tokenizer, and performing searches. This example highlights how 'sho' and 's' are tokenized differently at search time compared to index time. ```sql CREATE TABLE products ( id serial8 NOT NULL PRIMARY KEY, title text ); INSERT INTO products (title) VALUES ('shoes'), ('shirt'), ('shorts'), ('shoelaces'), ('socks'); CREATE INDEX idx_products ON products USING bm25 (id, (title::pdb.ngram(1, 10, 'prefix_only=true'))) WITH (key_field = 'id', search_tokenizer = 'unicode_words'); -- "sho" stays as one token → matches shoes, shorts, shoelaces SELECT id, title FROM products WHERE title ||| 'sho' ORDER BY id; -- "s" stays as one token → matches all five titles SELECT id, title FROM products WHERE title ||| 's' ORDER BY id; ``` -------------------------------- ### Install Prometheus Stack Source: https://docs.paradedb.com/deploy/self-hosted/kubernetes Installs the Prometheus community Helm chart for monitoring. Ensure Helm is installed and a Kubernetes cluster is running. ```bash helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm upgrade --atomic --install prometheus-community \ --create-namespace \ --namespace prometheus-community \ --values https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/main/docs/src/samples/monitoring/kube-stack-config.yaml \ prometheus-community/kube-prometheus-stack ``` -------------------------------- ### Install Extension Binaries with apt-get Source: https://docs.paradedb.com/deploy/third-party-extensions Update package lists and install prebuilt extension binaries using apt-get. This is a common method for extensions available in package repositories. ```bash apt-get update apt-get install -y --no-install-recommends postgresql-18-partman ``` -------------------------------- ### Example SQL Migration for users_service Source: https://docs.paradedb.com/deploy/logical-replication/multi-database This is a concrete example of the migration script applied to a 'users_service' database, moving 'users' and 'profiles' tables and creating corresponding views. ```sql BEGIN; -- Create new schema CREATE SCHEMA IF NOT EXISTS users_service; -- Move tables ALTER TABLE public.users SET SCHEMA users_service; ALTER TABLE public.profiles SET SCHEMA users_service; -- Create backwards-compatible views CREATE OR REPLACE VIEW public.users AS SELECT * FROM users_service.users; CREATE OR REPLACE VIEW public.profiles AS SELECT * FROM users_service.profiles; COMMIT; ``` -------------------------------- ### Get First Snippet Source: https://docs.paradedb.com/documentation/full-text/highlight Use the `limit` parameter set to 1 to retrieve only the first snippet. This example shows the SQL implementation. ```sql SELECT id, pdb.snippets(description, max_num_chars => 15, "limit" => 1) FROM mock_items WHERE description ||| 'running' LIMIT 5; ``` -------------------------------- ### Create Index with Regex Pattern Tokenizer Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/regex Example of creating a search index using the regex_pattern tokenizer to index words starting with 'h'. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.regex_pattern('(?i)\bh\w*'))) WITH (key_field='id'); ``` -------------------------------- ### Install Rails Gems Source: https://docs.paradedb.com/documentation/getting-started/environment Install all the gems specified in your Gemfile, including the newly added 'rails-paradedb' gem. ```bash bundle install ``` -------------------------------- ### Install ParadeDB Cluster Source: https://docs.paradedb.com/deploy/self-hosted/kubernetes Installs the ParadeDB Helm chart into the specified Kubernetes namespace using a values.yaml file and enabling monitoring. ```bash helm repo add paradedb https://paradedb.github.io/charts helm upgrade --atomic --install paradedb \ --namespace \ --values values.yaml \ --set cluster.monitoring.enabled=true \ paradedb/paradedb ``` -------------------------------- ### Install Citus and Configure PostgreSQL Source: https://docs.paradedb.com/deploy/citus Install the Citus extension and configure PostgreSQL to load both citus and pg_search extensions. Ensure citus is listed before pg_search in shared_preload_libraries. ```bash # Install Citus first curl https://install.citusdata.com/community/deb.sh | sudo bash apt-get install -y postgresql-18-citus-14.0 # Add both extensions to shared_preload_libraries sed -i "s/^shared_preload_libraries = .*/shared_preload_libraries = 'citus,pg_search'/' /var/lib/postgresql/data/postgresql.conf # Restart PostgreSQL # Then create extensions in your database ``` -------------------------------- ### Get First Snippet (SQLAlchemy) Source: https://docs.paradedb.com/documentation/full-text/highlight Use the `limit` parameter set to 1 to retrieve only the first snippet. This example shows the SQLAlchemy implementation. ```python from sqlalchemy import select from sqlalchemy.orm import Session from paradedb.sqlalchemy import pdb, search stmt = ( select(MockItem.id, pdb.snippets(MockItem.description, max_num_chars=15, limit=1).label("snippets")) .where(search.match_any(MockItem.description, "running")) .limit(5) ) with Session(engine) as session: session.execute(stmt).all() ``` -------------------------------- ### Regex Query Result Example Source: https://docs.paradedb.com/documentation/query-builder/overview This is an example of the expected output when querying for items matching the regex 'key.*rd' in the description. ```text description | rating | category --------------------------+--------+------------- Ergonomic metal keyboard | 4 | Electronics Plastic Keyboard | 4 | Electronics (2 rows) ``` -------------------------------- ### Copy GCP Example Terraform Variables Source: https://docs.paradedb.com/deploy/byoc Copies the example Terraform variable file for GCP to be used for BYOC configuration. ```bash cp gcp.example.tfvars byoc.tfvars ``` -------------------------------- ### Create Extension in ParadeDB Source: https://docs.paradedb.com/deploy/third-party-extensions Connect to ParadeDB using psql and execute the CREATE EXTENSION command to enable the installed extension. ```sql CREATE EXTENSION pg_partman; ``` -------------------------------- ### Install CloudNativePG Operator Source: https://docs.paradedb.com/deploy/self-hosted/kubernetes Installs the CloudNativePG operator Helm chart. Monitoring can be enabled using --set flags. Skip if already installed. ```bash helm repo add cnpg https://cloudnative-pg.github.io/charts helm upgrade --atomic --install cnpg \ --create-namespace \ --namespace cnpg-system \ --set monitoring.podMonitorEnabled=true \ --set monitoring.grafanaDashboard.create=true \ cnpg/cloudnative-pg ``` -------------------------------- ### Copy AWS Example Terraform Variables Source: https://docs.paradedb.com/deploy/byoc Copies the example Terraform variable file for AWS to be used for BYOC configuration. ```bash cp aws.example.tfvars byoc.tfvars ``` -------------------------------- ### Create Index with Trim Filter Source: https://docs.paradedb.com/documentation/token-filters/trim Example of creating an index using the literal_normalized tokenizer with the trim filter enabled. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.literal_normalized('trim=true'))) WITH (key_field='id'); ``` -------------------------------- ### Get Second Snippet (SQL) Source: https://docs.paradedb.com/documentation/full-text/highlight Use `limit` and `offset` parameters to retrieve the second snippet by skipping the first. This example shows the SQL implementation. ```sql SELECT id, pdb.snippets(description, max_num_chars => 15, "limit" => 1, "offset" => 1) FROM mock_items WHERE description ||| 'running' LIMIT 5; ``` -------------------------------- ### Install ParadeDB Agent Skill Source: https://docs.paradedb.com/documentation Install the paradedb-skill using npx to integrate ParadeDB context with your coding agent. This works with various coding assistants. ```bash npx skills add paradedb/agent-skills ``` -------------------------------- ### SQL Snippets Example Source: https://docs.paradedb.com/documentation/full-text/highlight Retrieve multiple highlighted snippets from a description column in SQL. This example uses the `pdb.snippets` function with a maximum character limit. ```SQL SELECT id, pdb.snippets(description, max_num_chars => 15) FROM mock_items WHERE description ||| 'artistic vase' LIMIT 5; ``` -------------------------------- ### Install ParadeDB on Droplet Source: https://docs.paradedb.com/deploy/cloud-platforms/digitalocean Installs ParadeDB on your DigitalOcean Droplet using a provided script. The 'tag' parameter pins the ParadeDB version. Refer to Docker Hub for available tags. ```bash curl -fsSL "https://paradedb.com/install.sh?tag=0.23.5-pg18" | sh ``` -------------------------------- ### Insert Mock Data for Phrase Query Example Source: https://docs.paradedb.com/documentation/full-text/phrase This SQL snippet inserts sample data into a mock_items table, which will be used to demonstrate phrase queries. ```sql INSERT INTO mock_items (description, rating, category) VALUES ('running sleek shoes', 5, 'Footwear'), ('shoes running', 5, 'Footwear'); ``` -------------------------------- ### SQL MIN Syntax Example Source: https://docs.paradedb.com/documentation/aggregates/metrics/minmax This is the equivalent SQL query using the standard MIN function after enabling custom aggregate scans. ```sql SELECT MIN(rating) FROM mock_items WHERE id @@@ pdb.all(); ``` -------------------------------- ### Install pg_search on RHEL 9 Source: https://docs.paradedb.com/deploy/self-hosted/extension Download and install the ParadeDB pg_search extension for RHEL 9. Ensure you use the correct architecture (x86_64 or aarch64). ```bash # Available arch versions are x86_64, aarch64 curl -L "https://github.com/paradedb/paradedb/releases/download/v0.23.5/pg_search_18-0.23.5-1PARADEDB.el9.x86_64.rpm" -o /tmp/pg_search.rpm sudo dnf install -y /tmp/*.rpm ``` -------------------------------- ### Install pg_search on RHEL 10 Source: https://docs.paradedb.com/deploy/self-hosted/extension Download and install the ParadeDB pg_search extension for RHEL 10. Ensure you use the correct architecture (x86_64 or aarch64). ```bash # Available arch versions are x86_64, aarch64 curl -L "https://github.com/paradedb/paradedb/releases/download/v0.23.5/pg_search_18-0.23.5-1PARADEDB.el10.x86_64.rpm" -o /tmp/pg_search.rpm sudo dnf install -y /tmp/*.rpm ``` -------------------------------- ### Filter Pushdown Example: Integer Equality Source: https://docs.paradedb.com/documentation/filtering Demonstrates filter pushdown for integer types using the equality operator. ```sql WHERE rating = 2 ``` -------------------------------- ### Filter Pushdown Example: Numeric Equality Source: https://docs.paradedb.com/documentation/filtering Demonstrates filter pushdown for numeric types using the equality operator. ```sql WHERE price = 99.99 ``` -------------------------------- ### Filter Pushdown Example: Boolean Equality Source: https://docs.paradedb.com/documentation/filtering Demonstrates filter pushdown for boolean types using the equality operator. ```sql WHERE in_stock = true ``` -------------------------------- ### Create BM25 Index using SQLAlchemy Source: https://docs.paradedb.com/documentation/indexing/create-index This SQLAlchemy example shows how to define and create a BM25 index with specified fields and options. ```python from sqlalchemy import Index from paradedb.sqlalchemy import indexing idx = Index( "search_idx", indexing.BM25Field(MockItem.id), indexing.BM25Field(MockItem.description), indexing.BM25Field(MockItem.category), indexing.BM25Field(MockItem.rating), indexing.BM25Field(MockItem.in_stock), indexing.BM25Field(MockItem.created_at), indexing.BM25Field(MockItem.metadata_), indexing.BM25Field(MockItem.weight_range), postgresql_using="bm25", postgresql_with={"key_field": "id"}, ) with engine.begin() as conn: idx.create(conn) ``` -------------------------------- ### Open Rails Console Source: https://docs.paradedb.com/documentation/getting-started/environment Start the Rails console to interact with your application and database, and to run queries. ```bash rails console ``` -------------------------------- ### Create Index with Chinese Compatible Tokenizer Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/chinese-compatible Example of creating a search index using the chinese_compatible tokenizer on a text field. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.chinese_compatible)) WITH (key_field='id'); ``` -------------------------------- ### Django Top K Query Example Source: https://docs.paradedb.com/documentation/sorting/topk This Django ORM example illustrates a Top K query, utilizing ParadeDB's MatchAny for text search and specifying order and limit. ```python from paradedb import MatchAny, ParadeDB MockItem.objects.filter( description=ParadeDB(MatchAny('running shoes')) ).order_by('rating').values('description', 'rating', 'category')[:5] ``` -------------------------------- ### Get Current ParadeDB Extension Version Source: https://docs.paradedb.com/deploy/upgrading Inspect the current version of the pg_search extension installed in your PostgreSQL database. ```sql SELECT extversion FROM pg_extension WHERE extname = 'pg_search'; ``` -------------------------------- ### Create Index with Edge Ngram Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/edge-ngrams Example of creating a search index using the Edge Ngram tokenizer with specified minimum and maximum gram lengths. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.edge_ngram(2,5))) WITH (key_field='id'); ``` -------------------------------- ### Example EXPLAIN Output for Distributed JOIN Source: https://docs.paradedb.com/deploy/citus This is a sample EXPLAIN output demonstrating a distributed JOIN. It shows key indicators like local JOIN execution on shards, use of BM25 for filtering, and index scans for lookups. ```text Sort (cost=12067.32..12317.32 rows=100000 width=64) Output: remote_scan.name, remote_scan.title Sort Key: remote_scan.name -> Custom Scan (Citus Adaptive) (cost=0.00..0.00 rows=100000 width=64) Output: remote_scan.name, remote_scan.title Task Count: 32 Tasks Shown: One of 32 -> Task Query: SELECT a.name, ar.title FROM (public.authors_102040 a JOIN public.articles_102008 ar ON (...)) Node: host=localhost port=5432 dbname=postgres -> Nested Loop (cost=10.15..18.20 rows=1 width=64) Output: a.name, ar.title Inner Unique: true -> Custom Scan (ParadeDB Base Scan) on public.articles_102008 ar (cost=10.00..10.01 rows=1 width=36) Output: ar.title, ar.author_id Table: articles_102008 Index: articles_search_idx_102008 Tantivy Query: {"with_index":{"query":{"with_index":{"query":{"match":{"field":"body","value":"PostgreSQL"}}}}}} -> Index Scan using authors_pkey_102040 on public.authors_102040 a (cost=0.15..8.17 rows=1 width=36) Output: a.id, a.name, a.bio Index Cond: (a.id = ar.author_id) ``` -------------------------------- ### Get First Snippet (Rails) Source: https://docs.paradedb.com/documentation/full-text/highlight Use the `limit` parameter set to 1 to retrieve only the first snippet. This example shows the Rails implementation. ```ruby MockItem.search(:description) .matching_any("running") .with_snippets(:description, max_chars: 15, limit: 1) .select(:id) .limit(5) ``` -------------------------------- ### Create Database on Publisher Source: https://docs.paradedb.com/deploy/logical-replication/getting-started Create the database that will be published for replication. ```bash sudo -u postgres -H createdb marketplace ``` -------------------------------- ### Equivalent Index Configurations Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/unicode Demonstrates two equivalent configurations for creating a search index using the default unicode tokenizer. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, description) WITH (key_field='id'); CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.unicode_words)) WITH (key_field='id'); ``` -------------------------------- ### Enter ParadeDB Shell Source: https://docs.paradedb.com/deploy/third-party-extensions Access the ParadeDB container's shell with root permissions to install extensions. ```bash docker exec -it --user root paradedb bash ``` -------------------------------- ### Get First Snippet (Django) Source: https://docs.paradedb.com/documentation/full-text/highlight Use the `limit` parameter set to 1 to retrieve only the first snippet. This example shows the Django implementation. ```python from paradedb import MatchAny, ParadeDB, Snippets MockItem.objects.filter( description=ParadeDB(MatchAny('running')) ).annotate( snippets=Snippets('description', max_num_chars=15, limit=1) ).values('id', 'snippets')[:5] ``` -------------------------------- ### Open TypeScript REPL (Bash) Source: https://docs.paradedb.com/documentation/getting-started/environment Start a Node.js TypeScript REPL using tsx for interactive development and testing. ```bash node --import tsx ``` -------------------------------- ### Get First Snippet (Drizzle) Source: https://docs.paradedb.com/documentation/full-text/highlight Use the `limit` parameter set to 1 to retrieve only the first snippet. This example shows the Drizzle implementation. ```ts import { search } from "@paradedb/drizzle-paradedb"; await db .select({ id: mockItems.id, snippets: search.snippets(mockItems.description, { maxNumChars: 15, limit: 1, }), }) .from(mockItems) .where(search.matchAny(mockItems.description, "running")) .limit(5); ``` -------------------------------- ### GCP CLI Authentication Source: https://docs.paradedb.com/deploy/byoc Initializes and logs in to the GCP CLI for authentication. Requires the GCP SDK to be installed. ```bash gcloud init gcloud auth application-default login ``` -------------------------------- ### Create Index with Japanese Lindera Tokenizer Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/lindera Example of creating a bm25 index using the Japanese Lindera tokenizer on a description column. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.lindera(japanese))) WITH (key_field='id'); ``` -------------------------------- ### Create ParadeDB Test Table (SQL) Source: https://docs.paradedb.com/documentation/getting-started/environment Use this SQL procedure to create a table populated with mock data for testing and getting started with ParadeDB. ```sql CALL paradedb.create_bm25_test_table( schema_name => 'public', table_name => 'mock_items' ); ``` -------------------------------- ### Create Index with Literal Tokenizer Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/literal Example of creating a search index using the literal tokenizer for the description field. The 'id' field is also included. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.literal)) WITH (key_field='id'); ``` -------------------------------- ### Get Second Snippet (Rails) Source: https://docs.paradedb.com/documentation/full-text/highlight Use `limit` and `offset` parameters to retrieve the second snippet by skipping the first. This example shows the Rails implementation. ```ruby MockItem.search(:description) .matching_any("running") .with_snippets(:description, max_chars: 15, limit: 1, offset: 1) .select(:id) .limit(5) ``` -------------------------------- ### Create New Rails App Source: https://docs.paradedb.com/documentation/getting-started/environment Initialize a new Rails application with PostgreSQL as the database adapter. ```bash rails new paradedb -d postgresql cd paradedb ``` -------------------------------- ### Get Second Snippet (SQLAlchemy) Source: https://docs.paradedb.com/documentation/full-text/highlight Use `limit` and `offset` parameters to retrieve the second snippet by skipping the first. This example shows the SQLAlchemy implementation. ```python from sqlalchemy import select from sqlalchemy.orm import Session from paradedb.sqlalchemy import pdb, search stmt = ( select( MockItem.id, pdb.snippets(MockItem.description, max_num_chars=15, limit=1, offset=1).label("snippets"), ) .where(search.match_any(MockItem.description, "running")) .limit(5) ) with Session(engine) as session: session.execute(stmt).all() ``` -------------------------------- ### Tokenizing Sample Text Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/unicode Shows how to tokenize a simple string using the default unicode tokenizer. ```sql SELECT 'Tokenize me!'::pdb.unicode_words::text[]; ``` -------------------------------- ### Get Second Snippet (Django) Source: https://docs.paradedb.com/documentation/full-text/highlight Use `limit` and `offset` parameters to retrieve the second snippet by skipping the first. This example shows the Django implementation. ```python from paradedb import MatchAny, ParadeDB, Snippets MockItem.objects.filter( description=ParadeDB(MatchAny('running')) ).annotate( snippets=Snippets('description', max_num_chars=15, limit=1, offset=1) ).values('id', 'snippets')[:5] ``` -------------------------------- ### Create Index with Korean Lindera Tokenizer Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/lindera Example of creating a bm25 index using the Korean Lindera tokenizer on a description column. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.lindera(korean))) WITH (key_field='id'); ``` -------------------------------- ### Get Second Snippet (Drizzle) Source: https://docs.paradedb.com/documentation/full-text/highlight Use `limit` and `offset` parameters to retrieve the second snippet by skipping the first. This example shows the Drizzle implementation. ```ts import { search } from "@paradedb/drizzle-paradedb"; await db .select({ id: mockItems.id, snippets: search.snippets(mockItems.description, { maxNumChars: 15, limit: 1, offset: 1, }), }) .from(mockItems) .where(search.matchAny(mockItems.description, "running")) .limit(5); ``` -------------------------------- ### Run Database Migrations Source: https://docs.paradedb.com/documentation/getting-started/environment Execute this command to apply all pending database migrations, including the new search index. ```bash rails db:migrate ``` -------------------------------- ### Drizzle ORM Snippets Example Source: https://docs.paradedb.com/documentation/full-text/highlight Get multiple highlighted snippets using Drizzle ORM with ParadeDB integration. Requires importing the `search` module. ```TypeScript import { search } from "@paradedb/drizzle-paradedb"; await db .select({ id: mockItems.id, snippets: search.snippets(mockItems.description, { maxNumChars: 15 }), }) .from(mockItems) .where(search.matchAny(mockItems.description, "artistic vase")) .limit(5); ``` -------------------------------- ### Rails Top Hits Aggregation Source: https://docs.paradedb.com/documentation/aggregates/metrics/tophits Example of using the top_hits aggregation in Rails to get the top 3 documents per rating category, sorted by creation date. ```Ruby MockItem.search(:id) .match_all .aggregate_by( :rating, agg: ParadeDB::Aggregations.top_hits( size: 3, sort: [{ created_at: "desc" }], docvalue_fields: %w[id created_at] ) ) ``` -------------------------------- ### SQL Term Queries Source: https://docs.paradedb.com/documentation/full-text/term Demonstrates two SQL term queries, one for 'running' and another for 'RUNNING', to find exact token matches in the 'description' column. ```SQL -- Term query 1 SELECT description, rating, category FROM mock_items WHERE description === 'running'; -- Term query 2 SELECT description, rating, category FROM mock_items WHERE description === 'RUNNING'; ``` -------------------------------- ### SQL MAX Syntax Example Source: https://docs.paradedb.com/documentation/aggregates/metrics/minmax This is the equivalent SQL query using the standard MAX function after enabling custom aggregate scans. ```sql SELECT MAX(rating) FROM mock_items WHERE id @@@ pdb.all(); ``` -------------------------------- ### Create Index with Chinese Lindera Tokenizer Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/lindera Example of creating a bm25 index using the Chinese Lindera tokenizer on a description column. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.lindera(chinese))) WITH (key_field='id'); ``` -------------------------------- ### Configure Publisher pg_hba.conf Source: https://docs.paradedb.com/deploy/logical-replication/getting-started Allow the subscriber to connect to the publisher by adding these lines to `pg_hba.conf`. ```ini local replication all peer host replication all 127.0.0.1/32 scram-sha-256 host replication all ::1/128 scram-sha-256 host replication all 192.168.0.0/24 scram-sha-256 ``` -------------------------------- ### Retrieve Byte Offsets for Snippets Source: https://docs.paradedb.com/documentation/full-text/highlight Use `snippet_positions` to get the byte offsets of highlighted text within the original document. The output is a 2D array of [start, end) byte indices. ```sql SELECT id, pdb.snippet(description), pdb.snippet_positions(description) FROM mock_items WHERE description ||| 'shoes' LIMIT 5; ``` ```typescript import { search } from "@paradedb/drizzle-paradedb"; await db .select({ id: mockItems.id, snippet: search.snippet(mockItems.description), snippetPositions: search.snippetPositions(mockItems.description), }) .from(mockItems) .where(search.matchAny(mockItems.description, "shoes")) .limit(5); ``` ```python from paradedb import MatchAny, ParadeDB, Snippet, SnippetPositions MockItem.objects.filter( description=ParadeDB(MatchAny('shoes')) ).annotate( snippet=Snippet('description'), snippet_positions=SnippetPositions('description') ).values('id', 'snippet', 'snippet_positions')[:5] ``` ```python from sqlalchemy import select from sqlalchemy.orm import Session from paradedb.sqlalchemy import pdb, search stmt = ( select( MockItem.id, pdb.snippet(MockItem.description).label("snippet"), pdb.snippet_positions(MockItem.description).label("snippet_positions"), ) .where(search.match_any(MockItem.description, "shoes")) .limit(5) ) with Session(engine) as session: session.execute(stmt).all() ``` ```ruby MockItem.search(:description) .matching_any("shoes") .with_snippet(:description) .with_snippet_positions(:description) .select(:id) .limit(5) ``` -------------------------------- ### Drizzle Proximity Chaining Source: https://docs.paradedb.com/documentation/full-text/proximity Chains proximity clauses in Drizzle ORM for ParadeDB. This example uses the search.proximity and related functions to build a complex proximity search. Ensure the @paradedb/drizzle-paradedb package is installed. ```ts import { search } from "@paradedb/drizzle-paradedb"; await db .select({ description: mockItems.description, rating: mockItems.rating, category: mockItems.category, }) .from(mockItems) .where( search.proximity( mockItems.description, search .proxStr("sleek") .within(1, "running") .within(2, search.proxArray("sneakers", search.proxRegex("sho.*`))), ), ); ``` -------------------------------- ### SQLAlchemy Term Queries Source: https://docs.paradedb.com/documentation/full-text/term Provides SQLAlchemy examples for term queries, demonstrating exact matches for 'running' and 'RUNNING' using ParadeDB's search module. ```Python from sqlalchemy import select from sqlalchemy.orm import Session from paradedb.sqlalchemy import search term_query_1 = ( select(MockItem.description, MockItem.rating, MockItem.category) .where(search.term(MockItem.description, "running")) ) term_query_2 = ( select(MockItem.description, MockItem.rating, MockItem.category) .where(search.term(MockItem.description, "RUNNING")) ) with Session(engine) as session: { "rows_query_1": session.execute(term_query_1).all(), "rows_query_2": session.execute(term_query_2).all(), } ``` -------------------------------- ### SQL Top Hits Aggregation Source: https://docs.paradedb.com/documentation/aggregates/metrics/tophits Example of using the top_hits aggregation in SQL to get the top 3 results sorted by 'created_at' for each 'rating' category. Includes 'id' and 'created_at' as docvalue fields. ```SQL SELECT pdb.agg('{"top_hits": {"size": 3, "sort": [{"created_at": "desc"}], "docvalue_fields": ["id", "created_at"]}}') FROM mock_items WHERE id @@@ pdb.all() GROUP BY rating; ``` -------------------------------- ### Set Filter Pushdown Source: https://docs.paradedb.com/documentation/filtering Demonstrates how to push down set filters over text fields, requiring the literal tokenizer for the indexed field. This example uses SQL for an IN clause. ```SQL SELECT description, rating, category FROM mock_items WHERE description @@@ 'shoes' AND category IN ('Footwear', 'Apparel'); ``` -------------------------------- ### Boost a regex query (Rails) Source: https://docs.paradedb.com/documentation/sorting/boost Use the `.regex` method with the `boost` option in Rails to apply a boost to regex matches. This example boosts 'key.*' by 2. ```ruby MockItem.search(:description) .regex("key.*", boost: 2) .with_score .select(:id, :description, :category) .order(search_score: :desc) .limit(5) ``` -------------------------------- ### Top Hits Aggregation with Basic Sorting Source: https://docs.paradedb.com/documentation/aggregates/metrics/tophits Demonstrates how to use the Top Hits aggregation to get the top 3 documents, sorted by 'created_at' in descending order, with specific fields returned. This example shows usage across multiple programming languages and frameworks. ```sql SELECT pdb.agg('{"top_hits": {"size": 3, "from": 1, "sort": [{"created_at": "desc"}], "docvalue_fields": ["id", "created_at"]}}') FROM mock_items WHERE id @@@ pdb.all() GROUP BY rating; ``` ```typescript import { search, type ParadeDBClient, } from "@paradedb/drizzle-paradedb"; // Assuming 'db' is your ParadeDB client instance // and 'mockItems' is your Drizzle schema for mock_items const result = await db.select({ agg: search.agg({ top_hits: { size: 3, from: 1, sort: [{ created_at: "desc" }], docvalue_fields: ["id", "created_at"], }, }), }) .from(mockItems) .where(search.all(mockItems.id)) .groupBy(mockItems.rating); ``` ```python from paradedb import Agg, All, ParadeDB # Assuming MockItem is your Django model MockItem.objects.filter( id=ParadeDB(All()) ).values('rating').annotate( agg=Agg('{"top_hits": {"size": 3, "from": 1, "sort": [{"created_at": "desc"}], "docvalue_fields": ["id", "created_at"]}}') ).values('agg') ``` ```python from sqlalchemy import select from sqlalchemy.orm import Session from paradedb.sqlalchemy import facets, pdb, search # Assuming MockItem is your SQLAlchemy model and engine is configured stmt = ( select( pdb.agg( facets.top_hits( size=3, from_=1, sort=[{"created_at": "desc"}], docvalue_fields=["id", "created_at"], ) ) ) .select_from(MockItem) .where(search.all(MockItem.id)) .group_by(MockItem.rating) ) with Session(engine) as session: session.execute(stmt).all() ``` ```ruby # Assuming MockItem is your Rails model MockItem.search(:id) .match_all .aggregate_by( :rating, agg: ParadeDB::Aggregations.top_hits( size: 3, from: 1, sort: [{ created_at: "desc" }], docvalue_fields: %w[id created_at] ) ) ``` -------------------------------- ### Create Index with Source Code Tokenizer Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/source-code This SQL snippet demonstrates how to create a full-text search index using the `bm25` algorithm and the `pdb.source_code` tokenizer on a description column. ```sql CREATE INDEX search_idx ON mock_items USING bm25 (id, (description::pdb.source_code)) WITH (key_field='id'); ``` -------------------------------- ### Sample GitLab CI Workflow with ParadeDB Source: https://docs.paradedb.com/deploy/ci/gitlab-ci This snippet shows a complete GitLab CI configuration to set up and use ParadeDB with a PostgreSQL service. It includes image selection, service configuration, environment variables, and example SQL commands for table creation, data retrieval, index creation, and searching. ```yaml paradedb-in-gitlab-ci: # The list of available tags can be found at https://hub.docker.com/r/paradedb/paradedb/tags image: paradedb/paradedb:latest services: - postgres variables: POSTGRES_USER: testuser POSTGRES_DB: testdb POSTGRES_HOST_AUTH_METHOD: trust script: - psql -h "postgres" -U testuser -d testdb -c "CALL paradedb.create_bm25_test_table(schema_name => 'public', table_name => 'mock_items');" - psql -h "postgres" -U testuser -d testdb -c "SELECT description, rating, category FROM mock_items LIMIT 3;" - psql -h "postgres" -U testuser -d testdb -c "CREATE INDEX search_idx ON mock_items USING bm25 (id, description, category, rating, in_stock, created_at, metadata, weight_range) WITH (key_field='id');" - psql -h "postgres" -U testuser -d testdb -c "SELECT description, rating, category FROM mock_items WHERE description @@@ 'shoes' OR category @@@ 'footwear' AND rating @@@ '>2' ORDER BY description LIMIT 5;" ``` -------------------------------- ### Create Table and Index for Phrase Queries Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/ngrams Sets up a table and index using the ngram tokenizer with `positions=true` to support exact substring matching with phrase queries. ```sql CREATE TABLE books (id SERIAL PRIMARY KEY, titles TEXT[]); INSERT INTO books (titles) VALUES (ARRAY['The Dragon Hatchling', 'Wings of Gold']), (ARRAY['Dragon Slayer', 'Hatchling Care']); CREATE INDEX ON books USING bm25 (id, (titles::pdb.ngram(4,4,'positions=true'))) WITH (key_field='id'); -- Phrase: matches exact substring "Dragon Hatchling" — only row 1 SELECT * FROM books WHERE titles ### 'Dragon Hatchling'; -- Match conjunction: matches all ngrams anywhere — also only row 1 here, -- but on larger datasets could match rows where the ngrams are scattered SELECT * FROM books WHERE titles ||| 'Dragon Hatchling'; DROP TABLE books; ``` -------------------------------- ### Boost a regex query (SQLAlchemy) Source: https://docs.paradedb.com/documentation/sorting/boost Use the `search.regex` function with the `boost` parameter in SQLAlchemy to apply a boost to regex matches. This example boosts 'key.*' by 2.0. ```python from sqlalchemy import desc, select from sqlalchemy.orm import Session from paradedb.sqlalchemy import pdb, search stmt = ( select( MockItem.id, MockItem.description, MockItem.category, pdb.score(MockItem.id).label("score"), ) .where(search.regex(MockItem.description, "key.*", boost=2.0)) .order_by(desc("score")) .limit(5) ) with Session(engine) as session: session.execute(stmt).all() ``` -------------------------------- ### Configure Publisher PostgreSQL Settings Source: https://docs.paradedb.com/deploy/logical-replication/getting-started Ensure these settings are present in `postgresql.conf` on the publisher for logical replication. ```ini listen_addresses = 'localhost,192.168.0.30' wal_level = logical max_replication_slots = 10 max_wal_senders = 10 ``` -------------------------------- ### Run Mock Items Table Migration Source: https://docs.paradedb.com/documentation/getting-started/environment Execute the pending database migrations to apply the changes, including the creation of the 'mock_items' table. ```bash alembic upgrade head ``` -------------------------------- ### Initialize Terraform for AWS Source: https://docs.paradedb.com/deploy/byoc Initializes the Terraform project for AWS infrastructure deployment. Ensure you are in the correct directory. ```bash terraform -chdir=infrastructure/aws init ``` -------------------------------- ### Index and Query with Aliased Tokenizers for Ordering Source: https://docs.paradedb.com/documentation/tokenizers/multiple-per-field Creates an index with an aliased simple tokenizer and demonstrates querying and ordering by the default literal tokenizer. This is recommended when literal tokenization is used alongside other tokenizers. ```SQL CREATE INDEX search_idx ON mock_items USING bm25 ( id, (description::pdb.literal), (description::pdb.simple('alias=description_simple')) ) WITH (key_field='id'); SELECT description, rating, category FROM mock_items WHERE description @@@ 'shoes' ORDER BY description LIMIT 5; ``` -------------------------------- ### Initialize Alembic for Migrations Source: https://docs.paradedb.com/documentation/getting-started/environment Initialize Alembic in your project to manage database schema migrations. ```bash alembic init migrations ``` -------------------------------- ### Configure Drizzle Kit (TypeScript) Source: https://docs.paradedb.com/documentation/getting-started/environment Set up the Drizzle Kit configuration file to point to your database and specify schema and table filters for migrations and introspection. ```typescript import { defineConfig } from "drizzle-kit"; export default defineConfig({ schema: "./db.ts", dialect: "postgresql", // Keep Drizzle Kit focused on `public.mock_items` so it // doesn't try to modify tables from preinstalled extensions. schemaFilter: ["public"], tablesFilter: ["mock_items"], dbCredentials: { url: "postgres://myuser:mypassword@localhost:5432/mydatabase", }, }); ``` -------------------------------- ### Install Drizzle and ParadeDB Dependencies (Bash) Source: https://docs.paradedb.com/documentation/getting-started/environment Install necessary npm packages for Drizzle ORM, postgres.js, and the ParadeDB Drizzle adapter in your TypeScript project. ```bash npm init -y npm pkg set type=module npm install drizzle-orm@1.0.0-rc.2 drizzle-kit@1.0.0-rc.2 postgres @paradedb/drizzle-paradedb@0.1.0 tsx ``` -------------------------------- ### Create a Full Database Dump Source: https://docs.paradedb.com/documentation/getting-started/load Use `pg_dump` with the custom format (-Fc) to create a backup of your entire PostgreSQL database. Ensure the `pg_dump` version is compatible with your database. Replace placeholders with your actual database credentials. ```bash pg_dump -Fc --no-acl --no-owner \ -h \ -U \ > old_db.dump ``` -------------------------------- ### Create Replication User on Publisher Source: https://docs.paradedb.com/deploy/logical-replication/getting-started Create a dedicated user for replication on the publisher PostgreSQL instance. ```bash sudo -u postgres createuser --pwprompt --replication replicator ``` -------------------------------- ### Create Subscription on ParadeDB Source: https://docs.paradedb.com/deploy/logical-replication/getting-started Set up a subscription on the ParadeDB subscriber to connect to the publisher's publication. Use `WITH (copy_data = false)` to skip initial data copy if needed. ```sql CREATE SUBSCRIPTION marketplace_sub CONNECTION 'host=192.168.0.30 port=5432 dbname=marketplace user=replicator password=passw0rd application_name=marketplace_sub' PUBLICATION marketplace_pub; ``` -------------------------------- ### Install pg_search on macOS 14 (Sonoma) Source: https://docs.paradedb.com/deploy/self-hosted/extension Download and install the ParadeDB pg_search extension for macOS 14 (Sonoma). Ensure you use the correct architecture (arm64). ```bash # Available arch version is arm64 curl -L "https://github.com/paradedb/paradedb/releases/download/v0.23.5/pg_search@18--0.23.5.arm64_sonoma.pkg" -o ~/Downloads/pg_search.pkg sudo installer -pkg ~/Downloads/pg_search.pkg -target / ``` -------------------------------- ### Install pg_search on macOS 15 (Sequoia) Source: https://docs.paradedb.com/deploy/self-hosted/extension Download and install the ParadeDB pg_search extension for macOS 15 (Sequoia). Ensure you use the correct architecture (arm64). ```bash # Available arch version is arm64 curl -L "https://github.com/paradedb/paradedb/releases/download/v0.23.5/pg_search@18--0.23.5.arm64_sequoia.pkg" -o ~/Downloads/pg_search.pkg sudo installer -pkg ~/Downloads/pg_search.pkg -target / ``` -------------------------------- ### SQLAlchemy: Top K executed by standard Postgres vs. ParadeDB Source: https://docs.paradedb.com/documentation/query-builder/compound/all Provides SQLAlchemy examples for Top K queries, demonstrating both standard PostgreSQL execution and optimization with ParadeDB's 'all' operator. ```python from sqlalchemy import select from sqlalchemy.orm import Session from paradedb.sqlalchemy import search standard_topn_stmt = ( select(MockItem.id, MockItem.description, MockItem.rating, MockItem.category) .where(MockItem.rating.is_not(None)) .order_by(MockItem.rating) .limit(5) ) paradedb_topn_stmt = ( select(MockItem.id, MockItem.description, MockItem.rating, MockItem.category) .where(MockItem.rating.is_not(None), search.all(MockItem.id)) .order_by(MockItem.rating) .limit(5) ) with Session(engine) as session: { "standard_rows": session.execute(standard_topn_stmt).all(), "paradedb_rows": session.execute(paradedb_topn_stmt).all(), } ``` -------------------------------- ### Install pg_search on Debian 12 Source: https://docs.paradedb.com/deploy/self-hosted/extension Download and install the ParadeDB pg_search extension for Debian 12 (Bookworm). Ensure you use the correct architecture (amd64 or arm64). ```bash # Available arch versions are amd64, arm64 curl -L "https://github.com/paradedb/paradedb/releases/download/v0.23.5/postgresql-18-pg-search_0.23.5-1PARADEDB-bookworm_amd64.deb" -o /tmp/pg_search.deb sudo apt-get install -y /tmp/*.deb ``` -------------------------------- ### Tokenize Text with Simple Tokenizer Source: https://docs.paradedb.com/documentation/tokenizers/available-tokenizers/simple Demonstrates how to use the Simple tokenizer to split a given string into an array of tokens. The output shows the resulting tokens after splitting on non-alphanumeric characters and lowercasing. ```sql SELECT 'Tokenize me!'::pdb.simple::text[]; ``` -------------------------------- ### Install pg_search on Debian 13 Source: https://docs.paradedb.com/deploy/self-hosted/extension Download and install the ParadeDB pg_search extension for Debian 13 (Trixie). Ensure you use the correct architecture (amd64 or arm64). ```bash # Available arch versions are amd64, arm64 curl -L "https://github.com/paradedb/paradedb/releases/download/v0.23.5/postgresql-18-pg-search_0.23.5-1PARADEDB-trixie_amd64.deb" -o /tmp/pg_search.deb sudo apt-get install -y /tmp/*.deb ``` -------------------------------- ### Install pg_search on Ubuntu 22.04 Source: https://docs.paradedb.com/deploy/self-hosted/extension Download and install the ParadeDB pg_search extension for Ubuntu 22.04 (Jammy). Ensure you use the correct architecture (amd64 or arm64). ```bash # Available arch versions are amd64, arm64 curl -L "https://github.com/paradedb/paradedb/releases/download/v0.23.5/postgresql-18-pg-search_0.23.5-1PARADEDB-jammy_amd64.deb" -o /tmp/pg_search.deb sudo apt-get install -y /tmp/*.deb ``` -------------------------------- ### Install pg_search on Ubuntu 24.04 Source: https://docs.paradedb.com/deploy/self-hosted/extension Download and install the ParadeDB pg_search extension for Ubuntu 24.04 (Noble). Ensure you use the correct architecture (amd64 or arm64). ```bash # Available arch versions are amd64, arm64 curl -L "https://github.com/paradedb/paradedb/releases/download/v0.23.5/postgresql-18-pg-search_0.23.5-1PARADEDB-noble_amd64.deb" -o /tmp/pg_search.deb sudo apt-get install -y /tmp/*.deb ``` -------------------------------- ### Table Creation for Aggregate Fallback (NUMERIC) Source: https://docs.paradedb.com/documentation/aggregates/limitations Example of creating a table with a NUMERIC column. Aggregate pushdown is not supported for NUMERIC columns, and queries will fall back to PostgreSQL for aggregation. ```sql -- Aggregates fall back to PostgreSQL CREATE TABLE products ( id SERIAL PRIMARY KEY, price NUMERIC(10,2) ); ``` -------------------------------- ### EXPLAIN Plan for Anti Join Pushdown Source: https://docs.paradedb.com/documentation/joins/overview This EXPLAIN output shows the query plan for an anti join, highlighting the 'ParadeDB Join Scan' which indicates successful join pushdown. It details the relation tree, join condition, and the underlying DataFusion physical plan. ```text QUERY PLAN ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Limit (cost=10.00..11.00 rows=5 width=11) -> Custom Scan (ParadeDB Join Scan) (cost=10.00..11.00 rows=5 width=11) Relation Tree: o ANTI m Join Cond: m.id = o.product_id Limit: 5 Order By: o.order_total desc DataFusion Physical Plan: : ProjectionExec: expr=[NULL as col_1, order_total@1 as col_2, ctid_0@0 as ctid_0] : SortExec: TopK(fetch=5), expr=[order_total@1 DESC], preserve_partitioning=[false] : HashJoinExec: mode=CollectLeft, join_type=RightAnti, on=[(id@0, product_id@1)], projection=[ctid_0@0, order_total@1] : CooperativeExec : PgSearchScan: segments=1, query={"with_index":{"query":{"match":{"field":"description","value":"keyboard","tokenizer":null,"distance":null,"transposition_cost_one":null,"prefix":null,"conjunction_mode":false}}}}) : ProjectionExec: expr=[ctid@0 as ctid_0, product_id@1 as product_id, order_total@2 as order_total] : CooperativeExec : PgSearchScan: segments=1, dynamic_filters=1, query="all" (15 rows) ```