### Install and Start Neon Local Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/005-zenith_local.md Installs the Neon CLI and starts the Neon local service with a single command. ```bash cargo install neon && neon start ``` -------------------------------- ### Start PostgreSQL Instance Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/003-laptop-cli.md Starts a PostgreSQL instance with preloaded extensions. Can be used to start a replica instance with the --replica flag. ```bash neon pg start [--replica] pgdata ``` -------------------------------- ### Start Cloud Compute Instances Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/006-laptop-cli-v2-CLI.md Start compute instances against cloud environments, streaming all changes directly to the cloud. ```bash -- you can start a compute instance against the cloud environment -- in this case all of the changes will be streamed into the cloud neon start https://neon:tecj/pipedpiper/northwind:main ``` ```bash neon start https://neon:tecj/pipedpiper/northwind:main ``` ```bash neon status -c https://neon:tecj/pipedpiper/northwind:main ``` ```bash neon commit -c https://neon:tecj/pipedpiper/northwind:main ``` ```bash neon branch -c https://neon:tecj/pipedpiper/northwind: experimental ``` -------------------------------- ### Start Minio Server Source: https://github.com/neondatabase/neon/blob/main/test_runner/README.md Command to start a local Minio S3-compatible server. This requires creating a directory for Minio data. ```bash # Start in Terminal 1 mkdir /tmp/minio_data minio server /tmp/minio_data --console-address 127.0.0.1:9001 --address 127.0.0.1:9000 ``` -------------------------------- ### Start Neon Compute Node as Replica Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/005-zenith_local.md Starts a Neon compute node, optionally as a replica. ```bash POST -d {"action": "start", "is_replica":"replica"} /pgs/:pg_name /actions ``` -------------------------------- ### Create Test User and Database for Neon Endpoint Source: https://github.com/neondatabase/neon/blob/main/control_plane/README.md These commands create a Neon endpoint with a superuser and necessary roles, then start the endpoint with a test user and database, mimicking cloud service setup. ```shell cargo neon endpoint create main --pg-version 16 --update-catalog true cargo neon endpoint start main --create-test-user true ``` -------------------------------- ### Install Neon and Create a PostgreSQL Instance Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/003-laptop-cli.md Installs the Neon CLI via Homebrew and creates a new PostgreSQL data directory with a default naming pattern. Lists available instances. ```bash > brew install pg-neon > neon pg create # creates pgdata with default pattern pgdata$i > neon pg list ID PGDATA USED STORAGE ENDPOINT primary1 pgdata1 0G neon-local localhost:5432 ``` -------------------------------- ### Start Local Neon Environment with Postgres 16 Source: https://github.com/neondatabase/neon/blob/main/control_plane/README.md Use these commands to initialize and start a local Neon development environment, creating a tenant and endpoint with PostgreSQL 16. ```shell cargo neon init cargo neon start cargo neon tenant create --set-default --pg-version 16 cargo neon endpoint create main --pg-version 16 cargo neon endpoint start main ``` -------------------------------- ### Install PostgreSQL Client on macOS Source: https://github.com/neondatabase/neon/blob/main/README.md Installs the PostgreSQL client library on macOS using Homebrew. ```bash # from https://stackoverflow.com/questions/44654216/correct-way-to-install-psql-without-full-postgres-on-macos brew install libpq brew link --force libpq ``` -------------------------------- ### Start Proxy with SQL over HTTP Source: https://github.com/neondatabase/neon/blob/main/proxy/README.md Starts the proxy with a specified certificate, key, authentication backend, and endpoint. It also enables the WebSocket service. ```bash LOGFMT=text ./target/debug/proxy -c server.crt -k server.key --auth-backend=postgres --auth-endpoint=postgres://stas@127.0.0.1:5432/stas --wss 0.0.0.0:4444 ``` -------------------------------- ### Install Fedora Dependencies Source: https://github.com/neondatabase/neon/blob/main/README.md Installs necessary packages for Fedora systems to build Neon. ```bash dnf install flex bison readline-devel zlib-devel openssl-devel \n libseccomp-devel perl clang cmake postgresql postgresql-contrib protobuf-compiler \n protobuf-devel libcurl-devel openssl poetry lsof libicu-devel libpq-devel python3-devel \n libffi-devel ``` -------------------------------- ### Install Rust using rustup Source: https://github.com/neondatabase/neon/blob/main/README.md Installs Rust using the recommended rustup script. This is a common step for Rust projects. ```bash curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` -------------------------------- ### Initialize Neon Repository Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/007-serverless-on-laptop.md Initializes a new Neon repository and starts the pageserver. This is the first step in both the current and proposed workflows. ```bash $ neon init ... created main branch new neon repository was created in .neon $ neon pageserver start Starting pageserver at '127.0.0.1:64000' in .neon Page server started ``` -------------------------------- ### Install Ubuntu/Debian Dependencies Source: https://github.com/neondatabase/neon/blob/main/README.md Installs necessary build dependencies and packages for Ubuntu or Debian systems. ```bash apt install build-essential libtool libreadline-dev zlib1g-dev flex bison libseccomp-dev \nlibssl-dev clang pkg-config libpq-dev cmake postgresql-client protobuf-compiler \nlibprotobuf-dev libcurl4-openssl-dev openssl python3-poetry lsof libicu-dev ``` -------------------------------- ### Run compute_ctl Source: https://github.com/neondatabase/neon/blob/main/compute_tools/README.md Example of how to run the compute_ctl tool with specified directories and connection strings. ```sh compute_ctl -D /var/db/postgres/compute \ -C 'postgresql://cloud_admin@localhost/postgres' \ -S /var/db/postgres/specs/current.json \ -b /usr/local/bin/postgres ``` -------------------------------- ### Start Local Compute Nodes Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/006-laptop-cli-v2-CLI.md Start compute instances locally, specifying the catalog path, branch, and optionally a port. ```bash -- here is how you a compute node neon start /home/pipedpiper/northwind:main -- starts a compute instance ``` ```bash neon start neon://neon.tech/northwind:main -- starts a compute instance in the cloud ``` ```bash -- you can start a compute node against any hash or branch neon start /home/pipedpiper/northwind:experimental --port 8008 -- start another compute instance (on different port) ``` ```bash -- you can start a compute node against any hash or branch neon start /home/pipedpiper/northwind: --port 8009 -- start another compute instance (on different port) ``` -------------------------------- ### Pageserver TOML Configuration Example Source: https://github.com/neondatabase/neon/blob/main/docs/settings.md This is an example of the initial configuration file created by 'pageserver --init'. It shows default values for basic pageserver settings. ```toml # Initial configuration file created by 'pageserver --init' listen_pg_addr = '127.0.0.1:64000' listen_http_addr = '127.0.0.1:9898' checkpoint_distance = '268435456' # in bytes checkpoint_timeout = '10m' gc_period = '1 hour' gc_horizon = '67108864' max_file_descriptors = '100' # initial superuser role name to use when creating a new tenant initial_superuser_name = 'cloud_admin' broker_endpoint = 'http://127.0.0.1:50051' # [remote_storage] ``` -------------------------------- ### Interact with Neon Database Source: https://github.com/neondatabase/neon/blob/main/docs/docker.md Example SQL commands to create a table, insert data, and select data from the Neon database. ```sql CREATE TABLE t(key int primary key, value text); INSERT into t values(1, 1); select * from t; ``` -------------------------------- ### Install Git LFS and Clone Dataset Source: https://github.com/neondatabase/neon/blob/main/test_runner/performance/pgvector/README.md Use this command to install Git Large File Storage (LFS) and then clone the dataset repository from Hugging Face. ```bash brew install git-lfs git-lfs clone https://huggingface.co/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-large-1536-1M ``` -------------------------------- ### Initialize and Start Neon Environment Source: https://github.com/neondatabase/neon/blob/main/test_runner/README.md Use `neon_env_builder` to configure and initialize a Neon environment with specific settings like the number of safekeepers and authentication methods. The `init_start()` method creates and starts the necessary components. ```python def test_foobar(neon_env_builder: NeonEnvBuilder): # Prescribe the environment. # We want to have 3 safekeeper nodes, and use JWT authentication in the # connections to the page server neon_env_builder.num_safekeepers = 3 neon_env_builder.set_pageserver_auth(True) # Now create the environment. This initializes the repository, and starts # up the page server and the safekeepers env = neon_env_builder.init_start() # Run the test ... ``` -------------------------------- ### Install compiledb Tool Source: https://github.com/neondatabase/neon/blob/main/docs/sourcetree.md Installs the `compiledb` tool as a development dependency using Poetry. This tool is used to generate a compilation database for C code. ```bash poetry add -D compiledb ``` -------------------------------- ### Proxy Incremental Metric Format Example Source: https://github.com/neondatabase/neon/blob/main/docs/consumption_metrics.md Example of an incremental metric format for proxy, detailing outbound traffic per client. Includes metric name, type, start and stop times, idempotency key, value, and endpoint ID. ```json { "metric": "proxy_io_bytes_per_client", "type": "incremental", "start_time": "2022-12-28T11:07:19.317310284Z", "stop_time": "2022-12-28T11:07:19.317310284Z", "idempotency_key": "2022-12-28 11:07:19.317310324 UTC-1-4019", "value": 12345454, "endpoint_id": "5d07d9ce9237c4cd845ea7918c0afa7d", } ``` -------------------------------- ### Compute Startup Sequence Diagram Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/040-Endpoint-Persistent-Unlogged-Files-Storage.md Illustrates the compute startup process, including fetching base backups, critical unlogged persistent files from EPUFS, and extensions from an extension server. ```mermaid sequenceDiagram autonumber participant co as Compute participant ps as Pageserver participant ep as EPUFS participant es as Extension server note over co: Bind endpoint ep-xxx par Get basebackup co->>+ps: Request basebackup @ LSN ps-)ps: Construct basebackup ps->>-co: Receive basebackup TAR @ LSN and Get startup-critical Unlogged Persistent Files co->>+ep: Get all UPFs of endpoint ep-xxx ep-)ep: Retrieve and gather all UPFs ep->>-co: TAR of UPFs and Get startup-critical extensions loop For every startup-critical extension co->>es: Get critical extension es->>co: Receive critical extension end end note over co: Start compute ``` -------------------------------- ### Apply Diesel migrations Source: https://github.com/neondatabase/neon/blob/main/docs/storage_controller.md Apply newly created migrations to the storage controller's database. This command updates the `schema.rs` file automatically. Ensure a database is running, for example, using `cargo neon start`. ```bash DATABASE_URL=... diesel migration run ``` -------------------------------- ### Install Cargo Hakari Source: https://github.com/neondatabase/neon/blob/main/docs/sourcetree.md If cargo hakari is not installed, use this command to install it. ```bash cargo install cargo-hakari ``` -------------------------------- ### Initialize Neon Repository Source: https://github.com/neondatabase/neon/blob/main/README.md Initialize the Neon repository, setting up necessary paths for binaries and data within the `.neon` directory. ```bash > cargo neon init Initializing pageserver node 1 at '127.0.0.1:64000' in ".neon" ``` -------------------------------- ### Install macOS cross-compiler Source: https://github.com/neondatabase/neon/blob/main/compute_tools/README.md Install the macOS cross-compiler toolchain for x86_64-unknown-linux-gnu. ```sh brew tap SergioBenitez/osxct brew install x86_64-unknown-linux-gnu ``` -------------------------------- ### Example Keys Before Migration Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/041-rel-sparse-keyspace.md Illustrates the structure of keys in the system before the sparse keyspace migration process begins. This includes old reldir keys and existing sparse keys. ```plain db1/reldir_key -> (table 1, table 2, table 3) ...db1 rel keys db2/reldir_key -> (table 4, table 5, table 6) ...db2 rel keys sparse_reldir_db2_table7 -> exists sparse_reldir_db1_table8 -> deleted ``` -------------------------------- ### Install Diesel CLI Source: https://github.com/neondatabase/neon/blob/main/docs/storage_controller.md Install the Diesel command-line interface, which is required for managing database migrations. ```bash cargo install diesel_cli ``` -------------------------------- ### Install Arch Linux Dependencies Source: https://github.com/neondatabase/neon/blob/main/README.md Installs required packages for Arch-based systems for Neon development. ```bash pacman -S base-devel readline zlib libseccomp openssl clang \npostgresql-libs cmake postgresql protobuf curl lsof ``` -------------------------------- ### Full Delta File Path Example Source: https://github.com/neondatabase/neon/blob/main/docs/pageserver-storage.md Demonstrates the complete path structure for a delta file within the Neon storage directory. ```text .neon/tenants/941ddc8604413b88b3d208bddf90396c/timelines/4af489b06af8eed9e27a841775616962/rel_1663_13990_2609_0_10_000000000169C348_0000000001702000 ``` -------------------------------- ### Create Default Tenant and Timeline Source: https://github.com/neondatabase/neon/blob/main/README.md Create an initial tenant and set it as the default for subsequent Neon local invocations. This also creates an initial timeline for the tenant. ```bash > cargo neon tenant create --set-default tenant 9ef87a5bf0d92544f6fafeeb3239695c successfully created on the pageserver Created an initial timeline 'de200bd42b49cc1814412c7e592dd6e9' at Lsn 0/16B5A50 for tenant: 9ef87a5bf0d92544f6fafeeb3239695c Setting tenant 9ef87a5bf0d92544f6fafeeb3239695c as a default one ``` -------------------------------- ### Example Metering Event JSON Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/021-metering.md This is an example of a metering event payload that might be generated by a component. ```json { "metric": "remote_storage_size", "type": "absolute", "time": "2022-12-28T11:07:19.317310284Z", "idempotency_key": "2022-12-28 11:07:19.317310324 UTC-1-4019", "value": 12345454, "tenant_id": "5d07d9ce9237c4cd845ea7918c0afa7d", "timeline_id": "a03ebb4f5922a1c56ff7485cc8854143" } ``` -------------------------------- ### Create Branch and Start Postgres Instance Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/007-serverless-on-laptop.md Creates a new branch and initializes a PostgreSQL instance on it. This step is part of the current workflow. ```bash $ neon branch heikki main branching at end of WAL: 0/15ECF68 $ neon pg create heikki Initializing Postgres on timeline 76cf9279915be7797095241638e64644... Extracting base backup to create postgres instance: path=.neon/pgdatadirs/pg1 port=55432 $ neon pg start pg1 Starting postgres node at 'host=127.0.0.1 port=55432 user=heikki' waiting for server to start.... done server started ``` -------------------------------- ### Install macOS Dependencies Source: https://github.com/neondatabase/neon/blob/main/README.md Installs XCode, Homebrew dependencies like protobuf, openssl, and flex for macOS. ```bash xcode-select --install brew install protobuf openssl flex bison icu4c pkg-config m4 # add openssl to PATH, required for ed25519 keys generation in neon_local echo 'export PATH="$(brew --prefix openssl)/bin:$PATH"' >> ~/.zshrc ``` -------------------------------- ### Start PostgreSQL Compute Endpoint Source: https://github.com/neondatabase/neon/blob/main/README.md Start the 'main' PostgreSQL compute node, which runs on a specific timeline. ```bash > cargo neon endpoint start main Starting new endpoint main (PostgreSQL v14) on timeline de200bd42b49cc1814412c7e592dd6e9 ... Starting postgres at 'postgresql://cloud_admin@127.0.0.1:55432/postgres' ``` -------------------------------- ### Branch File Structure Example Source: https://github.com/neondatabase/neon/blob/main/docs/pageserver-storage.md Illustrates the file organization on disk for different branches, showing how table data is stored and versioned. ```text main/orders_100 main/orders_100_200 main/orders_200 main/orders_200_300 main/orders_300 main/orders_300_400 main/orders_400 main/customers_100 main/customers_100_200 main/customers_200 child/orders_250_300 child/orders_300 child/orders_300_400 child/orders_400 ``` -------------------------------- ### Get Compute Status Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/2025-03-17-compute-prewarm.md Extends the existing GET /status API to include prewarm state information. ```APIDOC ## GET /status ### Description Extends the existing GET /status API to include prewarm state information, detailing the status, progress, and any errors related to prewarming. ### Method GET ### Endpoint /status ### Response #### Success Response (200) Returns the compute status including prewarm state. #### Response Example ```json { "existing_attributes": "...", "prewarm_state": { "status": "InProgress", "pages_total": 1000, "pages_processed": 500, "error": null } } ``` ### PrewarmState Details - **status** (PrewarmStatus) - The current status of the prewarming process (Off, Pending, InProgress, Completed, Failed, Skipped). - **pages_total** (integer) - Total number of pages to prewarm. - **pages_processed** (integer) - Number of pages prewarmed so far. - **error** (string, optional) - Optional prewarm error message if the status is Failed. ``` -------------------------------- ### Start PostgreSQL Endpoint on a Branch Source: https://github.com/neondatabase/neon/blob/main/README.md Start the 'migration_check' PostgreSQL compute node, which runs on the specified timeline branch. ```bash > cargo neon endpoint start migration_check Starting new endpoint migration_check (PostgreSQL v14) on timeline b3b863fa45fa9e57e615f9f2d944e601 ... Starting postgres at 'postgresql://cloud_admin@127.0.0.1:55434/postgres' ``` -------------------------------- ### Create a PostgreSQL instance for a tenant Source: https://github.com/neondatabase/neon/blob/main/docs/multitenancy.md Creates a new PostgreSQL instance named 'main' for a specific tenant. If no tenant ID is provided, it defaults to the tenant initialized with `neon_local init`. ```sh neon_local pg create main // default tenant from neon init ``` ```sh neon_local pg create main --tenant_id=ee6016ec31116c1b7c33dfdfca38892f ``` -------------------------------- ### Initialize and Clone Repository Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/006-laptop-cli-v2-CLI.md Initialize a new Neon repository or clone an existing one from a cloud URL. ```bash neon init ``` ```bash neon clone neon://neon.tech/piedpiper/northwind -- clones a repo to the northwind directory ``` -------------------------------- ### Install Python 3.11 on Ubuntu Source: https://github.com/neondatabase/neon/blob/main/docs/sourcetree.md Use this command to install Python 3.11 on Ubuntu systems if you encounter version issues. ```bash sudo add-apt-repository ppa:deadsnakes/ppa sudo apt update sudo apt install python3.11 ``` -------------------------------- ### Example Keys After Migration Image Layer Generation Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/041-rel-sparse-keyspace.md Shows the state of keys after the special image layer generation process for the sparse keyspace. Old reldir keys are emptied, and new sparse keys are created. ```plain db1/reldir_key -> () # we have to keep the key because it is part of `collect_keyspace`. ...db1 rel keys db2/reldir_key -> () ...db2 rel keys -- start image layer for the sparse keyspace at sparse_reldir_prefix at LSN 0/180 sparse_reldir_db1_table1 -> exists sparse_reldir_db1_table2 -> exists sparse_reldir_db1_table3 -> exists sparse_reldir_db2_table4 -> exists sparse_reldir_db2_table5 -> exists sparse_reldir_db2_table6 -> exists sparse_reldir_db2_table7 -> exists -- end image layer for the sparse keyspace at sparse_reldir_prefix+1 # The `sparse_reldir_db1_table8` key gets dropped as part of the image layer generation code for the sparse keyspace. # Note that the read path will stop reading if a key is not found in the image layer covering the key range so there # are no correctness issue. ``` -------------------------------- ### Install m4 on macOS Source: https://github.com/neondatabase/neon/blob/main/README.md Installs and links m4 on macOS if it's missing, often required for certain build processes. ```bash brew install m4 brew link --force m4 ``` -------------------------------- ### Create PostgreSQL Instance from Snapshot (Implicit Import) Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/003-laptop-cli.md Creates a new PostgreSQL instance directly from a base backup URL, implicitly importing the snapshot, starting the instance, and then cleaning up the downloaded snapshot. ```bash > neon pg create --snapshot basebackup://replication@localhost:5432/ Downloading snapshot... Done. Started postgres on localhost:5432 Destroying snapshot... Done. ``` -------------------------------- ### Connect to PostgreSQL Instance Source: https://github.com/neondatabase/neon/blob/main/README.md Connect to a PostgreSQL instance using psql. This example shows connecting to a local instance with specific host, port, and user. ```sql psql -p 55432 -h 127.0.0.1 -U cloud_admin postgres ``` -------------------------------- ### Start PostgreSQL with REST/GraphQL Proxy Source: https://github.com/neondatabase/neon/blob/main/docs/rfcs/003-laptop-cli.md Starts a REST or GraphQL proxy on top of a PostgreSQL master instance. This is an experimental feature. ```bash neon pg start-rest/graphql pgdata ```