### Build and Install Apache Xerces-C from Source Source: https://cloudberry.apache.org/docs/deployment/install-required-packages Extracts the Xerces-C source code, configures the build, compiles the library using parallel processing, runs tests, and installs it to the specified prefix. Log files are created for each step for troubleshooting. A symbolic link is created for the installation directory. ```shell tar xf "xerces-c-${XERCES_LATEST_RELEASE}.tar.gz" rm "xerces-c-${XERCES_LATEST_RELEASE}.tar.gz" cd xerces-c-${XERCES_LATEST_RELEASE} ./configure --prefix="${XERCES_INSTALL_PREFIX}-${XERCES_LATEST_RELEASE}" | tee configure-$(date "+%Y.%m.%d-%H.%M.%S").log make -j$(nproc) | tee make-$(date "+%Y.%m.%d-%H.%M.%S").log make check | tee make-check-$(date "+%Y.%m.%d-%H.%M.%S").log sudo make install | tee make-install-$(date "+%Y.%m.%d-%H.%M.%S").log sudo ln -s ${XERCES_INSTALL_PREFIX}-${XERCES_LATEST_RELEASE} ${XERCES_INSTALL_PREFIX} ``` -------------------------------- ### Clone Apache Cloudberry Bootcamp Repository Source: https://cloudberry.apache.org/docs/deployment/sandbox This command clones the Apache Cloudberry Bootcamp repository from GitHub. This repository contains scripts and configurations necessary for setting up the Cloudberry sandbox environment. Ensure Git is installed on your system. ```bash git clone https://github.com/apache/cloudberry-bootcamp.git ``` -------------------------------- ### Start Database in Multi-Container Deployment Source: https://cloudberry.apache.org/docs/deployment/sandbox Commands to manually start the database within a running multi-container Cloudberry Docker environment. This is necessary after starting the containers if the database was not automatically initiated. ```bash docker exec -it cbdb-cdw /bin/bash [gpadmin@cdw /] gpstart -a ``` -------------------------------- ### SQL: Example Setup for hstore and plpython3u Transform Source: https://cloudberry.apache.org/docs/1.x/sql-stmts/create-transform Provides example SQL commands to set up the necessary extensions (hstore and plpython3u) before creating a transform. This is a prerequisite for establishing custom data type handling between SQL and procedural languages. ```sql CREATE EXTENSION IF NOT EXISTS hstore; CREATE EXTENSION plpython3u; ``` -------------------------------- ### Install cbcopy Tool Source: https://cloudberry.apache.org/docs/next/sys-admin/migration-and-upgrade Steps to clone the cbcopy repository, build the binaries, and install them. Requires Go programming language version 1.19 or higher. ```bash git clone https://github.com/cloudberry-contrib/cbcopy.git cd cbcopy make make install ``` -------------------------------- ### SQL Tablespace Creation Example Source: https://cloudberry.apache.org/docs/next/sys-admin/migration-and-upgrade Example SQL command to manually create a tablespace in the target database. This is an alternative to using cbcopy's automatic global object handling. ```sql CREATE TABLESPACE custom_tablespace LOCATION '/path/to/tablespace'; ``` -------------------------------- ### Start gpmemwatcher Utility Source: https://cloudberry.apache.org/docs/next/sys-utilities/gpmemwatcher Example command to start the gpmemwatcher utility, specifying a host file that lists the servers from which to collect memory usage data. ```bash $ gpmemwatcher -f /home/gpadmin/hostmap.txt ``` -------------------------------- ### Example gpmovemirrors Input File Source: https://cloudberry.apache.org/docs/next/tutorials/best-practices/high-availability-best-practices An example of an input file for the `gpmovemirrors` utility, demonstrating how to specify three mirror segments for relocation. This file guides the `gpmovemirrors` command to move mirrors to their new block mirroring locations. ```text sdw2|50001|/data2/mirror/gpseg1 sdw3|50001|/data/mirror/gpseg1 sdw2|50001|/data2/mirror/gpseg2 sdw4|50001|/data/mirror/gpseg2 sdw3|50001|/data2/mirror/gpseg3 sdw1|50001|/data/mirror/gpseg3 ``` -------------------------------- ### Start Single Container Deployment with Docker Source: https://cloudberry.apache.org/docs/deployment/sandbox Command to start a previously stopped single Docker container deployment. This command assumes the Docker volume was persisted, allowing the container to resume with its previous state. ```bash docker start cbdb-cdw ``` -------------------------------- ### gpsync Command Example for Copying Installer Source: https://cloudberry.apache.org/docs/1.x/sys-utilities/gpsync This example illustrates copying an 'installer.tar' file to the root directory ('/') on all hosts specified in 'hostfile_gpssh'. The '=' acts as a substitution for each hostname in the file. ```bash gpsync -f hostfile_gpssh installer.tar =:/ ``` -------------------------------- ### Initialize and Start PXF Service Source: https://cloudberry.apache.org/docs/data-loading/load-data-using-pxf Commands to prepare and start the PXF service. `pxf prepare` initializes the configuration structure, and `pxf start` launches the Java process responsible for handling external data requests on each segment host. ```bash pxf prepare ``` ```bash pxf start ``` -------------------------------- ### Install Basic System Packages for Cloudberry Development Source: https://cloudberry.apache.org/docs/deployment/quick-build This command installs a comprehensive set of essential development packages required for building and running Apache Cloudberry, including compilers, libraries, and utilities. It uses `apt install -y` to install multiple packages. ```shell sudo apt install -y bison \ bzip2 \ cmake \ curl \ flex \ gcc \ g++ \ iproute2 \ iputils-ping \ language-pack-en \ locales \ libapr1-dev \ libbz2-dev \ libcurl4-gnutls-dev \ libevent-dev \ libkrb5-dev \ libipc-run-perl \ libldap2-dev \ libpam0g-dev \ libprotobuf-dev \ libreadline-dev \ libssl-dev \ libuv1-dev \ liblz4-dev \ libxerces-c-dev \ libxml2-dev \ libyaml-dev \ libzstd-dev \ libperl-dev \ make \ pkg-config \ protobuf-compiler \ python3-dev \ python3-pip \ python3-setuptools \ rsync ``` -------------------------------- ### Interpreting installcheck Test Output Source: https://cloudberry.apache.org/docs/deployment/post-installation Illustrates sample output from the Cloudberry installcheck test suite, showing individual test results (ok/FAILED) and a summary of passed and failed tests. It also points to files containing detailed failure information. ```text test tablespace ... ok 3236 ms (diff 76 ms) parallel group (20 tests): pg_lsn oid txid name char varchar int2 regproc text ... boolean ... ok 862 ms (diff 56 ms) char ... ok 419 ms (diff 87 ms) explain ... FAILED 310 ms (diff 139 ms) ======================== 1 of 658 tests failed. ======================== If any tests fail: * `regression.diffs`: Shows differences between actual and expected results * `regression.out`: Contains the complete test execution output The files will be located in `/home/gpadmin/cloudberry/src/test/regress/`. ``` -------------------------------- ### Example: Create a Simple Database - SQL Source: https://cloudberry.apache.org/docs/next/sql-stmts/create-database An example demonstrating the basic usage of the `CREATE DATABASE` command to create a new database named 'mydatabase'. ```sql CREATE DATABASE mydatabase; ``` -------------------------------- ### Example Installcheck Test Output (Text) Source: https://cloudberry.apache.org/docs/next/deployment/post-installation Illustrates the typical output format of the Cloudberry installcheck test suite, showing individual test results (pass/fail) and execution times. It also provides an example of the final summary indicating the number of failed tests. ```text test tablespace ... ok 3236 ms (diff 76 ms) parallel group (20 tests): pg_lsn oid txid name char varchar int2 regproc text ... boolean ... ok 862 ms (diff 56 ms) char ... ok 419 ms (diff 87 ms) explain ... FAILED 310 ms (diff 139 ms) ======================== 1 of 658 tests failed. ======================== ``` -------------------------------- ### Install gpbackup and gprestore Utilities with Go Source: https://cloudberry.apache.org/docs/1.x/sys-admin/backup-and-restore/perform-full-backup-and-restore Installs the `gpbackup` and `gprestore` utilities using the Go programming language. This command fetches the latest version of the specified GitHub repository and installs it. Ensure Golang (v1.11 or later) is installed and the Go PATH environment variable is set. ```bash go install github.com/apache/cloudberry-gpbackup@latest ``` -------------------------------- ### Drop and Create PostgreSQL Database using Utilities Source: https://cloudberry.apache.org/docs/next/tutorials/quick-trial-lessons/create-and-prepare-database Demonstrates how to drop an existing database using `dropdb` and create a new one with `createdb`. Includes verification steps using `psql -l` and explains database copying behavior from `template1`. Useful for database initialization and cleanup. ```bash [gpadmin@cdw ~]$ dropdb tutorial Output: ``` dropdb: error: database removal failed: ERROR: database "tutorial" does not exist ``` [gpadmin@cdw ~]$ createdb tutorial [gpadmin@cdw ~]$ psql -l # Verifies that this database has been created. List of databases Name | Owner | Encoding | Collate | Ctype | Access privileges -----------+---------+----------+-------------+-------------+--------------------- gpadmin | gpadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | postgres | gpadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | template0 | gpadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gpadmin + | | | | | gpadmin=CTc/gpadmin template1 | gpadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gpadmin + | | | | | gpadmin=CTc/gpadmin tutorial | gpadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | (5 rows) ``` -------------------------------- ### Install GCC 11+ for Cloudberry with PAX Support (Rocky Linux 8) Source: https://cloudberry.apache.org/docs/deployment/install-required-packages Installs a higher version of GCC and G++ (version 11+) on Rocky Linux 8 to enable PAX support during Cloudberry builds. It includes commands for temporary and permanent environment setup. ```shell sudo yum install -y gcc-toolset-11-gcc gcc-toolset-11-gcc-c++ scl enable gcc-toolset-11 bash # for temprory use sudo echo "source /opt/rh/gcc-toolset-11/enable" >> /etc/profile.d/gcc.sh sudo source /etc/profile.d/gcc.sh # for permanent use ``` ```shell gcc --version ``` ```shell g++ --version ``` -------------------------------- ### Set Up and Initialize Cloudberry Demo Cluster Source: https://cloudberry.apache.org/docs/deployment/quick-build This section covers the steps to set up a demo cluster for Cloudberry. It involves sourcing environment scripts and using `make create-demo-cluster` followed by initializing the demo environment and checking cluster configuration and version via `psql`. ```shell source /usr/local/cloudberry-db/greenplum_path.sh make create-demo-cluster -C ~/cloudberry source ~/cloudberry/gpAux/gpdemo/gpdemo-env.sh psql -P pager=off template1 -c 'SELECT * from gp_segment_configuration' psql template1 -c 'SELECT version()' ``` -------------------------------- ### Start Apache Cloudberry Cluster Source: https://cloudberry.apache.org/docs/1.x/cbdb-macos-compile This command is used to start the Apache Cloudberry cluster after installation. It should be executed from the root directory of the cloned source code. ```makefile # In the folder where your cloned the source code make installcheck-world ``` -------------------------------- ### Display gpinitsystem Help Information Source: https://cloudberry.apache.org/docs/sys-utilities/gpinitsystem Displays the help information for the gpinitsystem utility, outlining all available options and their usage. This is useful for understanding the full capabilities of the command. ```bash gpinitsystem -? | --help ``` -------------------------------- ### Install Sudo and Git via APT Source: https://cloudberry.apache.org/docs/deployment/quick-build This command installs the `sudo` and `git` packages using the APT package manager, ensuring these essential tools are available on the system. It updates the package list before installation. ```shell apt update && apt install -y sudo git ``` -------------------------------- ### gpstart Utility Synopsis and Help Source: https://cloudberry.apache.org/docs/next/sys-utilities/gpstart Displays the command-line options for starting an Apache Cloudberry system. This includes parameters for data directory, parallel processes, logging, and help information. It's the primary interface for initiating the cluster startup. ```bash gpstart [-d ] [-B ] [-R] [-m] [-y] [-a] [-t ] [-l ] [--skip-heap-checksum-validation] [-v | -q] gpstart -? | -h | --help gpstart --version ``` -------------------------------- ### gpmemreport: Extract Data with Start Time Example Source: https://cloudberry.apache.org/docs/1.x/sys-utilities/gpmemreport This example illustrates how to use the `gpmemreport` utility with a specified start time to filter the data extracted from a `gpmemwatcher` output file. It shows the command syntax using the `--start` option and verifies the output by listing the generated files, which now only include those from the specified timestamp onwards. ```bash $ gpmemreport cdw.ps.out.gz --start='2021-11-19 15:38:00' >>>21:11:19:15:37:18<<< >>>21:11:19:15:38:18<<< >>>21:11:19:15:39:18<<< ``` ```bash $ ls -thrl -rw-rw-r--. 1 gpadmin gpadmin 1.2K Nov 19 15:50 20211119-153818 -rw-rw-r--. 1 gpadmin gpadmin 1.2K Nov 19 15:50 20211119-153918 ``` -------------------------------- ### Start Apache Cloudberry System Source: https://cloudberry.apache.org/docs/sys-utilities/gpstart Starts the Apache Cloudberry system. This is the basic command to bring the system online. No specific dependencies or inputs are required beyond executing the command. ```bash gpstart ``` -------------------------------- ### gpstart - Maintenance Mode Example Source: https://cloudberry.apache.org/docs/sys-utilities/gpstart The '-m' option for gpstart allows starting only the coordinator instance for maintenance tasks. Connections are limited to utility mode in this configuration. ```bash PGOPTIONS='-c gp_role=utility' psql ``` -------------------------------- ### Start Multi-Container Deployment with Docker Compose Source: https://cloudberry.apache.org/docs/deployment/sandbox Command to start a previously stopped multi-container Docker deployment. This command restarts the containers defined in the docker-compose file. ```bash docker compose -f docker-compose-rockylinux9.yml start ``` -------------------------------- ### Set Up Cloudberry Demo Cluster Source: https://cloudberry.apache.org/docs/next/deployment/quick-build This snippet sets up a demo cluster for Cloudberry. It sources environment scripts and uses psql to query segment configuration and version information. ```bash source /usr/local/cloudberry-db/cloudberry-env.sh make create-demo-cluster -C ~/cloudberry source ~/cloudberry/gpAux/gpdemo/gpdemo-env.sh psql -P pager=off template1 -c 'SELECT * from gp_segment_configuration' psql template1 -c 'SELECT version()' ``` -------------------------------- ### Set Apache Xerces-C Installation Variables Source: https://cloudberry.apache.org/docs/deployment/install-required-packages Defines environment variables for specifying the Xerces-C release version and the installation prefix. These variables streamline commands and ensure consistency throughout the build process. ```shell XERCES_LATEST_RELEASE=3.3.0 XERCES_INSTALL_PREFIX="/usr/local/xerces-c" ``` -------------------------------- ### SQL: Example of Creating a Foreign Server Source: https://cloudberry.apache.org/docs/1.x/sql-stmts/create-server An example demonstrating how to create a foreign server named 'myserver' using the 'gpfdw1' foreign-data wrapper. This example includes specifying connection options such as host, database name, and port. ```sql CREATE SERVER myserver FOREIGN DATA WRAPPER gpfdw1 OPTIONS (host 'foo', dbname 'foodb', port '5432'); ``` -------------------------------- ### Install Apache Cloudberry Dependencies on Ubuntu Source: https://cloudberry.apache.org/docs/deployment/install-required-packages Installs a comprehensive list of development packages required for building Apache Cloudberry on Ubuntu systems. This includes compilers, build tools, libraries, and development headers. ```shell sudo apt install -y bison \ bzip2 \ cmake \ curl \ flex \ gcc \ g++ \ iproute2 \ iputils-ping \ language-pack-en \ locales \ libapr1-dev \ libbz2-dev \ libcurl4-gnutls-dev \ libevent-dev \ libkrb5-dev \ libipc-run-perl \ libldap2-dev \ libpam0g-dev \ libprotobuf-dev \ libreadline-dev \ libssl-dev \ libuv1-dev \ liblz4-dev \ libxerces-c-dev \ libxml2-dev \ libyaml-dev \ libzstd-dev \ libperl-dev \ make \ pkg-config \ protobuf-compiler \ python3-dev \ python3-pip \ python3-setuptools \ rsync ``` -------------------------------- ### Initialize Apache Cloudberry System with gpinitsystem Source: https://cloudberry.apache.org/docs/1.x/sys-utilities/gpinitsystem Initializes an Apache Cloudberry system using a configuration file and optional command-line parameters. It verifies configuration, establishes host connections, initializes coordinator and segment instances, and configures the system. Requires Cloudberry software installation and SSH key exchange between hosts. ```bash gpinitsystem -c \ [-h ] \ [-B ] \ [-p ] \ [-s \ [-P ] \ [-S | --standby_datadir=]] \ [-m | --max_connections=number>] \ [-b | --shared_buffers=] \ [-n | --locale=] [--lc-collate=] \ [--lc-ctype=] [--lc-messages=] \ [--lc-monetary=] [--lc-numeric=] \ [--lc-time=] [-e | --su_password=] \ [--mirror-mode={group|spread}] [-a] [-q] [-l ] [-D] \ [-I ] \ [-O ] ``` ```bash gpinitsystem -v | --version ``` ```bash gpinitsystem -? | --help ``` -------------------------------- ### gpreload Utility Example Command Source: https://cloudberry.apache.org/docs/1.x/sys-utilities/gpreload Demonstrates a practical example of how to run the gpreload utility to reload tables specified in a file. ```bash gpreload -d mytest --table-file data-tables.txt ``` -------------------------------- ### Get Help for gpinitstandby Utility Source: https://cloudberry.apache.org/docs/sys-utilities/gpinitstandby Displays the help message for the gpinitstandby utility, outlining all available options and their usage. ```bash gpinitstandby -? ``` -------------------------------- ### Install Basic System Packages for Cloudberry (Rocky Linux/Ubuntu) Source: https://cloudberry.apache.org/docs/deployment/install-required-packages Installs a comprehensive set of core development packages required for Apache Cloudberry. This command is applicable to Rocky Linux 8+ and Ubuntu 20.04+. ```shell sudo dnf install -y apr-devel \ bison \ bzip2-devel \ curl \ cmake3 \ diffutils \ flex \ gcc \ gcc-c++ \ glibc-langpack-en \ glibc-locale-source \ iproute \ krb5-devel \ libcurl-devel \ libevent-devel \ libxml2-devel \ libuuid-devel \ libzstd-devel \ lz4-devel \ net-tools \ openldap-devel \ openssl-devel \ openssh-server \ pam-devel \ perl \ perl-ExtUtils-Embed \ perl-Test-Simple \ perl-Env \ python3-devel \ python3-pip \ readline-devel \ rsync \ wget \ which \ zlib-devel ``` -------------------------------- ### Start Apache Cloudberry System Source: https://cloudberry.apache.org/docs/1.x/sys-utilities/gpstart This command starts a basic Apache Cloudberry system. It is the default command with no arguments. ```bash gpstart ``` -------------------------------- ### CREATE RESOURCE GROUP Examples Source: https://cloudberry.apache.org/docs/next/sql-stmts/create-resource-group Examples demonstrating how to create resource groups with different configurations. ```APIDOC ## CREATE RESOURCE GROUP Examples ### Example 1: CPU and Memory Limits Create a resource group with CPU and memory limit percentages of 35: ```sql CREATE RESOURCE GROUP rgroup1 WITH (CPU_MAX_PERCENT=35, MEMORY_LIMIT=35); ``` ### Example 2: Concurrency, Memory, and CPU Limits Create a resource group with a concurrent transaction limit of 20, a memory limit of 15, and a CPU limit of 25: ```sql CREATE RESOURCE GROUP rgroup2 WITH (CONCURRENCY=20, MEMORY_LIMIT=15, CPU_MAX_PERCENT=25); ``` ### Example 3: PL/Container Resources Create a resource group to manage PL/Container resources specifying a memory limit of 10, and a CPU limit of 10: ```sql CREATE RESOURCE GROUP plc_run1 WITH (MEMORY_LIMIT=10, CPU_MAX_PERCENT=10, CONCURRENCY=0); ``` ### Example 4: CPUSET and Memory Limit Create a resource group with a memory limit percentage of 11 to which you assign CPU core 1 on the coordinator host, and cores 1 to 3 on segment hosts: ```sql CREATE RESOURCE GROUP rgroup3 WITH (CPUSET='1;1-3', MEMORY_LIMIT=11); ``` ``` -------------------------------- ### Start Cloudberry Cluster with gpstart Source: https://cloudberry.apache.org/docs/tutorials/crash-course This snippet demonstrates how to start a Cloudberry cluster using the `gpstart -a` command. It includes the command itself and the typical output logs indicating a successful startup process, including segment instance startup and coordinator activation. Ensure the environment is correctly configured before execution. ```bash [gpadmin@cdw ~]$ gpstart -a ``` ```text 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Starting gpstart with args: -a 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Gathering information and validating the environment... 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Cloudberry Binary Version: 'postgres (Apache Cloudberry) 1.0.0 build dev' 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Cloudberry Catalog Version: '302206171' 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Starting Coordinator instance in admin mode 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-CoordinatorStart pg_ctl cmd is env GPSESSID=0000000000 GPERA=None $GPHOME/bin/pg_ctl -D /data0/database/coordinator/gpseg-1 -l /data0/database/coordinator/gpseg-1/log/startup.log -w -t 600 -o " -p 5432 -c gp_role=utility " start 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Obtaining Cloudberry Coordinator catalog information 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Obtaining Segment details from coordinator... 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Setting new coordinator era 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Coordinator Started... 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Shutting down coordinator 20230823:16:14:23:004256 gpstart:cdw:gpadmin-[INFO]:-Commencing parallel primary and mirror segment instance startup, please wait... 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:-Process results... 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:----------------------------------------------------- 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:- Successful segment starts = 4 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:- Failed segment starts = 0 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:- Skipped segment starts (segments are marked down in configuration) = 0 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:----------------------------------------------------- 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:-Successfully started 4 of 4 segment instances 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:----------------------------------------------------- 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:-Starting Coordinator instance cdw directory /data0/database/coordinator/gpseg-1 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:-CoordinatorStart pg_ctl cmd is env GPSESSID=0000000000 GPERA=45b5ca734de32094_230823161423 $GPHOME/bin/pg_ctl -D /data0/database/coordinator/gpseg-1 -l /data0/database/coordinator/gpseg-1/log/startup.log -w -t 600 -o " -p 5432 -c gp_role=dispatch " start 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:-Command pg_ctl reports Coordinator cdw instance active 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:-Connecting to db template1 on host localhost 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:-No standby coordinator configured. skipping... 20230823:16:14:24:004256 gpstart:cdw:gpadmin-[INFO]:-Database successfully started ``` -------------------------------- ### Build and Install Xerces-C Source: https://cloudberry.apache.org/docs/next/deployment/quick-build Downloads, compiles, and installs the Xerces-C XML parser library. This process involves fetching the source code, verifying its integrity, configuring the build, compiling, and installing it to a specified prefix. ```shell XERCES_LATEST_RELEASE=3.3.0 XERCES_INSTALL_PREFIX="/usr/local/xerces-c" wget -nv "https://dlcdn.apache.org//xerces/c/3/sources/xerces-c-${XERCES_LATEST_RELEASE}.tar.gz" echo "$(curl -sL https://dlcdn.apache.org//xerces/c/3/sources/xerces-c-${XERCES_LATEST_RELEASE}.tar.gz.sha256)" | sha256sum -c - tar xf "xerces-c-${XERCES_LATEST_RELEASE}.tar.gz" rm "xerces-c-${XERCES_LATEST_RELEASE}.tar.gz" cd xerces-c-${XERCES_LATEST_RELEASE} ./configure --prefix="${XERCES_INSTALL_PREFIX}-${XERCES_LATEST_RELEASE}" make -j$(nproc) make check sudo make install sudo ln -s ${XERCES_INSTALL_PREFIX}-${XERCES_LATEST_RELEASE} ${XERCES_INSTALL_PREFIX} ``` -------------------------------- ### Verify Apache Cloudberry Installation: Core Utilities Source: https://cloudberry.apache.org/docs/next/deployment/build-and-install Command to list the core utility binaries installed for Apache Cloudberry. Uses 'ls -l' to show details of files in the bin directory, expecting binaries like postgres and initdb. ```shell ls -l /usr/local/cloudberry-db/bin/ ``` -------------------------------- ### Start Apache Cloudberry Source: https://cloudberry.apache.org/docs/1.x/start-and-stop-cbdb-database Starts an initialized Apache Cloudberry system using the `gpstart` utility. This command initiates all `postgres` instances across the cluster in parallel. Ensure the system has been previously initialized by `gpinitsystem` and stopped by `gpstop`. ```bash $ gpstart ``` -------------------------------- ### Configure gpinitsystem_config for Initialization Source: https://cloudberry.apache.org/docs/next/cbdb-op-deploy-guide Displays the content of the gpinitsystem_config file, showing required and optional parameters for initializing the Greenplum system. Key parameters include segment data directories, coordinator hostname and directory, and mirror data directories. The example demonstrates a basic configuration. ```bash # FILE NAME: gpinitsystem_config # Configuration file needed by the gpinitsystem ######################################## #### REQUIRED PARAMETERS ######################################## #### Naming convention for utility-generated data directories. SEG_PREFIX=gpseg #### Base number by which primary segment port numbers #### are calculated. PORT_BASE=6000 #### File system location(s) where primary segment data directories #### will be created. The number of locations in the list dictate #### the number of primary segments that will get created per #### physical host (if multiple addresses for a host are listed in #### the hostfile, the number of segments will be spread evenly across #### the specified interface addresses). declare -a DATA_DIRECTORY=(/data0/primary) #### OS-configured hostname or IP address of the coordinator host. COORDINATOR_HOSTNAME=cbdb-coordinator #### File system location where the coordinator data directory #### will be created. COORDINATOR_DIRECTORY=/data0/coordinator #### Port number for the coordinator instance. COORDINATOR_PORT=5432 #### Shell utility used to connect to remote hosts. TRUSTED_SHELL=ssh #### Default server-side character set encoding. ENCODING=UNICODE ######################################## #### OPTIONAL MIRROR PARAMETERS ######################################## #### Base number by which mirror segment port numbers #### are calculated. MIRROR_PORT_BASE=7000 #### File system location(s) where mirror segment data directories #### will be created. The number of mirror locations must equal the #### number of primary locations as specified in the #### DATA_DIRECTORY parameter. declare -a MIRROR_DATA_DIRECTORY=(/data0/mirror) ######################################## #### OTHER OPTIONAL PARAMETERS ######################################## ``` -------------------------------- ### Build and Install Xerces-C Source: https://cloudberry.apache.org/docs/deployment/quick-build Downloads, compiles, and installs the Xerces-C XML parser library. It verifies the downloaded archive integrity using SHA256 checksum and creates a symbolic link for the installed version. ```bash # Build Xerces-C source code XERCES_LATEST_RELEASE=3.3.0 XERCES_INSTALL_PREFIX="/usr/local/xerces-c" wget -nv "https://dlcdn.apache.org//xerces/c/3/sources/xerces-c-${XERCES_LATEST_RELEASE}.tar.gz" echo "$(curl -sL https://dlcdn.apache.org//xerces/c/3/sources/xerces-c-${XERCES_LATEST_RELEASE}.tar.gz.sha256)" | sha256sum -c - tar xf "xerces-c-${XERCES_LATEST_RELEASE}.tar.gz" rm "xerces-c-${XERCES_LATEST_RELEASE}.tar.gz" cd xerces-c-${XERCES_LATEST_RELEASE} ./configure --prefix="${XERCES_INSTALL_PREFIX}-${XERCES_LATEST_RELEASE}" make -j$(nproc) make check sudo make install sudo ln -s ${XERCES_INSTALL_PREFIX}-${XERCES_LATEST_RELEASE} ${XERCES_INSTALL_PREFIX} ``` -------------------------------- ### CREATE RESOURCE GROUP Examples Source: https://cloudberry.apache.org/docs/sql-stmts/create-resource-group Examples demonstrating how to create resource groups with various configurations. ```APIDOC ## CREATE RESOURCE GROUP Examples This section provides examples of creating resource groups with different configurations. ### Example 1: CPU and Memory Limit Percentages Create a resource group with CPU and memory limit percentages of 35: ```sql CREATE RESOURCE GROUP rgroup1 WITH (CPU_MAX_PERCENT=35, MEMORY_LIMIT=35); ``` ### Example 2: Concurrency, Memory, and CPU Limits Create a resource group with a concurrent transaction limit of 20, a memory limit of 15, and a CPU limit of 25: ```sql CREATE RESOURCE GROUP rgroup2 WITH (CONCURRENCY=20, MEMORY_LIMIT=15, CPU_MAX_PERCENT=25); ``` ### Example 3: PL/Container Resources Create a resource group to manage PL/Container resources specifying a memory limit of 10, and a CPU limit of 10: ```sql CREATE RESOURCE GROUP plc_run1 WITH (MEMORY_LIMIT=10, CPU_MAX_PERCENT=10, CONCURRENCY=0); ``` ### Example 4: CPUSET and Memory Limit Create a resource group with a memory limit percentage of 11 to which you assign CPU core 1 on the coordinator host, and cores 1 to 3 on segment hosts: ```sql CREATE RESOURCE GROUP rgroup3 WITH (CPUSET='1;1-3', MEMORY_LIMIT=11); ``` ``` -------------------------------- ### Start Cloudberry Coordinator Only in Utility Mode Source: https://cloudberry.apache.org/docs/1.x/sys-utilities/gpstart This example demonstrates starting only the Cloudberry coordinator instance and connecting to it in utility mode. This is often used for maintenance tasks where segment instances are not required. ```bash gpstart -m PGOPTIONS='-c gp_role=utility' psql ``` -------------------------------- ### Initialize Apache Cloudberry System using Configuration File Source: https://cloudberry.apache.org/docs/sys-utilities/gpinitsystem Initializes an Apache Cloudberry system using a specified configuration file and optional command-line parameters. This utility verifies parameters, establishes host connections, initializes coordinator, standby, primary segment, and mirror segment instances, and configures the system. ```bash gpinitsystem -c \ [-h ] \ [-B ] \ [-p ] \ [-s \ [-P ] \ [-S | --standby_datadir=]] \ [-m | --max_connections=number>] \ [-b | --shared_buffers=] \ [-n | --locale=] [--lc-collate=] \ [--lc-ctype=] [--lc-messages=] \ [--lc-monetary=] [--lc-numeric=] \ [--lc-time=] [-e | --su_password=] \ [--mirror-mode={group|spread}] [-a] [-q] [-l ] [-D] \ [-I ] \ [-O ] ``` -------------------------------- ### Connect to Apache Database using psql Source: https://cloudberry.apache.org/docs/tutorials/quick-trial-lessons/data-loading This snippet shows the command to connect to the 'tutorial' database using the 'lily' user with the psql client. It is the initial step before executing data loading commands. ```bash [gpadmin@cdw faa]$ psql -U lily -d tutorial ``` -------------------------------- ### Start Apache Cloudberry Source: https://cloudberry.apache.org/docs/next/start-and-stop-cbdb-database Starts an initialized Apache Cloudberry system. This utility starts all postgres instances in the cluster in parallel. ```bash $ gpstart ``` -------------------------------- ### gpinitsystem_config File Example Source: https://cloudberry.apache.org/docs/cbdb-op-deploy-guide An example of the gpinitsystem_config file, showing required and optional parameters for initializing a Greenplum system. Key parameters include data directories, hostnames, and ports. ```ini # FILE NAME: gpinitsystem_config # Configuration file needed by the gpinitsystem ######################################## #### REQUIRED PARAMETERS ######################################## #### Naming convention for utility-generated data directories. SEG_PREFIX=gpseg #### Base number by which primary segment port numbers #### are calculated. PORT_BASE=6000 #### File system location(s) where primary segment data directories #### will be created. The number of locations in the list dictate #### the number of primary segments that will get created per #### physical host (if multiple addresses for a host are listed in #### the hostfile, the number of segments will be spread evenly across #### the specified interface addresses). declare -a DATA_DIRECTORY=(/data0/primary) #### OS-configured hostname or IP address of the coordinator host. COORDINATOR_HOSTNAME=cbdb-coordinator #### File system location where the coordinator data directory #### will be created. COORDINATOR_DIRECTORY=/data0/coordinator #### Port number for the coordinator instance. COORDINATOR_PORT=5432 #### Shell utility used to connect to remote hosts. TRUSTED_SHELL=ssh #### Default server-side character set encoding. ENCODING=UNICODE ######################################## #### OPTIONAL MIRROR PARAMETERS ######################################## #### Base number by which mirror segment port numbers #### are calculated. MIRROR_PORT_BASE=7000 #### File system location(s) where mirror segment data directories #### will be created. The number of mirror locations must equal the #### number of primary locations as specified in the #### DATA_DIRECTORY parameter. declare -a MIRROR_DATA_DIRECTORY=(/data0/mirror) ``` -------------------------------- ### Example: Cancel Process with Message Source: https://cloudberry.apache.org/docs/next/start-and-stop-cbdb-database This example demonstrates how to cancel a specific PostgreSQL backend process (PID 31905) and includes a custom message that will be returned to the client. This function is used when a query needs to be stopped gracefully. ```sql =# SELECT pg_cancel_backend(31905 ,'Admin canceled long-running query.'); ``` -------------------------------- ### Verify gpbackup and gprestore Installation Source: https://cloudberry.apache.org/docs/1.x/sys-admin/backup-and-restore/perform-full-backup-and-restore Checks the successful installation of `gpbackup` and `gprestore` utilities. This involves listing the contents of the binary directory and then running the version command for each utility to confirm they are accessible and functional. ```bash ls $HOME/go/bin gpbackup --version gprestore --version ``` -------------------------------- ### SQL/JSON Regex and String Matching (like_regex, starts with) Source: https://cloudberry.apache.org/docs/functions/json-functions-and-operators Explains and provides examples for using `like_regex` with flags and the `starts with` operator in SQL/JSON filter expressions for pattern matching and substring checks within JSON string values. ```sql SELECT json_query(column_name, '$[*] ? (@ like_regex "^ab.*c" flag "i")') FROM your_table; -- Result: "abc", "aBdC", "abdacb" SELECT json_query(column_name, '$[*] ? (@ starts with "John")') FROM your_table; -- Result: "John Smith" ``` -------------------------------- ### Example: Create Database - Apache Cloudberry SQL Source: https://cloudberry.apache.org/docs/1.x/sql-stmts/create-database Provides an example of how to create a basic database named 'mydatabase' using the CREATE DATABASE command in Apache Cloudberry. This is a fundamental operation for setting up new data storage. ```sql CREATE DATABASE mydatabase; ``` -------------------------------- ### Enable Additional Development Repositories for Cloudberry (Rocky Linux) Source: https://cloudberry.apache.org/docs/deployment/install-required-packages Enables the 'devel' or 'crb' repositories on Rocky Linux to install extra development tools and libraries needed for Cloudberry. These repositories are often disabled by default. ```shell sudo dnf install -y --enablerepo=devel libuv-devel libyaml-devel perl-IPC-Run protobuf-devel ``` ```shell sudo dnf install -y --enablerepo=crb libuv-devel libyaml-devel perl-IPC-Run protobuf-devel ``` -------------------------------- ### Install cbcopy Binaries Source: https://cloudberry.apache.org/docs/sys-admin/migration-and-upgrade Installs the `cbcopy` and `cbcopy_helper` binaries into the `$GPHOME/bin` directory. This command should be run after successfully building the binaries. ```bash make install ``` -------------------------------- ### Connect to PostgreSQL Database Source: https://cloudberry.apache.org/docs/next/tutorials/quick-trial-lessons/create-and-prepare-database Establishes a connection to the 'tutorial' PostgreSQL database using the 'psql' command-line utility as the user 'lily'. Requires password authentication. ```bash [gpadmin@cdw ~]$ psql -U lily tutorial ``` -------------------------------- ### Validate Cloudberry Installation with installcheck Source: https://cloudberry.apache.org/docs/deployment/post-installation Runs the installcheck test suite to verify basic Cloudberry functionalities. It demonstrates how to execute tests with the Orca query optimizer enabled and disabled by setting the PGOPTIONS environment variable. ```shell # Run tests with Orca optimizer enabled PGOPTIONS='-c optimizer=on' make --directory=~/cloudberry installcheck # Run tests with Orca optimizer disabled PGOPTIONS='-c optimizer=off' make --directory=~/cloudberry installcheck ``` -------------------------------- ### List Databases Command - Bash Source: https://cloudberry.apache.org/docs/tutorials/quick-trial-lessons/create-and-prepare-database Lists all available databases on the PostgreSQL server. The 'psql -l' command is used to connect to the server and query the system catalog for database information. ```bash [gpadmin@cdw ~]$ psql -l # Verifies that this database has been created. List of databases Name | Owner | Encoding | Collate | Ctype | Access privileges -----------+---------+----------+-------------+-------------+--------------------- gpadmin | gpadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | postgres | gpadmin | UTF8 | en_US.US.UTF-8 | en_US.UTF-8 | template0 | gpadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gpadmin + | | | | | gpadmin=CTc/gpadmin template1 | gpadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/gpadmin + | | | | | gpadmin=CTc/gpadmin tutorial | gpadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 | (5 rows) ``` -------------------------------- ### Start SSH Daemon if Not Running Source: https://cloudberry.apache.org/docs/1.x/cbdb-rockylinux9-compile Ensures the SSH daemon is running, which is crucial for Apache Cloudberry's inter-process communication. This script checks if `sshd` is active and starts it if necessary, configuring password authentication. ```shell if ! pgrep sshd > /dev/null; then echo "SSH daemon not running. Starting it now..." sudo ssh-keygen -A echo "PasswordAuthentication yes" | sudo tee -a /etc/ssh/sshd_config sudo /usr/sbin/sshd else echo "SSH daemon is already running" fi ``` -------------------------------- ### Start Apache Cloudberry DBMS Source: https://cloudberry.apache.org/docs/start-and-stop-cbdb-database Initiates an initialized Apache Cloudberry system. The `gpstart` utility starts all PostgreSQL instances across the cluster in parallel. This command should be run on the coordinator host. ```bash $ gpstart ``` -------------------------------- ### Display createuser Help Source: https://cloudberry.apache.org/docs/sys-utilities/createuser This snippet shows how to display the help information for the `createuser` command. This is useful for understanding all available options and their usage. ```bash createuser --help ``` -------------------------------- ### gplogfilter timestamp filtering example Source: https://cloudberry.apache.org/docs/next/sys-utilities/gplogfilter This example shows how to specify a start date and time for filtering log entries using the --begin option. The datetime format must be 'YYYY-MM-DD [hh:mm[:ss]]' and can be enclosed in quotes. ```bash gplogfilter -b '2013-05-23 14:33' ```