### Install and configure rpk Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/iceberg/iceberg-topics-gcp-biglake.adoc Install the Redpanda command-line tool (`rpk`) and create a profile for quickstart configurations. Ensure `~/.local/bin` is in your PATH. ```bash sudo apt-get install unzip curl -LO https://github.com/redpanda-data/redpanda/releases/latest/download/rpk-linux-amd64.zip && mkdir -p ~/.local/bin && export PATH="~/.local/bin:$PATH" && unzip rpk-linux-amd64.zip -d ~/.local/bin/ rpk profile create quickstart --from-profile rpk-profile.yaml ``` -------------------------------- ### Install and Import Dependencies Source: https://github.com/redpanda-data/docs/blob/main/modules/develop/partials/http-proxy.adoc Setup commands and import statements for NodeJS and Python clients. ```bash npm install axios ``` ```javascript const axios = require('axios'); const base_uri = 'http://:8082'; ``` ```javascript const axios = require('axios'); const base_uri = 'http://:30082'; ``` ```bash pip install requests ``` ```python import requests import json def pretty(text): print(json.dumps(text, indent=2)) base_uri = "http://:8082" ``` ```python import requests import json def pretty(text): print(json.dumps(text, indent=2)) base_uri = "http://:30082" ``` -------------------------------- ### Start Redpanda Tuner Service (Ubuntu) Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/manual/production/production-deployment.adoc After installing the APT package on Ubuntu, use systemctl to start the `redpanda-tuner` service. The service is already enabled by default. ```bash sudo systemctl start redpanda-tuner ``` -------------------------------- ### Install Docker and Docker Compose Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/iceberg/iceberg-topics-gcp-biglake.adoc Install Docker and Docker Compose on the VM instance by following the official Docker installation guide for Debian. ```bash ``` -------------------------------- ### Install and Run iPerf3 Server on Debian Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/manual/sizing.adoc Installs iPerf3 and starts it in server mode on a Debian-based system. This is the first step in testing network bandwidth between nodes. ```bash redpanda1:~ sudo apt -y update; sudo apt -y install iperf3 redpanda1:~ iperf3 -s ``` -------------------------------- ### Install and Update Dependencies Source: https://github.com/redpanda-data/docs/blob/main/README.adoc Updates the project packages before starting development. ```bash npm update ``` -------------------------------- ### Example of Created Cloud Topic Source: https://github.com/redpanda-data/docs/blob/main/modules/develop/pages/manage-topics/cloud-topics.adoc This is an example output showing a successfully created Cloud Topic. ```console TOPIC STATUS audit.analytics.may2025 OK ``` -------------------------------- ### Start MirrorMaker 2 Service Source: https://github.com/redpanda-data/docs/blob/main/modules/migrate/pages/data-migration.adoc Start the MirrorMaker 2 service using `systemctl`. ```bash sudo systemctl start mm2.service ``` -------------------------------- ### Install rpk on Linux Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/kubernetes/security/tls/k-secrets.adoc Instructions for installing the `rpk` command-line client on a Linux system. ```bash [loweralpha] include::get-started:partial$install-rpk-linux.adoc[tags=latest] ``` -------------------------------- ### Example usage Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-storage-status-mount.adoc Example of checking the status for a specific migration ID. ```bash rpk cluster storage status-mount 123 ``` -------------------------------- ### Install rpk using Homebrew Source: https://github.com/redpanda-data/docs/blob/main/modules/get-started/partials/install-rpk-homebrew.adoc Use this command to install or update the rpk tool. Ensure Homebrew is installed first. ```bash brew install redpanda-data/tap/redpanda ``` -------------------------------- ### Get Help for -X Options Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-x-options.adoc Run `rpk -X help` to get detailed information about supported `-X` options. ```bash rpk -X help ``` -------------------------------- ### Create ACL examples Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/security/authorization/acl.adoc Examples for creating ACLs with specific permissions, topics, and groups. ```bash rpk security acl create --allow-principal bar --operation all --topic foo --group g ``` ```bash rpk security acl create --allow-principal '*' --operation read --topic biz,baz ``` ```bash rpk security acl create --allow-principal User:buzz --operation write --transactional-id txn ``` -------------------------------- ### Transfer partition leadership examples Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-partitions-transfer-leadership.adoc Examples showing how to specify partition and broker IDs for leadership transfer. ```bash rpk cluster partitions transfer-leadership foo --partition 0:2 ``` ```bash rpk cluster partitions transfer-leadership --partition test-topic/0:2 ``` -------------------------------- ### Install BYOC plugin with credentials Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/partials/rpk-cloud/rpk-cloud-byoc-install.adoc Use the -X flag to provide your client ID and client secret during the installation process. ```bash rpk cloud byoc install -X cloud.client_id= -X cloud.client_secret= ``` -------------------------------- ### Start minikube Cluster Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/kubernetes/local-guide.adoc Start a minikube cluster with 4 nodes. ```bash minikube start --nodes 4 ``` -------------------------------- ### Start Redpanda service Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/partials/linux/start-redpanda.adoc Use systemctl to start the redpanda-tuner and redpanda services. ```bash sudo systemctl start redpanda-tuner redpanda ``` -------------------------------- ### Client Configuration Example Source: https://github.com/redpanda-data/docs/blob/main/modules/develop/pages/benchmark.adoc Example client configuration for Redpanda producer and consumer clients, including topic configurations. ```yaml topicConfig: | min.insync.replicas=2 flush.messages=1 flush.ms=0 producerConfig: | acks=all linger.ms=1 batch.size=131072 consumerConfig: | auto.offset.reset=earliest enable.auto.commit=false auto.commit.interval.ms=0 max.partition.fetch.bytes=131072 ``` -------------------------------- ### Install Redpanda Connect via CLI Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-connect/rpk-connect-install.adoc Basic command syntax for installing Redpanda Connect. ```bash rpk connect install [flags] ``` -------------------------------- ### Install rpk on macOS Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/kubernetes/security/tls/k-secrets.adoc Instructions for installing the `rpk` command-line client on macOS using Homebrew. ```bash [loweralpha] include::get-started:partial$install-rpk-homebrew.adoc[] ``` -------------------------------- ### Install Redpanda with Helm Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/kubernetes/local-guide.adoc Configures the Helm repository and installs cert-manager and Redpanda. ```bash helm repo add redpanda https://charts.redpanda.com helm repo add jetstack https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager --set crds.enabled=true --namespace cert-manager --create-namespace ``` ```bash helm repo add redpanda https://charts.redpanda.com/ helm repo update helm install redpanda redpanda/redpanda \ --version {latest-redpanda-helm-chart-version} \ --namespace \ --create-namespace \ --set external.domain=customredpandadomain.local \ --set statefulset.initContainers.setDataDirOwnership.enabled=true ``` -------------------------------- ### Install Benchmark Requirements Source: https://github.com/redpanda-data/docs/blob/main/modules/develop/pages/benchmark.adoc Installs necessary Python packages for the benchmark. Ensure you are in the correct directory before running. ```bash cd ../../bin # openmessaging-benchmark/bin python3 -m pip -r install requirements.txt ``` -------------------------------- ### Start Local Development Server Source: https://github.com/redpanda-data/docs/blob/main/README.adoc Builds the documentation and launches a local web server with live reloading. ```bash npm run start ``` -------------------------------- ### Install Redpanda with Helm --set flags Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/kubernetes/storage/k-emptydir.adoc Configure storage settings directly via command-line flags during Helm installation. ```bash helm upgrade --install redpanda redpanda/redpanda \ --namespace \ --create-namespace \ --set storage.hostPath="" \ --set storage.persistentVolume.enabled=false ``` -------------------------------- ### Install Docker Engine Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/iceberg/iceberg-topics-gcp-biglake.adoc Installs Docker Engine, CLI, and Compose plugins on Debian-based systems. Ensure Docker is installed before proceeding with Redpanda setup. ```bash sudo apt-get update sudo apt-get install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin ``` -------------------------------- ### Create a shadow link with configuration Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-shadow/rpk-shadow-create.adoc Examples showing how to specify a configuration file and bypass the confirmation prompt. ```bash rpk shadow create --config-file shadow-link.yaml ``` ```bash rpk shadow create -c shadow-link.yaml --no-confirm ``` -------------------------------- ### Enable and start Redpanda Console service Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/console/linux/deploy.adoc Enables the Redpanda Console systemd service to start on boot and immediately starts the service. This command is used after installing the Redpanda Console package. ```bash sudo systemctl enable --now redpanda-console ``` -------------------------------- ### View Configuration Files Source: https://github.com/redpanda-data/docs/blob/main/modules/get-started/pages/quick-start.adoc Configuration files included in the quickstart deployment. ```yaml include::test-resources:attachment$docker-compose/docker-compose.yml[] ``` ```yaml include::test-resources:attachment$docker-compose/rpk-profile.yaml[] ``` ```yaml include::test-resources:attachment$docker-compose/bootstrap.yml[] ``` -------------------------------- ### Check Go Installation Source: https://github.com/redpanda-data/docs/blob/main/tests/docker-compose/transform/README.adoc Verify that Go version 1.20 or later is installed on your system. This is a prerequisite for compiling the Wasm transform. ```bash go version ``` -------------------------------- ### Troubleshoot Redpanda version installation errors Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/partials/linux/install-redpanda.adoc Example of an error message encountered when attempting to install a specific version of Redpanda without including its dependencies. ```bash yum install redpanda=24.3.1~rc1-1 Reading package lists... Done Building dependency tree... Done Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: redpanda : Depends: redpanda-rpk (= 24.3.1~rc1-1) but it is not going to be installed or redpanda-rpk-fips (= 24.3.1~rc1-1) but it is not going to be installed Depends: redpanda-tuner (= 24.3.1~rc1-1) but it is not going to be installed E: Unable to correct problems, you have held broken packages. ``` ```bash apt install redpanda=24.3.1~rc1-1 Reading package lists... Done Building dependency tree... Done Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: redpanda : Depends: redpanda-rpk (= 24.3.1~rc1-1) but it is not going to be installed or redpanda-rpk-fips (= 24.3.1~rc1-1) but it is not going to be installed Depends: redpanda-tuner (= 24.3.1~rc1-1) but it is not going to be installed E: Unable to correct problems, you have held broken packages. ``` -------------------------------- ### Get help for rpk configuration options Source: https://github.com/redpanda-data/docs/blob/main/modules/get-started/pages/intro-to-rpk.adoc Display detailed descriptions for each available configuration option. ```bash rpk -X help ``` -------------------------------- ### Start Redpanda with I/O properties file Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/io-optimization.adoc Use a previously generated I/O configuration file to start Redpanda on identical hardware. ```bash rpk redpanda start --io-properties-file '' ``` -------------------------------- ### Start and Enable Redpanda Tuner Service (RHEL) Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/manual/production/production-deployment.adoc After installing the RPM package on RHEL, use systemctl to start and enable the `redpanda-tuner` service, which automatically tunes the Linux kernel on Redpanda broker node restarts. ```bash sudo systemctl start redpanda-tuner sudo systemctl enable redpanda-tuner ``` -------------------------------- ### Example Output of License Generation Source: https://github.com/redpanda-data/docs/blob/main/modules/get-started/pages/licensing/overview.adoc This is an example of the output you might receive after successfully generating and applying a trial license. It confirms the license save location and provides instructions for setting the cluster license. ```bash Successfully saved license to "/home//code/rp/redpanda/src/go/rpk/redpanda.license". Upload this license in Redpanda Console, or run: rpk cluster license set --path /home//rp/redpanda/src/go/rpk/redpanda.license This license expires on 2025-05-07. For more information, see: https://docs.redpanda.com/current/get-started/licensing/overview/#license-keys ``` -------------------------------- ### Deploy Redpanda with Docker Compose Source: https://github.com/redpanda-data/docs/blob/main/modules/get-started/pages/quick-start.adoc Downloads the quickstart archive and initializes the Redpanda cluster using Docker Compose. ```bash mkdir redpanda-quickstart && cd redpanda-quickstart && \ ifdef::page-component-version-is-latest[] curl -sSL https://docs.redpanda.com/redpanda-quickstart.tar.gz | tar xzf - && \ endif::[] ifndef::page-component-version-is-latest[] curl -sSL https://docs.redpanda.com/{page-component-version}-redpanda-quickstart.tar.gz | tar xzf - && \ endif::[] cd docker-compose && docker compose up -d ``` -------------------------------- ### Run Ansible Playbook (With TLS/SASL) Source: https://github.com/redpanda-data/docs/blob/main/modules/develop/pages/benchmark.adoc Execute the Ansible playbook to install and start the Redpanda cluster with TLS and SASL enabled. ```bash ansible-playbook deploy.yaml -e "tls_enabled=true sasl_enabled=true" ``` -------------------------------- ### Get Current Redpanda Version (macOS) Source: https://github.com/redpanda-data/docs/blob/main/modules/upgrade/pages/rolling-upgrade.adoc On macOS, use this command to list installed Redpanda versions via Homebrew. ```bash brew list --versions | grep redpanda ``` -------------------------------- ### List Shadow Links Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/disaster-recovery/shadowing/monitor.adoc Lists all existing shadow links. This command is useful for getting an overview of your shadow replication setup. ```APIDOC ## List Shadow Links ### Description Lists all existing shadow links. ### Method GET ### Endpoint /v1/shadow-links ### Request Example ```json { "example": "curl 'https://api.redpanda.com/v1/shadow-links' \ -H \"Content-Type: application/json\" \ -H \"Authorization: Bearer ${RP_CLOUD_TOKEN}\"" } ``` ### Response #### Success Response (200) - **shadow_links** (array) - A list of shadow link objects. #### Response Example ```json { "example": "[Array of shadow link objects]" } ``` ``` -------------------------------- ### Run Ansible Playbook (No TLS/SASL) Source: https://github.com/redpanda-data/docs/blob/main/modules/develop/pages/benchmark.adoc Execute the Ansible playbook to install and start the Redpanda cluster without TLS and SASL enabled. ```bash ansible-playbook deploy.yaml ``` -------------------------------- ### Example of rpk connect echo Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-connect/rpk-connect-echo.adoc Piping the normalized configuration output to less for easier viewing. ```bash rpk connect echo ./config.yaml | less ``` -------------------------------- ### List All Subjects with rpk Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-registry/rpk-registry-subject-list.adoc Use this command to display all subjects in the Redpanda registry. No specific setup is required beyond having rpk installed and configured. ```bash rpk registry subject list [flags] ``` -------------------------------- ### Download Redpanda Quickstart Files Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/iceberg/iceberg-topics-gcp-biglake.adoc Downloads and extracts the Redpanda self-managed quickstart files. This includes the necessary Docker Compose configurations for setting up Redpanda. ```bash mkdir redpanda-quickstart && cd redpanda-quickstart && \ curl -sSL https://docs.redpanda.com/redpanda-quickstart.tar.gz | tar xzf - && \ cd docker-compose ``` -------------------------------- ### List SASL Users with rpk Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-security/rpk-security-user-list.adoc Use this command to list all SASL users configured in Redpanda. No setup is required beyond having rpk installed and configured. ```bash rpk security user list [flags] ``` -------------------------------- ### Generate Shadowing Configuration with rpk Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/disaster-recovery/shadowing/setup.adoc Use rpk to create a sample configuration file or a template with documentation for Shadowing setup. ```bash # Generate a sample configuration file with placeholder values rpk shadow config generate -o shadow-config.yaml # Or generate a template with detailed field documentation rpk shadow config generate --print-template -o shadow-config-template.yaml ``` -------------------------------- ### GET /topics/{topic_name}/partitions/{partition_id}/records Source: https://github.com/redpanda-data/docs/blob/main/modules/develop/partials/http-proxy.adoc Retrieves events from a specific partition of a topic. You can specify the starting offset and other parameters to control the data retrieval. ```APIDOC ## GET /topics/{topic_name}/partitions/{partition_id}/records ### Description Retrieves events from a specific partition of a topic. You can specify the starting offset and other parameters to control the data retrieval. ### Method GET ### Endpoint /topics/{topic_name}/partitions/{partition_id}/records ### Query Parameters - **offset** (integer) - Required - The starting offset from which to retrieve records. - **timeout** (integer) - Optional - The maximum time in milliseconds to wait for records to become available. - **max_bytes** (integer) - Optional - The maximum number of bytes to return in the response. ### Request Example ```json { "example": "curl -s \"$base_uri/topics/test_topic/partitions/0/records?offset=0&timeout=1000&max_bytes=100000\" -H \"Accept: application/vnd.kafka.json.v2+json\"" } ``` ### Response #### Success Response (200) - **topic** (string) - The name of the topic. - **key** (string) - The key of the message. - **value** (string) - The value of the message. - **partition** (integer) - The partition the message belongs to. - **offset** (integer) - The offset of the message within the partition. #### Response Example ```json { "example": "[\"{\"topic\":\"test_topic\",\"key\":null,\"value\":\"Redpanda\",\"partition\":0,\"offset\":0}\"]" } ``` ``` -------------------------------- ### Generate configuration with rpk connect create Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-connect/rpk-connect-create.adoc Examples of using the rpk connect create command with specific component expressions. ```bash rpk connect create stdin/bloblang,awk/nats ``` ```bash rpk connect create file,http_server/protobuf/http_client ``` -------------------------------- ### Configure Connectors Chart Values for Redpanda Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/kafka-connect/k-deploy-kafka-connect.adoc Example YAML configuration for the connectors chart. Set bootstrap servers and TLS/SASL settings according to your Redpanda cluster setup. ```yaml # Connection to Redpanda brokers connectors: bootstrapServers: "" # Configure TLS if your Redpanda cluster has TLS enabled brokerTLS: enabled: true ca: secretRef: "redpanda-default-cert" secretNameOverwrite: "ca.crt" # Configure SASL if your Redpanda cluster has SASL enabled auth: sasl: enabled: false mechanism: "SCRAM-SHA-512" userName: "" secretRef: "" ``` -------------------------------- ### Example io-config.yaml output Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-iotune.adoc Sample content of the configuration file generated by the iotune command. ```yaml disks: - mountpoint: /var/lib/redpanda/data read_iops: 40952 read_bandwidth: 5638210048 write_iops: 6685 write_bandwidth: 1491679488 ``` -------------------------------- ### Filter with case-insensitive matching Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-config-list.adoc Use the `(?i)` flag within the --filter argument for case-insensitive matching of configuration keys. This example finds keys starting with 'batch' regardless of case. ```bash rpk cluster config list --filter="(?i)batch.*" ``` -------------------------------- ### List configuration properties matching a filter Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-config-list.adoc Use the --filter flag with a regular expression to find specific configuration keys. This example filters for keys starting with 'kafka'. ```bash rpk cluster config list --filter="kafka.*" ``` -------------------------------- ### Create a topic with default settings Source: https://github.com/redpanda-data/docs/blob/main/modules/develop/pages/manage-topics/config-topics.adoc Use this command to create a topic with default partitions and replicas. Ensure the topic name is unique. ```bash rpk topic create xyz ``` -------------------------------- ### Enable Protobuf Deserialization with Git Sync Source: https://github.com/redpanda-data/docs/blob/main/modules/console/pages/config/deserialization.adoc Configure Redpanda Console to enable Protobuf deserialization and synchronize Protobuf definitions from a Git repository. This setup is for standalone Helm chart installations. ```yaml config: serde: protobuf: enabled: true mappings: - topicName: orders valueProtoType: fake_models.Order keyProtoType: fake_models.OrderKey git: enabled: true repository: url: https://github.com/myorg/kafka-proto-files.git branch: master refreshInterval: 10m paths: - ./ ``` -------------------------------- ### Deploy Monitoring Stack with Ansible Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/manual/production/production-deployment-automation.adoc Run the deploy-monitor.yml playbook to install and configure a basic Prometheus and Grafana setup for Redpanda observability. This requires a [monitor] section in your hosts file and a valid private key for SSH access. ```bash ansible-playbook ansible/deploy-monitor.yml \ --private-key '' ``` -------------------------------- ### Provision Redpanda Cluster with Ansible Playbook Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/manual/production/production-deployment-automation.adoc Execute the main Ansible playbook to install and start a Redpanda cluster. This command assumes TLS encryption and Tiered Storage are enabled by default. Ensure you provide the correct path to your private SSH key. ```bash ansible-playbook --private-key -v ansible/provision-cluster.yml ``` -------------------------------- ### Generate Sample Configuration Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-shadow/rpk-shadow-config-generate.adoc Use this command to generate a sample configuration file for a shadow link and print it to standard output. Customize the placeholder values for your environment. ```bash rpk shadow config generate ``` -------------------------------- ### Start Redpanda with I/O properties string Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/io-optimization.adoc Pass the contents of an I/O configuration file directly as a string. ```bash rpk redpanda start --io-properties '' ``` -------------------------------- ### Start Redpanda FIPS container with a configuration file Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/security/fips-compliance.adoc Mount the redpanda.yaml file into the container at startup. ```bash docker run -d \ --name=redpanda \ -p 9092:9092 \ -p 9644:9644 \ -v /path/to/redpanda.yaml:/etc/redpanda/redpanda.yaml \ docker.redpanda.com/redpandadata/redpanda:-fips \ redpanda start --overprovisioned --smp 1 ``` -------------------------------- ### Install rpk on Debian/Ubuntu (FIPS) Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/partials/linux/install-fips.adoc Installs only the FIPS-compliant rpk tool on Debian/Ubuntu systems. Use this if you only need rpk and not the full Redpanda installation. ```bash sudo apt install -y redpanda-rpk-fips ``` -------------------------------- ### Install Redpanda on Debian/Ubuntu (FIPS) Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/partials/linux/install-fips.adoc Installs Redpanda on Debian/Ubuntu systems, ensuring FIPS compliance. This command also installs the base Redpanda package. ```bash sudo apt install redpanda-fips redpanda-rpk-fips ``` -------------------------------- ### Iceberg Partitioning Examples Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/iceberg/iceberg-performance-tuning.adoc Illustrates different ways to specify partition keys for an Iceberg table, including single columns, multiple columns, and transformations on timestamp columns. ```bash redpanda.iceberg.partition.spec=(col1) ``` ```bash redpanda.iceberg.partition.spec=(col1, col2) ``` ```bash redpanda.iceberg.partition.spec=(year(ts1), col1) ``` -------------------------------- ### Install Redpanda on RHEL (FIPS) Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/partials/linux/install-fips.adoc Installs Redpanda on RHEL systems, ensuring FIPS compliance. This command also installs the base Redpanda package. ```bash curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.rpm.sh' | \ sudo -E bash && sudo yum install redpanda -y ``` -------------------------------- ### List Supported -X Options Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-x-options.adoc Run `rpk -X list` to see all available configuration options that can be overridden with the `-X` flag. ```bash rpk -X list ``` -------------------------------- ### Basic command usage Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/partials/rpk-cloud/rpk-cloud-byoc-install.adoc The standard syntax for executing the BYOC installation command. ```bash rpk cloud byoc install [flags] ``` -------------------------------- ### Check CRD Installation Permissions Source: https://github.com/redpanda-data/docs/blob/main/modules/migrate/pages/kubernetes/helm-to-operator.adoc Verify that you have the necessary permissions to install Custom Resource Definitions (CRDs) in your Kubernetes cluster. This is a prerequisite for installing the Redpanda Operator. ```bash kubectl auth can-i create CustomResourceDefinition --all-namespaces ``` -------------------------------- ### Avro Schema Example Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/iceberg/specify-iceberg-schema.adoc This is an example of an Avro schema for a `ClickEvent`. ```avro { "type": "record", "name": "ClickEvent", "fields": [ { "name": "user_id", "type": "int" }, { "name": "event_type", "type": "string" }, { "name": "ts", "type": "string" } ] } ``` -------------------------------- ### Install cert-manager with Helm Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/kubernetes/local-guide.adoc Install cert-manager using Helm. This command adds the Jetstack Helm repository, updates it, and installs cert-manager in the 'cert-manager' namespace, enabling CRDs. ```bash helm repo add jetstack https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager --set crds.enabled=true --namespace cert-manager --create-namespace ``` -------------------------------- ### rpk plugin uninstall Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-plugin/rpk-plugin-uninstall.adoc Uninstall or remove an existing local plugin. This command lists locally installed plugins and removes the first plugin that matches the requested removal. If --include-shadowed is specified, this command also removes all shadowed plugins of the same name. To remove a command under a nested namespace, concatenate the namespace. For example, for the nested namespace `rpk foo bar`, use the name `foo_bar`. ```APIDOC ## rpk plugin uninstall ### Description Uninstall or remove an existing local plugin. This command lists locally installed plugins and removes the first plugin that matches the requested removal. If `--include-shadowed` is specified, this command also removes all shadowed plugins of the same name. To remove a command under a nested namespace, concatenate the namespace. For example, for the nested namespace `rpk foo bar`, use the name `foo_bar`. ### Usage ```bash rpk plugin uninstall [NAME] [flags] ``` ### Aliases ```bash uninstall, rm ``` ### Flags #### Query Parameters - **--help** (boolean) - Optional - Help for uninstall. - **--include-shadowed** (boolean) - Optional - Also remove shadowed plugins that have the same name. - **--config** (string) - Optional - Redpanda or `rpk` config file; default search paths are `/var/lib/redpanda/.config/rpk/rpk.yaml`, `$PWD/redpanda.yaml`, and `/etc/redpanda/redpanda.yaml`. - **-X, --config-opt** (stringArray) - Optional - Override `rpk` configuration settings. See xref:reference:rpk/rpk-x-options.adoc[`rpk -X`] or execute `rpk -X help` for inline detail or `rpk -X list` for terser detail. - **--profile** (string) - Optional - Profile to use. See xref:reference:rpk/rpk-profile.adoc[`rpk profile`] for more details. - **-v, --verbose** (boolean) - Optional - Enable verbose logging. ``` -------------------------------- ### Start Redpanda with well-known I/O settings Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/io-optimization.adoc Specify cloud vendor, VM type, and storage type when metadata API access is restricted. ```bash rpk redpanda start --well-known-io 'aws:i3.xlarge:default' ``` -------------------------------- ### Example Avro Schema Source: https://github.com/redpanda-data/docs/blob/main/modules/manage/pages/schema-reg/schema-reg-api.adoc This is an example Avro schema for sensor data. ```json { "type": "record", "name": "sensor_sample", "fields": [ { "name": "timestamp", "type": "long", "logicalType": "timestamp-millis" }, { "name": "identifier", "type": "string", "logicalType": "uuid" }, { "name": "value", "type": "long" } ] } ``` -------------------------------- ### Initialize Terraform Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/partials/high-availability.adoc Prepare the Terraform environment for the chosen cloud provider. ```bash cd deployment-automation/aws (or cd deployment-automation/azure, or cd deployment-automation/gcp) terraform init ``` -------------------------------- ### Run rpk iotune Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-iotune.adoc Basic command syntax for executing the I/O performance benchmark. ```bash rpk iotune [flags] ``` -------------------------------- ### Install Ansible requirements Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/partials/high-availability.adoc Install necessary Ansible dependencies before running playbooks. ```bash ansible-galaxy install -r ansible/requirements.yml ``` -------------------------------- ### Get eksctl create cluster Help Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/kubernetes/eks-guide.adoc Display all available options for creating an EKS cluster using eksctl. ```bash eksctl create cluster --help ``` -------------------------------- ### Get compatibility level usage Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-registry/rpk-registry-compatibility-level-get.adoc Syntax for executing the compatibility-level get command. ```bash rpk registry compatibility-level get [SUBJECT...] [flags] ``` -------------------------------- ### Install and Run Fio for Sequential Writes Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/manual/sizing.adoc This bash script installs the fio benchmarking tool and configures it to test sequential write performance on a specified block device. Ensure the job targets the correct device for accurate results. ```bash sudo apt -y update; sudo apt -y install fio cd /var/lib/redpanda/data sudo tee fio-seq-write.job >/dev/null << EOF [global] name=fio-seq-write filename=fio-seq-write [file1] size=10G ioengine=libaio iodepth=16 EOF sudo fio fio-seq-write.job ``` -------------------------------- ### Example Redpanda I/O Configuration Source: https://github.com/redpanda-data/docs/blob/main/modules/deploy/pages/redpanda/kubernetes/k-tune-workers.adoc An example of the `io-config.yaml` file generated by `rpk iotune`, which Redpanda reads at startup to optimize itself for the worker node's I/O capabilities. ```yaml disks: - mountpoint: / read_iops: 40952 read_bandwidth: 5638210048 write_iops: 6685 write_bandwidth: 1491679488 ``` -------------------------------- ### List all available plugins Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-plugin/rpk-plugin-list.adoc Fetches the remote manifest and prints plugins available for download. Prefixes installed plugins with an asterisk. Prints additional messages for locally installed plugins with SHA-256 sum mismatches or calculation failures. ```bash rpk plugin list ``` -------------------------------- ### Basic command usage Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-security/rpk-security-acl-create.adoc General syntax for the create command. ```bash rpk security acl create [flags] ``` -------------------------------- ### rpk redpanda start Source: https://github.com/redpanda-data/docs/blob/main/modules/reference/pages/rpk/rpk-redpanda/rpk-redpanda-start.adoc Starts a Redpanda node with specified configuration flags or modes. ```APIDOC ## CLI COMMAND: rpk redpanda start ### Description Starts the Redpanda process. This command supports various flags for network configuration, system tuning, and environment modes. ### Method CLI Command ### Parameters #### Flags - **--advertise-kafka-addr** (strings) - Optional - A comma-separated list of Kafka addresses to advertise. - **--advertise-pandaproxy-addr** (strings) - Optional - A comma-separated list of Pandaproxy addresses to advertise. - **--advertise-rpc-addr** (string) - Optional - The advertised RPC address. - **--check** (boolean) - Optional - When set to false, disables system checking before starting (default true). - **--install-dir** (string) - Optional - Directory where redpanda has been installed. - **--kafka-addr** (strings) - Optional - A comma-separated list of Kafka listener addresses to bind to. - **--mode** (string) - Optional - Sets well-known configuration properties for development or test environments (e.g., dev-container). - **--node-tuner-state-path** (string) - Optional - Alternative path to read the node tuner state file from. - **--pandaproxy-addr** (strings) - Optional - A comma-separated list of Pandaproxy listener addresses to bind to. - **--rpc-addr** (string) - Optional - The RPC address to bind to. - **--schema-registry-addr** (strings) - Optional - A comma-separated list of Schema Registry listener addresses to bind to. - **-s, --seeds** (strings) - Optional - A comma-separated list of seed nodes to connect to. - **--timeout** (duration) - Optional - Maximum time to wait for checks and tune processes (default 10s). - **--tune** (boolean) - Optional - When present, enables tuning before starting. - **--well-known-io** (string) - Optional - The cloud vendor and VM type. - **--config** (string) - Optional - Path to Redpanda or rpk config file. - **-X, --config-opt** (stringArray) - Optional - Override rpk configuration settings. - **--profile** (string) - Optional - Profile to use. - **-v, --verbose** (boolean) - Optional - Enable verbose logging. ```