### Build RisingWave Documentation

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/README.md

Instructions for installing necessary tools and serving the RisingWave developer guide locally. This allows for real-time previewing of changes as you edit the documentation.

```sh
# install tools
> cargo install mdbook mdbook-linkcheck mdbook-toc
# start a web server on localhost that you can visit to view the book,
# and it will automatically reload each time you edit a page.
> mdbook serve --open
```

--------------------------------

### Install Dependencies and Start Development Server

Source: https://github.com/risingwavelabs/risingwave/blob/main/dashboard/README.md

Installs project dependencies using npm and starts the Next.js development server for the RisingWave dashboard.

```bash
npm i
npm run dev
```

--------------------------------

### RisingWave Cluster Setup and Dashboard Testing

Source: https://github.com/risingwavelabs/risingwave/blob/main/dashboard/README.md

Commands to set up a full RisingWave cluster, clean data, and run end-to-end tests specifically for the dashboard's graph creation functionality.

```bash
# Start full cluster so prometheus is available.
./risedev k
./risedev clean-data
./risedev d full
# We maintain our own separate test data for testing the dashboard's rendering
# It provides nested MV-on-MV structures (nexmark and tpch only have 1-deep MV-on-MV).
# It also provides backpressure and lag scenarios.
./risedev slt e2e_test/dashboard/create_graph.slt.part --label dashboard
```

--------------------------------

### Example risedev.yml Configuration

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

An example configuration file for risedev.yml, specifying components to include in the development cluster.

```yaml
default:
  - use: minio
  - use: meta-node
  - use: compute-node
  - use: frontend
  - use: prometheus
  - use: grafana
  - use: kafka
    persist-data: true
```

--------------------------------

### Install RisingWave Standalone

Source: https://github.com/risingwavelabs/risingwave/blob/main/README.md

Installs RisingWave in standalone mode using a curl command. This is a quick way to get started with RisingWave.

```shell
curl -L https://risingwave.com/sh | sh
```

--------------------------------

### Start RisingWave Playground

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Starts the RisingWave playground, a simplified development environment where metadata, compute, and frontend nodes run in a single process. Logs are directed to standard output.

```shell
./risedev p # shortcut for ./risedev playground
```

```shell
cargo run --bin risingwave -- playground
```

```shell
psql -h localhost -p 4566 -d dev -U root
```

--------------------------------

### Start MinIO Server for Hummock Replay

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/storage/hummock_trace/README.md

Starts a MinIO server instance with specified credentials and configuration for storing Hummock data.

```bash
MINIO_ROOT_PASSWORD=hummockadmin \
MINIO_ROOT_USER=hummockadmin \
.risingwave/bin/minio server \
--address 127.0.0.1:9301 \
--console-address 127.0.0.1:9400 \
--config-dir .risingwave/config/minio \
.risingwave/data/minio
```

--------------------------------

### Install Dependencies and Run Python UDF Example

Source: https://github.com/risingwavelabs/risingwave/blob/main/e2e_test/udf/remote_python/README.md

Installs the necessary Python packages using requirements.txt and then runs the test script for the RisingWave Python UDF example. The UDF server will be accessible on port 8815.

```Shell
pip install -r requirements.txt
python3 test.py
```

--------------------------------

### Setup RisingWave Pipeline

Source: https://github.com/risingwavelabs/risingwave/blob/main/integration_tests/kafka-cdc-sink/README.md

Command to set up the RisingWave data pipeline. Requires cargo make to be installed.

```bash
cargo make setup
```

--------------------------------

### Start RisingWave Development Cluster

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Starts a RisingWave development cluster using the risedev tool.

```shell
./risedev d
```

--------------------------------

### Risingwave Standalone Mode Configuration

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/cmd_all/src/README.md

Example of configuring and starting Risingwave in standalone mode via CLI parameters. This mode is suitable for production environments.

```bash
standalone \
  --meta-opts="..." \
  --frontend-opts="..."
  # --compute-opts="..." not provided, so it won't be started.
```

--------------------------------

### RiseDev Component Configuration Environment Variables

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/risedevtool/README.md

Example environment variables set by RiseDev's configuration wizard, indicating that the setup is configured and MinIO and Rust build are enabled.

```env
RISEDEV_CONFIGURED=true
ENABLE_MINIO=true
ENABLE_BUILD_RUST=true
```

--------------------------------

### Starting RisingWave with HDFS/WebHDFS

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/design/multi-object-store.md

After configuring risedev, you can start RisingWave with the chosen HDFS backend using the risedev command-line tool.

```bash
# Start RisingWave with HDFS backend
./risedev d hdfs

# Start RisingWave with WebHDFS backend
./risedev d webhdfs

```

--------------------------------

### Start RisingWave with 'hm-trace' Profile

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/storage/hummock_trace/README.md

Starts RisingWave using risedev with a profile that enables Hummock tracing.

```bash
USE_HM_TRACE=true ./risedev d hummock-trace
```

--------------------------------

### Start MinIO Server (Replay Context)

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/storage/hummock_trace/README.md

Command to start a MinIO server, typically used before replaying Hummock operations.

```bash
MINIO_ROOT_PASSWORD=hummockadmin MINIO_ROOT_USER=hummockadmin .risi
ngwave/bin/minio server --address 127.0.0.1:9301 --console-address 127
.0.0.1:9400 --config-dir .risingwave/config/minio .risingwave/data/minio
```

--------------------------------

### Starting RiseDev with a Profile

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/connector/intro.md

Command to start the RisingWave cluster and associated external systems (like Kafka and MySQL) based on a specified RiseDev profile.

```sh
risedev d my-cool-profile
```

--------------------------------

### Install Dependencies on Debian-based Linux

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Installs required development tools and Rust for Debian-based Linux distributions.

```shell
sudo apt install make build-essential cmake protobuf-compiler curl postgresql-client tmux lld pkg-config libssl-dev libsasl2-dev libblas-dev liblapack-dev libomp-dev
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

--------------------------------

### PostgreSQL Docker Setup and Connection

Source: https://github.com/risingwavelabs/risingwave/blob/main/java/connector-node/README.md

Instructions for setting up a PostgreSQL instance using Docker and connecting to it.

```shell
# create postgresql in docker
docker run --name my-postgres -e POSTGRES_PASSWORD=connector -e POSTGRES_DB=test -e POSTGRES_USER=test -d -p 5432:5432 postgres
# connect postgresql
psql -h localhost -p 5432 -U test -d postgres
```

--------------------------------

### RiseDev Configuration Example

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/config/README.md

Example of specifying a custom configuration file path for a RiseDev profile.

```yaml
risedev:
  profile-name:
    config-path: src/config/ci.toml
    steps:
      - use: ...
```

--------------------------------

### Cargo-Make Task Execution Example

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/risedevtool/README.md

Illustrates the execution flow of cargo-make tasks during RiseDev setup, showing skipped tasks like `check-risedev-configured` and `download-grafana`, and executed tasks like `download-minio` and `build-risingwave`.

```toml
[cargo-make]
INFO - Skipping Task: check-risedev-configured
INFO - Running Task: download-minio
INFO - Running Task: download-mcli
INFO - Skipping Task: download-grafana
INFO - Skipping Task: download-prometheus
INFO - Running Task: build-risingwave
```

--------------------------------

### Install Dependencies on macOS

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Installs necessary development tools and Rust for macOS users.

```shell
brew install postgresql cmake protobuf tmux cyrus-sasl lld openssl@3 libomp
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

--------------------------------

### Start RisingWave Cluster with Hummock Tracing

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/storage/hummock_trace/README.md

Starts a RisingWave cluster with the Hummock tracing feature enabled.

```bash
USE_HM_TRACE=true ./risedev d hummock-trace
```

--------------------------------

### Start Docker and Build Image for Airflow Compaction

Source: https://github.com/risingwavelabs/risingwave/blob/main/integration_tests/iceberg-sink/README.md

Builds the Docker image and starts the Docker environment for the Iceberg compaction demo using Airflow.

```sh
docker-compose up --build
```

--------------------------------

### RisingWave Contribution Guide

Source: https://github.com/risingwavelabs/risingwave/blob/main/README.md

Information for developers interested in contributing to the RisingWave project.

```markdown
Please refer to [RisingWave Developer Guide](https://risingwavelabs.github.io/risingwave/) for more information.
```

--------------------------------

### RisingWave Pull Request Examples for New Connectors

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/connector/intro.md

Provides links to RisingWave pull requests that serve as examples for adding new connectors, specifically for user-managed and Docker-based MySQL connectors.

```markdown
#16449 (user-managed only MySQL)
```

```markdown
#16514 (Docker based MySQL)
```

--------------------------------

### Example General Script Location

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/connector/intro.md

Indicates that general-purpose scripts, usable across multiple tests, should be placed in the `e2e_test/commands/` directory.

```markdown
e2e_test/commands/
```

--------------------------------

### Sqllogictest End-to-End Test Example

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/connector/intro.md

An example of an end-to-end test case using sqllogictest-rs. It demonstrates enabling substitution, running bash commands with `system ok`, and using environment variables for configuration.

```sql
control substitution on

# Note: you can also use envvars in `system` commands, but usually it's not necessary since the CLI tools can load envvars themselves.
system ok
rpk topic create my_source -p 4
```

--------------------------------

### Cargo Build Jobs Configuration

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Example of how to lower compilation parallelism using CARGO_BUILD_JOBS to mitigate memory bottlenecks.

```shell
CARGO_BUILD_JOBS=2
```

--------------------------------

### Launch RisingWave Cluster with Dependencies

Source: https://github.com/risingwavelabs/risingwave/blob/main/integration_tests/cassandra-and-scylladb-sink/README.md

Starts the necessary services for the demo, including a RisingWave cluster, data generator, and Cassandra sink.

```sh
docker-compose up -d
```

--------------------------------

### RiseDev Debug Output Example

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/risedevtool/README.md

Example output when debugging a compute node, showing the status of managed services and indicating that the user needs to start the compute-node manually.

```plain
✅ tmux: session risedev
✅ minio: api http://127.0.0.1:9301/, console http://127.0.0.1:9400/
.. compute-node-5688: waiting for user-managed service online... (you should start it!)
.. dev cluster: starting 5 services for dev-compute-node...
```

--------------------------------

### Example System Command: MySQL Operations

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/connector/intro.md

Demonstrates executing MySQL commands within a test script, including dropping and creating a database, creating a table, and inserting data.

```bash
mysql -e "
    DROP DATABASE IF EXISTS testdb1; CREATE DATABASE testdb1;
    USE testdb1;
    CREATE TABLE tt1 (v1 int primary key, v2 timestamp);
    INSERT INTO tt1 VALUES (1, '2023-10-23 10:00:00');
"
```

--------------------------------

### Get Compute Node PID on OSX

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/benchmark-and-profile/cpu-profiling.md

Retrieves the Process ID (PID) of the 'compute-node' process on macOS.

```shell
pgrep compute-node
```

--------------------------------

### Build Workload Generator from Source

Source: https://github.com/risingwavelabs/risingwave/blob/main/integration_tests/README.md

Instructions to build the workload generator binary from its source code using Go.

```Go
cd datagen && go build
```

--------------------------------

### Example Wrapper Script for CLI Tools

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/connector/intro.md

Demonstrates the use of a wrapper script (e.g., `e2e_test/commands/mysql`) to handle CLI tool arguments implicitly, making test invocation more concise.

```markdown
e2e_test/commands/mysql
```

--------------------------------

### Profile CPU on OSX with Cargo Flamegraph

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/benchmark-and-profile/cpu-profiling.md

Installs the 'flamegraph' cargo crate and then uses it to profile a specified process ID (PID) on macOS, outputting the flamegraph to 'flamegraph.svg'.

```shell
cargo install flamegraph
sudo flamegraph -o flamegraph.svg --pid [pid]
```

--------------------------------

### Run jeprof in RisingWave Docker Image

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/benchmark-and-profile/memory-profiling.md

Starts a Docker container with the RisingWave image, mounting the current directory to `/dumps` for easy access to profiling files. Assumes jeprof is included in the image.

```bash
docker run -it --rm --entrypoint /bin/bash -v $(pwd):/dumps  ghcr.io/risingwavelabs/risingwave:latest
```

--------------------------------

### Integration Test Setup

Source: https://github.com/risingwavelabs/risingwave/blob/main/java/connector-node/README.md

Commands to set up the Python environment and run integration tests for the RisingWave Connector Node.

```bash
bash build-venv.sh
bash gen-stub.sh
PYTHONPATH=proto python3 integration_tests.py
```

--------------------------------

### VS Code Integration for Lints

Source: https://github.com/risingwavelabs/risingwave/blob/main/lints/README.md

Enables rust-analyzer integration for developing custom lints by duplicating the example VS Code settings file. This helps in getting real-time feedback on linting during development.

```bash
cp .vscode/settings.json.example .vscode/settings.json
```

--------------------------------

### Launch RisingWave Cluster

Source: https://github.com/risingwavelabs/risingwave/blob/main/integration_tests/sqlserver-sink/README.md

Starts the necessary services for the demo, including the RisingWave cluster, a data generator, and a SQL Server instance, using Docker Compose.

```sh
docker-compose up -d
```

--------------------------------

### RisingWave Data Sinking to Postgres Setup

Source: https://github.com/risingwavelabs/risingwave/blob/main/integration_tests/postgres-sink/README.md

Steps to set up RisingWave data sinking to an external PostgreSQL database. This involves starting the cluster, creating necessary SQL objects (source, materialized view, sink), and verifying data ingestion.

```sql
create_source.sql
```

```sql
create_mv.sql
```

```sql
create_sink.sql
```

--------------------------------

### RisingWave Configuration File Reference

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Details the configuration file for RisingWave, located at `src/config/risingwave.toml`. It specifies where to find all configurable variables and how to add new ones within sections like `[server]` or `[storage]`.

```rust
# Check src/common/src/config.rs for all configurable variables.
# Add new variables to appropriate sections (e.g., [server], [storage]) in src/config/risingwave.toml.
```

--------------------------------

### RisingWave Docker Deployment

Source: https://github.com/risingwavelabs/risingwave/blob/main/README.md

Instructions for deploying RisingWave using Docker Compose.

```markdown
For **Docker deployment**, please refer to [Docker Compose](https://docs.risingwave.com/docs/current/risingwave-docker-compose/).
```

--------------------------------

### Enable Heap Profiling Locally with risedev

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/benchmark-and-profile/memory-profiling.md

This snippet shows how to enable heap profiling for a local RisingWave cluster using `risedev`. It involves setting the `RISEDEV_ENABLE_HEAP_PROFILE` environment variable before starting the cluster. The profiler outputs `.heap` files upon significant memory allocation.

```shell
RISEDEV_ENABLE_HEAP_PROFILE=1 ./risedev d full
```

--------------------------------

### Example Migration File Structure

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/meta/model/migration/README.md

An example of a Rust migration file, demonstrating the structure for defining schema changes. This specific example adds an index column property.

```rust
src/m20240617_070131_index_column_properties.rs
```

--------------------------------

### Install cargo-flamegraph

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/benchmark-and-profile/microbenchmarks.md

Installs the `cargo-flamegraph` tool, which is used for generating flamegraphs.

```shell
cargo install flamegraph
```

--------------------------------

### Configure RisingWave Components

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Opens the configuration interface for enabling/disabling additional RisingWave components.

```shell
./risedev configure
```

--------------------------------

### Initialize New SQL Benchmark

Source: https://github.com/risingwavelabs/risingwave/blob/main/develop/sql_bench/README.md

Command to initialize a new benchmark configuration file. This creates a YAML file in the `benchmarks/` directory with a predefined structure for defining SQL benchmark parameters.

```bash
python main.py init <benchmark_name>
```

--------------------------------

### RisingWave Kubernetes Deployment

Source: https://github.com/risingwavelabs/risingwave/blob/main/README.md

Instructions for deploying RisingWave on Kubernetes using Helm or an Operator.

```markdown
For **Kubernetes deployment**, please refer to [Kubernetes with Helm](https://docs.risingwave.com/docs/current/risingwave-k8s-helm/) or [Kubernetes with Operator](https://docs.risingwave.com/docs/current/risingwave-kubernetes/).
```

--------------------------------

### risedev.yml Template and Profile Example

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/risedevtool/README.md

Demonstrates the structure of a risedev.yml file with a 'template' section defining a compute-node component and a 'profile' section using and overriding template configurations.

```yaml
template:
  compute-node:
    address: "127.0.0.1"
    port: 5688
    exporter-address: "127.0.0.1"
    exporter-port: 1222
    id: compute-node-${port}
    provide-minio: "minio*"
    user-managed: false

profile:
  ci-3cn-1fe:
    - use: compute-node
      port: 5687
      exporter-port: 1222
```

--------------------------------

### Start RisingWave Cluster for E2E Tests

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/tests/intro.md

Starts a full RisingWave cluster, which is a prerequisite for running end-to-end tests.

```shell
./risedev d
```

--------------------------------

### Install perf (Ubuntu)

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/benchmark-and-profile/microbenchmarks.md

Installs the `perf` performance analysis tool on Ubuntu systems. This is a prerequisite for generating flamegraphs.

```shell
sudo apt install linux-tools
```

--------------------------------

### Install Python 3.12 on macOS

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Installs Python version 3.12 on macOS, required for Embedded Python UDF feature.

```shell
brew install python@3.12
```

--------------------------------

### RPC Server Service Addition

Source: https://github.com/risingwavelabs/risingwave/blob/main/src/docs/development/how-to-write-a-rpc-service.md

Shows how to instantiate and add a custom service implementation to the RisingWave RPC server using `tonic`. It includes setting up middleware and starting the server.

```rust
let health_srv = HealthServiceImpl::new();
tokio::spawn(async move {
     tonic::transport::Server::builder()
         .layer(MetricsMiddlewareLayer::new(meta_metrics.clone()))
         .add_service(HealthServer::new(health_srv))
         .serve(address_info.listen_addr)
         .await
         .unwrap();
});
```

--------------------------------

### Launch RisingWave Cluster

Source: https://github.com/risingwavelabs/risingwave/blob/main/integration_tests/big-query-sink/README.md

Starts the RisingWave cluster and its dependencies using Docker Compose. Requires a GCP key file to be placed in the specified directory.

```sh
docker-compose up -d
```

--------------------------------

### Install perf (EC2)

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/benchmark-and-profile/microbenchmarks.md

Installs the `perf` performance analysis tool on EC2 instances, which may require specific package names.

```shell
sudo apt install linux-tools-aws
```

--------------------------------

### Install RisingWave Grafana Dashboard Dependencies

Source: https://github.com/risingwavelabs/risingwave/blob/main/grafana/README.md

Installs the necessary Python packages and sets up a virtual environment for generating the Grafana dashboard.

```bash
python3 -m venv 'venv'
source venv/bin/activate
pip install -r requirements.txt
```

--------------------------------

### Configure and Build Dashboard for Meta Node Deployment

Source: https://github.com/risingwavelabs/risingwave/blob/main/dashboard/README.md

Enables dashboard integration with the meta node and builds static files for standalone deployment.

```bash
./risedev configure enable dashboard
```

--------------------------------

### Enter Nix Shell

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Enters a Nix shell environment for automated dependency management.

```shell
nix develop ./develop/nix
```

--------------------------------

### Install Python 3.12 on Debian-based Linux

Source: https://github.com/risingwavelabs/risingwave/blob/main/docs/dev/src/build-and-run/intro.md

Installs Python version 3.12 on Debian-based Linux systems, required for Embedded Python UDF feature.

```shell
sudo apt install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.12 python3.12-dev
```