### Install Setup Tools

Source: https://github.com/ray-project/kuberay/blob/master/clients/python-apiserver-client/README.md

Install the setup-tools package using pip3. This is a prerequisite for local library installation.

```shell
pip3 install -U setuptools
```

--------------------------------

### Install Example ConfigMap

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/Autoscaling.md

Apply the provided ConfigMap to install the necessary code for the example. This ConfigMap contains the detached actor script.

```bash
kubectl apply -f test/cluster/cluster/detachedactor.yaml
```

--------------------------------

### Install and Start Local Ray

Source: https://github.com/ray-project/kuberay/blob/master/historyserver/DEVELOPMENT.md

Installs a specific version of Ray locally and starts the Ray head node, configuring it to route requests to the port-forwarded History Server.

```bash
pip uninstall -y ray
pip install -U "ray[default]==2.55.0"

ray start --head --num-cpus=1 --proxy-server-url=http://localhost:8080
```

--------------------------------

### Install KubeRay Python Client

Source: https://github.com/ray-project/kuberay/blob/master/clients/python-client/python_client_test/README.md

Install or upgrade pip and setuptools before installing the client. Navigate to the client directory and install it in editable mode.

```bash
pip install -U pip setuptools
```

```bash
cd clients/python-client && pip install -e .
```

--------------------------------

### Install Secure KubeRay API Server

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/DEVELOPMENT.md

Installs the KubeRay API server using the configuration from deploy/base for a secure setup. Ensure security proxy is built and loaded if needed.

```bash
#To use the configuration from deploy/base for secure API server
make install-secure
```

--------------------------------

### Install Python Client

Source: https://github.com/ray-project/kuberay/blob/master/clients/python-client/README.md

Install the Python client library using pip. Ensure setuptools is up-to-date.

```bash
pip install -U pip setuptools
```

--------------------------------

### Install KubeRay Operator and RayCluster

Source: https://github.com/ray-project/kuberay/blob/master/docs/development/release.md

Install the KubeRay operator and a sample RayCluster using Helm. Replace `<new-version>` with the actual chart version you want to install. This example uses Kind for cluster creation.

```bash
# Example using Kind
kind create cluster

# Install the operator using the specific version
helm install kuberay-operator kuberay/kuberay-operator --version <new-version>

# Install a sample RayCluster using the specific chart version
helm install raycluster kuberay/ray-cluster --version <new-version>
```

--------------------------------

### Install Code ConfigMap

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/HACluster.md

Deploy a ConfigMap containing example code. For production, package user code within a Docker image.

```bash
kubectl apply -f code_configmap.yaml
```

--------------------------------

### Install Insecure KubeRay API Server

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/DEVELOPMENT.md

Installs the KubeRay API server using the configuration from deploy/base for an insecure setup.

```bash
#To use the configuration from deploy/base for insecure API server
make install
```

--------------------------------

### Install Go Version Manager

Source: https://github.com/ray-project/kuberay/blob/master/ray-operator/DEVELOPMENT.md

Installs and sets up a specific Go version for development. Ensure this version is compatible with KubeRay.

```bash
go install golang.org/dl/go1.25.0@latest
go1.25.0 download
export GOROOT=$(go1.25.0 env GOROOT)
export PATH="$GOROOT/bin:$PATH"
```

--------------------------------

### Example: Get Service Request

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/HttpRequestSpec.md

This `curl` command demonstrates how to make a GET request to retrieve a service named 'test-v2' in the 'default' namespace.

```sh
curl --silent -X 'GET' \
  'http://localhost:31888/apis/v1/namespaces/default/services/test-v2' \
  -H 'accept: application/json'
```

--------------------------------

### Install Prometheus

Source: https://github.com/ray-project/kuberay/blob/master/benchmark/memory_benchmark/memory_benchmark.md

Installs Prometheus using a provided shell script. This is part of the setup for monitoring the benchmark experiments.

```sh
# Path: kuberay/
./install/prometheus/install.sh
```

--------------------------------

### Start Ray Locally

Source: https://github.com/ray-project/kuberay/blob/master/config/grafana/README.md

Start a local Ray instance to access the metrics and dashboards. Ensure Ray is running before copying dashboard files.

```bash
ray start --head
```

--------------------------------

### Start Development Server with Bun

Source: https://github.com/ray-project/kuberay/blob/master/dashboard/README.md

Use this command to start the development server for the KubeRay project using Bun. Open http://localhost:3000 in your browser to view the application.

```bash
bun dev
```

--------------------------------

### Install Ray

Source: https://github.com/ray-project/kuberay/blob/master/config/grafana/README.md

Install the Ray library with default extras. This is a prerequisite for updating the dashboards.

```bash
pip install "ray[default]"
```

--------------------------------

### Start Local KubeRay APIServer

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/README.md

Build and start the KubeRay APIServer from scratch in a local environment, deploying necessary components to a local Kind cluster.

```sh
make start-local-apiserver
```

--------------------------------

### Install KubeRay APIServer (Stable, No Security Proxy)

Source: https://github.com/ray-project/kuberay/blob/master/apiserversdk/docs/installation.md

Installs a stable version of KubeRay APIServer without the security proxy using Helm. Ensure Helm is installed and the KubeRay repo is added.

```sh
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update
# Install KubeRay APIServer without security proxy
helm install kuberay-apiserver kuberay/kuberay-apiserver --version 1.4.0 --set security=null
```

--------------------------------

### Install Prometheus Stack via Helm Chart

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/Monitoring.md

Navigate to the install/prometheus directory and execute the install script to set up the Prometheus Stack on a kind cluster. This command installs Prometheus, Grafana, and related components.

```shell
cd install/prometheus
bash install.sh
```

--------------------------------

### Spin up Kind Cluster and Run KubeRay Operator

Source: https://github.com/ray-project/kuberay/blob/master/historyserver/docs/set_up_collector.md

Sets up a Kind cluster, builds the KubeRay operator image, loads it into the cluster, and installs the operator using Helm. Ensure you have Go v1.24+, Docker, and Make installed.

```bash
git clone https://github.com/ray-project/kuberay.git
cd kuberay

kind create cluster --image=kindest/node:v1.29.0

IMG=kuberay/operator:latest make -C ray-operator docker-build

kind load docker-image kuberay/operator:latest

helm install kuberay-operator \
  --set image.repository=kuberay/operator \
  --set image.tag=latest \
  ./helm-chart/kuberay-operator

kubectl logs deployments/kuberay-operator
# or
k9s
```

--------------------------------

### Install APIServer with Security using Helm

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/SecuringImplementation.md

This Helm command installs the KubeRay APIServer with security configurations. Ensure you are in the helm-chart directory.

```bash
cd helm-chart
helm install apiserver kuberay-apiserver
```

--------------------------------

### Helm Chart Installation

Source: https://context7.com/ray-project/kuberay/llms.txt

Instructions for installing the KubeRay operator using Helm, including default and custom configurations.

```APIDOC
## Helm Chart Installation

Install the KubeRay operator using Helm with customizable configuration options.

### Installation Commands

```bash
# Add the KubeRay Helm repository
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update

# Install the operator with default settings
helm install kuberay-operator kuberay/kuberay-operator --namespace ray-system --create-namespace

# Install with custom values
helm install kuberay-operator kuberay/kuberay-operator \
  --namespace ray-system \
  --create-namespace \
  --set replicas=2 \
  --set image.tag=v1.3.0 \
  --set leaderElectionEnabled=true \
  --set metrics.enabled=true \
  --set batchScheduler.name=volcano
```

### Custom `values.yaml` Example

```yaml
# values.yaml for custom Helm installation
image:
  repository: quay.io/kuberay/operator
  tag: v1.3.0
  pullPolicy: IfNotPresent

replicas: 1
leaderElectionEnabled: true

# Batch scheduler integration (volcano, yunikorn, scheduler-plugins, kai-scheduler)
batchScheduler:
  name: ""

featureGates:
- name: RayClusterStatusConditions
  enabled: true
- name: RayJobDeletionPolicy
  enabled: true
- name: RayMultiHostIndexing
  enabled: true
- name: RayServiceIncrementalUpgrade
  enabled: false
- name: RayCronJob
  enabled: false

metrics:
  enabled: true
  serviceMonitor:
    enabled: false
    interval: 30s

configuration:
  enabled: false
  defaultContainerEnvs: []
  headSidecarContainers: []
  workerSidecarContainers: []

resources:
  limits:
    cpu: 100m
    memory: 512Mi

kubeClient:
  qps: 100.0
  burst: 200
```
```

--------------------------------

### Start Local Kuberay API Server

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/DEVELOPMENT.md

Starts the Kuberay API server on your development machine with the Go race detector enabled. Requires a connection to a Kubernetes cluster.

```bash
make run
```

--------------------------------

### Install KubeRay APIServer (Nightly, With Security Proxy)

Source: https://github.com/ray-project/kuberay/blob/master/helm-chart/kuberay-apiserver/README.md

Installs the nightly build of KubeRay APIServer from a local chart, with the default security proxy enabled. Ensure you have cloned the KubeRay repository and navigated to the chart directory.

```sh
# Step1: Clone KubeRay repository

# Step2: Navigate to `helm-chart/kuberay-apiserver`
cd helm-chart/kuberay-apiserver

# Step3: Install the KubeRay APIServer
helm install kuberay-apiserver .
```

--------------------------------

### Install Python Client in Editable Mode

Source: https://github.com/ray-project/kuberay/blob/master/clients/python-client/README.md

Install the Python client in editable mode from the client's directory. This is useful for development.

```bash
pip install -e .
```

--------------------------------

### Install KubeRay APIServer (Stable, With Security Proxy)

Source: https://github.com/ray-project/kuberay/blob/master/apiserversdk/docs/installation.md

Installs a stable version of KubeRay APIServer with the security proxy enabled using Helm. Note that the security proxy is experimental.

```sh
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update
# Install KubeRay APIServer
helm install kuberay-apiserver kuberay/kuberay-apiserver --version 1.4.0
```

--------------------------------

### Install KubeRay APIServer (Nightly, With Security Proxy)

Source: https://github.com/ray-project/kuberay/blob/master/apiserversdk/docs/installation.md

Installs the nightly version of KubeRay APIServer with the security proxy enabled. This requires cloning the repository and navigating to the Helm chart directory.

```sh
# Step1: Clone KubeRay repository

# Step2: Navigate to `helm-chart/kuberay-apiserver`
cd helm-chart/kuberay-apiserver

# Step3: Install the KubeRay apiserver
helm install kuberay-apiserver .
```

--------------------------------

### Example Output of 'make test'

Source: https://github.com/ray-project/kuberay/blob/master/ray-operator/DEVELOPMENT.md

This is an example of the console output you might see when running the 'make test' command, showing test execution progress and coverage.

```console
✗ make test
... go fmt ./...
go vet ./...
... setting up env vars
?    github.com/ray-project/kuberay/ray-operator [no test files]
ok   github.com/ray-project/kuberay/ray-operator/api/v1alpha1 0.023s coverage: 0.9% of statements
ok   github.com/ray-project/kuberay/ray-operator/controllers 9.587s coverage: 66.8% of statements
ok   github.com/ray-project/kuberay/ray-operator/controllers/common 0.016s coverage: 75.6% of statements
ok   github.com/ray-project/kuberay/ray-operator/controllers/utils 0.015s coverage: 31.4% of statements
```

--------------------------------

### Install KubeRay Python Client Locally

Source: https://github.com/ray-project/kuberay/blob/master/clients/python-apiserver-client/README.md

Install the KubeRay Python client in editable mode from its local directory. Ensure you are in the correct path.

```shell
pip3 install -e .
```

--------------------------------

### Install KubeRay Plugin with Krew

Source: https://github.com/ray-project/kuberay/blob/master/kubectl-plugin/DEVELOPMENT.md

Move the compiled KubeRay kubectl plugin executable to the Krew plugin directory for installation. This assumes Krew is used for plugin management.

```console
mv kubectl-ray ~/.krew/bin
```

--------------------------------

### Start Kuberay API Server Executable

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/DEVELOPMENT.md

This command starts the Kuberay API Server executable after it has been built. The server listens on port 8888 for HTTP and 8887 for RPC.

```bash
../bin/kuberay-apiserver
```

--------------------------------

### Install KubeRay APIServer (Nightly, No Security Proxy)

Source: https://github.com/ray-project/kuberay/blob/master/apiserversdk/docs/installation.md

Installs the nightly version of KubeRay APIServer without the security proxy. This involves cloning the repository and navigating to the Helm chart directory.

```sh
# Step1: Clone KubeRay repository

# Step2: Navigate to `helm-chart/kuberay-apiserver`
cd helm-chart/kuberay-apiserver

# Step3: Install the KubeRay apiserver
helm install kuberay-apiserver . --set security=null
```

--------------------------------

### Start KubeRay Dashboard Development Server

Source: https://github.com/ray-project/kuberay/blob/master/dashboard/README.md

Commands to start the KubeRay dashboard development server using npm, yarn, or pnpm. These commands are used for local development and hot-reloading.

```bash
npm run dev
```

```bash
yarn dev
```

```bash
pnpm dev
```

--------------------------------

### Verify Docker Buildx Installation

Source: https://github.com/ray-project/kuberay/blob/master/ray-operator/DEVELOPMENT.md

Check if the Docker Buildx plugin is installed and functioning correctly. This is a prerequisite for building multi-architecture container images.

```console
docker buildx version
```

--------------------------------

### Install and Search Kuberay Helm Charts

Source: https://github.com/ray-project/kuberay/blob/master/docs/release/helm-chart.md

Commands to add the Kuberay Helm repository, update local repository information, search for available charts, and install specific charts.

```sh
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update

# List all charts
helm search repo kuberay

# Install charts (If you want to install charts for a release candidate, add "--version vX.Y.Z-rc.0" to the command below.)
helm install kuberay-operator kuberay/kuberay-operator
helm install kuberay-apiserver kuberay/kuberay-apiserver
helm install raycluster kuberay/ray-cluster
```

--------------------------------

### Install KubeRay Operator using Kustomize (Stable)

Source: https://github.com/ray-project/kuberay/blob/master/docs/deploy/installation.md

Installs the CRD and KubeRay operator using Kustomize with a specific stable version tag. Ensure the correct ref is used.

```sh
# Install CRD and KubeRay operator
kubectl create -k "github.com/ray-project/kuberay/ray-operator/config/default?ref=v1.1.0&timeout=90s"
```

--------------------------------

### Install KubeRay Operator with Helm Override

Source: https://github.com/ray-project/kuberay/blob/master/ray-operator/config/overlays/test-overrides/README.md

This command demonstrates how to install the KubeRay operator using Helm, applying a specific override values file for CI release automation.

```bash
helm install kuberay-operator kuberay/kuberay-operator -f .buildkite/values-kuberay-operator-override.yaml
```

--------------------------------

### Ray Job Execution Log Example

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/JobSubmission.md

Example output of a Ray job's execution log, showing worker connection details, dashboard access information, and application-specific output like counter increments.

```text
2023-11-08 03:24:31,904	INFO worker.py:1329 -- Using address 10.244.2.2:6379 set in the environment variable RAY_ADDRESS
2023-11-08 03:24:31,905	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: 10.244.2.2:6379...
2023-11-08 03:24:31,921	INFO worker.py:1633 -- Connected to Ray cluster. View the dashboard at 10.244.2.2:8265
test_counter got 1
test_counter got 2
test_counter got 3
test_counter got 4
test_counter got 5
```

--------------------------------

### Set Up Kind and KubeRay Operator

Source: https://github.com/ray-project/kuberay/blob/master/historyserver/DEVELOPMENT.md

Initializes a Kind cluster, builds and loads the KubeRay operator image, and installs the operator via Helm. Ensure you are in the 'kuberay' directory before running.

```bash
# Spin up the kind cluster.
kind create cluster --image=kindest/node:v1.29.0

# Build and load the operator.
IMG=kuberay/operator:latest make -C ray-operator docker-build
kind load docker-image kuberay/operator:latest

# Install via Helm.
helm install kuberay-operator \
  --set image.repository=kuberay/operator \
  --set image.tag=latest \
  ./helm-chart/kuberay-operator
```

--------------------------------

### Copy Dashboards

Source: https://github.com/ray-project/kuberay/blob/master/config/grafana/README.md

Copy the latest Grafana dashboards from the Ray installation to the current directory. This command assumes Ray has been installed and started.

```bash
cp -r /tmp/ray/session_latest/metrics/grafana/dashboards/* .
```

--------------------------------

### Delete All Ray Clusters

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/Autoscaling.md

Use this command to remove all existing Ray clusters before starting the example to ensure a clean execution.

```bash
kubectl delete raycluster --all
```

--------------------------------

### Run Local KubeRay Documentation Preview

Source: https://github.com/ray-project/kuberay/blob/master/docs/development/development.md

Use this command to run a local preview of the KubeRay documentation. Ensure Docker is installed and navigate to the repository root. The preview will auto-update on file changes.

```sh
docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material
```

--------------------------------

### Install KubeRay APIServer using Helm

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/README.md

Deploys the KubeRay APIServer to a specified namespace using Helm. Ensure the Helm repository is added first.

```bash
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm -n ray-system install kuberay-apiserver kuberay/kuberay-apiserver -n ray-system --create-namespace
```

--------------------------------

### Example: Get Service Response

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/HttpRequestSpec.md

This JSON object represents the successful response when retrieving details for a Ray service. It includes configuration, cluster specifications, and status events.

```json
{
   "name":"test-v2",
   "namespace":"default",
   "user":"user",
   "serveConfigV2":"applications:\n  - name: fruit_app\n    import_path: fruit.deployment_graph\n    route_prefix: /fruit\n    runtime_env:\n      working_dir: \"https://github.com/ray-project/test_dag/archive/78b4a5da38796123d9f9ffff59bab2792a043e95.zip\"
    deployments:\n      - name: MangoStand\n        num_replicas: 2\n        max_replicas_per_node: 1\n        user_config:\n          price: 3\n        ray_actor_options:\n          num_cpus: 0.1\n      - name: OrangeStand\n        num_replicas: 1\n        user_config:\n          price: 2\n        ray_actor_options:\n          num_cpus: 0.1\n      - name: PearStand\n        num_replicas: 1\n        user_config:\n          price: 1\n        ray_actor_options:\n          num_cpus: 0.1\n      - name: FruitMarket\n        num_replicas: 1\n        ray_actor_options:\n          num_cpus: 0.1\n  - name: math_app\n    import_path: conditional_dag.serve_dag\n    route_prefix: /calc\n    runtime_env:\n      working_dir: \"https://github.com/ray-project/test_dag/archive/78b4a5da38796123d9f9ffff59bab2792a043e95.zip\"
    deployments:\n      - name: Adder\n        num_replicas: 1\n        user_config:\n          increment: 3\n        ray_actor_options:\n          num_cpus: 0.1\n      - name: Multiplier\n        num_replicas: 1\n        user_config:\n          factor: 5\n        ray_actor_options:\n          num_cpus: 0.1\n      - name: Router\n        num_replicas: 1\n",
   "clusterSpec":{
      "headGroupSpec":{
         "computeTemplate":"default-template",
         "image":"rayproject/ray:2.46.0-py310",
         "serviceType":"NodePort",
         "rayStartParams":{
            "dashboard-host":"0.0.0.0",
            "metrics-export-port":"8080"
         },
         "environment":{

         }
      },
      "workerGroupSpec":[
         {
            "groupName":"small-wg",
            "computeTemplate":"default-template",
            "image":"rayproject/ray:2.46.0-py310",
            "replicas":1,
            "minReplicas":1,
            "maxReplicas":5,
            "rayStartParams":{
               "node-ip-address":"$MY_POD_IP"
            },
            "environment":{

            }
         }
      ]
   },
   "rayServiceStatus":{
      "rayServiceEvents":[
         {
            "id":"test-v2.17ab175ec787d26e",
            "name":"test-v2-test-v2.17ab175ec787d26e",
            "createdAt":"2024-01-17T09:09:39Z",
            "firstTimestamp":"2024-01-17T09:09:39Z",
            "lastTimestamp":"2024-01-17T09:10:04Z",
            "reason":"ServiceNotReady",
            "message":"The service is not ready yet. Controller will perform a round of actions in 2s.",
            "type":"Normal",
            "count":13
         },
         {
            "id":"test-v2.17ab176292337141",
            "name":"test-v2-test-v2.17ab176292337141",
            "createdAt":"2024-01-17T09:09:56Z",
            "firstTimestamp":"2024-01-17T09:09:56Z",
            "lastTimestamp":"2024-01-17T09:09:56Z",
            "reason":"SubmittedServeDeployment",
            "message":"Controller sent API request to update Serve deployments on cluster test-v2-raycluster-lmk5c",
            "type":"Normal",
            "count":1
         },
         {
            "id":"test-v2.17ab1764fe827fe3",
            "name":"test-v2-test-v2.17ab1764fe827fe3",
            "createdAt":"2024-01-17T09:10:06Z",
            "firstTimestamp":"2024-01-17T09:10:06Z",
            "lastTimestamp":"2024-01-17T09:14:41Z",
            "reason":"Running",
            "message":"The Serve applicaton is now running and healthy.",
            "type":"Normal",
            "count":139
         },
         {
            "id":"test-v2.17ab1815ce6a98da",
            "name":"test-v2-test-v2.17ab1815ce6a98da",
            "createdAt":"2024-01-17T09:22:45Z",
            "firstTimestamp":"2024-01-17T09:22:45Z",
            "lastTimestamp":"2024-01-17T09:23:16Z",
            "reason":"ServiceNotReady",
            "message":"The service is not ready yet. Controller will perform a round of actions in 2s.",
            "type":"Normal",
            "count":16
         },
         {
            "id":"test-v2.17ab181a8576812d",
            "name":"test-v2-test-v2.17ab181a8576812d",
            "createdAt":"2024-01-17T09:23:06Z",
            "firstTimestamp":"2024-01-17T09:23:06Z",
            "lastTimestamp":"2024-01-17T09:23:06Z",
            "reason":"SubmittedServeDeployment",
            "message":"Controller sent API request to update Serve deployments on cluster test-v2-raycluster-b85bj",
            "type":"Normal",

```

--------------------------------

### Verify Kuberay API Server Health

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/DEVELOPMENT.md

Use this command to verify the KubeRay API server setup. Ensure the KubeRay Operator and API server are installed before running.

```bash
curl -I localhost:31888/healthz
```

--------------------------------

### Get Ray Job Details

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/JobSubmission.md

Retrieve detailed information about a submitted Ray job using its `submissionId`. This includes the job's status, start and end times, and runtime environment.

```bash
curl -X GET 'localhost:31888/apis/v1/namespaces/default/jobsubmissions/test-cluster/<submissionID>' \
  --header 'Content-Type: application/json'
```

--------------------------------

### Get ConfigMaps

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/Autoscaling.md

Verify that the ConfigMap has been successfully created by listing all ConfigMaps in the current namespace. Look for 'ray-example'.

```bash
kubectl get configmaps
```

--------------------------------

### Helm Chart Custom Values

Source: https://context7.com/ray-project/kuberay/llms.txt

Example `values.yaml` file for customizing KubeRay Helm chart installation. Configure image repository, tag, replicas, feature gates, and batch scheduler integration.

```yaml
# values.yaml for custom Helm installation
image:
  repository: quay.io/kuberay/operator
  tag: v1.3.0
  pullPolicy: IfNotPresent

replicas: 1
leaderElectionEnabled: true

# Batch scheduler integration (volcano, yunikorn, scheduler-plugins, kai-scheduler)
batchScheduler:
  name: ""

featureGates:
- name: RayClusterStatusConditions
  enabled: true
- name: RayJobDeletionPolicy
  enabled: true
- name: RayMultiHostIndexing
  enabled: true
- name: RayServiceIncrementalUpgrade
  enabled: false
- name: RayCronJob
  enabled: false

metrics:
  enabled: true
  serviceMonitor:
    enabled: false
    interval: 30s

configuration:
  enabled: false
  defaultContainerEnvs: []
  headSidecarContainers: []
  workerSidecarContainers: []

resources:
  limits:
    cpu: 100m
    memory: 512Mi

kubeClient:
  qps: 100.0
  burst: 200

```

--------------------------------

### Install Kuberay CRDs Separately

Source: https://github.com/ray-project/kuberay/blob/master/docs/deploy/helm.md

Installs only the Custom Resource Definitions (CRDs) for Kuberay. This is useful when different roles or permissions are required for CRD installation versus operator installation.

```bash
kubectl create -k "github.com/ray-project/kuberay/ray-operator/config/crd?ref=v1.1.0&timeout=90s"
```

--------------------------------

### Install Prometheus and Grafana

Source: https://github.com/ray-project/kuberay/blob/master/historyserver/docs/set_up_historyserver.md

Installs Prometheus and Grafana for monitoring Ray clusters. This script assumes Helm is installed and configured.

```bash
./install/prometheus/install.sh --auto-load-dashboard true
```

--------------------------------

### Install Kuberay Operator without CRDs via Helm

Source: https://github.com/ray-project/kuberay/blob/master/docs/deploy/helm.md

Installs the Kuberay operator using Helm, explicitly skipping the installation of CRDs. This is intended for scenarios where CRDs are installed separately, potentially by a cluster administrator.

```bash
helm install kuberay-operator kuberay/kuberay-operator --version 1.1.0 --skip-crds
```

--------------------------------

### Helm Chart Installation Commands

Source: https://context7.com/ray-project/kuberay/llms.txt

Install the KubeRay operator using Helm. Add the repository, update it, and install the operator with default or custom values.

```bash
# Add the KubeRay Helm repository
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update

# Install the operator with default settings
helm install kuberay-operator kuberay/kuberay-operator --namespace ray-system --create-namespace

# Install with custom values
helm install kuberay-operator kuberay/kuberay-operator \
  --namespace ray-system \
  --create-namespace \
  --set replicas=2 \
  --set image.tag=v1.3.0 \
  --set leaderElectionEnabled=true \
  --set metrics.enabled=true \
  --set batchScheduler.name=volcano

```

--------------------------------

### Install Nightly Kuberay Operator via Helm

Source: https://github.com/ray-project/kuberay/blob/master/docs/deploy/helm.md

Installs the latest nightly build of the Kuberay operator. This involves cloning the repository, navigating to the chart directory, and then installing.

```bash
# Step1: Clone KubeRay repository

# Step2: Move to `helm-chart/kuberay-operator`

# Step3: Install KubeRay operator
helm install kuberay-operator .
```

--------------------------------

### Install KubeRay Operator with Helm

Source: https://github.com/ray-project/kuberay/blob/master/ray-operator/DEVELOPMENT.md

Installs the KubeRay operator using a local Helm chart, specifying the custom image repository and tag. This also installs necessary CRDs.

```bash
# Command: helm install kuberay-operator --set image.repository={IMG_REPO} --set image.tag={IMG_TAG} ../helm-chart/kuberay-operator
helm install kuberay-operator --set image.repository=kuberay/operator --set image.tag=nightly ../helm-chart/kuberay-operator
```

--------------------------------

### Get KubeRay Compute Templates via HTTP GET

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/README.md

Retrieve a list of compute templates using an HTTP GET request to the KubeRay APIServer. This verifies the successful creation of templates.

```sh
curl --silent -X 'GET' \
    'http://localhost:31888/apis/v1/namespaces/ray-system/compute_templates' \
    -H 'accept: application/json'
```

--------------------------------

### Run KubeRay History Server

Source: https://github.com/ray-project/kuberay/blob/master/historyserver/README.md

Starts the history server service. Configure storage backend and log directory using command-line flags.

```bash
./output/bin/historyserver \
  --runtime-class-name=s3 \
  --ray-root-dir=/path/to/logs
```

--------------------------------

### Create and Bootstrap Docker Buildx Builder

Source: https://github.com/ray-project/kuberay/blob/master/ray-operator/DEVELOPMENT.md

If a suitable Docker Buildx builder instance is not found, use this command to create a new one and set it as the default. This prepares your environment for multi-architecture builds.

```console
docker buildx create --use --bootstrap
```

--------------------------------

### Install KubeRay Operator and RayCluster Helm Charts

Source: https://github.com/ray-project/kuberay/blob/master/docs/deploy/helm-cluster.md

Installs the KubeRay operator and a RayCluster custom resource using Helm. Ensure you have Helm installed and have added the KubeRay repository. For arm64 users, specify the image tag.

```sh
# Step 1: Create a KinD cluster
kind create cluster

# Step 2: Register a Helm chart repo
helm repo add kuberay https://ray-project.github.io/kuberay-helm/

# Step 3: Install both CRDs and KubeRay operator v1.1.0.
helm install kuberay-operator kuberay/kuberay-operator --version 1.1.0

# Step 4: Install a RayCluster custom resource
# (For x86_64 users)
helm install raycluster kuberay/ray-cluster --version 1.1.0
# (For arm64 users, e.g. Mac M1)
# See here for all available arm64 images: https://hub.docker.com/r/rayproject/ray/tags?page=1&name=aarch64
helm install raycluster kuberay/ray-cluster --version 1.1.0 --set image.tag=nightly-aarch64
```

--------------------------------

### Build and Deploy KubeRay Operator and API Server

Source: https://github.com/ray-project/kuberay/blob/master/clients/python-apiserver-client/README.md

Execute make commands to build operator images, load them into a Kind cluster, deploy the operator, and prepare the API server. This is part of the testing setup.

```shell
make operator-image cluster load-operator-image deploy-operator docker-image load-image install
```

--------------------------------

### Install KubeRay CRDs

Source: https://github.com/ray-project/kuberay/blob/master/ray-operator/DEVELOPMENT.md

Installs the Custom Resource Definitions (CRDs) for KubeRay. This is a prerequisite for deploying KubeRay components.

```bash
make install
```

--------------------------------

### Get Ray Resources with kubectl

Source: https://context7.com/ray-project/kuberay/llms.txt

Retrieve information about Ray clusters, jobs, and services using kubectl get.

```bash
kubectl get raycluster
```

```bash
kubectl get raycluster raycluster-kuberay -o yaml
```

```bash
kubectl get rayjob
```

```bash
kubectl get rayjob rayjob-sample -o wide
```

```bash
kubectl get rayservice
```

```bash
kubectl get rayservice rayservice-sample -o jsonpath='{.status}'
```

--------------------------------

### Build Kuberay API Server Executable

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/DEVELOPMENT.md

Use this command to build the Kuberay API Server executable. Ensure you are in the project root directory.

```bash
make build
```

--------------------------------

### Install PyGithub Dependency

Source: https://github.com/ray-project/kuberay/blob/master/docs/release/changelog.md

Install the PyGithub library, which is required for interacting with the GitHub API to fetch pull request information.

```console
pip install PyGithub
```

--------------------------------

### Install KubeRay Operator with Helm

Source: https://github.com/ray-project/kuberay/blob/master/benchmark/perf-tests/README.md

Install the KubeRay operator using Helm. Specify the chart name, repository, and version.

```sh
helm install kuberay-operator kuberay/kuberay-operator --version 1.1.0

```

--------------------------------

### Argo CD Application for Kuberay Operator (Skip CRDs)

Source: https://github.com/ray-project/kuberay/blob/master/docs/deploy/helm.md

Defines an Argo CD Application to install the Kuberay operator using a Helm chart, with CRDs installation skipped. This requires Argo CD v2.3.0+ and is intended to be used alongside a separate CRD installation.

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: ray-operator
spec:
  source:
    repoURL: https://github.com/ray-project/kuberay
    targetRevision: v1.0.0-rc.0
    path: helm-chart/kuberay-operator
    helm:
      skipCrds: true
  destination:
    server: https://kubernetes.default.svc
    namespace: ray-operator
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
...
```

--------------------------------

### Build and Deploy KubeRay Components Locally

Source: https://github.com/ray-project/kuberay/blob/master/apiserver/DEVELOPMENT.md

These commands are used to build new API server and operator images, create a local Kind cluster, load the images into the cluster, and deploy the operator. Use these for initial setup or after making changes to the core components.

```bash
make docker-image cluster load-image deploy
```

```bash
make docker-image operator-image cluster load-image load-operator-image deploy deploy-operator
```