KWOK

https://kwok.sigs.k8s.io
Admin
KWOK (Kubernetes WithOut Kubelet) is a toolkit for setting up a cluster of thousands of simulated...
Tokens:48,032
Snippets:659
Trust Score:9.4
Update:4 months ago
Show doc for...
Tokens:
Raw
### kwokctl Start Help Options

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_start

Displays the help message for the start command, detailing available flags.

```bash
  -h, --help   help for start

```

--------------------------------

### Basic KWOK GitHub Action Setup

Source: https://kwok.sigs.k8s.io/docs/user/github-action

Use this snippet to set up KWOK in your GitHub Actions. Specify the command to install ('kwok' or 'kwokctl') and optionally the KWOK version.

```yaml
- uses: kubernetes-sigs/kwok@main
  with:
    # Required: The command to install ('kwok' or 'kwokctl')
    command: ''
    # Optional: Specific version of command to install, defaults to latest release
    kwok-version: ''

```

--------------------------------

### Setup Audit Policy

Source: https://kwok.sigs.k8s.io/docs/user/kwokctl-auditing

Create an audit policy YAML file to define auditing rules. This example sets the audit level to Metadata.

```bash
cat <<EOF > audit-policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
EOF

```

--------------------------------

### Cluster Start Options

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_start_cluster

These options are specific to starting a cluster. Use the help flag for detailed information on each option.

```bash
  -h, --help               help for cluster
      --timeout duration   Timeout for waiting for the cluster to be started
      --wait duration      Wait for the cluster to be ready
```

--------------------------------

### Kubectl Output Example

Source: https://kwok.sigs.k8s.io/docs/user/all-in-one-image

Example output from 'kubectl get ns' command, showing the default namespaces and their active status.

```yaml
NAME              STATUS   AGE
default           Active   1s
kube-node-lease   Active   1s
kube-public       Active   1s
kube-system       Active   1s

```

--------------------------------

### kwokctl Start Command Usage

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_start

This is the general syntax for starting a cluster using the kwokctl command. It outlines the basic structure for invoking the start command.

```bash
kwokctl start [command] [flags]

```

--------------------------------

### Install KWOK with Golang

Source: https://kwok.sigs.k8s.io/docs/user/installation

Installs `kwokctl` and `kwok` binaries using Golang, fetching the latest release tag from the GitHub repository.

```bash
# KWOK repository
KWOK_REPO=kubernetes-sigs/kwok
# Get latest
KWOK_LATEST_RELEASE=$(curl "https://api.github.com/repos/${KWOK_REPO}/releases/latest" | jq -r '.tag_name')

go install sigs.k8s.io/kwok/cmd/{kwok,kwokctl}@${KWOK_LATEST_RELEASE}

```

--------------------------------

### Example Context Output

Source: https://kwok.sigs.k8s.io/docs/examples/performance/control-plane/multi-cluster

This is an example of the expected output after running 'kubectl config get-contexts' when two KWOK clusters are created.

```text
CURRENT   NAME           CLUSTER        AUTHINFO       NAMESPACE
          kwok-member1   kwok-member1   kwok-member1   
*         kwok-member2   kwok-member2   kwok-member2   
```

--------------------------------

### Apply Node Example

Source: https://kwok.sigs.k8s.io/docs/examples/argo

Apply a standard Kubernetes node example manifest. This ensures that node resources are correctly represented in the simulated cluster.

```bash
kubectl apply -f https://kwok.sigs.k8s.io/examples/node.yaml
```

--------------------------------

### Install Istio

Source: https://kwok.sigs.k8s.io/docs/examples/istio

Install Istio into the Kubernetes cluster using the `istioctl` command-line tool. The `-y` flag accepts all defaults.

```bash
istioctl install -y

```

--------------------------------

### Install KWOK with WinGet

Source: https://kwok.sigs.k8s.io/docs/user/installation

Installs the latest versions of `kwokctl` and `kwok` binaries on Windows using the WinGet package manager.

```bash
winget install --id=Kubernetes.kwok -e
winget install --id=Kubernetes.kwokctl -e

```

--------------------------------

### Start a Cluster Command

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_start_cluster

This is the base command to start a cluster using kwokctl. Use this command followed by flags to configure the cluster startup process.

```bash
kwokctl start cluster [flags]
```

--------------------------------

### Add KubeRay Helm Repo and Install Operator

Source: https://kwok.sigs.k8s.io/docs/examples/ray

Adds the KubeRay Helm repository and installs the KubeRay operator. Ensure Helm is installed and configured.

```bash
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm install kuberay-operator kuberay/kuberay-operator --version 1.4.2

```

--------------------------------

### Install KWOK with Homebrew

Source: https://kwok.sigs.k8s.io/docs/user/installation

Installs the latest versions of `kwokctl` and `kwok` binaries on Linux/macOS using the Homebrew package manager.

```bash
brew install kwok

```

--------------------------------

### Deploy KWOK and Pod Stages for Failure Injection

Source: https://kwok.sigs.k8s.io/docs/examples/injectfailure

This snippet shows the initial setup for the demo, including deploying KWOK, applying pod stage configurations, and injecting a specific failure into an initContainer.

```bash
# Demo: inject fault to initContainer in an existing minikube cluster
kubectl config use-context minikube

kubectl get nodes

# Deploy and install kwok
kubectl apply -f https://github.com/kubernetes-sigs/kwok/releases/download/v0.5.2/kwok.yaml --wait
kubectl get pod -n kube-system | grep kwok

# Deploy pod stages
kubectl apply -f https://github.com/kubernetes-sigs/kwok/releases/download/v0.5.2/stage-fast.yaml
kubectl apply -f https://github.com/kubernetes-sigs/kwok/raw/main/kustomize/stage/pod/chaos/pod-init-container-running-failed.yaml
kubectl get stages
```

--------------------------------

### Help for Components Command

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_components

Displays help information for the `kwokctl get components` command, including available flags.

```bash
kwokctl get components --help
```

--------------------------------

### KWOK CLI: kwokctl get Options

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get

This snippet shows the help flag for the `get` command. It is used to display help information for the command.

```bash
  -h, --help   help for get

```

--------------------------------

### Apply Node Configuration

Source: https://kwok.sigs.k8s.io/docs/examples/istio

Apply a Kubernetes node configuration file using `kubectl`. This example assumes the file is available at the provided URL.

```bash
kubectl apply -f https://kwok.sigs.k8s.io/examples/node.yaml

```

--------------------------------

### List Clusters with Help Flag

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_clusters

Display help information for the `kwokctl get clusters` command to understand all available options.

```bash
kwokctl get clusters -h
```

--------------------------------

### Download and Install kwok Binary

Source: https://kwok.sigs.k8s.io/docs/user/installation

Downloads the `kwok` binary for the current operating system and architecture from the GitHub releases page, makes it executable, and moves it to the system's PATH.

```bash
wget -O kwok -c "https://github.com/${KWOK_REPO}/releases/download/${KWOK_LATEST_RELEASE}/kwok-$(go env GOOS)-$(go env GOARCH)"
chmod +x kwok
sudo mv kwok /usr/local/bin/kwok

```

--------------------------------

### Test Argo Workflow

Source: https://kwok.sigs.k8s.io/docs/examples/argo

Submit a sample 'hello-world' workflow to Argo Workflows and monitor its execution. This verifies the Argo installation and KWOK simulation.

```bash
argo submit -n argo --watch https://raw.githubusercontent.com/argoproj/argo-workflows/master/examples/hello-world.yaml
```

--------------------------------

### Download and Install kwokctl Binary

Source: https://kwok.sigs.k8s.io/docs/user/installation

Downloads the `kwokctl` binary for the current operating system and architecture from the GitHub releases page, makes it executable, and moves it to the system's PATH.

```bash
wget -O kwokctl -c "https://github.com/${KWOK_REPO}/releases/download/${KWOK_LATEST_RELEASE}/kwokctl-$(go env GOOS)-$(go env GOARCH)"
chmod +x kwokctl
sudo mv kwokctl /usr/local/bin/kwokctl

```

--------------------------------

### VPA Configuration Example

Source: https://kwok.sigs.k8s.io/docs/examples/scalability/scale-using-vpa

Defines a VerticalPodAutoscaler resource targeting a Deployment named 'fake-app' with an 'Auto' update mode.

```yaml
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: fake-vpa
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: fake-app
  updatePolicy:
    updateMode: "Auto"

```

--------------------------------

### List Kubernetes Contexts

Source: https://kwok.sigs.k8s.io/docs/examples/performance/control-plane/multi-cluster

Verify the creation of KWOK-managed clusters by listing the available kubectl contexts. This helps confirm your multi-cluster setup.

```bash
kubectl config get-contexts
```

--------------------------------

### Deploy Argo Workflows

Source: https://kwok.sigs.k8s.io/docs/examples/argo

Create the 'argo' namespace and deploy Argo Workflows using its official installation manifest. This installs the Argo controller and UI.

```bash
kubectl create namespace argo
kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/download/v3.4.8/install.yaml
```

--------------------------------

### kwokctl create Command Structure

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_create

Basic structure for the `kwokctl create` command. Use this as a starting point for creating new clusters.

```bash
kwokctl create [command] [flags]
```

--------------------------------

### Deploy Sample Ray Job

Source: https://kwok.sigs.k8s.io/docs/examples/ray

Deploys a sample PyTorch MNIST Ray job to test the Kubernetes setup. This command uses a sample YAML file from the KubeRay repository.

```bash
kubectl apply -f https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/pytorch-mnist/ray-job.pytorch-mnist.yaml

```

--------------------------------

### Create Multiple Clusters with KWOK

Source: https://kwok.sigs.k8s.io/docs/examples/performance/control-plane/multi-cluster

Use this command to spin up multiple clusters for multi-cluster testing. Ensure KWOK is installed.

```bash
for ((i=1; i<=2; i ++)); do
    kwokctl create cluster --name member$i
done;
```

--------------------------------

### Deploy Kueue Controller-Manager

Source: https://kwok.sigs.k8s.io/docs/examples/kueue

Install the Kueue controller-manager using the provided manifests. This command deploys Kueue into your Kubernetes cluster.

```bash
kubectl apply --server-side -f https://github.com/kubernetes-sigs/kueue/releases/download/v0.14.4/manifests.yaml

```

--------------------------------

### Setup Cluster with DRA Enabled

Source: https://kwok.sigs.k8s.io/docs/examples/dra

Use this command to create a Kubernetes cluster with the Dynamic Resource Allocation feature gate enabled and the necessary runtime configuration for DRA.

```bash
kwokctl create cluster --kube-feature-gates="kube:DynamicResourceAllocation=true" --kube-runtime-config="resource.k8s.io/v1beta1=true"
```

--------------------------------

### Run All-in-one Cluster Image

Source: https://kwok.sigs.k8s.io/docs/user/all-in-one-image

Starts a KWOK all-in-one cluster image using Docker. This command exposes the cluster's API on port 8080.

```bash
docker run --rm -it -p 8080:8080 registry.k8s.io/kwok/cluster:v0.4.0-k8s.v1.28.0

```

--------------------------------

### List Pods with Wide Output

Source: https://kwok.sigs.k8s.io/docs/user/kwok-manage-nodes-and-pods

Get a detailed list of all pods in the current namespace, including their IP address and the node they are running on. This helps verify pod placement after creation.

```bash
kubectl get pod -o wide
```

--------------------------------

### Create Mock Ray Head ConfigMap

Source: https://kwok.sigs.k8s.io/docs/examples/ray

Creates a Kubernetes ConfigMap containing a Python script for a mock Ray head server. This script handles GET requests to simulate Ray job status.

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: mock-head
  namespace: default
data:
  server.py: |
    import json
    import http.server
    
    class MockRayHandler(http.server.BaseHTTPRequestHandler):
        """
        Handle GET /api/jobs/{job_id} requests for mock Ray head API
        """
        def do_GET(self):
            path = self.path.strip('/').split('/')
            
            if len(path) >= 3 and path[0] == 'api' and path[1] == 'jobs':
                self.send_response(200)
                self.send_header('Content-Type', 'application/json')
                self.end_headers()
                self.wfile.write(json.dumps({
                    "job_id": path[2],
                    "status": "SUCCEEDED"
                }).encode())
                return
            
            self.send_response(404)
            self.end_headers()
            self.wfile.write(b'Not Found')
    
    if __name__ == "__main__":
        http.server.HTTPServer(('', 8265), MockRayHandler).serve_forever()

```

--------------------------------

### Get Clusters Managed by kwokctl

Source: https://kwok.sigs.k8s.io/docs/user/kwokctl-manage-cluster

List all clusters currently managed by `kwokctl` using the `get clusters` command.

```bash
$ kwokctl get clusters
kwok

```

--------------------------------

### Create Deployment with Resource Claim Template

Source: https://kwok.sigs.k8s.io/docs/examples/dra

Creates a Deployment with a single replica that references a Resource Claim Template for GPU allocation. This is useful for scheduling pods that require specific hardware resources.

```bash
kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fake-pod
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fake-pod
  template:
    metadata:
      labels:
        app: fake-pod
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: type
                operator: In
                values:
                - kwok
      tolerations:
      - key: "kwok.x-k8s.io/node"
        operator: "Exists"
        effect: "NoSchedule"
      containers:
      - name: fake-container
        image: fake-image
        resources:
          claims:
          - name: gpu0
      resourceClaims:
      - name: gpu0
        resourceClaimTemplateName: single-gpu
EOF
```

--------------------------------

### Enable Istio Injection and Deploy Bookinfo Application

Source: https://kwok.sigs.k8s.io/docs/examples/istio

Enable Istio sidecar injection for the default namespace by applying a label. Then, deploy the Istio Bookinfo sample application.

```bash
kubectl label namespace default istio-injection=enabled
kubectl apply -f https://raw.githubusercontent.com/istio/istio/master/samples/bookinfo/platform/kube/bookinfo.yaml

```

--------------------------------

### KWOK CLI: kwokctl get Components Command

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get

This command lists the components of a KWOK cluster. It is a specific subcommand of `kwokctl get`.

```bash
kwokctl get components

```

--------------------------------

### KWOK CLI: kwokctl get Command Usage

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get

This is the general usage pattern for the `kwokctl get` command. It specifies the type of resource to retrieve.

```bash
kwokctl get [command] [flags]

```

--------------------------------

### kwokctl get artifacts Options

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_artifacts

These are the specific options available for the `kwokctl get artifacts` command, including filtering and specifying the cluster runtime.

```bash
      --filter string    Filter the list of (binary or image)
  -h, --help             help for artifacts
      --runtime string   Runtime of the cluster (binary or docker or finch or kind or kind-finch or kind-lima or kind-nerdctl or kind-podman or lima or nerdctl or podman)

```

--------------------------------

### Prepare Variables for Binary Release

Source: https://kwok.sigs.k8s.io/docs/user/installation

Sets up the `KWOK_REPO` and `KWOK_LATEST_RELEASE` variables, which are used to construct download URLs for KWOK binaries from GitHub releases.

```bash
# KWOK repository
KWOK_REPO=kubernetes-sigs/kwok
# Get latest
KWOK_LATEST_RELEASE=$(curl "https://api.github.com/repos/${KWOK_REPO}/releases/latest" | jq -r '.tag_name')

```

--------------------------------

### kwokctl get artifacts Command Usage

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_artifacts

This is the basic usage of the `kwokctl get artifacts` command. It lists binaries or images used by the cluster.

```bash
kwokctl get artifacts [flags]

```

--------------------------------

### Describe VPA Recommendations

Source: https://kwok.sigs.k8s.io/docs/examples/scalability/scale-using-vpa

Retrieves and displays the resource recommendations made by the VPA for the 'fake-vpa' resource.

```bash
kubectl describe vpa fake-vpa | awk '/Recommendation:/,/Events:/' | sed '$d'
```

--------------------------------

### Create a Sample Job with Kueue

Source: https://kwok.sigs.k8s.io/docs/examples/kueue

Apply a sample Kubernetes Job manifest that specifies a Kueue queue name and resource requests. Ensure the job is configured to be suspended initially.

```yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: sample-job
  namespace: default
  labels:
    kueue.x-k8s.io/queue-name: kwok-local-queue
spec:
  parallelism: 1
  completions: 1
  suspend: true
  template:
    spec:
      containers:
      - name: dummy-job
        image: gcr.io/k8s-staging-perf-tests/sleep:v0.1.0
        args: ["30s"]
        resources:
          requests:
            cpu: "1"
            memory: "200Mi"
      restartPolicy: Never
      nodeSelector:
        type: kwok
```

--------------------------------

### kwokctl get kubeconfig Options

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_kubeconfig

These are the specific options available for the `kwokctl get kubeconfig` command. They allow customization of the generated kubeconfig, such as specifying certificate groups, hosts, and user details.

```bash
--group strings              Signing certificate with the specified groups if modified (default [system:masters])
-h, --help                       help for kubeconfig
--host string                Override host[:port] for kubeconfig (default "127.0.0.1")
--insecure-skip-tls-verify   Skip server certificate verification
--user string                Signing certificate with the specified user if modified (default "kwok-admin")
```

--------------------------------

### Get Audit Logs

Source: https://kwok.sigs.k8s.io/docs/user/kwokctl-auditing

Retrieve audit logs from the `kwokctl` cluster.

```bash
kwokctl logs audit

```

--------------------------------

### Set Up Default Stage Custom Resources

Source: https://kwok.sigs.k8s.io/docs/user/kwok-in-cluster

Applies the default fast stage configuration for pod and node simulation, which is required for scenarios like scheduling and autoscaling.

```bash
kubectl apply -f "https://github.com/${KWOK_REPO}/releases/download/${KWOK_LATEST_RELEASE}/stage-fast.yaml"

```

--------------------------------

### Deploy Linkerd

Source: https://kwok.sigs.k8s.io/docs/examples/linkerd

Deploys Linkerd to the Kubernetes cluster, including CRDs and the main installation.

```bash
linkerd check --pre
linkerd install --crds | kubectl apply -f -
linkerd install | kubectl apply -f -

```

--------------------------------

### Download metrics-usage.yaml

Source: https://kwok.sigs.k8s.io/docs/examples/scalability/scale-using-hpa

Download the configuration file for the metrics server.

```bash
wget https://github.com/kubernetes-sigs/kwok/releases/download/v0.6.0/metrics-usage.yaml
```

--------------------------------

### Inherited Options for kwokctl

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_components

These options are inherited from the parent `kwokctl` command and can be used with `kwokctl get components`.

```bash
kwokctl --config ~/.kwok/kwok.yaml get components
```

```bash
kwokctl --dry-run get components
```

```bash
kwokctl --name my-cluster get components
```

```bash
kwokctl --v WARN get components
```

--------------------------------

### Build KWOK and KWOKCTL Binaries

Source: https://kwok.sigs.k8s.io/docs/contributing/development

Builds the `kwok` and `kwokctl` binaries locally. Binaries will be placed in `./bin/$(go env GOOS)/$(go env GOARCH)`. Platform-specific Kubernetes binaries may be required for non-Linux platforms.

```bash
make build

```

--------------------------------

### Build KWOK Image and KWOKCTL Binary with Docker

Source: https://kwok.sigs.k8s.io/docs/contributing/development

Builds the KWOK image and KWOKCTL binary using Docker. The resulting binaries will be in `./bin/$(go env GOOS)/$(go env GOARCH)` and the image will be tagged `localhost/kwok:${tag}`.

```bash
IMAGE_PREFIX=localhost BUILDER=docker make build build-image

```

--------------------------------

### Describe LimitRange Configuration

Source: https://kwok.sigs.k8s.io/docs/examples/scheduling/limit-range

Inspect the applied `LimitRange` configuration to confirm the CPU limits and minimums.

```bash
kubectl describe limitranges cpu-resource-constraint

```

--------------------------------

### Create a Kubernetes Cluster

Source: https://kwok.sigs.k8s.io/docs/examples/scheduling/node-affinity

Use `kwokctl` to create a new Kubernetes cluster for testing.

```bash
kwokctl create cluster

```

--------------------------------

### Get Current CoreDNS Configuration

Source: https://kwok.sigs.k8s.io/docs/examples/ray

Retrieves the current CoreDNS configuration from Kubernetes and saves it to a local YAML file for modification.

```bash
kubectl get configmap coredns -n kube-system -o yaml > coredns.yaml

```

--------------------------------

### Build Kubernetes Binaries from Source

Source: https://kwok.sigs.k8s.io/docs/user/kwokctl-platform-specific-binaries

Builds Kubernetes binaries from source for a specified version and configures kwokctl to use them. The binaries will be located in a specific cache directory.

```bash
KUBE_VERSION="v1.28.0"
SRC_DIR="${HOME}/.kwok/cache/kubernetes/${KUBE_VERSION}"
mkdir -p "${SRC_DIR}" && cd "${SRC_DIR}" &&
wget "https://dl.k8s.io/${KUBE_VERSION}/kubernetes-src.tar.gz" -O - | tar xz &&
make WHAT="cmd/kube-apiserver cmd/kube-controller-manager cmd/kube-scheduler" &&
cat <<EOF >> ~/.kwok/kwok.yaml
---
kind: KwokctlConfiguration
apiVersion: config.kwok.x-k8s.io/v1alpha1
options:
  kubeBinaryPrefix: $(pwd)/_output/local/bin/$(go env GOOS)/$(go env GOARCH)
---
EOF

```

--------------------------------

### Scale KWOK Nodes

Source: https://kwok.sigs.k8s.io/docs/examples/scheduling/pod-topology-spread-constraint

Creates a specified number of worker nodes for the KWOK cluster. This example scales to 4 nodes.

```bash
kwokctl scale node --replicas 4

```

--------------------------------

### Create Cluster with Binary Runtime

Source: https://kwok.sigs.k8s.io/docs/contributing/development

Creates a Kubernetes cluster using `kwokctl` with the `binary` runtime. Requires the `kwok` binary to be built separately and specified via `--kwok-controller-binary`.

```bash
./bin/$(go env GOOS)/$(go env GOARCH)/kwokctl create cluster \
  --runtime=binary \
  --kwok-controller-binary=./bin/$(go env GOOS)/$(go env GOARCH)/kwok

```

--------------------------------

### Set Up Default Resource Usage Custom Resources

Source: https://kwok.sigs.k8s.io/docs/user/kwok-in-cluster

Applies the default configuration for simulating resource usage (CPU and memory) for nodes, pods, and containers managed by `kwok`. This is optional.

```bash
kubectl apply -f "https://github.com/${KWOK_REPO}/releases/download/${KWOK_LATEST_RELEASE}/metrics-usage.yaml"

```

--------------------------------

### Set Output Format for Components

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_components

Specify the output format for the `kwokctl get components` command. Supported formats are 'name' and 'wide'.

```bash
kwokctl get components --output wide
```

--------------------------------

### Kubeconfig Setup for KWOK Cluster

Source: https://kwok.sigs.k8s.io/docs/user/all-in-one-image

Configuration for a kubeconfig file to connect to the KWOK all-in-one cluster running locally on port 8080.

```yaml
apiVersion: v1
clusters:
- cluster:
    server: http://127.0.0.1:8080
  name: kwok
contexts:
- context:
    cluster: kwok
  name: kwok
current-context: kwok
kind: Config
preferences: {}
users: null

```

--------------------------------

### Create KWOK Cluster with Metrics Server

Source: https://kwok.sigs.k8s.io/docs/examples/metrics-server

Create a KWOK cluster and enable the metrics server using the downloaded metrics usage configuration. This command sets up the necessary components for metrics collection.

```bash
kwokctl create cluster --enable-metrics-server -c metrics-usage.yaml

```

--------------------------------

### Create Kubernetes Cluster with Metrics Server

Source: https://kwok.sigs.k8s.io/docs/examples/scalability/scale-using-hpa

Create a Kubernetes cluster using kwokctl, enabling the metrics server and specifying a configuration file.

```bash
kwokctl create cluster --enable-metrics-server --config ./metrics-usage.yaml
```

--------------------------------

### Verify Fake Node Status

Source: https://kwok.sigs.k8s.io/docs/user/kwok-manage-nodes-and-pods

Check the status of the created fake node using `kubectl get node`. The node should be in the `Ready` state.

```bash
kubectl get node -o wide

```

--------------------------------

### Label Nodes for Topology

Source: https://kwok.sigs.k8s.io/docs/examples/scheduling/pod-topology-spread-constraint

Assigns topology labels to nodes, defining regions for scheduling. This example sets up two regions, 'us-west-1' and 'us-west-2'.

```bash
kubectl label nodes node-000000 topology.kubernetes.io/region=us-west-1
kubectl label nodes node-000001 topology.kubernetes.io/region=us-west-1
kubectl label nodes node-000002 topology.kubernetes.io/region=us-west-2
kubectl label nodes node-000003 topology.kubernetes.io/region=us-west-2

```

--------------------------------

### Get HPA Status After Scale-Up

Source: https://kwok.sigs.k8s.io/docs/examples/scalability/scale-using-hpa

Fetches and displays the current status of the Horizontal Pod Autoscaler, showing the updated number of replicas after the scale-up event.

```bash
kubectl get hpa
```

--------------------------------

### kwokctl config tidy Help Option

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_config_tidy

Displays help information for the `tidy` subcommand.

```bash
  -h, --help   help for tidy

```

--------------------------------

### Create Kubernetes Cluster with Metrics Server

Source: https://kwok.sigs.k8s.io/docs/examples/scalability/scale-using-vpa

Create a Kubernetes cluster using kwokctl, enabling the metrics server and specifying a configuration file and runtime.

```bash
kwokctl create cluster --enable-metrics-server --config ./metrics-usage.yaml --runtime binary
```

--------------------------------

### kwokctl get kubeconfig Command Usage

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_kubeconfig

This snippet shows the basic command structure for retrieving kubeconfig. It is used to print the cluster's kubeconfig file.

```bash
kwokctl get kubeconfig [flags]
```

--------------------------------

### Create Kubernetes Cluster with Kind

Source: https://kwok.sigs.k8s.io/docs/examples/istio

Use `kwokctl` to create a Kubernetes cluster using the Kind runtime. This is a prerequisite for deploying Istio.

```bash
kwokctl create cluster --runtime kind

```

--------------------------------

### Describe Workload Details

Source: https://kwok.sigs.k8s.io/docs/examples/kueue

Get detailed information about a specific workload, including its spec, status, conditions, and events. This is useful for debugging admission and execution problems.

```bash
kubectl describe workload job-sample-job-d7b38
```

--------------------------------

### List Clusters with Dry Run Flag

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_clusters

Execute the `kwokctl get clusters` command in dry-run mode to see the command that would be executed without actually running it.

```bash
kwokctl get clusters --dry-run
```

--------------------------------

### CEL Expression for Dynamic Memory Usage

Source: https://kwok.sigs.k8s.io/docs/user/resource-usage-configuration

Example of a CEL expression to simulate memory usage that grows linearly with time, calculated based on the pod's uptime.

```cel
expression: 'Quantity("1Mi") * (pod.SinceSecond() / 60.0)'

```

--------------------------------

### kwokctl create Options

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_create

Common options available for the `kwokctl create` command, including help flags.

```bash
-h, --help   help for create
```

--------------------------------

### Help Option for View Command

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_config_view

Displays help information for the `kwokctl config view` command.

```bash
-h, --help   help for view
```

--------------------------------

### CEL Expression for CPU Usage from Annotation

Source: https://kwok.sigs.k8s.io/docs/user/resource-usage-configuration

Example of a CEL expression to extract CPU resource usage from pod annotations or use a default value if the annotation is not present.

```cel
expression: |
  "kwok.x-k8s.io/usage-cpu" in pod.metadata.annotations
  ? Quantity(pod.metadata.annotations["kwok.x-k8s.io/usage-cpu"])
  : Quantity("1m")  

```

--------------------------------

### Clean Up Resources After Failure Injection Demo

Source: https://kwok.sigs.k8s.io/docs/examples/injectfailure

This snippet outlines the steps to remove the resources created during the failure injection demo, including the failed pod, the virtual GPU node, and the KWOK deployment.

```bash
# Clean up
kubectl delete -f ${WORK_DIR}/failed-pod.yaml
kubectl delete -f ${WORK_DIR}/virtual-gpu-node.yaml
kubectl delete -f https://github.com/kubernetes-sigs/kwok/releases/download/v0.5.2/kwok.yaml
```

--------------------------------

### Kwokctl Create Cluster Options

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_create_cluster

Lists the available command-line options for `kwokctl create cluster`. These options allow customization of cluster components, container images, binary paths, and network ports.

```bash
      --components strings                      Default of components (default [etcd,kube-apiserver,kube-controller-manager,kube-scheduler,kwok-controller])
      --controller-port uint32                  Port of kwok-controller given to the host
      --disable strings                         Disable list of components
      --disable-qps-limits                      Disable QPS limits for components
      --enable strings                          Enable list of components
      --enable-crds strings                     List of CRDs to enable
      --etcd-binary string                      Binary of etcd, only for binary runtime (default "https://github.com/etcd-io/etcd/releases/download/v3.6.10/etcd-v3.6.10-linux-amd64.tar.gz#etcd")
      --etcd-image string                       Image of etcd, only for docker/podman/nerdctl runtime
                                                '${KWOK_KUBE_IMAGE_PREFIX}/etcd:${KWOK_ETCD_VERSION}'
                                                 (default "registry.k8s.io/etcd:3.6.10-0")
      --etcd-port uint32                        Port of etcd given to the host. The behavior is unstable for kind/kind-podman runtime and may be modified in the future
      --etcd-prefix string                      prefix of the key (default "/registry")
      --etcd-quota-backend-size string          Quota backend size for etcd (default "8Gi")
      --extra-args component=key=value          Pass a single extra arg key-value pair to the component in the format component=key=value
      --heartbeat-factor float                  Scale factor for all about heartbeat (default 5)
  -h, --help                                    help for cluster
      --jaeger-binary string                    Binary of Jaeger, only for binary runtime (default "https://github.com/jaegertracing/jaeger/releases/download/v1.76.0/jaeger-1.76.0-linux-amd64.tar.gz#jaeger-all-in-one")
      --jaeger-image string                     Image of Jaeger, only for docker/podman/nerdctl/kind/kind-podman runtime
                                                '${KWOK_JAEGER_IMAGE_PREFIX}/all-in-one:${KWOK_JAEGER_VERSION}'
                                                 (default "docker.io/jaegertracing/all-in-one:1.76.0")
      --jaeger-port uint32                      Port to expose Jaeger UI
      --kind-binary string                      Binary of kind, only for kind/kind-podman runtime
                                                 (default "https://github.com/kubernetes-sigs/kind/releases/download/v0.32.0/kind-linux-amd64")
      --kind-node-image string                  Image of kind node, only for kind/kind-podman runtime
                                                '${KWOK_KIND_NODE_IMAGE_PREFIX}/node:${KWOK_KUBE_VERSION}'
                                                 (default "docker.io/kindest/node:v1.36.1")
      --kube-admission                          Enable admission for kube-apiserver, only for non kind/kind-podman runtime (default true)
      --kube-apiserver-binary string            Binary of kube-apiserver, only for binary runtime
                                                 (default "https://dl.k8s.io/release/v1.36.1/bin/linux/amd64/kube-apiserver")
      --kube-apiserver-image string             Image of kube-apiserver, only for docker/podman/nerdctl runtime
                                                '${KWOK_KUBE_IMAGE_PREFIX}/kube-apiserver:${KWOK_KUBE_VERSION}'
                                                 (default "registry.k8s.io/kube-apiserver:v1.36.1")
      --kube-apiserver-insecure-port uint32     Insecure port of the apiserver
      --kube-apiserver-port uint32              Port of the apiserver (default random)
      --kube-audit-policy string                Path to the file that defines the audit policy configuration
      --kube-authorization                      Enable authorization for kube-apiserver, only for non kind/kind-podman runtime (default true)
      --kube-controller-manager-binary string   Binary of kube-controller-manager, only for binary runtime
                                                 (default "https://dl.k8s.io/release/v1.36.1/bin/linux/amd64/kube-controller-manager")
      --kube-controller-manager-image string    Image of kube-controller-manager, only for docker/podman/nerdctl runtime
                                                '${KWOK_KUBE_IMAGE_PREFIX}/kube-controller-manager:${KWOK_KUBE_VERSION}'
                                                 (default "registry.k8s.io/kube-controller-manager:v1.36.1")
      --kube-controller-manager-port uint32     Port of kube-controller-manager given to the host, only for binary and docker/podman/nerdctl runtime
      --kube-feature-gates string               A set of key=value pairs that describe feature gates for alpha/experimental features of Kubernetes
      --kube-runtime-config string              A set of key=value pairs that enable or disable built-in APIs

```

--------------------------------

### Kwokctl Kubectl Help Option

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_kubectl

This option displays help information for the `kwokctl kubectl` command.

```bash
  -h, --help   help for kubectl
```

--------------------------------

### Create Cluster with Docker Runtime

Source: https://kwok.sigs.k8s.io/docs/contributing/development

Creates a Kubernetes cluster using `kwokctl` with Docker as the runtime. Assumes the KWOK image has been built.

```bash
./bin/$(go env GOOS)/$(go env GOARCH)/kwokctl create cluster \
  --runtime=docker

```

--------------------------------

### Verify Scaled Deployments

Source: https://kwok.sigs.k8s.io/docs/examples/scale-with-custom-resource

Verifies the creation and status of scaled Deployments across all namespaces using `kubectl get deployments`. This command helps confirm that the scaling operation was successful.

```bash
kubectl get deployments -A

```

--------------------------------

### Add Virtual GPU Node and Create Pod with InitContainer Failure

Source: https://kwok.sigs.k8s.io/docs/examples/injectfailure

This section details adding a virtual GPU node and then creating a pod that is designed to fail its initContainer, simulating a preflight check error.

```bash
# Add a virtual-gpu node
grep allocatable ${WORK_DIR}/virtual-gpu-node.yaml -A 15 
kubectl apply -f ${WORK_DIR}/virtual-gpu-node.yaml
kubectl get nodes
kubectl describe node virtual-gpu-node | grep Allocatable -A 15 

# Create a pod and inject an error to initContainer to simulate a preflight check error
grep failed ${WORK_DIR}/failed-pod.yaml
kubectl apply -f ${WORK_DIR}/failed-pod.yaml
kubectl get pod
kubectl describe pod failed-pod | grep "Init Containers" -A 10
```

--------------------------------

### KWOK CLI Usage

Source: https://kwok.sigs.k8s.io/docs/generated/kwok

Basic usage of the kwok command. This is the entry point for running the KWOK simulator.

```bash
kwok [flags]
```

--------------------------------

### Run Grafana Docker Container

Source: https://kwok.sigs.k8s.io/docs/examples/prometheus

Start a Grafana instance using Docker to visualize Prometheus data. This command runs Grafana in detached mode and maps the default Grafana port.

```bash
docker run -d --name=grafana -p 3000:3000 docker.io/grafana/grafana:9.4.7

```

--------------------------------

### View Available Clusters

Source: https://kwok.sigs.k8s.io/docs/examples/scheduling/node-affinity

Verify that the cluster was created successfully by listing all available clusters.

```bash
kwokctl get clusters

```

--------------------------------

### List Clusters with Verbosity Level

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_clusters

Set the log level verbosity for the `kwokctl get clusters` command. Accepts keywords like DEBUG, INFO, WARN, ERROR or numerical values.

```bash
kwokctl get clusters -v WARN
```

--------------------------------

### Create Cluster with Binary Runtime

Source: https://kwok.sigs.k8s.io/docs/user/kwokctl-platform-specific-binaries

Creates a Kubernetes cluster using kwokctl with the 'binary' runtime. This command assumes the necessary Kubernetes binaries have been configured.

```bash
kwokctl create cluster \
  --runtime=binary

```

--------------------------------

### Verify Scaled Pods

Source: https://kwok.sigs.k8s.io/docs/examples/scale-with-custom-resource

Verifies the creation and status of pods associated with the scaled Deployments across all namespaces using `kubectl get pods`. This confirms that the Deployment controller has spawned the expected pods.

```bash
kubectl get pods -A

```

--------------------------------

### kwokctl etcdctl Help Option

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_etcdctl

Displays help information for the `kwokctl etcdctl` command. This is useful for understanding available subcommands and their specific options.

```bash
kwokctl etcdctl -h

```

--------------------------------

### Create Pod and Inject Linkerd Sidecar

Source: https://kwok.sigs.k8s.io/docs/examples/linkerd

Applies a sample application (emojivoto) and then injects the Linkerd sidecar proxy into its deployments.

```bash
kubectl apply -f https://run.linkerd.io/emojivoto.yml
kubectl get -n emojivoto deploy -o yaml | \
  linkerd inject - | \
  kubectl apply -f -

```

--------------------------------

### Disable Service Account Token in Pod Spec

Source: https://kwok.sigs.k8s.io/docs/user/all-in-one-image

Example snippet for a Kubernetes Pod specification to disable the automatic mounting of the service account token. This is often required when running the all-in-one image within a pod.

```yaml
...
spec:
  automountServiceAccountToken: false
...

```

--------------------------------

### Scale nodes with specific CPU allocatable resources

Source: https://kwok.sigs.k8s.io/docs/examples/scheduling/requests-and-limits

Deploy worker nodes and configure their allocatable CPU resources using `kwokctl scale`.

```bash
kwokctl scale node node-1 --replicas 1 --param '.allocatable.cpu="4000m"'
kwokctl scale node node-2 --replicas 1 --param '.allocatable.cpu="2000m"'

```

--------------------------------

### kwokctl kectl Help Option

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_kectl

This snippet shows the help flag for the kectl command, which is standard for most CLI tools.

```bash
  -h, --help   help for kectl
```

--------------------------------

### Check Applied CPU Limits

Source: https://kwok.sigs.k8s.io/docs/examples/scalability/scale-using-vpa

Lists pods and filters for lines containing 'cpu:' to show the newly applied CPU limits after VPA updates.

```bash
kubectl get pod -o yaml | grep -i cpu:
```

--------------------------------

### List KWOK Components

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_get_components

Use this command to list the components of the KWOK system. It supports different output formats.

```bash
kwokctl get components [flags]
```

--------------------------------

### Create KWOK Cluster with Prometheus Enabled

Source: https://kwok.sigs.k8s.io/docs/examples/prometheus

Use the `kwokctl create cluster` command with the `--prometheus-port` flag to enable Prometheus monitoring on your cluster. Specify the desired port for Prometheus.

```bash
kwokctl create cluster --prometheus-port 9090

```

--------------------------------

### kwokctl snapshot Options

Source: https://kwok.sigs.k8s.io/docs/generated/kwokctl_snapshot

Help flag for the snapshot command. Displays available options for the snapshot subcommand.

```bash
  -h, --help   help for snapshot
```
Tokens:
Raw