### Install Latest Version from Source Source: https://chaos-mesh.org/docs/production-installation-using-helm Clone the repository and install the latest version using Helm. ```bash # Clone repository git clone https://github.com/chaos-mesh/chaos-mesh.git cd chaos-mesh helm install chaos-mesh helm/chaos-mesh -n=chaos-mesh ``` -------------------------------- ### Example HelloWorldChaos YAML Source: https://chaos-mesh.org/docs/add-new-chaos-experiment-type An example of how to describe the HelloWorldChaos custom resource in a YAML file. Replace placeholders with your specific values. ```yaml apiVersion: chaos-mesh.org/v1alpha1 kind: HelloWorldChaos metadata: name: namespace: spec: duration: #... ``` -------------------------------- ### Example: Simulate Disk Read Load Source: https://chaos-mesh.org/docs/simulate-disk-pressure-in-physical-nodes This example demonstrates how to simulate a disk read load of 1000GB using 7 processes, reading from /dev/zero. ```bash chaosd attack disk add-payload read -s 1000G -n 7 -p /dev/zero ``` -------------------------------- ### Install Chaos Mesh (Default) Source: https://chaos-mesh.org/docs/quick-start Run this script to install Chaos Mesh in a test environment. The `install.sh` script is deprecated starting from v2.8.2. ```bash curl -sSL https://mirrors.chaos-mesh.org/v2.8.2/install.sh | bash ``` -------------------------------- ### Chaos Controller Manager Log Output Example Source: https://chaos-mesh.org/docs/extend-chaos-daemon-interface Example output showing the successful application of a helloworld chaos experiment. ```text 2023-07-16T08:20:46.823Z INFO records records/controller.go:149 apply chaos {"id": "busybox/busybox-0/busybox"} 2023-07-16T08:20:46.823Z INFO helloworldchaos helloworldchaos/types.go:27 Apply helloworld chaos ``` -------------------------------- ### Chaos Daemon Log Output Example Source: https://chaos-mesh.org/docs/extend-chaos-daemon-interface Example output showing the execution of commands by the Chaos Daemon on target pods. ```text 2023-07-16T08:20:46.833Z INFO chaos-daemon.daemon-server chaosdaemon/server.go:187 ExecHelloWorldChaos {"namespacedName": "chaos-mesh/hello-busybox", "request": "container_id:\"docker://5e01e76efdec6aa0934afc15bb80e121d58b43c529a6696a01a242f7ac68f201\""} 2023-07-16T08:20:46.834Z INFO chaos-daemon.daemon-server.background-process-manager.process-builder pb/chaosdaemon.pb.go:4568 build command {"namespacedName": "chaos-mesh/hello-busybox", "command": "/usr/local/bin/nsexec -m /proc/104710/ns/mnt -- sh -c ps aux"} 2023-07-16T08:20:46.841Z INFO chaos-daemon.daemon-server chaosdaemon/server.go:187 cmd output {"namespacedName": "chaos-mesh/hello-busybox", "output": "PID USER TIME COMMAND\n 1 root 0:00 sh -c echo Container is Running ; sleep 3600\n"} 2023-07-16T08:20:46.856Z INFO chaos-daemon.daemon-server chaosdaemon/server.go:187 ExecHelloWorldChaos {"namespacedName": "chaos-mesh/hello-busybox", "request": "container_id:\"docker://bab4f632a0358529f7d72d35e014b8c2ce57438102d99d6174dd9df52d093e99\""} 2023-07-16T08:20:46.864Z INFO chaos-daemon.daemon-server.background-process-manager.process-builder pb/chaosdaemon.pb.go:4568 build command {"namespacedName": "chaos-mesh/hello-busybox", "command": "/usr/local/bin/nsexec -m /proc/104841/ns/mnt -- sh -c ps aux"} 2023-07-16T08:20:46.867Z INFO chaos-daemon.daemon-server chaosdaemon/server.go:187 cmd output {"namespacedName": "chaos-mesh/hello-busybox", "output": "PID USER TIME COMMAND\n 1 root 0:00 sh -c echo Container is Running ; sleep 3600\n"} ``` -------------------------------- ### Start Minikube Cluster Source: https://chaos-mesh.org/docs/configure-development-environment Start a local Kubernetes cluster using Minikube. This provides an environment to run Chaos Mesh. ```bash minikube start ``` -------------------------------- ### Get Help for Cache Limit Fault Source: https://chaos-mesh.org/docs/simulate-redis-chaos-on-physical-nodes Run this command to get detailed help for simulating cache limit faults. ```bash chaosd attack redis cache-limit -h ``` -------------------------------- ### Chaosd Example for Disk Write Load Source: https://chaos-mesh.org/docs/simulate-disk-pressure-in-physical-nodes This example demonstrates how to simulate disk write load using chaosd. It specifies a size of 2GB and uses 8 processes for writing. ```bash chaosd attack disk add-payload write -s 2G -n 8 ``` -------------------------------- ### Install Chaos Mesh for Kind Source: https://chaos-mesh.org/docs/quick-start Installs Chaos Mesh locally for a `kind` Kubernetes cluster. The `install.sh` script is deprecated starting from v2.8.2. ```bash curl -sSL https://mirrors.chaos-mesh.org/v2.8.2/install.sh | bash -s -- --local kind ``` -------------------------------- ### Example Helm list output Source: https://chaos-mesh.org/docs/uninstallation Sample output showing a deployed Chaos Mesh release. ```text NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION chaos-mesh-playground chaos-mesh 1 2021-12-01 22:58:18.037052401 +0800 CST deployed chaos-mesh-2.1.0 2.1.0 ``` -------------------------------- ### Get Help for Cache Expiration Fault Source: https://chaos-mesh.org/docs/simulate-redis-chaos-on-physical-nodes Run this command to get detailed help for simulating cache expiration faults. ```bash chaosd attack redis cache-expiration -h ``` -------------------------------- ### Install Chaos Mesh (Default Docker) Source: https://chaos-mesh.org/docs/production-installation-using-helm Install Chaos Mesh using Helm with default settings, suitable for Docker runtime. Specify the version for a particular release. ```bash # Default to /var/run/docker.sock helm install chaos-mesh chaos-mesh/chaos-mesh -n=chaos-mesh --version 2.8.2 ``` -------------------------------- ### Install Chaos Mesh Grafana Data Source via CLI Source: https://chaos-mesh.org/docs/use-grafana-data-source Use the Grafana CLI to install the Chaos Mesh data source plugin. Ensure you have the Grafana CLI installed and configured. ```bash grafana-cli plugins install chaosmeshorg-datasource ``` -------------------------------- ### Example kubectl describe Output Source: https://chaos-mesh.org/docs/inspect-chaos-experiments Sample output showing the Status and Events fields of a pod chaos experiment. ```text ... Status: Conditions: Reason: Status: False Type: Paused Reason: Status: True Type: Selected Reason: Status: True Type: AllInjected Reason: Status: False Type: AllRecovered Experiment: Container Records: Id: tidb-cluster/basic-tikv-0 Phase: Injected Selector Key: . Desired Phase: Run Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal FinalizerInited 39s finalizer Finalizer has been inited Normal Paused 39s desiredphase Experiment has been paused Normal Updated 39s finalizer Successfully update finalizer of resource Normal Updated 39s records Successfully update records of resource Normal Updated 39s desiredphase Successfully update desiredPhase of resource Normal Started 17s desiredphase Experiment has started Normal Updated 17s desiredphase Successfully update desiredPhase of resource Normal Applied 17s records Successfully apply chaos for tidb-cluster/basic-tikv-0 Normal Updated 17s records Successfully update records of resource ``` -------------------------------- ### Simulate DNS Faults Source: https://chaos-mesh.org/docs/simulate-network-chaos-in-physical-nodes Examples for mapping hosts to IPs or updating the DNS server configuration. ```bash chaosd attack network dns --dns-ip 123.123.123.123 --dns-domain-name chaos-mesh.org ``` ```bash chaosd attack network dns --dns-server 123.123.123.123 ``` -------------------------------- ### Example Service Account Token Output Source: https://chaos-mesh.org/docs/manage-user-permissions This is an example output from describing a Service Account secret, showing the token used for API authentication. Copy the 'token' field value for subsequent logins. ```text Name: account-default-manager-vfmot-token-n4tg8 Namespace: default Labels: Annotations: kubernetes.io/service-account.name: account-default-manager-vfmot kubernetes.io/service-account.uid: b71b3bf4-cd5e-4efb-8bf6-ff9a55fd7e07 Type: kubernetes.io/service-account-token Data ==== ca.crt: 1111 bytes namespace: 7 bytes token: eyJhbG... ``` -------------------------------- ### Install Chaos Mesh (Containerd) Source: https://chaos-mesh.org/docs/production-installation-using-helm Install Chaos Mesh for Containerd runtime, specifying the correct socket path. Ensure the --version flag is used for specific releases. ```bash helm install chaos-mesh chaos-mesh/chaos-mesh -n=chaos-mesh --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/containerd/containerd.sock --version 2.8.2 ``` -------------------------------- ### Install Chaos Mesh (CRI-O) Source: https://chaos-mesh.org/docs/production-installation-using-helm Install Chaos Mesh for CRI-O runtime, specifying the correct socket path. Use the --version flag to target a specific release. ```bash helm install chaos-mesh chaos-mesh/chaos-mesh -n=chaos-mesh --set chaosDaemon.runtime=crio --set chaosDaemon.socketPath=/var/run/crio/crio.sock --version 2.8.2 ``` -------------------------------- ### Container command configuration Source: https://chaos-mesh.org/docs/create-chaos-mesh-workflow Example of defining container commands using a string array format. ```json ["wget", "-q", "http://httpbin.org/status/201"] ``` -------------------------------- ### Start Chaosd Server Source: https://chaos-mesh.org/docs/simulate-redis-chaos-on-physical-nodes Run the Chaosd server in service mode on the specified port. ```bash chaosd server --port 31767 ``` -------------------------------- ### Pod Status Output Source: https://chaos-mesh.org/docs/add-new-chaos-experiment-type Example output showing the running status of the test pod. ```text NAME READY STATUS RESTARTS AGE hello-minikube-77b6f68484-dg4sw 1/1 Running 0 2m ``` -------------------------------- ### Check Helm Installation Source: https://chaos-mesh.org/docs/production-installation-using-helm Verify if Helm is installed by checking its version. Ensure the output includes Version, GitCommit, GitTreeState, and GoVersion. ```bash helm version ``` -------------------------------- ### Example Output of All Chaosd Experiments via Service Mode Source: https://chaos-mesh.org/docs/chaosd-search-recover This is an example of the JSON output when retrieving all experiments from the Chaosd service. It includes details for each experiment. ```json [{"id":1,"uid":"ddc5ca81-b677-4595-b691-0ce57bedb156","created_at":"2021-10-18T16:01:18.563542491+08:00","updated_at":"2021-10-18T16:07:27.87111393+08:00","status":"success","kind":"stress","action":"mem","recover_command":"{\"schedule\":\"\",\"duration\":\"\",\"action\":\"mem\",\"kind\":\"stress\",\"uid\":\"ddc5ca81-b677-4595-b691-0ce57bedb156\",\"Load\":0,\"Workers\":0,\"Size\":\"100MB\",\"Options\":null,\"StressngPid\":0}","launch_mode":"svr"}] ``` -------------------------------- ### Start Chaosd server in service mode Source: https://chaos-mesh.org/docs/simulate-physical-machine-chaos Commands to start the Chaosd server. The first command uses TLS certificates for secure communication, while the second uses an insecure port. ```bash chaosd server --https-port 31768 --CA=/etc/chaosd/pki/ca.crt --cert=/etc/chaosd/pki/chaosd.crt --key=/etc/chaosd/pki/chaosd.key ``` ```bash chaosd server --port 31767 ``` -------------------------------- ### Install Chaos Mesh (MicroK8s) Source: https://chaos-mesh.org/docs/production-installation-using-helm Install Chaos Mesh for MicroK8s, setting the Containerd socket path to the MicroK8s specific location. Specify the --version for the desired release. ```bash helm install chaos-mesh chaos-mesh/chaos-mesh -n=chaos-mesh --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/var/snap/microk8s/common/run/containerd.sock --version 2.8.2 ``` -------------------------------- ### Example Byteman Rule Configuration Source: https://chaos-mesh.org/docs/simulate-jvm-application-chaos This is an example of a Byteman rule configuration file for modifying return values. Ensure line breaks are escaped as '\n' when used as a string value. ```java RULE modify return value CLASS Main METHOD getnum AT ENTRY IF true DO return 9999 ENDRULE ``` ```string \nRULE modify return value\nCLASS Main\nMETHOD getnum\nAT ENTRY\nIF true\nDO return 9999\nENDRULE\n ``` -------------------------------- ### Install Chaos Mesh with Helm Source: https://chaos-mesh.org/docs/configure-development-environment Install Chaos Mesh into the Minikube cluster using Helm. This command deploys Chaos Mesh with a specific namespace. ```bash helm upgrade --install chaos-mesh-debug ./helm/chaos-mesh -n=chaos-mesh-debug --create-namespace ``` -------------------------------- ### Install Chaos Mesh (K3s) Source: https://chaos-mesh.org/docs/production-installation-using-helm Install Chaos Mesh for K3s, configuring the Containerd socket path specific to K3s environments. Use the --version flag to select the desired release. ```bash helm install chaos-mesh chaos-mesh/chaos-mesh -n=chaos-mesh --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/k3s/containerd/containerd.sock --version 2.8.2 ``` -------------------------------- ### Controller Log Output Source: https://chaos-mesh.org/docs/add-new-chaos-experiment-type Example log output showing the HelloWorldChaos execution message. ```text 2023-07-16T06:19:40.068Z INFO records records/controller.go:149 apply chaos {"id": "default/hello-minikube-77b6f68484-dg4sw/echo-server"} 2023-07-16T06:19:40.068Z INFO helloworldchaos helloworldchaos/types.go:26 Hello world! ``` -------------------------------- ### Chaosd Network Duplication Command Example Source: https://chaos-mesh.org/docs/simulate-network-chaos-in-physical-nodes An example command to simulate network duplication, targeting a specific device and IP address with a 50% packet duplication rate. Ensure the specified device and IP are valid in your environment. ```bash chaosd attack network duplicate -d eth0 -i 172.16.4.4 --percent 50 ``` -------------------------------- ### Get Help for Network Delay Simulation with Chaosd Source: https://chaos-mesh.org/docs/simulate-network-chaos-in-physical-nodes View the help information for the chaosd network delay command to understand available flags and configurations. ```bash chaosd attack network delay --help ``` ```bash delay network Usage: chaosd attack network delay [flags] Flags: -c, --correlation string correlation is percentage (10 is 10%) (default "0") -d, --device string the network interface to impact -e, --egress-port string only impact egress traffic to these destination ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp -h, --help help for delay -H, --hostname string only impact traffic to these hostnames -i, --ip string only impact egress traffic to these IP addresses -j, --jitter string jitter time, time units: ns, us (or µs), ms, s, m, h. -l, --latency string delay egress time, time units: ns, us (or µs), ms, s, m, h. -p, --protocol string only impact traffic using this IP protocol, supported: tcp, udp, icmp, all -s, --source-port string only impact egress traffic from these source ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp Global Flags: --log-level string the log level of chaosd, the value can be 'debug', 'info', 'warn' and 'error' ``` -------------------------------- ### Define a PodChaos template step Source: https://chaos-mesh.org/docs/create-chaos-mesh-workflow Example of a template step configured for a PodChaos experiment. ```yaml name: workflow-pod-chaos templateType: PodChaos deadline: 40s podChaos: action: pod-kill mode: one selector: labelSelectors: 'app': 'hello-kubernetes' ``` -------------------------------- ### Install Chaos Mesh for MicroK8s Source: https://chaos-mesh.org/docs/quick-start Installs Chaos Mesh for a MicroK8s Kubernetes cluster. The `install.sh` script is deprecated starting from v2.8.2. ```bash curl -sSL https://mirrors.chaos-mesh.org/v2.8.2/install.sh | bash -s -- --microk8s ``` -------------------------------- ### Get Chaosd Clock Attack Help Source: https://chaos-mesh.org/docs/simulate-time-chaos-on-physical-nodes Use this command to view the available options and flags for simulating clock-related faults with Chaosd. ```bash chaosd attack clock -h ``` -------------------------------- ### Install Chaos Mesh for K3s Source: https://chaos-mesh.org/docs/quick-start Installs Chaos Mesh for a K3s Kubernetes cluster. The `install.sh` script is deprecated starting from v2.8.2. ```bash curl -sSL https://mirrors.chaos-mesh.org/v2.8.2/install.sh | bash -s -- --k3s ``` -------------------------------- ### Install CI Environment and Checkout Repository Source: https://chaos-mesh.org/docs/integrate-chaos-mesh-into-github-actions This section configures the build job to run on 'ubuntu-latest', creates a Kind cluster using 'helm/kind-action', prints cluster information, and checks out the GitHub repository. ```yaml jobs: build: runs-on: ubuntu-latest steps: - name: Creating kind cluster uses: helm/kind-action@v1.0.0-rc.1 - name: Print cluster information run: | kubectl config view kubectl cluster-info kubectl get nodes kubectl get pods -n kube-system helm version kubectl version - uses: actions/checkout@v2 ``` -------------------------------- ### ChaosD CLI Commands Source: https://chaos-mesh.org/docs/simulate-heavy-stress-in-physical-nodes Examples of using the chaosd command-line interface to simulate memory stress and recover experiments. ```APIDOC ## Simulate Memory Stress using CLI ### Description Simulates memory stress on a system using the chaosd CLI. ### Method CLI Command ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```bash chaosd attack stress mem --workers 2 --size 100M ``` ### Response #### Success Response Indicates successful initiation of the stress test and provides a UID for recovery. #### Response Example ``` Attack stress mem successfully, uid: c2bff2f5-3aac-4ace-b7a6-322946ae6f13 ``` ## Recover Experiment using CLI ### Description Recovers or terminates a previously started experiment using its unique identifier (UID). ### Method CLI Command ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```bash chaosd recover c2bff2f5-3aac-4ace-b7a6-322946ae6f13 ``` ### Response #### Success Response Confirms the successful termination of the specified experiment. #### Response Example ``` Recover c2bff2f5-3aac-4ace-b7a6-322946ae6f13 successfully ``` ``` -------------------------------- ### Install Chaos Mesh for Kind with Specific Version Source: https://chaos-mesh.org/docs/quick-start Installs Chaos Mesh locally for a `kind` Kubernetes cluster, specifying a particular `kind` version. The `install.sh` script is deprecated starting from v2.8.2. ```bash curl -sSL https://mirrors.chaos-mesh.org/v2.8.2/install.sh | bash -s -- --local kind --kind-version v0.20.0 ``` -------------------------------- ### Apply Target Application Pod Definition Source: https://chaos-mesh.org/docs/simulate-jvm-application-chaos Apply the 'app.yaml' file to create the 'helloworld' Pod in the 'helloworld' namespace. ```bash kubectl apply -f app.yaml ``` -------------------------------- ### Get Help for Port Occupation Command Source: https://chaos-mesh.org/docs/simulate-network-chaos-in-physical-nodes Run this command to view the available flags and usage for simulating port occupation. ```bash chaosd attack network port --help ``` -------------------------------- ### Install Chaos Mesh using Helm Source: https://chaos-mesh.org/docs/offline-installation Install Chaos Mesh using Helm, specifying the namespace and the Docker registry address. This command assumes Helm is installed and configured. ```bash helm install chaos-mesh helm/chaos-mesh -n=chaos-mesh --set images.registry=$DOCKER_REGISTRY ``` -------------------------------- ### Deploy Test Pod Source: https://chaos-mesh.org/docs/extend-chaos-daemon-interface Deploy a sample application for testing the chaos experiment. ```bash kubectl apply -f https://raw.githubusercontent.com/chaos-mesh/apps/master/ping/busybox-statefulset.yaml ``` -------------------------------- ### Enable FilterNamespace during installation Source: https://chaos-mesh.org/docs/configure-enabled-namespace Use the Helm install command with the controllerManager.enableFilterNamespace flag to enable namespace filtering. ```bash helm install chaos-mesh chaos-mesh/chaos-mesh -n chaos-mesh --set controllerManager.enableFilterNamespace=true --version 2.8.2 ``` -------------------------------- ### Deploy Application Source: https://chaos-mesh.org/docs/integrate-chaos-mesh-into-github-actions This step deploys a sample application by applying a Kubernetes resource definition from a GitHub raw URL. It is intended to set up the environment for fault injection. ```yaml - name: Deploy an application run: | kubectl apply -f https://raw.githubusercontent.com/chaos-mesh/apps/master/ping/busybox-statefulset.yaml ``` -------------------------------- ### Get Help for Redis Faults Source: https://chaos-mesh.org/docs/simulate-redis-chaos-on-physical-nodes Use this command to get detailed help for Redis fault simulation commands. ```bash chaosd attack redis [command] --help ``` -------------------------------- ### Observe Target Application Logs Source: https://chaos-mesh.org/docs/simulate-jvm-application-chaos Stream the logs of the 'helloworld' Pod to observe its behavior before injecting faults. The application is expected to output 'Hello World' with an incrementing number every second. ```bash kubectl -n helloworld logs -f helloworld ``` -------------------------------- ### View JVM rule-file help Source: https://chaos-mesh.org/docs/simulate-jvm-application-chaos-in-physical-nodes Displays help information for the rule-file command used to inject faults via Byteman configuration files. ```bash chaosd attack jvm rule-file --help ``` -------------------------------- ### ConditionalBranch expression examples Source: https://chaos-mesh.org/docs/create-chaos-mesh-workflow Examples of boolean expressions used to determine if a conditional branch should execute based on task results. ```text exitCode == 0 ``` -------------------------------- ### View Network Partition Help Source: https://chaos-mesh.org/docs/simulate-network-chaos-in-physical-nodes Displays the help menu for the network partition attack command. ```bash chaosd attack network partition --help ``` -------------------------------- ### Create Experiment via Chaosd Command-Line Source: https://chaos-mesh.org/docs/chaosd-search-recover This command initiates a CPU stress experiment using Chaosd in command-line mode. Save the returned UID for later recovery. ```bash chaosd attack stress cpu --workers 2 --load 10 ``` -------------------------------- ### Install Kernel Headers on Ubuntu/Debian Source: https://chaos-mesh.org/docs/simulate-block-chaos-on-kubernetes Install the necessary kernel headers for your current kernel version on Ubuntu or Debian systems, required for compiling the chaos-driver module. ```bash apt install linux-headers-$(uname -r) ``` -------------------------------- ### View Host Shutdown Experiment Help Source: https://chaos-mesh.org/docs/simulate-host-console-in-physical-nodes Run this command to see the available flags and usage for host shutdown experiments. ```bash chaosd attack host shutdown -h ``` -------------------------------- ### Create a workflow using kubectl Source: https://chaos-mesh.org/docs/create-chaos-mesh-workflow Commands to deploy a workflow from a local file or a remote network URL. ```bash kubectl create -f ``` ```bash kubectl create -f https://raw.githubusercontent.com/chaos-mesh/chaos-mesh/master/examples/workflow/serial.yaml ``` -------------------------------- ### Install Chaos Mesh with DNS Server disabled Source: https://chaos-mesh.org/docs/simulate-dns-chaos-on-kubernetes Use this command to install Chaos Mesh while explicitly disabling the default deployment of the Chaos DNS Server. ```bash helm install chaos-mesh chaos-mesh/chaos-mesh --namespace=chaos-mesh --version 2.8.2 --set dnsServer.create=false ``` -------------------------------- ### Example Output of Searched Chaosd Experiment Source: https://chaos-mesh.org/docs/chaosd-search-recover This is an example of the output when searching for a specific Chaosd experiment. It displays details like UID, KIND, STATUS, and configuration. ```text UID KIND ACTION STATUS CREATE TIME CONFIGURATION --------------------------------------- --------- -------- ----------- --------------------------- --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1f6c1253-522a-43d9-83f8-42607102b3b9 network delay destroyed 2021-11-02T15:14:07+08:00 {"schedule":"","duration":"","action":"delay","kind":"network","uid":"1f6c1253-522a-43d9-83f8-42607102b3b9","latency":"2s","jitter":"0ms","correlation":"0","device":"eth0","ip-address":"220.181.38.251","ip-protocol":"all"} ``` -------------------------------- ### Install Kernel Headers on CentOS/Fedora Source: https://chaos-mesh.org/docs/simulate-block-chaos-on-kubernetes Install the necessary kernel development headers for your current kernel version on CentOS or Fedora systems, required for compiling the chaos-driver module. ```bash yum install kernel-devel-$(uname -r) ``` -------------------------------- ### Apply experiment configuration Source: https://chaos-mesh.org/docs/simulate-physical-machine-chaos Command to create the experiment in the Kubernetes cluster using the defined YAML file. ```bash kubectl apply -f physicalmachine.yaml ``` -------------------------------- ### Download Chaos Mesh Grafana Data Source Plugin Source: https://chaos-mesh.org/docs/use-grafana-data-source Download the plugin zip package using curl. This is an alternative to the CLI installation method. Ensure you have curl installed. ```bash curl -LO https://github.com/chaos-mesh/datasource/releases/download/v3.0.0/chaosmeshorg-datasource-3.0.0.zip ``` -------------------------------- ### View CPU stress configuration options Source: https://chaos-mesh.org/docs/simulate-heavy-stress-in-physical-nodes Displays help information for CPU stress simulation flags. ```bash chaosd attack stress cpu --help ``` -------------------------------- ### View Exception Injection Help Source: https://chaos-mesh.org/docs/simulate-jvm-application-chaos-in-physical-nodes Shows usage and configuration flags for the JVM exception injection command. ```bash chaosd attack jvm exception --help ``` ```text throw specified exception for specified method Usage: chaosd attack jvm exception [options] [flags] Flags: -c, --class string Java class name --exception string the exception which needs to throw for action 'exception' -h, --help help for exception -m, --method string the method name in Java class Global Flags: --log-level string the log level of chaosd. The value can be 'debug', 'info', 'warn' and 'error' --pid int the pid of Java process which needs to attach --port int the port of agent server (default 9288) --uid string the experiment ID ``` -------------------------------- ### Create Test Deployment Source: https://chaos-mesh.org/docs/add-new-chaos-experiment-type Deploys an echo server to serve as the target for the chaos experiment. ```bash kubectl create deployment hello-minikube --image=kicbase/echo-server:1.0 ``` -------------------------------- ### Get All Chaosd Experiments via Service Mode Source: https://chaos-mesh.org/docs/chaosd-search-recover Retrieve all experiments from the Chaosd service by making a GET request to the `/api/experiments/` path. This is useful for programmatic access to experiment data. ```bash curl -X GET 127.0.0.1:31767/api/experiments/ ``` -------------------------------- ### View Garbage Collection Help Source: https://chaos-mesh.org/docs/simulate-jvm-application-chaos-in-physical-nodes Shows usage and configuration flags for the JVM garbage collection command. ```bash chaosd attack jvm gc --help ``` ```text trigger GC for JVM Usage: chaosd attack jvm gc [flags] Flags: -h, --help help for gc Global Flags: --log-level string the log level of chaosd. The value can be 'debug', 'info', 'warn' and 'error' --pid int the pid of Java process which needs to attach --port int the port of agent server (default 9288) --uid string the experiment ID ``` -------------------------------- ### Create Experiment via Chaosd Service API Source: https://chaos-mesh.org/docs/chaosd-search-recover Initiate a CPU stress experiment by sending a POST request to the Chaosd service. The response will contain the experiment's UID, which is needed for recovery. ```bash curl -X POST 172.16.112.130:31767/api/attack/stress -H "Content-Type:application/json" -d '{"load":10, "action":"cpu","workers":1}' ``` -------------------------------- ### Implement gRPC Server Logic Source: https://chaos-mesh.org/docs/extend-chaos-daemon-interface Create the server implementation in the Chaos Daemon to handle the new gRPC request. ```go package chaosdaemon import ( "context" "github.com/golang/protobuf/ptypes/empty" "github.com/chaos-mesh/chaos-mesh/pkg/bpm" "github.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb" ) func (s *DaemonServer) ExecHelloWorldChaos(ctx context.Context, req *pb.ExecHelloWorldRequest) (*empty.Empty, error) { log := s.getLoggerFromContext(ctx) log.Info("ExecHelloWorldChaos", "request", req) pid, err := s.crClient.GetPidFromContainerID(ctx, req.ContainerId) if err != nil { return nil, err } cmd := bpm.DefaultProcessBuilder("sh", "-c", "ps aux"). SetContext(ctx). SetNS(pid, bpm.MountNS). Build(ctx) out, err := cmd.Output() if err != nil { return nil, err } if len(out) != 0 { log.Info("cmd output", "output", string(out)) } return &empty.Empty{}, nil } ``` -------------------------------- ### HelloWorldChaos Configuration Source: https://chaos-mesh.org/docs/add-new-chaos-experiment-type YAML definition for the HelloWorldChaos experiment targeting the hello-minikube deployment. ```yaml apiVersion: chaos-mesh.org/v1alpha1 kind: HelloWorldChaos metadata: name: hello-world namespace: chaos-mesh spec: selector: labelSelectors: app: hello-minikube mode: one duration: 1h ``` -------------------------------- ### Get Base64 Encoded Chaos Experiment Configuration Source: https://chaos-mesh.org/docs/integrate-chaos-mesh-into-github-actions This command demonstrates how to get the base64-encoded value of a chaos experiment configuration file named 'chaos.yaml'. This encoded value is used as an environment variable in the GitHub Actions workflow. ```bash base64 chaos.yaml ``` -------------------------------- ### Create Workflow using kubectl Source: https://chaos-mesh.org/docs/create-chaos-mesh-workflow Demonstrates how to create a Chaos Mesh workflow using a local YAML file or a YAML file from a network URL with `kubectl`. ```APIDOC ## Create Workflow using kubectl ### Description Create a Chaos Mesh workflow using a local YAML file or a YAML file from the network with the `kubectl create -f` command. ### Commands **Create from local file:** ```bash kubectl create -f ``` **Create from network URL:** ```bash kubectl create -f https://raw.githubusercontent.com/chaos-mesh/chaos-mesh/master/examples/workflow/serial.yaml ``` ``` -------------------------------- ### Install Chaos Mesh CRDs for Older Kubernetes Versions Source: https://chaos-mesh.org/docs/faqs When installing Chaos Mesh on Kubernetes versions prior to v1.16, manually create CRDs using the v1beta1 version and disable validation. This ensures compatibility with older Kubernetes APIs. ```bash kubectl create -f https://mirrors.chaos-mesh.org/v2.1.0/crd-v1beta1.yaml --validate=false ``` -------------------------------- ### Install Chaos Mesh with Remote Debugging Enabled Source: https://chaos-mesh.org/docs/configure-development-environment Install Chaos Mesh using Helm, enabling remote debugging and disabling leader election for easier debugging. Leader election is disabled to ensure only one controller-manager instance is created. ```bash helm upgrade --install chaos-mesh-debug ./helm/chaos-mesh -n=chaos-mesh-debug --create-namespace --set chaosDlv.enable=true --set controllerManager.leaderElection.enabled=false ``` -------------------------------- ### View JVM Latency Help Source: https://chaos-mesh.org/docs/simulate-jvm-application-chaos-in-physical-nodes Displays help information for the JVM latency injection command. ```bash chaosd attack jvm latency --help ``` ```text inject latency to specified method Usage: chaosd attack jvm latency [options] [flags] Flags: -c, --class string Java class name -h, --help help for latency --latency int the latency duration, unit ms -m, --method string the method name in Java class Global Flags: --log-level string the log level of chaosd. The value can be 'debug', 'info', 'warn' and 'error' --pid int the pid of Java process which needs to attach --port int the port of agent server (default 9288) --uid string the experiment ID ``` -------------------------------- ### DNS Fault Command Help Source: https://chaos-mesh.org/docs/simulate-network-chaos-in-physical-nodes Displays help information for the DNS fault simulation command. ```bash chaosd attack network dns --help ``` -------------------------------- ### GET /api/experiments/ Source: https://chaos-mesh.org/docs/chaosd-search-recover Retrieves a list of all experiments managed by the Chaosd service. ```APIDOC ## GET /api/experiments/ ### Description Retrieves a list of all experiments currently managed by the Chaosd service. ### Method GET ### Endpoint /api/experiments/ ### Response #### Success Response (200) - **id** (int) - Unique identifier of the experiment record - **uid** (string) - Unique identifier of the experiment - **created_at** (string) - Timestamp of creation - **updated_at** (string) - Timestamp of last update - **status** (string) - Current status of the experiment - **kind** (string) - Type of attack - **action** (string) - Action performed - **recover_command** (string) - JSON string containing recovery configuration - **launch_mode** (string) - Mode in which the experiment was launched #### Response Example [ { "id": 1, "uid": "ddc5ca81-b677-4595-b691-0ce57bedb156", "created_at": "2021-10-18T16:01:18.563542491+08:00", "updated_at": "2021-10-18T16:07:27.87111393+08:00", "status": "success", "kind": "stress", "action": "mem", "recover_command": "{\"schedule\":\"\",\"duration\":\"\",\"action\":\"mem\",\"kind\":\"stress\",\"uid\":\"ddc5ca81-b677-4595-b691-0ce57bedb156\",\"Load\":0,\"Workers\":0,\"Size\":\"100MB\",\"Options\":null,\"StressngPid\":0}", "launch_mode": "svr" } ] ``` -------------------------------- ### Configure Starting Deadline Seconds for NetworkChaos Source: https://chaos-mesh.org/docs/define-scheduling-rules This configuration illustrates the use of `startingDeadlineSeconds` to manage missed experiments. When set to 5, it limits the lookback period for missed tasks, potentially preventing experiment creation if the window is too small relative to experiment duration and concurrency policies. ```yaml spec: schedule: '* * * * *' type: 'NetworkChaos' networkChaos: action: delay mode: one selector: namespaces: - default labelSelectors: 'app': 'web-show' startingDeadlineSeconds: 5 delay: latency: '10ms' duration: '70s' ``` -------------------------------- ### Get HelloWorldChaos CRD Source: https://chaos-mesh.org/docs/add-new-chaos-experiment-type Retrieves the status and details of the registered HelloWorldChaos CRD. ```bash kubectl get crd helloworldchaos.chaos-mesh.org ``` -------------------------------- ### Disable Safe Mode Source: https://chaos-mesh.org/docs/production-installation-using-helm Disable authentication for the Chaos Mesh dashboard during installation or upgrade. ```bash helm install chaos-mesh chaos-mesh/chaos-mesh -n=chaos-mesh --set dashboard.securityMode=false --version 2.8.2 ``` -------------------------------- ### Create Namespace for Target Application Source: https://chaos-mesh.org/docs/simulate-jvm-application-chaos Use this kubectl command to create a new Kubernetes namespace named 'helloworld' for the target application. ```bash kubectl create namespace helloworld ``` -------------------------------- ### Simulate Network Latency Source: https://chaos-mesh.org/docs/simulate-network-chaos-in-physical-nodes This section provides an example command to simulate network latency. ```APIDOC ## POST /chaosd/attack/network/delay ### Description Simulates network latency by introducing delay on network traffic. ### Method POST ### Endpoint /chaosd/attack/network/delay ### Parameters #### Query Parameters - **device** (string) - Required - The network interface to impact (e.g., eth0). - **ip** (string) - Optional - The IP address to impact. - **latency** (string) - Required - The amount of delay to introduce (e.g., 10ms). Time units: ns, us, ms, s, m, h. - **correlation** (string) - Optional - The correlation between the current latency and the previous one (percentage, e.g., "10"). Defaults to "0". - **egress-port** (string) - Optional - Only impact egress traffic to these destination ports. Use ',' to separate or indicate a range (e.g., "80,8001:8010"). Requires `-p` to be set to `tcp` or `udp`. - **hostname** (string) - Optional - Only impact traffic to these hostnames. Cannot be empty if `ip` is not set. - **jitter** (string) - Optional - The range of the length of network delay time (e.g., "1ms"). Time units: ns, us, ms, s, m, h. - **protocol** (string) - Optional - The IP protocol to impact. Supported: "tcp", "udp", "icmp", "all". - **source-port** (string) - Optional - Only impact egress traffic from these source ports. Use ',' to separate or indicate a range (e.g., "80,8001:8010"). Requires `-p` to be set to `tcp` or `udp`. ### Request Example ```bash chaosd attack network delay -d eth0 -i 172.16.4.4 -l 10ms ``` ### Response #### Success Response (200) - **uid** (string) - The unique identifier for the attack. #### Response Example ```json { "uid": "4b23a0b5-e193-4b27-90a7-3e04235f32ab" } ``` ``` -------------------------------- ### View JVM stress command help Source: https://chaos-mesh.org/docs/simulate-jvm-application-chaos-in-physical-nodes Displays usage and configuration options for the JVM stress attack command. ```bash chaosd attack jvm stress --help ``` -------------------------------- ### Define a parallel template step Source: https://chaos-mesh.org/docs/create-chaos-mesh-workflow Example of a template step configured for parallel execution. ```yaml name: the-entry templateType: Parallel deadline: 240s children: - workflow-stress-chaos - workflow-network-chaos - workflow-pod-chaos ``` -------------------------------- ### Simulate Disk Fill using Service Mode Source: https://chaos-mesh.org/docs/simulate-disk-pressure-in-physical-nodes Use this command to simulate disk fill by sending a POST request to the Chaos Mesh API. Ensure the path is writable and specify either 'size' or 'percent' for the fill amount. ```bash curl -X POST 172.16.112.130:31767/api/attack/disk -H "Content-Type:application/json" -d '{"action":"fill","path":"/tmp/test", "fill-by-fallocate":true,"percent":"50"}' ```