### Dataset and Runtime Ready Status Example Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/automatic_clean_up_data_operation.md This output shows an example of a ready Dataset and AlluxioRuntime. The 'Bound' phase and capacity information confirm successful setup. ```shell NAME UFS TOTAL SIZE CACHED CACHE CAPACITY CACHED PERCENTAGE PHASE AGE hbase 1.21GiB 0.00B 2.00GiB 0.0% Bound 75s ``` -------------------------------- ### Install Gorilla WebSocket Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/gorilla/websocket/README.md Use go get to install the Gorilla WebSocket package. ```bash go get github.com/gorilla/websocket ``` -------------------------------- ### Install Mergo Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/imdario/mergo/README.md Install the Mergo library using go get. ```go go get github.com/imdario/mergo ``` -------------------------------- ### Install automaxprocs Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/go.uber.org/automaxprocs/README.md Install the automaxprocs package using go get. ```bash go get -u go.uber.org/automaxprocs ``` -------------------------------- ### Install Cobra Library Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/spf13/cobra/README.md Use 'go get' to install the latest version of the Cobra library. ```bash go get -u github.com/spf13/cobra@latest ``` -------------------------------- ### Basic Structured Logging Example Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/sigs.k8s.io/controller-runtime/TMP-LOGGING.md Compares traditional Go logging with controller-runtime's structured logging for starting reconciliation. ```go log.Printf("starting reconciliation for pod %s/%s", podNamespace, podName) ``` ```go logger.Info("starting reconciliation", "pod", req.NamespacedName) ``` -------------------------------- ### Example Application Pod Node Information Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/troubleshooting/debug-fuse.md An example output of 'kubectl get pods -owide', showing Pod details and the node it is running on. ```shell kubectl get pods nginx-0 -owide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-0 0/1 ContainerCreating 0 17m testnode ``` -------------------------------- ### Install Ginkgo Locally Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/onsi/ginkgo/v2/CONTRIBUTING.md Installs the Ginkgo testing framework locally using `go install`. This is a prerequisite for running tests and other development commands. ```bash go install ./... ``` -------------------------------- ### Install json-iterator/go Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/json-iterator/go/README.md Use the go get command to install the json-iterator/go library. ```go go get github.com/json-iterator/go ``` -------------------------------- ### Install uuid Package Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/google/uuid/README.md Install the uuid package using the go get command. ```shell go get github.com/google/uuid ``` -------------------------------- ### Start Dataset Component Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/dev_with_kind.md Navigate to the dataset command directory and execute the start script. This script sets an environment variable for the init image and starts the component with metrics enabled. ```shell cd ~/go/src/github.com/fluid-cloudnative/fluid/cmd/dataset && sh start.sh ``` ```shell #! /bin/bash export ALLUXIO_INIT_IMAGE_ENV=registry.cn-hangzhou.aliyuncs.com/fluid/init-users:v0.4.0-a8ba7c9 go run main.go start \ --metrics-addr=127.0.0.1:8082 ``` -------------------------------- ### Install multierr Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/go.uber.org/multierr/README.md Install the latest version of the multierr package using go get. ```bash go get -u go.uber.org/multierr@latest ``` -------------------------------- ### Start Minio Demo Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/accelerate_s3_minio.md Starts a standalone Minio instance locally as a remote S3 service using Docker. This is for demonstration purposes only. ```shell docker run -ti -p 9000:9000 --name minio minio/minio server /data ``` -------------------------------- ### Start Alluxio Component Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/dev_with_kind.md Navigate to the Alluxio command directory and execute the start script. This script sets environment variables for the init image and mount root before running the main start command. ```shell cd ~/go/src/github.com/fluid-cloudnative/fluid/cmd/alluxio && sh start.sh ``` ```shell #! /bin/bash export ALLUXIO_INIT_IMAGE_ENV=registry.cn-hangzhou.aliyuncs.com/fluid/init-users:v0.4.0-a8ba7c9 export MOUNT_ROOT=/alluxio-mnt go run main.go start ``` -------------------------------- ### Clone Repository and Setup Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/go.uber.org/automaxprocs/CONTRIBUTING.md Steps to fork, clone the repository, and set up the local development environment for automaxprocs. ```bash mkdir -p $GOPATH/src/go.uber.org cd $GOPATH/src/go.uber.org git clone git@github.com:your_github_username/automaxprocs.git cd automaxprocs git remote add upstream https://github.com/uber-go/automaxprocs.git git fetch upstream ``` -------------------------------- ### Install YAML Package for Go Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/go.yaml.in/yaml/v3/README.md Install the yaml v3 package using the go get command. ```bash go get go.yaml.in/yaml/v3 ``` -------------------------------- ### Download and Unzip async-profiler Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/profiling.md Download the async-profiler tool and extract it to your system. This is the initial setup step before profiling. ```bash $ wget https://github.com/jvm-profiling-tools/async-profiler/releases/download/v1.7.1/async-profiler-1.7.1-linux-x64.tar.gz $ tar -zxf async-profiler-1.7.1-linux-x64.tar.gz ``` -------------------------------- ### Install fluid-databackup using Helm Source: https://github.com/fluid-cloudnative/fluid/blob/master/charts/fluid-databackup/alluxio/README.md Installs the fluid-databackup chart. Ensure prerequisites are met before installation. ```shell helm install charts/fluid-databackup ``` -------------------------------- ### Install gomonkey (below v2.1.0) Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/agiledragon/gomonkey/v2/README.md Use this command to install versions of gomonkey prior to v2.1.0. ```bash $ go get github.com/agiledragon/gomonkey@v2.0.2 ``` -------------------------------- ### Initialize Procfs and Get Stats Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/prometheus/procfs/README.md Initialize the proc filesystem and retrieve general statistics. This is a common starting point for accessing procfs data. ```go fs, err := procfs.NewFS("/proc") stats, err := fs.Stat() ``` -------------------------------- ### Ginkgo Spec Example Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/onsi/ginkgo/v2/README.md An example of a Ginkgo spec demonstrating BeforeEach, When, Context, It, and SpecTimeout for testing library book checkout functionality. It includes setup, assertions, and handling of various scenarios like book availability and checkout status. ```go import ( . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" ... ) var _ = Describe("Checking books out of the library", Label("library"), func() { var library *libraries.Library var book *books.Book var valjean *users.User BeforeEach(func() { library = libraries.NewClient() book = &books.Book{ Title: "Les Miserables", Author: "Victor Hugo", } valjean = users.NewUser("Jean Valjean") }) When("the library has the book in question", func() { BeforeEach(func(ctx SpecContext) { Expect(library.Store(ctx, book)).To(Succeed()) }) Context("and the book is available", func() { It("lends it to the reader", func(ctx SpecContext) { Expect(valjean.Checkout(ctx, library, "Les Miserables")).To(Succeed()) Expect(valjean.Books()).To(ContainElement(book)) Expect(library.UserWithBook(ctx, book)).To(Equal(valjean)) }, SpecTimeout(time.Second * 5)) }) Context("but the book has already been checked out", func() { var javert *users.User BeforeEach(func(ctx SpecContext) { javert = users.NewUser("Javert") Expect(javert.Checkout(ctx, library, "Les Miserables")).To(Succeed()) }) It("tells the user", func(ctx SpecContext) { err := valjean.Checkout(ctx, library, "Les Miserables") Expect(err).To(MatchError("Les Miserables is currently checked out")) }, SpecTimeout(time.Second * 5)) It("lets the user place a hold and get notified later", func(ctx SpecContext) { Expect(valjean.Hold(ctx, library, "Les Miserables")).To(Succeed()) Expect(valjean.Holds(ctx)).To(ContainElement(book)) By("when Javert returns the book") Expect(javert.Return(ctx, library, book)).To(Succeed()) By("it eventually informs Valjean") notification := "Les Miserables is ready for pick up" Eventually(ctx, valjean.Notifications).Should(ContainElement(notification)) Expect(valjean.Checkout(ctx, library, "Les Miserables")).To(Succeed()) Expect(valjean.Books(ctx)).To(ContainElement(book)) Expect(valjean.Holds(ctx)).To(BeEmpty()) }, SpecTimeout(time.Second * 10)) }) }) When("the library does not have the book in question", func() { It("tells the reader the book is unavailable", func(ctx SpecContext) { err := valjean.Checkout(ctx, library, "Les Miserables") Expect(err).To(MatchError("Les Miserables is not in the library catalog")) }, SpecTimeout(time.Second * 5)) }) }) ``` -------------------------------- ### Example Release Announcement Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/sigs.k8s.io/controller-runtime/RELEASE.md An example of how to announce a new release on a Slack channel, including version, dependency information, and a link to the release page. ```text :announce: Controller-Runtime v0.12.0 has been released! This release includes a Kubernetes dependency bump to v1.24. For more info, see the release page: https://github.com/kubernetes-sigs/controller-runtime/releases. :tada: Thanks to all our contributors! ``` -------------------------------- ### Install gomonkey (v2.1.0 and above) Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/agiledragon/gomonkey/v2/README.md Use this command to install gomonkey version v2.1.0 or later. ```bash $ go get github.com/agiledragon/gomonkey/v2@v2.11.0 ``` -------------------------------- ### CacheRuntime Configuration Example Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/generic_cache_runtime_integration.md Example JSON configuration provided by Fluid to components, detailing mounts, access modes, and component-specific settings for Master, Worker, and Client. ```json { "mounts": [ { "mountPoint": "s3://test", "options": { "access": "minioadmin", "endpoint_url": "http://minio:9000", "path_style": "true", "region_name": "us-east-1", "secret": "minioadmin" }, "encryptOptions": { "access-key": "/etc/fluid/secrets/minio-secret/access-key", "secret-key": "/etc/fluid/secrets/minio-secret/secret-key" }, "name": "minio", "path": "/minio" } ], "accessModes": [ "ReadWriteMany" ], "targetPath": "/runtime-mnt/cache/default/curvine-demo/cache-fuse", "master": { "enabled": true, "name": "curvine-demo-master", "options": { "key1": "master-value1" }, "replicas": 1, "service": { "name": "svc-curvine-demo-master" } }, "worker": { "enabled": true, "name": "curvine-demo-worker", "options": { "key1": "worker-value1" }, "replicas": 1, "service": { "name": "svc-curvine-demo-worker" } }, "client": { "enabled": true, "name": "curvine-demo-client", "options": { "key1": "value1" }, "service": { "name": "" } } } ``` -------------------------------- ### Install Fluid with Helm Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/userguide/install.md Installs the Fluid chart using Helm. The output shows the release name, deployment status, and revision. ```shell helm install fluid fluid/fluid NAME: fluid LAST DEPLOYED: Wed May 24 18:17:16 2023 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None ``` -------------------------------- ### Set Up Workspace for SSD Acceleration Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/accelerate_data_by_mem_or_ssd.md Create a dedicated directory for your SSD acceleration setup. ```shell $ mkdir /ssd $ cd /ssd ``` -------------------------------- ### Download Docker Image for Documentation Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/README.md Pull the necessary Docker image for building Fluid documentation. Ensure Docker is installed. ```shell docker pull registry.cn-hangzhou.aliyuncs.com/docs-fluid/doc-build ``` -------------------------------- ### Example: Merging Structs Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/imdario/mergo/README.md Demonstrates a practical example of merging two Foo structs. The destination struct's zero values are updated with values from the source struct. ```go package main import ( "fmt" "github.com/imdario/mergo" ) type Foo struct { A string B int64 } func main() { src := Foo{ A: "one", B: 2, } dest := Foo{ A: "two", } mergo.Merge(&dest, src) fmt.Println(dest) // Will print // {two 2} } ``` -------------------------------- ### Node Driver Registrar Start Script Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/dev_with_kind.md Shell script to configure environment variables and start the node-driver-registrar. It removes any existing registration socket, then runs the Go application to register the CSI driver with the kubelet using specified paths. ```shell #! /bin/bash set -x export TMPDIR=/root/go/tmp export GO111MODULE=on export GOMODCACHE=/root/go/pkg/mod export GOPROXY=https://goproxy.io export GOPATH=/home/work/go export GOROOT=/home/work/go/local/go export GOBIN=/home/work/go/bin export PATH=$PATH:$GOBIN:$GOROOT/bin if [ ! -d $TMPDIR ]; then mkdir -p $TMPDIR fi # delete reg socket if exist rm -rf /var/lib/kubelet/plugins_registry/fuse.csi.fluid.io-reg.sock go run main.go \ --kubelet-registration-path="/var/lib/kubelet/csi-plugins/fuse.csi.fluid.io/csi.sock" \ --csi-address="/var/lib/kubelet/csi-plugins/fuse.csi.fluid.io/csi.sock" \ --reg-path="/var/lib/kubelet/plugins_registry" \ --v=5 ``` -------------------------------- ### Install Fluid System Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/how_to_develop.md Deploy the Fluid system components to the Kubernetes cluster. ```shell $ kubectl apply -k config/fluid ``` -------------------------------- ### Deploy Prometheus using kubectl Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/operation/monitoring.md Quickly deploy Prometheus using a provided YAML file. This method is suitable for testing but not recommended for production environments. Refer to the official Prometheus installation guide for production setups. ```shell cd community kubectl apply -f integration/prometheus/prometheus.yaml ``` -------------------------------- ### Set Up Workspace Directory Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/dataset_scaling.md Create and navigate to a new directory for the dataset scaling demo. This prepares your environment for the subsequent steps. ```shell $ mkdir /dataset_scale $ cd /dataset_scale ``` -------------------------------- ### Set Up Workspace Directory Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/data_co_locality.md Create a dedicated directory for the co-locality demo and navigate into it. This directory will serve as the workspace for the demonstration. ```shell mkdir /co-locality cd /co-locality ``` -------------------------------- ### Start Docker Container for Documentation Build Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/README.md Run a Docker container, mounting your Fluid documentation path to the container's data directory. This prepares the environment for building the documentation. ```shell docker run -it -v :/data/ fluid/doc-build:0.2.0 ``` -------------------------------- ### Create Workspace Directory Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/accelerate_data_accessing.md Prepare a local directory for the demonstration. This involves creating a new directory and navigating into it. ```shell mkdir /accelerate cd /accelerate ``` -------------------------------- ### SpdyStream Server Example Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/moby/spdystream/README.md Listens for incoming TCP connections, establishes SpdyStream connections, and serves streams using a mirroring handler. ```go package main import ( "github.com/moby/spdystream" "net" ) func main() { listener, err := net.Listen("tcp", "localhost:8080") if err != nil { panic(err) } for { conn, err := listener.Accept() if err != nil { panic(err) } spdyConn, err := spdystream.NewConnection(conn, true) if err != nil { panic(err) } go spdyConn.Serve(spdystream.MirrorStreamHandler) } } ``` -------------------------------- ### Install Latest JSON-Patch Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/evanphx/json-patch/README.md Install the latest version of the jsonpatch library using go get. ```bash go get -u github.com/evanphx/json-patch/v5 ``` -------------------------------- ### Create Root Logger with Implementation Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/go-logr/logr/README.md Demonstrates how to create the initial 'root' logger in an application's main function, specifying the desired logging implementation. ```go func main() { // ... other setup code ... // Create the "root" logger. We have chosen the "logimpl" implementation, // which takes some initial parameters and returns a logr.Logger. logger := logimpl.New(param1, param2) // ... other setup code ... } ``` -------------------------------- ### Set Up Workspace for Data Preloading Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/data_warmup.md Create and navigate to a new directory to set up your workspace for data preloading operations. ```bash mkdir /warmup cd /warmup ``` -------------------------------- ### Define a Basic WebService Route Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/emicklei/go-restful/v3/README.md Example of defining a RESTful WebService with path, consumes, produces, and a GET route. The route is mapped to the findUser function. ```go ws := new(restful.WebService) ws. Path("/users"). Consumes(restful.MIME_XML, restful.MIME_JSON). Produces(restful.MIME_JSON, restful.MIME_XML) ws.Route(ws.GET("/{user-id}").To(u.findUser). Doc("get a user"). Param(ws.PathParameter("user-id", "identifier of the user").DataType("string")). Writes(User{})) ... func (u UserResource) findUser(request *restful.Request, response *restful.Response) { id := request.PathParameter("user-id") ... } ``` -------------------------------- ### Start Fluid Job with AlluxioRuntime Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/operation/monitoring.md Define and apply a Dataset and AlluxioRuntime configuration for a Fluid job. This example demonstrates how to set up the runtime, including storage configuration and Prometheus monitoring settings. ```yaml apiVersion: data.fluid.io/v1alpha1 kind: Dataset metadata: name: spark spec: mounts: - mountPoint: https://mirrors.bit.edu.cn/apache/spark/ name: spark --- apiVersion: data.fluid.io/v1alpha1 kind: AlluxioRuntime metadata: name: spark spec: replicas: 2 tieredstore: levels: - mediumtype: MEM path: /dev/shm quota: 1Gi high: "0.95" low: "0.7" # By default, after v0.5.0, aluxio runtime has Prometheus data turned on, if you need to turn it off you can actively set disablePrometheus: true # disablePrometheus: false ``` -------------------------------- ### Quick Start: Import automaxprocs Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/go.uber.org/automaxprocs/README.md Import the automaxprocs package to automatically set GOMAXPROCS on application startup. No further configuration is needed. ```go import _ "go.uber.org/automaxprocs" func main() { // Your application logic here. } ``` -------------------------------- ### Manually Install CRDs with Helm Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/userguide/faq.md Use this command to manually install CRDs if using a Helm version below 3, as older versions do not automatically install them. ```bash kubectl create -f fluid/crds ``` -------------------------------- ### SpdyStream Client Example Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/moby/spdystream/README.md Connects to a mirroring server without authentication, creates a stream, writes data, reads a response, and closes the stream. ```go package main import ( "fmt" "github.com/moby/spdystream" "net" "net/http" ) func main() { conn, err := net.Dial("tcp", "localhost:8080") if err != nil { panic(err) } spdyConn, err := spdystream.NewConnection(conn, false) if err != nil { panic(err) } go spdyConn.Serve(spdystream.NoOpStreamHandler) stream, err := spdyConn.CreateStream(http.Header{}, nil, false) if err != nil { panic(err) } stream.Wait() fmt.Fprint(stream, "Writing to stream") buf := make([]byte, 25) stream.Read(buf) fmt.Println(string(buf)) stream.Close() } ``` -------------------------------- ### Install Fluid on Kubernetes Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/arm64.md Creates a namespace for Fluid and installs the Fluid Helm chart. This command assumes the Helm chart is named 'fluid' and will be installed in the default namespace if none is specified. ```shell $ kubectl create ns fluid-system $ helm install fluid fluid ``` -------------------------------- ### Set Up Workspace Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/accelerate_data_accessing_by_hdfs.md Create a dedicated directory for HDFS-related operations. Navigate into this directory to organize your configuration files. ```shell mkdir /hdfs cd /hdfs ``` -------------------------------- ### Install Stable JSON-Patch Versions Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/evanphx/json-patch/README.md Install specific stable versions of the jsonpatch library. ```bash go get -u gopkg.in/evanphx/json-patch.v5 ``` ```bash go get -u gopkg.in/evanphx/json-patch.v4 ``` -------------------------------- ### Deploy Fluid Samples Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/how_to_develop.md Apply sample configurations to test the Fluid implementation. ```shell $ kubectl apply -k config/samples ``` -------------------------------- ### Basic fsnotify Watcher Example Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/fsnotify/fsnotify/README.md This Go code demonstrates how to create a new fsnotify watcher, set up goroutines to listen for events and errors, and add a path to monitor. It includes deferring the watcher's close and blocking the main goroutine indefinitely. ```go package main import ( "log" "github.com/fsnotify/fsnotify" ) func main() { // Create new watcher. watcher, err := fsnotify.NewWatcher() if err != nil { log.Fatal(err) } defer watcher.Close() // Start listening for events. go func() { for { select { case event, ok := <-watcher.Events: if !ok { return } log.Println("event:", event) if event.Has(fsnotify.Write) { log.Println("modified file:", event.Name) } case err, ok := <-watcher.Errors: if !ok { return } log.Println("error:", err) } } }() // Add a path. err = watcher.Add("/tmp") if err != nil { log.Fatal(err) } // Block main goroutine forever. <-make(chan struct{}) } ``` -------------------------------- ### Check Sample Pods Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/how_to_develop.md Verify that the sample pods are running after deployment. ```shell $ kubectl get pod NAME READY STATUS RESTARTS AGE cifar10-fuse-vb6l4 1/1 Running 0 6m15s cifar10-fuse-vtqpx 1/1 Running 0 6m15s cifar10-master-0 2/2 Running 0 8m24s cifar10-worker-729xz 2/2 Running 0 6m15s cifar10-worker-d6kmd 2/2 Running 0 6m15s nginx-0 1/1 Running 0 8m30s nginx-1 1/1 Running 0 8m30s ``` -------------------------------- ### Create and Copy Test File to Minio Bucket Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/thinruntime.md These bash commands create a test file, copy it to the Minio bucket 'my-first-bucket', and then display its content. Ensure the Minio pod name is correct. ```bash $ kubectl create -f minio.yaml ``` ```bash $ kubectl exec -it minio-69c555f4cf-np59j -- bash -c "echo fluid-minio-test > testfile" ``` ```bash $ kubectl exec -it minio-69c555f4cf-np59j -- bash -c "mc cp ./testfile local/my-first-bucket/" ``` ```bash $ kubectl exec -it minio-69c555f4cf-np59j -- bash -c "mc cat local/my-first-bucket/testfile" ``` -------------------------------- ### Install jq Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/operation/dataset_auto_scaling.md Install the jq tool for parsing JSON, using yum on CentOS. ```shell $ yum install -y jq ``` -------------------------------- ### Install Test Dependencies Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/go.uber.org/automaxprocs/CONTRIBUTING.md Installs the necessary dependencies required for running tests in the automaxprocs project. ```bash make dependencies ``` -------------------------------- ### Gzip Stateless Compression with Bufio Writer Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/klauspost/compress/README.md Demonstrates how to use gzip with stateless compression and a bufio.Writer for controlled memory usage. Replace ioutil.Discard with your actual output destination. ```go import ( "bufio" "compress/gzip" "io/ioutil" ) // replace 'ioutil.Discard' with your output. gzw, err := gzip.NewWriterLevel(ioutil.Discard, gzip.StatelessCompression) if err != nil { return err } def gzw.Close() w := bufio.NewWriterSize(gzw, 4096) def w.Flush() // Write to 'w' ``` -------------------------------- ### Install Cobra CLI Generator Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/spf13/cobra/README.md Install the 'cobra-cli' command-line program to generate Cobra application scaffolding. ```bash go install github.com/spf13/cobra-cli@latest ``` -------------------------------- ### Install YAML Package Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/imdario/mergo/README.md If tests are failing due to a missing package, execute this command to install gopkg.in/yaml.v2. ```go go get gopkg.in/yaml.v2 ``` -------------------------------- ### Verbose Logging with Formatted Output (Level 2) Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/k8s.io/klog/v2/README.md This example shows how to use glog.V with Infoln for formatted verbose output, including variables. ```go glog.V(2).Infoln("Processed", nItems, "elements") ``` -------------------------------- ### Install Kubebuilder for Integration Tests Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/how_to_develop.md Download and install Kubebuilder, a tool for building Kubernetes APIs and controllers, which is used for integration testing. ```shell $ os=$(go env GOOS) $ arch=$(go env GOARCH) $ curl -L https://go.kubebuilder.io/dl/2.3.1/${os}/${arch} | tar -xz -C /tmp/ $ sudo mv /tmp/kubebuilder_2.3.1_${os}_${arch} /usr/local/kubebuilder $ export PATH=$PATH:/usr/local/kubebuilder/bin ``` -------------------------------- ### Git Commit Sign-off Example Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/docker/go-units/CONTRIBUTING.md An example of how to format the sign-off line in a git commit message. This confirms the Developer Certificate of Origin. ```git Signed-off-by: Joe Smith ``` -------------------------------- ### Quick Start with Logger Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/go.uber.org/zap/README.md Use Logger when performance and type safety are critical. It is faster than SugaredLogger and allocates less, but only supports structured logging. ```go logger, _ := zap.NewProduction() deferr logger.Sync() logger.Info("failed to fetch URL", // Structured context as strongly typed Field values. zap.String("url", url), zap.Int("attempt", 3), zap.Duration("backoff", time.Second), ) ``` -------------------------------- ### Example Application Pod Events Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/troubleshooting/debug-fuse.md An example of Kubernetes Pod events, highlighting a 'FailedMount' warning related to a Fuse Pod not being ready. ```shell kubectl describe po nginx-0 ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 30s default-scheduler Successfully assigned default/nginx-0 to testnode Warning FailedMount 1s kubelet MountVolume.MountDevice failed for volume "default-shared-data" : rpc error: code = Unknown desc = fuse pod on node testnode is not ready ``` -------------------------------- ### Parse Version and Constraint, then Validate Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/Masterminds/semver/v3/README.md Demonstrates creating a version constraint and a version, then validating the version against the constraint. It shows how to access validation error messages if the validation fails. ```go c, err := semver.NewConstraint("<= 1.2.3, >= 1.4") if err != nil { // Handle constraint not being parseable. } v, err := semver.NewVersion("1.3") if err != nil { // Handle version not being parseable. } // Validate a version against a constraint. a, msgs := c.Validate(v) // a is false for _, m := range msgs { fmt.Println(m) // Loops over the errors which would read // "1.3 is greater than 1.2.3" // "1.3 is less than 1.4" } ``` -------------------------------- ### Install Argo Workflow Controller Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/vineyard/accelerate_kubeflow_pipelines.md Installs the Argo workflow controller into a dedicated 'argo' namespace. This controller serves as the backend for Kubeflow pipelines. ```shell $ kubectl create ns argo $ kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/download/v3.4.8/install.yaml ``` -------------------------------- ### Coexisting with glog Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/k8s.io/klog/v2/README.md Illustrates how to use klog/v2 alongside the original glog library. This example synchronizes flags and directs all logs to stderr. ```Go package main import ( "flag" "k8s.io/klog/v2" // Import glog to demonstrate coexistence // "github.com/golang/glog" ) func main() { // Initialize klog flags. klog.InitFlags(nil) // If you were also using glog, you would initialize its flags here. // glog.InitFlags() // Set alsologtostderr to true to ensure all logs go to stderr. // This flag is common to both klog and glog. flag.Set("alsologtostderr", "true") klog.Info("This is a klog message.") // If glog were used, its messages would also appear on stderr. // glog.Info("This is a glog message.") // klog.Flush() is called automatically on exit. // glog.Flush() would also be called if glog was active. } ``` -------------------------------- ### Create and Check Version Constraint Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/Masterminds/semver/v3/README.md Demonstrates how to create a new version constraint and check if a given version satisfies it. Handles potential errors during constraint or version parsing. ```go c, err := semver.NewConstraint(">= 1.2.3") if err != nil { // Handle constraint not being parsable. } v, err := semver.NewVersion("1.3") if err != nil { // Handle version not being parsable. } // Check if the version meets the constraints. The variable a will be true. a := c.Check(v) ``` -------------------------------- ### Example Failing Fuse Pod Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/troubleshooting/debug-fuse.md An example output of filtering for Fuse Pods on a specific node, showing a Fuse Pod in 'CrashLoopBackOff' state. ```shell kubectl get po -owide | grep testnode | grep fuse shared-data-alluxio-fuse-w6lcp 0/1 CrashLoopBackOff 10 29m 192.168.0.233 testnode ``` -------------------------------- ### CephFS Mount Type Example Source: https://github.com/fluid-cloudnative/fluid/blob/master/addons/cephfs/dev-guide/cephfs.md Example output from a mount command showing the filesystem type for CephFS, which can be used to configure `fileSystemType` in ThinRuntimeProfile. ```shell ceph-fuse on /runtime-mnt/thin/default/ceph-demo/thin-fuse type fuse.ceph-fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) ``` -------------------------------- ### HDFS Client Initialization Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/samples/accelerate_data_accessing_by_hdfs.md Initialize the FileSystem with the Alluxio HDFS URL. Ensure the HDFS_URL domain name follows the specified rule. ```java final String HDFS_URL = "alluxio://hadoop-master-0.default.svc.cluster.local:"+ System.getenv("HADOOP_PORT") + "/hadoop"; Configuration conf = new Configuration(); FileSystem fs = FileSystem.get(URI.create(HDFS_URL), conf); ``` -------------------------------- ### Install Fluid for Older Kubernetes Versions Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/userguide/install.md A specific Helm installation command for Kubernetes versions prior to v1.17, disabling a critical Fuse pod. ```shell helm install --set runtime.criticalFusePod=false fluid fluid.tgz ``` -------------------------------- ### Install Fluid CSI Driver Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/userguide/faq.md If the CSI driver is not installed, use this command to apply the driver configuration. This is often a necessary step to resolve volume attachment timeouts. ```bash kubectl apply -f charts/fluid/fluid/templates/CSI/driver.yaml ``` -------------------------------- ### Install and Verify ktctl for WebHook Debugging Source: https://github.com/fluid-cloudnative/fluid/blob/master/docs/en/dev/how_to_develop.md Install the kt-connect tool (ktctl) to proxy access to WebHooks in the cluster to your local machine. This is useful for debugging WebHook components. ```shell # 1. Install kt-connect(https://github.com/alibaba/kt-connect) $ curl -OL https://github.com/alibaba/kt-connect/releases/download/v0.3.7/ktctl_0.3.7_Linux_x86_64.tar.gz $ tar zxf ktctl_0.3.7_Linux_x86_64.tar.gz $ mv ktctl /usr/local/bin/ktctl $ ktctl --version ``` -------------------------------- ### Basic Usage with logr Source: https://github.com/fluid-cloudnative/fluid/blob/master/vendor/github.com/go-logr/zapr/README.md Demonstrates how to initialize a logr.Logger using zapr and log a message. Ensure zap and zapr are imported. ```go package main import ( "fmt" "go.uber.org/zap" "github.com/go-logr/logr" "github.com/go-logr/zapr" ) func main() { var log logr.Logger zapLog, err := zap.NewDevelopment() if err != nil { panic(fmt.Sprintf("who watches the watchmen (%v)?", err)) } log = zapr.NewLogger(zapLog) log.Info("Logr in action!", "the answer", 42) } ```