KServe

https://kserve.github.io/website/docs
Admin
KServe is a serverless model serving platform for machine learning and generative AI models,...
Tokens:350,285
Snippets:3,933
Trust Score:8.3
Update:1 month ago
Show doc for...
Tokens:
Raw
### Quick Install via Scripts

Source: https://kserve.github.io/website/docs/install/llmisvc-install

Commands to perform a quick installation using provided setup scripts.

```bash
cd kserve  
  
# Install dependencies + LLMInferenceService  
./hack/setup/quick-install/llmisvc-full-install-helm.sh  
  
# Or use with-manifest version (no clone needed, includes embedded manifests)  
./hack/setup/quick-install/llmisvc-full-install-helm-with-manifest.sh  
```

--------------------------------

### Example Installation Definition

Source: https://kserve.github.io/website/docs/install/overview

A YAML definition file for generating installation scripts, specifying tools and component configurations.

```yaml
# quick-install/definitions/kserve-standard-mode-full-install.definition  
DESCRIPTION: Install KServe Standard Mode using Helm  
RELEASE: true  
  
TOOLS:  
  - helm  
  - kustomize  
  - yq  
  
COMPONENTS:  
  - name: cert-manager-helm  
  - name: istio-helm  
  - name: kserve-helm  
    env:  
      DEPLOYMENT_MODE: Standard  
      ENABLE_KSERVE: true  
      ENABLE_LLMISVC: false
```

--------------------------------

### Local Documentation Build

Source: https://kserve.github.io/website/docs/developer-guide/contribution

Commands to install dependencies and start the local documentation preview server.

```bash
cd website
npm install
npm start
```

--------------------------------

### Example Script Structure

Source: https://kserve.github.io/website/docs/install/overview

A template for creating installation scripts that source common utilities and implement install/uninstall functions.

```bash
#!/bin/bash  
# Source common utilities  
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"  
source "${SCRIPT_DIR}/../common.sh"  
  
install() {  
  log_info "Installing cert-manager..."  
  helm repo add jetstack https://charts.jetstack.io  
  helm install cert-manager jetstack/cert-manager \
    --namespace cert-manager \
    --create-namespace \
    --set crds.enabled=true  
  
  wait_for_pods "cert-manager" "app.kubernetes.io/instance=cert-manager" 300  
  log_success "cert-manager installed successfully"  
}  
  
uninstall() {  
  log_info "Uninstalling cert-manager..."  
  helm uninstall cert-manager -n cert-manager  
  kubectl delete namespace cert-manager --wait=true  
}  
  
# Main execution  
if [ "$UNINSTALL" = true ]; then uninstall; exit 0; fi  
install
```

--------------------------------

### Install LLMInferenceService via Quick Install Script

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install the LLMInferenceService controller using the quick install script.

```bash
curl -sL "https://github.com/kserve/kserve/releases/download/v0.20.0/llmisvc-full-install-with-manifests.sh" | bash  
```

--------------------------------

### BaseKServeModel start Method

Source: https://kserve.github.io/website/docs/reference/python-runtime-sdk/python-runtime-sdk-api

Handler for performing model setup.

```python
def start()  
```

--------------------------------

### Quick Install KServe in Standard Mode via Scripts

Source: https://kserve.github.io/website/docs/install/kserve-install

Use the setup script or the manifest-based curl command to install KServe in Standard mode.

```bash
cd kserve  
  
# Install dependencies + KServe  
./hack/setup/quick-install/kserve-standard-mode-full-install-helm.sh  
  
# Or use with-manifest version (no clone needed, includes embedded manifests)  
curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-standard-mode-full-install-with-manifests.sh  
| bash  
```

--------------------------------

### Install Gateway Instance

Source: https://kserve.github.io/website/docs/admin-guide/kubernetes-deployment-llmisvc

Script to install the Gateway instance.

```bash
infra/gateway-api/manage.gateway-api-gw.sh  
```

--------------------------------

### Install Paddleserver Runtime

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/paddle

Navigate to the paddleserver directory and install the runtime using Uv.

```bash
cd python/paddleserver
uv sync
```

--------------------------------

### Install KServe Controller via Quick Install Script

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install the KServe controller using standard or Knative mode scripts.

```bash
# Standard mode  
curl -sL "https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-standard-mode-full-install-with-manifests.sh" | bash  
  
# Knative mode (serverless)  
curl -sL "https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-knative-mode-full-install-with-manifests.sh" | bash  
```

--------------------------------

### Install PMMLServer Runtime

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/pmml

Navigate to the pmmlserver directory and install dependencies using Uv.

```bash
cd python/pmmlserver
uv sync
```

--------------------------------

### Install LLMInferenceService via Kustomize

Source: https://kserve.github.io/website/docs/install/llmisvc-install

Commands to install the LLMInferenceService standalone, as an addon, or to install LLMInferenceServiceConfigs.

```bash
# Install LLMInferenceService standalone  
kubectl apply -k config/overlays/standalone/llmisvc  
```

```bash
# Install only LLMInferenceService component (no base resources, reuses existing namespace)  
kubectl apply -k config/overlays/addons/llmisvc  
```

```bash
kubectl apply -k config/llmisvcconfig  
```

--------------------------------

### Install LLMInferenceService via Kustomize

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install the LLMInferenceService controller using Kustomize mode.

```bash
# LLMIsvc  
./hack/kserve-install.sh --kustomize --type llmisvc  
```

--------------------------------

### Install Infrastructure Dependencies via Quick Install Script

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install specific infrastructure dependencies for Standard, Knative, or LLMInferenceService modes.

```bash
curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-standard-mode-dependency-install.sh | bash  
```

```bash
curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-knative-mode-dependency-install.sh | bash  
```

```bash
curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/llmisvc-dependency-install.sh | bash  
```

--------------------------------

### Install the xgbserver runtime

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/xgboost

Installs the xgbserver runtime dependencies using Uv.

```bash
cd python/xgbserver  
uv sync  
```

--------------------------------

### Install All Components: Controllers and Resources

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install LLMInferenceService dependencies, CRDs, controllers, and cluster resources.

```bash
# Install LLMInferenceService Dependencies  
curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/llmisvc-dependency-install.sh | bash  
  
  
# Install CRDs  
kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-crds.yaml  
# Install Controllers  
kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.20.0/kserve.yaml  
  
# Install ClusterServingRuntimes/LLMIsvcConfigs  
kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-cluster-resources.yaml  
```

--------------------------------

### Install kserve-storage via Poetry

Source: https://kserve.github.io/website/docs/reference/python-runtime-sdk

Installation command using the uv package manager.

```bash
uv add kserve-storage
```

--------------------------------

### Install All Components via Kustomize

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install all KServe components including LLMInferenceService and LocalModel using Kustomize.

```bash
# All components (Standard mode)  
./hack/kserve-install.sh --kustomize --standard --type kserve,llmisvc,localmodel  
```

```bash
# All components (Knative mode)  
./hack/kserve-install.sh --kustomize --knative --type kserve,llmisvc,localmodel  
```

--------------------------------

### Install lgbserver Runtime

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/lightgbm

Installs the lgbserver runtime dependencies using Uv.

```bash
cd python/lgbserver  
uv sync  
```

--------------------------------

### Install GatewayClass

Source: https://kserve.github.io/website/docs/admin-guide/kubernetes-deployment-llmisvc

Script to install the GatewayClass resource.

```bash
infra/gateway-api/manage.gateway-api-gwclass.sh  
```

--------------------------------

### Install All Components: Dependencies

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install dependencies for KServe in either Standard or Knative mode.

```bash
# Option A: Standard Mode Dependencies  
curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-standard-mode-dependency-install.sh | bash  
  
# OR  
  
# Option B: Knative Mode Dependencies  
curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-knative-mode-dependency-install.sh | bash  
```

--------------------------------

### Install kserve-storage via pip

Source: https://kserve.github.io/website/docs/reference/python-runtime-sdk

Standard installation command for the kserve-storage package.

```bash
pip install kserve-storage
```

--------------------------------

### Install sklearnserver Runtime

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/sklearn

Installs the sklearnserver runtime dependencies using Uv.

```bash
cd python/sklearnserver  
uv sync  
```

--------------------------------

### Install KServe with LocalModel via Kustomize

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install KServe and LocalModel support using Kustomize mode.

```bash
# Standard mode  
./hack/kserve-install.sh --kustomize --standard --type kserve,localmodel  
  
# Knative mode (serverless)  
./hack/kserve-install.sh --kustomize --knative --type kserve,localmodel  
```

--------------------------------

### Start Local Paddle Server

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/paddle

Start the local server by specifying the model directory and model name.

```bash
python3 paddleserver --model_dir /path/to/model_dir --model_name paddle-resnet50
```

--------------------------------

### Install Standard Mode Dependencies

Source: https://kserve.github.io/website/docs/install/dependencies

Executes the automated installation script for all required dependencies in Standard mode.

```bash
# Install dependencies for Standard mode  
 curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-standard-mode-dependency-install.sh | bash  
```

--------------------------------

### Enable tracing with defaults

Source: https://kserve.github.io/website/docs/reference/crd-api

Example showing how to enable distributed tracing using default settings.

```yaml
tracing: 
```

--------------------------------

### Install LLMInferenceServiceConfigs via Helm

Source: https://kserve.github.io/website/docs/install/llmisvc-install

Command to install pre-configured templates for common LLM frameworks.

```bash
helm install kserve-runtime-configs oci://ghcr.io/kserve/charts/kserve-runtime-configs \
  --version v0.20.0 \
  --namespace kserve \
  --set kserve.llmisvcConfigs.enabled=true  
```

--------------------------------

### Install All Components via Helm

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install all KServe components including LLMInferenceService and LocalModel using Helm.

```bash
# All components (Standard mode)  
./hack/kserve-install.sh --kserve-version v0.20.0 --type kserve,llmisvc,localmodel --standard  
  
# All components (Knative mode)  
./hack/kserve-install.sh --kserve-version v0.20.0 --type kserve,llmisvc,localmodel --knative  
```

--------------------------------

### Install LLMIsvc Components Individually

Source: https://kserve.github.io/website/docs/install/dependencies

Installs specific infrastructure components for LLMIsvc mode manually using individual setup scripts.

```bash
# External load balancer (platform-specific)  
./hack/setup/infra/external-lb/manage.external-lb.sh  
  
# cert-manager  
./hack/setup/infra/manage.cert-manager-helm.sh  
  
# Gateway API Extension CRDs  
./hack/setup/infra/gateway-api/manage.gateway-api-extension-crd.sh  
  
# Gateway API CRDs  
./hack/setup/infra/gateway-api/manage.gateway-api-crd.sh  
  
# Envoy Gateway  
./hack/setup/infra/manage.envoy-gateway-helm.sh  
  
# Envoy AI Gateway  
./hack/setup/infra/manage.envoy-ai-gateway-helm.sh  
  
# Gateway API GatewayClass  
./hack/setup/infra/gateway-api/manage.gateway-api-gwclass.sh  
  
# Gateway API Gateway  
./hack/setup/infra/gateway-api/manage.gateway-api-gw.sh  
  
# LWS Operator  
./hack/setup/infra/manage.lws-operator.sh  
```

--------------------------------

### Disable storage initializer

Source: https://kserve.github.io/website/docs/reference/crd-api

Example configuration to disable the storage-initializer initContainer.

```yaml
storageInitializer: enabled: false
```

--------------------------------

### Explicitly enable storage initializer

Source: https://kserve.github.io/website/docs/reference/crd-api

Example configuration to explicitly enable the storage-initializer, which is the default behavior.

```yaml
storageInitializer: enabled: true
```

--------------------------------

### Custom tracing configuration

Source: https://kserve.github.io/website/docs/reference/crd-api

Example showing how to provide custom configuration for the distributed tracing exporter and sampler.

```yaml
tracing: exporterEndpoint: "http://my-collector:317" sampler: "parentbased_traceidratio" samplerArg: "0.1" exporter: "otlp"
```

--------------------------------

### InferenceGraph Switch configuration

Source: https://kserve.github.io/website/docs/reference/crd-api

Example of a Switch router type that routes requests to specific services based on input conditions.

```yaml
kind: InferenceGraph  
metadata:  
  
	name: abtest  
  
spec:  
  
	nodes:  
	  mymodel:  
	    routerType: Switch  
	    routes:  
	    - service: mymodel1  
	      condition: "{ .input.userId == 1 }"  
	    - service: mymodel2  
	      condition: "{ .input.userId == 2 }"
```

--------------------------------

### InferenceGraph complex classification flow

Source: https://kserve.github.io/website/docs/reference/crd-api

Example of a complex flow using both Sequence and Switch routers for multi-stage classification.

```yaml
kind: InferenceGraph  
metadata:  
  
	name: dog-breed-classification  
  
spec:  
  
	nodes:  
	  root:  
	    routerType: Sequence  
	    routes:  
	    - service: cat-dog-classifier  
	    - nodeName: breed-classifier  
	      data: $request  
	  breed-classifier:  
	    routerType: Switch  
	    routes:  
	    - service: dog-breed-classifier  
	      condition: { .predictions.class == "dog" }  
	    - service: cat-breed-classifier  
	      condition: { .predictions.class == "cat" }
```

--------------------------------

### InferenceGraph Sequence configuration

Source: https://kserve.github.io/website/docs/reference/crd-api

Example of a Sequence router type that chains multiple models where the output of one serves as input to the next.

```yaml
kind: InferenceGraph  
metadata:  
  
	name: model-chainer  
  
spec:  
  
	nodes:  
	  root:  
	    routerType: Sequence  
	    routes:  
	    - service: mymodel-s1  
	    - service: mymodel-s2  
	      data: $response  
	    - service: mymodel-s3  
	      data: $response
```

--------------------------------

### Quick Install KServe in Knative Mode via Scripts

Source: https://kserve.github.io/website/docs/install/kserve-install

Use the setup script or the manifest-based curl command to install KServe in Knative mode.

```bash
cd kserve  
  
# Install dependencies + KServe  
./hack/setup/quick-install/kserve-knative-mode-full-install-helm.sh  
  
# Or use with-manifest version (no clone needed, includes embedded manifests)  
curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-knative-mode-full-install-with-manifests.sh  
| bash  
```

--------------------------------

### Launch Jupyter notebook

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/onnx

Command to start the Jupyter server and open the provided ONNX example notebook.

```bash
jupyter notebook mosaic-onnx.ipynb  
```

--------------------------------

### Run KServe quick install script

Source: https://kserve.github.io/website/docs/developer-guide

Clone the repository and execute the installation script with various flags for different deployment modes.

```bash
# Clone the repository if you haven't already  
git clone https://github.com/kserve/kserve.git  
cd kserve  
  
# Run the quick install script with Standard deployment mode (without Knative, using Gateway API + Istio)  
./hack/kserve-install.sh -r --kustomize  
  
# Run the quick install script with Knative mode (installs Knative + Istio)  
./hack/kserve-install.sh -s --kustomize  
  
# For LLMInferenceService  
./hack/kserve-install.sh --type localmodel --kustomize  
  
# For LLMInferenceService  
./hack/kserve-install.sh --type llmisvc --kustomize  
  
# To install only Standard mode dependencies (without KServe)  
./hack/kserve-install.sh -d --standard  
  
# To install only Knative mode dependencies (without KServe)  
./hack/kserve-install.sh -d --knative  
  
# To install only LLMInferenceService dependencies (without LLMInferenceService)  
./hack/kserve-install.sh -d --type llmisvc  
  
# To include KEDA (Kubernetes Event-driven Autoscaling)  
./hack/kserve-install.sh -k  
```

--------------------------------

### C# HttpClient Request

Source: https://kserve.github.io/website/docs/reference/oip/get-v-2-health-ready

Example of using HttpClient in C# to perform a GET request to the health ready endpoint.

```csharp
var client = new HttpClient();  
var request = new HttpRequestMessage(HttpMethod.Get, "https://kserve.github.io/v2/health/ready");  
var response = await client.SendAsync(request);  
response.EnsureSuccessStatusCode();  
Console.WriteLine(await response.Content.ReadAsStringAsync());
```

--------------------------------

### Setup cloud-provider-kind for LoadBalancer

Source: https://kserve.github.io/website/docs/model-serving/generative-inference/llmisvc/llmisvc-dependencies

Commands to create a kind cluster and run cloud-provider-kind to simulate cloud LoadBalancer behavior.

```bash
# Create kind cluster  
kind create cluster -n "kserve-llm"  
  
# Install cloud-provider-kind  
go install sigs.k8s.io/cloud-provider-kind@latest  
  
# Run cloud-provider-kind in background  
cloud-provider-kind > /dev/null 2>&1 &  
```

--------------------------------

### C# HttpClient Model Ready Request

Source: https://kserve.github.io/website/docs/reference/oip/get-v-2-models-model-name-versions-model-version-ready

Example of using HttpClient in C# to perform a GET request to the model ready endpoint.

```csharp
var client = new HttpClient();  
var request = new HttpRequestMessage(HttpMethod.Get, "https://kserve.github.io");  
var response = await client.SendAsync(request);  
response.EnsureSuccessStatusCode();  
Console.WriteLine(await response.Content.ReadAsStringAsync());
```

--------------------------------

### Start minikube

Source: https://kserve.github.io/website/docs/developer-guide

Initialize a minikube cluster with specific resource allocations.

```bash
minikube start --driver=docker --memory=8192 --cpus=4  
```

--------------------------------

### Install KServe SDK via uv

Source: https://kserve.github.io/website/docs/reference/controlplane-client/controlplane-client-sdk

Install the SDK using uv by navigating to the kserve python directory and syncing the environment.

```bash
cd kserve/python/kserve
uv sync
```

--------------------------------

### Install KServe dependencies

Source: https://kserve.github.io/website/docs/developer-guide

Installs KServe dependencies using the quick install script with the -d flag.

```bash
./hack/kserve-install.sh -d --type kserve  
```

--------------------------------

### Install LLMInferenceService Addon via Helm

Source: https://kserve.github.io/website/docs/install/llmisvc-install

Command to install LLMInferenceService resources as an addon when KServe is already installed.

```bash
helm install kserve-llmisvc-resources oci://ghcr.io/kserve/charts/kserve-llmisvc-resources \
  --version v0.20.0 \
  --create-namespace \
  --namespace kserve \
  --set kserve.createSharedResources=false  
```

--------------------------------

### Install KServe Addon Resources with Helm

Source: https://kserve.github.io/website/docs/install/kserve-install

Install KServe resources when LLMIsvc is already installed by disabling shared resources.

```bash
helm install kserve-resources oci://ghcr.io/kserve/charts/kserve-resources \
  --version v0.20.0 \
  --namespace kserve \
  --set kserve.createSharedResources=false \
  --wait  
```

--------------------------------

### Setup grpcurl Environment

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/triton/torchscript

Downloads the required proto file and sets environment variables for the gRPC request.

```bash
# Download the proto file  
curl -O https://raw.githubusercontent.com/kserve/kserve/master/docs/predict-api/v2/grpc_predict_v2.proto  
  
INPUT_PATH=input-grpc.json  
PROTO_FILE=grpc_predict_v2.proto  
SERVICE_HOSTNAME=$(kubectl get inferenceservice torchscript-cifar10 -o jsonpath='{.status.url}' | cut -d "/" -f 3)  
```

--------------------------------

### Install dependencies for Knative mode via script

Source: https://kserve.github.io/website/docs/install/dependencies

Use the pre-generated dependency installation script to install all required components for Knative mode.

```bash
# Install dependencies for Knative mode  
curl -fsSL https://github.com/kserve/kserve/releases/download/v0.20.0/kserve-knative-mode-dependency-install.sh | bash  
```

--------------------------------

### Sample pod output

Source: https://kserve.github.io/website/docs/model-serving/storage/providers/oci

Example output showing the pod status after deployment.

```text
NAME                                                     READY   STATUS    RESTARTS      AGE  
sklearn-iris-oci-predictor-00001-deployment-58fc6564d7   3/3     Running   1 (39m ago)   40m
```

--------------------------------

### Verify git installation

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Run this command to verify that git is installed.

```bash
git --version  
```

--------------------------------

### Execute the LangChain example script

Source: https://kserve.github.io/website/docs/model-serving/generative-inference/sdk-integration

Run the created Python script to verify the integration and view the model responses.

```bash
python3 sample_langchain.py
```

--------------------------------

### Verify helm installation

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Run this command to verify that helm is installed.

```bash
helm version  
```

--------------------------------

### Explain API Request with instances

Source: https://kserve.github.io/website/docs/concepts/architecture/data-plane/v1-protocol

Example curl command for the Explain API using the 'instances' key.

```bash
curl -X POST http://localhost:8000/v1/models/mymodel:explain \
  -d '{"instances": [[1.0, 2.0, 5.0], [5.0, 6.0, 5.0]]}'
```

--------------------------------

### Setup Environment for gRPC Testing

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/spark-mllib

Downloads the required proto file and sets environment variables for the service hostname.

```bash
# Download the proto file  
curl -O https://raw.githubusercontent.com/kserve/open-inference-protocol/main/specification/protocol/open_inference_grpc.proto  
  
INPUT_PATH=iris-input-grpc.json  
PROTO_FILE=open_inference_grpc.proto  
SERVICE_HOSTNAME=$(kubectl get inferenceservice spark-iris-grpc -o jsonpath='{.status.url}' | cut -d "/" -f 3)
```

--------------------------------

### Configure startupProbe in ServingRuntime

Source: https://kserve.github.io/website/docs/model-serving/generative-inference/multi-node

Adjust the startupProbe settings in your ServingRuntime to accommodate large model loading times.

```yaml
..  
      startupProbe:  
        failureThreshold: 40  
        periodSeconds: 30  
        successThreshold: 1  
        timeoutSeconds: 30  
        initialDelaySeconds: 60  
..
```

--------------------------------

### Manage kind cluster with script

Source: https://kserve.github.io/website/docs/developer-guide

Use the provided management script to set up a kind cluster with a local registry.

```bash
hack/setup/dev/manage.kind-with-registry.sh  
```

--------------------------------

### Install Kafka Event Source

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/kafka

Command to install the Kafka Event Source for Knative.

```bash
kubectl apply -f https://github.com/knative-sandbox/eventing-kafka/releases/download/knative-v1.9.1/source.yaml  
```

--------------------------------

### Sample in-container session output

Source: https://kserve.github.io/website/docs/model-serving/storage/providers/oci

Example output from inspecting the /mnt directory inside the runtime container.

```text
sklearn-iris-oci-predictor:/$ cd /mnt  
sklearn-iris-oci-predictor:/mnt$ ls -l  
total 0  
lrwxrwxrwx 1 1010 root 20 Jan 27 10:35 models -> /proc/38/root/models  
  
sklearn-iris-oci-predictor:/mnt$ cd /mnt/models   
sklearn-iris-oci-predictor:/mnt/models$ ls -l  
total 8  
-rw-r--r-- 1 root root 5408 Jan 26 15:58 model.joblib
```

--------------------------------

### Install Python dependencies

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/onnx

Install the necessary packages for image processing and API requests.

```bash
pip install numpy pillow protobuf requests  
```

--------------------------------

### Install KServe Controller via Kustomize

Source: https://kserve.github.io/website/docs/getting-started/quickstart-guide

Install KServe using Kustomize mode with standard or Knative configurations.

```bash
# Standard mode  
./hack/kserve-install.sh --kustomize --standard  
  
# Knative mode (serverless)  
./hack/kserve-install.sh --kustomize --knative  
```

--------------------------------

### Prepare Environment for grpcurl

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/paddle

Downloads the required proto file and sets environment variables for testing.

```bash
# Download the proto file  
curl -O https://raw.githubusercontent.com/kserve/open-inference-protocol/main/specification/protocol/open_inference_grpc.proto  
  
INPUT_PATH=jay-v2-grpc.json  
PROTO_FILE=open_inference_grpc.proto  
SERVICE_HOSTNAME=$(kubectl get inferenceservice paddle-v2-resnet50-grpc -o jsonpath='{.status.url}' | cut -d "/" -f 3)  
```

--------------------------------

### Start LightGBM Server Locally

Source: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/lightgbm

Starts the local model server using the lgbserver package with specified model directory and name.

```bash
python3 lgbserver --model_dir /path/to/model_dir --model_name lightgbm-iris  
```

--------------------------------

### CLI Tool Installation Commands

Source: https://kserve.github.io/website/docs/install/overview

Commands to install Helm, optionally specifying a version.

```bash
# Install Helm  
./hack/setup/cli/install-helm.sh  
  
# Install with specific version  
HELM_VERSION=v3.13.0 ./hack/setup/cli/install-helm.sh
```
Tokens:
Raw