Seldon Core

https://docs.seldon.ai/seldon-core-2
Admin
Seldon Core is an open-source platform for deploying machine learning models on Kubernetes, offering...
Tokens:196,854
Snippets:1,359
Trust Score:4.8
Update:11 months ago
Show doc for...
Tokens:
Raw
### Install Tritonclient

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/tritonclient-examples

Installs the Tritonclient library with all optional dependencies. This is a prerequisite for running the following examples.

```bash
pip install tritonclient[all]
```

--------------------------------

### Start A/B Experiment

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/experiment-versions

Starts the defined A/B experiment using the provided YAML configuration file. This command initiates the traffic splitting between the specified candidate models.

```bash
seldon experiment start -f ./experiments/ab-default-model.yaml
```

--------------------------------

### Install Seldon Core v2 Servers using Helm

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment

Installs example Seldon Core v2 servers, such as 'mlserver-0' and 'triton-0', in the 'seldon-mesh' namespace. These servers are used to host machine learning models for inference.

```bash
helm upgrade seldon-core-v2-servers seldon-charts/seldon-core-v2-servers \
--namespace seldon-mesh \
--install
```

--------------------------------

### Start Seldon Core Experiment (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-experiments

This command starts a Seldon Core experiment defined in a YAML file. It initiates the traffic routing and A/B testing setup.

```bash
seldon experiment start -f ./experiments/ab-default-model.yaml

```

--------------------------------

### Install MLServer CLI

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/batch-examples-local

Installs the MLServer command-line interface using pip. This tool is used for managing and interacting with ML models, including running batch inference jobs.

```bash
pip install mlserver
```

--------------------------------

### Starting and Managing Seldon Experiments

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-experiments

This section illustrates starting and monitoring a Seldon experiment. It includes defining an experiment configuration, starting the experiment, and checking its status. The status output provides details on the experiment's active state and readiness.

```bash
cat ./experiments/add1020.yaml
```

```yaml
apiVersion: mlops.seldon.io/v1alpha1
kind: Experiment
metadata:
  name: add1020
spec:
  default: add10
  candidates:
  - name: add10
    weight: 50
  - name: add20
    weight: 50

```

```bash
seldon experiment start -f ./experiments/add1020.yaml
```

```bash
seldon experiment status add1020 -w | jq -M .
```

```json
{
  "experimentName": "add1020",
  "active": true,
  "candidatesReady": true,
  "mirrorReady": true,
  "statusDescription": "experiment active",
  "kubernetesMeta": {}
}

```

--------------------------------

### Start and Monitor Experiment Status (Bash/JSON)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-examples

These commands initiate a Seldon Core experiment using a defined YAML file and then check its status. The 'experiment start' command begins the A/B test, and 'experiment status -w' monitors it, providing JSON output indicating if the experiment is active and ready.

```bash
seldon experiment start -f ./experiments/ab-default-model.yaml

```

```bash
seldon experiment status experiment-sample -w | jq -M .

```

--------------------------------

### Install Seldon Core 2 using Helm

Source: https://docs.seldon.ai/seldon-core-2/installation/learning-environment

This snippet outlines the steps to install Seldon Core 2 using Helm. It covers creating namespaces, adding Helm repositories, installing CRDs, the operator, runtimes, and servers. It also includes commands to check the installation status.

```bash
kubectl create ns seldon-mesh || echo "Namespace seldon-mesh already exists"
```

```bash
helm repo add seldon-charts https://seldonio.github.io/helm-charts/
helm repo update seldon-charts
```

```bash
helm upgrade seldon-core-v2-crds seldon-charts/seldon-core-v2-crds \
--namespace default \
--install
```

```bash
helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
--namespace seldon-mesh --set controller.clusterwide=true \
--install
```

```bash
helm upgrade seldon-core-v2-runtime seldon-charts/seldon-core-v2-runtime \
--namespace seldon-mesh \
--install
```

```bash
helm upgrade seldon-core-v2-servers seldon-charts/seldon-core-v2-servers \
--namespace seldon-mesh \
--install
```

```bash
kubectl get pods -n seldon-mesh
```

--------------------------------

### Start Mirror Experiment using Seldon CLI (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-experiments

Starts the mirror experiment defined in './experiments/addmul10-mirror.yaml' using the `seldon experiment start` command. This command initiates the traffic mirroring process.

```bash
seldon experiment start -f ./experiments/addmul10-mirror.yaml

```

--------------------------------

### Start Modified Default Model Experiment - Bash

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/experiment-versions

Starts a new Seldon Core experiment with the updated configuration. This command applies the new traffic splitting rules, redirecting traffic according to the modified default and candidate models.

```bash
seldon experiment start -f ./experiments/ab-default-model3.yaml
```

--------------------------------

### Seldon CLI Basic Usage

Source: https://docs.seldon.ai/seldon-core-2/resources/apis/cli

Demonstrates the basic command to display the help information for the Seldon CLI.

```sh
seldon --help
```

--------------------------------

### Install Seldon CLI (Local Build)

Source: https://docs.seldon.ai/seldon-core-2/resources/apis/cli

Instructions for building the Seldon CLI locally from source. This method requires Go version 1.21.1 or higher. It involves cloning the Seldon Core repository, navigating to the operator directory, and running the build command.

```bash
git clone https://github.com/SeldonIO/seldon-core --branch=v2
cd seldon-core/operator
make build-seldon
```

--------------------------------

### Get Income Explainer Model Status (Bash & JSON)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/k8s-examples

This command retrieves the status of the 'income-explainer' model in JSON format and pipes it to `jq` for pretty-printing. It shows the model's readiness conditions and replica count.

```bash
kubectl get model income-explainer -n ${NAMESPACE} -o jsonpath='{.status}' | jq -M .
```

--------------------------------

### Install Istio Ingress Gateway with Helm

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment/istio

Deploys the Istio Ingress Gateway, which acts as the entry point for external traffic into the service mesh. This command installs the gateway component within the 'istio-system' namespace.

```bash
helm install istio-ingressgateway istio/gateway -n istio-system
```

--------------------------------

### Install Istio Base Component with Helm

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment/istio

Installs the base Istio component using Helm, which includes essential resources for Istio's operation. This command targets the 'istio-system' namespace.

```bash
helm install istio-base istio/base -n istio-system
```

--------------------------------

### Install Seldon Core v2 Runtimes using Helm

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment

Deploys the Seldon Core v2 runtimes within the 'seldon-mesh' namespace. Runtimes are essential components for executing Seldon Core inference services.

```bash
helm upgrade seldon-core-v2-runtime seldon-charts/seldon-core-v2-runtime \
--namespace seldon-mesh \
--install
```

--------------------------------

### Deploy Tensorflow Model

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-examples

Instructions on how to deploy a TensorFlow model using kubectl or the seldon-cli.

```APIDOC
## Deploy Tensorflow Model

This section covers deploying a TensorFlow model configuration.

### Method

- `kubectl apply -f <model-config-file>`
- `seldon model load -f <model-config-file>`

### Endpoint

N/A (Local file deployment)

### Parameters

#### Request Body (model config file)

- **apiVersion** (string) - Required - Specifies the API version for the model configuration.
- **kind** (string) - Required - Set to 'Model' for model deployments.
- **metadata.name** (string) - Required - The name of the model.
- **spec.storageUri** (string) - Required - The URI where the model artifacts are stored.
- **spec.requirements** (array of strings) - Optional - A list of package requirements for the model.
- **spec.memory** (string) - Optional - Memory allocation for the model.

### Request Example

```yaml
apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
  name: tfsimple1
spec:
  storageUri: "gs://seldon-models/triton/simple"
  requirements:
  - tensorflow
  memory: 100Ki
```

### Response

#### Success Response (Deployment)

- `model.mlops.seldon.io/<model-name> created` (kubectl)
- `{}` (seldon-cli, indicates successful loading)

#### Response Example

```json
// kubectl output example
model.mlops.seldon.io/tfsimple1 created

// seldon-cli output example
{}
```
```

--------------------------------

### Install Go License Tools

Source: https://docs.seldon.ai/seldon-core-2/resources/development/licenses

Navigates to the go-license-tools directory within the cloned repository and installs the Python package. This makes the license generation commands available.

```shell
cd kubeflow-testing/py/kubeflow/testing/go-license-tools/
python setup.py install
```

--------------------------------

### Install Seldon CLI (Linux)

Source: https://docs.seldon.ai/seldon-core-2/resources/apis/cli

Instructions for installing the Seldon CLI on Linux using prebuilt binaries. This method requires a Unix-like architecture and glibc 2.25+. The process involves downloading the binary, moving it to a folder, and making it executable.

```bash
mv seldon-linux-amd64 seldon
chmod u+x seldon
```

--------------------------------

### Start, Monitor, and Stop Seldon Experiment

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-experiments

Commands to manage a Seldon Core experiment. This includes starting the experiment, checking its status, and stopping it.

```bash
seldon experiment start -f ./experiments/sklearn-mirror.yaml

```

```bash
seldon experiment status sklearn-mirror -w | jq -M .

```

```bash
seldon experiment stop sklearn-mirror

```

--------------------------------

### Start Experiment in Seldon Core (CLI)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-experiments

Initiates the 'addmul10' experiment using the 'seldon experiment start' command with the specified YAML configuration file. This command triggers the experiment execution.

```bash
seldon experiment start -f ./experiments/addmul10.yaml

```

--------------------------------

### Install Istio Control Plane (Istiod) with Helm

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment/istio

Installs Istiod, the Istio control plane, using Helm. The '--wait' flag ensures that the command waits for the installation to complete before returning. This is crucial for subsequent steps that depend on the control plane being ready.

```bash
helm install istiod istio/istiod -n istio-system --wait
```

--------------------------------

### List Model Artifacts (Google Cloud Storage)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

Lists all files within a specified Google Cloud Storage bucket, useful for inspecting model artifacts and configuration files. This command helps in understanding the available resources before deployment.

```bash
!gcloud storage ls --recursive gs://seldon-models/scv2/samples/mlserver_1.4.0/income-sklearn/classifier
```

--------------------------------

### Setup Seldon Core Ecosystem with Ansible

Source: https://docs.seldon.ai/seldon-core-2/installation/learning-environment

Installs necessary ecosystem dependencies such as Kafka, Prometheus, OpenTelemetry, and Jaeger. It can be configured to install within a specified namespace.

```bash
ansible-playbook playbooks/setup-ecosystem.yaml
```

```bash
ansible-playbook playbooks/setup-ecosystem.yaml -e seldon_mesh_namespace=<mynamespace>
```

--------------------------------

### Deploy Seldon Iris Model with Traefik Ingress

Source: https://docs.seldon.ai/seldon-core-2/integrations/service-meshes/traefik

This configuration deploys a Seldon Iris model, sets up a Traefik Service, and defines an IngressRoute for accessing the model. It also includes a Traefik Middleware to add a custom 'seldon-model: iris' header to requests. This setup assumes a Kubernetes environment with Traefik installed.

```yaml
apiVersion: v1
kind: Service
metadata:
  name: myapps
  namespace: seldon-mesh
spec:
  ports:
  - name: web
    port: 80
    protocol: TCP
  selector:
    app: traefik-ingress-lb
  type: LoadBalancer
---
apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
  name: iris
  namespace: seldon-mesh
spec:
  requirements:
  - sklearn
  storageUri: gs://seldon-models/mlserver/iris
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: iris
  namespace: seldon-mesh
spec:
  entryPoints:
  - web
  routes:
  - kind: Rule
    match: PathPrefix(`/`)
    middlewares:
    - name: iris-header
    services:
    - name: seldon-mesh
      port: 80
      scheme: h2c
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: iris-header
  namespace: seldon-mesh
spec:
  headers:
    customRequestHeaders:
      seldon-model: iris

```

--------------------------------

### Install Prometheus Adapter using Helm

Source: https://docs.seldon.ai/seldon-core-2/user-guide/scaling/hpa-overview/hpa-setup

Installs the Prometheus Adapter using Helm, setting the Prometheus URL and namespace. This command assumes Prometheus is running in the same namespace. If Prometheus is in a different namespace or uses a custom port, the --set parameters need to be adjusted accordingly.

```bash
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install --set prometheus.url='http://seldon-monitoring-prometheus' hpa-metrics prometheus-community/prometheus-adapter -n seldon-monitoring
```

--------------------------------

### View Model Settings (model-settings.json)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

Displays the content of the model-settings.json file, which contains crucial information for loading and configuring the MLServer model. It specifies the model name, implementation, and parameters like the model URI and version.

```bash
!gsutil cat gs://seldon-models/scv2/samples/mlserver_1.4.0/income-sklearn/classifier/model-settings.json
```

--------------------------------

### Install Seldon Core v2 Setup with Kafka Configuration

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/k8s-clusterwide

This Helm command installs the Seldon Core v2 setup, enabling cluster-wide controller functionality and configuring Kafka topic prefixes and consumer group prefixes for isolation. This is useful when sharing a Kafka cluster with other applications.

```bash
helm upgrade --install seldon-v2 ../k8s/helm-charts/seldon-core-v2-setup/ -n seldon-mesh \
    --set controller.clusterwide=true \
    --set kafka.topicPrefix=myorg \
    --set kafka.consumerGroupIdPrefix=myorg
```

--------------------------------

### Applying Seldon Core Model Deployment to Kubernetes

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

This command applies the `preprocessor.yaml` configuration file to the Kubernetes cluster using `kubectl`. This action creates the Seldon Core Model resource, making the preprocessing step available for use within the Seldon deployment.

```bash
!kubectl apply -f ../../../samples/quickstart/models/preprocessor/preprocessor.yaml -n seldon-mesh
```

--------------------------------

### Wait for Model Ready

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-examples

Commands to verify that the deployed model is ready for inference.

```APIDOC
## Wait for Model Ready

Verify that the model has been successfully deployed and is ready to serve requests.

### Method

- `kubectl wait --for condition=ready --timeout=<timeout> model <model-name>`
- `seldon model status <model-name> -w ModelAvailable`

### Endpoint

N/A (Resource status check)

### Parameters

#### Path Parameters

- **model-name** (string) - Required - The name of the model to check.

#### Query Parameters

- **-w ModelAvailable** (flag) - Required (for seldon-cli) - Specifies to wait until the model is available.
- **--timeout** (string) - Optional (for kubectl) - Maximum time to wait for the condition.

### Response

#### Success Response (Model Ready)

- `model.mlops.seldon.io/<model-name> condition met` (kubectl)
- JSON output indicating model availability (seldon-cli, typically `{}` if available)

#### Response Example

```bash
// kubectl output example
model.mlops.seldon.io/tfsimple1 condition met

// seldon-cli output example
{}
```
```

--------------------------------

### Customize Seldon Core 2 Installation with Ansible Variables

Source: https://docs.seldon.ai/seldon-core-2/installation/learning-environment

Allows for customization of the Seldon Core 2 installation using Ansible extra variables. This example demonstrates setting a custom namespace, disabling Prometheus, and loading variables from a file.

```bash
ansible-playbook playbooks/seldon-all.yaml -e seldon_mesh_namespace=my-seldon-mesh -e install_prometheus=no -e @playbooks/vars/set-custom-images.yaml
```

--------------------------------

### Install Seldon Core v2 Certificates using Helm

Source: https://docs.seldon.ai/seldon-core-2/resources/security

Installs the Seldon Core v2 certificates Helm chart into the specified namespace. This chart provides example certificate issuers and certificates, often used with cert-manager.

```bash
helm install seldon-v2-certs k8s/helm-charts/seldon-core-v2-certs/ -n seldon-mesh
```

--------------------------------

### Listing Cloud Storage Contents for Model Artifacts

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

This command lists the contents of a specified Google Cloud Storage bucket, which are necessary model artifacts for deployment. It shows the `model-settings.json`, `model.py`, and `preprocessor.yaml` files required by Seldon Core.

```bash
!gcloud storage ls --recursive gs://seldon-models/scv2/samples/preprocessor
```

--------------------------------

### Deploy LightGBM Model with Kubectl

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/model-zoo

Applies the LightGBM model configuration using kubectl and waits for the model to be ready. This involves creating the model resource and ensuring it is in a deployable state before proceeding to inference.

```bash
kubectl apply -f ./models/income-lgb.yaml -n ${NAMESPACE}
kubectl wait --for condition=ready --timeout=300s model income-lgb -n ${NAMESPACE}
```

--------------------------------

### Get MLServer Status

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/k8s-examples

This command retrieves the status of the MLServer running in the Kubernetes cluster using `kubectl get server`. The output, formatted with `jq`, provides details about the server's readiness, replica status, and loaded models.

```bash
kubectl get server mlserver -n ${NAMESPACE} -o jsonpath='{.status}' | jq -M .
```

--------------------------------

### Get Seldon Model Status

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/k8s-examples

This snippet retrieves the status of the Seldon Model named 'iris' using `kubectl get` and formats the output using `jq`. The status provides information about the model's readiness and replica count.

```bash
kubectl get model iris -n ${NAMESPACE} -o jsonpath='{.status}' | jq -M .
```

--------------------------------

### Check Seldon Core v2 Installation Status

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment

Verifies the installation of Seldon Core v2 components by listing all pods in the 'seldon-mesh' namespace. This command helps in confirming that the operator, runtimes, and servers are running as expected.

```bash
kubectl get pods -n seldon-mesh
```

--------------------------------

### View Pipeline Configuration (YAML)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

Displays the YAML configuration for the Seldon Pipeline, defining its steps and input/output mappings. This YAML is crucial for understanding the pipeline's structure and how models are connected.

```yaml
apiVersion: mlops.seldon.io/v1alpha1
kind: Pipeline
metadata:
  name: income-classifier-app
spec:
  steps:
    - name: preprocessor
    - name: income-classifier
      inputs:
      - preprocessor
  output:
    steps:
    - income-classifier
```

--------------------------------

### Apply Pipeline Configuration (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

Applies the Seldon Pipeline configuration to the Kubernetes cluster using kubectl. This command deploys the defined pipeline, making it available for inference.

```bash
!kubectl apply -f ../../../samples/quickstart/pipelines/income-classifier-app.yaml -n seldon-mesh
```

--------------------------------

### Install Seldon Core v2 Operator (Cluster-Wide)

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment

Installs the Seldon Core v2 operator in a cluster-wide mode within the 'seldon-mesh' namespace. This allows the operator to manage Seldon resources across the entire Kubernetes cluster.

```bash
helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
--namespace seldon-mesh --set controller.clusterwide=true \
--install
```

--------------------------------

### Install Seldon Core v2 Operator (Namespace Specific)

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment

Installs the Seldon Core v2 operator in a specific namespace ('op-ns1') and configures it to watch two other namespaces ('ns1', 'ns2'). This allows for fine-grained control over resource management.

```bash
helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
--namespace op-ns1 \
--set controller.clusterwide=true \
--set "controller.watchNamespaces={ns1,ns2}" \
--install
```

--------------------------------

### Install Seldon Core v2 CRDS using Helm

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment

Installs the custom resource definitions (CRDS) for Seldon Core v2 using Helm. This step is essential for Seldon Core to manage its resources within a Kubernetes cluster.

```bash
helm upgrade seldon-core-v2-crds seldon-charts/seldon-core-v2-crds \
--namespace default \
--install
```

--------------------------------

### Install Second Seldon Core v2 Operator (Skipping Cluster Role)

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment

Installs a second Seldon Core v2 operator in 'op-ns2', configured to watch 'ns3' and 'ns4'. It uses 'controller.skipClusterRoleCreation=true' to avoid duplicating cluster roles created by the first operator.

```bash
helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \
--namespace op-ns2 \
--set controller.clusterwide=true \
--set "controller.watchNamespaces={ns3,ns4}" \
--set controller.skipClusterRoleCreation=true \
--install
```

--------------------------------

### LightGBM Model Deployment and Inference

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/model-zoo

This section details how to deploy the LightGBM model, perform inference, and clean up resources using kubectl.

```APIDOC
## POST /v2/models/{model_name}/infer

### Description
Performs inference using the deployed LightGBM model.

### Method
POST

### Endpoint
`/v2/models/income-lgb/infer`

### Parameters
#### Query Parameters
None

#### Request Body
- **parameters** (object) - Required - Contains inference parameters, such as content type.
  - **content_type** (string) - Required - Specifies the content type of the input data (e.g., "pd" for pandas DataFrame).
- **inputs** (array) - Required - An array of input tensors for the model.
  - **name** (string) - Required - The name of the input tensor.
  - **shape** (array) - Required - The shape of the input tensor.
  - **datatype** (string) - Required - The data type of the input tensor (e.g., "INT64").
  - **data** (array) - Required - The actual input data.

### Request Example
```json
{
  "parameters": {"content_type": "pd"},
  "inputs": [
    {"name": "Age", "shape": [1, 1], "datatype": "INT64", "data": [47]},
    {"name": "Workclass", "shape": [1, 1], "datatype": "INT64", "data": [4]},
    {"name": "Education", "shape": [1, 1], "datatype": "INT64", "data": [1]},
    {"name": "Marital Status", "shape": [1, 1], "datatype": "INT64", "data": [1]},
    {"name": "Occupation", "shape": [1, 1], "datatype": "INT64", "data": [1]},
    {"name": "Relationship", "shape": [1, 1], "datatype": "INT64", "data": [3]},
    {"name": "Race", "shape": [1, 1], "datatype": "INT64", "data": [4]},
    {"name": "Sex", "shape": [1, 1], "datatype": "INT64", "data": [1]},
    {"name": "Capital Gain", "shape": [1, 1], "datatype": "INT64", "data": [0]},
    {"name": "Capital Loss", "shape": [1, 1], "datatype": "INT64", "data": [0]},
    {"name": "Hours per week", "shape": [1, 1], "datatype": "INT64", "data": [40]},
    {"name": "Country", "shape": [1, 1], "datatype": "INT64", "data": [9]}
  ]
}
```

### Response
#### Success Response (200)
- **model_name** (string) - The name of the model.
- **model_version** (string) - The version of the model.
- **id** (string) - A unique identifier for the inference request.
- **parameters** (object) - Inference parameters returned by the model.
- **outputs** (array) - An array of output tensors from the model.
  - **name** (string) - The name of the output tensor.
  - **shape** (array) - The shape of the output tensor.
  - **datatype** (string) - The data type of the output tensor.
  - **data** (array) - The actual output data.

#### Response Example
```json
{
	"model_name": "income-lgb_1",
	"model_version": "1",
	"id": "4437a71e-9af1-4e3b-aa4b-cb95d2cd86b9",
	"parameters": {},
	"outputs": [
		{
			"name": "predict",
			"shape": [
				1,
				1
			],
			"datatype": "FP64",
			"data": [
				0.06279460120044741
			]
		}
	]
}
```
```

--------------------------------

### Install Seldon Core v2 Setup with Helm

Source: https://docs.seldon.ai/seldon-core-2/integrations/confluent/schema-registry

This command installs or upgrades the 'seldon-core-v2-setup' Helm chart in the 'seldon-mesh' namespace. It configures the Schema Registry integration by setting the security configuration path to '/mnt/schema-registry'. This ensures that Seldon Core services can access the Schema Registry credentials.

```bash
helm upgrade seldon-core-v2-setup seldon-charts/seldon-core-v2-setup \n  --namespace seldon-mesh \n  --set security.schemaRegistry.configPath=/mnt/schema-registry \n  --install
```

--------------------------------

### Seldon Core Model Deployment Configuration

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

This YAML file defines a Seldon Core Model resource. It specifies the `storageUri` where the model artifacts are located in cloud storage and lists the `requirements` needed for deployment, including dependencies for the preprocessor and the main model.

```yaml
apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
    name: preprocessor
spec:
    storageUri: "gs://seldon-models/scv2/samples/preprocessor"
    requirements:
    - income-classifier-deps
```

--------------------------------

### GET /v2/models/{model-name}

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-examples

Retrieves metadata for a specific model, including its name, versions, platform, and input/output schema.

```APIDOC
## Get Model Metadata

Retrieves the metadata associated with a deployed model.

### Method

`GET`

### Endpoint

`/v2/models/{model-name}`

### Parameters

#### Path Parameters

- **model-name** (string) - Required - The name of the model for which to retrieve metadata.

### Request Example

```bash
curl --location 'http://${SELDON_INFER_HOST}/v2/models/tfsimple1'
```

### Response

#### Success Response (200)

- **name** (string) - The name of the model.
- **versions** (array of strings) - A list of available model versions.
- **platform** (string) - The inference platform used by the model (e.g., 'tensorflow_graphdef').
- **inputs** (array of objects) - Schema definition for model inputs.
  - **name** (string) - Input name.
  - **datatype** (string) - Data type of the input.
  - **shape** (array of integers) - Shape of the input tensor.
- **outputs** (array of objects) - Schema definition for model outputs.
  - **name** (string) - Output name.
  - **datatype** (string) - Data type of the output.
  - **shape** (array of integers) - Shape of the output tensor.

#### Response Example

```json
{
	"name": "tfsimple1_1",
	"versions": [
		"1"
	],
	"platform": "tensorflow_graphdef",
	"inputs": [
		{
			"name": "INPUT0",
			"datatype": "INT32",
			"shape": [
				-1,
				16
			]
		},
		{
			"name": "INPUT1",
			"datatype": "INT32",
			"shape": [
				-1,
				16
			]
		}
	],
	"outputs": [
		{
			"name": "OUTPUT0",
			"datatype": "INT32",
			"shape": [
				-1,
				16
			]
		},
		{
			"name": "OUTPUT1",
			"datatype": "INT32",
			"shape": [
				-1,
				16
			]
		}
	]
}
```
```

--------------------------------

### Deploy SKLearn Income Model using Seldon CLI

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/explainer-examples

This snippet demonstrates deploying the SKLearn Income Model using the `seldon-cli`. It includes commands to unload a previous model (if necessary), load the income model using its YAML configuration, check its status, and perform an inference request.

```bash
seldon model unload mnist-pytorch

seldon model load -f ./models/income.yaml

seldon model status income -w ModelAvailable

seldon model infer income \
  '{"inputs": [{"name": "predict", "shape": [1, 12], "datatype": "FP32", "data": [[47,4,1,1,1,3,4,1,0,0,40,9]]}]}'
```

--------------------------------

### Define and Load a Simple Seldon Pipeline

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/pipeline-to-pipeline

Defines a basic Seldon pipeline named 'tfsimple' that uses the 'tfsimple1' model. The pipeline configuration specifies the steps involved and the expected output. The `seldon pipeline load` command then registers this pipeline.

```bash
cat ./pipelines/tfsimple.yaml

# The content of tfsimple.yaml would be:
# apiVersion: mlops.seldon.io/v1alpha1
# kind: Pipeline
# metadata:
#   name: tfsimple
# spec:
#   steps:
#     - name: tfsimple1
#   output:
#     steps:
#     - tfsimple1

seldon pipeline load -f ./pipelines/tfsimple.yaml
```

--------------------------------

### Restart Experiment with Updated Configuration

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/experiment-versions

Starts the experiment again with the new configuration that includes 'iris3'. This command applies the updated traffic splitting rules to the running experiment.

```bash
seldon experiment start -f ./experiments/ab-default-model2.yaml
```

--------------------------------

### Load LightGBM Model with Seldon CLI

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/model-zoo

Loads the LightGBM model into Seldon Core using the seldon-cli command. This prepares the model for deployment and management within the Seldon ecosystem.

```bash
seldon model load -f ./models/income-lgb.yaml
```

--------------------------------

### Minimal SeldonRuntime Configuration (YAML)

Source: https://docs.seldon.ai/seldon-core-2/installation/advanced-configurations/seldonruntime

This minimal YAML configuration illustrates the basic setup for a SeldonRuntime resource. It specifies the target namespace ('seldon-mesh') and the SeldonConfig to be used as the base for the installation.

```yaml
apiVersion: mlops.seldon.io/v1alpha1
kind: SeldonRuntime
metadata:
  name: seldon
  namespace: seldon-mesh
spec:
  seldonConfig: default
```

--------------------------------

### Create Income Explainer Model Resource (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/k8s-examples

This command applies the income explainer model's YAML definition to a Kubernetes cluster, creating the explainer resource in the specified namespace. Ensure the NAMESPACE variable is set.

```bash
kubectl create -f ./models/income-explainer.yaml -n ${NAMESPACE}
```

--------------------------------

### Infer with A/B Split

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/experiment-versions

Performs inference requests on the 'iris' model endpoint after an A/B experiment has started. The output demonstrates a traffic split between 'iris' and 'iris2' models, showing the distribution of responses.

```bash
seldon model infer iris -i 100 \
  '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
```

--------------------------------

### Display Income Explainer Model Configuration (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/k8s-examples

This command displays the content of the income explainer model's YAML configuration file. It defines an explainer of type 'anchor_tabular' and references the 'income' model.

```bash
cat ./models/income-explainer.yaml
```

--------------------------------

### Get Income Model Status (Bash & JSON)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/k8s-examples

This command retrieves the status of the 'income' model in JSON format and pipes it to `jq` for pretty-printing. It shows the model's readiness conditions and replica count.

```bash
kubectl get model income -n ${NAMESPACE} -o jsonpath='{.status}' | jq -M .
```

--------------------------------

### Clean up deployed resources

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/k8s-examples

This snippet demonstrates how to remove the deployed experiment and model resources from the Kubernetes cluster. It uses `kubectl delete` with the corresponding YAML files and namespace to revert the changes made during the experiment setup.

```bash
kubectl delete -f ./experiments/ab-default-model.yaml -n ${NAMESPACE}
kubectl delete -f ./models/sklearn1.yaml -n ${NAMESPACE}
kubectl delete -f ./models/sklearn2.yaml -n ${NAMESPACE}

```

--------------------------------

### Set Up Pipeline Inference Endpoint and Headers (Python)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

Constructs the inference endpoint URL and defines the necessary headers for sending requests to the deployed Seldon Pipeline. The 'Seldon-Model' header is essential for routing requests to the pipeline.

```python
pipeline_endpoint = f"http://{MESH_IP}/v2/models/income-classifier-app/infer"
pipeline_headers = {
    "Seldon-Model": "income-classifier-app.pipeline"
}
```

--------------------------------

### Get Tensorflow Model Metadata via seldon-cli

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-examples

Retrieves the metadata for the 'tfsimple1' model using the Seldon CLI. The output includes details about the model's name, versions, platform, and its input/output specifications.

```bash
seldon model metadata tfsimple1

```

--------------------------------

### Deploy Income Explainer Model using kubectl (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/income

Applies the income explainer model configuration to the Kubernetes cluster using kubectl. It requires the NAMESPACE environment variable to be set.

```bash
kubectl apply -f ../../models/income-explainer.yaml -n ${NAMESPACE}
```

--------------------------------

### Send Inference Request to Pipeline (Python)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

Sends a POST request with inference data to the Seldon Pipeline's endpoint and retrieves the JSON response. This demonstrates how to interact with the deployed pipeline to get predictions.

```python
pipeline_response = requests.post(
    pipeline_endpoint, json=text_inference_request, headers=pipeline_headers
)
pipeline_response.json()
```

--------------------------------

### Apply Model and Pipeline Deployments

Source: https://docs.seldon.ai/seldon-core-2/user-guide/inference/batch-examples-k8s

Applies the Kubernetes manifests for deploying the Iris and TFSimple models and their associated pipelines to the specified namespace.

```bash
kubectl apply -f models/sklearn-iris-gs.yaml -n ${NAMESPACE}
kubectl apply -f pipelines/iris.yaml -n ${NAMESPACE}

kubectl apply -f models/tfsimple1.yaml -n ${NAMESPACE}
kubectl apply -f pipelines/tfsimple.yaml -n ${NAMESPACE}
```

--------------------------------

### Deploy Model Manifest (Kubernetes)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

Applies the Model manifest to the Kubernetes cluster using `kubectl`. This command creates the `income-classifier` Model resource within the specified `seldon-mesh` namespace, making the model available for serving.

```bash
!kubectl apply -f ../../../samples/quickstart/models/sklearn-income-classifier.yaml -n seldon-mesh
```

--------------------------------

### Get Service Mesh IP (Kubernetes)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/quickstart

Retrieves the external IP address of the Seldon mesh service using `kubectl`. This IP is essential for constructing the inference endpoint to send requests to the deployed model.

```bash
MESH_IP = !kubectl get svc seldon-mesh -n seldon-mesh -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
MESH_IP = MESH_IP[0]
```

--------------------------------

### Deploy Seldon Core Pipeline using seldon-cli

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/income

This snippet demonstrates loading the pipeline definition using seldon-cli and then checking its status. It confirms that the pipeline is ready and provides detailed information about its configuration and current state.

```bash
seldon pipeline load -f ../../pipelines/income.yaml

seldon pipeline status income-production -w PipelineReady | jq -M .

```

--------------------------------

### Perform Inference on Income Explainer (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/k8s-examples

This command sends an inference request to the 'income-explainer' model to get an explanation for the input data. It uses the same input format as the model inference. The MESH_IP variable should be set to the inference host's IP address.

```bash
seldon model infer income-explainer --inference-host ${MESH_IP}:80 \
     '{"inputs": [{"name": "predict", "shape": [1, 12], "datatype": "FP32", "data": [[47,4,1,1,1,3,4,1,0,0,40,9]]}]}'
```

--------------------------------

### Deploy SKLearn Income Model using Kubectl

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/explainer-examples

This snippet shows how to deploy the SKLearn Income Model using a YAML configuration file with `kubectl`. It first displays the model configuration and then applies it to the Kubernetes cluster, waits for the model to be ready, and performs an inference request.

```bash
cat ./models/income.yaml

kubectl apply -f ./models/income.yaml -n ${NAMESPACE}
kubectl wait --for condition=ready --timeout=300s model income -n ${NAMESPACE}
curl --location 'http://${SELDON_INFER_HOST}/v2/models/income/infer' \
	--header 'Content-Type: application/json'  \
    --data '{"inputs": [{"name": "predict", "shape": [1, 12], "datatype": "FP32", "data": [[47,4,1,1,1,3,4,1,0,0,40,9]]}]}'
```

--------------------------------

### Add and Update Helm Repository for Seldon Core

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment

This snippet demonstrates how to add the Seldon Core Helm charts repository and update the local Helm repository cache. This is a prerequisite for installing Seldon Core components using Helm.

```bash
helm repo add seldon-charts https://seldonio.github.io/helm-charts/
helm repo update seldon-charts
```

--------------------------------

### Deploy SKLearn Model using seldon-cli (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-examples

Loads a Seldon Core Model definition from a YAML file using the seldon-cli tool. This is an alternative to using kubectl for model deployment.

```bash
seldon model load -f ./models/sklearn-iris-gs.yaml

```

--------------------------------

### Get Seldon Pipeline Output Details (JSON)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/cifar10

This is an example of the detailed JSON output obtained from inspecting a Seldon pipeline. It specifies the model and output name, along with its datatype, shape, and content, useful for programmatic access and understanding.

```json
seldon.default.model.cifar10-drift.outputs	cifeij8fh5ss738i5bp0	{"name":"is_drift", "datatype":"INT64", "shape":["1", "1"], "parameters":{"content_type":{"stringParam":"np"}}, "contents":{"int64Contents":["0"]}}
```

--------------------------------

### Check Model Readiness and Get Metadata with Python Triton Client

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/tritonclient-examples

Checks if a specific model is ready for inference on the Triton server and retrieves its metadata. This includes model name, versions, platform, inputs, and outputs. Requires the model name and optional headers.

```python
model_name = "tfsimple1"
headers = {"seldon-model": model_name}

print("model ready:", grpc_triton_client.is_model_ready(model_name, headers=headers))
print(grpc_triton_client.get_model_metadata(model_name, headers=headers))

```

--------------------------------

### Deploy AnchorText Explainer with kubectl (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/explainer-examples

This section shows how to deploy the AnchorText explainer model to Kubernetes using kubectl. It includes commands to apply the explainer configuration and wait for its readiness.

```bash
kubectl apply -f ./models/moviesentiment-explainer.yaml -n ${NAMESPACE}

```

```bash
kubectl wait --for condition=ready --timeout=300s model sentiment-explainer -n ${NAMESPACE}

```

--------------------------------

### Get Tensorflow Model Metadata via curl

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/local-examples

Fetches the metadata for the 'tfsimple1' model using a curl command to the Seldon inference server's /v2/models endpoint. This reveals model details like name, versions, platform, inputs, and outputs.

```bash
curl --location 'http://${SELDON_INFER_HOST}/v2/models/tfsimple1'

```

--------------------------------

### Get Seldon Core 2 External IP Address via Istio

Source: https://docs.seldon.ai/seldon-core-2/installation/production-environment/istio

Retrieves the external IP address of the Seldon mesh service managed by Istio. This IP address is used to access the Seldon Core 2 instance from outside the Kubernetes cluster and is essential for testing the installation.

```bash
ISTIO_INGRESS=$(kubectl get svc seldon-mesh -n seldon-mesh -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

echo "Seldon Core 2: http://$ISTIO_INGRESS"
```

--------------------------------

### Display Iris Pipeline Configuration (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/batch-examples-local

Displays the content of the 'iris.yaml' file, which defines a Seldon Core pipeline orchestrating the Iris model. This allows review of the pipeline's structure and steps before deployment.

```bash
cat pipelines/iris.yaml
```

--------------------------------

### Check TensorFlow Batch Inference Output

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/batch-examples-local

This command snippet demonstrates how to inspect the output of the TensorFlow batch inference. It uses 'cat' to read the output file and 'head -n 1' to get the first line, then pipes it to 'jq' for pretty-printing JSON output, allowing for quick verification of the inference results.

```bash
cat /tmp/tfsimple-pipeline-output.txt | head -n 1 | jq
```

--------------------------------

### Load AnchorText Explainer with seldon-cli (Bash)

Source: https://docs.seldon.ai/seldon-core-2/user-guide/examples/explainer-examples

This command uses the seldon-cli tool to load the AnchorText explainer model configuration, providing an alternative to kubectl for deployment.

```bash
seldon model load -f ./models/moviesentiment-explainer.yaml

```

--------------------------------

### Get Traefik Ingress IP Address

Source: https://docs.seldon.ai/seldon-core-2/integrations/service-meshes/traefik

This Python code snippet retrieves the external IP address of the Traefik service deployed in a Kubernetes cluster. It uses `kubectl` to query the service details and stores the IP address in an environment variable named `INGRESS_IP`. This is useful for configuring external access to services managed by Traefik.

```python
INGRESS_IP=!kubectl get svc traefik -n traefik-v2 -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
INGRESS_IP=INGRESS_IP[0]
import os
os.environ['INGRESS_IP'] = INGRESS_IP
INGRESS_IP

```
Tokens:
Raw