MLServer

https://docs.seldon.ai/mlserver
Admin
MLServer is a high-performance, flexible, and extensible inference server for machine learning...
Tokens:52,327
Snippets:741
Trust Score:4.8
Update:4 months ago
Show doc for...
Tokens:
Raw
### Example Output of Summaries

Source: https://docs.seldon.ai/mlserver/getting-started

Example output of the fetched movie summaries.

```text
Barbie is a 2023 American fantasy comedy film directed by Greta Gerwig and written by Gerwig and Noah Baumbach. Based on the Barbie fashion dolls by Mattel, it is the first live-action Barbie film after numerous computer-animated direct-to-video and streaming television films. The film stars Margot Robbie as Barbie and Ryan Gosling as Ken, and follows the two on a journey of self-discovery following an existential crisis. The film also features an ensemble cast that includes America Ferrera, Kate McKinnon, Issa Rae, Rhea Perlman, and Will Ferrell...

Oppenheimer is a 2023 biographical thriller film written and directed by Christopher Nolan. Based on the 2005 biography American Prometheus by Kai Bird and Martin J. Sherwin, the film chronicles the life of J. Robert Oppenheimer, a theoretical physicist who was pivotal in developing the first nuclear weapons as part of the Manhattan Project, and thereby ushering in the Atomic Age. Cillian Murphy stars as Oppenheimer, with Emily Blunt as Oppenheimer's wife Katherine "Kitty" Oppenheimer; Matt Damon as General Leslie Groves, director of the Manhattan Project; and Robert Downey Jr. as Lewis Strauss, a senior member of the United States Atomic Energy Commission. The ensemble supporting cast includes Florence Pugh, Josh Hartnett, Casey Affleck, Rami Malek, Gary Oldman and Kenneth Branagh...
```

--------------------------------

### Query Documentation

Source: https://docs.seldon.ai/mlserver/getting-started

Example of how to query the documentation dynamically using an HTTP GET request with an 'ask' parameter.

```http
GET https://docs.seldon.ai/mlserver/getting-started.md?ask=<question>
```

--------------------------------

### Install MLServer, spaCy, and wikipedia-api

Source: https://docs.seldon.ai/mlserver/getting-started

Installs the necessary Python libraries for the document similarity use case.

```bash
pip install mlserver spacy wikipedia-api
```

--------------------------------

### Requirements File

Source: https://docs.seldon.ai/mlserver/getting-started

Example of a requirements.txt file for service dependencies.

```python

```

--------------------------------

### Query Documentation Example

Source: https://docs.seldon.ai/mlserver/runtimes/catboost

Example of how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/runtimes/catboost.md?ask=<question>
```

--------------------------------

### Start MLServer

Source: https://docs.seldon.ai/mlserver/getting-started

Command to start MLServer with a specified model.

```bash
mlserver start similarity_model
```

--------------------------------

### Querying Documentation Dynamically

Source: https://docs.seldon.ai/mlserver/examples

Demonstrates how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter to get answers to specific questions.

```HTTP
GET https://docs.seldon.ai/mlserver/examples.md?ask=<question>
```

--------------------------------

### Example Similarity Score

Source: https://docs.seldon.ai/mlserver/getting-started

Example output of the similarity score calculation.

```text
0.9866910567224084
```

--------------------------------

### Start MLServer Command

Source: https://docs.seldon.ai/mlserver/getting-started

Command to start the MLServer with a specified model directory.

```bash
mlserver start similarity_model/
```

--------------------------------

### List Docker Images

Source: https://docs.seldon.ai/mlserver/getting-started

Command to verify that the Docker image was successfully built.

```bash
docker images
```

--------------------------------

### Run Docker Container

Source: https://docs.seldon.ai/mlserver/getting-started

Command to test the built Docker image by running it as a container.

```bash
docker run -it --rm -p 8080:8080 fancy_ml_service
```

--------------------------------

### Create Model Directory

Source: https://docs.seldon.ai/mlserver/getting-started

Creates a directory for the model files.

```bash
mkdir -p similarity_model
```

--------------------------------

### MLServer Settings

Source: https://docs.seldon.ai/mlserver/getting-started

Configuration for parallel workers in MLServer.

```json
{
    "parallel_workers": 3
}
```

--------------------------------

### Example GET request with 'ask' parameter

Source: https://docs.seldon.ai/mlserver/runtimes/mlflow

Demonstrates how to query the documentation dynamically using the 'ask' query parameter in an HTTP GET request.

```http
GET https://docs.seldon.ai/mlserver/runtimes/mlflow.md?ask=<question>
```

--------------------------------

### Querying Documentation via HTTP GET

Source: https://docs.seldon.ai/mlserver/runtimes/alibi-detect

Example of how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/runtimes/alibi-detect.md?ask=<question>
```

--------------------------------

### Querying Documentation Dynamically

Source: https://docs.seldon.ai/mlserver/user-guide/deployment

This example shows how to perform an HTTP GET request to query the documentation dynamically with an 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/user-guide/deployment.md?ask=<question>
```

--------------------------------

### Querying documentation via HTTP GET request

Source: https://docs.seldon.ai/mlserver/runtimes/huggingface

Example of how to query the documentation dynamically using an HTTP GET request with an 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/runtimes/huggingface.md?ask=<question>
```

--------------------------------

### Import necessary libraries

Source: https://docs.seldon.ai/mlserver/getting-started

Imports the StringCodec for encoding inputs and the requests library for making HTTP calls.

```python
from mlserver.codecs import StringCodec
import requests
```

--------------------------------

### Install mlserver-alibi-explain

Source: https://docs.seldon.ai/mlserver/runtimes/alibi-explain

Installs the MLServer runtime compatible with Alibi-Explain.

```bash
pip install mlserver mlserver-alibi-explain
```

--------------------------------

### Querying Documentation Dynamically

Source: https://docs.seldon.ai/mlserver/user-guide

Demonstrates how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/user-guide.md?ask=<question>
```

--------------------------------

### Example inference request payload

Source: https://docs.seldon.ai/mlserver/getting-started

An example of the JSON structure for the inference request payload.

```json
{
    "inputs": [{
        "name": "docs",
        "shape": [2, 1],
        "datatype": "BYTES",
        "parameters": {"content_type": "str"},
        "data": [
            "Barbie is a 2023 American fantasy comedy...",
            "Oppenheimer is a 2023 biographical thriller..."
            ]
        }]
    }
```

--------------------------------

### Example inference response

Source: https://docs.seldon.ai/mlserver/getting-started

An example of the JSON response received from MLServer, showing the similarity score.

```json
{
    "model_name": "doc-sim-model",
    "id": "a4665ddb-1868-4523-bd00-a25902d9b124",
    "parameters": {},
    "outputs": [{
        "name": "output-0",
        "shape": [1],
        "datatype": "FP64",
        "parameters": {"content_type": "np"},
        "data": [0.9866910567224084]
    }]
}
```

--------------------------------

### Python Model Implementation

Source: https://docs.seldon.ai/mlserver/getting-started

Example Python code for a model that calculates similarity between two documents using spaCy.

```python
from mlserver.codecs import decode_args
from mlserver import MLModel
from typing import List
import numpy as np
import spacy

class MyKulModel(MLModel):

    async def load(self):
        self.model = spacy.load("en_core_web_lg")

    @decode_args
    async def predict(self, docs: List[str]) -> np.ndarray:

        doc1 = self.model(docs[0])
        doc2 = self.model(docs[1])

        return np.array(doc1.similarity(doc2))
```

--------------------------------

### Querying documentation dynamically

Source: https://docs.seldon.ai/mlserver/user-guide/deployment/seldon-core

This example shows how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/user-guide/deployment/seldon-core.md?ask=<question>
```

--------------------------------

### Start MLServer

Source: https://docs.seldon.ai/mlserver/examples/custom-json

Command to start the MLServer with the specified configuration files.

```shell
mlserver start .

```

--------------------------------

### Import Wikipedia API

Source: https://docs.seldon.ai/mlserver/getting-started

Imports the necessary library to fetch Wikipedia content.

```python
import wikipediaapi
```

--------------------------------

### Querying Documentation

Source: https://docs.seldon.ai/mlserver/runtimes/lightgbm

Example of how to query the documentation dynamically using an HTTP GET request with an 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/runtimes/lightgbm.md?ask=<question>
```

--------------------------------

### Initialize Wikipedia API

Source: https://docs.seldon.ai/mlserver/getting-started

Initializes the Wikipedia API client with a user agent and language.

```python
wiki_wiki = wikipediaapi.Wikipedia('MyMovieEval (example@example.com)', 'en')
```

--------------------------------

### Example similarity output

Source: https://docs.seldon.ai/mlserver/getting-started

The expected output when printing the similarity score.

```text
Our movies are 98.6691% similar
```

--------------------------------

### Build Docker Image

Source: https://docs.seldon.ai/mlserver/getting-started

Command to build a Docker image for the similarity model using MLServer.

```python
mlserver build similarity_model/ -t 'fancy_ml_service'
```

--------------------------------

### Querying Documentation Dynamically

Source: https://docs.seldon.ai/mlserver/user-guide/metrics

This example shows how to query the documentation dynamically by performing an HTTP GET request on the current page URL with the `ask` query parameter.

```http
GET https://docs.seldon.ai/mlserver/user-guide/metrics.md?ask=<question>
```

--------------------------------

### Querying Documentation Dynamically

Source: https://docs.seldon.ai/mlserver/api-reference

This example shows how to perform an HTTP GET request to query the documentation dynamically using the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/api-reference.md?ask=<question>
```

--------------------------------

### Querying Documentation

Source: https://docs.seldon.ai/mlserver/runtimes/custom

Example of how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/runtimes/custom.md?ask=<question>
```

--------------------------------

### MLServer Start Command

Source: https://docs.seldon.ai/mlserver/api-reference/cli

Start serving a machine learning model with MLServer.

```bash
mlserver start [OPTIONS] FOLDER
```

--------------------------------

### Querying Documentation via HTTP GET

Source: https://docs.seldon.ai/mlserver/api-reference/modelsettings

Demonstrates how to perform an HTTP GET request to query the documentation dynamically by appending the 'ask' query parameter to the URL.

```http
GET https://docs.seldon.ai/mlserver/api-reference/modelsettings.md?ask=<question>
```

--------------------------------

### Load spaCy Model

Source: https://docs.seldon.ai/mlserver/getting-started

Loads the English language model for spaCy.

```python
import spacy
nlp = spacy.load("en_core_web_lg")
```

--------------------------------

### Load Wiki Summaries

Source: https://docs.seldon.ai/mlserver/getting-started

Python code to load summaries of films from Wikipedia.

```python
deep_impact    = wiki_wiki.page('Deep_Impact_(film)').summary
armageddon     = wiki_wiki.page('Armageddon_(1998_film)').summary

antz           = wiki_wiki.page('Antz').summary
a_bugs_life    = wiki_wiki.page("A_Bug's_Life").summary

the_dark_night = wiki_wiki.page('The_Dark_Knight').summary
mamma_mia      = wiki_wiki.page('Mamma_Mia!_(film)').summary
```

--------------------------------

### Parallel POST Requests

Source: https://docs.seldon.ai/mlserver/getting-started

Python code to map similarity score requests in parallel.

```python
results = list(
    map(get_sim_score, (deep_impact, antz, the_dark_night), (armageddon, a_bugs_life, mamma_mia)))
results
```

--------------------------------

### Query Documentation Dynamically

Source: https://docs.seldon.ai/mlserver/examples/cassava

Example of how to perform an HTTP GET request to query the documentation dynamically using the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/examples/cassava.md?ask=<question>
```

--------------------------------

### Fetch Movie Summaries

Source: https://docs.seldon.ai/mlserver/getting-started

Fetches summaries for the 'Barbie' and 'Oppenheimer' films from Wikipedia and prints them.

```python
barbie = wiki_wiki.page('Barbie_(film)').summary
oppenheimer = wiki_wiki.page('Oppenheimer_(film)').summary

print(barbie)
print()
print(oppenheimer)
```

--------------------------------

### Querying Documentation Dynamically

Source: https://docs.seldon.ai/mlserver/user-guide/deployment/kserve

Example of how to perform an HTTP GET request to query the documentation dynamically with an 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/user-guide/deployment/kserve.md?ask=<question>
```

--------------------------------

### Querying Documentation Dynamically

Source: https://docs.seldon.ai/mlserver/user-guide/openapi

Example of how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/user-guide/openapi.md?ask=<question>
```

--------------------------------

### Querying Documentation Example

Source: https://docs.seldon.ai/mlserver/api-reference/pythonapi/mlmodel

Perform an HTTP GET request on the current page URL with the `ask` query parameter to query the documentation dynamically.

```bash
GET https://docs.seldon.ai/mlserver/api-reference/pythonapi/mlmodel.md?ask=<question>
```

--------------------------------

### Install mlserver-alibi-detect

Source: https://docs.seldon.ai/mlserver/runtimes/alibi-detect

Command to install the mlserver-alibi-detect runtime alongside mlserver.

```bash
pip install mlserver mlserver-alibi-detect
```

--------------------------------

### Sequential Similarity Score Calculation

Source: https://docs.seldon.ai/mlserver/getting-started

Python code to calculate similarity scores sequentially.

```python
for movie1, movie2 in zip((deep_impact, antz, the_dark_night), (armageddon, a_bugs_life, mamma_mia)):
    print(get_sim_score(movie1, movie2))
```

--------------------------------

### Agent Instructions: Querying Documentation

Source: https://docs.seldon.ai/mlserver/api-reference/modelparameters

Example of how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/api-reference/modelparameters.md?ask=<question>
```

--------------------------------

### Install mlserver

Source: https://docs.seldon.ai/mlserver

Install the mlserver package.

```bash
pip install mlserver
```

--------------------------------

### Install MLServer and Catboost Runtime

Source: https://docs.seldon.ai/mlserver/runtimes/catboost

Command to install the MLServer and MLServer-Catboost packages.

```bash
pip install mlserver mlserver-catboost
```

--------------------------------

### Model Settings Configuration

Source: https://docs.seldon.ai/mlserver/getting-started

JSON configuration file for MLServer specifying the model name and implementation class.

```json
{
    "name": "doc-sim-model",
    "implementation": "my_model.MyKulModel"
}
```

--------------------------------

### Install alibi and alibi_detect libraries

Source: https://docs.seldon.ai/mlserver/examples/alibi-detect

Installs the necessary Python libraries for dataset dependencies and detector configuration.

```python
!pip install alibi alibi_detect
```

--------------------------------

### Start mlserver

Source: https://docs.seldon.ai/mlserver/examples/alibi-detect

Command to start the mlserver with the specified configuration.

```shell
mlserver start .
```

--------------------------------

### Install dependencies

Source: https://docs.seldon.ai/mlserver/examples/cassava

Install all the libraries required to run the code.

```bash
pip install -r requirements.txt
```

--------------------------------

### AlibiDetectRuntime model-settings.json example

Source: https://docs.seldon.ai/mlserver/runtimes/alibi-detect

Example configuration for the AlibiDetectRuntime in model-settings.json, including batch_size.

```json
{
  "name": "drift-detector",
  "implementation": "mlserver_alibi_detect.AlibiDetectRuntime",
  "parameters": {
    "uri": "./alibi-detect-artifact/",
    "extra": {
      "batch_size": 5
    }
  }
}
```

--------------------------------

### Start MLServer

Source: https://docs.seldon.ai/mlserver/examples/cassava

Command to start the MLServer with the configured model.

```bash
mlserver start model/
```

--------------------------------

### Install MLServer and HuggingFace runtime

Source: https://docs.seldon.ai/mlserver/runtimes/huggingface

Command to install the MLServer package along with the HuggingFace runtime.

```bash
pip install mlserver mlserver-huggingface
```

--------------------------------

### Querying Documentation

Source: https://docs.seldon.ai/mlserver/api-reference/pythonapi/types

Instructions on how to query the documentation dynamically using an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/api-reference/pythonapi/types.md?ask=<question>
```

--------------------------------

### start() method signature

Source: https://docs.seldon.ai/mlserver/api-reference/pythonapi/metrics

Signature for the start method.

```python
start()
```

--------------------------------

### Querying Documentation

Source: https://docs.seldon.ai/mlserver/runtimes/mllib

Example of how to query the documentation dynamically using an HTTP GET request with an 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/runtimes/mllib.md?ask=<question>
```

--------------------------------

### Install MLServer and LightGBM runtime

Source: https://docs.seldon.ai/mlserver/runtimes/lightgbm

Command to install the MLServer runtime along with the LightGBM package.

```bash
pip install mlserver mlserver-lightgbm
```

--------------------------------

### Query Documentation via HTTP GET

Source: https://docs.seldon.ai/mlserver/api-reference/settings

Perform an HTTP GET request on the current page URL with the `ask` query parameter to query the documentation dynamically.

```http
GET https://docs.seldon.ai/mlserver/api-reference/settings.md?ask=<question>
```

--------------------------------

### Querying documentation with 'ask' parameter

Source: https://docs.seldon.ai/mlserver/api-reference/pythonapi/metrics

Example of how to perform an HTTP GET request to query the documentation dynamically using the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/api-reference/pythonapi/metrics.md?ask=<question>
```

--------------------------------

### Python App

Source: https://docs.seldon.ai/mlserver/examples/cassava

The starting point for this tutorial is the python script app.py. This script loads a pre-trained Tensorflow model, loads the Cassava dataset, preprocesses examples, makes predictions, and plots the results.

```python
from helpers import plot, preprocess
import tensorflow as tf
import tensorflow_datasets as tfds
import tensorflow_hub as hub

# Fixes an issue with Jax and TF competing for GPU
tf.config.experimental.set_visible_devices([], 'GPU')

# Load the model
model_path = './model'
classifier = hub.KerasLayer(model_path)

# Load the dataset and store the class names
dataset, info = tfds.load('cassava', with_info=True)
class_names = info.features['label'].names + ['unknown']

# Select a batch of examples and plot them
batch_size = 9
batch = dataset['validation'].map(preprocess).batch(batch_size).as_numpy_iterator()
examples = next(batch)
plot(examples, class_names)

# Generate predictions for the batch and plot them against their labels
predictions = classifier(examples['image'])
predictions_max = tf.argmax(predictions, axis=-1)
print(predictions_max)
plot(examples, class_names, predictions_max)
```

--------------------------------

### Install MLServer and SKLearn Runtime

Source: https://docs.seldon.ai/mlserver/runtimes/sklearn

Installs the MLServer core package along with the Scikit-Learn runtime.

```bash
pip install mlserver mlserver-sklearn
```

--------------------------------

### Querying Documentation via HTTP GET

Source: https://docs.seldon.ai/mlserver/runtimes/sklearn

Demonstrates how to ask a question about the documentation using an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/runtimes/sklearn.md?ask=<question>
```

--------------------------------

### Querying Documentation

Source: https://docs.seldon.ai/mlserver/api-reference/cli

Perform an HTTP GET request on a documentation URL with the 'ask' query parameter to get dynamic information.

```http
GET https://docs.seldon.ai/mlserver/api-reference/cli.md?ask=<question>
```

--------------------------------

### Install MLServer XGBoost runtime

Source: https://docs.seldon.ai/mlserver/runtimes/xgboost

Command to install the MLServer XGBoost runtime alongside mlserver.

```bash
pip install mlserver mlserver-xgboost
```

--------------------------------

### Install MLServer and MLlib Runtime

Source: https://docs.seldon.ai/mlserver/runtimes/mllib

Installs the MLServer and the Spark MLlib runtime using pip.

```bash
pip install mlserver mlserver-mllib
```

--------------------------------

### Run the no-zookeeper kafka broker

Source: https://docs.seldon.ai/mlserver/examples/kafka

Command to start the Kafka broker.

```shell
!./kafka_2.12-2.8.0/bin/kafka-server-start.sh ./kafka_2.12-2.8.0/config/kraft/server.properties
```

--------------------------------

### Install MLServer and MLflow runtime

Source: https://docs.seldon.ai/mlserver/runtimes/mlflow

Command to install the MLserver package along with the MLflow runtime.

```bash
pip install mlserver mlserver-mlflow
```

--------------------------------

### Querying Documentation Dynamically

Source: https://docs.seldon.ai/mlserver/user-guide/streaming

To get additional information not directly available on a page, perform an HTTP GET request with the 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/user-guide/streaming.md?ask=<question>
```

--------------------------------

### Custom Metrics Example

Source: https://docs.seldon.ai/mlserver/user-guide/metrics

Example of how to register and log custom metrics within a custom inference runtime.

```python
import mlserver

from mlserver.types import InferenceRequest, InferenceResponse

class MyCustomRuntime(mlserver.MLModel):
  async def load(self) -> bool:
    self._model = load_my_custom_model()
    mlserver.register("my_custom_metric", "This is a custom metric example")
    return True

  async def predict(self, payload: InferenceRequest) -> InferenceResponse:
    mlserver.log(my_custom_metric=34)
    # TODO: Replace for custom logic to run inference
    return self._model.predict(payload)
```

--------------------------------

### Get JSON response

Source: https://docs.seldon.ai/mlserver/getting-started

Retrieves the JSON response from the MLServer.

```python
r.json()
```

--------------------------------

### Request Input with Custom Parameters

Source: https://docs.seldon.ai/mlserver/user-guide/adaptive-batching

Example of how custom parameters are structured in an individual request.

```python
types.RequestInput(
    name="parameters-np",
    shape=[1],
    datatype="BYTES",
    data=[],
    parameters=types.Parameters(
        custom-param='value-1',
    )
)
```

```python
types.RequestInput(
    name="parameters-np",
    shape=[1],
    datatype="BYTES",
    data=[],
    parameters=types.Parameters(
        custom-param='value-2',
    )
)
```

--------------------------------

### Clone the repository

Source: https://docs.seldon.ai/mlserver/examples/cassava

The easiest way to run this example is to clone the repository.

```bash
git clone https://github.com/SeldonIO/cassava-example.git
```

--------------------------------

### Agent Instructions: Querying This Documentation

Source: https://docs.seldon.ai/mlserver/runtimes

Instructions on how to dynamically query the documentation using an HTTP GET request with an 'ask' query parameter.

```http
GET https://docs.seldon.ai/mlserver/runtimes.md?ask=<question>
```

--------------------------------

### Test Similarity Score Function

Source: https://docs.seldon.ai/mlserver/getting-started

Python code to test the similarity score function.

```python
get_sim_score(deep_impact, armageddon)
```

--------------------------------

### Calculate Similarity Score

Source: https://docs.seldon.ai/mlserver/getting-started

Calculates and prints the similarity score between the two document objects.

```python
doc1.similarity(doc2)
```

--------------------------------

### Run a custom Docker image

Source: https://docs.seldon.ai/mlserver/examples/custom

Start a container from the custom-built Docker image and map the port.

```bash
docker run -it --rm -p 8080:8080 my-custom-numpyro-server:0.1.0
```

--------------------------------

### Print similarity score

Source: https://docs.seldon.ai/mlserver/getting-started

Prints the calculated similarity score between the two movies, formatted as a percentage.

```python
print(f"Our movies are {round(r.json()['outputs'][0]['data'][0] * 100, 4)}% similar!")
```

--------------------------------

### Compare Document Similarity

Source: https://docs.seldon.ai/mlserver/getting-started

Compares the similarity between the two fetched movie summaries using spaCy.

```python
doc1 = nlp(barbie)
doc2 = nlp(oppenheimer)
```

--------------------------------

### Similarity Score Function

Source: https://docs.seldon.ai/mlserver/getting-started

Python function to calculate similarity score between two documents using MLServer.

```python
def get_sim_score(movie1, movie2):
    response = requests.post(
        'http://0.0.0.0:8080/v2/models/doc-sim-model/infer',
        json={
            "inputs": [
                StringCodec.encode_input(name='docs', payload=[movie1, movie2], use_bytes=False).model_dump()
            ]
        })
    return response.json()['outputs'][0]['data'][0]
```

--------------------------------

### Send POST request to MLServer

Source: https://docs.seldon.ai/mlserver/getting-started

Sends a POST request to the MLServer endpoint with the prepared inference request payload.

```python
r = requests.post('http://0.0.0.0:8080/v2/models/doc-sim-model/infer', json=inference_request)
```

--------------------------------

### Prepare inference request payload

Source: https://docs.seldon.ai/mlserver/getting-started

Constructs the inference request payload using StringCodec to encode the input documents.

```python
inference_request = {
    "inputs": [
        StringCodec.encode_input(name='docs', payload=[barbie, oppenheimer], use_bytes=False).model_dump()
    ]
}
print(inference_request)
```

--------------------------------

### Download spaCy language model

Source: https://docs.seldon.ai/mlserver/getting-started

Downloads the English language model required by spaCy for natural language processing tasks.

```bash
python -m spacy download en_core_web_lg
```

--------------------------------

### Querying Documentation

Source: https://docs.seldon.ai/mlserver/examples/xgboost

Perform an HTTP GET request on the current page URL with the `ask` query parameter to dynamically query the documentation.

```http
GET https://docs.seldon.ai/mlserver/examples/xgboost.md?ask=<question>
```

--------------------------------

### Run the Python App

Source: https://docs.seldon.ai/mlserver/examples/cassava

Try running the app.py script yourself.

```bash
python app.py
```

--------------------------------

### MLServer Init Command

Source: https://docs.seldon.ai/mlserver/api-reference/cli

Generate a base project template for MLServer.

```bash
mlserver init [OPTIONS]
```

--------------------------------

### Querying Documentation

Source: https://docs.seldon.ai/mlserver/examples/custom-json

This section explains how to query the documentation dynamically for additional information not directly present on the page. It involves performing an HTTP GET request to the page URL with an 'ask' query parameter.

```bash
GET https://docs.seldon.ai/mlserver/examples/custom-json.md?ask=<question>
```

--------------------------------

### Install mlserver-sklearn

Source: https://docs.seldon.ai/mlserver

Install the mlserver-sklearn package to serve scikit-learn models.

```bash
pip install mlserver-sklearn
```
Tokens:
Raw