MLflow

https://mlflow.org/docs/latest
Admin
MLflow is an open-source platform for managing the end-to-end machine learning lifecycle, including...
Tokens:6,080,191
Snippets:49,461
Trust Score:9.7
Update:1 week ago
Show doc for...
Tokens:
Raw
### Install dependencies for Agno basic example

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/agno

Install the required packages for the basic Agno example: mlflow>=3.3, agno, anthropic, and yfinance.

```bash
pip install 'mlflow>=3.3' agno anthropic yfinance  
```

--------------------------------

### Get user example with environment setup and output

Source: https://mlflow.org/docs/latest/api_reference/_modules/mlflow/server/auth/client.html

Example usage of the get_user method, including the required environment variable setup for authentication, the Python code to create and retrieve a user, and the expected output. The user must exist, otherwise a RestException is raised.

```bash
export MLFLOW_TRACKING_USERNAME=admin
export MLFLOW_TRACKING_PASSWORD=password
```

```python
from mlflow.server.auth.client import AuthServiceClient

client = AuthServiceClient("tracking_uri")
client.create_user("newuser", "newpassword")
user = client.get_user("newuser")

print(f"user_id: {user.id}")
print(f"username: {user.username}")
print(f"password_hash: {user.password_hash}")
print(f"is_admin: {user.is_admin}")
```

```text
user_id: 3
username: newuser
password_hash: REDACTED
is_admin: False
```

--------------------------------

### Install MLflow and example plugin

Source: https://mlflow.org/docs/latest/ml/plugins

Clones the MLflow repository and installs the example plugin in editable mode. This is the first step to try the built-in test plugin.

```bash
# Clone MLflow and install example plugin  

git clone https://github.com/mlflow/mlflow  

cd mlflow  

pip install -e . tests/resources/mlflow-test-plugin  
```

--------------------------------

### Install MLflow Helm chart with example values

Source: https://mlflow.org/docs/latest/self-hosting/kubernetes-helm

Installs the MLflow Helm chart from a local checkout using the example values file. Creates the 'mlflow' namespace if it does not exist.

```bash
helm install mlflow ./charts \

  --namespace mlflow \

  --create-namespace \

  -f charts/example-mlflow-charts.yaml
```

--------------------------------

### Install MLflow and start the server

Source: https://mlflow.org/docs/latest/genai/eval-monitor/scorers/llm-judge/custom-judges/create-custom-judge

Install the MLflow package and start the server to access the UI Judge Builder. Run these commands in a terminal.

```bash
pip install mlflow  

mlflow server  

```

--------------------------------

### Install required packages

Source: https://mlflow.org/docs/latest/genai/eval-monitor/notebooks/quickstart-eval

Install the required packages for the quickstart: mlflow and openai.

```python
pip install mlflow openai  
```

--------------------------------

### Install MLflow and example plugin

Source: https://mlflow.org/docs/latest/plugins.html

Clones the MLflow repository and installs the example plugin using pip.

```bash
# Clone MLflow and install example plugin  

git clone https://github.com/mlflow/mlflow  

cd mlflow  

pip install -e . tests/resources/mlflow-test-plugin  

```

--------------------------------

### Install MLflow and start the server

Source: https://mlflow.org/docs/latest/genai/governance/ai-gateway/quickstart

Install MLflow with the genai extra and start the tracking server on port 5000. The AI Gateway is built into the server and requires a SQL-based backend store and FastAPI, which are default.

```bash
pip install 'mlflow[genai]'  

mlflow server --port 5000  

```

--------------------------------

### Hermes setup wizard output

Source: https://mlflow.org/docs/latest/genai/governance/ai-gateway/coding-agents/hermes-agent

Example output from the setup wizard showing provider selection. Select option 36 for custom endpoint.

```text
┌─────────────────────────────────────────────────────────┐

│             ⚕ Hermes Agent Setup Wizard                │

├─────────────────────────────────────────────────────────┤

│  Let's configure your Hermes Agent installation.       │

│  Press Ctrl+C at any time to exit.                     │

└─────────────────────────────────────────────────────────┘

  

  ...

  

  Select provider:

  Select by number, Enter to confirm.

  

  (●)  1. Nous Portal (Nous Research subscription)

  (○)  2. OpenRouter (100+ models, pay-per-use)

  (○)  3. LM Studio (local desktop app with built-in model server)

  ...

  (○) 36. Custom endpoint (enter URL manually)

  (○) 37. Configure auxiliary models...

  (○) 38. Leave unchanged

  

  Choice [default 1]: 36
```

--------------------------------

### Install MLflow with GenAI extra and start server (uv)

Source: https://mlflow.org/docs/latest/genai/tracing/token-usage-cost

Install the MLflow package with the [genai] extra (version >= 3.10.0) using uv and start the MLflow server. This enables cost tracking and other GenAI features.

```bash
uv add 'mlflow[genai]>=3.10.0'  

uv run mlflow server  

```

--------------------------------

### Quickstart: Log OpenAI traces with mlflow-tracing

Source: https://mlflow.org/docs/latest/genai/tracing/lightweight-sdk

Quickstart example for logging traces to an experiment on a remote MLflow server using the Production Tracing SDK with OpenAI. Sets the tracking URI and experiment, enables auto-tracing via mlflow.openai.autolog(), then uses OpenAI as usual to automatically log traces.

```python
import mlflow
import openai

# Set the tracking URI to your MLflow server
mlflow.set_tracking_uri("http://your-mlflow-server:5000")
mlflow.set_experiment("genai-production-monitoring")

# Enable auto-tracing for OpenAI
mlflow.openai.autolog()

# Use OpenAI as usual - traces will be automatically logged
client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4o-mini", messages=[{"role": "user", "content": "What is MLflow?"}]
)

print(response.choices[0].message.content)
```

--------------------------------

### Run the setup command for MLflow tracing

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/qwen_code

Starts an interactive flow to configure MLflow tracing for Qwen Code. The output shows the prompts and resulting configuration files. Use --non-interactive or -y to skip the interactive flow and use defaults.

```text
~/my-project $ mlflow-qwen-code setup

Configure MLflow tracing for Qwen Code

? Where should MLflow tracing be installed?

  ● Project  ./.qwen/  (default)

  ○ User  ~/.qwen/

  ↑/↓ to move, enter to select

✓ Registered Stop hook in /path/to/project/.qwen/settings.json

  MLflow tracking URI [http://localhost:5000] http://localhost:5000
  MLflow experiment ID [0] 1

✓ Wrote tracing config to /path/to/project/.qwen/mlflow-tracing.json

Next steps

  1. Start the MLflow tracking server in a separate terminal:

       mlflow server --port 5000

  2. Launch qwen — traces appear at http://localhost:5000 after each turn.
```

--------------------------------

### Start MLflow Tracking Server

Source: https://mlflow.org/docs/latest/api_reference/typescript_api/mlflow-core/index.html

Install mlflow and start the tracking server with a SQLite backend on port 5000.

```bash
pip install mlflow  
mlflow server --backend-store-uri sqlite:///mlruns.db --port 5000
Copy

```

--------------------------------

### Install MLflow with GenAI extra and start server (pip)

Source: https://mlflow.org/docs/latest/genai/tracing/token-usage-cost

Install the MLflow package with the [genai] extra (version >= 3.10.0) and start the MLflow server. This enables cost tracking and other GenAI features.

```bash
pip install 'mlflow[genai]>=3.10.0'  

mlflow server  

```

--------------------------------

### Install dependencies for DSPy quickstart

Source: https://mlflow.org/docs/latest/genai/flavors/dspy/notebooks/dspy_quickstart

Install the required packages: datasets, openai, dspy (>=3.0.3), and mlflow (>=3.4.0). Run this in a notebook or Python environment before proceeding.

```python
%pip install -U datasets openai "dspy>=3.0.3" "mlflow>=3.4.0"  
```

--------------------------------

### Set up MLflow experiment in Python

Source: https://mlflow.org/docs/latest/getting-started/intro-quickstart/index.html

Import MLflow and set the experiment name to 'MLflow Quickstart'. Run this in a notebook cell to start tracking.

```python
import mlflow

mlflow.set_experiment("MLflow Quickstart")
```

--------------------------------

### Run Hermes model setup wizard

Source: https://mlflow.org/docs/latest/genai/governance/ai-gateway/coding-agents/hermes-agent

Starts the interactive Hermes Agent setup wizard to configure the model provider. Choose option 36 for custom endpoint.

```bash
hermes setup model
```

--------------------------------

### Start the MLflow AI Gateway with mlflow gateway start

Source: https://mlflow.org/docs/latest/api_reference/_modules/mlflow/deployments/mlflow.html

Command to start the MLflow AI Gateway with a specified configuration file. This is the first step in the example for using the MlflowDeploymentClient.

```bash
mlflow gateway start --config-path path/to/config.yaml
```

--------------------------------

### Install MLflow, Semantic Kernel, and OpenAI

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/semantic_kernel

Install the required libraries for using MLflow tracing with Semantic Kernel. The example uses OpenAI, but the tutorial extends to all providers supported by Semantic Kernel.

```bash
pip install 'mlflow>=3.2.0' semantic_kernel openai  
```

--------------------------------

### Docker Compose setup commands

Source: https://mlflow.org/docs/latest/self-hosting/index.html

Commands to clone the MLflow repository, navigate to the docker-compose directory, copy the environment file, and start the services. After running, open http://localhost:5000 to view the UI.

```bash
git clone https://github.com/mlflow/mlflow.git  

cd mlflow/docker-compose  

cp .env.dev.example .env  

docker compose up -d  

# Open http://localhost:5000 in your browser to view the UI.  

```

--------------------------------

### Install OpenAI SDK

Source: https://mlflow.org/docs/latest/genai/eval-monitor/quickstart

Install the OpenAI Python package, required for the code examples in this guide. MLflow's evaluation framework works with any LLM provider, but these examples use the OpenAI SDK.

```bash
pip install openai
```

--------------------------------

### Run MLflow with Docker Compose

Source: https://mlflow.org/docs/latest/self-hosting

Clone the MLflow repository, navigate to the docker-compose directory, copy the environment example file, and start the services. The UI is then available at http://localhost:5000.

```bash
git clone https://github.com/mlflow/mlflow.git

cd mlflow/docker-compose

cp .env.dev.example .env

docker compose up -d

# Open http://localhost:5000 in your browser to view the UI.
```

--------------------------------

### Get Latest Versions Example

Source: https://mlflow.org/docs/latest/api_reference/_modules/mlflow/tracking/client.html

Shows the setup for retrieving the latest versions of a registered model using MlflowClient.

```python
import mlflow.sklearn
from mlflow import MlflowClient
from mlflow.models import infer_signature
from sklearn.datasets import make_regression
from sklearn.ensemble import RandomForestRegressor


def print_models_info(mv):
    for m in mv:
        print(f"name: {m.name}")
        print(f"latest version: {m.version}")
        print(f"run_id: {m.run_id}")
        print(f"current_stage: {m.current_stage}")
```

--------------------------------

### Create client and list endpoints with get_deploy_client

Source: https://mlflow.org/docs/latest/api_reference/_modules/mlflow/deployments/mlflow.html

Python example showing how to create a deployment client and list endpoints. It asserts that the returned endpoints match the expected structure.

```python
from mlflow.deployments import get_deploy_client

client = get_deploy_client("http://localhost:5000")
endpoints = client.list_endpoints()
assert [e.dict() for e in endpoints] == [
    {
        "name": "chat",
        "endpoint_type": "llm/v1/chat",
        "model": {"name": "gpt-4o-mini", "provider": "openai"},
        "endpoint_url": "http://localhost:5000/gateway/chat/invocations",
    },
]
```

--------------------------------

### Use plugin with custom tracking URI and launch UI

Source: https://mlflow.org/docs/latest/ml/plugins

Sets the MLFLOW_TRACKING_URI environment variable to use the file-plugin scheme, runs a tracking example, and starts the MLflow UI to view results. The UI is accessible at http://localhost:5000.

```bash
# Use the plugin with custom tracking URI scheme  

MLFLOW_TRACKING_URI=file-plugin:$(PWD)/mlruns python examples/quickstart/mlflow_tracking.py  

  

# Launch MLflow UI to view results  

mlflow server --backend-store-uri ./mlruns  
```

--------------------------------

### mlflow.login() example

Source: https://mlflow.org/docs/latest/api_reference/_modules/mlflow/utils/credentials.html

Shows how to call mlflow.login() to configure authentication and start a tracking run. Requires the Databricks SDK to be installed.

```python
import mlflow

mlflow.login()
with mlflow.start_run():
    mlflow.log_param("p", 0)
```

--------------------------------

### Command-line example: build Docker image and run local deployment

Source: https://mlflow.org/docs/latest/api_reference/_modules/mlflow/sagemaker.html

Shows the equivalent CLI commands to build a Docker image and run a local SageMaker deployment. Uses the --target sagemaker flag and -C options for port and image.

```bash
mlflow models build-docker --name "mlflow-pyfunc"
mlflow deployments run-local --target sagemaker \
        --name my-local-deployment \
        --model-uri "/mlruns/0/abc/model" \
        --flavor python_function \
        -C port=5000 \
        -C image="mlflow-pyfunc"
```

--------------------------------

### Configuration examples with MLflow CLI

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/claude_code

Examples for setting up tracing with a custom tracking URI, a Databricks backend with a specific experiment ID, or a specific experiment name.

```bash
# Set up with custom tracking URI
mlflow autolog claude -u file://./custom-mlruns
mlflow autolog claude -u sqlite:///mlflow.db

# Set up with Databricks backend and a specific experiment ID
mlflow autolog claude -u databricks -e 123456789

# Set up with a specific experiment name
mlflow autolog claude -n "My AI Project"
```

--------------------------------

### Install LangChain Dependencies

Source: https://mlflow.org/docs/latest/genai/tracing/app-instrumentation/automatic

Installs LangChain and the LangChain OpenAI integration, required for the multi-framework example.

```bash
pip install --upgrade langchain langchain-openai  

```

--------------------------------

### Start MLflow AI Gateway and list endpoints with MlflowDeploymentClient

Source: https://mlflow.org/docs/latest/api_reference/python_api/mlflow.deployments.html

First, start the MLflow AI Gateway with the command `mlflow gateway start --config-path path/to/config.yaml`. Then, create a client using `get_deploy_client` and list endpoints. The example asserts that the returned endpoints match the expected list with a chat endpoint.

```bash
mlflow gateway start --config-path path/to/config.yaml

```

```python
from mlflow.deployments import get_deploy_client

client = get_deploy_client("http://localhost:5000")
endpoints = client.list_endpoints()
assert [e.dict() for e in endpoints] == [
    {
        "name": "chat",
        "endpoint_type": "llm/v1/chat",
        "model": {"name": "gpt-4o-mini", "provider": "openai"},
        "endpoint_url": "http://localhost:5000/gateway/chat/invocations",
    },
]

```

--------------------------------

### Get Model Version Setup

Source: https://mlflow.org/docs/latest/api_reference/_modules/mlflow/tracking/client.html

Initial setup code for retrieving a model version using the MlflowClient.

```python
import mlflow.sklearn
from mlflow import MlflowClient
from mlflow.models import infer_signature
from sklearn.datasets import make_regression
from sklearn.ensemble import RandomForestRegressor

X, y = make_regression(n_features=4, n_informative=2, random_state=0, shuffle=False)
```

--------------------------------

### Install confirmed working Python versions

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/langchain

Installs the specific Python package versions that have been confirmed to work with the example.

```shell
pip install openai==3.3.1 langchain==1.3.17 langchain-openai==1.6.0 mlflow==3.15.1 tiktoken==0.14.0
```

--------------------------------

### Get Deployment Client and Manage Deployments

Source: https://mlflow.org/docs/latest/api_reference/_modules/mlflow/deployments/interface.html

Example showing how to retrieve a deployment client for a specific target and perform common lifecycle operations like creating, predicting, listing, updating, and deleting a deployment.

```python
from mlflow.deployments import get_deploy_client
import pandas as pd

client = get_deploy_client("redisai")
# Deploy the model stored at artifact path 'myModel' under run with ID 'someRunId'. The
# model artifacts are fetched from the current tracking server and then used for deployment.
client.create_deployment("spamDetector", "runs:/someRunId/myModel")
# Load a CSV of emails and score it against our deployment
emails_df = pd.read_csv("...")
prediction_df = client.predict_deployment("spamDetector", emails_df)
# List all deployments, get details of our particular deployment
print(client.list_deployments())
print(client.get_deployment("spamDetector"))
# Update our deployment to serve a different model
client.update_deployment("spamDetector", "runs:/anotherRunId/myModel")
# Delete our deployment
client.delete_deployment("spamDetector")
```

--------------------------------

### Set up OpenAI client and enable auto-tracing

Source: https://mlflow.org/docs/latest/genai/eval-monitor/scorers/custom/tutorial

Optional setup for using OpenAI models. Imports mlflow, os, and openai, enables auto-tracing with mlflow.openai.autolog(), creates an OpenAI client, and selects the model 'gpt-4o-mini'. Requires OPENAI_API_KEY to be set in the environment.

```python
import mlflow  

import os  

import openai  

  

# Ensure your OPENAI_API_KEY is set in your environment  

# os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>" # Uncomment and set if not globally configured  

  

# Enable auto-tracing for OpenAI  

mlflow.openai.autolog()  

  

# Create an OpenAI client  

client = openai.OpenAI()  

  

# Select an LLM  

model_name = "gpt-4o-mini"  
```

--------------------------------

### Manual installation commands for n8n-nodes-mlflow

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/n8n

Commands to manually install the n8n-nodes-mlflow package in the n8n directory and start n8n.

```bash
cd ~/.n8n  
npm install n8n-nodes-mlflow  
n8n start  
```

--------------------------------

### Run the plugin setup command

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/claude_code

Inside Claude Code, run this command and follow the wizard, providing the required values (tracking URI, experiment, scope).

```text
/mlflow-tracing:setup
```

--------------------------------

### Start Open WebUI

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/open-webui

Run this command to start the Open WebUI server.

```bash
open-webui serve
```

--------------------------------

### Start scorer with sampling configuration

Source: https://mlflow.org/docs/latest/api_reference/_modules/mlflow/genai/scorers/base.html

Example from the start() method docstring. Shows how to register a RelevanceToQuery scorer and start it with a sampling rate of 0.5, then start it again with a filter string. Requires an MLflow experiment to be set.

```python
import mlflow
from mlflow.genai.scorers import RelevanceToQuery, ScorerSamplingConfig

# Start scorer with 50% sampling rate
mlflow.set_experiment("my_genai_app")
scorer = RelevanceToQuery().register()
active_scorer = scorer.start(sampling_config=ScorerSamplingConfig(sample_rate=0.5))
print(f"Scorer is evaluating {active_scorer.sample_rate * 100}% of traces")

# Start scorer with filter to only evaluate specific traces
filtered_scorer = scorer.start(
    sampling_config=ScorerSamplingConfig(
        sample_rate=1.0, filter_string="YOUR_FILTER_STRING"
    )
)
```

--------------------------------

### Install the @mlflow/codex npm package

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/codex

Installs the @mlflow/codex npm package globally. If you prefer not to install globally, you can invoke the bin through npx (e.g., npx @mlflow/codex setup).

```bash
npm install -g @mlflow/codex
```

--------------------------------

### Run mlflow-codex setup interactively

Source: https://mlflow.org/docs/latest/genai/tracing/integrations/listing/codex

Runs the interactive setup command `mlflow-codex setup`. It prompts for installation location (Project ./.codex/ default or User ~/.codex/), then creates config.toml with a notify hook and writes mlflow-tracing.json. Pass --non-interactive / -y to skip the interactive flow and use default values. Setup refuses to overwrite a pre-existing notify = ... entry.

```text
~/my-project $ mlflow-codex setup

Configure MLflow tracing for Codex CLI

? Where should MLflow tracing be installed?

  ● Project  ./.codex/  (default)

  ○ User  ~/.codex/

  ↑/↓ to move, enter to select

✓ Created /path/to/project/.codex/config.toml with notify hook

  MLflow tracking URI [http://localhost:5000] http://localhost:5000
  MLflow experiment ID [0] 1

✓ Wrote tracing config to /path/to/project/.codex/mlflow-tracing.json

Next steps

  1. Start the MLflow tracking server in a separate terminal:

       mlflow server --port 5000

  2. Launch codex — traces appear at http://localhost:5000 after each turn.
```
Tokens:
Raw