### Install and Run MLflow Server Source: https://mlops-coding-course.fmind.dev/5.%20Refining/5.5 Commands to install MLflow using uv, verify the installation, and start the MLflow UI server locally. It also includes a Docker Compose configuration for a more permanent setup of the MLflow server. ```shell uv add mlflow ``` ```shell uv run mlflow doctor uv run mlflow server ``` ```yaml services: mlflow: image: ghcr.io/mlflow/mlflow:v2.14.1 ports: - 5000:5000 environment: - MLFLOW_HOST=0.0.0.0 command: mlflow server ``` -------------------------------- ### Reusable Setup Composite Action (GitHub Actions) Source: https://mlops-coding-course.fmind.dev/5.%20Refining/5.3 A composite action designed to streamline CI/CD workflows by encapsulating common setup steps. It installs the 'uv' package manager with caching enabled and sets up the project's Python version based on a .python-version file. ```yaml name: Setup description: Setup for project workflows runs: using: composite steps: - name: Install uv uses: astral-sh/setup-uv@v5 with: enable-cache: true - name: Setup Python uses: actions/setup-python@v5 with: python-version-file: .python-version ``` -------------------------------- ### Install and Run MLflow Server Source: https://mlops-coding-course.fmind.dev/5.%20Refining/5.6 Installs MLflow using 'uv' and then starts the MLflow tracking server. This is the initial setup for using MLflow's features locally. ```bash uv add mlflow uv run mlflow doctor uv run mlflow server ``` -------------------------------- ### Initialize and Sync MLOps Project with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.3 Demonstrates the command-line steps to initialize a new MLOps project using uv and then synchronize its dependencies. This involves creating a project directory, running 'uv init' to configure pyproject.toml, and 'uv sync' to install dependencies and create a lock file. ```bash mkdir my-mlops-project && cd my-mlops-project ``` ```bash uv init ``` ```bash uv sync ``` -------------------------------- ### Install uv using curl Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Installs uv using a curl command, which downloads and executes an installation script. This script handles platform detection and the installation process automatically. ```shell curl -LsSf https://astral.sh/uv/install.sh | sh ``` -------------------------------- ### Replace pipx: Install a tool with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Demonstrates installing command-line tools in isolated environments using uv's `tool install` command, which replicates `pipx install`. This installs the specified tool globally accessible. ```shell uv tool install ruff ``` -------------------------------- ### Install uv using pip or pipx Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Alternative methods to install uv using existing package managers. Users can choose between pip or pipx based on their current setup. ```shell # Using pip pip install uv ``` ```shell # Using pipx pipx install uv ``` -------------------------------- ### Replace pip: Install packages with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Demonstrates how to use uv to install Python packages, mimicking the functionality of `pip install`. This command installs specified packages into the current environment. ```shell uv pip install requests numpy pandas ``` -------------------------------- ### Install and Manage pre-commit Hooks Source: https://mlops-coding-course.fmind.dev/5.%20Refining/5.2 These commands are used to install and manage pre-commit hooks for different Git actions. 'pre-commit install' sets up hooks to run automatically before each commit. Optional commands allow installation for 'pre-push' and 'commit-msg' actions. ```bash # Install hooks to run automatically before each commit uv run pre-commit install # (Optional) Install hooks for other git actions uv run pre-commit install --hook-type pre-push uv run pre-commit install --hook-type commit-msg ``` -------------------------------- ### Build and Install Package, then Run Entrypoint Source: https://mlops-coding-course.fmind.dev/3.%20Productionizing/3.3 This shell command sequence outlines the process of building a Python package into a wheel file, installing it using `pip`, and then running its declared entrypoint as a standard command-line tool. This is the typical workflow for distributing and using installed packages. ```shell # 1. Build the package into a wheel file in the dist/ directory uv build --wheel # 2. Install the package from the generated wheel file pip install dist/bikes-*.whl # 3. Run the entrypoint as a native command bikes config.yml --extras key=value ``` -------------------------------- ### Configure Private Package Repositories with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.3 Provides an example of how to configure custom and private package repositories in pyproject.toml for uv. It shows how to define a source for a private repository, which can then be used alongside public PyPI packages. ```toml # pyproject.toml [tool.uv.sources] private-repo = { url = "https://my-private-pypi.example.com/simple" } ``` -------------------------------- ### Replace pipx: List installed tools with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Shows how to list all command-line tools installed via uv's `tool` command, similar to `pipx list`. This provides an overview of available tools. ```shell uv tool list ``` -------------------------------- ### Replace pip: Install from requirements.txt with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Shows how to use uv to install packages listed in a `requirements.txt` file, equivalent to `pip install -r requirements.txt`. This is useful for managing project dependencies. ```shell uv pip install -r requirements.txt ``` -------------------------------- ### Replace pyenv: Install a Python version with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Demonstrates installing a specific Python version using uv's `python install` command, replacing the need for pyenv. uv downloads and manages the specified Python version. ```shell uv python install 3.12 ``` -------------------------------- ### GitHub Actions Workflow Example Source: https://mlops-coding-course.fmind.dev/6.%20Sharing/6.0 A basic example of a GitHub Actions workflow file defining CI/CD automation triggered by repository events. ```yaml name: CI/CD Pipeline on: push: branches: [ main ] pull_request: branches: [ main ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up Python 3.10 uses: actions/setup-python@v3 with: python-version: "3.10" - name: Install dependencies run: | python -m pip install --upgrade pip pip install flake8 pytest if [ -f requirements.txt ]; then pip install -r requirements.txt; fi - name: Lint with flake8 run: | # stop the build if there are Python syntax errors or undefined names flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics - name: Test with pytest run: pytest ``` -------------------------------- ### Replace pyenv: List available Python versions with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Shows how to list all Python versions that uv can install or is currently managing, similar to `pyenv install --list`. This helps users see available options. ```shell uv python list ``` -------------------------------- ### Install Python Version with Pyenv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/archives/1.2 Installs a specific version of Python using the pyenv command-line tool. This is essential for ensuring project compatibility with required Python environments. No specific inputs or outputs are detailed beyond the action of installation. ```bash pyenv install 3.13 ``` -------------------------------- ### Install and Run Ruff Linter (Python) Source: https://mlops-coding-course.fmind.dev/4.%20Validating/4.1 This snippet demonstrates how to install the Ruff linter using 'uv' and then run it to check the codebase. It's essential for enforcing code quality and catching potential issues early in the development cycle. ```shell # Install Ruff into your "check" dependency group uv add --group check ruff # Run Ruff to lint your codebase uv run ruff check src/ tests/ ``` -------------------------------- ### Replace pip: Generate requirements.txt with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Illustrates how to use uv to generate a `requirements.txt` file from the currently installed packages, similar to `pip freeze > requirements.txt`. This helps in documenting project dependencies. ```shell uv pip freeze > requirements.txt ``` -------------------------------- ### Command-Line Interface for Layered Configurations Source: https://mlops-coding-course.fmind.dev/3.%20Productionizing/3.4 This example shows how to provide configuration files to an application using the command-line interface. It demonstrates layering a default configuration with a training-specific configuration and an additional flag. ```bash $ bikes defaults.yaml training.yaml --verbose ``` -------------------------------- ### Cookiecutter post_gen_project.py Hook Example Source: https://mlops-coding-course.fmind.dev/6.%20Sharing/6.4 Example of a Cookiecutter post-generation hook script written in Python. This script demonstrates conditional logic to remove a `requirements.txt` file if a package manager other than pip was selected during project setup. ```python import os ``` -------------------------------- ### Justfile Configuration Example Source: https://mlops-coding-course.fmind.dev/5.%20Refining/5.1 Demonstrates the basic structure and configuration of a 'justfile'. This includes setting up required command-line tools, configuring Just's behavior (like dotenv loading), defining global variables, and setting a default task. ```justfile # Main configuration file for Just. See docs: https://just.systems/man/en/ # REQUIRES: Ensure necessary command-line tools are available. docker := require("docker") find := require("find") rm := require("rm") uv := require("uv") # SETTINGS: Configure Just's behavior. set dotenv-load := true # Automatically load environment variables from a .env file. # VARIABLES: Define global constants for your tasks. PACKAGE := "bikes" REPOSITORY := "bikes" SOURCES := "src" TESTS := "tests" # DEFAULTS: Define the default action when 'just' is run without arguments. default: @just --list # Display a list of available tasks. ``` -------------------------------- ### Configure Mypy Plugins for Pandera and Pydantic Source: https://mlops-coding-course.fmind.dev/4.%20Validating/4.0 This snippet shows how to enable Mypy plugins for libraries like Pandera and Pydantic. It requires the specified plugins to be installed and configured in your Mypy setup. ```Python plugins = ["pandera.mypy", "pydantic.mypy"] ``` -------------------------------- ### Automate GitHub Ruleset Installation using a Just Task Source: https://mlops-coding-course.fmind.dev/6.%20Sharing/6.6 This bash script, intended to be used with the 'just' task runner, automates the installation of GitHub rulesets. It fetches repository and owner information using the 'gh' CLI and then uses the 'gh api' command to POST the ruleset definition to the repository. Ensure the 'gh' CLI is authenticated. ```bash # install github rulesets [group('install')] install-rulesets: #!/usr/bin/env bash set -euo pipefail repo=$(gh repo view --json=name --jq=.name) owner=$(gh repo view --json=owner --jq=.owner.login) gh api --method POST -H "Accept: application/vnd.github+json" \ "/repos/$owner/$repo/rulesets" --input=".github/rulesets/main.json" ``` -------------------------------- ### Python DVC Pipeline Snippet Example Source: https://mlops-coding-course.fmind.dev/0.%20Overview/0.5 This Python code snippet demonstrates a basic DVC (Data Version Control) pipeline. It requires the DVC library to be installed and configured in your project. The output is a DVC pipeline configuration that can be executed to manage data and model versions. ```python import dvc.api def run_dvc_pipeline(): # Example: Reproduce a DVC pipeline stage # Replace 'stage_name' with the actual name of your DVC stage try: dvc.api.reproduce('stage_name') print("DVC pipeline stage 'stage_name' reproduced successfully.") except Exception as e: print(f"Error reproducing DVC pipeline stage: {e}") if __name__ == "__main__": # This is a placeholder. Actual DVC pipeline usage involves dvc.yaml file. print("This script demonstrates conceptual usage. Refer to your dvc.yaml for actual pipeline definition.") # run_dvc_pipeline() # Uncomment to attempt reproduction if dvc.yaml is set up ``` -------------------------------- ### Run Commands with uv Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.3 Shows how to execute scripts and command-line tools within the uv-managed virtual environment using the 'uv run' command. This avoids the need for manual environment activation. ```bash # Run a Python script $ uv run python my_app/main.py # Run a command-line tool installed in the environment $ uv run pytest # Run your project's main entrypoint script $ uv run bikes --help ``` -------------------------------- ### Baseline Dockerfile for Python MLOps Project Source: https://mlops-coding-course.fmind.dev/5.%20Refining/5.4 A simple Dockerfile for a Python MLOps project using 'uv' for package management. It starts from a pre-built Python and 'uv' image, copies a wheel file, installs it, and sets a default command. This Dockerfile is designed for efficiency and ease of use. ```dockerfile # Dockerfile Reference: https://docs.docker.com/engine/reference/builder/ # 1. Start from a lean, official base image with Python and uv pre-installed. FROM ghcr.io/astral-sh/uv:python3.13-bookworm # 2. Copy the pre-built Python wheel file into the image. COPY dist/*.whl . # 3. Install the wheel file into the system's Python environment. RUN uv pip install --system *.whl # 4. Define the default command to run when the container starts. CMD ["bikes", "--help"] ``` -------------------------------- ### Example YAML Configuration for Training Job Source: https://mlops-coding-course.fmind.dev/3.%20Productionizing/3.4 This YAML file defines parameters for a training job, including the job kind, input data reader and path, and target data reader and path. It demonstrates a human-readable way to specify settings separate from code. ```yaml # Example: confs/training.yaml job: KIND: TrainingJob # Specifies the type of job to run inputs: KIND: ParquetReader # Defines the reader for input data path: data/inputs.parquet # Path to the input dataset targets: KIND: ParquetReader # Defines the reader for target data path: data/targets.parquet # Path to the target dataset ``` -------------------------------- ### Nested MLflow Runs for Hyperparameter Tuning (Python) Source: https://mlops-coding-course.fmind.dev/5.%20Refining/5.5 Illustrates the use of nested runs in MLflow for organizing complex experiments, such as hyperparameter tuning. Each child run, started with `nested=True`, logs specific parameters and metrics, allowing for structured exploration of different configurations. Requires MLflow to be installed. ```python import mlflow params = [0.01, 0.02, 0.03] for p in params: with mlflow.start_run(run_name=f"alpha_{p}") as child_run: mlflow.log_param("alpha", p) # ... training logic ... # Assume val_loss is calculated during training val_loss = 0.1 # Example value mlflow.log_metric("val_loss", val_loss) ``` -------------------------------- ### Verify uv Installation Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.2 Checks the installed version of uv to confirm the installation was successful. This is a simple command-line check. ```shell uv --version ``` -------------------------------- ### Execute Entrypoint During Development with uv Source: https://mlops-coding-course.fmind.dev/3.%20Productionizing/3.3 This shell command demonstrates how to execute a package's entrypoint directly from the source code during development using `uv run`. This bypasses the need for building and installing the package, making the development cycle faster. It shows how to pass arguments to the entrypoint. ```shell # The 'uv run' command executes an entrypoint from the current project $ uv run bikes config.yml --extras key=value ``` -------------------------------- ### Install Poetry using pipx Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/archives/1.3 Commands to install pipx and then use it to install Poetry. pipx installs packages in isolated environments, preventing dependency conflicts. ```bash # Install pipx on your system python -m pip install pipx # install poetry using pipx pipx install poetry ``` -------------------------------- ### Start and Log MLflow Run Source: https://mlops-coding-course.fmind.dev/5.%20Refining/5.5 Python code demonstrating how to start a new MLflow run within a context manager. This allows for the addition of descriptive metadata, logging of system metrics, and provides the run ID for reference. The actual model training code would be placed inside this context. ```python with mlflow.start_run( run_name="Forecast Model with Feature Engineering", description="Training a model with an enhanced feature set.", log_system_metrics=True, ) as run: # Your model training and evaluation code goes here print(f"MLflow Run ID: {run.info.run_id}") ``` -------------------------------- ### Initialize Cookiecutter Template Package Source: https://mlops-coding-course.fmind.dev/6.%20Sharing/6.4 Command to initialize a new project from a specified Cookiecutter template hosted on GitHub. ```bash cookiecutter gh:fmind/cookiecutter-mlops-package ``` -------------------------------- ### Initialize and Commit Git Repository Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/1.5 This snippet demonstrates the basic Git commands for initializing a local repository, adding files, committing changes, and pushing them to a remote repository. It's crucial for version control in any MLOps project. ```bash git clone [Your-GitHub-Repository-URL] cd [repository-name] git add . git commit -m "feat: Initial project structure and data exploration" git push origin main ``` -------------------------------- ### Installing Production Dependencies with Poetry Source: https://mlops-coding-course.fmind.dev/1.%20Initializing/archives/1.3 Command to install only the main (production) dependencies for a project managed by Poetry. This is useful for deployment to ensure only essential packages are installed. ```bash poetry install --only main ```