### Dockerfile for MPI Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/kfmpi_plugin/README.md

A Dockerfile example that includes necessary installations for MPI and Horovod, essential for distributed training. Lines 40-51 and 66 are emphasized for MPI/Horovod setup.

```docker
FROM ubuntu:20.04

RUN apt-get update && apt-get install -y --no-install-recommends 
    build-essential 
    git 
    wget 
    python3 
    python3-pip 
    && rm -rf /var/lib/apt/lists/*

RUN pip3 install --no-cache-dir --upgrade pip
RUN pip3 install --no-cache-dir tensorflow
RUN pip3 install --no-cache-dir horovod
RUN pip3 install --no-cache-dir flytekit
RUN pip3 install --no-cache-dir flytekitplugins-kfmpi

# Example: Copying your training script
# COPY mpi_mnist.py /app/mpi_mnist.py

# CMD ["pyflyte-execute", "--inputs", "/app/inputs.json", "--output", "/app/outputs.json", "--", "mpi_mnist.py", "horovod_training_wf"]

# Example for running a specific script
# ENTRYPOINT ["python3", "/app/mpi_mnist.py"]

# Example for running a Flyte workflow
# ENTRYPOINT ["pyflyte", "run", "--remote", "mpi_mnist.py", "horovod_training_wf"]

# Example for building a container image with MPI and Horovod installed
# This Dockerfile is a simplified example and might need adjustments based on specific requirements.
# Ensure that the necessary MPI libraries and Horovod are correctly installed.
# For more complex setups, consider using pre-built MPI images or specific Horovod installation guides.

# Lines 40-51, 66 are emphasized in the original document for MPI/Horovod setup.
# These lines would typically contain commands like:
# RUN apt-get update && apt-get install -y openmpi-bin openmpi-common libopenmpi-dev
# RUN pip install horovod[keras,tensorflow,torch]
# RUN apt-get install -y --no-install-recommends 
#     libsm6 
#     libxext6 
#     libxrender1 
#     xvfb 
#     && rm -rf /var/lib/apt/lists/*

# Example of a command that might be emphasized:
# RUN apt-get update && apt-get install -y openmpi-bin openmpi-common libopenmpi-dev && rm -rf /var/lib/apt/lists/*
# RUN pip install --no-cache-dir horovod[tensorflow]

```

--------------------------------

### Flyte Example Project Structure

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/README.md

Demonstrates the typical file and directory structure for a Flyte example project. This includes configuration files, Dockerfile for packaging, and Python scripts containing the Flyte workflows and tasks.

```bash
example_project
├── README.md  # High-level description of the example project
├── Dockerfile  # Dockerfile for packaging up the project requirements
├── requirements.in  # Minimal python requirements for the project
├── requirements.txt  # Compiled python requirements using pip-compile
└── example_project  # Python package containing examples with the same name as the project
    ├── __init__.py
    ├── example_01.py
    ├── example_02.py
    ├── ...
    └── example_n.py
```

--------------------------------

### Flytelab Examples

Source: https://github.com/flyteorg/flytesnacks/blob/master/docs/tutorials/index.md

Examples from the Flytelab repository, including building an online weather forecasting application.

```python
from flytekit.sdk.types import Types
from flytekit.configuration import common
from flytekit.sdk.workflow import workflow, Input, Output
from flytekit.sdk.tasks import python_task, kwtypes

@python_task(
    inputs=kwtypes(name=str),
    outputs=kwtypes(output=str),
)
def greet(name: str) -> str:
    return "Hello " + name

@workflow
def greeting_workflow(name: str = "World") -> str:
    return greet(name=name)

```

--------------------------------

### PERIAN Flyte Agent Setup

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/perian_agent/README.md

This section provides guidance on setting up the PERIAN Flyte Agent. It directs users to the official PERIAN Flyte Agent setup guide for detailed instructions.

```APIDOC
PERIAN Flyte Agent Setup Guide:
  Refer to the official PERIAN documentation for detailed setup instructions.
  URL: https://perian.io/docs/flyte-setup-guide
```

--------------------------------

### Dockerfile for DBT Example

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/dbt_plugin/README.md

This Dockerfile defines the environment for the Flytekit dbt example. It installs Python dependencies like flytekitplugins-dbt and dbt-postgres, includes the jaffle-shop dbt project, and sets up a PostgreSQL database.

```docker
FROM ghcr.io/flyteorg/flytecookbook:core

RUN apt-get update && apt-get install -y --no-install-recommends
    # Install dbt and postgres adapter
    pip install "flytekitplugins-dbt[postgres]"

# Copy the dbt project into the image
COPY ./dbt_project /dbt_project

# Set the working directory
WORKDIR /dbt_project

# Start postgres
# This is a simplified example; in a real scenario, you'd manage the DB lifecycle more robustly.
# For the purpose of this example, we assume postgres is available externally or managed by flytectl demo.

# Define the entrypoint or command if needed, though Flyte handles task execution.
# CMD ["bash"]
```

--------------------------------

### Run Spark Examples

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/k8s_spark_plugin/README.md

Commands to execute Spark examples on a Flyte cluster using the `pyflyte` CLI. Demonstrates how to run PySpark Pi and DataFrame passing examples remotely.

```bash
pyflyte run --remote pyspark_pi.py my_spark
```

```bash
pyflyte run --remote dataframe_passing.py \
  my_smart_structured_dataset
```

--------------------------------

### Pandera Quick Start Example

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/pandera_plugin/README.md

Demonstrates a basic Pandera schema definition and its application in a Python function processed by Flytekit. It includes defining column types, constraints, and custom checks, along with an example of how invalid data triggers informative errors.

```python
import pandas as pd
import pandera as pa
from pandera.typing import DataFrame, Series

class Schema(pa.SchemaModel):
    column_1: Series[int] = pa.Field(ge=0)
    column_2: Series[float] = pa.Field(gt=0, lt=100)
    column_3: Series[str] = pa.Field(str_startswith="prefix")

    @pa.check("column_3")
    def check_str_length(cls, series):
        return series.str.len() > 5

@pa.check_types
def processing_fn(df: DataFrame[Schema]) -> DataFrame[Schema]:
    df["column_1"] = df["column_1"] * 2
    df["column_2"] = df["column_2"] * 0.5
    df["column_3"] = df["column_3"] + "_suffix"
    return df

raw_df = pd.DataFrame({
   "column_1": [1, 2, 3],
   "column_2": [1.5, 2.21, 3.9],
   "column_3": ["prefix_a", "prefix_b", "prefix_c"],
})
processed_df = processing_fn(raw_df)
print(processed_df)
```

--------------------------------

### Run File Sensor Example

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/sensor/README.md

This command demonstrates how to execute the file sensor example on a remote Flyte cluster. It requires the Flyte CLI to be installed and configured.

```bash
pyflyte run --remote \
  https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/sensor/sensor/file_sensor_example.py wf
```

--------------------------------

### Run TensorFlow Example

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/kftensorflow_plugin/README.md

Executes a distributed TensorFlow training example on a Flyte cluster. This command assumes the TensorFlow plugin is installed and configured.

```bash
pyflyte run --remote tf_mnist.py \
  mnist_tensorflow_workflow
```

--------------------------------

### Install Flytekit Pandera Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/pandera_plugin/README.md

Installs the necessary plugin to enable Pandera integration with Flytekit.

```bash
pip install flytekitplugins-pandera
```

--------------------------------

### Install DBT Postgres Adapter

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/dbt_plugin/README.md

Installs the dbt-postgres adapter along with dbt-core. This is an example for connecting to a PostgreSQL database; other adapters like dbt-redshift, dbt-snowflake, or dbt-bigquery may be needed depending on the database.

```bash
pip install dbt-postgres
```

--------------------------------

### Install Neptune Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/neptune_plugin/README.md

Command to install the Flyte Neptune plugin using pip.

```bash
pip install flytekitplugins-neptune
```

--------------------------------

### Install Dolt Plugin and CLI

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/dolt_plugin/README.md

Installs the Flyte Dolt plugin using pip and the Dolt command-line tool from the latest release.

```bash
pip install flytekitplugins.dolt
sudo bash -c 'curl -L https://github.com/dolthub/dolt/releases/latest/download/install.sh | sudo bash'
```

--------------------------------

### Install Papermill Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/exploratory_data_analysis/README.md

Installs the Papermill plugin for Flyte, enabling the use of Jupyter Notebooks within Flyte pipelines.

```bash
pip install flytekitplugins-papermill
```

--------------------------------

### Install MLFlow Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/mlflow_plugin/README.md

Installs the Flyte MLFlow plugin using pip. This is the first step to enable MLFlow integration.

```bash
pip install flytekitplugins-mlflow
```

--------------------------------

### Install whylogs Flyte Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/whylogs_plugin/README.md

Installs the whylogs plugin for Flyte using pip, making the whylogs integration available in your environment.

```bash
pip install flytekitplugins-whylogs
```

--------------------------------

### Install Comet ML Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/comet_ml_plugin/README.md

Installs the necessary Flyte plugin for Comet ML integration using pip.

```bash
pip install flytekitplugins-comet-ml
```

--------------------------------

### Feature Engineering Examples

Source: https://github.com/flyteorg/flytesnacks/blob/master/docs/tutorials/index.md

Examples demonstrating data cleaning, feature engineering, and data serving using tools like Papermill and Feast within Flyte.

```python
from flytekit.sdk.types import Types
from flytekit.configuration import common
from flytekit.sdk.workflow import workflow, Input, Output
from flytekit.sdk.tasks import python_task, kwtypes

@python_task(
    inputs=kwtypes(name=str),
    outputs=kwtypes(output=str),
)
def greet(name: str) -> str:
    return "Hello " + name

@workflow
def greeting_workflow(name: str = "World") -> str:
    return greet(name=name)

```

--------------------------------

### Install Flyte Ray Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/ray_plugin/README.md

Installs the FlyteKit Ray plugin using pip. This is the first step to enable Ray integration within Flyte.

```bash
pip install flytekitplugins-ray
```

--------------------------------

### Install Spark Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/k8s_spark_plugin/README.md

Installs the necessary Flyte Spark plugin package using pip. This command should be run in your Flyte environment.

```bash
pip install flytekitplugins-spark
```

--------------------------------

### Install Dask Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/k8s_dask_plugin/README.md

Installs the Flyte Dask plugin using pip. This is the first step to enable Dask integration with Flyte.

```shell
pip install flytekitplugins-dask
```

--------------------------------

### Install Ollama Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/ollama_plugin/README.md

This command installs the necessary Flyte plugin for Ollama integration, enabling the serving of LLMs within Flyte tasks.

```bash
pip install flytekitplugins-inference
```

--------------------------------

### Install Memray Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/memray_plugin/README.md

Installs the Memray plugin for Flyte using pip. This is the first step to enable memory tracking for your Flyte tasks.

```bash
pip install flytekitplugins-memray
```

--------------------------------

### Install FlyteInteractive Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/flyteinteractive_plugin/README.md

Installs the FlyteInteractive plugin using pip. This plugin is necessary to enable interactive task development features within Flyte.

```bash
pip install flytekitplugins-flyteinteractive
```

--------------------------------

### Basic Flyte Concepts

Source: https://github.com/flyteorg/flytesnacks/blob/master/flyte_tests.txt

Illustrates fundamental concepts in Flyte, including documenting workflows, a simple 'hello world' example, defining named outputs, creating shell tasks, and basic workflow structure.

```python
from flytekit import task, workflow

@task
def greet(name: str) -> str:
    """This task greets the given name."""
    return f"Hello {name}"

@workflow
def greeting_workflow(name: str) -> str:
    """This workflow calls the greet task."""
    return greet(name=name)
```

```python
from flytekit import task, workflow

@task
def hello_world_task() -> str:
    return "Hello, World!"

@workflow
def hello_world_workflow() -> str:
    return hello_world_task()
```

```python
from flytekit import task, workflow

@task
def task_with_named_outputs(x: int, y: int) -> (int, int):
    return x + y, x * y

@workflow
def named_outputs_workflow(a: int, b: int) -> int:
    sum_val, prod_val = task_with_named_outputs(x=a, y=b)
    return sum_val
```

```python
from flytekit import task, workflow

@task
def shell_task_example() -> str:
    return "echo 'Hello from shell'"

@workflow
def shell_workflow() -> str:
    return shell_task_example()
```

```python
from flytekit import task, workflow

@task
def add_task(x: int, y: int) -> int:
    return x + y

@workflow
def simple_workflow(a: int, b: int) -> int:
    return add_task(x=a, y=b)
```

--------------------------------

### Dockerfile for BLASTX

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/blast/README.md

This Dockerfile is used to set up the environment for running BLASTX searches. It includes instructions for installing necessary software and dependencies.

```docker
FROM ubuntu:20.04

RUN apt-get update && apt-get install -y --no-install-recommends \
    wget \
    build-essential \
    git \
    && rm -rf /var/lib/apt/lists/*

# Install BLAST+
RUN wget https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.13.0+-x64-linux.tar.gz \
    && tar -xzf ncbi-blast-2.13.0+-x64-linux.tar.gz \
    && mv ncbi-blast-2.13.0+/bin/* /usr/local/bin/ \
    && rm -rf ncbi-blast-2.13.0+ ncbi-blast-2.13.0+-x64-linux.tar.gz

# Install Biopython
RUN pip install biopython

# Copy Flytekit and the blastx example code
COPY ./flytekit /flytekit
COPY ./examples/blast /blast

WORKDIR /blast

# Set up the BLAST database (example: downloading a small database)
# In a real scenario, you would download a comprehensive database.
# RUN update-ncbi-ncbi-acc --download-blast-db all

# Example command to create a dummy database for testing if needed
# RUN echo -e ">seq1\nACGTACGT"
# RUN echo -e ">seq2\nTTGCCTGG"
# RUN makeblastdb -in dummy_sequences.fasta -dbtype nucl -out dummy_db

# Set environment variables if needed
# ENV BLASTDB=/path/to/your/blast/databases

CMD ["bash"]
```

--------------------------------

### Model Training Examples

Source: https://github.com/flyteorg/flytesnacks/blob/master/docs/tutorials/index.md

This section covers various machine learning model training examples using different frameworks and datasets. It includes classification, regression, and NLP tasks.

```python
from flytekit.sdk.types import Types
from flytekit.configuration import common
from flytekit.sdk.workflow import workflow, Input, Output
from flytekit.sdk.tasks import python_task, kwtypes

@python_task(
    inputs=kwtypes(name=str),
    outputs=kwtypes(output=str),
)
def greet(name: str) -> str:
    return "Hello " + name

@workflow
def greeting_workflow(name: str = "World") -> str:
    return greet(name=name)

```

--------------------------------

### Pull Example Docker Image

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/dbt_plugin/README.md

Pulls a pre-built Docker image containing the Flytekit dbt example, including necessary dependencies and the jaffle-shop dbt project.

```bash
docker pull ghcr.io/flyteorg/flytecookbook:dbt_example-latest
```

--------------------------------

### Clone Flytesnacks Repo and Run Example

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/dbt_plugin/README.md

Clones the flytesnacks repository and runs the dbt example directly from the local repository. This is an alternative method to running the pre-built Docker image.

```bash
git clone https://github.com/flyteorg/flytesnacks
cd flytesnacks/examples/dbt_example
pyflyte run --remote \
    --image ghcr.io/flyteorg/flytecookbook:dbt_example-latest \
    dbt_plugin/dbt_example.py wf
```

--------------------------------

### Install Flytekit

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/basics/basics/basic_interactive_mode.ipynb

Upgrades the flytekit version to 1.14.0 or higher. This is a prerequisite for using the interactive mode features described in this guide.

```bash
!pip install flytekit==1.14.0
```

--------------------------------

### Install BigQuery Agent

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/bigquery_agent/README.md

Command to install the BigQuery agent plugin for Flytekit. This is a prerequisite for using the agent.

```bash
pip install flytekitplugins-bigquery
```

--------------------------------

### Data Types and IO Examples

Source: https://github.com/flyteorg/flytesnacks/blob/master/flyte_tests.txt

Demonstrates how to use various data types and handle input/output operations in Flyte, including dataclasses, enums, files, folders, and structured datasets.

```python
from flytekit import task, workflow
from dataclasses import dataclass

@dataclass
class MyData:
    name: str
    value: int

@task
def process_dataclass(data: MyData) -> str:
    return f"Processed: {data.name} with value {data.value}"

@workflow
def dataclass_workflow() -> str:
    my_data = MyData(name="example", value=123)
    return process_dataclass(data=my_data)
```

```python
from flytekit import task, workflow
from enum import Enum

class Status(Enum):
    PENDING = "pending"
    SUCCESS = "success"
    FAILURE = "failure"

@task
def process_enum(status: Status) -> str:
    return f"Current status: {status.value}"

@workflow
def enum_workflow() -> str:
    return process_enum(status=Status.SUCCESS)
```

```python
from flytekit import task, workflow
from flytekit.types.file import FlyteFile

@task
def create_file() -> FlyteFile:
    with open("my_output_file.txt", "w") as f:
        f.write("This is the file content.")
    return FlyteFile("my_output_file.txt")

@task
def read_file(file: FlyteFile) -> str:
    with open(file.path, "r") as f:
        content = f.read()
    return content

@workflow
def file_workflow() -> str:
    created_file = create_file()
    return read_file(file=created_file)
```

```python
from flytekit import task, workflow
from flytekit.types.directory import FlyteDirectory
import os

@task
def create_directory() -> FlyteDirectory:
    os.makedirs("my_output_directory", exist_ok=True)
    with open("my_output_directory/file1.txt", "w") as f:
        f.write("content1")
    with open("my_output_directory/file2.txt", "w") as f:
        f.write("content2")
    return FlyteDirectory("my_output_directory")

@task
def list_directory(directory: FlyteDirectory) -> list:
    return os.listdir(directory.path)

@workflow
def directory_workflow() -> list:
    created_dir = create_directory()
    return list_directory(directory=created_dir)
```

```python
from flytekit import task, workflow
from flytekit.types.schema import FlyteSchema
from typing import Dict, List

@task
def create_structured_dataset() -> FlyteSchema:
    data = [
        {"col1": 1, "col2": "a"},
        {"col1": 2, "col2": "b"},
    ]
    return FlyteSchema.from_dataframe(pd.DataFrame(data))

@task
def process_structured_dataset(schema: FlyteSchema) -> Dict[str, List]:
    df = schema.open_python_dataframe()
    return df.to_dict(orient='list')

@workflow
def structured_dataset_workflow() -> Dict[str, List]:
    created_schema = create_structured_dataset()
    return process_structured_dataset(schema=created_schema)
```

--------------------------------

### Initialize Dolt Repository

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/dolt_plugin/README.md

Creates a new Dolt repository in the current directory, which is a prerequisite for using the DoltTable plugin.

```bash
mkdir foo
cd foo
dolt init
```

--------------------------------

### Install Slurm Agent

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/slurm_agent/README.md

Command to install the Slurm agent plugin for Flyte. This enables integration with Slurm clusters.

```bash
pip install flytekitplugins-slurm
```

--------------------------------

### Install MPI Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/kfmpi_plugin/README.md

Installs the Flyte MPI plugin using pip. This is a prerequisite for using the MPI operator.

```bash
pip install flytekitplugins-kfmpi
```

--------------------------------

### Install TensorFlow Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/kftensorflow_plugin/README.md

Installs the Kubeflow TensorFlow plugin for Flyte. This is a prerequisite for running distributed TensorFlow jobs.

```bash
pip install flytekitplugins-kftensorflow
```

--------------------------------

### Sphinx-Gallery Prompt Example

Source: https://github.com/flyteorg/flytesnacks/blob/master/docs/README.md

Demonstrates how to format command-line prompts within Sphinx-Gallery documentation using the 'prompt' directive.

```rst
.. prompt::bash\n\n   flytectl --version
```

--------------------------------

### Install Spark Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/databricks_agent/README.md

Installs the Spark plugin for Flyte, which includes the Databricks agent. This is a prerequisite for using the Databricks integration.

```bash
pip install flytekitplugins-spark
```

--------------------------------

### Build FlyteSnacks Documentation

Source: https://github.com/flyteorg/flytesnacks/blob/master/docs/README.md

This snippet shows the command to build the documentation for FlyteSnacks. It requires installing development dependencies first.

```bash
make html
```

--------------------------------

### Start Flyte Demo Cluster

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/dbt_plugin/README.md

Starts a local Flyte demo cluster using the flytectl command-line tool. This is often used for local development and testing of Flyte workflows.

```bash
flytectl demo start
```

--------------------------------

### Install OpenAI Batch Agent

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/openai_batch_agent/README.md

Command to install the OpenAI Batch agent plugin for Flytekit. This is a prerequisite for using the agent.

```bash
pip install flytekitplugins-openai
```

--------------------------------

### Bioinformatics Examples

Source: https://github.com/flyteorg/flytesnacks/blob/master/docs/tutorials/index.md

Tutorials on performing computational biology tasks with Flyte, such as nucleotide sequence querying using BLASTX.

```python
from flytekit.sdk.types import Types
from flytekit.configuration import common
from flytekit.sdk.workflow import workflow, Input, Output
from flytekit.sdk.tasks import python_task, kwtypes

@python_task(
    inputs=kwtypes(name=str),
    outputs=kwtypes(output=str),
)
def greet(name: str) -> str:
    return "Hello " + name

@workflow
def greeting_workflow(name: str = "World") -> str:
    return greet(name=name)

```

--------------------------------

### Install Kubeflow PyTorch Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/kfpytorch_plugin/README.md

Installs the Flyte plugin for Kubeflow PyTorch training. This is a prerequisite for using the distributed training capabilities.

```bash
pip install flytekitplugins-kfpytorch
```

--------------------------------

### Install Flytekit DuckDB Plugin

Source: https://github.com/flyteorg/flytesnacks/blob/master/examples/duckdb_plugin/README.md

Command to install the Flytekit DuckDB plugin, enabling DuckDB integration within Flyte workflows.

```bash
pip install flytekitplugins-duckdb
```