### Build and Run Kedro-Viz in Docker

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-docker/README.md

Steps to download, build, and run Kedro-Viz within a Docker container for pipeline visualization. This includes installing Kedro-Viz and starting the viz server.

```bash
pip download -d data --no-deps kedro-viz
kedro docker build
kedro docker cmd bash --docker-args="-it -u=0 -p=4141:4141"
pip install data/*.whl
kedro viz --host=0.0.0.0 --no-browser
```

--------------------------------

### Install kedro-datasets

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/README.md

Install the kedro-datasets package using pip.

```bash
pip install kedro-datasets
```

--------------------------------

### Install kedro-datasets[langfuse-promptdataset]

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Command to install the SDK mode for basic Langfuse integration without LangChain dependencies.

```bash
pip install "kedro-datasets[langfuse-promptdataset]"
```

--------------------------------

### Install Kedro Datasets with Langfuse Support

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Install the full package for all Langfuse datasets or specific extras for TraceDataset. The full installation includes support for OpenAI and LangChain.

```bash
pip install "kedro-datasets[langfuse]"  # Full installation
```

--------------------------------

### Install kedro-datasets[langfuse]

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Command to install the full package for complete functionality, including LangChain integration.

```bash
pip install "kedro-datasets[langfuse]"
```

--------------------------------

### Install Airspeed Velocity

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_benchmarks/README.md

Installs the airspeed velocity (asv) benchmarking tool. This is a prerequisite for running the benchmarks.

```bash
pip install asv
```

--------------------------------

### Install Opik TraceDataset

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Install the Opik TraceDataset with optional dependencies for AutoGen or all Opik datasets.

```bash
pip install "kedro-datasets[opik-tracedataset]"
```

```bash
pip install "kedro-datasets[opik-tracedataset-autogen]"
```

```bash
pip install "kedro-datasets[opik]"
```

--------------------------------

### Install Documentation Dependencies

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/docs/readme.md

Installs all required packages for building and serving the documentation locally. Ensure you have an active Conda environment.

```bash
pip install -e ".[docs]"
```

--------------------------------

### Install Kedro Datasets with MLRun

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/mlrun/README.md

Install the kedro-datasets package with MLRun support using pip.

```bash
pip install "kedro-datasets[mlrun]"
```

--------------------------------

### Run Documentation Locally

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/docs/readme.md

Starts the MkDocs development server to view the documentation in your browser. Access it at http://127.0.0.1:8000/pages/.

```bash
mkdocs serve
```

--------------------------------

### Install Langfuse EvaluationDataset

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Install the necessary package for the Langfuse EvaluationDataset. Can be installed individually or as part of the full Langfuse integration.

```bash
pip install "kedro-datasets[langfuse-evaluationdataset]"
```

```bash
pip install "kedro-datasets[langfuse]"
```

--------------------------------

### Opik Client Initialization with Credentials

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Provides an example of how to configure credentials for initializing the Opik client. This is crucial for authentication and accessing Opik resources.

```python
credentials = {
    "api_key": "opik_...",  # pragma: allowlist secret
    "workspace": "your-workspace",
}

```

--------------------------------

### Install kedro-datasets with group dependencies

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/README.md

Install kedro-datasets along with dependencies for a specific dataset group, such as 'pandas'.

```bash
pip install "kedro-datasets[<group>]"
```

--------------------------------

### Install Test Requirements and Pre-commit Hooks

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/CONTRIBUTING.md

Installs the necessary requirements for running tests and sets up pre-commit hooks for the kedro-datasets plugin.

```bash
make plugin=kedro-datasets install-test-requirements
make install-pre-commit
```

--------------------------------

### Install Kedro-Docker

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-docker/README.md

Install the Kedro-Docker plugin using pip. This command should be run in your Python environment.

```bash
pip install kedro-docker
```

--------------------------------

### Install Kedro Datasets with Opik Support

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Shows the command to install the necessary packages for Kedro Datasets, including Opik support. This ensures all required dependencies are met for using Opik features.

```bash
pip install "kedro-datasets[opik]"  # Full installation

```

--------------------------------

### Spark Configuration Example

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets/spark/README.md

Configure Spark settings for Kafka integration by adding relevant properties to `conf/base/spark.yml`. This example shows driver max result size and scheduler mode.

```yaml
spark.driver.maxResultSize: 3g
spark.scheduler.mode: FAIR
```

--------------------------------

### Quick Start: MLRun Datasets in Python

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/mlrun/README.md

Demonstrates saving and loading ML models, DataFrames, and logging metrics using MLRun datasets in Python. Ensure 'trained_model' and 'df' are defined before use.

```python
from kedro_datasets_experimental.mlrun import MLRunModel, MLRunDataframeDataset, MLRunResult

# Save and load ML models
model_dataset = MLRunModel(
    key="trained_model",
    framework="sklearn",
    model_format="pkl",
)
model_dataset.save(trained_model)
loaded_model = model_dataset.load()

# Save and load DataFrames
df_dataset = MLRunDataframeDataset(key="processed_data")
df_dataset.save(df)
loaded_df = df_dataset.load()

# Log metrics and results
result_dataset = MLRunResult(key="metrics", flatten=True)
result_dataset.save({"accuracy": 0.95, "loss": 0.05})
```

--------------------------------

### Install Test Requirements

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-airflow/CONTRIBUTING.md

Installs test requirements for the kedro-airflow plugin, including 'behave'.

```bash
make plugin=kedro-airflow install-test-requirements
make install-pre-commit
```

--------------------------------

### Install Kedro Datasets with Opik Support

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Installs the kedro-datasets package with the necessary extras for Opik evaluation datasets. This command ensures all dependencies for using the EvaluationDataset are met.

```bash
pip install "kedro-datasets[opik-evaluationdataset]"
```

--------------------------------

### Install TraceDataset with AutoGen Dependencies

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

For AutoGen mode, install with OpenTelemetry dependencies to enable tracing of AutoGen agent conversations.

```bash
pip install "kedro-datasets[langfuse-tracedataset-autogen]"
```

--------------------------------

### Install pandas ExcelDataset dependencies

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/README.md

Install kedro-datasets and the specific dependencies for the pandas ExcelDataset.

```bash
pip install "kedro-datasets[pandas-exceldataset]"
```

--------------------------------

### YAML Prompt Format Example

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Example of a prompt structured in YAML format, an alternative to JSON for AI prompts. Supports role-based messaging.

```yaml
- role: system
  content: You are a helpful assistant.
- role: user
  content: "Hello, {question}"
```

--------------------------------

### Run Kedro-Datasets Benchmarks

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_benchmarks/README.md

Executes the installed airspeed velocity benchmarks. Ensure you are in the correct directory before running.

```bash
asv run
```

--------------------------------

### Install kedro-datasets with specific dataset dependencies

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/README.md

Install kedro-datasets with dependencies for a particular dataset type within a group, like 'pandas-exceldataset'.

```bash
pip install "kedro-datasets[<group>-<dataset>]"
```

--------------------------------

### Install Optional Dependencies for ParquetDataset

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/RELEASE.md

Install optional dependencies for ParquetDataset using the new PEP 685 compliant naming convention.

```bash
pip install kedro-datasets[pandas-parquetdataset]
```

--------------------------------

### Install Kedro-Airflow Plugin

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-airflow/README.md

Install the kedro-airflow plugin using pip. This is the first step to enable Kedro pipeline deployment to Airflow.

```bash
pip install kedro-airflow
```

--------------------------------

### Code Block Component Example

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/docs/readme.md

Shows a basic Python function example within a code block.

```python
def hello_world():
    print("Hello, world!")
```

```python
def hello_world():
    print("Hello, world!")
```

--------------------------------

### Note Component Example

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/docs/readme.md

Demonstrates the usage of a note block for highlighting important information within the documentation.

```text
> **Note**
> This is a note block to highlight important information.
```

```bash
> This is a note block to highlight important information.
```

--------------------------------

### JSON Chat Prompt Example

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Example of a chat prompt in JSON format, defining system and human messages with a placeholder.

```json
[
  {
    "role": "system",
    "content": "You are a helpful insurance support assistant."
  },
  {
    "role": "human",
    "content": "{user_query}"
  }
]
```

--------------------------------

### Install Test Requirements for Kedro Docker

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-docker/CONTRIBUTING.md

Installs the necessary test requirements, including 'behave', for running end-to-end tests for the kedro-docker plugin.

```bash
make plugin=kedro-docker install-test-requirements
```

```bash
make install-pre-commit
```

--------------------------------

### Quick Start: Initialize TraceDataset for OpenAI Mode

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Initialize the TraceDataset with credentials for OpenAI mode and load the wrapped client. API calls made with this client will be automatically traced.

```python
from kedro_datasets_experimental.langfuse import TraceDataset

dataset = TraceDataset(
    credentials={
        "public_key": "pk_...",
        "secret_key": "sk_...",  # pragma: allowlist secret
        "openai": {"api_key": "sk-..."},  # pragma: allowlist secret
    },
    mode="openai",
)

client = dataset.load()
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}],
)
# The call is automatically traced in Langfuse
```

--------------------------------

### Catalog Configuration for Local Sync

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Example Kedro catalog configuration for PromptDataset using the 'local' sync policy for development.

```yaml
intent_prompt:
  type: kedro_datasets_experimental.langfuse.PromptDataset
  filepath: data/prompts/intent.json
  prompt_name: "intent-classifier"
  prompt_type: "chat"
  credentials: langfuse_credentials
  sync_policy: local  # Local files are source of truth
  mode: langchain
  save_args:
    labels: ["development", "v2.1"]
```

--------------------------------

### Start IPython Session in Docker

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-docker/README.md

Launches an IPython session within the Docker container, allowing for interactive development and debugging of the Kedro project.

```bash
kedro docker ipython
```

--------------------------------

### Type Hinting Example in Python

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/CONTRIBUTING.md

Demonstrates the use of PEP 484 type hints for user-facing functions and class methods in Python code.

```python
def count_truthy(elements: List[Any]) -> int:
    return sum(1 for elem in elements if elem)
```

--------------------------------

### After: Opik Catalog Entry

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Example of a Kedro catalog entry updated for Opik's EvaluationDataset.

```yaml
evaluation_dataset:
  type: kedro_datasets_experimental.opik.EvaluationDataset
  dataset_name: intent-detection-eval
  filepath: data/evaluation/intent_items.json
  sync_policy: local
  credentials: opik_credentials

# conf/local/credentials.yml
opik_credentials:
  api_key: "opik_..."  # pragma: allowlist secret
```

--------------------------------

### YAML Chat Prompt Format

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Example of a chat prompt structure in YAML format, including system and human roles.

```yaml
- role: system
  content: You are an expert analyst.
- role: human
  content: "{query}"
```

--------------------------------

### JSON Text Prompt Format

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Example of a simple text prompt template in JSON format with a variable placeholder.

```json
"Classify the following text as positive, negative, or neutral: {input}"
```

--------------------------------

### Advanced PromptDataset Configuration

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Shows a full configuration for PromptDataset, including custom host, sync policy, mode, and arguments for loading and saving. Useful for complex setups.

```python
# Full configuration with custom host
dataset = PromptDataset(
    filepath="prompts/support.yaml",
    prompt_name="customer-support",
    prompt_type="chat",
    sync_policy="remote",
    mode="langchain",
    credentials={
        "public_key": "pk_...",
        "secret_key": "sk_...",  # pragma: allowlist secret
        "host": "https://your-langfuse.com",
    },
    load_args={"label": "staging"},
    save_args={"labels": ["staging", "v1.2"]},
)
```

--------------------------------

### MLRun Model YAML Catalog Configuration

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/mlrun/README.md

Example of configuring an MLRunModel dataset in the Kedro catalog for scikit-learn models.

```yaml
trained_model:
  type: kedro_datasets_experimental.mlrun.MLRunModel
  key: rf_classifier
  framework: sklearn

evaluation_metrics:
  type: kedro_datasets_experimental.mlrun.MLRunResult
  key: eval_metrics
  flatten: true
```

--------------------------------

### Credentials Management Configuration

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Example of how to configure Opik credentials in Kedro's conf/local/credentials.yml. Ensure sensitive information is not committed to version control.

```yaml
# conf/local/credentials.yml
# Store securely - not in version control
opik_credentials:
  api_key: "opik_your_api_key"  # pragma: allowlist secret
  workspace: "your-workspace"
  project_name: "your-project"  # optional
```

--------------------------------

### Credentials Management for Opik TraceDataset

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Example of how to configure credentials for the Opik TraceDataset, including API keys and endpoints for different modes.

```yaml
# conf/local/credentials.yml
opik_credentials:
  api_key: "opik_your_api_key"  # pragma: allowlist secret
  workspace: "your-workspace"
  project_name: "your-project"  # optional
  url_override: "https://your-opik-instance.com"  # optional, self-hosted only
  openai:  # required for mode: openai
    api_key: "sk-..."  # pragma: allowlist secret
    base_url: "https://api.openai.com/v1"  # optional
  endpoint: "https://www.comet.com/opik/api/v1/private/otel/v1/traces"  # required for mode: autogen
```

--------------------------------

### RAG Applications Use Case

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Loads a prompt template for Retrieval-Augmented Generation (RAG) and formats it with retrieved context and user question. This example demonstrates combining context with a query for synthesis.

```python
# Retrieval-Augmented Generation
rag_dataset = PromptDataset(
    filepath="prompts/rag.json",
    prompt_name="rag-synthesizer",
    prompt_type="chat",
    sync_policy="remote",
    load_args={"label": "production"},
)

# Combine retrieved context with user query
template = rag_dataset.load()
final_prompt = template.format(
    context="\n".join(retrieved_chunks),
    question=user_question,
    instructions="Be concise and cite sources.",
)
```

--------------------------------

### Basic PromptDataset Configuration

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Demonstrates the minimal configuration required to initialize a PromptDataset. Ensure Langfuse credentials are provided.

```python
from kedro_datasets_experimental.langfuse import PromptDataset

# Minimal configuration
dataset = PromptDataset(
    filepath="prompts/intent.json",
    prompt_name="intent-classifier",
    credentials={
        "public_key": "pk_...",
        "secret_key": "sk_...",  # pragma: allowlist secret
    },
)
```

--------------------------------

### Install pandas group dependencies

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/README.md

Install kedro-datasets and the dependencies required for the pandas dataset group.

```bash
pip install 'kedro-datasets[pandas]'
```

--------------------------------

### Load and Use PromptDataset in Python

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Demonstrates how to load a prompt using PromptDataset and access its content. Requires Opik credentials.

```python
from kedro_datasets_experimental.opik import PromptDataset

# Load and use a prompt
dataset = PromptDataset(
    filepath="prompts/customer_support.json",
    prompt_name="customer_support_v1",
    credentials={
        "api_key": "opik_...",  # pragma: allowlist secret
        "workspace": "my-workspace",
    },
)

# Returns Opik Prompt object
prompt = dataset.load()
```

--------------------------------

### Load and Evaluate Data with EvaluationDataset

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Demonstrates how to initialize EvaluationDataset, load data into an opik.Dataset, and run an evaluation using the opik.evaluate function. Ensure Opik credentials and filepath are correctly configured.

```python
from kedro_datasets_experimental.opik import EvaluationDataset
from opik.evaluation import evaluate

dataset = EvaluationDataset(
    dataset_name="intent-detection-eval",
    credentials={"api_key": "opik_..."},  # pragma: allowlist secret
    filepath="data/evaluation/intent_items.json",
)

# Load returns an opik.Dataset ready for experiments
eval_dataset = dataset.load()
evaluate(
    dataset=eval_dataset,
    task=my_task,
    scoring_functions=[my_scorer],
    experiment_name="my-experiment",
)
```

--------------------------------

### Initialize EvaluationDataset with Version

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Instantiate `EvaluationDataset` with a specific version for loading data. Ensure credentials are provided.

```python
dataset = EvaluationDataset(
    dataset_name="intent-detection-eval",
    credentials={
        "public_key": "pk_...",
        "secret_key": "sk_...",  # pragma: allowlist secret
    },
    sync_policy="remote",
    version="2026-01-15T00:00:00Z",
)

snapshot = dataset.load()
```

--------------------------------

### Catalog Configuration: Remote Sync Policy (Production)

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Configure the PromptDataset for production environments where Opik is the source of truth for prompts.

```yaml
production_prompt:
  type: kedro_datasets_experimental.opik.PromptDataset
  filepath: data/prompts/production.json
  prompt_name: customer_support_v1
  prompt_type: chat
  credentials: opik_credentials
  sync_policy: remote  # Opik is source of truth
  mode: sdk
```

--------------------------------

### Catalog Configuration: Local Sync Policy (Development)

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Configure the PromptDataset for local development where local files are the source of truth. Includes metadata for tracking.

```yaml
dev_prompt:
  type: kedro_datasets_experimental.opik.PromptDataset
  filepath: data/prompts/customer_support.json
  prompt_name: customer_support_v1
  prompt_type: chat
  credentials: opik_credentials
  sync_policy: local  # Local files are source of truth
  mode: langchain
  save_args:
    metadata:
      environment: development
      version: "2.1"
```

--------------------------------

### Python API: Advanced PromptDataset Configuration

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Instantiate a PromptDataset with full configuration, including sync policy, mode, and metadata for saving.

```python
# Full configuration with metadata
dataset = PromptDataset(
    filepath="prompts/assistant.yaml",
    prompt_name="assistant_v2",
    prompt_type="chat",
    sync_policy="remote",
    mode="langchain",
    credentials={
        "api_key": "opik_...",  # pragma: allowlist secret
        "workspace": "my-workspace",
        "project_name": "customer-support",
    },
    save_args={
        "metadata": {"environment": "production", "team": "ml-ops", "version": "2.0.0"}
    },
)
```

--------------------------------

### Run Kedro-Viz if Already Installed in Docker

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-docker/README.md

Execute Kedro-Viz directly within the Docker container if it's already installed. This command maps the necessary port for accessing the visualization.

```bash
kedro docker cmd --docker-args="-p=4141:4141" kedro viz --host=0.0.0.0
```

--------------------------------

### Start Jupyter Lab in Docker

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-docker/README.md

Starts a Jupyter Lab server inside the Docker container. Similar to Jupyter Notebook, the `--port` option can be used for port mapping.

```bash
kedro docker jupyter lab
```

--------------------------------

### Python API: Basic PromptDataset Usage

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Instantiate a PromptDataset with minimal configuration for basic usage.

```python
from kedro_datasets_experimental.opik import PromptDataset

# Minimal configuration
dataset = PromptDataset(
    filepath="prompts/support.json",
    prompt_name="support_assistant",
    credentials={
        "api_key": "opik_...",  # pragma: allowlist secret
        "workspace": "my-workspace",
    },
)
```

--------------------------------

### Start Jupyter Notebook in Docker

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-docker/README.md

Starts a Jupyter Notebook server inside the Docker container. The `--port` option maps a host port to the container's default port (8888).

```bash
kedro docker jupyter notebook
```

--------------------------------

### MLRunAbstractDataset YAML Catalog Configuration

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/mlrun/README.md

Example of how to configure MLRunAbstractDataset in the Kedro catalog.

```yaml
generic_artifact:
  type: kedro_datasets_experimental.mlrun.MLRunAbstractDataset
  key: my_artifact
```

--------------------------------

### Before: Langfuse Catalog Entry

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Example of a Kedro catalog entry for Langfuse's EvaluationDataset.

```yaml
evaluation_dataset:
  type: kedro_datasets_experimental.langfuse.EvaluationDataset
  dataset_name: intent-detection-eval
  filepath: data/evaluation/intent_items.json
  sync_policy: local
  credentials: langfuse_credentials

# conf/local/credentials.yml
langfuse_credentials:
  public_key: "pk_..."
  secret_key: "sk_..."  # pragma: allowlist secret
```

--------------------------------

### Troubleshooting: Missing OpenTelemetry Dependencies

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Solution for 'DatasetError: AutoGen mode requires OpenTelemetry.'. Install the necessary package.

```bash
pip install "kedro-datasets[opik-tracedataset-autogen]"
```

--------------------------------

### Load and Use PromptDataset

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Demonstrates loading a prompt from a JSON file using PromptDataset and accessing its content and version.

```python
from kedro_datasets_experimental.langfuse import PromptDataset

# Load and use a prompt
dataset = PromptDataset(
    filepath="prompts/intent.json",
    prompt_name="intent-classifier",
    credentials={
        "public_key": "pk_...",
        "secret_key": "sk_...",  # pragma: allowlist secret
    },
)

# Returns Langfuse prompt object
# langfuse.model.TextPromptClient in case of `prompt_type=text`
# langfuse.model.ChatPromptClient in case of `prompt_type=chat`
prompt = dataset.load()
```

--------------------------------

### Initialize TraceDataset for SDK Mode

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Instantiate TraceDataset for SDK mode to use the @client.track decorator for manual tracing. Provide Opik credentials and project name.

```python
dataset = TraceDataset(
    credentials={
        "api_key": "opik_...",  # pragma: allowlist secret
        "workspace": "my-workspace",
        "project_name": "kedro-sdk-demo",
    },
    mode="sdk",
)

client = dataset.load()


@client.track(name="my_workflow")
def run_step(x: int) -> int:
    return x * 2


run_step(21)  # Trace sent to Opik
```

--------------------------------

### Missing Credentials Error

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Example of a `DatasetError` that occurs when required Langfuse credentials, such as 'public_key', are missing during dataset initialization or operation.

```text
DatasetError: Missing required Langfuse credential: 'public_key'
```

--------------------------------

### SparkHiveDataset Initialization and Usage

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/docs/api/kedro_datasets/spark.SparkHiveDataset.md

This snippet outlines the primary methods and parameters for initializing and using the SparkHiveDataset. It demonstrates how to specify Hive table details and data catalog configuration.

```APIDOC
## SparkHiveDataset

`SparkHiveDataset` loads and saves data to Hive tables using Apache Spark.

### Parameters

- **catalog**: `kedro.io.DataCatalog` - The Kedro data catalog instance.
- **spark**: `pyspark.sql.SparkSession` - The SparkSession instance.
- **table_name**: `str` - The name of the Hive table.
- **schema**: `str` (optional) - The Hive schema name.
- **save_args**: `dict` (optional) - Additional arguments for saving data.
- **load_args**: `dict` (optional) - Additional arguments for loading data.

### Methods

- **__init__(self, catalog, spark, table_name, schema=None, save_args=None, load_args=None)**: Initializes the SparkHiveDataset.
- **load()**: Loads data from the specified Hive table.
- **save(data)**: Saves the provided data to the specified Hive table.
- **describe()**: Returns a description of the dataset.

### Example Usage

```python
from pyspark.sql import SparkSession
from kedro.io import DataCatalog
from kedro_datasets.spark import SparkHiveDataset

# Assuming spark and catalog are already initialized
spark = SparkSession.builder.appName("SparkHiveDatasetExample").getOrCreate()
catalog = DataCatalog()

hive_dataset = SparkHiveDataset(
    catalog=catalog,
    spark=spark,
    table_name="my_hive_table",
    schema="my_schema"
)

# Load data
data = hive_dataset.load()

# Save data (assuming 'new_data' is a Spark DataFrame)
# hive_dataset.save(new_data)
```
```

--------------------------------

### JSON Chat Prompt Format

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Example of a chat prompt structure in JSON format, including system and human roles.

```json
[
  {
    "role": "system",
    "content": "You are a helpful assistant."
  },
  {
    "role": "human",
    "content": "{user_input}"
  }
]
```

--------------------------------

### Version-Specific Loading

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Shows how to load specific versions of a prompt using either a version number or a label. 'production' label is preferred over a specific version number if both are available.

```python
# Load specific versions
historical_dataset = PromptDataset(load_args={"version": 3})  # Load version 3

labeled_dataset = PromptDataset(
    load_args={"label": "production"}  # Load production label
)
```

--------------------------------

### Labeling Strategy for Version Management

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/langfuse/README.md

Saves a prompt content, which auto-creates a new version, and then demonstrates applying semantic versioning labels for organization. Labels help manage different stages or types of prompts.

```python
# Semantic versioning with labels
dataset.save(prompt_content)  # Auto-creates new version

# Apply labels for organization
dataset = PromptDataset(save_args={"labels": ["v2.1.0", "production", "stable"]})
```

--------------------------------

### Opik Scorer Implementation

Source: https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets_experimental/opik/README.md

Example of a scorer implemented as a class inheriting from `base_metric.BaseMetric` for Opik, receiving a bundled dataset_item and task_outputs, and returning a ScoreResult.

```python
from opik.evaluation.metrics import base_metric, score_result


class MyScorer(base_metric.BaseMetric):
    def score(
        self, dataset_item: dict, task_outputs: dict, **kwargs
    ) -> score_result.ScoreResult:
        expected = dataset_item.get("expected_output", "")
        actual = task_outputs.get("output", "")
        value = float(actual == expected)
        return score_result.ScoreResult(name=self.name, value=value, reason="...")
```