### Setup OpenAI Client for LLM Inference

Source: https://context7.com/abhinand5/medembed/llms.txt

Configures an OpenAI-compatible client to connect to LLM inference servers, including local deployments. Requires base URL and API key.

```python
from openai import OpenAI

def setup_openai_client(base_url: str, api_key: str) -> OpenAI:
    return OpenAI(base_url=base_url, api_key=api_key)

# Setup client for local vLLM server
client = setup_openai_client(
    base_url="http://localhost:4000/v1",
    api_key="your-api-key"
)

# Test the connection
def test_openai_api(client: OpenAI, model_name: str) -> str:
    response = client.chat.completions.create(
        model=model_name,
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the world series in 2020?"}
        ]
    )
    return response.choices[0].message.content

result = test_openai_api(client, "vllm-llama3.1-70b-4bit")
print(f"API Test Result: {result}")
```

--------------------------------

### Set up Python Path

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Ensures that the project's source directory is accessible for imports. This is typically used at the beginning of scripts.

```python
import os
import sys

sys.path.insert(0, os.path.abspath("../src/"))
```

--------------------------------

### Load Configuration from YAML File

Source: https://context7.com/abhinand5/medembed/llms.txt

Loads configuration settings from a specified YAML file. This function is essential for initializing the application with parameters defined in the configuration.

```python
import yaml

def load_config(config_path: str = 'config/data_v1.yaml') -> dict:
    with open(config_path, 'r') as file:
        return yaml.safe_load(file)

config = load_config()
print(f"Using model: {config['openai']['model_name']}")
print(f"Processing {config['data']['num_samples']} samples")
# Output: Using model: vllm-llama3.1-70b-4bit
# Output: Processing 5000 samples
```

--------------------------------

### Train BGE Models with Contrastive Learning

Source: https://context7.com/abhinand5/medembed/llms.txt

Fine-tunes BGE models on medical triplet data using the FlagEmbedding library. This script configures hyperparameters, data paths, and training epochs for the fine-tuning process.

```bash
#!/bin/bash

MODEL_NAME="BAAI/bge-base-en-v1.5"
TRAIN_DATA_PATH="./data/triplets.jsonl"
OUTPUT_DIR="/workspace/medical-bge-base-v0"
LEARNING_RATE=2e-5

torchrun \
    -m FlagEmbedding.baai_general_embedding.finetune.run \
    --output_dir ${OUTPUT_DIR} \
    --model_name_or_path ${MODEL_NAME} \
    --train_data ${TRAIN_DATA_PATH} \
    --learning_rate ${LEARNING_RATE} \
    --bf16 \
    --num_train_epochs 3 \
    --per_device_train_batch_size 32 \
    --dataloader_drop_last True \
    --normlized True \
    --temperature 0.02 \
    --query_max_len 128 \
    --passage_max_len 256 \
    --negatives_cross_device \
    --logging_steps 10 \
    --save_steps 500 \
    --query_instruction_for_retrieval "" \
    --warmup_ratio 0.05 \
    --max_grad_norm 1.0 \
    --lr_scheduler_type cosine

# Training data format (triplets.jsonl):
# {"query": "chest pain symptoms", "pos": ["Patient presents with chest pain..."], "neg": ["Patient reports headache..."]}
```

--------------------------------

### Create Flat Pairs Dataset for Contrastive Learning

Source: https://context7.com/abhinand5/medembed/llms.txt

Transforms a hierarchical dataset into a flat structure suitable for contrastive learning. It extracts QA pairs, associates them with sample IDs, and prepares them for model training.

```python
import nanoid
from datasets import Dataset

def add_summary_id(sample: Dict) -> Dict:
    sample["id"] = nanoid.generate()
    return sample

def process_dataset(ds: Dataset) -> Dataset:
    ds = ds.map(add_summary_id)
    ds_faulty = ds.filter(lambda x: len(x["qa_pairs"]) == 0)
    print(f"[WARNING] {len(ds_faulty)} faulty samples found in the dataset.")
    return ds

def create_pairs_dataset(ds: Dataset) -> Dataset:
    pairs_data = []
    for sample in ds:
        pairs = sample["qa_pairs"]
        for data in pairs:
            data["note_id"] = sample["id"]
            data["patient_id"] = sample["patient_id"]
            pairs_data.append(data)
    return Dataset.from_list(pairs_data)

# Process and create pairs
ds = process_dataset(ds)
ds_pairs = create_pairs_dataset(ds)

print(f"Total training pairs: {len(ds_pairs)}")
# Output: Total training pairs: 30000 (assuming 6 pairs per 5000 notes)

# Push to Hugging Face Hub
ds.select_columns(["id", "patient_id", "note"]).push_to_hub(
    "your-username/medembed-corpus",
    "corpus",
    split="train"
)
ds_pairs.push_to_hub("your-username/medembed-pairs", split="train")
```

--------------------------------

### Print Selected Metrics

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Outputs the list of metrics currently being used for evaluation.

```python
print(selected_metrics[2:])
```

--------------------------------

### Compare and Rank Models

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Performs head-to-head model comparison and generates a styled ranking table.

```python
# Compare all models
win_matrix, win_percentages = compare_models(df, metrics=["avg"])
plot_comparison(win_matrix, win_percentages, df['model'].unique())
```

```python
# Print ranking
ranking = pd.Series(win_percentages, index=df['model'].unique()).sort_values(ascending=False)
print("Model Ranking based on Head-to-Head Wins:")
# for model, percentage in ranking.items():
#     print(f"{model}: {percentage:.2f}%")
ranking = ranking.to_frame(name="win_percent")

ranking.style.background_gradient(cmap='YlGnBu', axis=0, subset=["win_percent"])
```

--------------------------------

### Plot Model Comparison Results

Source: https://context7.com/abhinand5/medembed/llms.txt

Visualizes model comparison results using heatmaps for win matrices and bar plots for win percentages. This function requires the win matrix, win percentages, and model names as input. It saves the plots as PNG files.

```python
import matplotlib.pyplot as plt
import seaborn as sns

def plot_comparison(win_matrix, win_percentages, models):
    """
    Plot heatmap of win matrix and bar plot of win percentages.
    """
    # Heatmap of head-to-head wins
    plt.figure(figsize=(12, 10))
    sns.heatmap(win_matrix, annot=True, fmt='.0f', cmap='YlGnBu',
                xticklabels=models, yticklabels=models)
    plt.title('Head-to-Head Comparison of Models (Number of Wins)')
    plt.xlabel('Model (Columns)')
    plt.ylabel('Model (Rows)')
    plt.tight_layout()
    plt.savefig('head2head_heatmap.png', dpi=150)
    plt.show()

    # Bar plot of win percentages
    plt.figure(figsize=(12, 6))
    plt.bar(models, win_percentages)
    plt.title('Model Performance: Percentage of Head-to-Head Wins')
    plt.xlabel('Model')
    plt.ylabel('Win Percentage')
    plt.ylim(0, 100)
    plt.xticks(rotation=45, ha='right')
    plt.tight_layout()
    plt.savefig('win_percentages.png', dpi=150)
    plt.show()

# Generate comparison visualizations
plot_comparison(win_matrix, win_percentages, df['model'].unique())
```

--------------------------------

### Create Training Pairs Dataset

Source: https://context7.com/abhinand5/medembed/llms.txt

Processes individual samples to generate QA pairs and adds them to the sample. This function is mapped over a dataset for parallel processing.

```python
import json
from typing import List, Dict

def parse_qa_pairs(json_str: str) -> List[Dict]:
    try:
        json_str = json_str.replace("```json", "").replace("```", "")
        data = json.loads(json_str)
        return data.get("qa_pairs", [])
    except Exception as e:
        print(f"Error parsing JSON: {e}")
        return []

def create_training_pairs_ds(client, model_name: str, sample: Dict) -> Dict:
    note = sample['note']
    json_output = generate_questions_and_answers(client, model_name, note)
    qa_pairs = parse_qa_pairs(json_output)

    sample["raw_output"] = json_output
    sample["qa_pairs"] = qa_pairs
    return sample

# Process dataset with generated QA pairs
from datasets import Dataset

ds = Dataset.from_pandas(df)
ds = ds.map(
    lambda x: create_training_pairs_ds(client, "vllm-llama3.1-70b-4bit", x),
    num_proc=4  # Parallel processing
)

print(f"Processed {len(ds)} samples")
print(f"Sample QA pairs: {ds[0]['qa_pairs'][:2]}")
```

--------------------------------

### Import Core Libraries

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Imports essential libraries for data manipulation, numerical operations, plotting, and color mapping. These are fundamental for most analyses.

```python
import numpy as np
import pandas as pd

from pprint import pprint

import matplotlib.pyplot as plt
from matplotlib.colors import LinearSegmentedColormap
```

--------------------------------

### Print Model Ranking

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Generates a pandas Series of model rankings based on head-to-head win percentages and prints them in descending order. This provides a clear overview of model performance.

```python
# Print ranking
ranking = pd.Series(win_percentages, index=df['model'].unique()).sort_values(ascending=False)
print("Model Ranking based on Head-to-Head Wins:")
# for model, percentage in ranking.items():
```

--------------------------------

### Compare Models and Plot Results

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Compares all models based on the 'avg' metric using a head-to-head approach and generates plots visualizing the comparison results. This provides an overall performance ranking.

```python
# Compare all models
win_matrix, win_percentages = compare_models(df, metrics=["avg"])
plot_comparison(win_matrix, win_percentages, df['model'].unique())
```

--------------------------------

### Model Training

Source: https://context7.com/abhinand5/medembed/llms.txt

Configures and executes the training process for embedding models.

```APIDOC
## POST /trainer/train

### Description
Trains an embedding model using specified hyperparameters such as batch size, learning rate, and compression settings.

### Method
POST

### Endpoint
/trainer/train

### Request Body
- **batch_size** (int) - Optional - Number of samples per batch
- **nbits** (int) - Optional - Compression bits for indexes
- **maxsteps** (int) - Optional - Maximum training steps
- **use_ib_negatives** (bool) - Optional - Use in-batch negatives
- **dim** (int) - Optional - Embedding dimensions
- **learning_rate** (float) - Optional - Learning rate for the optimizer
- **doc_maxlen** (int) - Optional - Maximum document length
- **use_relu** (bool) - Optional - Enable or disable ReLU
- **warmup_steps** (string/int) - Optional - Number of warmup steps
```

--------------------------------

### Import MedEmbed Modules

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Imports specific functions for data styling, colormap creation, and model comparison from the med_embed library.

```python
from med_embed.compare.df_styles import style_dataframe, create_custom_colormap
from med_embed.compare.head2head import compare_models, plot_comparison
```

--------------------------------

### plot_comparison

Source: https://context7.com/abhinand5/medembed/llms.txt

Visualizes model comparison results using heatmaps and bar plots.

```APIDOC
## POST /plot_comparison

### Description
Generates visual representations of model performance, including a heatmap of win counts and a bar chart of win percentages.

### Method
POST

### Endpoint
/plot_comparison

### Request Body
- **win_matrix** (ndarray) - Required - The win matrix generated by compare_models
- **win_percentages** (ndarray) - Required - The win percentages generated by compare_models
- **models** (list) - Required - List of model names
```

--------------------------------

### Train ColBERT Models with RAGatouille

Source: https://context7.com/abhinand5/medembed/llms.txt

Trains late-interaction ColBERT models using the RAGatouille library. This involves preparing training data, including mining hard negatives, and initializing the RAGTrainer.

```python
import os
from pathlib import Path
from tqdm.auto import tqdm
from datasets import load_dataset
from ragatouille import RAGTrainer

DATA_DIR = os.environ.get("DATA_DIR", "./processed-data")

def get_data_ready(ds):
    pairs = []
    full_corpus = []

    for sample in tqdm(ds):
        pairs.append((sample["query"], sample["information"]))
        full_corpus.append(sample["information"])

    return pairs, full_corpus

# Load training dataset
ds = load_dataset("abhinand/clini-colbert-pairs-v0.2-dev", split="train")
print(f"Loaded {len(ds)} training pairs")

# Initialize trainer
trainer = RAGTrainer(
    model_name="Med-ColBERT",
    pretrained_model_name="colbert-ir/colbertv2.0",
    language_code="en"
)

# Prepare training data with hard negatives
os.makedirs(DATA_DIR, exist_ok=True)
pairs, full_corpus = get_data_ready(ds)

trainer.prepare_training_data(
    raw_data=pairs,
    data_out_path=DATA_DIR,
    all_documents=full_corpus,
    num_new_negatives=10,
    mine_hard_negatives=True
)

```

--------------------------------

### Compare Embedding Models

Source: https://context7.com/abhinand5/medembed/llms.txt

Performs a head-to-head comparison of embedding models across various tasks and metrics. It calculates win matrices and win percentages. Requires a pandas DataFrame with model performance data.

```python
import numpy as np
import pandas as pd

def compare_models(df, models=None, metrics=None):
    """
    Perform head-to-head comparison of models across all tasks and metrics.

    :param df: DataFrame with columns 'task_name', 'model', and various metrics
    :param models: List of models to compare. If None, use all models
    :param metrics: List of metrics to use. If None, use all numeric columns
    :return: Tuple of (win_matrix, win_percentages)
    """
    if models is None:
        models = df['model'].unique()
    else:
        df = df[df['model'].isin(models)]

    if metrics is None:
        metrics = df.select_dtypes(include=[np.number]).columns.drop(
            ['task_name', 'model'], errors='ignore'
        )

    n_models = len(models)
    win_matrix = np.zeros((n_models, n_models))

    for task in df['task_name'].unique():
        task_data = df[df['task_name'] == task].set_index('model')
        for metric in metrics:
            for i, model1 in enumerate(models):
                for j, model2 in enumerate(models):
                    if i != j:
                        if task_data.loc[model1, metric] > task_data.loc[model2, metric]:
                            win_matrix[i, j] += 1

    total_comparisons = len(df['task_name'].unique()) * len(metrics) * (n_models - 1)
    win_percentages = win_matrix.sum(axis=1) / total_comparisons * 100

    return win_matrix, win_percentages

# Load benchmark results
df = pd.concat([
    pd.read_csv('results/med_emb_small_v1_results.csv'),
    pd.read_csv('results/med_emb_small_v2_results.csv'),
], ignore_index=True)

# Define metrics for comparison
METRICS = ['ndcg_at_1', 'ndcg_at_5', 'ndcg_at_10', 'map_at_5', 'map_at_10',
           'recall_at_1', 'recall_at_5', 'recall_at_10', 'precision_at_1',
           'precision_at_5', 'mrr_at_1', 'mrr_at_5']

# Add average metric
df["avg"] = df[METRICS].mean(axis=1)

# Compare models using average metric
win_matrix, win_percentages = compare_models(df, metrics=["avg"])

# Print ranking
ranking = pd.Series(win_percentages, index=df['model'].unique()).sort_values(ascending=False)
print("Model Ranking based on Head-to-Head Wins:")
for model, percentage in ranking.items():
    print(f"{model}: {percentage:.2f}%")
```

--------------------------------

### YAML Configuration for Data Pipeline

Source: https://context7.com/abhinand5/medembed/llms.txt

Defines the complete configuration for a data generation pipeline, including LLM settings, data sources, and output destinations. Ensure sensitive information like API keys are handled securely.

```yaml
# config/data_v1.yaml
openai:
  api_key: ""  # Add your API key or leave empty for local servers
  base_url: "http://localhost:4000/v1"
  model_name: "vllm-llama3.1-70b-4bit"

data:
  dataset_name: "starmpcc/Asclepius-Synthetic-Clinical-Notes"
  num_samples: 5000
  random_state: 42

processing:
  num_proc: 4

output:
  corpus_hub: "abhinand/clini-colbert-pairs-v1-dev"
  pairs_hub: "abhinand/clini-colbert-pairs-v1-dev"
```

--------------------------------

### Rank Models by Win Percentage

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Ranks models based on win percentages and applies a background gradient style to the resulting dataframe.

```python
# Print ranking
ranking = pd.Series(win_percentages, index=df['model'].unique()).sort_values(ascending=False)
print("Model Ranking based on Head-to-Head Wins:")
# for model, percentage in ranking.items():
#     print(f"{model}: {percentage:.2f}%")
ranking = ranking.to_frame(name="win_percent")

ranking.style.background_gradient(cmap='YlGnBu', axis=0, subset=["win_percent"])
```

```python
# Print ranking
ranking = pd.Series(win_percentages, index=df['model'].unique()).sort_values(ascending=False)
print("""Model Ranking based on Head-to-Head Wins on 5 Benchmarks:
- ArguAna
- MedicalQARetrieval
- NFCorpus
- PublicHealthQA
- TRECCOVID

Metrics -> nDCG @ 1, nDCG @ 5, nDCG @ 10, 
           MAP @ 5, MAP @ 10, Recall @ 1, 
           Recall @ 5, Recall @ 10, Precision @ 1, 
           Precision @ 5, MRR @ 1, MRR @ 5, Average
""")
# for model, percentage in ranking.items():
#     print(f"{model}: {percentage:.2f}%")
ranking = ranking.to_frame(name="win_percent")

ranking.style.background_gradient(cmap='YlGnBu', axis=0, subset=["win_percent"])
```

--------------------------------

### Create Custom Colormap for DataFrame Styling

Source: https://context7.com/abhinand5/medembed/llms.txt

Creates a custom red-white-green colormap for visualizing performance metrics in DataFrames. Use this when you need a specific color gradient for highlighting performance differences.

```python
import matplotlib.pyplot as plt
from matplotlib.colors import LinearSegmentedColormap

def create_custom_colormap():
    """Create red-white-green colormap for performance visualization."""
    colors = ['#FFCCCB', '#FFFFFF', '#006400']  # light red, white, dark green
    return LinearSegmentedColormap.from_list('custom', colors, N=100)
```

--------------------------------

### Style DataFrame for ArguAna Task

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Filters the DataFrame for the 'ArguAna' task, selects the relevant metrics (including the newly calculated 'avg'), and styles it using a custom colormap. The output is a pandas Styler object.

```python
_m1_df = df.loc[df["task_name"] == "ArguAna", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

--------------------------------

### Generate QA Pairs from Clinical Notes

Source: https://context7.com/abhinand5/medembed/llms.txt

Generates diverse query-information pairs from clinical notes using an LLM. Outputs JSON with keyword, natural language, and procedure-related queries, along with extracted information and relevance scores.

```python
from openai import OpenAI

def generate_questions_and_answers(client: OpenAI, model_name: str, note: str) -> str:
    prompt = f"""You are an advanced medical AI assistant tasked with generating diverse, realistic queries and extracting relevant information from clinical notes.

Given the following clinical note, please:
1. Generate 6 diverse queries (2 keyword-based, 2 natural language, 2 procedure-related)
2. For each query, extract the most relevant information from the note
3. Include relevance scores (0-1) for each pair

Clinical Note:
{note}

Generate the query-information pairs in JSON format with qa_pairs array."""

    response = client.chat.completions.create(
        model=model_name,
        messages=[
            {"role": "system", "content": "You are a medical AI assistant specializing in clinical notes analysis."},
            {"role": "user", "content": prompt}
        ],
        max_tokens=4096,
        temperature=0.8,
    )
    return response.choices[0].message.content

# Example usage
clinical_note = """
Patient presents with chest pain radiating to left arm. ECG shows ST elevation.
Troponin levels elevated. Diagnosed with acute myocardial infarction.
Started on aspirin, heparin, and scheduled for cardiac catheterization.
"""

json_output = generate_questions_and_answers(client, "vllm-llama3.1-70b-4bit", clinical_note)
# Output: {"qa_pairs": [
#   {"query": "chest pain cardiac symptoms", "query_type": "keyword",
#    "information": "Patient presents with chest pain radiating to left arm.", "relevance_score": 0.95},
#   {"query": "What was the patient diagnosed with?", "query_type": "natural_language",
#    "information": "Diagnosed with acute myocardial infarction.", "relevance_score": 1.0},
#   ...
# ]}
```

--------------------------------

### Load Asclepius Clinical Data

Source: https://context7.com/abhinand5/medembed/llms.txt

Loads clinical notes from the Asclepius dataset on Hugging Face. Supports sampling for development or testing.

```python
import pandas as pd
from datasets import load_dataset

def load_asclepius_data(dataset_name: str, num_samples: int, random_state: int) -> pd.DataFrame:
    dataset = load_dataset(dataset_name)
    df = pd.DataFrame(dataset['train'])
    return df.sample(n=num_samples, random_state=random_state) if num_samples else df

# Load 5000 samples from the Asclepius clinical notes dataset
df = load_asclepius_data(
    dataset_name="starmpcc/Asclepius-Synthetic-Clinical-Notes",
    num_samples=5000,
    random_state=42
)

print(f"Loaded {len(df)} clinical notes")
print(df.columns.tolist())
# Output: Loaded 5000 clinical notes
# Output: ['patient_id', 'note', ...]
```

--------------------------------

### Load and Filter Model Results

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Loads multiple CSV result files and filters the dataframe for selected models.

```python
selected_models = [
    "BAAI/bge-small-en-v1.5",
    "./medical-bge-small-v1-mix1",
    "BAAI/bge-base-en-v1.5",
    "./medical-bge-base-v0-mix2",
    "BAAI/bge-large-en-v1.5",
    "medical-bge-large-mix2",
]

df = pd.concat([
    pd.read_csv("../results/med_emb_small_v1_results.csv"),
    pd.read_csv("../results/med_emb_base_v0_results.csv"),
    pd.read_csv("../results/med_emb_large_v0_results.csv")
], ignore_index=True)

df = df[df["model"].isin(selected_models)].reset_index(drop=True)
```

--------------------------------

### Process Dataset and Filter Faulty Samples

Source: https://context7.com/abhinand5/medembed/llms.txt

Processes a dataset by adding unique IDs to each sample and filtering out samples with no QA pairs. It also warns about the number of faulty samples found.

```python
import nanoid
from datasets import Dataset

def add_summary_id(sample: Dict) -> Dict:
    sample["id"] = nanoid.generate()
    return sample

def process_dataset(ds: Dataset) -> Dataset:
    ds = ds.map(add_summary_id)
    ds_faulty = ds.filter(lambda x: len(x["qa_pairs"]) == 0)
    print(f"[WARNING] {len(ds_faulty)} faulty samples found in the dataset.")
    return ds
```

--------------------------------

### Load and Inspect Evaluation Data

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Loads evaluation results from CSV files and inspects the distribution of models and tasks.

```python
df = pd.concat([
    pd.read_csv('../results/med_emb_base_v0_results.csv'),
    pd.read_csv('../results/med_emb_base_results.csv'),
], ignore_index=True)
df["model"].value_counts()
```

```python
df["task_name"].value_counts()
```

```python
df = pd.read_csv("../results/med_emb_large_v0_results.csv")
df["model"].value_counts()
```

```python
df["task_name"].value_counts()
```

--------------------------------

### Define Metrics List

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Defines a list of metrics to be used for evaluating and comparing models. This list can be modified to include or exclude specific metrics.

```python
METRICS = [
    'task_name',
    'model',
    'ndcg_at_1',
    'ndcg_at_5',
    'ndcg_at_10',
    'map_at_5',
    'map_at_10',
    'recall_at_1',
    'recall_at_5',
    'recall_at_10',
    'precision_at_1',
    'precision_at_5',
    'mrr_at_1',
    'mrr_at_5',
    # 'nauc_ndcg_at_10_max',
    # 'nauc_ndcg_at_10_std',
    # 'languages'
]
```

--------------------------------

### Load and Use MedEmbed Sentence Transformer Model

Source: https://context7.com/abhinand5/medembed/llms.txt

Loads a pre-trained MedEmbed model from Hugging Face using the Sentence Transformers library. This model can then be used to encode medical queries and documents into embeddings for retrieval tasks.

```python
from sentence_transformers import SentenceTransformer

# Load MedEmbed model (available sizes: small, base, large)
model = SentenceTransformer("abhinand/MedEmbed-small-v0.1")

# Encode medical queries and documents
queries = [
    "symptoms of myocardial infarction",
    "What medications are used for diabetes treatment?",
    "post-operative care after cardiac surgery"
]

documents = [
    "Patient presents with chest pain radiating to left arm, elevated troponin levels consistent with acute MI.",
    "Metformin 500mg twice daily prescribed for type 2 diabetes management along with lifestyle modifications.",
    "Following coronary artery bypass grafting, patient should monitor for signs of infection and attend cardiac rehabilitation."
]

# Generate embeddings
query_embeddings = model.encode(queries, normalize_embeddings=True)
doc_embeddings = model.encode(documents, normalize_embeddings=True)

# Compute similarity scores
import numpy as np
similarities = np.dot(query_embeddings, doc_embeddings.T)

print("Similarity Matrix:")
for i, query in enumerate(queries):
    print(f"\nQuery: {query}")
    for j, doc in enumerate(documents):
        print(f"  Doc {j+1}: {similarities[i][j]:.4f}")

# Output:
# Query: symptoms of myocardial infarction
#   Doc 1: 0.8234 (highest - correct match)
#   Doc 2: 0.3421
#   Doc 3: 0.4532
```

--------------------------------

### compare_models

Source: https://context7.com/abhinand5/medembed/llms.txt

Performs head-to-head comparison of multiple embedding models across tasks and metrics.

```APIDOC
## POST /compare_models

### Description
Computes win matrices and win percentages for models based on provided benchmark data.

### Method
POST

### Endpoint
/compare_models

### Request Body
- **df** (DataFrame) - Required - DataFrame containing 'task_name', 'model', and metrics
- **models** (list) - Optional - List of models to compare
- **metrics** (list) - Optional - List of metrics to evaluate

### Response
#### Success Response (200)
- **win_matrix** (ndarray) - Matrix of head-to-head wins
- **win_percentages** (ndarray) - Calculated win percentages for each model
```

--------------------------------

### Display Task-Specific Dataframes

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Filters the dataframe by task name and displays it with a custom colormap.

```python
_m1_df = df.loc[df["task_name"] == "ArguAna", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

```python
_m1_df = df.loc[df["task_name"] == "NFCorpus", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

```python
_m1_df = df.loc[df["task_name"] == "MedicalQARetrieval", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

```python
_m1_df = df.loc[df["task_name"] == "PublicHealthQA", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

```python
_m1_df = df.loc[df["task_name"] == "TRECCOVID", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

--------------------------------

### Style DataFrame for NFCorpus Task

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Filters the DataFrame for the 'NFCorpus' task and applies custom styling with a colormap to the selected metrics. This aids in visually inspecting the results for this task.

```python
_m1_df = df.loc[df["task_name"] == "NFCorpus", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

--------------------------------

### Display Task Name Value Counts

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Calculates and displays the frequency of each task name within the DataFrame. This helps understand the distribution of tasks in the results.

```python
df["task_name"].value_counts()
```

--------------------------------

### Style DataFrame for PublicHealthQA Task

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Filters the DataFrame for the 'PublicHealthQA' task and applies custom styling with a colormap. This visualization helps in analyzing performance trends for this task.

```python
_m1_df = df.loc[df["task_name"] == "PublicHealthQA", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

--------------------------------

### Display Task-Specific Performance

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Filters the dataframe by task name and displays the results using a custom styler.

```python
_m1_df = df.loc[df["task_name"] == "ArguAna", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

```python
_m1_df = df.loc[df["task_name"] == "MedicalQARetrieval", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

```python
_m1_df = df.loc[df["task_name"] == "NFCorpus", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

```python
_m1_df = df.loc[df["task_name"] == "PublicHealthQA", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

```python
_m1_df = df.loc[df["task_name"] == "TRECCOVID", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

--------------------------------

### Style DataFrame with Conditional Formatting

Source: https://context7.com/abhinand5/medembed/llms.txt

Applies conditional color formatting to numeric columns of a DataFrame for visual analysis. It scales colors based on the min/max values in each column and sets text color for readability. Subset the columns to apply formatting.

```python
import matplotlib.pyplot as plt
from matplotlib.colors import LinearSegmentedColormap

def create_custom_colormap():
    """Create red-white-green colormap for performance visualization."""
    colors = ['#FFCCCB', '#FFFFFF', '#006400']  # light red, white, dark green
    return LinearSegmentedColormap.from_list('custom', colors, N=100)

def style_dataframe(df, colormap='viridis'):
    """Apply color scaling to numeric columns in DataFrame."""
    def color_scale(s):
        norm = (s - s.min()) / (s.max() - s.min())
        colors = plt.get_cmap(colormap)(norm)
        return ['background-color: rgba({}, {}, {}, 0.7); color: {}'.format(
            int(r*255), int(g*255), int(b*255),
            'black' if r * 0.299 + g * 0.587 + b * 0.114 > 0.729 else 'white')
            for r, g, b, _ in colors]

    return df.style.apply(color_scale, axis=0, subset=df.columns[2:])

# Display styled results for a specific task
task_df = df.loc[df["task_name"] == "MedicalQARetrieval", METRICS + ['model', 'task_name', 'avg']]
styled_df = style_dataframe(task_df, colormap=create_custom_colormap())
display(styled_df)  # In Jupyter notebook
```

--------------------------------

### Style DataFrame for MedicalQARetrieval Task

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Filters the DataFrame for the 'MedicalQARetrieval' task, selects specified metrics, and applies custom styling with a colormap. This is useful for visualizing performance on this specific task.

```python
_m1_df = df.loc[df["task_name"] == "MedicalQARetrieval", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

--------------------------------

### Style DataFrame for TRECCOVID Task

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Filters the DataFrame for the 'TRECCOVID' task and applies custom styling with a colormap. This is useful for detailed examination of results on the TRECCOVID dataset.

```python
_m1_df = df.loc[df["task_name"] == "TRECCOVID", selected_metrics]
display(style_dataframe(_m1_df, colormap=create_custom_colormap()))
```

--------------------------------

### Add Unique Identifiers to Samples

Source: https://context7.com/abhinand5/medembed/llms.txt

Adds a unique nanoid identifier to each sample in the dataset. This is a prerequisite for processing and creating pairs.

```python
import nanoid
from datasets import Dataset

def add_summary_id(sample: Dict) -> Dict:
    sample["id"] = nanoid.generate()
    return sample
```

--------------------------------

### Train MedEmbed Model

Source: https://context7.com/abhinand5/medembed/llms.txt

Trains an embedding model using specified parameters such as batch size, embedding dimensions, and learning rate. Ensure data is preprocessed and suitable for training.

```python
trainer.train(
    batch_size=64,
    nbits=4,              # Compression bits for indexes
    maxsteps=500000,      # Maximum training steps
    use_ib_negatives=True,  # Use in-batch negatives
    dim=128,              # Embedding dimensions
    learning_rate=5e-6,   # Learning rate (3e-6 to 3e-5 works best)
    doc_maxlen=256,       # Maximum document length
    use_relu=False,       # Disable ReLU
    warmup_steps="auto",  # 10% warmup
)
print("Training Complete!")
```

--------------------------------

### Parse QA Pairs from JSON String

Source: https://context7.com/abhinand5/medembed/llms.txt

Parses a JSON string to extract QA pairs. It cleans the input string and handles potential JSON parsing errors.

```python
import json
from typing import List, Dict

def parse_qa_pairs(json_str: str) -> List[Dict]:
    try:
        json_str = json_str.replace("```json", "").replace("```", "")
        data = json.loads(json_str)
        return data.get("qa_pairs", [])
    except Exception as e:
        print(f"Error parsing JSON: {e}")
        return []
```

--------------------------------

### Load and Concatenate Results

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Loads results from multiple CSV files into a single pandas DataFrame and displays the value counts for the 'model' column.

```python
df = pd.concat([
    pd.read_csv('../results/med_emb_small_v1_results.csv'),
    pd.read_csv('../results/med_emb_small_v2_results.csv'),
], ignore_index=True)
df["model"].value_counts()
```

--------------------------------

### Calculate Average Metric

Source: https://github.com/abhinand5/medembed/blob/main/notebooks/bge_result_analysis.ipynb

Calculates the average of specified metrics for each row and adds it as a new 'avg' column to the DataFrame. Excludes 'task_name' and 'model' from averaging.

```python
selected_metrics = METRICS.copy()

df["avg"] = df[selected_metrics].iloc[:, 2:].mean(axis=1)

selected_metrics.append("avg")
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.