### Install and Run BetterChat UI

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Install the necessary dependencies and start the BetterChat development server.

```bash
npm install
npm run dev
```

--------------------------------

### Start OpenAI-Compatible API Server

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Run this command to launch the API server compatible with OpenAI's API, allowing integration with various clients.

```bash
python3 -m fastchat.serve.openai_api_server \
    --host localhost \
    --port 8000
```

--------------------------------

### Install scipdf-parser

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/README.md

Install the scipdf-parser library from GitHub to convert PDF documents to text.

```bash
# Install scipdf from GitHub to convert PDFs to text
pip install git+https://github.com/titipata/scipdf_parser
```

--------------------------------

### Launch Gradio Web Server

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Use this command to start the default Gradio web server for interacting with models via a web UI.

```bash
python3 -m fastchat.serve.gradio_web_server
```

--------------------------------

### Install Dependencies for Guideline Scrapers

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/README.md

Install the necessary Python packages for the guideline web scrapers and cleaning code.

```bash
# Install dependencies
pip install -r guidelines/requirements.txt
```

--------------------------------

### Install Rust and CMake on Mac

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Install necessary build tools (Rust and CMake) if you are running on a Mac system.

```bash
brew install rust cmake
```

--------------------------------

### Example Benchmark Class

Source: https://github.com/epfllm/meditron/blob/main/evaluation/README.md

Define a new benchmark by extending the `Benchmark` class. Ensure the benchmark is registered on the Huggingface dataset hub.

```python
class MyBenchmark(Benchmark):
    '''
    MyBenchmark is <Your Description>

    Huggingface card: https://huggingface.co/datasets/<MyBenchmark>
    '''
    def __init__(self, name='pubmedqa') -> None:
        super().__init__(name)
        self.hub_name = "<MyBenchmark>"
        self.dir_name = '<Directory for MyBenchmark>'
        self.path = os.path.join(ROOT_DIR, 'benchmarks', 'datasets', self.dir_name)
        self.splits = ['train', 'validation', 'test']
        self.subsets = ['<If subset exist, specify here>']

    @staticmethod
    def custom_preprocessing(row):
        '''Add your custom preprocessing code here'''
        return row
```

--------------------------------

### Launch FastChat Controller

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Start the FastChat controller using the command `python3 -m fastchat.serve.controller`. The controller manages the distributed workers and is essential for the web GUI serving.

```bash
python3 -m fastchat.serve.controller
```

--------------------------------

### Load Meditron Model from HuggingFace

Source: https://github.com/epfllm/meditron/blob/main/README.md

Load the Meditron model and tokenizer using the HuggingFace transformers library. Ensure you have the library installed.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("epfl-llm/meditron-70b")
model = AutoModelForCausalLM.from_pretrained("epfl-llm/meditron-70b")
```

--------------------------------

### Register One-Shot Medical Conversation Template

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Register a custom conversation template named 'one_shot_medical' in FastChat's conversation module. This template includes a system message and example user-assistant turns for medical-related dialogues.

```python
register_conv_template(
        Conversation(
            name="one_shot_medical",
            system_message=system_msg_meditron,
            roles=("User", "Assistant"),
            messages=(
                (
                    "User",
                    "What happens if listeria is left untreated?",
                ),
                (
                    "Assistant",
                    "If listeria infection, or listeriosis, is left untreated, it can lead to severe health complications, particularly in certain high-risk groups. Here's a general overview of the potential outcomes:\n\n   1. Spread of the Infection: Untreated listeriosis can spread beyond the gut to other parts of the body, including the nervous system. This can lead to more severe conditions like meningitis (inflammation of the membranes surrounding the brain and spinal cord) and septicemia (a serious blood infection).\n   2. Increased Risk for Certain Groups: Pregnant women, newborns, the elderly, and individuals with weakened immune systems are at a higher risk of severe complications. In pregnant women, listeriosis can lead to miscarriage, stillbirth, premature delivery, or life-threatening infection of the newborn.\n   3. Neurological Effects: Listeriosis can cause severe neurological symptoms like headaches, stiff neck, confusion, loss of balance, and convulsions, especially when the infection spreads to the nervous system.\n   4. Long-Term Health Impacts: For some, particularly those with pre-existing health conditions or weakened immune systems, the health impacts of listeriosis can be long-lasting and may not fully resolve even with treatment.\n   5. Fatalities: In severe cases, particularly among high-risk groups, listeriosis can be fatal.\n\n   It's important to note that early diagnosis and appropriate treatment, typically with antibiotics, can greatly improve the prognosis for those with listeriosis. Therefore, seeking medical attention promptly if listeriosis is suspected is crucial."
               )
           ),
            offset=2,
            sep_style=SeparatorStyle.ADD_COLON_SINGLE,
            sep="\n### ",
            stop_str=["<", ">", "###", "Assistant:", "User:"]
       )
   )
```

--------------------------------

### Download and Pre-process All Clinical Guidelines

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/README.md

Execute this script to download and pre-process all 46,000 clinical practice guidelines.

```bash
./guidelines/download.sh
```

--------------------------------

### Run Finetuning Pipeline

Source: https://github.com/epfllm/meditron/blob/main/finetuning/README.md

Execute the `sft.py` script to initiate the fine-tuning process. Ensure that paths in the `CHECKPOINTS` constant and the `N_DOCS` dictionary within the script are updated to match your environment and dataset.

```bash
python finetuning/sft.py --checkpoint=meditron --size=7 --run_name=pubmedqa
```

--------------------------------

### Supervised Finetuning Script

Source: https://github.com/epfllm/meditron/blob/main/README.md

Use this script to initiate a multi-node supervised finetuning process. Ensure correct paths are set in sft.py and finetune_sft.sh.

```bash
cd finetuning
python sft.py \
    --checkpoint=baseline \
    --size=70 \
    --run_name=cotmedqa \
    --data /pure-mlo-scratch/zechen/meditron/benchmarks/ft_preprocessed/medqa_cot_train.jsonl \
    --val /pure-mlo-scratch/zechen/meditron/benchmarks/ft_preprocessed/medqa_cot_validation.jsonl \
    --micro_batch=4
    --nodes=4 \
    --addr=<RANK0_HOST_NAME> \
    --save_interval=200 \
    --pp=4 \
    --seq 4096 \
    --rank=<CURRENT_RANK>
```

--------------------------------

### Download All GAP-Replay Datasets

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/README.md

Run this script to download all datasets and combine them into a single GAP-Replay corpus.

```bash
./download.sh
```

--------------------------------

### Download Replay Data

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/replay/README.md

Use this command to download Replay data. Adjust the --keep parameter to control the fraction of data retained and --out to specify the output file path.

```bash
python replay.py --keep=0.0004 --out=../data/replay.jsonl
```

--------------------------------

### Clone FastChat Repository

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Clone the FastChat repository and navigate to its directory. This is the initial step for setting up the deployment environment.

```bash
git clone https://github.com/lm-sys/FastChat.git
cd FastChat
```

--------------------------------

### Run inference_pipeline.sh with parameters

Source: https://github.com/epfllm/meditron/blob/main/evaluation/README.md

Use this script to run the inference-evaluation pipeline. Specify model, benchmark, and prompting strategies.

```bash
./inference_pipeline.sh
        -c <model_name> or <checkpoint_name>
        -b <benchmark name>
        -s <num_shots> (in-context learning shots number, default is zero)
        -r 0 (cot prompting; 0: disabled, 1: enabled)
        -e vllm (inference backend, currently support vllm only)
        -m 0 (in-context learning with multiple seeds; 0: disabled, 1: enabled)
        -t 0 (self-consistency cot prompting; 0: disabled, 1: enabled)
        -d 32 (setting batch_size 32 for inference per gpu
```

```bash
bash inference_pipeline.sh \
        -b pubmedqa
        -c meditron-70b \
        -s 5
        -m 1
```

```bash
bash inference_pipeline.sh \
        -b medqa
        -c meditron-70b-cotmedqa \
        -r 1
        -t 1
```

--------------------------------

### Download Replay Data

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/README.md

This script downloads and sub-samples replay data from the RedPajama-v1 dataset.

```bash
./replay/download.sh
```

--------------------------------

### Launch Meditron Model Worker

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Launch a model worker for the 'meditron-70b' model on 8 GPUs using the provided shell script. Ensure the process completes and shows 'Uvicorn running on ...' before proceeding.

```bash
./fastchat.sh \
    -c meditron-70b \
    -m single \
    -n 8
```

--------------------------------

### Download StarCoder Data

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/replay/README.md

Use this command to download StarCoder data. Adjust the --keep parameter to control the fraction of data retained and --out to specify the output file path. Ensure you have accepted the StarCoderData TOS.

```bash
python starcoder.py --keep=0.1 --out=../data/starcoder.jsonl
```

--------------------------------

### Load Clinical Guidelines Dataset

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/README.md

Load the open-access clinical guidelines dataset from the HuggingFace datasets hub.

```python
from datasets import load_dataset

dataset = load_dataset("epfl-llm/guidelines")
```

--------------------------------

### Download spaCy English Language Model

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/README.md

Download the small English language model for spaCy, used for text processing.

```bash
# Use spacy to get the English language pipeline
python -m spacy download en_core_web_sm 
```

--------------------------------

### Checkout Meditron Deployment Branch

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Clone the FastChat repository again and checkout the 'meditron-deploy' branch. This branch contains modifications specifically for deploying Meditron models.

```bash
git clone https://github.com/eric11eca/FastChat.git
cd FastChat
git checkout meditron-deploy
```

--------------------------------

### Model Checkpoint Configuration

Source: https://github.com/epfllm/meditron/blob/main/evaluation/README.md

Add models to `checkpoints` in `inference_pipeline.sh` by specifying their local path or Huggingface repository name.

```bash
["med42"]="m42-health/med42-70b" # Huggingface repo name
    ["meditron-70b"]="$meditron-70b/hf_checkpoints/raw/iter_23000/ # local path
```

--------------------------------

### Clone and Checkout BetterChat UI

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Clone the BetterChat repository and checkout the specific branch for the Meditron UI integration.

```bash
git clone https://github.com/eric11eca/BetterChatGPT
cd BetterChatGPT
git checkout meditron-ui
```

--------------------------------

### Downsample Replay Data to 400M Tokens

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/replay/README.md

This command downsamples Replay data to approximately 400 million tokens. The --keep parameter is set to 0.0004, and the output path is specified.

```bash
python replay.py --keep=0.0004 --out=/pure-mlo-scratch/alhernan/data/replay/replay-400M.json
```

--------------------------------

### Meditron Model Adapter for FastChat

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Implement a custom model adapter for Meditron models by subclassing BaseModelAdapter. This involves defining methods to match model paths, load models, and retrieve default conversation templates.

```python
class MeditronAdapter(BaseModelAdapter):
    """The model adapter for Meditron"""

    def match(self, model_path: str):
        return "meditron" in model_path.lower()

    def load_model(self, model_path: str, from_pretrained_kwargs: dict):
        model, tokenizer = super().load_model(model_path, from_pretrained_kwargs)
        model.config.eos_token_id = tokenizer.eos_token_id
        model.config.pad_token_id = tokenizer.pad_token_id
        return model, tokenizer

    def get_default_conv_template(self, model_path: str) -> Conversation:
        return get_conv_template("one_shot") # or meditron_one_shot, meditron
```

--------------------------------

### Convert Megatron-LLM Checkpoints to Huggingface Format

Source: https://github.com/epfllm/meditron/blob/main/evaluation/README.md

Use the `megatron2hf.sh` script to convert Megatron-LLM checkpoints to Huggingface format for inference and evaluation.

```bash
NUM_IN_SHARDS=8 # number of input model shards
NUM_OUT_SHARDS=8 # number of output model shards
INPUT_DIR=<path to your Megatron checkpoint>
OUTPUT_DIR=<path to save your HF model weights>
UNSHARDED_DIR=<path for stroing unsharded Megatron checkpoint, temporary>

./megatron2hf.sh
```

--------------------------------

### Download PubMed Papers and Abstracts

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/README.md

Use this script to download and pre-process PubMed papers and abstracts via the S2ORC API.

```bash
./pubmed/download.sh
```

--------------------------------

### Downsample StarCoder Data to 41B Tokens

Source: https://github.com/epfllm/meditron/blob/main/gap-replay/replay/README.md

This command downsamples StarCoder data to approximately 41 billion tokens. The --keep parameter is set to 0.09, and the output path is specified.

```bash
python starcoder.py --keep=0.09 --out=/pure-mlo-scratch/alhernan/data/starcoder/starcoder_41B.json
```

--------------------------------

### Register Zero-Shot Medical Conversation Template

Source: https://github.com/epfllm/meditron/blob/main/deployment/README.md

Register a custom conversation template named 'zero_shot_medical' in FastChat's conversation module. This template is designed for zero-shot medical inquiries and uses a specified system message and separator style.

```python
register_conv_template(
        Conversation(
            name="zero_shot_medical",
            system_message=system_msg_meditron,
            roles=("User", "Assistant"),
            sep_style=SeparatorStyle.ADD_COLON_SINGLE,
            sep="\n### ",
            stop_str=["<", ">", "###", "Assistant:", "User:"]
        )
    )
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.