### Fine-tune LLMs using Medical and General Safety Demonstrations

Source: https://github.com/ai4life-group/med-safety-bench/blob/main/README.md

This script fine-tunes LLMs using curated datasets of medical safety, general safety, or a combination of both. It utilizes specific data splits for training.

```text
training/finetuning.py
```

--------------------------------

### Prompt LLMs with Harmful Requests

Source: https://github.com/ai4life-group/med-safety-bench/blob/main/README.md

This script is used to prompt various LLMs, including general and medical models, with harmful requests. It's part of the experimental evaluation pipeline.

```text
exps/exp01_prompt_models.py
```

--------------------------------

### Evaluate Medical Performance of LLMs

Source: https://github.com/ai4life-group/med-safety-bench/blob/main/README.md

This shell script executes an inference pipeline to evaluate the medical performance of LLMs, both before and after fine-tuning. It's crucial for assessing improvements in medical safety.

```text
meditron/evaluation/inference_pipeline.sh
```

--------------------------------

### Evaluate LLM Responses to Harmful Requests

Source: https://github.com/ai4life-group/med-safety-bench/blob/main/README.md

This script evaluates the responses generated by LLMs when presented with harmful requests. It is used for both general and medical LLMs.

```text
exps/exp02_eval_responses.py
```

--------------------------------

### Generate Harmful Medical Requests with Llama-3-8B-Instruct

Source: https://github.com/ai4life-group/med-safety-bench/blob/main/README.md

This script generates harmful medical requests using Llama-3-8B-Instruct, specifically forcing responses to begin with 'Sure'. This is used for expanding the dataset with more harmful requests.

```text
exps/adv_attack/generate_prompts.py
```

--------------------------------

### MedSafetyBench Citation

Source: https://github.com/ai4life-group/med-safety-bench/blob/main/README.md

This is the citation information for the MedSafetyBench paper, including title, authors, journal, and year.

```bibtex
@article{han2024medsafetybench,
  title={MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models},
  author={Han, Tessa and Kumar, Aounon and Agarwal, Chirag and Lakkaraju, Himabindu},
  journal={NeurIPS},
  year={2024}
}
```

--------------------------------

### Generate Safe Responses to Harmful Medical Requests

Source: https://github.com/ai4life-group/med-safety-bench/blob/main/README.md

This script is responsible for generating safe responses to harmful medical requests. It is a key component in creating the safety demonstrations within the dataset.

```text
exps/exp03_generate_safe_responses.py
```

--------------------------------

### Generate Harmful Medical Requests with Llama2-7b-chat

Source: https://github.com/ai4life-group/med-safety-bench/blob/main/README.md

This script generates harmful medical requests by jailbreaking using Llama2-7b-chat. It is part of the dataset generation process.

```text
exps/adv_attack/gcg.py
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.