### Download SafetyBench Data with Shell Script Source: https://github.com/thu-coai/safetybench/blob/main/README.md Use this shell script to download test questions and few-shot examples from Hugging Face. ```shell #!/bin/bash # Download test questions # Chinese test questions wget https://huggingface.co/datasets/thu-coai/SafetyBench/resolve/main/test_zh.json # English test questions wget https://huggingface.co/datasets/thu-coai/SafetyBench/resolve/main/test_en.json # Chinese subset test questions wget https://huggingface.co/datasets/thu-coai/SafetyBench/resolve/main/test_zh_subset.json # Download few-shot examples # Chinese few-shot examples wget https://huggingface.co/datasets/thu-coai/SafetyBench/resolve/main/dev_zh.json # English few-shot examples wget https://huggingface.co/datasets/thu-coai/SafetyBench/resolve/main/dev_en.json ``` -------------------------------- ### Submission File Format Example Source: https://github.com/thu-coai/safetybench/blob/main/README.md Prepare a UTF-8 encoded JSON file with question IDs as keys and predicted answers (0->A, 1->B, 2->C, 3->D) as values. ```json { "0": 0, "1": 1, "2": 3, "3": 2 } ``` -------------------------------- ### Download SafetyBench Data with Python Datasets Library Source: https://github.com/thu-coai/safetybench/blob/main/README.md Use the Hugging Face datasets library in Python to download test questions and few-shot examples. ```python from datasets import load_dataset # Load test questions test_zh = load_dataset("thu-coai/SafetyBench", "test_zh") test_en = load_dataset("thu-coai/SafetyBench", "test_en") test_zh_subset = load_dataset("thu-coai/SafetyBench", "test_zh_subset") # Load few-shot examples dev_zh = load_dataset("thu-coai/SafetyBench", "dev_zh") dev_en = load_dataset("thu-coai/SafetyBench", "dev_en") ``` -------------------------------- ### Citation for SafetyBench Source: https://github.com/thu-coai/safetybench/blob/main/README.md BibTeX citation for the SafetyBench paper. ```bibtex @article{zhang2023safetybench, title={SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions}, author={Zhexin Zhang and Leqi Lei and Lindong Wu and Rui Sun and Yongkang Huang and Chong Long and Xiao Liu and Xuanyu Lei and Jie Tang and Minlie Huang}, journal={arXiv preprint arXiv:2309.07045}, year={2023} } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.