### Install TrustLLM from GitHub

Source: https://github.com/howiehwong/trustllm/blob/main/docs/index.md

Install the TrustLLM package directly from its GitHub repository. This method is recommended for the latest updates.

```shell
git clone git@github.com:HowieHwong/TrustLLM.git
cd TrustLLM/trustllm_pkg
pip install .
```

--------------------------------

### Configure OpenAI and Perspective API Keys

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Set your OpenAI and Perspective API keys before starting evaluations. This is a prerequisite for most TrustLLM functionalities.

```python
from trustllm import config

config.openai_key = 'your-openai-api-key'

config.perspective_key = 'your-perspective-api-key'
```

--------------------------------

### Install TrustLLM via pip

Source: https://github.com/howiehwong/trustllm/blob/main/docs/index.md

Install the TrustLLM package using pip. Note that this method is deprecated.

```shell
pip install trustllm
```

--------------------------------

### Generate Results for a Local Llama2 Model

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/generation_details.md

Example of initializing LLMGeneration for a specific local model, Llama-2-7b-chat-hf, for safety testing. This configuration assumes the model and dataset are locally available.

```python
from trustllm.generation.generation import LLMGeneration

llm_gen = LLMGeneration(
    model_path="meta-llama/Llama-2-7b-chat-hf", 
    test_type="safety", 
    data_path="TrustLLM"
)

llm_gen.generation_results()
```

--------------------------------

### Install TrustLLM via conda

Source: https://github.com/howiehwong/trustllm/blob/main/docs/index.md

Install the TrustLLM package using conda. Note that this method is deprecated.

```shell
conda install -c conda-forge trustllm
```

--------------------------------

### Create Conda Environment

Source: https://github.com/howiehwong/trustllm/blob/main/docs/index.md

Create a new conda environment for TrustLLM with Python 3.9. This is a recommended first step before installing the package.

```shell
conda create --name trustllm python=3.9
```

--------------------------------

### Initialize LLMGeneration with Online Model

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/generation_details.md

Instantiate the LLMGeneration class with `online_model=True` to utilize remote models. This is useful when local computing resources are insufficient.

```python
from trustllm.generation.generation import LLMGeneration

llm_gen = LLMGeneration(
    model_path="meta-llama/Llama-2-7b-chat-hf", 
    test_type="safety", 
    data_path="TrustLLM",
    online_model=True
)

llm_gen.generation_results()
```

--------------------------------

### Initialize LLMGeneration for Local Models

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/generation_details.md

Instantiate the LLMGeneration class to generate results for a local model. Specify the model path, test type, and dataset path. The default is to use local models.

```python
from trustllm.generation.generation import LLMGeneration

llm_gen = LLMGeneration(
    model_path="your model name", 
    test_type="test section", 
    data_path="your dataset file path",
    model_name="", 
    online_model=False,
    use_deepinfra=False,
    use_replicate=False,
    repetition_penalty=1.0,
    num_gpus=1, 
    max_new_tokens=512, 
    debug=False
)

llm_gen.generation_results()
```

--------------------------------

### Initialize Safety Evaluator

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Sets up the SafetyEval class for performing various safety assessments. Requires importing necessary modules from trustllm.

```python
from trustllm import safety
from trustllm import file_process
from trustllm import config

evaluator = safety.SafetyEval()
```

--------------------------------

### Initialize Truthfulness Evaluator

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Sets up the TruthfulnessEval class for performing various truthfulness assessments. Requires importing necessary modules from trustllm.

```python
from trustllm import truthfulness
from trustllm import file_process
from trustllm import config

evaluator = truthfulness.TruthfulnessEval()
```

--------------------------------

### Configure API Keys for Online Models

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/generation_details.md

Set API keys and credentials for various online LLM providers before using them for generation. Ensure you replace placeholder values with your actual API keys.

```python
from trustllm import config

config.deepinfra_api = "deepinfra api"

config.claude_api = "claude api"

config.openai_key = "openai api"

config.palm_api = "palm api"

config.ernie_client_id = "ernie client id"

config.ernie_client_secret = "ernie client secret"

config.ernie_api = "ernie api"
```

--------------------------------

### LLM Generation with TrustLLM

Source: https://github.com/howiehwong/trustllm/blob/main/README.md

Initializes and runs LLM generation using the LLMGeneration class. Specify model path, test type, and data path.

```python
from trustllm.generation.generation import LLMGeneration

llm_gen = LLMGeneration(
    model_path="your model name", 
    test_type="test section", 
    data_path="your dataset file path",
    model_name="", 
    online_model=False, 
    use_deepinfra=False,
    use_replicate=False,
    repetition_penalty=1.0,
    num_gpus=1, 
    max_new_tokens=512, 
    debug=False,
    device='cuda:0'
)

llm_gen.generation_results()
```

--------------------------------

### Run Safety Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Initiate the safety evaluation pipeline, specifying paths for jailbreak, exaggerated safety, and misuse data. Optionally enable and configure toxicity evaluation.

```python
safety_results = run_safety(
    jailbreak_path="path_to_jailbreak_data.json",
    exaggerated_safety_path="path_to_exaggerated_safety_data.json",
    misuse_path="path_to_misuse_data.json",
    toxicity_eval=True,
    toxicity_path="path_to_toxicity_data.json",
    jailbreak_eval_type="total"
)
```

--------------------------------

### Initialize Robustness Evaluator

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Initializes the RobustnessEval class from the trustllm library. This is a preliminary step before performing any robustness evaluations.

```python
from trustllm import robustness
from trustllm import file_process
from trustllm import config

evaluator = robustness.RobustnessEval()
```

--------------------------------

### Configure Online Model APIs

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/generation_details.md

Set your DeepInfra or Replicate API keys before using online models. These are required for authentication and access.

```python
from trustllm import config

config.deepinfra_api = "deepinfra api"

config.replicate_api = "replicate api"
```

--------------------------------

### Process Dataset for LLM Generation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/index.md

Load a JSON dataset, iterate through its elements, and add a 'res' key with the generated output from an LLM. This prepares the data for evaluation.

```python
import json

filename = 'dataset_path.json'

# Load the data from the file
with open(filename, 'r') as file:
    data = json.load(file)

# Process each dictionary and add the 'res' key with the generated output
for element in data:
    element['res'] = generation(element['prompt'])  # Replace 'generation' with your function

# Write the modified data back to the file
with open(filename, 'w') as file:
    json.dump(data, file, indent=4)
```

--------------------------------

### Configure Azure OpenAI API Settings

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

If using Azure OpenAI, configure the necessary settings including the engine name and API base URL.

```python
config.azure_openai = True

config.azure_engine = "your-azure-engine-name"

config.azure_api_base = "your-azure-api-url (openai.base_url)"
```

--------------------------------

### Evaluate Toxicity (from Saved Results)

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Evaluates toxicity using pre-saved evaluated jailbreak results loaded from a JSON file.

```python
# If you have saved evaluated jailbreak results
toxicity_data = file_process.load_json('evaluated_jailbreak_results_json_path') # load eval data for toxicity evaluation
print(evaluator.toxicity_eval(toxicity_data))
```

--------------------------------

### Evaluate Jailbreak (Total)

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Performs a total jailbreak evaluation, returning an overall rate of response to adversarial prompts. Requires loading jailbreak data from a JSON file.

```python
jailbreak_data = file_process.load_json('jailbreak_data_json_path')
print(evaluator.jailbreak_eval(jailbreak_data, eval_type='total'))
```

--------------------------------

### Run Robustness Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Conduct robustness evaluation by specifying paths to datasets for adversarial GLUE, adversarial instructions, OOD detection, and OOD generalization.

```python
robustness_results = run_robustness(
    advglue_path="path_to_advglue_data.json",
    advinstruction_path="path_to_advinstruction_data.json",
    ood_detection_path="path_to_ood_detection_data.json",
    ood_generalization_path="path_to_ood_generalization_data.json"
)
```

--------------------------------

### Initialize Fairness Evaluator

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Initializes the FairnessEval class from the trustllm library. This is a preliminary step before performing any fairness evaluations.

```python
from trustllm import fairness
from trustllm import file_process
from trustllm import config

evaluator = fairness.FairnessEval()
```

--------------------------------

### Evaluate Toxicity (from Jailbreak Data)

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Evaluates toxicity using data derived from a prior jailbreak evaluation. If evaluated jailbreak results are not saved, they are computed first.

```python
# If you have not saved evaluated jailbreak results
jailbreak_data = file_process.load_json('jailbreak_data_json_path')
eval_data, _ = evaluator.jailbreak_eval(jailbreak_data, eval_type='total/single', return_data=True)
print(evaluator.toxicity_eval(eval_data))
```

--------------------------------

### Evaluate Jailbreak (Single)

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Performs a single jailbreak evaluation, returning a dictionary of response rates for each type of jailbreak. Requires loading jailbreak data from a JSON file.

```python
jailbreak_data = file_process.load_json('jailbreak_data_json_path')
print(evaluator.jailbreak_eval(jailbreak_data, eval_type='single'))
```

--------------------------------

### Download TrustLLM Dataset

Source: https://github.com/howiehwong/trustllm/blob/main/docs/index.md

Download the TrustLLM dataset using the provided Python function. Specify a save path for the downloaded dataset.

```python
from trustllm.dataset_download import download_dataset

download_dataset(save_path='save_path')
```

--------------------------------

### Dataset Temperature Configuration

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/generation_details.md

Defines the temperature setting for various dataset files. A temperature of 0 is used for classification tasks for precision, while 1 is used for generation tasks to encourage diversity.

```python
file_config = {
        "disparagement.json": 1.0,
        "preference_force.json": 1.0,
        "preference_plain.json": 1.0,
        "stereotype_agreement.json": 1.0,
        "stereotype_stereoset_classification.json": 0.0,
        "stereotype_query_test.json":1.0,
        "stereotype_recognition.json":0.0,
        "external.json": 0.0,
        "hallucination.json": 0.0,
        "golden_advfactuality.json": 1.0,
        "internal.json": 1.0,
        "sycophancy.json": 1.0,
        "ood_detection.json":1.0,
        "ood_generalization.json":0.0,
        "AdvGLUE.json":0.0,
        "AdvInstruction.json":1.0,
        "jailbreak.json":1.0,
        "exaggerated_safety.json": 1.0,
        "misuse.json":1.0,
        "privacy_awareness_confAIde.json":0.0,
        "privacy_awareness_query.json": 1.0,
        "privacy_leakage.json": 1.0,
        "awareness.json": 0.0,
        "implicit_ETHICS.json": 0.0,
        "implicit_SocialChemistry101.json": 0.0
    }
```

--------------------------------

### Initialize Privacy Evaluator

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Initializes the PrivacyEval class from the trustllm library. This is a preliminary step before performing any privacy evaluations.

```python
from trustllm import privacy
from trustllm import file_process
from trustllm import config

evaluator = privacy.PrivacyEval()
```

--------------------------------

### Initialize Ethics Evaluator

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Initializes the EthicsEval class from the trustllm library. This is a preliminary step before performing any machine ethics evaluations.

```python
from trustllm import ethics
from trustllm import file_process
from trustllm import config

evaluator = ethics.EthicsEval()
```

--------------------------------

### Run Truthfulness Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/README.md

Executes the truthfulness evaluation pipeline by providing paths to the necessary data files.

```python
from trustllm.task.pipeline import run_truthfulness

truthfulness_results = run_truthfulness(  
    internal_path="path_to_internal_consistency_data.json",  
    external_path="path_to_external_consistency_data.json",  
    hallucination_path="path_to_hallucination_data.json",  
    sycophancy_path="path_to_sycophancy_data.json",
    advfact_path="path_to_advfact_data.json"
)
```

--------------------------------

### Perform OOD Generalization Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads OOD generalization data and evaluates it using the ood_generalization method of the RobustnessEval class.

```python
ood_generalization_data = file_process.load_json('ood_generalization_data_json_path')
print(evaluator.ood_generalization(ood_generalization_data))
```

--------------------------------

### Perform OOD Detection Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads OOD detection data and evaluates it using the ood_detection method of the RobustnessEval class.

```python
ood_detection_data = file_process.load_json('ood_detection_data_json_path')
print(evaluator.ood_detection(ood_detection_data))
```

--------------------------------

### Perform Preference Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads preference data and evaluates it using the preference_eval method of the FairnessEval class.

```python
preference_data = file_process.load_json('preference_data_json_path')
print(evaluator.preference_eval(preference_data))
```

--------------------------------

### Run Fairness Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Perform fairness evaluation by providing paths to datasets for stereotype recognition, stereotype agreement, stereotype queries, disparagement, and preference biases.

```python
fairness_results = run_fairness(
    stereotype_recognition_path="path_to_stereotype_recognition_data.json",
    stereotype_agreement_path="path_to_stereotype_agreement_data.json",
    stereotype_query_test_path="path_to_stereotype_query_test_data.json",
    disparagement_path="path_to_disparagement_data.json",
    preference_path="path_to_preference_data.json"
)
```

--------------------------------

### Run Privacy Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Execute the privacy evaluation pipeline, providing paths to datasets for privacy conformity, privacy awareness queries, and privacy leakage scenarios.

```python
privacy_results = run_privacy(
    privacy_confAIde_path="path_to_privacy_confaide_data.json",
    privacy_awareness_query_path="path_to_privacy_awareness_query_data.json",
    privacy_leakage_path="path_to_privacy_leakage_data.json"
)
```

--------------------------------

### Run Truthfulness Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Execute the truthfulness evaluation pipeline by providing paths to relevant JSON datasets for internal consistency, external consistency, hallucinations, sycophancy, and adversarial factuality.

```python
truthfulness_results = run_truthfulness(
    internal_path="path_to_internal_consistency_data.json",
    external_path="path_to_external_consistency_data.json",
    hallucination_path="path_to_hallucination_data.json",
    sycophancy_path="path_to_sycophancy_data.json",
    advfact_path="path_to_advfact_data.json"
)
```

--------------------------------

### Perform AdvGlue Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads AdvGlue data and evaluates it using the advglue_eval method of the RobustnessEval class. This is also used for advinstruction data.

```python
advglue_data = file_process.load_json('advglue_data_json_path')
print(evaluator.advglue_eval(advglue_data))

advinstruction_data = file_process.load_json('advinstruction_data_json_path')
print(evaluator.advglue_eval(advinstruction_data))
```

--------------------------------

### Run Ethics Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Initiates the ethics evaluation process by specifying paths to relevant data files. The function returns a dictionary with detailed results.

```python
results = run_ethics(
    explicit_ethics_path="path_to_explicit_ethics_data.json",
    implicit_ethics_path="path_to_implicit_ethics_data.json",
    awareness_path="path_to_awareness_data.json"
)
```

--------------------------------

### Perform Awareness Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads awareness data and evaluates it using the awareness_eval method of the EthicsEval class.

```python
awareness_data = file_process.load_json('awareness_data_json_path')
print(evaluator.awareness_eval(awareness_data))
```

--------------------------------

### TrustLLM Project Citation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/index.md

Use this BibTeX entry for citing the TrustLLM project in academic work. It includes authors, title, year, and publication details.

```text
@misc{sun2024trustllm,
      title={TrustLLM: Trustworthiness in Large Language Models},
      author={Lichao Sun and Yue Huang and Haoran Wang and Siyuan Wu and Qihui Zhang and Chujie Gao and Yixin Huang and Wenhan Lyu and Yixuan Zhang and Xiner Li and Zhengliang Liu and Yixin Liu and Yijue Wang and Zhikun Zhang and Bhavya Kailkhura and Caiming Xiong and Chaowei Xiao and Chunyuan Li and Eric Xing and Furong Huang and Hao Liu and Heng Ji and Hongyi Wang and Huan Zhang and Huaxiu Yao and Manolis Kellis and Marinka Zitnik and Meng Jiang and Mohit Bansal and James Zou and Jian Pei and Jian Liu and Jianfeng Gao and Jiawei Han and Jieyu Zhao and Jiliang Tang and Jindong Wang and John Mitchell and Kai Shu and Kaidi Xu and Kai-Wei Chang and Lifang He and Lifu Huang and Michael Backes and Neil Zhenqiang Gong and Philip S. Yu and Pin-Yu Chen and Quanquan Gu and Ran Xu and Rex Ying and Shuiwang Ji and Suman Jana and Tianlong Chen and Tianming Liu and Tianyi Zhou and Willian Wang and Xiang Li and Xiangliang Zhang and Xiao Wang and Xing Xie and Xun Chen and Xuyu Wang and Yan Liu and Yanfang Ye and Yinzhi Cao and Yong Chen and Yue Zhao},
      year={2024},
      eprint={2401.05561},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

--------------------------------

### Evaluate Misuse

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads misuse data from a JSON file and performs a misuse evaluation using the SafetyEval class.

```python
misuse_data = file_process.load_json('misuse_data_json_path')
print(evaluator.misuse_eval(misuse_data))
```

--------------------------------

### Evaluate Adversarial Factuality

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads adversarial factuality data from a JSON file and performs the evaluation using the TruthfulnessEval class.

```python
adv_fact_data = file_process.load_json('adv_fact_data_json_path')
print(evaluator.advfact_eval(adv_fact_data))
```

--------------------------------

### Evaluate Hallucination

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads hallucination data from a JSON file and performs a hallucination evaluation using the TruthfulnessEval class.

```python
hallucination_data = file_process.load_json('hallucination_data_json_path')
print(evaluator.hallucination_eval(hallucination_data))
```

--------------------------------

### Perform Privacy Awareness Query Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads privacy awareness query data and evaluates it using the awareness_query_eval method of the PrivacyEval class. Supports 'normal' and 'aug' types.

```python
privacy_awareness_query_data = file_process.load_json('privacy_awareness_query_data_json_path')
print(evaluator.awareness_query_eval(privacy_awareness_query_data, type='normal'))
print(evaluator.awareness_query_eval(privacy_awareness_query_data, type='aug'))
```

--------------------------------

### Evaluate Misinformation (External)

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads external misinformation data from a JSON file and performs an external evaluation using the TruthfulnessEval class.

```python
misinformation_external_data = file_process.load_json('misinformation_external_data_json_path')
print(evaluator.external_eval(misinformation_external_data))
```

--------------------------------

### Evaluate Misinformation (Internal)

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads internal misinformation data from a JSON file and performs an internal evaluation using the TruthfulnessEval class.

```python
misinformation_internal_data = file_process.load_json('misinformation_internal_data_json_path')
print(evaluator.internal_eval(misinformation_internal_data))
```

--------------------------------

### Perform Stereotype Recognition Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads stereotype recognition data and evaluates it using the stereotype_recognition_eval method of the FairnessEval class.

```python
stereotype_recognition_data = file_process.load_json('stereotype_recognition_data_json_path')
print(evaluator.stereotype_recognition_eval(stereotype_recognition_data))
```

--------------------------------

### Perform Stereotype Query Test Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads stereotype query test data and evaluates it using the stereotype_query_eval method of the FairnessEval class.

```python
stereotype_query_test_data = file_process.load_json('stereotype_query_test_data_json_path')
print(evaluator.stereotype_query_eval(stereotype_query_test_data))
```

--------------------------------

### Perform Privacy Awareness ConfAIde Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads privacy awareness data for ConfAIde and evaluates it using the ConfAIDe_eval method of the PrivacyEval class.

```python
privacy_confAIde_data = file_process.load_json('privacy_confAIde_data_json_path')
print(evaluator.ConfAIDe_eval(privacy_confAIde_data))
```

--------------------------------

### Perform Implicit Ethics Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads implicit ethics data and evaluates it using the implicit_ethics_eval method of the EthicsEval class. Supports 'ETHICS' and 'social_norm' eval_types.

```python
implicit_ethics_data = file_process.load_json('implicit_ethics_data_json_path')
# evaluate ETHICS dataset
print(evaluator.implicit_ethics_eval(implicit_ethics_data, eval_type='ETHICS'))
# evaluate social_norm dataset
print(evaluator.implicit_ethics_eval(implicit_ethics_data, eval_type='social_norm'))
```

--------------------------------

### Perform Disparagement Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads disparagement data and evaluates it using the disparagement_eval method of the FairnessEval class.

```python
disparagement_data = file_process.load_json('disparagement_data_json_path')
print(evaluator.disparagement_eval(disparagement_data))
```

--------------------------------

### Perform Stereotype Agreement Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads stereotype agreement data and evaluates it using the stereotype_agreement_eval method of the FairnessEval class.

```python
stereotype_agreement_data = file_process.load_json('stereotype_agreement_data_json_path')
print(evaluator.stereotype_agreement_eval(stereotype_agreement_data))
```

--------------------------------

### Evaluate Sycophancy

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads sycophancy data from a JSON file and performs sycophancy evaluation. Supports 'persona' and 'preference' evaluation types.

```python
sycophancy_data = file_process.load_json('sycophancy_data_json_path')
print(evaluator.sycophancy_eval(sycophancy_data, eval_type='persona'))
print(evaluator.sycophancy_eval(sycophancy_data, eval_type='preference'))
```

--------------------------------

### Perform Explicit Ethics Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads explicit ethics data and evaluates it using the explicit_ethics_eval method of the EthicsEval class. Supports 'low' and 'high' eval_types.

```python
explicit_ethics_data = file_process.load_json('explicit_ethics_data_json_path')
print(evaluator.explicit_ethics_eval(explicit_ethics_data, eval_type='low'))
print(evaluator.explicit_ethics_eval(explicit_ethics_data, eval_type='high'))
```

--------------------------------

### Evaluate Exaggerated Safety

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads exaggerated safety data from a JSON file and performs an exaggerated safety evaluation using the SafetyEval class.

```python
exaggerated_data = file_process.load_json('exaggerated_data_json_path')
print(evaluator.exaggerated_eval(exaggerated_data))
```

--------------------------------

### Perform Privacy Leakage Evaluation

Source: https://github.com/howiehwong/trustllm/blob/main/docs/guides/evaluation.md

Loads privacy leakage data and evaluates it using the leakage_eval method of the PrivacyEval class.

```python
privacy_leakage_data = file_process.load_json('privacy_leakage_data_json_path')
print(evaluator.leakage_eval(privacy_leakage_data))
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.