### Setup HumanEval Repository

Source: https://github.com/openai/simple-evals

Clones the HumanEval repository from GitHub and installs it locally in editable mode. This is a prerequisite for running evaluations using the HumanEval benchmark.

```shell
git clone https://github.com/openai/human-eval
pip install -e human-eval
```

--------------------------------

### Install HumanEval Benchmark

Source: https://github.com/openai/simple-evals

Installs the HumanEval benchmark and its dependencies for Python programming evaluations. This involves cloning the repository and installing it as an editable package.

```bash
git clone https://github.com/openai/human-eval
pip install -e human-eval
```

--------------------------------

### Run LLM Evaluations

Source: https://github.com/openai/simple-evals

Launches evaluations for a specified model and number of examples. This command initiates the evaluation process, typically interacting with APIs like OpenAI.

```shell
python -m simple-evals.simple_evals --model <model_name> --examples <num_examples>
```

--------------------------------

### Install OpenAI Python Library

Source: https://github.com/openai/simple-evals

Installs the official Python client library for the OpenAI API. This is necessary for interacting with OpenAI models for evaluations or other tasks.

```bash
pip install openai
```

--------------------------------

### Example JSON Configuration

Source: https://github.com/openai/simple-evals

A sample JSON object representing a server color mode configuration. This is a common pattern for client-server communication or state management.

```JSON
{
  "resolvedServerColorMode": "day"
}
```

--------------------------------

### Install Anthropic Python Client

Source: https://github.com/openai/simple-evals

Installs the official Anthropic Python client library using pip. This client is needed for running evaluations with Anthropic models.

```shell
pip install anthropic
```

--------------------------------

### Run LLM Evaluations

Source: https://github.com/openai/simple-evals

Launches evaluations for a specified language model using a given number of examples. This command typically interacts with the OpenAI API by default and requires model name and example count as parameters.

```bash
python -m simple-evals.simple_evals --model <model_name> --examples <num_examples>
```

--------------------------------

### Install OpenAI Python Client

Source: https://github.com/openai/simple-evals

Installs the official OpenAI Python client library using pip. This library is required to interact with the OpenAI API for model evaluations.

```shell
pip install openai
```

--------------------------------

### Install Anthropic Python Library

Source: https://github.com/openai/simple-evals

Installs the official Python client library for the Anthropic API, used for interacting with Claude models. This package is required for running evaluations or applications that utilize Anthropic's models.

```bash
pip install anthropic
```

--------------------------------

### List Available Models for Evaluation

Source: https://github.com/openai/simple-evals

Executes a Python command to list all models that can be evaluated using the simple-evals tool. This helps in identifying the correct model names for subsequent evaluation runs.

```shell
python -m simple-evals.simple_evals --list-models
```

--------------------------------

### List Available LLM Models

Source: https://github.com/openai/simple-evals

Executes a Python command to list all the language models that can be evaluated using the simple-evals framework. This command is a prerequisite for running specific evaluations.

```bash
python -m simple-evals.simple_evals --list-models
```

--------------------------------

### OpenAI API Assistant System Message

Source: https://github.com/openai/simple-evals

This snippet documents the default system message used for assistants in the OpenAI API. It specifies the exact string that defines the assistant's persona and helpfulness.

```APIDOC
OpenAI API Assistant System Message:
  "You are a helpful assistant."

  Description:
    This is the default system message provided to the assistant in the OpenAI API. It sets the persona and behavior for the AI model, guiding it to be helpful.
  
  Context:
    Referenced in the OpenAI API documentation for introduction and system message configuration.
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.