Vivaria

https://vivaria.metr.org
Admin
Vivaria is METR's tool for running evaluations and conducting agent elicitation research through a...
Tokens:64,860
Snippets:549
Trust Score:9.8
Update:5 months ago
Show doc for...
Tokens:
Raw
### start

Source: https://vivaria.metr.org/reference/cli

Starts a task environment for testing, QA, or baselining. It builds a Docker image, starts a container, and runs the specified task within it.

```APIDOC
## start

### Description
Start a task environment that you can use to manually test a task, or as an environment for a QA run or a human baseline.
Builds a Docker image for a particular task, starts a container from that image, and runs TaskFamily#start in the container.

### Parameters
#### Path Parameters
- **taskId** (str) - Required - The task to test.

#### Query Parameters
- **dont_cache** (bool) - Optional - Rebuild the task environment primary machine's Docker image from scratch. Defaults to `False`.
- **ssh** (bool) - Optional - SSH into the task environment after starting it. Defaults to `False`.
- **ssh_user** (SSHUser) - Optional - User to SSH into the task environment as. Defaults to `'root'`.
- **task_family_path** (str | None) - Optional - Path to a task family directory to use. If not provided, Vivaria may look up the task family directory from a Git repo that it's configured to use. Defaults to `None`.
- **env_file_path** (str | None) - Optional - Path to a file of environment variables that Vivaria will set in some TaskFamily methods. You can only provide this argument if you also provide task_family_path. If neither task_family_path nor env_file_path is provided, Vivaria will read environment variables from a file called secrets.env in a Git repo that Vivaria is configured to use. Defaults to `None`.
- **ignore_workdir** (bool) - Optional - Start task from the current commit while ignoring any uncommitted changes. Defaults to `False`.
- **k8s** (bool | None) - Optional - Start the task environment in a Kubernetes cluster. Defaults to `None`.
```

--------------------------------

### Install Vivaria using the install script

Source: https://vivaria.metr.org/tutorials/set-up-docker-compose

Run this script on macOS and Linux to automate the installation of Vivaria.

```bash
curl -fsSL https://raw.githubusercontent.com/METR/vivaria/main/scripts/install.sh | bash - 

```

--------------------------------

### Start Task Environment via Direct Upload

Source: https://vivaria.metr.org/tutorials/start-task-environment

This method works with or without Git support. It zips the task code and uploads it to Vivaria to start the environment.

```bash
viv task start count_odds/main --task-family-path vivaria/examples/count_odds

```

--------------------------------

### Start Task Environment

Source: https://vivaria.metr.org/reference/cli

Starts a task environment for manual testing, QA runs, or human baselines. Builds a Docker image, starts a container, and runs TaskFamily#start within it. Supports options for caching, SSH access, and specifying task/environment files.

```python
@typechecked
def start(  # noqa: PLR0913
    self,
    taskId: str,  # noqa: ANN001, RUF100, N803 (CLI argument so can't change)
    dont_cache: bool = False,
    ssh: bool = False,
    ssh_user: SSHUser = "root",
    task_family_path: str | None = None,
    env_file_path: str | None = None,
    ignore_workdir: bool = False,
    k8s: bool | None = None,
) -> None:
    """Start a task environment.

    Start a task environment that you can use to manually test a task, or as an environment
    for a QA run or a human baseline.

    Builds a Docker image for a particular task, starts a container from that image, and runs
    TaskFamily#start in the container.

    Args:
        taskId: The task to test.
        dont_cache: Rebuild the task environment primary machine's Docker image from scratch.
        ssh: SSH into the task environment after starting it.
        ssh_user: User to SSH into the task environment as.
        task_family_path: Path to a task family directory to use. If not provided, Vivaria may
            look up the task family directory from a Git repo that it's configured to use.
        env_file_path: Path to a file of environment variables that Vivaria will set in some
            TaskFamily methods. You can only provide this argument if you also provide
            task_family_path. If neither task_family_path nor env_file_path is provided,
            Vivaria will read environment variables from a file called secrets.env in a Git repo
            that Vivaria is configured to use.
        ignore_workdir: Start task from the current commit while ignoring any uncommitted
            changes.
        k8s: Start the task environment in a Kubernetes cluster.
    """
    if task_family_path is None:
        if env_file_path is not None:
            err_exit("env_file_path cannot be provided without task_family_path")
        task_source = self._setup_task_commit(ignore_workdir=ignore_workdir)
    else:

```

--------------------------------

### Start Vivaria with Docker Compose

Source: https://vivaria.metr.org/tutorials/set-up-docker-compose

Starts Vivaria services using Docker Compose. Ensure API keys are set in .env.server and VIVARIA_DOCKER_GID is configured if necessary.

```bash
docker compose up --pull always --detach --wait

```

--------------------------------

### Start a Task Environment with Vivaria CLI

Source: https://vivaria.metr.org/reference/cli

Use this command to start a task environment for manual testing, QA runs, or human baselines. It builds a Docker image, starts a container, and runs TaskFamily#start. Options include rebuilding the image from scratch, SSHing into the environment, specifying a task family path, providing environment variables, ignoring working directory changes, and running in a Kubernetes cluster.

```python
from viv_cli.main import VivCli

cli = VivCli()
cli.start(taskId='my_task_id', dont_cache=True, ssh=True, ssh_user='dev', task_family_path='/path/to/task_family', env_file_path='/path/to/env.env', ignore_workdir=True, k8s=True)
```

--------------------------------

### Upload and Start Task Environment

Source: https://vivaria.metr.org/reference/cli

Uploads a task family and starts its environment. Handles environment caching and optional Kubernetes deployment. If SSH is enabled, it attempts to connect via SSH after the environment is ready.

```python
task_source = viv_api.upload_task_family(
    pathlib.Path(task_family_path).expanduser(),
    pathlib.Path(env_file_path).expanduser() if env_file_path is not None else None,
)

response_lines = viv_api.start_task_environment(
    taskId,
    task_source,
    dont_cache,
    k8s=k8s,
)

final_json = self._get_final_json_from_response(response_lines)
if final_json is None:
    return

environment_name = final_json.get("environmentName")
if environment_name is None:
    return

_set_last_task_environment_name(environment_name)

if ssh:
    self.ssh(environment_name=environment_name, user=ssh_user)
```

--------------------------------

### Task Start Environment

Source: https://vivaria.metr.org/reference/cli

Starts a task environment, optionally allowing SSH access, specifying task family paths, environment files, and destruction behavior.

```APIDOC
## Task Start Environment

### Description
Starts a task environment. This command can be used to SSH into the task environment, specify user credentials, control verbosity, define task family paths, and manage environment files. It also supports options for destroying the environment after use and ignoring the working directory.

### Parameters
#### Path Parameters
- **taskId** (string) - Required - The ID of the task to start.

#### Query Parameters
- **ssh** (boolean) - Optional - Whether to SSH into the task environment after starting it.
- **ssh_user** (string) - Optional - The user to SSH into the task environment as.
- **verbose** (boolean) - Optional - Log the output of all tests, on success or failure.
- **task_family_path** (string) - Optional - Path to a task family directory to use. If not provided, Vivaria may look up the task family directory from a Git repo.
- **env_file_path** (string) - Optional - Path to a file of environment variables. Can only be provided if `task_family_path` is also provided.
- **destroy** (boolean) - Optional - Destroy the task environment after running tests.
- **ignore_workdir** (boolean) - Optional - Run tests on the current commit while ignoring any uncommitted changes.
- **k8s** (boolean) - Optional - Start the task environment in a Kubernetes cluster.
- **dont_cache** (boolean) - Optional - Do not use cached task environments.
- **test_name** (string) - Optional - The name of the test to run.
```

--------------------------------

### test

Source: https://vivaria.metr.org/reference/cli

Starts a task environment and runs tests within it. This command is used for testing specific tasks and can be configured with various options for environment setup, test execution, and cleanup.

```APIDOC
## test

### Description
Starts a task environment and runs tests. This command is useful for validating task functionality before deployment or for debugging.

### Method
`test(taskId, test_name='', dont_cache=False, ssh=False, ssh_user='root', verbose=False, task_family_path=None, env_file_path=None, destroy=False, ignore_workdir=False, k8s=None)`

### Parameters
#### Path Parameters
- **taskId** (str) - Required - The task to test.

#### Query Parameters
- **test_name** (str) - Optional - Test file to run tests from. Defaults to `''`.
- **dont_cache** (bool) - Optional - Rebuild the task environment primary machine's Docker image from scratch. Defaults to `False`.
- **ssh** (bool) - Optional - SSH into the task environment after starting it. Defaults to `False`.
- **ssh_user** (SSHUser) - Optional - User to SSH into the task environment as. Defaults to `'root'`.
- **verbose** (bool) - Optional - Log the output of all tests, on success or failure. Defaults to `False`.
- **task_family_path** (str | None) - Optional - Path to a task family directory to use. If not provided, Vivaria may look up the task family directory from a Git repo that it's configured to use. Defaults to `None`.
- **env_file_path** (str | None) - Optional - Path to a file of environment variables that Vivaria will set in some TaskFamily methods. You can only provide this argument if you also provide task_family_path. If neither task_family_path nor env_file_path is provided, Vivaria will read environment variables from a file called secrets.env in a Git repo that Vivaria is configured to use. Defaults to `None`.
- **destroy** (bool) - Optional - Destroy the task environment after running tests. Defaults to `False`.
- **ignore_workdir** (bool) - Optional - Run tests on the current commit while ignoring any uncommitted changes. Defaults to `False`.
- **k8s** (bool | None) - Optional - Start the task environment in a Kubernetes cluster. Defaults to `None`.

### Request Example
```json
{
  "taskId": "example_task_id",
  "test_name": "test_suite.py",
  "dont_cache": true,
  "ssh": true,
  "ssh_user": "dev_user",
  "verbose": true,
  "task_family_path": "/path/to/task/family",
  "env_file_path": "/path/to/env.vars",
  "destroy": true,
  "ignore_workdir": true,
  "k8s": true
}
```

### Response
#### Success Response (200)
- **testStatusCode** (int) - The status code of the test execution.
- **environmentName** (str) - The name of the created task environment.

#### Response Example
```json
{
  "testStatusCode": 0,
  "environmentName": "task-env-12345"
}
```
```

--------------------------------

### Start Task Environment with Git Push

Source: https://vivaria.metr.org/tutorials/start-task-environment

Use this command when your Vivaria instance has Git support. It commits and pushes local changes before starting the environment.

```bash
cd path/to/my-tasks-repo
viv task start count_odds/main

```

--------------------------------

### Start Task Environment and Run Tests

Source: https://vivaria.metr.org/reference/cli

Use this command to start a task environment and execute tests within it. It supports options for specifying test files, controlling caching, enabling SSH access, and managing environment destruction. Ensure the taskId is provided.

```python
def test(
    self,
    taskId: str,  # noqa: ANN001, RUF100, N803 (CLI argument so can't change)
    test_name: str = "",
    dont_cache: bool = False,
    ssh: bool = False,
    ssh_user: SSHUser = "root",
    verbose: bool = False,
    task_family_path: str | None = None,
    env_file_path: str | None = None,
    destroy: bool = False,
    ignore_workdir: bool = False,
    k8s: bool | None = None,
) -> None:
    """Start a task environment and run tests.

    Args:
        taskId: The task to test.
        test_name: Test file to run tests from.
        dont_cache: Rebuild the task environment primary machine's Docker image from scratch.
        ssh: SSH into the task environment after starting it.
        ssh_user: User to SSH into the task environment as.
        verbose: Log the output of all tests, on success or failure.
        task_family_path: Path to a task family directory to use. If not provided, Vivaria may
            look up the task family directory from a Git repo that it's configured to use.
        env_file_path: Path to a file of environment variables that Vivaria will set in some
            TaskFamily methods. You can only provide this argument if you also provide
            task_family_path. If neither task_family_path nor env_file_path is provided,
            Vivaria will read environment variables from a file called secrets.env in a Git repo
            that Vivaria is configured to use.
        destroy: Destroy the task environment after running tests.
        ignore_workdir: Run tests on the current commit while ignoring any uncommitted
            changes.
        k8s: Start the task environment in a Kubernetes cluster.
    """
    if task_family_path is None:
        if env_file_path is not None:
            err_exit("env_file_path cannot be provided without task_family_path")

        task_source = self._setup_task_commit(ignore_workdir=ignore_workdir)
    else:
        task_source = viv_api.upload_task_family(
            task_family_path=pathlib.Path(task_family_path).expanduser(),
            env_file_path=pathlib.Path(env_file_path).expanduser()
            if env_file_path is not None
            else None,
        )

    response_lines = viv_api.start_task_test_environment(
        taskId,
        task_source,
        dont_cache,
        test_name,
        include_final_json=True,
        verbose=verbose,
        destroy_on_exit=destroy,
        k8s=k8s,
    )

    final_json = self._get_final_json_from_response(response_lines)
    if final_json is None:
        return

    test_status_code = final_json.get("testStatusCode")

    environment_name = final_json.get("environmentName")
    if environment_name is None:
        sys.exit(test_status_code or 0)

    _set_last_task_environment_name(environment_name)

    if ssh:
        self.ssh(environment_name=environment_name, user=ssh_user)
    else:
        sys.exit(test_status_code or 0)

```

--------------------------------

### Install the viv CLI

Source: https://vivaria.metr.org/tutorials/set-up-docker-compose

Installs the viv CLI and its dependencies using pip. This command should be run after activating the virtual environment.

```bash
pip install -e cli

```

--------------------------------

### Start Task Environment

Source: https://vivaria.metr.org/reference/cli

Starts a task environment for testing. It can optionally SSH into the environment, use a specific task family path, load environment variables from a file, destroy the environment after tests, run tests ignoring uncommitted changes, or start the environment in Kubernetes. If task_family_path is not provided, Vivaria attempts to infer it from a configured Git repository. If neither task_family_path nor env_file_path is provided, Vivaria reads environment variables from secrets.env in a configured Git repository.

```python
if task_family_path is None:
            if env_file_path is not None:
                err_exit("env_file_path cannot be provided without task_family_path")

            task_source = self._setup_task_commit(ignore_workdir=ignore_workdir)
        else:
            task_source = viv_api.upload_task_family(
                task_family_path=pathlib.Path(task_family_path).expanduser(),
                env_file_path=pathlib.Path(env_file_path).expanduser()
                if env_file_path is not None
                else None,
            )

        response_lines = viv_api.start_task_test_environment(
            taskId,
            task_source,
            dont_cache,
            test_name,
            include_final_json=True,
            verbose=verbose,
            destroy_on_exit=destroy,
            k8s=k8s,
        )

        final_json = self._get_final_json_from_response(response_lines)
        if final_json is None:
            return

        test_status_code = final_json.get("testStatusCode")

        environment_name = final_json.get("environmentName")
        if environment_name is None:
            sys.exit(test_status_code or 0)

        _set_last_task_environment_name(environment_name)

        if ssh:
            self.ssh(environment_name=environment_name, user=ssh_user)
        else:
            sys.exit(test_status_code or 0)
```

--------------------------------

### Get Run Details

Source: https://vivaria.metr.org/reference/cli

Fetches and displays the details of a specific run. The output is formatted as JSON.

```python
@typechecked
def get_run(self, run_id: int) -> None:
    """Get a run."""
    print(json.dumps(viv_api.get_run(run_id), indent=2))
```

--------------------------------

### Run Agent Task with Vivaria API

Source: https://vivaria.metr.org/reference/cli

This snippet demonstrates how to use the `viv_api.setup_and_run_agent` function to execute a task. It handles various configurations for agent starting state, task sources, and usage limits. Ensure that `agent_starting_state` and `agent_starting_state_file` are not specified simultaneously. If `task_family_path` is provided, it uses `viv_api.upload_task_family`; otherwise, it defaults to `viv_api.GitRepoTaskSource`.

```python
if agent_starting_state is not None and agent_starting_state_file is not None:
    err_exit("Cannot specify both agent starting state and agent starting state file")

agent_starting_state = agent_starting_state or agent_starting_state_file

starting_state = _get_input_json(agent_starting_state, "agent starting state")
settings_override = _get_input_json(agent_settings_override, "agent settings override")

task_parts = task.split("@")
task_id = task_parts[0]
task_branch = task_parts[1] if len(task_parts) > 1 else "main"

if batch_concurrency_limit is not None:
    if batch_name is None:
        err_exit("To use --batch-concurrency-limit, you must also specify --batch-name")
    if batch_concurrency_limit < 0:
        err_exit("--batch-concurrency-limit must not be negative")

if task_family_path is not None:
    task_source: viv_api.TaskSource = viv_api.upload_task_family(
        task_family_path=pathlib.Path(task_family_path).expanduser(),
        env_file_path=pathlib.Path(env_file_path).expanduser()
        if env_file_path is not None
        else None,
    )
else:
    task_source = viv_api.GitRepoTaskSource(
        type="gitRepo",
        repoName=task_repo or get_user_config().tasksRepoSlug,
        commitId=None,
    )

if priority is None and low_priority is not None:
    priority = "low" if low_priority else "high"

viv_api.setup_and_run_agent(
    {
        "agentRepoName": repo,
        "agentBranch": branch,
        "agentCommitId": commit,
        "uploadedAgentPath": uploaded_agent_path,
        "taskId": task_id,
        "taskBranch": task_branch,
        "name": name,
        "metadata": metadata,
        "usageLimits": {
            "tokens": max_tokens,
            "actions": max_actions,
            "total_seconds": max_total_seconds,
            "cost": max_cost,
        },
        "checkpoint": {
            "tokens": checkpoint_tokens,
            "actions": checkpoint_actions,
            "total_seconds": checkpoint_total_seconds,
            "cost": checkpoint_cost,
        },
        "requiresHumanIntervention": intervention,
        "agentStartingState": starting_state,
        "agentSettingsOverride": settings_override,
        "agentSettingsPack": agent_settings_pack,
        "priority": priority,
        # TODO: Stop sending isLowPriority once Vivaria instances stop expecting it.
        "isLowPriority": priority != "high",
        "parentRunId": parent,
        "batchName": str(batch_name) if batch_name is not None else None,
        "batchConcurrencyLimit": batch_concurrency_limit,
        "dangerouslyIgnoreGlobalLimits": dangerously_ignore_global_limits,
        "keepTaskEnvironmentRunning": keep_task_environment_running,
        "taskSource": task_source,
        "isK8s": k8s,
    },
    verbose=verbose,
    open_browser=open_browser,
)
```

--------------------------------

### Configure the viv CLI for Docker Compose (macOS/Linux)

Source: https://vivaria.metr.org/tutorials/set-up-docker-compose

Sets up the viv CLI to work with Docker Compose. This script should be run after installing the CLI.

```bash
./scripts/configure-cli-for-docker-compose.sh

```

--------------------------------

### Task Test

Source: https://vivaria.metr.org/reference/cli

Start a task environment and run tests. Allows for specifying test files, caching behavior, and SSH options.

```APIDOC
## Task Test

### Description
Start a task environment and run tests. Allows for specifying test files, caching behavior, and SSH options.

### Method
`viv task test`

### Parameters
#### Path Parameters
- **taskId** (string) - Required - The task to test.

#### Query Parameters
- **test_name** (string) - Optional - Test file to run tests from.
- **dont_cache** (boolean) - Optional - Rebuild the task environment primary machine's Docker image from scratch.
- **ssh** (boolean) - Optional - Whether to use SSH for running tests.
- **ssh_user** (SSHUser) - Optional - The user to use for SSH. Defaults to 'root'.
- **verbose** (boolean) - Optional - Enable verbose output.
- **task_family_path** (string) - Optional - Path to the task family.
- **env_file_path** (string) - Optional - Path to the environment file.
- **destroy** (boolean) - Optional - Destroy the task environment after tests are run.
- **ignore_workdir** (boolean) - Optional - Ignore the working directory.
- **k8s** (boolean) - Optional - Use Kubernetes for running tests.

### Request Example
```bash
viv task test my-task-id --test-name=test_module.py --dont-cache --ssh --verbose
```
```

--------------------------------

### run Command

Source: https://vivaria.metr.org/reference/cli

The `run` command is used to construct a task environment and execute an agent. It supports various configurations for task execution, resource limits, and environment setup.

```APIDOC
## run Command

### Description
Construct a task environment and run an agent in it. You can either run this command from a clone of an agent repo on your computer, or you can specify the repo, branch, and commit to use.

### Method
`run`

### Parameters
#### Path Parameters
- **task** (str) - Required - The task to run. Specified as `taskId@ref`, with the ref defaulting to `origin/main`. The ref can be a branch, tag, or commit.

#### Optional Parameters
- **path** (str | None) - The path to the git repo containing the agent code. Defaults to the current directory. Should not be specified if the `repo`, `branch`, and `commit` arguments, or the `agent_path` argument, are specified instead.
- **yes** (bool) - Whether to skip the confirmation prompt before starting the agent.
- **verbose** (bool) - Whether to print verbose output.
- **open_browser** (bool) - Whether to open the agent run page in the default browser.
- **max_tokens** (int) - The maximum number of tokens the agent can use. Defaults to 300,000.
- **max_actions** (int) - The maximum number of actions the agent can take. Defaults to 1,000.
- **max_total_seconds** (int) - The maximum number of seconds the agent can run for. Defaults to 604,800 (7 days).
- **max_cost** (float) - The maximum cost of the tokens the agent can use. The currency depends on the Vivaria installation you're using. Defaults to 100.
- **checkpoint_tokens** (int | None) - If provided, the agent will pause and wait for human input after using this many tokens.
- **checkpoint_actions** (int | None) - If provided, the agent will pause and wait for human input after this many actions.
- **checkpoint_total_seconds** (int | None) - If provided, the agent will pause and wait for human input after running for this many seconds.
- **checkpoint_cost** (float | None) - If provided, the agent will pause and wait for human input after incurring this much cost.
- **intervention** (bool) - Whether to allow human intervention during the agent's run.
- **agent_starting_state** (str | dict | None) - The starting state of the agent.
- **agent_starting_state_file** (str | None) - Path to a file containing the agent's starting state.
- **agent_settings_override** (str | dict | None) - Override agent settings.
- **agent_settings_pack** (str | None) - Path to a pack of agent settings.
- **name** (str | None) - The name of the agent run.
- **metadata** (dict[str, str]) - Metadata to associate with the agent run. Defaults to an empty dictionary.
- **repo** (str | None) - The repository to clone the agent from.
- **branch** (str | None) - The branch to use from the specified repository.
- **commit** (str | None) - The commit hash to use from the specified repository.
- **priority** (Literal['low', 'high'] | None) - The priority of the agent run.
- **low_priority** (bool | None) - Whether to run the agent with low priority.
- **parent** (int | None) - The ID of the parent agent run.
- **batch_name** (str | None) - The name of the batch this agent run belongs to.
- **batch_concurrency_limit** (int | None) - The concurrency limit for the batch.
- **dangerously_ignore_global_limits** (bool) - Whether to ignore global limits. Defaults to False.
- **keep_task_environment_running** (bool) - Whether to keep the task environment running after the agent finishes. Defaults to False.
- **agent_path** (str | None) - Path to the agent code.
- **task_family_path** (str | None) - Path to the task family.
- **env_file_path** (str | None) - Path to a file of environment variables that Vivaria will set in some TaskFamily methods. You can only provide this argument if you also provide task_family_path. If neither task_family_path nor env_file_path is provided, Vivaria will read environment variables from a file called secrets.env in a Git repo that Vivaria is configured to use.
- **k8s** (bool | None) - Run the agent in a Kubernetes cluster.
- **task_repo** (str | None) - Optionally specify the task repository. Should include the owner name, e.g. METR/mp4-tasks.

### Return Value
None
```

--------------------------------

### Update Run Using JSON File

Source: https://vivaria.metr.org/reference/cli

This example demonstrates updating a run using a JSON file that contains the update data. The file can include fields like 'score' and 'workPeriods'.

```bash
cat <<EOF > patch.json
{
   "score": 0.95,
   "workPeriods": [
       {"start": 1614556800000, "end": 1614556900000}
   ]
}
EOF
viv update-run 12345 "Setting work periods" patch.json
```

--------------------------------

### Setup Git Commit for Task

Source: https://vivaria.metr.org/reference/cli

Sets up a Git repository source for a task, creating a permalink to the current commit. Useful for tasks that depend on specific code versions.

```python
def _setup_task_commit(self, ignore_workdir: bool = False) -> viv_api.GitRepoTaskSource:
    """Set up git commit for task environment."""
    org, repo = gh.get_org_and_repo()
    _, commit, permalink = gh.create_working_tree_permalink(
        org=org, repo=repo, ignore_workdir=ignore_workdir
    )
    print("GitHub permalink to task commit:", permalink)
    return {"type": "gitRepo", "repoName": f"{org}/{repo}", "commitId": commit}

```

--------------------------------

### viv config get

Source: https://vivaria.metr.org/reference/cli

Retrieves the value of a specified configuration key from the user's configuration.

```APIDOC
## viv config get

### Description
Get the value of a config key.

### Parameters
#### Path Parameters
- **key** (str) - Required - The configuration key to retrieve.

### Request Example
```bash
viv config get <key>
```

### Response
#### Success Response (200)
- Prints the key and its corresponding value.

#### Response Example
```
<key>: <value>
```
```

--------------------------------

### Run Docker Compose with specific GID (macOS/Linux)

Source: https://vivaria.metr.org/tutorials/set-up-docker-compose

Starts Vivaria using Docker Compose, explicitly setting the VIVARIA_DOCKER_GID environment variable to resolve potential permission denied errors when connecting to the Docker daemon socket.

```bash
VIVARIA_DOCKER_GID=<number> docker compose up --pull always --detach --wait

```

--------------------------------

### ssh

Source: https://vivaria.metr.org/reference/cli

SSH into the agent container or aux VM for a run ID. Starts the agent container if necessary for agent containers, or uses the provisioned user for aux VMs.

```APIDOC
## ssh(run_id, user='root', aux_vm=False)

### Description
SSH into the agent container or aux VM for a run ID.

For agent containers: Starts the agent container if necessary and uses the given user (defaulting to root).

For aux VMs: Uses the provisioned aux VM user.

### Parameters
#### Path Parameters
- **run_id** (int) - Required - The ID of the run.
- **user** (SSHUser) - Optional - The user to connect as (default: 'root').
- **aux_vm** (bool) - Optional - Whether to SSH into the aux VM (default: False).
```

--------------------------------

### Start Agent from a Previous State

Source: https://vivaria.metr.org/tutorials/run-agent

Resume an agent's execution from a saved state by providing the path to a JSON file containing the state. This allows for continuity in long-running tasks.

```bash
viv run count_odds/main --agent_starting_state_file state.json

```

--------------------------------

### SSH into Agent Container or Aux VM

Source: https://vivaria.metr.org/reference/cli

Connects to the agent container or an auxiliary VM for a given run ID. Starts the agent container if it's not running. For aux VMs, it uses the provisioned user.

```python
def ssh(self, run_id: int, user: SSHUser = "root", aux_vm: bool = False) -> None:
    """SSH into the agent container or aux VM for a run ID.

    For agent containers: Starts the agent container if necessary and uses the given user
    (defaulting to root).

    For aux VMs: Uses the provisioned aux VM user.
    """
    if aux_vm:
        aux_vm_details = viv_api.get_aux_vm_details(run_id=run_id)
        with _temp_key_file(aux_vm_details) as f:
            opts = _aux_vm_ssh_opts(f.name, aux_vm_details)
            self._ssh.ssh(opts)
    else:
        viv_api.start_agent_container(run_id)
        ip_address = viv_api.get_agent_container_ip_address(run_id)
        env = viv_api.get_env_for_run(run_id, user)
        opts = _container_ssh_opts(ip_address, user, env)
        self._ssh.ssh(opts)
```

--------------------------------

### Submit Task Solution and Get Score

Source: https://vivaria.metr.org/tutorials/start-task-environment

Submit a solution to the task environment and receive a score. The submission is provided as a string argument.

```bash
viv task score --submission "2"

```

--------------------------------

### SSH into Run

Source: https://vivaria.metr.org/reference/cli

Initiates an SSH connection to the agent container or an auxiliary VM for a given run ID. It can start the agent container if necessary and uses the specified user.

```APIDOC
## ssh

### Description
SSH into the agent container or aux VM for a run ID. For agent containers, starts the agent container if necessary and uses the given user (defaulting to root). For aux VMs, uses the provisioned aux VM user.

### Parameters
#### Path Parameters
- **run_id** (int) - Required - ID of the run to SSH into.
- **user** (SSHUser) - Optional - User to SSH into the agent container as. Defaults to "root".
- **aux_vm** (bool) - Optional - Whether to SSH into the auxiliary VM. Defaults to false.
```

--------------------------------

### Configure the viv CLI for Docker Compose (Windows PowerShell)

Source: https://vivaria.metr.org/tutorials/set-up-docker-compose

Sets up the viv CLI to work with Docker Compose on Windows. This script should be run after installing the CLI.

```powershell
.\scripts\configure-cli-for-docker-compose.ps1

```

--------------------------------

### Initialize Pauser Class

Source: https://vivaria.metr.org/reference/pyhooks

Initializes the Pauser class with necessary environment configurations, a sleeper, a request function, and a flag to record pauses. The start time defaults to the current timestamp if not provided.

```python
class Pauser:
    """Manages delays in retrying RPCs, and sending pause/unpause requests to the server"""

    _envs: CommonEnvs
    _start: int
    _end: Optional[int]
    _state: State
    _sleeper: Sleeper
    _request_fn: RequestFn
    _record_pause: bool

    class State(Enum):
        NO_PAUSE = auto()
        PAUSE_REQUESTED = auto()
        PAUSE_FAILED = auto()
        PAUSE_SUCCEEDED = auto()

    def __init__(
        self,
        envs: CommonEnvs,
        sleeper: Sleeper,
        request_fn: RequestFn,
        record_pause: bool,
        start: int | None,
    ):
        self._envs = envs
        self._start = start if start is not None else timestamp_now()
        self._end = None
        self._state = self.State.NO_PAUSE
        self._sleeper = sleeper
        self._request_fn = request_fn
        self._record_pause = record_pause
```

--------------------------------

### SSH into Agent Container or Aux VM

Source: https://vivaria.metr.org/reference/cli

Connects to a run's agent container or auxiliary VM via SSH. For agent containers, it starts the container if needed. For aux VMs, it uses the provisioned user.

```python
aux_vm_details = viv_api.get_aux_vm_details(run_id=run_id)
            with _temp_key_file(aux_vm_details) as f:
                opts = _aux_vm_ssh_opts(f.name, aux_vm_details)
                self._ssh.ssh(opts)
```

```python
viv_api.start_agent_container(run_id)
            ip_address = viv_api.get_agent_container_ip_address(run_id)
            env = viv_api.get_env_for_run(run_id, user)
            opts = _container_ssh_opts(ip_address, user, env)
            self._ssh.ssh(opts)
```

--------------------------------

### `__init__`

Source: https://vivaria.metr.org/reference/cli

Initializes the CLI environment. It sets up global options and instantiates command groups.

```APIDOC
## `__init__`

### Description
Initialise the CLI.

### Parameters
* `dev` (bool) - Optional - If true, enables development mode.

### Source Code
`cli/viv_cli/main.py`
```

--------------------------------

### Read Task Instructions from Environment

Source: https://vivaria.metr.org/tutorials/start-task-environment

View the task instructions file located at /home/agent/instructions.txt from within the task environment.

```bash
cat /home/agent/instructions.txt

```

--------------------------------

### Initialize Task Environment

Source: https://vivaria.metr.org/reference/cli

Initializes the task command group, setting up necessary components like SSH clients.

```python
class Task:
    """Task environment management.

    Group within the CLI for managing task environments.
    """

    def __init__(self) -> None:
        """Initialize the task command group."""
        self._ssh = SSH()

```

--------------------------------

### frame

Source: https://vivaria.metr.org/reference/pyhooks

Decorator to automatically manage frame start and end.

```APIDOC
## `frame(self, name: str)`

### Description
A decorator that automatically calls `start_frame` before the decorated function executes and `end_frame` after it completes. This simplifies the management of tracing frames.

### Parameters
* `name` (str) - The name to assign to the frame.
```

--------------------------------

### List all config values

Source: https://vivaria.metr.org/reference/cli

Prints the user config path, current config, default config, and environment variable overrides.

```python
@typechecked
def list(self) -> None:
    """Print config and config path."""
    print(
        "user config path:",
        f"\t{user_config_path}",
        json.dumps(get_config_from_file(), indent=2),
        "",
        "default config:\n",
        json.dumps(default_config.model_dump(), indent=2),
        "",
        "environment variable overrides:",
        "\n".join(f"\t{k}: {v} ({os.environ.get(v, '')!r})" for k, v in env_overrides),
        sep="\n",
    )
    print(
        "\ncurrent config including env overrides:\n",
        json.dumps(get_user_config().model_dump(), indent=2),
    )
```

--------------------------------

### Get Run

Source: https://vivaria.metr.org/reference/cli

Retrieves and prints details of a specific run identified by its ID.

```APIDOC
## get_run

### Description
Get a run.

### Method
```python
def get_run(self, run_id: int) -> None
```

### Parameters
* **run_id** (int) - The ID of the run to retrieve.
```

--------------------------------

### get_input

Source: https://vivaria.metr.org/reference/pyhooks

Gets input from the user, falling back to default if not in intervention mode.

```APIDOC
## get_input

### Description
Gets input from the user. If not in intervention mode, it uses the `default_input`.

### Method
`get_input` (public method)

### Parameters
- **description** (string) - Required - A description of the input required.
- **default_input** (string) - Required - The default input to use if user input is not available.

### Returns
- **string** - The user-provided input or the default input.
```

--------------------------------

### Get Run Usage

Source: https://vivaria.metr.org/reference/cli

Retrieves and prints the time and token usage for a specific run.

```APIDOC
## get_run_usage

### Description
Get the time and token usage of an agent run.

### Method
```python
def get_run_usage(self, run_id: int, branch_number: int = 0) -> None
```

### Parameters
* **run_id** (int) - The ID of the run.
* **branch_number** (int, optional) - The branch number for usage. Defaults to 0.
```

--------------------------------

### Vivaria CLI Initialization

Source: https://vivaria.metr.org/reference/cli

Initializes the Vivaria CLI, optionally enabling developer mode.

```APIDOC
## Vivaria CLI Initialization

### Description
Initializes the Vivaria CLI, optionally enabling developer mode.

### Method
`__init__`

### Parameters
- **dev** (bool) - Optional - Enables developer mode.

### Attributes
- **config**: Manages configuration settings.
- **task**: Manages task-related operations.
- **run_batch**: Manages batch execution of tasks.
```

--------------------------------

### Get Run Status

Source: https://vivaria.metr.org/reference/cli

Retrieves and prints the status of a specific run identified by its ID.

```APIDOC
## get_run_status

### Description
Get the status of a run.

### Method
```python
def get_run_status(self, run_id: int) -> None
```

### Parameters
* **run_id** (int) - The ID of the run to get the status for.
```

--------------------------------

### Initialize Vivaria CLI

Source: https://vivaria.metr.org/reference/cli

Initializes the Command Line Interface for Vivaria. Sets the development mode.

```python
def __init__(self, dev: bool = False) -> None:
    """Initialise the CLI."""
    GlobalOptions.dev_mode = dev
    self._ssh = SSH()
    # Add groups of commands
    self.config = Config()
    self.task = Task()
    self.run_batch = RunBatch()
```

--------------------------------

### Get Run Status

Source: https://vivaria.metr.org/reference/cli

Retrieves and prints the status of a specific run in JSON format.

```python
print(json.dumps(viv_api.get_run_status(run_id), indent=2))
```

--------------------------------

### Vivaria CLI Main Entry Point

Source: https://vivaria.metr.org/reference/cli

The main entry point for the Vivaria CLI. It handles configuration loading, Sentry initialization, and command dispatching using `fire.Fire`.

```python
def main() -> None:
    """Main entry point for the CLI."""
    _move_old_config_files()

    # We can't use get_user_config here because the user's config might be invalid.
    config = get_user_config_dict()

    # TODO: improve type hints if Sentry releases their types
    def sentry_before_send(event: Any, hint: Any) -> Any:  # noqa: ANN401
        if "exc_info" in hint:
            _, exc_value, _ = hint["exc_info"]
            if isinstance(exc_value, KeyboardInterrupt):
                return None
        return event

    sentry_sdk.init(
        dsn=config.get("sentryDsn"),
        # Enable performance monitoring
        enable_tracing=True,
        traces_sample_rate=1.0,
        profiles_sample_rate=1.0,
        before_send=sentry_before_send,
    )
    sentry_sdk.set_tag("api_url", config.get("apiUrl"))
    try:
        fire.Fire(Vivaria)
    except TypeCheckError as e:
        err_exit(str(e))
```

--------------------------------

### Run a Task with Vivaria CLI

Source: https://vivaria.metr.org/reference/cli

Use the `run` command to construct a task environment and execute an agent. Specify the task ID and optionally a Git ref. You can control agent behavior with numerous parameters like token limits, action counts, and cost.

```python
run(
    task,
    path=None,
    yes=False,
    verbose=False,
    open_browser=False,
    max_tokens=300_000,
    max_actions=1_000,
    max_total_seconds=60 * 60 * 24 * 7,
    max_cost=100,
    checkpoint_tokens=None,
    checkpoint_actions=None,
    checkpoint_total_seconds=None,
    checkpoint_cost=None,
    intervention=False,
    agent_starting_state=None,
    agent_starting_state_file=None,
    agent_settings_override=None,
    agent_settings_pack=None,
    name=None,
    metadata={},
    repo=None,
    branch=None,
    commit=None,
    priority=None,
    low_priority=None,
    parent=None,
    batch_name=None,
    batch_concurrency_limit=None,
    dangerously_ignore_global_limits=False,
    keep_task_environment_running=False,
    agent_path=None,
    task_family_path=None,
    env_file_path=None,
    k8s=None,
    task_repo=None,
)
```

--------------------------------

### Get Run Status

Source: https://vivaria.metr.org/reference/cli

Retrieves the current status of a specified run. The output is formatted as JSON.

```python
@typechecked
def get_run_status(self, run_id: int) -> None:
    """Get the status of a run."""
    print(json.dumps(viv_api.get_run_status(run_id), indent=2))
```

--------------------------------

### Run Task

Source: https://vivaria.metr.org/reference/cli

Executes a specified task with various configuration options.

```APIDOC
## Run Task

### Description
Executes a specified task with various configuration options, including path, verbosity, and resource limits.

### Method
`run`

### Parameters
- **task** (str) - Required - The name or identifier of the task to run.
- **path** (str | None) - Optional - The path to the task or related files.
- **yes** (bool) - Optional - Skips confirmation prompts.
- **verbose** (bool) - Optional - Enables verbose output.
- **open_browser** (bool) - Optional - Opens the task in a web browser.
- **max_tokens** (int) - Optional - Maximum number of tokens for the task.
- **max_actions** (int) - Optional - Maximum number of actions the agent can perform.
- **max_total_seconds** (int) - Optional - Maximum execution time in seconds.
- **max_cost** (float) - Optional - Maximum cost allowed for the task.
- **checkpoint_tokens** (int | None) - Optional - Token count for checkpointing.
- **checkpoint_actions** (int | None) - Optional - Action count for checkpointing.
- **checkpoint_total_seconds** (int | None) - Optional - Time in seconds for checkpointing.
- **checkpoint_cost** (float | None) - Optional - Cost for checkpointing.
- **intervention** (bool) - Optional - Enables intervention mode.
- **agent_starting_state** (str | dict | None) - Optional - Specifies the initial state of the agent.
- **agent_starting_state_file** (str | None) - Optional - Path to a file containing the agent's starting state.
- **agent_settings_override** (str | dict | None) - Optional - Overrides agent settings.
- **agent_settings_pack** (str | None) - Optional - Specifies a pack of agent settings.
```

--------------------------------

### Get Agent State

Source: https://vivaria.metr.org/reference/cli

Retrieves and prints the last state of an agent within a specific run.

```APIDOC
## get_agent_state

### Description
Get the last state of an agent run.

### Method
```python
def get_agent_state(self, run_id: int, index: int, agent_branch_number: int = 0) -> None
```

### Parameters
* **run_id** (int) - The ID of the run.
* **index** (int) - The index of the agent within the run.
* **agent_branch_number** (int, optional) - The branch number of the agent. Defaults to 0.
```

--------------------------------

### Get a Specific Run

Source: https://vivaria.metr.org/reference/cli

Retrieves and prints a specific run's details in JSON format.

```python
print(json.dumps(viv_api.get_run(run_id), indent=2))
```

--------------------------------

### Define File Build Step Specification

Source: https://vivaria.metr.org/reference/metr_task_standard

Use this TypedDict to define a file build step, specifying the source file path and its destination in the VM.

```python
class FileBuildStep(TypedDict):
    """
    A specification for a file build step.

    Attributes:
        type (Literal["file"]): The type of build step.
        source (str): A relative file path in the task family directory.
        destination (str): A file path in the VM (absolute or relative to /root).
    """

    type: Literal["file"]
    source: str
    destination: str
```

--------------------------------

### Get config value

Source: https://vivaria.metr.org/reference/cli

Retrieves the value of a specific configuration key. Exits if the key is not set.

```python
@typechecked
def get(self, key: str) -> None:
    """Get the value of a config key."""
    # Not get_user_config().dict() so that we can still get values if the config is invalid
    user_config = get_user_config_dict()
    if key not in user_config:
        err_exit(f"{key} not set")
    print(f"{key}: {user_config[key]}")
```

--------------------------------

### Create a virtual environment for the viv CLI (macOS/Linux)

Source: https://vivaria.metr.org/tutorials/set-up-docker-compose

Creates and activates a Python virtual environment named 'viv' in the ~/.venvs directory. Ensure Python 3.11 or higher is available.

```bash
mkdir -p ~/.venvs && python3 -m venv ~/.venvs/viv && source ~/.venvs/viv/bin/activate

```
Tokens:
Raw