### Setup Gradescope Autograder Environment (Bash)

Source: https://otter-grader.readthedocs.io/en/latest/workflow/otter_generate/container_image

The `setup.sh` script is required by Gradescope to install necessary software and dependencies for the autograder. It installs packages like Pandoc and Mamba, sets up the PATH, and creates a Conda environment using `environment.yml`.

```bash
#!/usr/bin/env bash

export DEBIAN_FRONTEND=noninteractive
apt-get clean
apt-get update
apt-get install -y wget texlive-xetex texlive-fonts-recommended texlive-plain-generic \
    build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libgit2-dev texlive-lang-chinese

# install pandoc
wget -nv https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb \
    -O /tmp/pandoc.deb
dpkg -i /tmp/pandoc.deb

# install mamba
if [ $(uname -p) = "arm" ] || [ $(uname -p) = "aarch64" ] ; \
    then wget -nv https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh \
        -O /autograder/source/mamba_install.sh ; \
    else wget -nv https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh \
        -O /autograder/source/mamba_install.sh ; \
fi
chmod +x /autograder/source/mamba_install.sh
/autograder/source/mamba_install.sh -b
echo "export PATH=/root/miniforge3/bin:\$PATH" >> /root/.bashrc

export PATH=/root/miniforge3/bin:$PATH
export TAR="/bin/tar"

# install dependencies with mamba
mamba env create -f /autograder/source/environment.yml
mamba install -n otter-env -c conda-forge nb_conda_kernels
mamba run -n otter-env bash -c "playwright install-deps && playwright install chromium"

```

--------------------------------

### Otter Plugin Configuration Example

Source: https://otter-grader.readthedocs.io/en/latest/plugins/creating_plugins

Example JSON structure for configuring Otter plugins in `otter_config.json`. It demonstrates how to specify plugin packages, either as strings for default configurations or as dictionaries for custom metadata.

```json
{
    "plugins": [
        {
            "my_otter_plugin_package.MyOtterPlugin": {
                "some_metadata_key": "some_value"
            }
        },
        "my_otter_plugin_package.MyOtherOtterPlugin"
    ]
}
```

--------------------------------

### Tutorial Directory Structure

Source: https://otter-grader.readthedocs.io/en/latest/tutorial

Illustrates the expected file and directory layout for the Otter-Grader tutorial, including the master notebook, requirements file, and submission examples.

```text
tutorial
├── demo.ipynb
├── requirements.txt
└── submissions
    ├── ipynbs
    │   ├── demo-fails1.ipynb
    │   ├── demo-fails2.ipynb
    │   ├── demo-fails2Hidden.ipynb
    │   ├── demo-fails3.ipynb
    │   ├── demo-fails3Hidden.ipynb
    │   └── demo-passesAll.ipynb
    └── zips
        ├── demo-fails1.zip
        ├── demo-fails2.zip
        ├── demo-fails2Hidden.zip
        ├── demo-fails3.zip
        ├── demo-fails3Hidden.zip
        └── demo-passesAll.zip
```

--------------------------------

### R Markdown BEGIN TESTS Example

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/rmd_format

Demonstrates how to use HTML comments to mark the start of test cells in R Markdown for Otter Assign.

```R Markdown
<!-- # BEGIN TESTS -->
```

--------------------------------

### Otter Grade Output Example

Source: https://otter-grader.readthedocs.io/en/latest/_sources/tutorial

Shows the expected format of the 'final_grades.csv' file generated by the 'otter grade' command, detailing scores for each submission.

```text
file,q1,q2,q3
fails3Hidden.ipynb,1.0,1.0,0.5
passesAll.ipynb,1.0,1.0,1.0
fails1.ipynb,0.6666666666666666,1.0,1.0
fails2Hidden.ipynb,1.0,0.5,1.0
fails3.ipynb,1.0,1.0,0.375
fails2.ipynb,1.0,0.0,1.0
```

--------------------------------

### Otter Assignment Configuration for Environment Saving

Source: https://otter-grader.readthedocs.io/en/latest/logging

An example Otter configuration file that specifies the notebook name and enables the saving of the execution environment. This is crucial for grading assignments in environments that may differ from the student's original setup.

```json
{
    "notebook": "hw00.ipynb",
    "save_environment": true
}
```

--------------------------------

### Example Directory Structure

Source: https://otter-grader.readthedocs.io/en/latest/workflow/otter_generate/index

This illustrates a typical directory structure for an assignment development folder, including source notebooks, test files, data, and requirements.

```text
hw00-dev
├── data.csv
├── hidden-tests
│   ├── q1.py
│   └── q2.py  # etc.
├── hw00-sol.ipynb
├── hw00.ipynb
├── requirements.txt
├── tests
│   ├── q1.py
│   └── q2.py  # etc.
└── utils.py
```

--------------------------------

### Otter Assign Command Usage

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_assign/usage

Demonstrates the basic command-line syntax for Otter Assign, specifying the master notebook and the output directory. Includes examples for both Python and R notebooks.

```bash
otter assign hw00.ipynb dist
```

```bash
otter assign hw00.Rmd dist
```

```bash
otter assign --no-run-tests hw00.ipynb dist
```

--------------------------------

### Python Solution Removal Example (Student View)

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_assign/notebook_format

Illustrates how the Python code from the previous example appears to students after Otter Assign applies solution removal rules.

```python
def square(x):
    ...

nine = ...
```

--------------------------------

### R Assignment Setup Script

Source: https://otter-grader.readthedocs.io/en/latest/_sources/workflow/otter_generate/container_image

The shell script template used for setting up the environment for R assignments on Gradescope. It installs necessary software and dependencies, including R packages if specified.

```shell
.. literalinclude:: ../../_static/r_setup.sh
    :language: shell
```

--------------------------------

### Python Assignment Setup Script

Source: https://otter-grader.readthedocs.io/en/latest/_sources/workflow/otter_generate/container_image

The shell script template used for setting up the environment for Python assignments on Gradescope. It installs necessary software and dependencies, including R packages if specified.

```shell
.. literalinclude:: ../../_static/python_setup.sh
    :language: shell
```

--------------------------------

### Notebook Seeding Example (Original)

Source: https://otter-grader.readthedocs.io/en/latest/seeding

Illustrates the original code within notebook cells before the autograder injects seeding commands. This shows the state of the code as written by the student.

```python
x = 2 ** np.arange(5)
```

```python
y = np.random.normal(100)
```

--------------------------------

### Install Otter Grader with pip

Source: https://otter-grader.readthedocs.io/en/latest/_sources/index

Installs the Otter Grader Python package using pip, the standard package installer for Python.

```console
pip install otter-grader
```

--------------------------------

### R Example with Function Definition

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_assign/notebook_format

Demonstrates an R code block defining a 'square' function and its usage.

```r
# BEGIN SOLUTION
square <- function(x) {
    return(x ^ 2)
}
# END SOLUTION
x2 <- square(25)
```

--------------------------------

### Otter Generate CLI Usage Examples

Source: https://otter-grader.readthedocs.io/en/latest/workflow/otter_generate/index

Demonstrates various ways to use the 'otter generate' command to create autograder zip files. It covers default usage, specifying data files, custom test directories, additional Python scripts, and R assignments.

```bash
otter generate
```

```bash
otter generate data.csv
```

```bash
otter generate -t hidden-tests data.csv
```

```bash
otter generate -t hidden-tests data.csv utils.py
```

```bash
otter generate -t hidden-tests -l r data.csv
```

--------------------------------

### Bash Script for R Assignment Setup

Source: https://otter-grader.readthedocs.io/en/latest/workflow/otter_generate/container_image

A comprehensive bash script to prepare an autograder environment for R assignments. It installs essential system packages, pandoc, mamba, and then uses mamba to create and populate a dedicated conda environment with R dependencies specified in an environment.yml file.

```bash
#!/usr/bin/env bash

export DEBIAN_FRONTEND=noninteractive
apt-get clean
apt-get update
apt-get install -y wget texlive-xetex texlive-fonts-recommended texlive-plain-generic \
    build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libgit2-dev texlive-lang-chinese
apt-get install -y libnlopt-dev cmake libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev \
    apt-utils libpoppler-cpp-dev libavfilter-dev  libharfbuzz-dev libfribidi-dev imagemagick \
    libmagick++-dev texlive-xetex texlive-fonts-recommended texlive-plain-generic \
    build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libgit2-dev texlive-lang-chinese \
    libxft-dev

# install pandoc
wget -nv https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb \
    -O /tmp/pandoc.deb
dpkg -i /tmp/pandoc.deb

# install mamba
if [ $(uname -p) = "arm" ] || [ $(uname -p) = "aarch64" ] ; \
    then wget -nv https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh \
        -O /autograder/source/mamba_install.sh ; \
    else wget -nv https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh \
        -O /autograder/source/mamba_install.sh ; \
fi
chmod +x /autograder/source/mamba_install.sh
/autograder/source/mamba_install.sh -b
echo "export PATH=/root/miniforge3/bin:\$PATH" >> /root/.bashrc

export PATH=/root/miniforge3/bin:$PATH
export TAR="/bin/tar"

# install dependencies with mamba
mamba env create -f /autograder/source/environment.yml
mamba install -n otter-env -c conda-forge nb_conda_kernels
mamba run -n otter-env bash -c "playwright install-deps && playwright install chromium"
mamba run -n otter-env Rscript /autograder/source/requirements.r

# set mamba shell
mamba shell init --shell bash

```

--------------------------------

### Define R Dependencies (R)

Source: https://otter-grader.readthedocs.io/en/latest/workflow/otter_generate/container_image

The `requirements.r` file lists R packages that need to be installed for assignments that use R. This file is used by the autograder's setup process to ensure all necessary R libraries are available.

```r
# Example content for requirements.r
install.packages("tidyverse", repos = "http://cran.us.r-project.org")
install.packages("rmarkdown", repos = "http://cran.us.r-project.org")

```

--------------------------------

### Mamba Commands for Environment Management

Source: https://otter-grader.readthedocs.io/en/latest/workflow/otter_generate/container_image

Demonstrates common Mamba commands used for initializing the shell, creating environments from YAML files, installing packages, and running commands within specific environments. These are essential for managing dependencies in the autograder.

```APIDOC
Mamba Shell Initialization:
  mamba shell init --shell bash
    Initializes Mamba's shell integration for the current session or user profile.

Mamba Environment Creation:
  mamba env create -f /path/to/environment.yml
    Creates a new conda environment based on the specifications in the provided YAML file.
    Parameters:
      -f, --file: Path to the environment.yml file.

Mamba Package Installation:
  mamba install -n <env_name> <package_name>
    Installs a specific package into a named conda environment.
    Parameters:
      -n, --name: The name of the target environment.
      <package_name>: The name of the package to install.

Mamba Run Command:
  mamba run -n <env_name> <command>
    Executes a command within the specified conda environment.
    Parameters:
      -n, --name: The name of the environment to activate before running the command.
      <command>: The command and its arguments to execute.
    Example:
      mamba run -n otter-env bash -c "playwright install-deps && playwright install chromium"
        Installs Playwright dependencies and browsers within the 'otter-env' environment.
```

--------------------------------

### Install Otter Grader (Python)

Source: https://otter-grader.readthedocs.io/en/latest/index

Installs the Otter Grader Python package using pipx or pip. This command makes the `otter` binary available for command-line use. It requires Python 3.9+.

```bash
pipx install otter-grader

# or
pip install otter-grader
```

--------------------------------

### Install Otter Grader with pipx

Source: https://otter-grader.readthedocs.io/en/latest/_sources/index

Installs the Otter Grader Python package using pipx, a tool for installing and running Python applications in isolated environments.

```console
pipx install otter-grader
```

--------------------------------

### Python Doctest: Simple Assertion

Source: https://otter-grader.readthedocs.io/en/latest/test_files/python_format

An example of a Python doctest case that asserts the equality of two values. It demonstrates the basic `>>>` prompt and expected output format.

```Python
>>> 1 == 1
True
```

--------------------------------

### Manually-Graded Written Question Example (With Prompt)

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/rmd_format

An example of a manually-graded written question that includes a custom prompt. The prompt is delimited using HTML comments, followed by the solution block.

```HTML
<!--
# BEGIN QUESTION
name: q6
manual: true
-->

**Question 6:** Fill in the blank.

<!-- # BEGIN PROMPT -->

The mitochondria is the ___________ of the cell.

<!-- # END PROMPT -->

<!-- # BEGIN SOLUTION-->

powerhouse

<!-- # END SOLUTION -->

<!-- # END QUESTION -->

```

--------------------------------

### R Seeding Example

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/notebook_format

Demonstrates how to set a random number generator seed in R using `set.seed`.

```r
set.seed(rng_seed)
runif(1000)
```

--------------------------------

### Python Example with Print Statement

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_assign/notebook_format

Illustrates a Python code block with a print statement, potentially used in an assignment context.

```python
pi = 3.14
if True:
    ...
    print('A circle with radius', radius, 'has area', area)

def circumference(r):
    # Next, define a circumference function.
    pass
```

--------------------------------

### Configure Rate Limiting (JSON)

Source: https://otter-grader.readthedocs.io/en/latest/_sources/plugins/builtin/rate_limiting

Example JSON configuration for the RateLimiting plugin in `otter_config.json`. This sets the maximum allowed submissions and the time window duration.

```json
{
    "plugins": [
        {
            "otter.plugins.builtin.RateLimiting": {
                "allowed_submissions": 5,
                "days": 1
            }
        }
    ]
}
```

--------------------------------

### Python Seeding Example

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/notebook_format

Demonstrates how to set a random number generator seed in Python using a variable that will be overwritten by Otter Assign.

```python
import numpy as np
rng_seed = 42
```

--------------------------------

### Python Script Seeding Example (Seeded)

Source: https://otter-grader.readthedocs.io/en/latest/seeding

Shows a Python script after the autograder has prepended seeding commands for NumPy and the random module. This ensures the entire script execution is reproducible.

```python
np.random.seed(42)
random.seed(42)
import numpy as np

def sigmoid(t):
    return 1 / (1 + np.exp(-1 * t))
```

--------------------------------

### Autograded Question Configuration

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_assign/notebook_format

Example configuration for an autograded question named 'q1' that should be included in the filtered PDF.

```yaml
# BEGIN QUESTION
name: q1
export: true
```

--------------------------------

### Sample Exception-Based Test File Structure

Source: https://otter-grader.readthedocs.io/en/latest/test_files/python_format

Provides an example of an exception-based test file using Otter's `@test_case` decorator. It shows how to define test cases with specific arguments (like 'sieve' or 'env') and includes assertions for testing a student's 'sieve' function.

```python
from otter.test_files import test_case

OK_FORMAT = False

name = "q1"
points = 2

@test_case()
def test_low_primes(sieve):
    assert sieve(1) == set()
    assert sieve(2) == {2}
    assert sieve(3) == {3}

@test_case(points=2, hidden=True)
def test_higher_primes(env):
    assert env["sieve"](20) == {2, 3, 5, 7, 11, 13, 17, 19}
    assert sieve(100) == {2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59,
        61, 67, 71, 73, 79, 83, 89, 97}
```

--------------------------------

### Python Doctest: Loop and Conditional

Source: https://otter-grader.readthedocs.io/en/latest/test_files/python_format

An example of a Python doctest case involving a loop and a conditional statement. It shows how to handle multi-line output and the `...` continuation prompt.

```Python
>>> for i in range(4):
...     print(i == 1)
False
True
False
False
```

--------------------------------

### Python Script Library Seeding Example (Python)

Source: https://otter-grader.readthedocs.io/en/latest/_sources/seeding

Shows how Otter seeds Python scripts by prepending seeding commands for `numpy` and `random` at the beginning of the script. This applies a consistent seed to the entire script execution.

```python
np.random.seed(42)
random.seed(42)
import numpy as np

def sigmoid(t):
    return 1 / (1 + np.exp(-1 * t))
```

--------------------------------

### Python Script Seeding Example (Original)

Source: https://otter-grader.readthedocs.io/en/latest/seeding

Presents an original Python script that will be processed by the autograder. This code snippet shows a typical function definition before any seeding is applied.

```python
import numpy as np

def sigmoid(t):
    return 1 / (1 + np.exp(-1 * t))
```

--------------------------------

### Notebook Library Seeding Example (Python)

Source: https://otter-grader.readthedocs.io/en/latest/_sources/seeding

Demonstrates how Otter injects seeding code for libraries like NumPy and `random` into notebook cells. This ensures that all computations within a cell are seeded with the instructor-provided value.

```python
np.random.seed(42)
random.seed(42)
x = 2 ** np.arange(5)
```

--------------------------------

### R Markdown Assignment Config Example

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/rmd_format

Illustrates specifying assignment generation arguments, such as init_cell and export_cell, using HTML comments in R Markdown.

```R Markdown
<!--
# ASSIGNMENT CONFIG
init_cell: false
export_cell: true
generate: true
# etc.
-->
```

--------------------------------

### Otter Assign Question Configuration Usage

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/notebook_format

Example of how to specify question configuration, including the name and export setting, within an Otter Assign notebook.

```python
# BEGIN QUESTION
name: q1
export: true


```

--------------------------------

### Notebook Seeding Example (Seeded)

Source: https://otter-grader.readthedocs.io/en/latest/seeding

Demonstrates how notebook cells are modified by the autograder to include seeding for both NumPy and the standard random library. This ensures that the execution of each cell is reproducible with a specific seed.

```python
np.random.seed(42)
random.seed(42)
x = 2 ** np.arange(5)
```

```python
np.random.seed(42)
random.seed(42)
y = np.random.normal(100)
```

--------------------------------

### Rate Limiting Plugin Configuration

Source: https://otter-grader.readthedocs.io/en/latest/plugins/builtin/rate_limiting

Example of how to configure the Rate Limiting plugin in `otter_config.json` to allow 5 submissions per day. This JSON structure defines the plugin and its parameters.

```JSON
{
    "plugins": [
        {
            "otter.plugins.builtin.RateLimiting": {
                "allowed_submissions": 5,
                "days": 1
            }
        }
    ]
}
```

--------------------------------

### Python Solution Removal Example (Original)

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_assign/notebook_format

Demonstrates how Otter Assign's solution removal rules are applied to Python code, showing lines marked with '# SOLUTION' or '# SOLUTION NO PROMPT'.

```python
def square(x):
    y = x * x # SOLUTION NO PROMPT
    return y # SOLUTION

nine = square(3) # SOLUTION
```

--------------------------------

### Otter Generate CLI Usage

Source: https://otter-grader.readthedocs.io/en/latest/_sources/workflow/otter_generate/index

Examples of using the `otter generate` command to create autograder zip files. This utility packages tests, requirements, and other necessary files for grading. It supports specifying test directories, including requirements.txt, adding arbitrary files, and indicating the assignment language (e.g., R).

```console
otter generate
```

```console
otter generate data.csv
```

```console
otter generate -t hidden-tests data.csv
```

```console
otter generate -t hidden-tests data.csv utils.py
```

```console
otter generate -t hidden-tests -l r data.csv
```

--------------------------------

### Otter Environment Serialization Configuration (JSON)

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_check/dot_otter_files

An example of an Otter configuration file (.otter) demonstrating how to enable environment serialization and specify variable types for restricted serialization. This is crucial for grading assignments from serialized environments.

```json
{
    "notebook": "hw00.ipynb",
    "save_environment": true,
    "variables": {
        "df": "pandas.core.frame.DataFrame",
        "fn": "builtins.function",
        "arr": "numpy.ndarray"
    }
}
```

--------------------------------

### Plugin Initialization Context

Source: https://otter-grader.readthedocs.io/en/latest/plugins/creating_plugins

Plugins are instantiated with context arguments that provide information about the submission and configuration. The default __init__ method stores these in instance variables.

```APIDOC
Plugin.__init__(submission_path: str, submission_metadata: dict, plugin_config: dict)
  - Initializes a plugin instance.
  - Parameters:
    - submission_path: The absolute path to the submission being executed.
    - submission_metadata: Information about the submission, read from submission_metadata.json.
    - plugin_config: Parsed configurations from the 'plugins' key in otter_config.json.
  - Note: Some events may not have these arguments available and will receive falsey defaults (e.g., {} for submission_metadata).
```

--------------------------------

### Get Variable Type Example (IPython)

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_check/dot_otter_files

Demonstrates the usage of the otter.utils.get_variable_type function in an IPython environment. This function returns the fully-qualified type string of an object, which is used in .otter files for variable serialization.

```python
from otter.utils import get_variable_type
import numpy as np
import pandas as pd
get_variable_type(np.array([]))
get_variable_type(pd.DataFrame())
fn = lambda x: x
get_variable_type(fn)
```

--------------------------------

### Python get_variable_type Example for Serialization

Source: https://otter-grader.readthedocs.io/en/latest/otter_check/dot_otter_files

Demonstrates using `otter.utils.get_variable_type` to get fully-qualified type strings for Python objects like NumPy arrays, Pandas DataFrames, and functions. This is crucial for specifying variable types during environment serialization.

```python
In [1]: from otter.utils import get_variable_type

In [2]: import numpy as np

In [3]: import pandas as pd

In [4]: get_variable_type(np.array([]))
Out[4]: 'numpy.ndarray'

In [5]: get_variable_type(pd.DataFrame())
Out[5]: 'pandas.core.frame.DataFrame'

In [6]: fn = lambda x: x

In [7]: get_variable_type(fn)
Out[7]: 'builtins.function'
```

--------------------------------

### Run Otter Assign Command

Source: https://otter-grader.readthedocs.io/en/latest/tutorial

Demonstrates how to execute the `otter assign` command to process a master notebook and generate assignment distribution files.

```shell
otter assign demo.ipynb dist
```

--------------------------------

### Otter Configuration for Environment Saving

Source: https://otter-grader.readthedocs.io/en/latest/_sources/logging

An example JSON configuration file for Otter, specifying the notebook to be graded and enabling the saving of execution environments. This is crucial for grading assignments using saved environments.

```json
{
    "notebook": "hw00.ipynb",
    "save_environment": true
}
```

--------------------------------

### Otter Assign Output Structure

Source: https://otter-grader.readthedocs.io/en/latest/_sources/tutorial

Illustrates the directory structure created by the 'otter assign' command, containing autograder files and student notebooks.

```text
tutorial/dist
    ├── autograder
    │   ├── autograder.zip
    │   ├── demo-sol.pdf
    │   ├── demo.ipynb
    │   ├── otter_config.json
    │   └── requirements.txt
    └── student
        └── demo.ipynb
```

--------------------------------

### R Test Structure Example

Source: https://otter-grader.readthedocs.io/en/latest/_sources/test_files/r_format

Demonstrates the structure of an R test file using the ottr package. It defines a global 'test' variable as a list containing ottr::TestCase instances, each with a name and a code block for assertions.

```r
test = list(
    name = "q1",
    cases = list(
        ottr::TestCase$new(
            name = "q1a",
            code = {
                testthat::expect_true(ans.1 > 1)
                testthat::expect_true(ans.1 < 2)
            }
        ),
        ottr::TestCase$new(
            name = "q1b",
            hidden = TRUE,
            code = {
                tol = 1e-5
                actual_answer = 1.27324
                testthat::expect_true(ans.1 > actual_answer - tol)
                testthat::expect_true(ans.1 < actual_answer + tol)
            }
        )
    )
)
```

--------------------------------

### Initialize Otter Notebook Grader

Source: https://otter-grader.readthedocs.io/en/latest/otter_check/index

Demonstrates how to import the Otter library and instantiate the `Notebook` class. The `Notebook` class can be initialized with an optional path to the directory containing test files, defaulting to './tests'.

```python
import otter
grader = otter.Notebook()
```

```python
grader = otter.Notebook("hw00-tests")
```

--------------------------------

### Install ottr R Package

Source: https://otter-grader.readthedocs.io/en/latest/_sources/index

Installs the 'ottr' R package, which is required for autograding R assignments with Otter Grader.

```r
install.packages("ottr")
```

--------------------------------

### Install Ottr Package (R)

Source: https://otter-grader.readthedocs.io/en/latest/index

Installs the `ottr` R package, which is required for autograding R assignments. This command should be run within an R environment.

```r
install.packages("ottr")
```

--------------------------------

### Get Variable Type String

Source: https://otter-grader.readthedocs.io/en/latest/logging

Utility function to get the fully-qualified type string for a Python object, used for selective environment saving.

```python
import pandas as pd
from otter.utils import get_variable_type

df = pd.DataFrame()
variable_type = get_variable_type(df)
print(variable_type) # Output: 'pandas.core.frame.DataFrame'
```

--------------------------------

### notebook_export Plugin Event

Source: https://otter-grader.readthedocs.io/en/latest/plugins/creating_plugins

This event is called when a student calls `otter.Notebook.add_plugin_files`. It should return a list of file paths to be included in the exported zip file.

```APIDOC
notebook_export(*args, **kwargs) -> list[str]
  - Called when a student calls `otter.Notebook.add_plugin_files`.
  - Receives additional args and kwargs passed to `Notebook.add_plugin_files`.
  - Does NOT receive submission_path, submission_metadata, or plugin_config.
  - Must return a list of strings corresponding to file paths to be included in the export zip file.
  - All necessary information must be gathered from the arguments.
```

--------------------------------

### Plugin Configuration with Parameters

Source: https://otter-grader.readthedocs.io/en/latest/_sources/plugins/index

Configures plugins by mapping their importable names to specific configuration dictionaries within the 'plugins' key of otter_config.json.

```json
{
    "plugins": [
        {
            "mypackage.MyOtterPlugin": {
                "key1": "value1",
                "key2": "value2"
            }
        },
        "mypackage.MyOtherOtterPlugin"
    ]
}
```

--------------------------------

### Manually-Graded Code Question Example

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/rmd_format

An example of a manually-graded question in R Markdown that involves generating a plot. It includes question metadata, a prompt, and a solution block with R code.

```HTML
<!--
# BEGIN QUESTION
name: q7
manual: true
-->

**Question 7:** Plot $f(x) = \cos e^x$ on $[0,10]$.

<!-- # BEGIN SOLUTION -->

```{r}
# BEGIN SOLUTION
x = seq(0, 10, 0.01)
y = cos(exp(x))
ggplot(data.frame(x, y), aes(x=x, y=y)) +
    geom_line()
# END SOLUTION
```

<!-- # END SOLUTION -->

<!-- # END QUESTION -->

```

--------------------------------

### Otter Generate Command

Source: https://otter-grader.readthedocs.io/en/latest/_sources/tutorial

Generates a zip file containing tests and requirements from a directory. This is typically done invisibly by Otter Assign but can be run manually.

```bash
otter generate
```

--------------------------------

### Otter Assign Output Directory Structure

Source: https://otter-grader.readthedocs.io/en/latest/tutorial

Shows the structure of the `dist` directory created by `otter assign`, containing subdirectories for autograder files and student notebooks.

```text
tutorial/dist
├── autograder
│   ├── autograder.zip
│   ├── demo-sol.pdf
│   ├── demo.ipynb
│   ├── otter_config.json
│   └── requirements.txt
└── student
    └── demo.ipynb
```

--------------------------------

### Configure Plugins

Source: https://otter-grader.readthedocs.io/en/latest/workflow/otter_generate/index

Lists plugins and their configurations in the `plugins` key. Plugins without configurations are listed as strings, while those requiring configuration are dictionaries mapping the plugin name to its configuration subdictionary.

```JSON
{
    "plugins": [
        "somepackage.SomeOtterPlugin",
        {
            "somepackage.SomeOtherOtterPlugin": {
            "some_config_key": "some_config_value"
            }
        }
    ]
}
```

--------------------------------

### from_notebook Plugin Event

Source: https://otter-grader.readthedocs.io/en/latest/plugins/creating_plugins

This event is triggered when a student calls `otter.Notebook.run_plugin`. It receives any additional arguments passed to `run_plugin` and should not rely on submission context.

```APIDOC
from_notebook(*args, **kwargs)
  - Called when a student calls `otter.Notebook.run_plugin`.
  - Receives additional args and kwargs passed to `Notebook.run_plugin`.
  - Does NOT receive submission_path, submission_metadata, or plugin_config.
  - Should not return anything but can modify state or provide feedback.
  - All necessary information must be gathered from the arguments.
```

--------------------------------

### Manually-Graded Written Question Example (No Prompt)

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/rmd_format

An example of a manually-graded written question without a custom prompt. Otter Assign automatically replaces the solution with placeholder text if no prompt is provided.

```HTML
<!--
# BEGIN QUESTION
name: q5
manual: true
-->

**Question 5:** Simplify $\sum_{i=1}^n n$.

<!-- # BEGIN SOLUTION -->

$\frac{n(n+1)}{2}$

<!-- # END SOLUTION -->

<!-- # END QUESTION -->

```

--------------------------------

### Otter Assign Command

Source: https://otter-grader.readthedocs.io/en/latest/_sources/tutorial

Assigns an Otter assignment by processing a master notebook. It generates an autograder zip file, solutions PDF, and student notebooks. Requires the notebook file as input and an output directory.

```bash
otter assign demo.ipynb dist
```

--------------------------------

### Grade Zip Submissions with Otter Grade

Source: https://otter-grader.readthedocs.io/en/latest/_sources/tutorial

This command grades zip file exports from Otter's `Notebook.export` method. It specifies the autograder files, enables verbose output, indicates that submissions are zip files using `--ext zip`, and points to the directory containing the zip submissions.

```console
otter grade -n demo -a dist/autograder/demo-autograder_*.zip -v --ext zip submissions/zips
```

--------------------------------

### R Test Cell Examples

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/rmd_format

Provides examples of test cells in R Markdown using the `testthat` package. These cells should raise an error if the test fails, ensuring correct assignment grading.

```R
testthat::expect_true(some_bool)

```

```R
testthat::expect_equal(some_value, 1.04)

```

--------------------------------

### R Markdown Autograded Question Example

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/rmd_format

A comprehensive example of an autograded question in R Markdown, including question configuration, solution code, and test code, all wrapped in HTML comments.

```R Markdown
<!--
# BEGIN QUESTION
name: q1
manual: false
points:
    - 1
    - 1
-->

**Question 1:** Find the radius of a circle that has a 90 deg. arc of length 2. Assign this
value to `ans.1`

<!-- # BEGIN SOLUTION -->

```{r}
ans.1 <- 2 * 2 * pi * 2 / pi / pi / 2 # SOLUTION
```

<!-- # END SOLUTION -->

<!-- # BEGIN TESTS -->

```{r}
expect_true(ans.1 > 1)
expect_true(ans.1 < 2)
```

```{r}
# HIDDEN
tol = 1e-5
actual_answer = 1.27324
expect_true(ans.1 > actual_answer - tol)
expect_true(ans.1 < actual_answer + tol)
```

<!-- # END TESTS -->

<!-- # END QUESTION -->
```

--------------------------------

### Run Otter Assign Command

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/usage

Command-line interface for Otter Assign to generate distribution versions of notebooks. It creates 'dist' folders containing autograder and student versions.

```bash
otter assign hw00.ipynb dist
```

```bash
otter assign hw00.Rmd dist
```

```bash
otter assign --no-run-tests hw00.ipynb dist
```

--------------------------------

### Python Solution Removal with BEGIN/END Blocks

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_assign/notebook_format

Shows Python code with BEGIN/END SOLUTION and BEGIN/END SOLUTION NO PROMPT markers, demonstrating how these blocks are handled during solution removal.

```python
pi = 3.14
if True:
    # BEGIN SOLUTION
    radius = 3
    area = radius * pi * pi
    # END SOLUTION
    print('A circle with radius', radius, 'has area', area)

def circumference(r):
    # BEGIN SOLUTION NO PROMPT
    return 2 * pi * r
    # END SOLUTION
    """ # BEGIN PROMPT
```

--------------------------------

### R Test Cell Example

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_assign/notebook_format

Demonstrates an R test cell in Otter Assign. Test cells in R notebooks determine passing or failing based on whether they raise an error, rather than checking output. This example uses testthat::expect_equal.

```r
. = " # BEGIN TEST CONFIG
hidden: true
points: 1
" # END TEST CONFIG
testthat::expect_equal(sieve(3), c(2, 3))
```

--------------------------------

### before_execution Plugin Event

Source: https://otter-grader.readthedocs.io/en/latest/plugins/creating_plugins

This event is called before the student's submission is executed. It receives the submission content and should return the content to be executed.

```APIDOC
before_execution(submission: Union[nbformat.NotebookNode, str]) -> Union[nbformat.NotebookNode, str]
  - Called before execution of the student's submission.
  - Receives the submission content (NotebookNode for notebooks, str for scripts).
  - Must return the content (NotebookNode or str) that will be executed in place of the original submission.
  - Allows for modification or preprocessing of the submission before execution.
```

--------------------------------

### AbstractOtterPlugin Methods

Source: https://otter-grader.readthedocs.io/en/latest/plugins/creating_plugins

Documentation for the methods available within the AbstractOtterPlugin class, which serve as hooks for custom plugin behavior during the Otter grading process.

```APIDOC
AbstractOtterPlugin:
  before_grading(*args, **kwargs)
    Plugin event run before the execution of the submission which can modify the dictionary of grading configurations.
    Parameters:
      config (otter.run.run_autograder.autograder_config.AutograderConfig): the autograder config
    Raises:
      PluginEventNotSupportedException: if the event is not supported by this plugin

  before_execution(*args, **kwargs)
    Plugin event run before the execution of the submission which can modify the dictionary of grading configurations.
    Parameters:
      config (otter.run.run_autograder.autograder_config.AutograderConfig): the autograder config
    Raises:
      PluginEventNotSupportedException: if the event is not supported by this plugin

  during_assign(assignment)
    Plugin event run during the execution of Otter Assign after output directories are written. Assignment configurations are passed in via the `assignment` argument.
    Parameters:
      assignment (otter.assign.assignment.Assignment): the `Assignment` instance with configurations for the assignment; used similar to an `Attrdict` where keys are accessed with the dot syntax (e.g. `assignment.master` is the path to the master notebook)
    Raises:
      PluginEventNotSupportedException: if the event is not supported by this plugin

  during_generate(otter_config, assignment)
    Plugin event run during the execution of Otter Generate that can modify the configurations to be written to `otter_config.json`.
    Parameters:
      otter_config (dict): the dictionary of Otter configurations to be written to `otter_config.json` in the zip file
      assignment (otter.assign.assignment.Assignment): the `Assignment` instance with configurations for the assignment if Otter Assign was used to generate this zip file; will be set to `None` if Otter Assign is not being used
    Raises:
      PluginEventNotSupportedException: if the event is not supported by this plugin

  from_notebook(*args, **kwargs)
    Plugin event run by students as they work through a notebook via the `Notebook` API (see `Notebook.run_plugin`). Accepts arbitrary arguments and has no return.
    Parameters:
      *args: arguments for the plugin event
      **kwargs: keyword arguments for the plugin event
    Raises:
      PluginEventNotSupportedException: if the event is not supported by this plugin

  generate_report()
    Plugin event run after grades are written to disk. This event should return a string that gets printed to stdout as a part of the plugin report.
    Returns:
      the string to be included in the report
    Return type:
      str
    Raises:
      PluginEventNotSupportedException: if the event is not supported by this plugin

  notebook_export(*args, **kwargs)
    Plugin event run when a student calls `Notebook.export`. Accepts arbitrary arguments and should return a list of file paths to include in the exported zip file.
    Parameters:
      *args: arguments for the plugin event
      **kwargs: keyword arguments for the plugin event
    Returns:
      the list of file paths to include in the export
    Return type:
      list[str]
    Raises:
      PluginEventNotSupportedException: if the event is not supported by this plugin

  after_grading()
    Plugin event run after grades are written to disk. This event should return a string that gets printed to stdout as a part of the plugin report.
    Returns:
      the string to be included in the report
    Return type:
      str
    Raises:
      PluginEventNotSupportedException: if the event is not supported by this plugin
```

--------------------------------

### Otter Assign Command and Behavior

Source: https://otter-grader.readthedocs.io/en/latest/otter_assign/usage

Details the `otter assign` command, its required arguments, automatic language detection, and the default steps performed during assignment creation. It also mentions customizable flags and the requirement to save master notebooks with outputs.

```APIDOC
otter assign <master> <result>

  - The `otter assign` command is used to create autograded assignments from a master notebook.
  - Required arguments:
    - `master`: Path to the master notebook.
    - `result`: Path at which output should be written.
  - Automatic language detection: Otter Assign automatically recognizes the notebook's language from kernel metadata. This can be overridden with the `-l` flag.

  Default behaviors:
  1. Filter test cells and create test objects.
  2. Add Otter initialization, export, and `Notebook.check_all` cells.
  3. Clear outputs and write questions, prompts, and solutions to a new `autograder` notebook.
  4. Write all tests to the `autograder` notebook's metadata.
  5. Copy the autograder notebook (solutions removed) to a `student` directory.
  6. Remove hidden tests from the `student` notebook.
  7. Copy specified `files` to `autograder` and `student` directories.
  8. Generate a Gradescope autograder zipfile from the `autograder` directory.
  9. Run all tests in `autograder/tests` on the solutions notebook to ensure they pass.

  Customization: Behaviors can be customized using command-line flags (refer to CLI Reference).

  Important Note:
  - Ensure all cells in the master notebook are run and saved *with outputs* before running `otter assign`.
```

--------------------------------

### Sample OK Test Structure in Python

Source: https://otter-grader.readthedocs.io/en/latest/_sources/test_files/python_format

Demonstrates the required structure for OK-formatted test files used with Otter Grader. It includes the `OK_FORMAT` flag and a `test` dictionary defining test suites, cases, and their properties like code execution and visibility.

```python
OK_FORMAT = True

test = {
    "name": "q1",             # name of the test
    "points": 1,              # number of points for the entire suite
    "all_or_nothing": False,  # whether points for this test file are all-or-nothing
    "suites": [               # list of suites, only 1 suite allowed!
        {
            "cases": [                  # list of test cases
                {
                    "code": r"""
                    >>> 1 == 1          # note that in any subsequence line of a multiline
                    True                # statement, the prompt becomes ... (see below)
                    """,
                    "hidden": False,    # used to determine case visibility on Gradescope
                    "locked": False,    # ignored by Otter
                },
                {
                    "code": r"""
                    >>> for i in range(4):
                    ...     print(i == 1)
                    False
                    True
                    False
                    False
                    """,
                    "hidden": False,
                    "locked": False,
                },
            ],
            "scored": False,            # ignored by Otter
            "setup": "",                # ignored by Otter
            "teardown": "",             # ignored by Otter
            "type": "doctest"           # the type of test; only "doctest" allowed
        },
    ]
}
```

--------------------------------

### Otter Grade Output Table

Source: https://otter-grader.readthedocs.io/en/latest/_sources/tutorial

A formatted table representing the grades for submitted notebooks, as generated by the 'otter grade' command.

```text
file,q1,q2,q3
fails3Hidden.ipynb,1.0,1.0,0.5
passesAll.ipynb,1.0,1.0,1.0
fails1.ipynb,0.6666666666666666,1.0,1.0
fails2Hidden.ipynb,1.0,0.5,1.0
fails3.ipynb,1.0,1.0,0.375
fails2.ipynb,1.0,0.0,1.0
```

--------------------------------

### Python Exception-Based Test Example

Source: https://otter-grader.readthedocs.io/en/latest/_sources/otter_assign/notebook_format

Illustrates an exception-based Python test cell with metadata and a test function that requires arguments.

```python
""" # BEGIN TEST CONFIG
points: 0.5
""" # END TEST CONFIG
def test_validity(arr):
    assert len(arr) == 10
    assert (0 <= arr <= 1).all()

test_validity(arr)
```