### Clone CodaLab Repository and Set Up Environment

Source: https://github.com/codalab/codalab-competitions/blob/develop/README.md

Clones the CodaLab Competitions repository, navigates into the directory, copies the sample environment file, installs docker-compose, and starts the services.

```bash
$ git clone https://github.com/codalab/codalab-competitions
$ cd codalab-competitions
$ cp .env_sample .env
$ pip install docker-compose
$ docker-compose up -d
```

--------------------------------

### Clone Repository and Setup Environment

Source: https://github.com/codalab/codalab-competitions/wiki/Google-Cloud-setup

Clones the CodaLab Competitions repository and installs dependencies. Uses `.env_sample` for configuration; use `.env_production_sample` for production.

```bash
$ sudo apt-get install git python-pip -y
$ sudo pip install docker-compose
$ git clone https://github.com/codalab/codalab-competitions.git 
$ cd codalab-competitions
$ git checkout google-cloud-storage

# FOR PRODUCTION YOU SHOULD USE .env_production_sample instead of .env_sample !!!
$ cp .env_sample .env
```

--------------------------------

### Start Docker Service on Ubuntu

Source: https://github.com/codalab/codalab-competitions/wiki/Installing-Docker

Command to start the Docker service on Ubuntu.

```bash
sudo service docker start
```

--------------------------------

### Install Docker

Source: https://github.com/codalab/codalab-competitions/wiki/Minio-storage

Installs Docker on the system. Requires reconnecting after execution.

```bash
curl https://get.docker.com | sudo sh
sudo usermod -aG docker $USER
# reconnect
```

--------------------------------

### Install Docker on Ubuntu

Source: https://github.com/codalab/codalab-competitions/wiki/User_Using-your-own-compute-workers

Installs Docker using a convenience script. Ensure to log out and back in after adding your user to the docker group.

```bash
curl https://get.docker.com | sudo sh
```

```bash
sudo usermod -aG docker $USER
```

```bash
docker run hello-world
```

--------------------------------

### Approve Emails Configuration Example

Source: https://github.com/codalab/codalab-competitions/blob/develop/scripts/stress_testing/README.md

Example content for the 'approve_emails.txt' file, showing email addresses and wildcard patterns for approval.

```plaintext
admin@admin.com
*@codalab.org
*@*.edu
```

--------------------------------

### Install Git on Ubuntu

Source: https://github.com/codalab/codalab-competitions/wiki/Dev_Getting-Started-on-UNIX-based-Systems

Use this command to install the Git version control system on Ubuntu systems. Ensure you have sudo privileges.

```bash
sudo apt-get install git
```

--------------------------------

### Example: competition_zip_name with short arguments

Source: https://github.com/codalab/codalab-competitions/wiki/Old-Creating-A-Test-Competition-Bundle

This example demonstrates using short-form arguments for the `competition_zip_name` command to configure the number of phases, phase length, deletion of temporary files, and auto-migration.

```bash
docker exec -it django python manage.py -p 6 -1 9 -d False -a True competition_zip_name
```

--------------------------------

### Start Docker VM and Set Environment

Source: https://github.com/codalab/codalab-competitions/wiki/Installing-Docker

Commands to start the default Docker virtual machine and evaluate its environment variables for use in the current shell session.

```bash
> docker-machine start default
> eval $(docker-machine env default)
```

--------------------------------

### Install Docker on VM

Source: https://github.com/codalab/codalab-competitions/wiki/Google-Cloud-setup

Installs Docker on a fresh VM. Ensure to re-login after adding the user to the docker group.

```bash
$ curl https://get.docker.com | sudo sh
$ sudo usermod -aG docker $USER
# and then re-login
```

--------------------------------

### Install PyLint

Source: https://github.com/codalab/codalab-competitions/wiki/Dev_Code-checkers

Install PyLint using pip. Ensure you are in your virtual environment.

```bash
pip install pylint
```

--------------------------------

### Metadata File Command Example

Source: https://github.com/codalab/codalab-competitions/wiki/User_Building-an-Ingestion-Program-for-a-Competition

Example of a metadata file specifying the command to execute a participant's submission.

```yaml
command: python $program/run.py $input $output
```

--------------------------------

### Install Testing Dependencies and Run Script

Source: https://github.com/codalab/codalab-competitions/wiki/Disaster-recovery-instructions

Install necessary Python packages for the auto-testing script and then execute the script to test the restored instance. Provide domain, username, and password as arguments.

```bash
$ sudo apt-get install -y python3-pip
$ pip3 install tqdm requests
$ python3 /path/to/codalab-competitions/scripts/test_restored_instance.py -w True -d <domain> -u <username> -p <password>
```

--------------------------------

### Start Minio Service

Source: https://github.com/codalab/codalab-competitions/wiki/Minio-storage

Starts the Minio service using a Docker Compose file. This command pulls the image and launches the container.

```bash
docker-compose -f docker-compose-minio.yml up -d 
```

--------------------------------

### Install Homebrew on macOS

Source: https://github.com/codalab/codalab-competitions/wiki/Dev_Getting-Started-on-Mac

Use this command to install Homebrew, a package manager for macOS, if you don't have it already. It simplifies the installation of other development tools.

```shell
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
```

--------------------------------

### Example: competition_zip_name with long arguments

Source: https://github.com/codalab/codalab-competitions/wiki/Old-Creating-A-Test-Competition-Bundle

This example shows how to use the full argument names with the `competition_zip_name` command to specify the number of phases, phase length, deletion of temporary files, and auto-migration.

```bash
docker exec -it django python manage.py --numphases 6 --phaselength 9 --delete False --automigrate True competition_zip_name
```

--------------------------------

### Install Docker and Add User to Docker Group (Linux)

Source: https://github.com/codalab/codalab-competitions/blob/develop/README.md

Installs Docker and adds the current user to the docker group. This is a prerequisite for setting up a local CodaLab instance.

```bash
$ wget -qO- https://get.docker.com/ | sh
$ sudo usermod -aG docker $USER
```

--------------------------------

### Install and Configure rclone for Minio

Source: https://github.com/codalab/codalab-competitions/wiki/Minio-storage

Install rclone using the provided script and configure it to connect to your Minio server. You may need to run 'rclone config' multiple times if adding more servers.

```bash
$ curl https://rclone.org/install.sh | sudo bash
$ rclone config # add the servers

# If you need to add more servers, `rclone config` again
```

--------------------------------

### Install Git using Homebrew on macOS

Source: https://github.com/codalab/codalab-competitions/wiki/Dev_Getting-Started-on-Mac

Install Git, a version control system, using Homebrew. This is one of several methods to install Git on a Mac.

```shell
brew install git
```

--------------------------------

### Enter Minio Client Shell

Source: https://github.com/codalab/codalab-competitions/wiki/Minio-storage

Starts an interactive shell within a Minio Client container for managing Minio storage.

```bash
$ docker run -it --entrypoint=/bin/sh minio/mc
```

--------------------------------

### Scoring Program Output Example

Source: https://github.com/codalab/codalab-competitions/wiki/Example_Hello_world

Example of the content expected in the 'scores.txt' file generated by the scoring program for a sample submission.

```text
correct:1
```

--------------------------------

### Ingestion Program Arguments Example

Source: https://github.com/codalab/codalab-competitions/wiki/User_Building-an-Ingestion-Program-for-a-Competition

Illustrates the command structure and available directory arguments for an ingestion program.

```bash
command: python $ingestion_program/test.py $ingestion_program $input $output $hidden $shared $submission_program
```

--------------------------------

### Setup Daily Database Backups

Source: https://github.com/codalab/codalab-competitions/wiki/Production-Server-Settings

Configure a cron job to perform daily backups of the database using the provided script. Ensure the script path is correct for your environment.

```bash
crontab -e
```

```bash
@daily /path/to/codalab-competitions/scripts/pg_dump.py
```

--------------------------------

### Create Competition

Source: https://github.com/codalab/codalab-competitions/wiki/Old-Debugging

Initializes a new competition with specified title, description, logo, and creator details. Options include forcing user creation and setting the number of phases and their start dates.

```bash
python create_competition .. --title "My Competition" --description "A sample competition" --logo "logo.png" --force_user_create --creator "user@example.com" --numphases 3 --phasedates "2023-01-01,2023-02-01,2023-03-01"
```

--------------------------------

### Python Scoring Script Example (evaluate.py)

Source: https://github.com/codalab/codalab-competitions/wiki/Example_Hello_world

An example Python script that verifies if the submission data matches the expected 'Hello World!' output.

```python
# evaluate.py - checks that the submission data matches the truth data, which is "Hello World!"
```

--------------------------------

### Install and Compile Compass CSS

Source: https://github.com/codalab/codalab-competitions/wiki/UX_UX-To-Do's

Commands to install Compass and compile SCSS files. Use 'compass compile' for a one-time generation or 'compass watch .' to automatically recompile on changes.

```bash
gem update –system
gem install compass
```

```bash
cd codalab\apps\web\static\foundation
compass compile
```

```bash
cd codalab\apps\web\static\foundation
compass watch .
```

--------------------------------

### View Report with view_report.py

Source: https://github.com/codalab/codalab-competitions/blob/develop/scripts/stress_testing/README.md

Starts a web server to view processed submission data. Run in the background and ignore output using nohup.

```bash
nohup python view_report.py > /dev/null 2>&1 &

```

--------------------------------

### Start CodaLab Competitions Server

Source: https://github.com/codalab/codalab-competitions/wiki/Google-Cloud-setup

Starts the CodaLab Competitions server using Docker Compose in detached mode. For production, consider disabling the compute worker.

```bash
docker-compose up -d
```

--------------------------------

### Basic Minio Docker Compose Setup

Source: https://github.com/codalab/codalab-competitions/wiki/Minio-storage

Defines a Docker Compose service for a basic Minio instance without SSL. Suitable for local testing.

```yaml
version: '2' 
services:
  minio:
    image: minio/minio
    environment:
      MINIO_ROOT_USER: 1234567890
      MINIO_ROOT_PASSWORD: 1234567890
    ports: ["9000:9000"]
    command: server --address ":9000" /data
    volumes: 
      - ~/data:/data
      - ~/minio_config:/root/.minio
    container_name: minio
```

--------------------------------

### Run Docker Compose

Source: https://github.com/codalab/codalab-competitions/wiki/Setup-Local-Competitions

Start the CodaLab Competitions services in detached mode using docker-compose. Use 'docker-compose logs -f' to monitor the logs.

```bash
docker-compose up -d
```

```bash
docker-compose logs -f
```

--------------------------------

### Annotated competition.yaml Example

Source: https://github.com/codalab/codalab-competitions/wiki/User_Building-a-Competition-Bundle

An example of a competition.yaml file with annotations explaining each configuration element. This file defines the competition's title, description, image, registration requirements, end date, administrators, and HTML pages.

```yaml
# Build an example competition
---
# The title of the competition
title: Example Competition
# A description of the competition
description: This is a competition to test the competition bundle system. It should be able to create a competition from this bundle. The goal is to compute the closest value of pi possible.
# A logo/image for the competition
image: logo.jpg
# Does this competition require participant approval by the organizer
has_registration: True
# When is this competition finished. It is valid to not include an end_date, which means the competition remains open perpetually.
end_date: 2013-12-31
# You can specify admins here, use their codalab username (CaSe sensitive!) they will automatically be added as participants
admin_names: tony,eric
# Each competition has a set of html pages for potential participants to read and review and for participants to use to interact with the competition. These are the specifications for those pages.
html: 
    # Basic overview (first impression) of the challenge
    overview: overview.html
    # What are the metrics being used for this challenge, how is it being scored.
    evaluation: evaluation.html
    # Terms of participation, including data licensing, results submission, et al
    terms: terms_and_conditions.html
    # Where to find the data, how to download it.
    data: data.html
    # An extra page
    page_name: extra.html

```

--------------------------------

### Install Docker Engine on Ubuntu Linux 14.04

Source: https://github.com/codalab/codalab-competitions/wiki/Installing-Docker

Commands to add the Docker repository, update package lists, and install the Docker engine. It also includes a command to add the current user to the docker group.

```bash
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
```

```bash
sudo sh -c "echo deb https://apt.dockerproject.org/repo ubuntu-trusty main > /etc/apt/sources.list.d/docker.list"
```

```bash
sudo apt-get update
```

```bash
sudo apt-get install docker-engine
```

```bash
sudo useradd $USER docker
```

--------------------------------

### Leaderboard Configuration Example

Source: https://github.com/codalab/codalab-competitions/wiki/Organizer_Codalab-competition-YAML-definition-language

Defines a leaderboard with multiple score columns, including a computed average score and execution time. Uses anchors and aliases for referencing.

```yaml
leaderboards:
    Results: &RESULTS
        label: RESULTS
        rank: 1
    columns:
        set1_score:
            leaderboard: *RESULTS
            label: Set 1
            numeric_format: 4
            rank: 2
            sorting: desc
        set2_score:
            leaderboard: *RESULTS
            label: Set 2
            numeric_format: 4
            rank: 3
        ave_score:
            leaderboard: *RESULTS
            label: < Rank >
            numeric_format: 4
            rank: 1
            computed:
                operation: Avg
                fields: set1_score, set2_score
        ExecutionTime:
            leaderboard: *RESULTS
            label: ExecutionTime
            numeric_format: 2
            rank: 4
```

--------------------------------

### Verify Docker Compose Installation

Source: https://github.com/codalab/codalab-competitions/wiki/Setup-Local-Competitions

Check if docker-compose is installed by running 'docker-compose ps'. Refer to Docker-Compose installation guide if not found.

```bash
docker-compose ps
```

--------------------------------

### Verify Docker Installation on macOS

Source: https://github.com/codalab/codalab-competitions/wiki/Dev_Getting-Started-on-Mac

Run this command after installing Docker-CE for Mac to ensure Docker is functioning correctly. A successful output indicates your Docker setup is ready.

```shell
docker run hello-world
```

--------------------------------

### Get Competition Data

Source: https://github.com/codalab/codalab-competitions/wiki/User_CompetitionAPI

This endpoint allows you to retrieve JSON data for a specific competition using its ID. The example shows how to make a GET request using JavaScript and Ajax.

```APIDOC
## GET /api/competition/<Competition ID>/

### Description
Retrieves JSON data for a specific competition.

### Method
GET

### Endpoint
http://codalabtest.cloudapp.net/api/competition/<Competition ID>/

### Parameters
#### Path Parameters
- **Competition ID** (integer) - Required - The unique identifier for the competition.

### Request Example
```javascript
function getJson(url) {
    return JSON.parse($.ajax({
     type: 'GET',
     url: url,
     dataType: 'json',
     global: false,
     async: false,
     success: function (data) {
        return data;
     }
   }).responseText);
 }

var competition = getJson("http://codalabtest.cloudapp.net/api/competition/1/");
```

### Response
#### Success Response (200)
- **id** (integer) - The competition ID.
- **title** (string) - The title of the competition.
- **image_url** (string) - The URL of the competition's logo.
- **description** (string) - The description of the competition.

#### Response Example
```json
{
  "id": 1,
  "title": "Example Competition",
  "image_url": "http://example.com/logo.png",
  "description": "<p>This is an example competition description.</p>"
}
```
```

--------------------------------

### Format and Mount a New Drive for Minio

Source: https://github.com/codalab/codalab-competitions/wiki/Minio-storage

Commands to format a new drive with ext4 and configure it to mount on boot. Ensure to replace '/dev/vdb' with your actual drive identifier.

```bash
# Format drive
sudo mkfs.ext4 /dev/vdb

# Mount it on boot, by adding this to /etc/fstab
/dev/vdb        /mnt/data       ext4    defaults        0 0

# Do this to make mount look for newly added drives
sudo mount -a
```

--------------------------------

### Start Docker Compose Services

Source: https://github.com/codalab/codalab-competitions/wiki/Instance-Hosts-Commands-and-Debugging

Starts all containers that are not currently running.

```bash
docker-compose start
```

--------------------------------

### Start Compute Worker with Docker Compose

Source: https://github.com/codalab/codalab-competitions/wiki/Old-Setup-Guide-Docker

Use docker-compose to start the compute worker service.

```bash
docker-compose start worker_compute
```

--------------------------------

### Install Certbot for SSL

Source: https://github.com/codalab/codalab-competitions/wiki/Minio-storage

Installs Certbot, a tool for obtaining and renewing SSL certificates, on Debian/Ubuntu systems.

```bash
sudo apt-get update
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install certbot 
```

--------------------------------

### Create and Set Permissions for Directories

Source: https://github.com/codalab/codalab-competitions/wiki/Setup-Local-Competitions

Create 'var' and '/tmp/codalab' directories and set ownership to your user. If permission issues arise, use 'sudo chmod 777 /tmp/codalab'.

```bash
mkdir var && sudo chown youruser:youruser -R var
mkdir /tmp/codalab && sudo chown youruser:youruser -R /tmp/codalab
```

```bash
sudo chmod 777 /tmp/codalab
```

--------------------------------

### Create and Start Compute Worker

Source: https://github.com/codalab/codalab-competitions/wiki/Old-Submissions

Commands to create and start the compute worker service using docker-compose. This is a prerequisite for making submissions.

```bash
docker-compose create worker_compute
docker-compose start worker_compute
```

--------------------------------

### Scores.txt Format Example

Source: https://github.com/codalab/codalab-competitions/wiki/Organizer_Codalab-competition-YAML-definition-language

Example content for the scores.txt file, which is required output from a scoring program. Specifies values for different score columns.

```text
set1_score: 0.1234
set2_score: -2.4321
ExecutionTime: 0.5
```

--------------------------------

### Configure Site Domain and Name

Source: https://github.com/codalab/codalab-competitions/wiki/E-mail-notifications

These settings define the domain and human-readable name for the website, used in email communications. The `initialize.py` script inserts these into the database.

```python
CODALAB_SITE_DOMAIN = 'codalab.org'
CODALAB_SITE_NAME = 'CodaLab'
```

--------------------------------

### Install Emacs Text Editor

Source: https://github.com/codalab/codalab-competitions/wiki/Restoring-from-VM-Backups-on-Google-Cloud

Install the Emacs text editor if you prefer it over Vim for editing configuration files. This command is run on the Ubuntu instance.

```bash
sudo apt install emacs
```

--------------------------------

### Set up Compute Worker Environment

Source: https://github.com/codalab/codalab-competitions/wiki/Old-Submissions

These commands are necessary for setting up the local environment for compute workers. Ensure you have the correct user permissions for the var directory and /tmp/codalab.

```bash
cd codalab-competitions
mkdir var
sudo chown youruser:youruser -R var
mkdir /tmp/codalab
sudo hown youruser:youruser -R /tmp/codalab
sudo chmod 777 /tmp/codalab
```

--------------------------------

### Initialize Vue App and Vuetify Theme

Source: https://github.com/codalab/codalab-competitions/blob/develop/codalab/apps/health/templates/health/storage.html

Sets up the Vue instance with Vuetify for UI components and defines the theme colors. This is the main entry point for the storage application's frontend logic.

```javascript
new Vue({
  el: '#storage-app',
  vuetify: new Vuetify({
    theme: {
      themes: {
        light: {
          primary: '#4b7695',
          secondary: '#b0bec5',
          accent: '#8c9eff',
          error: '#b71c1c',
        },
      },
    },
  }),
  data: function () {
    return {
      loading: true,
      snapshotDate: null,
      storageBucketName: "",
      storageTotalUsage: 0,
      overviewLineChart: null,
      defaultOverviewLineChartConfig: {
        type: 'line',
        data: {
          datasets: [
            {
              label: 'Storage usage history',
              backgroundColor: 'rgb(255, 99, 132)',
              radius: 0,
              borderColor: 'rgb(255, 99, 132)',
              borderWidth: 1,
              data: [],
            }
          ],
        },
        options: {
          responsive: true,
          interaction: {
            mode: 'nearest',
            axis: 'x',
            intersect: false
          },
          scales: {
            x: {
              type: 'time',
              ticks: {
                source: 'auto',
              }
            }
          }
        }
      },
      storageUsageHistoryDateRange: [moment().subtract(1, 'months').format("YYYY-MM-DD"), moment().format("YYYY-MM-DD")],
      storageUsageHistoryMenu: false,
      storageCompetitionTableHeaders: [
        {text: 'id', value: 'id'},
        {text: 'title', value: 'title'},
        {text: 'creator', value: 'creator'},
        {text: 'is active', value: 'is_active'},
        {text: 'bundle', value: 'bundle'},
        {text: 'datasets', value: 'datasets'},
        {text: 'submissions', value: 'submissions'},
        {text: 'dumps', value: 'dumps'},
        {text: 'total', value: 'total'}
      ],
      storageCompetitionTableData: [],
      storageCompetitionTableSortBy: 'total',
      storageCompetitionTableSortDesc: true,
      competitionPieChart: null,
      defaultCompetitionPieChartConfig: {
        type: 'pie',
        data: {
          labels: [],
          datasets: [
            {
              label: 'Competition distribution',
              backgroundColor: [],
              hoverOffset: 4,
              data: []
            }
          ]
        },
        options: {
          responsive: true,
          plugins: {
            legend: {
              position: 'left',
            },
            title: {
              display: true,
              text: 'Competition distribution'
            }
          }
        }
      },
      competitionDetailPieChart: null,
      defaultCompetitionDetailPieChartConfig: {
        type: 'pie',
        data: {
          labels: [],
          datasets: [
            {
              label: 'Competition details',
              backgroundColor: [],
              hoverOffset: 4,
              data: []
            }
          ]
        },
        options: {
          responsive: true,
          plugins: {
            legend: {
              position: 'top',
            },
            title: {
              display: true,
              text: 'Competition details'
            }
          }
        }
      },
      selectedCompetitionId: null,
      storageUserTableHeaders: [
        {text: 'id', value: 'id'},
        {text: 'email', value: 'email'},
        {text: 'username', value: 'username'},
        {text: 'competitions', value: 'competitions'},
        {text: 'datasets', value: 'datasets'},
        {text: 'submissions', value: 'submissions'},
        {text: 'total', value: 'total'}
      ],
      storageUserTableData: [],
      storageUserTableSortBy: 'total',
      storageUserTableSortDesc: true,
      userPieChart: null,
      defaultUserPieChartConfig: {
        type: 'pie',
        data: {
          labels: [],
          datasets: [
            {
              label: 'User distribution',
              backgroundColor: [],
              hoverOffset: 4,
              data: []
            }
          ]
        },
        options: {
          responsive: true,
          plugins: {
            legend: {
              position: 'left',
            },
            title: {
              display: true,
              text: 'User distribution'
            }
          }
        }
      },
      userDetailPieChart: null,
      defaultUserDetailPieChartConfig: {
        type: 'pie',
        data: {
          labels: [],
          datasets: [
            {
              label: 'User details',
              backgroundColor: [],
              hoverOffset: 4,
              data: []
            }
          ]
        },
        options: {
          responsive: true,
          plugins: {
            legend: {
              position: 'top',
            },
            title: {
              display: true,
              text: 'User details'
            }
          }
        }
      },
      selectedUserId: null,
    }
  },
  mounted: function () {
    this.createOverviewLineChart();
    this.createCompetitionPieChart();
    this.createCompetitionDetailPieChart();
    this.createUserPieChart();
    this.createUserDetailPieChart();
    this.getStorageAnalytics();
    this.getStorageUsageHistory();
  },
  methods: {
    getStorageAnalytics() {
      fetch("/api/storage/analytics")
        .then(response => {
          if(response.ok) {
            return response.json();
          }
          throw new Error('Something went wrong');
        })
        .then(json => {
          this.snapshotDate = json.created_at;
          this.storageBucketName = json.bucket;
          this.storageTotalUsage = json.total_usage;
          this.updateCompetitionTableData(json.competitions_details);
          this.updateUserTableData(json.users_details);
          this.updateCompetitionPieChartData(json.competitions_details);
          if (json.competitions_details.length > 0) {
            const heaviestCompetition = json.competitions_details.reduce((prev, cur) => (prev.total > cur.total) ? prev : cur);
            this.updateCompetitionDetailPieChartData(heaviestCompetition);
          }
          this.updateUserPieChartData(json.users_details);
          if (json.users_details.length > 0) {
            const heaviestUser = json.users_details.reduce((prev, cur) => (prev.total > cur.total) ? prev : cur);
            this.updateUserDetailPieChartData(heaviestUser);
          }
        })
        .catch(error => {
          console.log("error while fetching /api/storage/analytics", error);
        })
        .finally(() => {
          this.loading = false;
        });
    },
    getStorageUsageHistory() {
      fetch("/api/storage/usage-history?" + new URLSearchParams({
        from: this.storageUsageHistoryDateRange[0],
        to: this.storageUsageHistoryDateRange[1],
      }))
        .then(response => {
          if(response.ok) {
            return response.json();
          }
          throw new Error('Something went wrong');
        })
        .then(json => {
          this.updateOverviewLineChartData(json);
        })
        .catch(error => {
          console.log("error while fetching /api/storage/usage-history", error);
        });
    },
    createOverviewLineChart() {

```

--------------------------------

### Clone and Configure Codelab Competitions

Source: https://github.com/codalab/codalab-competitions/wiki/Old-Setup-Guide-Docker

Clone the Codelab Competitions repository and copy the sample environment file. Edit the .env file to set the BROKER_URL obtained from the worker management screen.

```bash
git clone git@github.com:codalab/codalab-competitions.git
```

```bash
cd codalab-competitions
```

```bash
cp .env_sample .env
```

```dotenv
BROKER_URL=pyamqp://cd980e2d-78f9-4707-868d-bdfdd071...
```

--------------------------------

### Example Password Reset Email

Source: https://github.com/codalab/codalab-competitions/wiki/E-mail-notifications

This is an example of a password reset email sent to a user, demonstrating the use of site name in the subject and site domain in the body.

```text
Subject: [CodaLab] Password Reset E-mail
From: info@codalab.org
To: someuser@somewhere.com
Date: Thu, 03 Oct 2013 22:42:55 -0000

You're receiving this e-mail because you or someone else has requested a password 
for your user account at codalab.org. It can be safely ignored if you did not request 
a password reset. Click the link below to reset your password.

http://codalab.org/accounts/password/reset/key/1-3le-6640241efe4e5b77f4e5/

In case you forgot, your username is someuser.

Thanks for using our site!
```

--------------------------------

### Copy SSL Certificate Files for Minio

Source: https://github.com/codalab/codalab-competitions/wiki/Minio-storage

After obtaining certificates, copy the fullchain and private key to the certs directory. These commands create symlinks to the live certificate files.

```bash
sudo cp -Lr /etc/letsencrypt/live/minio.YOURHOST.com-0001/fullchain.pem public.crt
sudo cp -Lr /etc/letsencrypt/live/minio.YOURHOST.com-0001/privkey.pem private.key
```

--------------------------------

### Install Xcode Command Line Tools on macOS

Source: https://github.com/codalab/codalab-competitions/wiki/Dev_Getting-Started-on-Mac

Install only the Xcode command-line tools if you prefer not to download the full Xcode package. This is often sufficient for development purposes.

```shell
xcode-select --install
```

--------------------------------

### Minio Service with SSL Docker Run

Source: https://github.com/codalab/codalab-competitions/wiki/Minio-storage

Launches a Minio server with SSL enabled using a direct Docker run command. Requires pre-configured SSL certificates.

```bash
docker run -p 443:443 \
  -e "MINIO_ACCESS_KEY=1234567890" \
  -e "MINIO_SECRET_KEY=1234567890" \
  -v ~/data:/data \
  -v ~/minio_config:/root/.minio \
  -d \
  minio/minio server --address ":443" /data
```

--------------------------------

### Clone Repo and Install Requirements

Source: https://github.com/codalab/codalab-competitions/wiki/Administering-compute-workers-using-Fabric

Clone the codalab-competitions repository and install necessary Python packages, including Fabric. It's recommended to perform this within a Python virtual environment.

```bash
git clone git@github.com:codalab/codalab-competitions.git
cd codalab-competitions/codalab

# Preferably do this in a Python virtual environment!
pip install fabric -r requirements/common.txt
```

--------------------------------

### Initialize Vue App for Submissions

Source: https://github.com/codalab/codalab-competitions/blob/develop/codalab/apps/web/templates/web/my/submissions.html

Sets up the Vue instance for the submissions page, defining data, computed properties, and methods for fetching and managing submissions.

```javascript
var COMPETITION_ID = {{ competition.id }}
var IS_SUPERUSER_OR_STAFF = {{is_superuser_or_staff|yesno:"true,false" }}
new Vue({
  el: '#app',
  vuetify: new Vuetify(),
  data: function () {
    return {
      search: '',
      expanded_rows: [],
      loading: true,
      headers: [
        {text: 'Submitted', value: 'submitted_at', width: '200px'},
        {text: 'Participant', value: 'username'},
        {text: 'Participant Submission #', value: 'participant_submission_number'},
        {text: 'Phase', value: 'phase_number'},
        {text: 'Submission ID', value: 'id'},
        {text: 'Filename', value: 'filename'},
        {text: 'Size (bytes)', value: 'size'},
        {text: 'Status', value: 'status'},
        {text: 'Leaderboard', value: 'leaderboard'},
        {text: 'Results', value: 'results'},
        {text: '', value: 'actions'},
      ],
      footer_properties: {
        'items-per-page-options': [50, 100, -1],
      },
      submissions: [],
      phase: null,
      phases: [],
      loading: true,
      is_superuser_or_staff: IS_SUPERUSER_OR_STAFF
    }
  },
  computed: {
    filtered_submissions() {
      return this.submissions.filter(s => {
        if (this.phase) {
          return this.phase.phasenumber == s.phase_number
        } else {
          return true
        }
      })
    }
  },
  mounted() {
    this.loading = false
    this.get_submissions()
    this.get_phases()
  },
  methods: {
    async get_submissions() {
      try {
        this.submissions = await $.get(`/api/competition/${COMPETITION_ID}/submissions/`)
      } catch (e) {
        console.log(e)
        console.log(e.response)
      }
    },
    async get_phases() {
      try {
        const resp = await $.get(`/api/competition/${COMPETITION_ID}/phases/`)
        this.phases = resp[0].phases
        this.loading = false
      } catch (e) {
        console.log(e)
        console.log(e.response)
      }
    },
    submission_clicked(submission) {
      const index = this.expanded_rows.indexOf(submission)
      if (index === -1) {
        this.expanded_rows.push(submission)
      } else {
        this.expanded_rows.splice(index, 1)
      }
    },
    get_item_class(item) {
      return `item-status-${item.status}`
    },
    async delete_submission(submission) {
      if (confirm("Are you sure you want to delete this submission?")) {
        try {
          const resp = await $.post(`/competitions/submission_delete/${submission.id}/`)
          this.get_submissions()
        } catch (e) {
          console.log("Error deleting submission..")
          console.log(e.response)
        }
      }
    },
    async toggle_leaderboard_for_submission(submission) {
      try {
        const resp = await $.post(`/competitions/toggle_leaderboard/${submission.id}/`)
        this.get_submissions()
      } catch (e) {
        console.log("Error toggling leaderboard..")
        console.log(e.response)
      }
    },
    async mark_submission_failed(submission) {
      try {
        const resp = await $.post(`/competitions/mark_as_failed/${submission.id}/`)
        this.get_submissions()
      } catch (e) {
        console.log("Error marking as failed..")
        console.log(e.response)
      }
    },
    async re_run_submission(submission) {
      try {
        const resp = await $.post(`/competitions/submission_re_run/${submission.id}/`)
        this.get_submissions()
      } catch (e) {
        console.log("Error deleting submission..")
        console.log(e.response)
      }
    },
    async re_run_all_submissions() {
      if(!this.phase) {
        alert("Cannot re-run all submissions, no phase selected.")
        return false
      }
      if (confirm("Are you sure? This could take hours!")) {
        try {
          const resp = await $.post(`/competitions/submission_re_run_all/${this.phase.id}/`)
          this.get_submissions()
        } catch (e) {
          console.log("Error deleting submission..")
          console.log(e.response)
        }
      }
    },
    async migrate_submission(submission) {
      try {
        const resp = await $.post(`/competitions/submission_migrate/${submission.id}/`)
        this.get_submissions()
      } catch (e) {
        console.log("Error deleting submission..")
        console.log(e.response)
      }
    }
  }
})

```

--------------------------------

### Build and Start Latest RabbitMQ Version

Source: https://github.com/codalab/codalab-competitions/wiki/RabbitMQ-upgrade

Updates the project to the latest version, builds the RabbitMQ image, and starts the new container in detached mode. This is part of the upgrade process.

```bash
git pull
docker build rabbit
docker-compose up -d
```

--------------------------------

### Copy Production Environment Sample

Source: https://github.com/codalab/codalab-competitions/wiki/Disaster-recovery-instructions

If a previous .env file is not available, copy the production sample file to create a new .env file. This file will be used to configure environment variables.

```bash
cp .env_production_sample .env
```

--------------------------------

### Configure SSL Certificate Paths in .env

Source: https://github.com/codalab/codalab-competitions/wiki/SSL

Specify the paths to your SSL certificate and private key files in the .env file. Ensure the certificate and key files are placed in the certs/ folder.

```bash
SSL_CERTIFICATE=/app/certs/localhost.crt
SSL_CERTIFICATE_KEY=/app/certs/localhost.key
```

--------------------------------

### Run CSS Build Command

Source: https://github.com/codalab/codalab-competitions/wiki/Old-Compiling-Less-To-Css

Execute this command in the root 'codalab' directory to compile LESS files to CSS. Ensure 'package.json' is present in the directory.

```bash
npm run build-css
```

--------------------------------

### Run PyLint on CodaLab Project

Source: https://github.com/codalab/codalab-competitions/wiki/Dev_Code-checkers

Navigate to the CodaLab project directory and run PyLint to scan the entire project.

```bash
pylint codalab
```

--------------------------------

### Configure Compute Worker Environment

Source: https://github.com/codalab/codalab-competitions/wiki/User_Using-your-own-compute-workers

Sets up the `.env` file with broker connection details. Replace `localhost` with your server's IP or URL if necessary.

```dotenv
BROKER_URL=pyamqp://6ad9ac58-88e3-4a22-9be7-6ed5126ef388:40546f9d-0f2e-4413-a6b2-a1d86cad2b30@localhost/37324ab2-ee78-4e8d-a6ee-6089a159d253
BROKER_USE_SSL=True
```

--------------------------------

### Initialize Vue App for Admin Competitions Manager

Source: https://github.com/codalab/codalab-competitions/blob/develop/codalab/apps/web/templates/web/admin_competitions_manager.html

Sets up the Vue instance for the admin competitions manager page, including Vuetify theme configuration and initial data.

```javascript
const csrftoken = document.querySelector('[name=csrfmiddlewaretoken]').value;
new Vue({
    el: '#admin-competitions-manager-app',
    vuetify: new Vuetify({
        theme: {
            themes: {
                light: {
                    primary: '#4b7695',
                    secondary: '#b0bec5',
                    accent: '#8c9eff',
                    error: '#b71c1c',
                },
            },
        },
    }),
    data: function () {
        return {
            defaultUpperBoundLimit: 300,
            defaultUpperBoundLimitLoading: false,
            competitionsTableSelected: [],
            competitionsTableHeaders: [
                {text: 'ID', value: 'id'},
                {text: 'Title', value: 'title'},
                {text: 'Organizer', value: 'creator'},
                {text: 'Start date', value: 'start_date'},
                {text: 'End date', value: 'end_date'},
                {text: 'Upper bound of the max sumbission size', value: 'upper_bound_max_submission_size'},
                {text: 'Current max sumbission size per phase', value: 'max_submission_sizes'},
                {text: 'Apply the upper bound limit to all phases', value: 'sync_button', sortable: false}
            ],
            competitionsTableItems: [],
            competitionsTableLoading: false,
            competitionsTableSortBy: 'total',
            competitionsTableSortDesc: true,
            competitionsTableSearch: '',
            saving: false,
            synchronizing: false,
            competitionsIdToUpdate: []
        };
    },
    mounted: function () {
        this.getCompetitions();
        this.getDefaultUpperBoundLimit();
    },
    methods: {
        getCompetitions() {
            this.competitionsTableLoading = true;
            fetch("/api/admin/competitions/list")
                .then(response => {
                    if(response.ok) {
                        return response.json();
                    }
                    throw new Error('Something went wrong');
                })
                .then(json => {
                    this.competitionsTableItems = json;
                })
                .catch(error => {
                    console.log("error while fetching /api/admin/competitions", error);
                })
                .finally(() => {
                    this.competitionsTableLoading = false;
                });
        },
        save(item) {
            // Mark the competition as competition to send through the API
            var newCompIdToUpdate = [item.id];
            // Propagate the modification to all selected rows and mark them as competitions to send through the API
            for (const selectedCompetition of this.competitionsTableSelected) {
                var competition = this.competitionsTableItems.find(competition => competition.id == selectedCompetition.id);
                competition.upper_bound_max_submission_size = item.upper_bound_max_submission_size;
                newCompIdToUpdate.push(selectedCompetition.id);
            }
            // Add them to already existing marked competitions
            this.competitionsIdToUpdate = [...new Set([...this.competitionsIdToUpdate, ...newCompIdToUpdate])];
        },
        saveAll() {
            var competitionsUpdated = [];
            for (const id of this.competitionsIdToUpdate) {
                const competition = this.competitionsTableItems.find(competition => competition.id == id);
                competitionsUpdated.push({
                    id: id,
                    upper_bound_max_submission_size: competition.upper_bound_max_submission_size
                });
            }
            if (competitionsUpdated.length > 0) {
                const competitionsJson = { competitions: competitionsUpdated }
                this.saving = true;
                fetch("/api/admin/competitions/update", {
                    method: 'POST',
                    headers: {
                        'X-CSRFToken': csrftoken,
                        'Content-Type': 'application/json',
                    },
                    body: JSON.stringify(competitionsJson)
                })
                .then(response => {
                    if(response.ok) {
                        return response.json();
                    }
                    throw new Error('Something went wrong');
                })
                .then(json => {
                    this.competitionsIdToUpdate = [];
                    for (const competition of json) {
                        const index = this.competitionsTableItems.findIndex(comp => comp.id == competition.id);
                        if (index >= 0) {
                            // Using method splice for Vue reactivity (updating an element is not supported otherwise)
                            this.competitionsTableItems.splice(index, 1, competition);
                        } else {
                            console.log("error: No matching competition to update");
                        }
                    }
                })
                .catch(error => {
                    console.log("error while fetching /api/admin/competitions", error);
                })
                .finally(() => {
                    this.saving = false;
                });
            }
        },
        synchronize(competition) {
            this.synchronizing = true;
            fetch("/api/admin/competition/" + competition.id + "/apply_upper_bound_limit", {
                method: 'PATCH',
                headers: {
                    'X-CSRFToken': csrftoken,
                }
            })
            .then(response => {
                if(response.ok) {
                    return response.json();
                }
                throw new Error('Something went wrong');
            })
            .then(competition => {
                const index = this.competitionsTableItems.findIndex(comp => comp.id == competition.id);
                if (index >= 0) {
                    // Using method splice for Vue reactivity (updating an element is not supported otherwise)
                    this.competitionsTableItems.splice(index, 1, competition);
                } else {
                    console.log("error: No matching competition to update");
                }
            })
            .catch(error => {
                console.log("error while fetching /api/admin/competition/" + competition.id + "/apply_upper_bound_limit", error);
            })
            .finally(() => {
                this.synchronizing = false;
            });
        },
        getDefaultUpperBoundLimit() {
            this.defaultUpperBoundLimitLoading = true;
            fetch("/api/admin/competitions/default_upper_bound_limit")
                .then(response => {
                    if(response.ok) {
                        return response.json();
                    }
                    throw new Error('Something went wrong');
                })
                .then(json => {
                    this.defaultUpperBoundLimit = json;
                })
                .catch(error => {
                    conso
```

--------------------------------

### CLI Commands for Setting Permissions

Source: https://github.com/codalab/codalab-competitions/wiki/Dev_Groups-and-permissions-discussion

Illustrates different syntaxes for setting permissions on bundles, worksheets, or other groups using the CLI.

```bash
set-perm <group_spec> <none|r(ead)|a(ll)> <bundle_spec>
set-perm -w <group_spec> <none|r(ead)|a(ll)> <worksheet_spec>
set-perm -g <group_spec> <none|r(ead)|a(ll)> <group_spec>
```

```bash
set-bundle-perm <group_spec> <none|r(ead)|a(ll)> <bundle_spec>
set-worksheet-perm <group_spec> <none|r(ead)|a(ll)> <worksheet_spec>
set-group-perm <group_spec> <none|r(ead)|a(ll)> <group_spec>
```

```bash
set-perm <group_spec> <none|r(ead)|a(ll)> <object_spec>
```

--------------------------------

### Clone CodaLab Competitions Repository

Source: https://github.com/codalab/codalab-competitions/wiki/Setup-Local-Competitions

Clone the CodaLab Competitions repository into your local 'src' directory. Ensure you have Git installed.

```bash
cd ~
mkdir src
cd src
git clone https://github.com/codalab/codalab-competitions.git
```

--------------------------------

### Write CodaLab Scores to scores.txt

Source: https://github.com/codalab/codalab-competitions/wiki/User_DetailedResultsPage

This snippet demonstrates how to create a directory if it doesn't exist and then write performance metrics to a 'scores.txt' file in the CodaLab format. Ensure the output directory is correctly specified.

```python
if not os.path.exists(output_dir):
			os.makedirs(output_dir)
			
	##writing codaLab scores.txt##
	output_filename = os.path.join(output_dir, 'scores.txt')              
	output_file = open(output_filename, 'wb')
	output_file.write("SENSITIVITY: %.3f\n" % (float(88) / float(100)))
	output_file.write("CANDIDATES: %.3f\n" % (float(120) / float(50)))
	output_file.write("TP: 88\n")
	output_file.write("FP: 20\n")
	output_file.write("FN: 12\n")
	output_file.write("TN: 32")
	output_file.close()
	
	##CodaLab stuff ends	
	createHTML(outputDir)
```

--------------------------------

### Example scores.txt Content

Source: https://github.com/codalab/codalab-competitions/wiki/User_Tracking-Competition-History

Demonstrates the content of a scores.txt file, indicating correctness of the current submission and the overall audit score.

```shell
correct:1.0
audit_score:0.3
```