### Initialize File Backend and Get File

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/io.html

Demonstrates initializing a Petrel backend and then retrieving a file using its get method.

```python
>>> # Initialize a file backend and call its methods
>>> import mmeval.fileio as fileio
>>> backend = fileio.get_file_backend(backend_args={'backend': 'petrel'})
>>> backend.get('s3://path/of/your/file')
```

--------------------------------

### Install MMEval with All Dependencies

Source: https://mmeval.readthedocs.io/en/latest/get_started/installation.html

Install MMEval along with all required dependencies for metrics.

```bash
pip install 'mmeval[all]'

```

--------------------------------

### Initialize PetrelBackend

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/petrel_backend.html

Initializes the PetrelBackend with optional path mapping and memcached support. Requires the 'petrel_client' to be installed.

```python
backend = PetrelBackend()
filepath1 = 'petrel://path/of/file'
filepath2 = 'cluster-name:petrel://path/of/file'
backend.get(filepath1)  # get data from default cluster
client.get(filepath2)  # get data from 'cluster-name' cluster
```

--------------------------------

### Install MMEval

Source: https://mmeval.readthedocs.io/en/latest/get_started/installation.html

Install MMEval using pip. Requires Python 3.6+.

```bash
pip install mmeval

```

--------------------------------

### Directly Get File Using Unified I/O Function

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/io.html

Shows how to use the unified 'get' function to retrieve a file, which internally handles backend selection.

```python
>>> # Directory call unified I/O functions
>>> fileio.get('s3://path/of/your/file')
```

--------------------------------

### Initialize Memcached Backend

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/memcached_backend.html

Initializes the MemcachedBackend with server and client configuration files. Ensure the 'memcached' library is installed.

```python
from pathlib import Path
from typing import Union

from .base import BaseStorageBackend



class MemcachedBackend(BaseStorageBackend):
    """Memcached storage backend.

    Attributes:
        server_list_cfg (str): Config file for memcached server list.
        client_cfg (str): Config file for memcached client.
        sys_path (str, optional): Additional path to be appended to `sys.path`.
            Defaults to None.
    """

    def __init__(self, server_list_cfg, client_cfg, sys_path=None):
        if sys_path is not None:
            import sys
            sys.path.append(sys_path)
        try:
            import mc
        except ImportError:
            raise ImportError(
                'Please install memcached to enable MemcachedBackend.')

        self.server_list_cfg = server_list_cfg
        self.client_cfg = client_cfg
        self._client = mc.MemcachedClient.GetInstance(self.server_list_cfg,
                                                      self.client_cfg)
        # mc.pyvector servers as a point which points to a memory cache
        self._mc_buffer = mc.pyvector()
```

--------------------------------

### List Files and Directories with PetrelBackend

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.fileio.PetrelBackend.html

Instantiate PetrelBackend and use list_dir_or_file to iterate through contents. This example shows listing both files and directories.

```python
backend = PetrelBackend()
dir_path = 'petrel://path/of/dir'
# list those files and directories in current directory
for file_path in backend.list_dir_or_file(dir_path):
    print(file_path)
```

--------------------------------

### Initialize and Use MemcachedBackend

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.fileio.MemcachedBackend.html

Demonstrates how to initialize the MemcachedBackend with server and client configuration files and retrieve data using the get method.

```python
server_list_cfg = '/path/of/server_list.conf'
client_cfg = '/path/of/mc.conf'
backend = MemcachedBackend(server_list_cfg, client_cfg)
backend.get('/path/of/file')
```

--------------------------------

### Install MPI4Py and OpenMPI

Source: https://mmeval.readthedocs.io/en/latest/tutorials/dist_evaluation.html

Install the necessary packages for using MPI4Py as a distributed communication backend. Conda is recommended for installation.

```bash
conda install openmpi
conda install mpi4py

```

--------------------------------

### PaddleDist Initialization

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/core/dist_backends/paddle_dist.html

Initializes the PaddleDist backend. Raises an ImportError if Paddle is not installed.

```python
paddle = try_import('paddle')
paddle_dist = try_import('paddle.distributed')

class PaddleDist(TensorBaseDistBackend):
    """A distributed communication backend for paddle.distributed."""

    def __init__(self) -> None:
        super().__init__()
        if paddle is None:
            raise ImportError(f'For availability of {self.__class__.__name__},'
                              ' please install paddle first.')
```

--------------------------------

### List Available Distributed Backends

Source: https://mmeval.readthedocs.io/en/latest/tutorials/dist_evaluation.html

Use this command to see all supported distributed communication backends in MMEval. Ensure MMEval is installed.

```python
import mmeval

print(mmeval.core.dist.list_all_backends())
```

--------------------------------

### ROUGE Example Usage

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.ROUGE.html

Demonstrates how to initialize and use the ROUGE metric to compute scores.

```APIDOC
## Example
```python
>>> from mmeval import ROUGE
>>> predictions = ['the cat is on the mat']
>>> references = [['a cat is on the mat']]
>>> metric = ROUGE(rouge_keys='L')
>>> metric.add(predictions, references)
>>> results = metric.compute_metric()
{'rougeL_fmeasure': 0.8333333,
 'rougeL_precision': 0.8333333,
 'rougeL_recall': 0.8333333}
```
```

--------------------------------

### LmdbBackend Initialization

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/lmdb_backend.html

Initializes the LmdbBackend with specified database path and environment parameters. It requires the 'lmdb' library to be installed.

```APIDOC
## LmdbBackend

### Description
Lmdb storage backend.

### Parameters
#### Args
- **db_path** (str) - Lmdb database path.
- **readonly** (bool) - Lmdb environment parameter. If True, disallow any write operations. Defaults to True.
- **lock** (bool) - Lmdb environment parameter. If False, when concurrent access occurs, do not lock the database. Defaults to False.
- **readahead** (bool) - Lmdb environment parameter. If False, disable the OS filesystem readahead mechanism, which may improve random read performance when a database is larger than RAM. Defaults to False.
- **kwargs** - Keyword arguments passed to `lmdb.open`.

### Attributes
- **db_path** (str) - Lmdb database path.
```

--------------------------------

### ProposalRecall Metric Example

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/proposal_recall.html

Instantiate and use the ProposalRecall metric with sample predictions and ground truths. This example demonstrates how to generate random bounding boxes and scores for predictions and ground truths, then evaluate them using the metric.

```python
import numpy as np
from mmeval import ProposalRecall

proposal_recall = ProposalRecall()
def _gen_bboxes(num_bboxes, img_w=256, img_h=256):
    # random generate bounding boxes in 'xyxy' formart.
    x = np.random.rand(num_bboxes, ) * img_w
    y = np.random.rand(num_bboxes, ) * img_h
    w = np.random.rand(num_bboxes, ) * (img_w - x)
    h = np.random.rand(num_bboxes, ) * (img_h - y)
    return np.stack([x, y, x + w, y + h], axis=1)

prediction = {
    'bboxes': _gen_bboxes(10),
    'scores': np.random.rand(10, ),
}
groundtruth = {
    'bboxes': _gen_bboxes(10),
}
proposal_recall(predictions=[prediction, ], groundtruths=[groundtruth, ])  # doctest: +ELLIPSIS  # noqa: E501
```

--------------------------------

### Example: Accumulating Batches

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.core.BaseMetric.html

Illustrates how to accumulate results over multiple batches using the `add` method and then compute the final metric using the `compute` method.

```APIDOC
## Example: Accumulating Batches

### Description
Illustrates how to accumulate results over multiple batches using the `add` method and then compute the final metric using the `compute` method.

### Code Example
```python
>>> for i in range(10):
>>>     predicts = np.random.randint(0, 4, size=(10,)) # Corrected labels assignment
>>>     labels = predicts = np.random.randint(0, 4, size=(10,)) # Corrected labels assignment
>>>     accuracy.add(predicts, labels)
>>> accuracy.compute()
```
```

--------------------------------

### Initialize LmdbBackend

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/lmdb_backend.html

Initializes the LmdbBackend with the database path and optional LMDB environment parameters. Requires the 'lmdb' library to be installed.

```python
from pathlib import Path
from typing import Union

from .base import BaseStorageBackend



class LmdbBackend(BaseStorageBackend):
    """Lmdb storage backend.

    Args:
        db_path (str): Lmdb database path.
        readonly (bool): Lmdb environment parameter. If True, disallow any
            write operations. Defaults to True.
        lock (bool): Lmdb environment parameter. If False, when concurrent
            access occurs, do not lock the database. Defaults to False.
        readahead (bool): Lmdb environment parameter. If False, disable the OS
            filesystem readahead mechanism, which may improve random read
            performance when a database is larger than RAM. Defaults to False.
        **kwargs: Keyword arguments passed to `lmdb.open`.

    Attributes:
        db_path (str): Lmdb database path.
    """

    def __init__(
            self,
            db_path,
            readonly=True,
            lock=False,
            readahead=False,
            **kwargs):
        try:
            import lmdb  # noqa: F401
        except ImportError:
            raise ImportError(
                'Please run "pip install lmdb" to enable LmdbBackend.')

        self.db_path = str(db_path)
        self.readonly = readonly
        self.lock = lock
        self.readahead = readahead
        self.kwargs = kwargs
        self._client = None

```

--------------------------------

### Get File Backend by Backend Arguments

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/io.html

Instantiates a file backend using provided backend arguments, specifying the backend type directly.

```python
>>> # get file backend based on the backend_args
>>> backend = get_file_backend(backend_args={'backend': 'petrel'})
```

--------------------------------

### Example: Implementing Accuracy Metric

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.core.BaseMetric.html

Demonstrates how to implement a custom accuracy metric by subclassing BaseMetric and overriding the `add` and `compute_metric` methods.

```APIDOC
## Example: Implementing Accuracy Metric

### Description
Demonstrates how to implement a custom accuracy metric by subclassing BaseMetric and overriding the `add` and `compute_metric` methods.

### Code Example
```python
>>> import numpy as np
>>> from mmeval.core import BaseMetric
>>>
>>> class Accuracy(BaseMetric):
...     def add(self, predictions, labels):
...         self._results.append((predictions, labels))
...     def compute_metric(self, results):
...         predictions = np.concatenate([res[0] for res in results])
...         labels = np.concatenate([res[1] for res in results])
...         correct = (predictions == labels)
...         accuracy = sum(correct) / len(predictions)
...         return {'accuracy': accuracy}
```
```

--------------------------------

### Compute Top-K Accuracy with Thresholds

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/accuracy.html

Calculate top-k accuracy with specified thresholds. This example demonstrates how to initialize the Accuracy metric with custom top-k values and thresholds, and then compute the results.

```python
>>> labels = np.asarray([0, 1, 2, 3])
>>> preds = np.asarray([
    [0.7, 0.1, 0.1, 0.1],
    [0.1, 0.3, 0.4, 0.2],
    [0.3, 0.4, 0.2, 0.1],
    [0.0, 0.0, 0.1, 0.9]])
>>> accuracy = Accuracy(topk=(1, 2, 3))
>>> accuracy(preds, labels)
{'top1': 0.5, 'top2': 0.75, 'top3': 1.0}
>>> accuracy = Accuracy(topk=2, thrs=(0.1, 0.5))
>>> accuracy(preds, labels)
{'top2_thr-0.10': 0.75, 'top2_thr-0.50': 0.5}
```

--------------------------------

### Get Data from Memcached

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/memcached_backend.html

Retrieves data from Memcached using the provided filepath. The data is returned as a bytes object. Requires the 'memcached' library to be installed.

```python
def get(self, filepath: Union[str, Path]):
        """Get values according to the filepath.

        Args:
            filepath (str or Path): Path to read data.

        Returns:
            bytes: Expected bytes object.

        Examples:
            >>> server_list_cfg = '/path/of/server_list.conf'
            >>> client_cfg = '/path/of/mc.conf'
            >>> backend = MemcachedBackend(server_list_cfg, client_cfg)
            >>> backend.get('/path/of/file')
            b'hello world'
        """
        filepath = str(filepath)
        import mc
        self._client.Get(filepath, self._mc_buffer)
        value_buf = mc.ConvertBuffer(self._mc_buffer)
        return value_buf
```

--------------------------------

### Accuracy Metric Example

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/core/base_metric.html

An example demonstrating how to implement a custom Accuracy metric by subclassing BaseMetric.

```APIDOC
>>> import numpy as np
>>> from mmeval.core import BaseMetric
>>> 
>>> class Accuracy(BaseMetric):
...     def add(self, predictions, labels):
...         self._results.append((predictions, labels))
...     def compute_metric(self, results):
...         predictions = np.concatenate([res[0] for res in results])
...         labels = np.concatenate([res[1] for res in results])
...         correct = (predictions == labels)
...         accuracy = sum(correct) / len(predictions)
...         return {'accuracy': accuracy}
>>> 
>>> accuracy = Accuracy()
>>> accuracy(predictions=[1, 2, 3, 4], labels=[1, 2, 3, 1])
{'accuracy': 0.75}
```

--------------------------------

### Initialize and Use MpiiPCKAccuracy

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.MpiiPCKAccuracy.html

Demonstrates how to initialize the MpiiPCKAccuracy metric with custom parameters and compute accuracy using sample predictions and ground truths.

```python
>>> from mmeval import MpiiPCKAccuracy
>>> import numpy as np
>>> num_keypoints = 16
>>> keypoints = np.random.random((1, num_keypoints, 2)) * 10
>>> predictions = [{'coords': keypoints}]
>>> keypoints_visible = np.ones((1, num_keypoints)).astype(bool)
>>> head_size = np.random.random((1, 2)) * 10
>>> groundtruths = [{
...     'coords': keypoints + 1.0,
...     'mask': keypoints_visible,
...     'head_size': head_size,
... }]
>>> mpii_pckh_metric = MpiiPCKAccuracy(thr=0.3, norm_item='head')
>>> mpii_pckh_metric(predictions, groundtruths)
OrderedDict([('Head', 100.0), ('Shoulder', 100.0), ('Elbow', 100.0),
('Wrist', 100.0), ('Hip', 100.0), ('Knee', 100.0), ('Ankle', 100.0),
('PCKh', 100.0), ('PCKh@0.1', 100.0)])

```

--------------------------------

### PetrelBackend Initialization

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/petrel_backend.html

Initializes the PetrelBackend with optional path mapping and memcached support.

```APIDOC
## PetrelBackend(path_mapping: Optional[dict] = None, enable_mc: bool = True)

### Description
Initializes the PetrelBackend with optional path mapping and memcached support.

### Parameters
- **path_mapping** (dict, optional): Path mapping dict from local path to Petrel path. Defaults to None.
- **enable_mc** (bool, optional): Whether to enable memcached support. Defaults to True.

### Raises
- ImportError: If petrel_client is not installed.
```

--------------------------------

### KeypointNME Metric Initialization and Usage Example

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/keypoint_nme.html

Demonstrates how to initialize and use the KeypointNME metric for evaluating keypoint predictions. It shows setting up the metric with a specific normalization mode and dataset metadata, and then applying it to sample predictions and ground truths.

```python
>>> from mmeval.metrics import KeypointNME
>>> import numpy as np
>>> aflw_dataset_meta = {
...     'dataset_name': 'aflw',
...     'num_keypoints': 19,
...     'sigmas': np.array([]),
... }
>>> nme_metric = KeypointNME(
...     norm_mode='use_norm_item',
...     norm_item=norm_item,
...     dataset_meta=aflw_dataset_meta)
>>> batch_size = 2
>>> predictions = [{
...     'coords': np.zeros((1, 19, 2))
... } for _ in range(batch_size)]
>>> groundtruths = [{
...     'coords': np.zeros((1, 19, 2)) + 0.5,
...     'mask': np.ones((1, 19)).astype(bool),
...     'box_size': np.ones((1, 1)) * i * 20
... } for i in range(batch_size)]
>>> norm_item = 'box_size'
>>> nme_metric(predictions, groundtruths)
OrderedDict([('NME', 0.03535533892480951)])
```

--------------------------------

### COCO Detection Example Usage

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/coco_detection.html

Demonstrates how to format predictions and ground truths for COCO detection evaluation. This example generates dummy bounding boxes and masks, then calls the coco_det_metric function.

```python
img_w, img_h = 256, 256
num_bboxes = 10
pred_boxes = _gen_bboxes(
    num_bboxes=num_bboxes,
    img_w=img_w,
    img_h=img_h)
pred_masks = _gen_masks(
    bboxes=pred_boxes,
    img_w=img_w,
    img_h=img_h)
prediction = {
    'img_id': img_id,
    'bboxes': pred_boxes,
    'scores': np.random.rand(num_bboxes, ),
    'labels': np.random.randint(0, num_classes, size=(num_bboxes, )), 
    'masks': pred_masks
}
gt_boxes = _gen_bboxes(
    num_bboxes=num_bboxes,
    img_w=img_w,
    img_h=img_h)
gt_masks = _gen_masks(
    bboxes=pred_boxes,
    img_w=img_w,
    img_h=img_h)
groundtruth = {
    'img_id': img_id,
    'width': img_w,
    'height': img_h,
    'bboxes': gt_boxes,
    'labels': np.random.randint(0, num_classes, size=(num_bboxes, )), 
    'masks': gt_masks,
    'ignore_flags': np.zeros(num_bboxes)
}
coco_det_metric(predictions=[prediction, ], groundtruths=[groundtruth, ])  # doctest: +ELLIPSIS  # noqa: E501
{'bbox_mAP': ..., 'bbox_mAP_50': ..., ...,
 'segm_mAP': ..., 'segm_mAP_50': ..., ...,
 'bbox_result': ..., 'segm_result': ..., ...}
```

--------------------------------

### COCODetection Metric Usage Example

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.COCODetection.html

Example of how to use the COCODetection metric with sample predictions and ground truths. This demonstrates the expected input format for predictions and ground truths, and the output structure of the computed metrics.

```python
>>> groundtruth = {
...     'img_id': img_id,
...     'width': img_w,
...     'height': img_h,
...     'bboxes': gt_boxes,
...     'labels': np.random.randint(0, num_classes, size=(num_bboxes, )),
...     'masks': gt_masks,
...     'ignore_flags': np.zeros(num_bboxes)
... }
>>> coco_det_metric(predictions=[prediction, ], groundtruths=[groundtruth, ])  
{'bbox_mAP': ..., 'bbox_mAP_50': ..., ...,
 'segm_mAP': ..., 'segm_mAP_50': ..., ...,
 'bbox_result': ..., 'segm_result': ..., ...}
```

--------------------------------

### Registering default backends

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/registry_utils.html

These lines demonstrate the direct registration of several common storage backends with their respective names and prefixes. This is typically done once at module initialization.

```python
register_backend('local', LocalBackend, prefixes='')
register_backend('memcached', MemcachedBackend)
register_backend('lmdb', LmdbBackend)
register_backend('petrel', PetrelBackend, prefixes='petrel')
register_backend('http', HTTPBackend, prefixes=['http', 'https'])

```

--------------------------------

### get_weights_indices

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.NaturalImageQualityEvaluator.html

Get weights and indices for interpolation.

```APIDOC
## get_weights_indices

### Description
Get weights and indices for interpolation. 

### Parameters
#### Parameters
- **input_length** (int) - Length of the input sequence.
- **output_length** (int) - Length of the output sequence.
- **scale** (float) - The scale factor for interpolation.
- **kernel** (Callable) - The interpolation kernel function.
- **kernel_width** (float) - The width of the kernel.
```

--------------------------------

### get

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/io.html

Reads the content of a file as bytes.

```APIDOC
## get

### Description
Reads bytes from a given ``filepath`` with 'rb' mode.

### Arguments
- **filepath** (str or Path): Path to read data.
- **backend_args** (dict, optional): Arguments to instantiate the corresponding backend. Defaults to None.

### Returns
- bytes: Expected bytes object.

### Examples
```python
filepath = '/path/of/file'
get(filepath)
```

### Returns Example
```python
b'hello world'
```
```

--------------------------------

### Initialize and Use VOCMeanAP

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.VOCMeanAP.html

Demonstrates how to initialize the VOCMeanAP metric and use it to evaluate predictions against ground truths. This includes generating sample bounding boxes and labels for both predictions and ground truths.

```python
import numpy as np
from mmeval import VOCMeanAP

num_classes = 4
voc_map = VOCMeanAP(num_classes=4)

def _gen_bboxes(num_bboxes, img_w=256, img_h=256):
    # random generate bounding boxes in 'xyxy' formart.
    x = np.random.rand(num_bboxes, ) * img_w
    y = np.random.rand(num_bboxes, ) * img_h
    w = np.random.rand(num_bboxes, ) * (img_w - x)
    h = np.random.rand(num_bboxes, ) * (img_h - y)
    return np.stack([x, y, x + w, y + h], axis=1)

prediction = {
    'bboxes': _gen_bboxes(10),
    'scores': np.random.rand(10, ),
    'labels': np.random.randint(0, num_classes, size=(10, ))
}
groundtruth = {
    'bboxes': _gen_bboxes(10),
    'labels': np.random.randint(0, num_classes, size=(10, )),
    'bboxes_ignore': _gen_bboxes(5),
    'labels_ignore': np.random.randint(0, num_classes, size=(5, ))
}
voc_map(predictions=[prediction, ], groundtruths=[groundtruth, ])
```

--------------------------------

### get

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.fileio.PetrelBackend.html

Reads bytes from a given file path in Petrel storage.

```APIDOC
## get(_filepath: Union[str, pathlib.Path]) -> bytes

### Description
Reads bytes from a given `filepath` with ‘rb’ mode.

### Parameters
* **filepath** (str or Path) - Path to read data.

### Returns
* **bytes** - Bytes read from filepath.

### Request Example
```python
backend = PetrelBackend()
filepath = 'petrel://path/of/file'
backend.get(filepath)
```
```

--------------------------------

### List Directories and Files with LocalBackend

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/local_backend.html

Instantiate LocalBackend and use list_dir_or_file to scan a directory. This method can list files, directories, or both, with options to filter by suffix and scan recursively.

```python
backend = LocalBackend()
dir_path = '/path/of/dir'
# list those files and directories in current directory
for file_path in backend.list_dir_or_file(dir_path):
    print(file_path)
# only list files
for file_path in backend.list_dir_or_file(dir_path, list_dir=False):
    print(file_path)
# only list directories
for file_path in backend.list_dir_or_file(dir_path, list_file=False):
    print(file_path)
# only list files ending with specified suffixes
for file_path in backend.list_dir_or_file(dir_path, suffix='.txt'):
    print(file_path)
# list all files and directory recursively
for file_path in backend.list_dir_or_file(dir_path, recursive=True):
    print(file_path)
```

--------------------------------

### AVAMeanAP Initialization and Usage

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/ava_map.html

Initialize the AVAMeanAP metric with necessary file paths and parameters, then add predictions and compute the mAP.

```APIDOC
## AVAMeanAP

### Description

AVA evaluation metric. This metric computes mAP using the ava evaluation toolkit provided by the author.

### Parameters

- **ann_file** (str) - Required - The annotation file path.
- **label_file** (str) - Required - The label file path.
- **exclude_file** (Optional[str]) - Optional - The excluded timestamp file path. Defaults to None.
- **num_classes** (int) - Optional - Number of classes. Defaults to 81.
- **custom_classes** (Optional[list(int)]) - Optional - A subset of class ids from origin dataset. Defaults to None.
- **verbose** (bool) - Optional - Whether to print messages in the evaluation process. Defaults to True.

### Method Signature

```python
AVAMeanAP(ann_file: str, label_file: str, exclude_file: Optional[str] = None, num_classes: int = 81, custom_classes: Optional[List[int]] = None, verbose: bool = True, **kwargs)
```

### Examples

```python
from mmeval import AVAMeanAP
import numpy as np

ann_file = 'tests/test_metrics/ava_detection_gt.csv'
label_file = 'tests/test_metrics/ava_action_list.txt'
num_classes = 4
av_metric = AVAMeanAP(ann_file=ann_file, label_file=label_file,
                       num_classes=4)

predictions = [
{
    'video_id': '3reY9zJKhqN',
    'timestamp': 1774,
    'outputs': [
        np.array([[0.362, 0.156, 0.969, 0.666, 0.106],
                  [0.442, 0.083, 0.721, 0.947, 0.162]]),
        np.array([[0.288, 0.365, 0.766, 0.551, 0.706],
                  [0.178, 0.296, 0.707, 0.995, 0.223]]),
        np.array([[0.417, 0.167, 0.843, 0.939, 0.015],
                  [0.35, 0.421, 0.57, 0.689, 0.427]]) ]
},
{
    'video_id': 'HmR8SmNIoxu',
    'timestamp': 1384,
    'outputs': [
        np.array([[0.256, 0.338, 0.726, 0.799, 0.563],
                  [0.071, 0.256, 0.64, 0.75, 0.297]]),
        np.array([[0.326, 0.036, 0.513, 0.991, 0.405],
                  [0.351, 0.035, 0.729, 0.936, 0.945]]),
        np.array([[0.051, 0.005, 0.975, 0.942, 0.424],
                  [0.347, 0.05, 0.97, 0.944, 0.396]])]
},
{
    'video_id': '5HNXoce1raG',
    'timestamp': 1097,
    'outputs': [
        np.array([[0.39, 0.087, 0.833, 0.616, 0.447],
                  [0.461, 0.212, 0.627, 0.527, 0.036]]),
        np.array([[0.022, 0.394, 0.93, 0.527, 0.109],
                  [0.208, 0.462, 0.874, 0.948, 0.954]]),
        np.array([[0.206, 0.456, 0.564, 0.725, 0.685],
                  [0.106, 0.445, 0.782, 0.673, 0.367]])]}
]
av_metric(predictions)
{'mAP@0.5IOU': 0.027777778}
```
```

--------------------------------

### get_size_from_scale

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.NaturalImageQualityEvaluator.html

Get the output size given input size and scale factor.

```APIDOC
## get_size_from_scale

### Description
Get the output size given input size and scale factor. 

### Parameters
#### Parameters
- **input_size** (Tuple) - The size of the input image.
- **scale_factor** (List[float]) - The resize factor.

### Returns
The size of the output image. 

### Return type
List[int]
```

--------------------------------

### DOTA Metric Example

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/dota_map.html

Demonstrates how to initialize and use the DOTAMetric for evaluating object detection predictions against ground truth. It includes helper functions for generating bounding boxes and sample data structures for predictions and ground truths.

```python
import numpy as np
from mmeval import DOTAMetric

num_classes = 15
dota_metric = DOTAMetric(num_classes=15)

def _gen_bboxes(num_bboxes, img_w=256, img_h=256):
    # random generate bounding boxes in 'xywha' formart.
    x = np.random.rand(num_bboxes, ) * img_w
    y = np.random.rand(num_bboxes, ) * img_h
    w = np.random.rand(num_bboxes, ) * (img_w - x)
    h = np.random.rand(num_bboxes, ) * (img_h - y)
    a = np.random.rand(num_bboxes, ) * np.pi / 2
    return np.stack([x, y, w, h, a], axis=1)
prediction = {
    'bboxes': _gen_bboxes(10),
    'scores': np.random.rand(10, ),
    'labels': np.random.randint(0, num_classes, size=(10, ))
}
groundtruth = {
    'bboxes': _gen_bboxes(10),
    'labels': np.random.randint(0, num_classes, size=(10, )),
    'bboxes_ignore': _gen_bboxes(5),
    'labels_ignore': np.random.randint(0, num_classes, size=(5, ))
}
dota_metric(predictions=[prediction, ], groundtruths=[groundtruth, ])  # doctest: +ELLIPSIS  # noqa: E501
```

--------------------------------

### Get OneFlow World Size

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/core/dist_backends/oneflow_dist.html

Retrieves the total number of processes in the distributed environment.

```python
return flow.env.get_world_size()
```

--------------------------------

### Get Paddle World Size

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/core/dist_backends/paddle_dist.html

Retrieves the total number of processes in the current distributed group.

```python
@property
def world_size(self) -> int:
        """Returns the world size of the current process group."""
        return paddle_dist.get_world_size()
```

--------------------------------

### Initialize LmdbBackend

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.fileio.LmdbBackend.html

Instantiate the LmdbBackend with the path to the LMDB database. The backend is read-only by default.

```python
backend = LmdbBackend('path/to/lmdb')
```

--------------------------------

### Initialize and Use DOTAMetric

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.DOTAMeanAP.html

Demonstrates how to initialize the DOTAMetric with a specified number of classes and then use it to evaluate predictions against ground truths. This includes generating random bounding boxes for both predictions and ground truths.

```python
import numpy as np
from mmeval import DOTAMetric
num_classes = 15
dota_metric = DOTAMetric(num_classes=15)

def _gen_bboxes(num_bboxes, img_w=256, img_h=256):
    # random generate bounding boxes in 'xywha' formart.
    x = np.random.rand(num_bboxes, ) * img_w
    y = np.random.rand(num_bboxes, ) * img_h
    w = np.random.rand(num_bboxes, ) * (img_w - x)
    h = np.random.rand(num_bboxes, ) * (img_h - y)
    a = np.random.rand(num_bboxes, ) * np.pi / 2
    return np.stack([x, y, w, h, a], axis=1)
prediction = {
    'bboxes': _gen_bboxes(10),
    'scores': np.random.rand(10, ),
    'labels': np.random.randint(0, num_classes, size=(10, ))
}
groundtruth = {
    'bboxes': _gen_bboxes(10),
    'labels': np.random.randint(0, num_classes, size=(10, )),
    'bboxes_ignore': _gen_bboxes(5),
    'labels_ignore': np.random.randint(0, num_classes, size=(5, ))
}
dota_metric(predictions=[prediction, ], groundtruths=[groundtruth, ])  
{'mAP@0.5': ..., 'mAP': ...}
```

--------------------------------

### Get Paddle Rank

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/core/dist_backends/paddle_dist.html

Retrieves the rank index of the current process within the distributed group.

```python
@property
def rank(self) -> int:
        """Returns the rank index of the current process group."""
        return paddle_dist.get_rank()
```

--------------------------------

### ROUGE Initialization and Usage

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/rouge.html

Demonstrates how to initialize the ROUGE metric with custom parameters and compute scores.

```APIDOC
## ROUGE

### Description

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a set of metrics used for evaluating automatic summarization and machine translation. This class implements the ROUGE metric.

### Parameters

- **rouge_keys** (Union[List, Tuple, int, str], optional): Specifies which ROUGE scores to compute. Can be a list, tuple, int (1-9), or string ('L'). Defaults to (1, 2, 'L').
- **use_stemmer** (bool, optional): Whether to use stemming from the nltk library. Defaults to False.
- **normalizer** (Optional[Callable], optional): A function to normalize text before tokenization. Defaults to None.
- **tokenizer** (Union[Callable, str, None], optional): A tokenizer function or a string identifier ('en', 'cn'). Defaults to None, which infers the language.
- **accumulate** (str, optional): How to accumulate scores when multiple references are present. Options are 'best' or 'avg'. Defaults to 'best'.
- **lowercase** (bool, optional): Whether to convert text to lowercase before processing. Defaults to True.
- **kwargs**: Additional keyword arguments passed to the base metric class.

### Method

`ROUGE`

### Examples

```python
>>> from mmeval import ROUGE
>>> predictions = ['the cat is on the mat']
>>> references = [['a cat is on the mat']]
>>> metric = ROUGE(rouge_keys='L')
>>> metric.add(predictions, references)
>>> results = metric.compute_metric()
{'rougeL_fmeasure': 0.8333333, 'rougeL_precision': 0.8333333, 'rougeL_recall': 0.8333333}
```

### Method

`add`

### Parameters

- **predictions** (Sequence[str]): An iterable of predicted sentences.
- **references** (Sequence[Sequence[str]]): An iterable of referenced sentences. Each predicted sentence may correspond to multiple referenced sentences.

### Method

`compute_metric`

### Description

Computes the final ROUGE scores based on the accumulated results.

### Returns

- dict: A dictionary containing the computed ROUGE scores for each specified rouge key (e.g., 'rougeL_fmeasure', 'rouge1_precision').
```

--------------------------------

### Get OneFlow Rank

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/core/dist_backends/oneflow_dist.html

Retrieves the rank index of the current process within the distributed environment.

```python
return flow.env.get_rank()
```

--------------------------------

### Get File Backend with URI and Backend Arguments

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/io.html

Obtains a file backend instance using both a URI for prefix detection and specific backend arguments. The 'backend' key in backend_args takes precedence.

```python
>>> # backend name has a higher priority if 'backend' in backend_args
>>> backend = get_file_backend(uri, backend_args={'backend': 'petrel'})
```

--------------------------------

### LMDB Client Initialization

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/lmdb_backend.html

Internal method to open and return an LMDB client instance using the configured database path and environment parameters.

```python
    def _get_client(self):
        import lmdb

        return lmdb.open(
            self.db_path,
            readonly=self.readonly,
            lock=self.lock,
            readahead=self.readahead,
            **self.kwargs)

```

--------------------------------

### Perplexity Metric Example

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/perplexity.html

Instantiate the Perplexity metric and compute perplexity on random predictions and targets. This demonstrates basic usage.

```python
from mmeval import Perplexity
import numpy as np

preds = np.random.rand(2, 4, 2)
targets = np.random.randint(low=0, high=2, size=(2, 4))
metric = Perplexity()
result = metric(preds, targets)  # doctest: +ELLIPSIS
{'perplexity': ...}
```

--------------------------------

### list_all_backends()

Source: https://mmeval.readthedocs.io/en/latest/genindex.html

Lists all available distribution backends. Part of the mmeval.core module.

```APIDOC
## list_all_backends()

### Description
Retrieves a list of all registered distribution backend names.

### Module
mmeval.core
```

--------------------------------

### Read bytes from a file

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.fileio.LocalBackend.html

Use `get` to read the raw byte content of a file. Ensure the file exists before calling.

```python
>>> backend = LocalBackend()
>>> filepath = '/path/of/file'
>>> backend.get(filepath)
b'hello world'
```

--------------------------------

### Accuracy Initialization and Usage

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/accuracy.html

Demonstrates how to initialize the Accuracy metric with different top-k values and thresholds, and how to use it to compute accuracy from predictions and labels.

```APIDOC
## Accuracy Class

### Description
Computes top-k accuracy with specified thresholds.

### Parameters

- **topk** (Union[int, Sequence[int]], optional): The top-k values to compute accuracy for. Defaults to (1,).
- **thrs** (Union[float, Sequence[Union[float, None]], None], optional): The thresholds to apply for accuracy computation. Defaults to 0..
- **kwargs**: Additional keyword arguments passed to the base class.

### Method Signature

```python
Accuracy(topk: Union[int, Sequence[int]] = (1, ), thrs: Union[float, Sequence[Union[float, None]], None] = 0., **kwargs)
```

### Example Usage

Computing top-k accuracy with specified threshold:

```python
>>> labels = np.asarray([0, 1, 2, 3])
>>> preds = np.asarray([
    [0.7, 0.1, 0.1, 0.1],
    [0.1, 0.3, 0.4, 0.2],
    [0.3, 0.4, 0.2, 0.1],
    [0.0, 0.0, 0.1, 0.9]])
>>> accuracy = Accuracy(topk=(1, 2, 3))
>>> accuracy(preds, labels)
{'top1': 0.5, 'top2': 0.75, 'top3': 1.0}
>>> accuracy = Accuracy(topk=2, thrs=(0.1, 0.5))
>>> accuracy(preds, labels)
{'top2_thr-0.10': 0.75, 'top2_thr-0.50': 0.5}
```

Accumulate batch:

```python
>>> for i in range(10):
...     labels = torch.randint(0, 4, size=(100, ))
...     predicts = torch.randint(0, 4, size=(100, ))
...     accuracy.add(predicts, labels)
>>> accuracy.compute()  # doctest: +SKIP
```
```

--------------------------------

### Get Normalization Factor

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/keypoint_nme.html

Helper method to compute the normalization factor for NME calculation, typically using inter-ocular distance.

```python
    def _get_norm_factor(self, gt_coords: np.ndarray) -> np.ndarray:
        """Get the normalization factor. Generally inter-ocular distance
        measured as the Euclidean distance between the outer corners of the
        eyes is used.

        Args:

```

--------------------------------

### TorchCPUDist Properties

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/core/dist_backends/torch_cpu.html

Provides properties to check if the distributed environment is initialized, and to get the current process rank and world size.

```python
    @property
    def is_initialized(self) -> bool:
        """Returns True if the distributed environment has been initialized.

        Returns:
            bool: Returns True if the distributed environment has been
            initialized, otherwise returns False.
        """
        return torch_dist.is_initialized()

    @property
    def rank(self) -> int:
        """Returns the rank index of the current process group."""
        return torch_dist.get_rank()

    @property
    def world_size(self) -> int:
        """Returns the world size of the current process group."""
        return torch_dist.get_world_size()

```

--------------------------------

### Check OneFlow Initialization

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/core/dist_backends/oneflow_dist.html

Checks if the distributed environment has been initialized by attempting to get the world size. Raises ValueError if not initialized.

```python
try:
            flow.env.get_world_size()
            is_init = True
        except ValueError:
            is_init = False
        return is_init
```

--------------------------------

### Initialize NaturalImageQualityEvaluator

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.NaturalImageQualityEvaluator.html

Instantiate the NIQE evaluator with optional parameters for cropping, input order, color conversion, and channel order. This is the primary way to set up the metric for use.

```python
from mmeval import NaturalImageQualityEvaluator
import numpy as np

niqe = NaturalImageQualityEvaluator()
preds = np.random.randint(0, 255, size=(3, 32, 32))
niqe(preds)
```

--------------------------------

### BaseStorageBackend Class

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/fileio/backends/base.html

The BaseStorageBackend class is an abstract base class for all storage backends. It requires implementations for `get()` and `get_text()` methods.

```APIDOC
## Class BaseStorageBackend

### Description
Abstract class of storage backends. All backends need to implement two apis: :meth:`get()` and :meth:`get_text()`.

- :meth:`get()` reads the file as a byte stream.
- :meth:`get_text()` reads the file as texts.

### Methods

- **name** (property)
  Returns the name of the storage backend class.

- **get**(filepath)
  Abstract method to read a file as a byte stream.
  - **filepath** (str): The path to the file.

- **get_text**(filepath)
  Abstract method to read a file as text.
  - **filepath** (str): The path to the file.
```

--------------------------------

### Initialize AVAMeanAP Metric

Source: https://mmeval.readthedocs.io/en/latest/_modules/mmeval/metrics/ava_map.html

Instantiate the AVAMeanAP metric with necessary file paths and configuration. The `ann_file` and `label_file` are mandatory. `num_classes` specifies the total number of classes, and `custom_classes` can be used to filter for a subset of classes.

```python
from mmeval import AVAMeanAP
import numpy as np

ann_file = 'tests/test_metrics/ava_detection_gt.csv'
label_file = 'tests/test_metrics/ava_action_list.txt'
num_classes = 4
ava_metric = AVAMeanAP(ann_file=ann_file, label_file=label_file,
                       num_classes=4)
```

--------------------------------

### Initialize COCODetection Metric

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.COCODetection.html

Instantiate the COCODetection metric for evaluating 'bbox' and 'segm' tasks. Ensure 'pycocotools' is installed for mask evaluation.

```python
import numpy as np
from mmeval import COCODetection
try:
    from mmeval.metrics.utils.coco_wrapper import mask_util
except ImportError as e:
    mask_util = None

num_classes = 4
fake_dataset_metas = {
    'classes': tuple([str(i) for i in range(num_classes)])
}

coco_det_metric = COCODetection(
    dataset_meta=fake_dataset_metas,
    metric=['bbox', 'segm']
)
```

--------------------------------

### Initialize and Use KeypointNME Metric

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.KeypointNME.html

Demonstrates how to initialize the KeypointNME metric with 'use_norm_item' normalization mode and then use it to evaluate predictions against ground truths. Ensure 'norm_item' is defined before calling the metric.

```python
from mmeval.metrics import KeypointNME
import numpy as np
aflw_dataset_meta = {
    'dataset_name': 'aflw',
    'num_keypoints': 19,
    'sigmas': np.array([]),
}
nme_metric = KeypointNME(
    norm_mode='use_norm_item',
    norm_item=norm_item,
    dataset_meta=aflw_dataset_meta)
batch_size = 2
predictions = [{
    'coords': np.zeros((1, 19, 2))
} for _ in range(batch_size)]
groundtruths = [{
    'coords': np.zeros((1, 19, 2)) + 0.5,
    'mask': np.ones((1, 19)).astype(bool),
    'box_size': np.ones((1, 1)) * i * 20
} for i in range(batch_size)]
norm_item = 'box_size'
nme_metric(predictions, groundtruths)

```

--------------------------------

### Initialize and Use ROUGE Metric

Source: https://mmeval.readthedocs.io/en/latest/api/generated/mmeval.metrics.ROUGE.html

Instantiate the ROUGE metric with specific configurations like 'rouge_keys' and then add predictions and references to compute the scores. This example demonstrates calculating the 'rougeL_fmeasure', 'rougeL_precision', and 'rougeL_recall'.

```python
from mmeval import ROUGE
predictions = ['the cat is on the mat']
references = [['a cat is on the mat']]
metric = ROUGE(rouge_keys='L')
metric.add(predictions, references)
results = metric.compute_metric()
print(results)
```