### MetaSGD Initialization and Adaptation Example

Source: http://learn2learn.net/docs/learn2learn.algorithms

Illustrates the usage of the MetaSGD algorithm, which is similar to MAML but learns per-parameter learning rates for fast adaptation. The example shows initialization, cloning, loss calculation, adaptation, and backpropagation.

```python
linear = l2l.algorithms.MetaSGD(nn.Linear(20, 10), lr=0.01)
clone = linear.clone()
error = loss(clone(X), y)
clone.adapt(error)
error = loss(clone(X), y)
error.backward()
```

--------------------------------

### LearnableOptimizer Example

Source: http://learn2learn.net/docs/learn2learn.optim

Demonstrates how to use LearnableOptimizer for meta-descent. The meta-optimizer updates the transform, and a standard optimizer updates the meta-optimizer itself.

```python
linear = nn.Linear(784, 10)
transform = l2l.optim.ModuleTransform(torch.nn.Linear)
metaopt = l2l.optim.LearnableOptimizer(linear, transform, lr=0.01)
opt = torch.optim.SGD(metaopt.parameters(), lr=0.001)

metaopt.zero_grad()
opt.zero_grad()
error = loss(linear(X), y)
error.backward()
opt.step()  # 
unpdate metaopt
metaopt.step()  # update linear
```

--------------------------------

### GBML Algorithm Implementation

Source: http://learn2learn.net/docs/learn2learn.algorithms

Example of initializing and training the GBML algorithm with a custom transform and one adaptation step.

```python
model = SmallCNN()
transform = l2l.optim.ModuleTransform(torch.nn.Linear)
gbml = l2l.algorithms.GBML(
    module=model,
    transform=transform,
    lr=0.01,
    adapt_transform=True,
)
gbml.to(device)
opt = torch.optim.SGD(gbml.parameters(), lr=0.001)

# Training with 1 adaptation step
for iteration in range(10):
    opt.zero_grad()
    task_model = gbml.clone()
    loss = compute_loss(task_model)
    task_model.adapt(loss)
    loss.backward()
    opt.step()
```

--------------------------------

### MAML Initialization and Adaptation Example

Source: http://learn2learn.net/docs/learn2learn.algorithms

Demonstrates how to initialize and use the MAML algorithm for meta-learning. This involves cloning the model, calculating a loss, adapting the cloned model, and performing backpropagation.

```python
linear = l2l.algorithms.MAML(nn.Linear(20, 10), lr=0.01)
clone = linear.clone()
error = loss(clone(X), y)
clone.adapt(error)
error = loss(clone(X), y)
error.backward()
```

--------------------------------

### LightningMAML PyTorch Lightning Module

Source: http://learn2learn.net/docs/learn2learn.algorithms

Example of setting up and training a meta-learning model using PyTorch Lightning with the LightningMAML module.

```python
tasksets = l2l.vision.benchmarks.get_tasksets('omniglot')
model = l2l.vision.models.OmniglotFC(28**2, args.ways)
maml = LightningMAML(classifier, adaptation_lr=0.1, **dict_args)
episodic_data = EpisodicBatcher(tasksets.train, tasksets.validation, tasksets.test)
trainer = pl.Trainer.from_argparse_args(args)
trainer.fit(maml, episodic_data)
```

--------------------------------

### Train Prototypical Networks with PyTorch Lightning

Source: http://learn2learn.net/docs/learn2learn.algorithms

Example of setting up and training a Prototypical Networks model using PyTorch Lightning. Requires tasksets, a feature extractor, and an EpisodicBatcher.

```python
tasksets = l2l.vision.benchmarks.get_tasksets('mini-imagenet')
features = Convnet()  # init model
protonet = LightningPrototypicalNetworks(features, **dict_args)
episodic_data = EpisodicBatcher(tasksets.train, tasksets.validation, tasksets.test)
trainer = pl.Trainer.from_argparse_args(args)
trainer.fit(protonet, episodic_data)
```

--------------------------------

### Train ANIL Model with PyTorch Lightning

Source: http://learn2learn.net/docs/learn2learn.algorithms

Example of setting up and training an ANIL model using PyTorch Lightning. Requires tasksets, a model, and an EpisodicBatcher.

```python
tasksets = l2l.vision.benchmarks.get_tasksets('omniglot')
model = l2l.vision.models.OmniglotFC(28**2, args.ways)
anil = LightningANIL(model.features, model.classifier, adaptation_lr=0.1, **dict_args)
episodic_data = EpisodicBatcher(tasksets.train, tasksets.validation, tasksets.test)
trainer = pl.Trainer.from_argparse_args(args)
trainer.fit(anil, episodic_data)
```

--------------------------------

### Instantiate MetaDataset with torchvision.datasets.MNIST

Source: http://learn2learn.net/docs/learn2learn.data

Wraps a classification dataset to enable fast indexing of samples within classes. This example shows how to wrap the MNIST dataset.

```python
mnist = torchvision.datasets.MNIST(root="/tmp/mnist", train=True)
mist = l2l.data.MetaDataset(mnist)
```

--------------------------------

### Get Pretrained Backbone

Source: http://learn2learn.net/docs/learn2learn.vision

Retrieve a pretrained backbone model for a specified dataset. Ensure weights are downloaded if not available.

```python
backbone = l2l.vision.models.get_pretrained_backbone(
    model='resnet12',
    dataset='mini-imagenet',
    root='~/.data',
    download=True,
)
```

--------------------------------

### Get Omniglot Benchmark Tasksets

Source: http://learn2learn.net/docs/learn2learn.vision

Retrieves the train, validation, and test tasksets for the Omniglot benchmark. Allows sampling tasks directly or accessing tasksets via attributes.

```python
train_tasks, validation_tasks, test_tasks = l2l.vision.benchmarks.get_tasksets('omniglot')
batch = train_tasks.sample()

or:

tasksets = l2l.vision.benchmarks.get_tasksets('omniglot')
batch = tasksets.train.sample()
```

--------------------------------

### Load and Prepare Quickdraw Dataset

Source: http://learn2learn.net/docs/learn2learn.vision

Instantiates the Quickdraw dataset for training and wraps it with MetaDataset and TaskDataset for few-shot learning.

```python
train_dataset = l2l.vision.datasets.Quickdraw(root='./data', mode='train')
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskDataset(dataset=train_dataset, num_tasks=1000)
```

--------------------------------

### Initialize and Train LightningMetaOptNet

Source: http://learn2learn.net/docs/learn2learn.algorithms

This snippet shows how to initialize the LightningMetaOptNet module with a feature extractor and optional arguments, then train it using episodic data with a PyTorch Lightning Trainer. Ensure `args` are properly configured for the trainer.

```python
tasksets = l2l.vision.benchmarks.get_tasksets('mini-imagenet')
features = Convnet()  # init model
metaoptnet = LightningMetaOptNet(features, **dict_args)
episodic_data = EpisodicBatcher(tasksets.train, tasksets.validation, tasksets.test)
trainer = pl.Trainer.from_argparse_args(args)
trainer.fit(metaoptnet, episodic_data)
```

--------------------------------

### Initialize TieredImagenet Dataset

Source: http://learn2learn.net/docs/learn2learn.vision

Initializes the TieredImagenet dataset for training. Ensure the root directory exists and set download=True if the data is not present.

```python
train_dataset = l2l.vision.datasets.TieredImagenet(root='./data', mode='train', download=True)
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskDataset(dataset=train_dataset, num_tasks=1000)
```

--------------------------------

### Initialize CIFARFS Dataset

Source: http://learn2learn.net/docs/learn2learn.vision

Initializes the CIFARFS dataset for training. The 'ways' parameter is used in the TaskGenerator.

```python
train_dataset = l2l.vision.datasets.CIFARFS(root='./data', mode='train')
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskGenerator(dataset=train_dataset, ways=ways)
```

--------------------------------

### Partition Task into Support and Query Sets

Source: http://learn2learn.net/docs/learn2learn.data

Partitions data and labels into support and query sets based on a specified number of shots per class. Assumes an equal number of samples per class.

```python
X, y = taskset.sample()
(X_support, y_support), (X_query, y_query) = partition_task(X, y, shots=5)
```

--------------------------------

### Load VGGFlower102 Dataset for Few-Shot Learning

Source: http://learn2learn.net/docs/learn2learn.vision

Loads the VGGFlower102 dataset in training mode and prepares it for few-shot learning tasks. Ensure the 'data' directory exists.

```python
train_dataset = l2l.vision.datasets.VGGFlower102(root='./data', mode='train')
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskDataset(dataset=train_dataset, num_tasks=1000)
```

--------------------------------

### Initialize FC100 Dataset

Source: http://learn2learn.net/docs/learn2learn.vision

Initializes the FC100 dataset for training. This dataset is based on CIFAR100 with specific class splits.

```python
train_dataset = l2l.vision.datasets.FC100(root='./data', mode='train')
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskDataset(dataset=train_dataset, num_tasks=1000)
```

--------------------------------

### Load FGVCFungi Dataset for Few-Shot Learning

Source: http://learn2learn.net/docs/learn2learn.vision

Loads the FGVCFungi dataset in training mode and prepares it for few-shot learning tasks. Ensure the 'data' directory exists and that you have agreed to the dataset's terms of use.

```python
train_dataset = l2l.vision.datasets.FGVCFungi(root='./data', mode='train')
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskDataset(dataset=train_dataset, num_tasks=1000)
```

--------------------------------

### Load FGVCAircraft Dataset for Few-Shot Learning

Source: http://learn2learn.net/docs/learn2learn.vision

Loads the FGVCAircraft dataset in training mode, enabling download if necessary, and prepares it for few-shot learning tasks. Ensure the 'data' directory exists.

```python
train_dataset = l2l.vision.datasets.FGVCAircraft(root='./data', mode='train', download=True)
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskDataset(dataset=train_dataset, num_tasks=1000)
```

--------------------------------

### List Available Vision Benchmarks

Source: http://learn2learn.net/docs/learn2learn.vision

Iterates through and prints the names of all available benchmarks in the learn2learn vision module.

```python
for name in l2l.vision.benchmarks.list_tasksets():
    print(name)
    tasksets = l2l.vision.benchmarks.get_tasksets(name)
```

--------------------------------

### MetaSGD

Source: http://learn2learn.net/docs/learn2learn.algorithms

High-level implementation of Meta-SGD. It wraps an arbitrary nn.Module and augments it with clone() and adapt() methods, learning per-parameter learning rates for fast adaptation.

```APIDOC
## MetaSGD

### Description
High-level implementation of _Meta-SGD_. This class wraps an arbitrary nn.Module and augments it with `clone()` and `adapt` methods. It behaves similarly to `MAML`, but in addition a set of per-parameters learning rates are learned for fast-adaptation.

### Arguments
* **model** (Module) - Module to be wrapped.
* **lr** (float) - Initialization value of the per-parameter fast adaptation learning rates.
* **first_order** (bool, _optional_ , default=False) - Whether to use the first-order version.
* **lrs** (list of Parameters, _optional_ , default=None) - If not None, overrides `lr`, and uses the list as learning rates for fast-adaptation.

### Method: adapt
#### `adapt(self, loss, first_order=None)`

**Description**
Akin to `MAML.adapt()` but for MetaSGD: it updates the model with the learnable per-parameter learning rates.

### Method: clone
#### `clone(self)`

**Description**
Akin to `MAML.clone()` but for MetaSGD: it includes a set of learnable fast-adaptation learning rates.
```

--------------------------------

### Load Full Omniglot Dataset

Source: http://learn2learn.net/docs/learn2learn.vision

Load the FullOmniglot dataset with specified transformations and download if necessary. Wrap with MetaDataset for meta-learning tasks.

```python
omniglot = l2l.vision.datasets.FullOmniglot(root='./data',
                                            transform=transforms.Compose([
                                                transforms.Resize(28, interpolation=LANCZOS),
                                                transforms.ToTensor(),
                                                lambda x: 1.0 - x,
                                            ]),
                                            download=True)
omniglot = l2l.data.MetaDataset(omniglot)
```

--------------------------------

### Instantiate ResNet12 Network

Source: http://learn2learn.net/docs/learn2learn.vision

Instantiate ResNet12, a 12-layer residual network for few-shot learning. Configure output size, hidden size, pooling, filter width, and dropout rates for residual layers and embedding.

```python
model = ResNet12(output_size=ways, hidden_size=1600, avg_pool=False)
```

--------------------------------

### Instantiate OmniglotFC Network

Source: http://learn2learn.net/docs/learn2learn.vision

Use OmniglotFC for fully-connected networks in Omniglot experiments. It requires input and output sizes, and optionally accepts hidden layer sizes.

```python
net = OmniglotFC(input_size=28**2,
                 output_size=10,
                 sizes=[64, 64, 64])
```

--------------------------------

### ParameterUpdate with Custom Transform

Source: http://learn2learn.net/docs/learn2learn.optim

Shows how to use ParameterUpdate to apply custom updates to model parameters. It computes gradients, passes them through a transform, and then updates the parameters.

```python
model = torch.nn.Linear()
transform = l2l.optim.KroneckerTransform(l2l.nn.KroneckerLinear)
get_update = ParameterUpdate(model, transform)
opt = torch.optim.SGD(model.parameters() + get_update.parameters())

for iteration in range(10):
    opt.zero_grad()
    error = loss(model(X), y)
    updates = get_update(
        error,
        model.parameters(),
        create_graph=True,
    )
    l2l.update_module(model, updates)
    opt.step()
```

--------------------------------

### MAML

Source: http://learn2learn.net/docs/learn2learn.algorithms

High-level implementation of Model-Agnostic Meta-Learning. It wraps an arbitrary nn.Module and augments it with clone() and adapt() methods. Supports first-order approximation via the first_order flag.

```APIDOC
## MAML

### Description
High-level implementation of _Model-Agnostic Meta-Learning_. This class wraps an arbitrary nn.Module and augments it with `clone()` and `adapt()` methods. For the first-order version of MAML (i.e. FOMAML), set the `first_order` flag to `True` upon initialization.

### Arguments
* **model** (Module) - Module to be wrapped.
* **lr** (float) - Fast adaptation learning rate.
* **first_order** (bool, _optional_ , default=False) - Whether to use the first-order approximation of MAML. (FOMAML)
* **allow_unused** (bool, _optional_ , default=None) - Whether to allow differentiation of unused parameters. Defaults to `allow_nograd`.
* **allow_nograd** (bool, _optional_ , default=False) - Whether to allow adaptation with parameters that have `requires_grad = False`.

### Method: adapt
#### `adapt(self, loss, first_order=None, allow_unused=None, allow_nograd=None)`

**Description**
Takes a gradient step on the loss and updates the cloned parameters in place.

**Arguments**
* **loss** (Tensor) - Loss to minimize upon update.
* **first_order** (bool, _optional_ , default=None) - Whether to use first- or second-order updates. Defaults to self.first_order.
* **allow_unused** (bool, _optional_ , default=None) - Whether to allow differentiation of unused parameters. Defaults to self.allow_unused.
* **allow_nograd** (bool, _optional_ , default=None) - Whether to allow adaptation with parameters that have `requires_grad = False`. Defaults to self.allow_nograd.

### Method: clone
#### `clone(self, first_order=None, allow_unused=None, allow_nograd=None)`

**Description**
Returns a `MAML`-wrapped copy of the module whose parameters and buffers are `torch.clone`d from the original module. This implies that back-propagating losses on the cloned module will populate the buffers of the original module. For more information, refer to learn2learn.clone_module().

**Arguments**
* **first_order** (bool, _optional_ , default=None) - Whether the clone uses first- or second-order updates. Defaults to self.first_order.
* **allow_unused** (bool, _optional_ , default=None) - Whether to allow differentiation of unused parameters. Defaults to self.allow_unused.
* **allow_nograd** (bool, _optional_ , default=False) - Whether to allow adaptation with parameters that have `requires_grad = False`. Defaults to self.allow_nograd.
```

--------------------------------

### Load DescribableTextures Dataset

Source: http://learn2learn.net/docs/learn2learn.vision

Loads the DescribableTextures dataset for training. It is then wrapped with MetaDataset and TaskDataset for few-shot learning.

```python
train_dataset = l2l.vision.datasets.DescribableTextures(root='./data', mode='train')
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskDataset(dataset=train_dataset, num_tasks=1000)
```

--------------------------------

### Instantiate WRN28 Model

Source: http://learn2learn.net/docs/learn2learn.vision

Instantiate the WRN28 model for feature extraction or classification. Adjust hidden_size based on dataset requirements.

```python
model = WRN28(output_size=ways, hidden_size=1600, avg_pool=False)
```

--------------------------------

### Create TaskDataset with NWays, KShots, and LoadData transforms

Source: http://learn2learn.net/docs/learn2learn.data

Generates a set of tasks from a given MetaDataset using specified task transformations. Tasks are sampled lazily upon indexing.

```python
dataset = l2l.data.MetaDataset(MyDataset())
transforms = [
    l2l.data.transforms.NWays(dataset, n=5),
    l2l.data.transforms.KShots(dataset, k=1),
    l2l.data.transforms.LoadData(dataset),
]
taskset = TaskDataset(dataset, transforms, num_tasks=20000)
for task in taskset:
    X, y = task
```

--------------------------------

### Load Mini-Imagenet Dataset

Source: http://learn2learn.net/docs/learn2learn.vision

Load the Mini-Imagenet dataset for a specific mode (train, validation, or test). Wrap with MetaDataset and TaskGenerator for meta-learning.

```python
train_dataset = l2l.vision.datasets.MiniImagenet(root='./data', mode='train')
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskGenerator(dataset=train_dataset, ways=ways)
```

--------------------------------

### Convert Dataset to OnDeviceDataset

Source: http://learn2learn.net/docs/learn2learn.data

Converts a dataset to a TensorDataset and optionally moves it to a specified device. Useful for accelerating training with smaller datasets.

```python
transforms = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,)),
    lambda x: x.view(1, 28, 28),
])
mnist = MNIST('~/data')
mnist_ondevice = OnDeviceDataset(mnist, device='cuda', transform=transforms)
mnist_meta = MetaDataset(mnist_ondevice)
```

--------------------------------

### Load CUBirds200 Dataset

Source: http://learn2learn.net/docs/learn2learn.vision

Loads the CUBirds200 dataset for training. It is then wrapped with MetaDataset and TaskDataset for few-shot learning.

```python
train_dataset = l2l.vision.datasets.CUBirds200(root='./data', mode='train')
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskDataset(dataset=train_dataset, num_tasks=1000)
```

--------------------------------

### Create UnionMetaDataset from CIFARFS datasets

Source: http://learn2learn.net/docs/learn2learn.data

Combines multiple MetaDatasets into a single union. Labels are remapped to be consecutive.

```python
train = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="train")
train = l2l.data.MetaDataset(train)
valid = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="validation")
valid = l2l.data.MetaDataset(valid)
test = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="test")
test = l2l.data.MetaDataset(test)
union = UnionMetaDataset([train, valid, test])
assert len(union.labels) == 100
```

--------------------------------

### Instantiate OmniglotCNN Network

Source: http://learn2learn.net/docs/learn2learn.vision

Instantiate OmniglotCNN for convolutional networks on Omniglot datasets. This model assumes input shapes of (1, 28, 28) and can be configured with hidden size and number of layers.

```python
model = OmniglotCNN(output_size=20, hidden_size=128, layers=3)
```

--------------------------------

### Quickdraw Dataset

Source: http://learn2learn.net/docs/learn2learn.vision

The Quickdraw dataset consists of approximately 50 million drawing images across 345 object categories, each represented as a 28x28 pixel black-and-white array. It is suitable for few-shot learning tasks.

```APIDOC
## Quickdraw (Dataset)

### Description
The Quickdraw dataset was originally introduced by Google Creative Lab in 2017 and then re-purposed for few-shot learning in Triantafillou et al., 2020. The dataset consists of roughly 50M drawing images of 345 objects. Each image was hand-drawn by human annotators and is represented as black-and-white 28x28 pixel array. We follow the train-validation-test splits of Triantafillou et al., 2020. (241 classes for train, 52 for validation, and 52 for test.)

### Arguments
* **root** (str) - Path to download the data.
* **mode** (str, _optional_ , default='train') - Which split to use. Must be 'train', 'validation', or 'test'.
* **transform** (Transform, _optional_ , default=None) - Input pre-processing.
* **target_transform** (Transform, _optional_ , default=None) - Target pre-processing.
* **download** (bool, _optional_ , default=False) - Whether to download the dataset.

### Example
```python
train_dataset = l2l.vision.datasets.Quickdraw(root='./data', mode='train')
train_dataset = l2l.data.MetaDataset(train_dataset)
train_generator = l2l.data.TaskDataset(dataset=train_dataset, num_tasks=1000)
```
```

--------------------------------

### DifferentiableSGD Update

Source: http://learn2learn.net/docs/learn2learn.optim

Illustrates how to use DifferentiableSGD to apply gradients to a module's parameters in a differentiable manner. The module is updated in-place.

```python
sgd = DifferentiableSGD(0.1)
gradients = torch.autograd.grad(
    loss,
    model.parameters(),
    create_gaph=True)
sgd(model, gradients)  # model is updated in-place
```

--------------------------------

### clone_module

Source: http://learn2learn.net/docs/learn2learn

Creates a copy of a module, preserving the computational graph for backpropagation.

```APIDOC
## clone_module(module, memo=None)

### Description
Creates a copy of a module, whose parameters/buffers/submodules are created using PyTorch's torch.clone(). This implies that the computational graph is kept, and you can compute the derivatives of the new modules' parameters w.r.t the original parameters.

### Arguments
* **module** (Module) - Module to be cloned.

### Return
* (Module) - The cloned module.

### Example
```python
net = nn.Sequential(Linear(20, 10), nn.ReLU(), nn.Linear(10, 2))
clone = clone_module(net)
error = loss(clone(X), y)
error.backward()  # Gradients are back-propagate all the way to net.
```
```

--------------------------------

### KShots

Source: http://learn2learn.net/docs/learn2learn.data

Keeps K samples for each present label in a dataset. Allows sampling with or without replacement.

```APIDOC
## KShots

### Description
Keeps K samples for each present label.

### Arguments
* **dataset** (Dataset) - The dataset from which to load the sample.
* **k** (int, optional, default=1) - The number of samples per label.
* **replacement** (bool, optional, default=False) - Whether to sample with replacement.
```

--------------------------------

### LoadData Transform

Source: http://learn2learn.net/docs/learn2learn.data

Loads a sample from the dataset given its index. This is a task transformation.

```APIDOC
## LoadData

### Description
Loads a sample from the dataset given its index.

### Arguments
* **dataset** (Dataset) - The dataset from which to load the sample.
```

--------------------------------

### LearnableOptimizer

Source: http://learn2learn.net/docs/learn2learn.optim

A PyTorch Optimizer with a learnable transform, designed for meta-descent and hypergradient algorithms. It updates a given module using a specified transform and allows for the differentiable computation of both module and transform gradients.

```APIDOC
## LearnableOptimizer

### Description
A PyTorch Optimizer with learnable transform, enabling the implementation of meta-descent / hypergradient algorithms. This optimizer takes a Module and a gradient transform. At each step, the gradient of the module is passed through the transforms, and the module differentiably update -- i.e. when the next backward is called, gradients of both the module and the transform are computed. In turn, the transform can be updated via your favorite optmizer.

### Arguments
  * **model** (Module) - Module to be updated.
  * **transform** (Module) - Transform used to compute updates of the model.
  * **lr** (float) - Learning rate.

### Method
`LearnableOptimizer(model, transform, lr)`

### Example
```python
linear = nn.Linear(784, 10)
transform = l2l.optim.ModuleTransform(torch.nn.Linear)
metaopt = l2l.optim.LearnableOptimizer(linear, transform, lr=0.01)
opt = torch.optim.SGD(metaopt.parameters(), lr=0.001)

metaopt.zero_grad()
opt.zero_grad()
error = loss(linear(X), y)
error.backward()
opt.step()  # 
metaopt.step()  # 
```

### Method
`zero_grad(self)`

### Description
Only reset target parameters.
```

--------------------------------

### list_tasksets

Source: http://learn2learn.net/docs/learn2learn.vision

Returns a list of all available benchmark names within the learn2learn vision module. This function helps users discover which datasets and task configurations are pre-defined.

```APIDOC
## learn2learn.vision.benchmarks.list_tasksets()

### Description
Returns a list of all available benchmarks.

### Example
```python
for name in l2l.vision.benchmarks.list_tasksets():
    print(name)
    tasksets = l2l.vision.benchmarks.get_tasksets(name)
```
```

--------------------------------

### OnDeviceDataset

Source: http://learn2learn.net/docs/learn2learn.data

Converts a dataset into a TensorDataset and optionally moves it to a specified device (CPU or GPU). Useful for accelerating training with small datasets.

```APIDOC
## OnDeviceDataset

### Description
Converts an entire dataset into a TensorDataset, and optionally puts it on the desired device. Useful to accelerate training with relatively small datasets. If the device is cpu and cuda is available, the TensorDataset will live in pinned memory.

### Arguments
* **dataset** (Dataset) - Dataset to put on a device.
* **device** (torch.device, optional, default=None) - Device of dataset. Defaults to CPU.
* **transform** (transform, optional, default=None) - Transform to apply on the first variate of the dataset's samples X.

### Request Example
```python
transforms = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,)),
    lambda x: x.view(1, 28, 28),
])
mnist = MNIST('~/data')
mnist_ondevice = OnDeviceDataset(mnist, device='cuda', transform=transforms)
mist_meta = MetaDataset(mnist_ondevice)
```
```

--------------------------------

### Create Infinite DataLoader Iterator

Source: http://learn2learn.net/docs/learn2learn.data

Creates an iterator that infinitely loops over a given DataLoader. Ensures a fixed number of iterations regardless of the DataLoader's length.

```python
dataloader = DataLoader(dataset, shuffle=True, batch_size=32)
inf_dataloader = InfiniteIterator(dataloader)
for iteration in range(10000):  # guaranteed to reach 10,000 regardless of len(dataloader)
    X, y = next(inf_dataloader)
```

--------------------------------

### Clone a PyTorch Module

Source: http://learn2learn.net/docs/learn2learn

Creates a copy of a module, preserving the computational graph for backpropagation. Use this when you need to modify a module's parameters or perform operations on a copy without affecting the original.

```python
net = nn.Sequential(Linear(20, 10), nn.ReLU(), nn.Linear(10, 2))
clone = clone_module(net)
error = loss(clone(X), y)
error.backward()  # Gradients are back-propagate all the way to net.
```

--------------------------------

### ConsecutiveLabels

Source: http://learn2learn.net/docs/learn2learn.data

Re-orders samples in a task description to be sorted in consecutive order.

```APIDOC
## ConsecutiveLabels

### Description
Re-orders the samples in the task description such that they are sorted in consecutive order.
Note: when used before `RemapLabels`, the labels will be homogeneously clustered, but in no specific order.

### Arguments
* **dataset** (Dataset) - The dataset from which to load the sample.
```

--------------------------------

### UnionMetaDataset

Source: http://learn2learn.net/docs/learn2learn.data

Takes multiple MetaDatasets and constructs their union. Labels from all datasets are remapped to be in consecutive order.

```APIDOC
## UnionMetaDataset

### Description
Takes multiple MetaDataests and constructs their union.
Note: The labels of all datasets are remapped to be in consecutive order. (i.e. the same label in two datasets will be to two different labels in the union)

### Arguments
* **datasets** (list of Dataset) - A list of torch Datasets.

### Example
```python
train = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="train")
train = l2l.data.MetaDataset(train)
valid = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="validation")
valid = l2l.data.MetaDataset(valid)
test = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="test")
test = l2l.data.MetaDataset(test)
union = UnionMetaDataset([train, valid, test])
assert len(union.labels) == 100
```
```

--------------------------------

### Instantiate CNN4 Network

Source: http://learn2learn.net/docs/learn2learn.vision

Use CNN4 for convolutional networks on MiniImagenet datasets. It assumes input shapes of (3, 84, 84) and can be customized with hidden size, layers, channels, max-pooling, and embedding size.

```python
model = CNN4(output_size=20, hidden_size=128, layers=3)
```

--------------------------------

### FusedNWaysKShots

Source: http://learn2learn.net/docs/learn2learn.data

An efficient implementation combining filtering, N-ways selection, and K-shots sampling.

```APIDOC
## FusedNWaysKShots

### Description
Efficient implementation of FilterLabels, NWays, and KShots.

### Arguments
* **dataset** (Dataset) - The dataset from which to load the sample.
* **n** (int, optional, default=2) - Number of labels to sample from the task description's labels.
* **k** (int, optional, default=1) - The number of samples per label.
* **replacement** (bool, optional, default=False) - Whether to sample shots with replacement.
* **filter_labels** (list, optional, default=None) - The list of labels to include. Defaults to all labels in the dataset.
```

--------------------------------

### TaskDataset

Source: http://learn2learn.net/docs/learn2learn.data

Creates a set of tasks from a given Dataset, optionally using task transformations. Tasks are lazily sampled upon indexing.

```APIDOC
## TaskDataset

### Description
Creates a set of tasks from a given Dataset. In addition to the Dataset, TaskDataset accepts a list of task transformations (`task_transforms`) which define the kind of tasks sampled from the dataset. The tasks are lazily sampled upon indexing (or calling the `.sample()` method), and their descriptions cached for later use. If `num_tasks` is -1, the TaskDataset will not cache task descriptions and instead continuously resample new ones. In this case, the length of the TaskDataset is set to 1.
For more information on tasks and task descriptions, please refer to the documentation of task transforms.

### Arguments
* **dataset** (Dataset) - Dataset of data to compute tasks.
* **task_transforms** (list, _optional_, default=None) - List of task transformations.
* **num_tasks** (int, _optional_, default=-1) - Number of tasks to generate.

### Example
```python
dataset = l2l.data.MetaDataset(MyDataset())
transforms = [
    l2l.data.transforms.NWays(dataset, n=5),
    l2l.data.transforms.KShots(dataset, k=1),
    l2l.data.transforms.LoadData(dataset),
]
taskset = TaskDataset(dataset, transforms, num_tasks=20000)
for task in taskset:
    X, y = task
```
```

--------------------------------

### MetaDataset

Source: http://learn2learn.net/docs/learn2learn.data

Wraps a classification dataset to enable fast indexing of samples within classes. It provides `labels_to_indices` and `indices_to_labels` attributes for efficient task creation.

```APIDOC
## MetaDataset

### Description
Wraps a classification dataset to enable fast indexing of samples within classes. This class exposes two attributes specific to the wrapped dataset: `labels_to_indices`: maps a class label to a list of sample indices with that label. `indices_to_labels`: maps a sample index to its corresponding class label.

Those dictionary attributes are often used to quickly create few-shot classification tasks. They can be passed as arguments upon instantiation, or automatically built on-the-fly. If the wrapped dataset has an attribute `_bookkeeping_path`, then the built attributes will be cached on disk and reloaded upon the next instantiation. This caching strategy is useful for large datasets (e.g. ImageNet-1k) where the first instantiation can take several hours.
Note that if only one of `labels_to_indices` or `indices_to_labels` is provided, this class builds the other one from it.

### Arguments
* **dataset** (Dataset) - A torch Dataset.
* **labels_to_indices** (dict, **optional**, default=None) - A dictionary mapping labels to the indices of their samples.
* **indices_to_labels** (dict, **optional**, default=None) - A dictionary mapping sample indices to their corresponding label.

### Example
```python
mnist = torchvision.datasets.MNIST(root="/tmp/mnist", train=True)
mnist = l2l.data.MetaDataset(mnist)
```
```

--------------------------------

### partition_task

Source: http://learn2learn.net/docs/learn2learn.data

Partitions a classification task's data and labels into support and query sets, with a specified number of shots per class for the support set.

```APIDOC
## partition_task

### Description
Partitions a classification task into support and query sets. The support set will contain `shots` samples per class, the query will take the remaining samples. Assumes each class in `labels` is associated with the same number of samples in `data`.

### Arguments
* **data** (Tensor) - Data to be partitioned into support and query.
* **labels** (Tensor) - Labels of each data sample, used for partitioning.
* **shots** (int, optional, default=1) - Number of data samples per class in the support set.

### Request Example
```python
X, y = taskset.sample()
(X_support, y_support), (X_query, y_query) = partition_task(X, y, shots=5)
```
```

--------------------------------

### KroneckerTransform for Linear Layers

Source: http://learn2learn.net/docs/learn2learn.optim

Utilize KroneckerTransform with KroneckerLinear for memory and computationally efficient optimization of parameters that admit a Kronecker factorization. This maps gradients to updates.

```python
classifier = torch.nn.Linear(784, 10, bias=False)
kronecker_transform = KroneckerTransform(l2l.nn.KroneckerLinear)
kronecker_update = kronecker_transform(classifier.weight)
loss(classifier(X), y).backward()
update = kronecker_update(classifier.weight.grad)
classifier.weight.data.add_(-lr, update)  # Not a differentiable update. See l2l.optim.DifferentiableSGD.
```

--------------------------------

### InfiniteIterator

Source: http://learn2learn.net/docs/learn2learn.data

Creates an iterator that infinitely loops over a given dataloader, ensuring continuous data availability.

```APIDOC
## InfiniteIterator

### Description
Infinitely loops over a given iterator.

### Arguments
* **dataloader** (iterator) - Iterator to loop over.

### Request Example
```python
dataloader = DataLoader(dataset, shuffle=True, batch_size=32)
inf_dataloader = InfiniteIterator(dataloader)
for iteration in range(10000):  # guaranteed to reach 10,000 regardless of len(dataloader)
    X, y = next(inf_dataloader)
```
```

--------------------------------

### Update Module Parameters with Differentiability

Source: http://learn2learn.net/docs/learn2learn

Updates a module's parameters in-place while preserving differentiability. This is useful for meta-learning algorithms where parameter updates need to be part of the computational graph. If `updates` is not provided, it uses the tensors in the `.update` attributes.

```python
error = loss(model(X), y)
grads = torch.autograd.grad(
    error,
    model.parameters(),
    create_graph=True,
)
updates = [-lr * g for g in grads]
l2l.update_module(model, updates=updates)
```

--------------------------------

### Apply Random Class Rotation Transform

Source: http://learn2learn.net/docs/learn2learn.vision

Creates a RandomClassRotation transform to apply specified rotations to images within a class.

```python
transform = RandomClassRotation([0, 90, 180, 270])
```

--------------------------------

### get_tasksets

Source: http://learn2learn.net/docs/learn2learn.vision

Retrieves tasksets for a specified benchmark, including standard data and task transformations. It returns a namedtuple containing train, validation, and test TaskDatasets.

```APIDOC
## learn2learn.vision.benchmarks.get_tasksets(name, train_ways=5, train_samples=10, test_ways=5, test_samples=10, num_tasks=-1, root='~/data', device=None, **kwargs)

### Description
Returns the tasksets for a particular benchmark, using literature standard data and task transformations. The returned object is a namedtuple with attributes `train`, `validation`, `test` which correspond to their respective TaskDatasets.

### Arguments
* **name** (str) - The name of the benchmark. Full list in `list_tasksets()`.
* **train_ways** (int, _optional_ , default=5) - The number of classes per train tasks.
* **train_samples** (int, _optional_ , default=10) - The number of samples per train tasks.
* **test_ways** (int, _optional_ , default=5) - The number of classes per test tasks. Also used for validation tasks.
* **test_samples** (int, _optional_ , default=10) - The number of samples per test tasks. Also used for validation tasks.
* **num_tasks** (int, _optional_ , default=-1) - The number of tasks in each TaskDataset.
* **device** (torch.Device, _optional_ , default=None) - If not None, tasksets are loaded as Tensors on `device`.
* **root** (str, _optional_ , default='~/data') - Where the data is stored.

### Example
```python
train_tasks, validation_tasks, test_tasks = l2l.vision.benchmarks.get_tasksets('omniglot')
batch = train_tasks.sample()

or:

tasksets = l2l.vision.benchmarks.get_tasksets('omniglot')
batch = tasksets.train.sample()
```
```

--------------------------------

### RemapLabels

Source: http://learn2learn.net/docs/learn2learn.data

Maps the labels of samples from K classes to a consecutive range from 0 to K.

```APIDOC
## RemapLabels

### Description
Given samples from K classes, maps the labels to 0, ..., K.

### Arguments
* **dataset** (Dataset) - The dataset from which to load the sample.
```

--------------------------------

### ParameterUpdate

Source: http://learn2learn.net/docs/learn2learn.optim

A convenience class for implementing custom update functions. It computes parameter updates by first calculating gradients and then passing them through a specified transform, supporting parameters that may not require updates.

```APIDOC
## ParameterUpdate

### Description
Convenience class to implement custom update functions. Objects instantiated from this class behave similarly to `torch.autograd.grad`, but return parameter updates as opposed to gradients. Concretely, the gradients are first computed, then fed to their respective transform whose output is finally returned to the user. Additionally, this class supports parameters that might not require updates by setting the `allow_nograd` flag to True. In this case, the returned update is `None`.

### Arguments
  * **parameters** (list) - Parameters of the model to update.
  * **transform** (callable) - A callable that returns an instantiated transform given a parameter.

### Method
`ParameterUpdate(parameters, transform)`

### Example
```python
model = torch.nn.Linear()
transform = l2l.optim.KroneckerTransform(l2l.nn.KroneckerLinear)
get_update = ParameterUpdate(model, transform)
opt = torch.optim.SGD(model.parameters() + get_update.parameters())

for iteration in range(10):
    opt.zero_grad()
    error = loss(model(X), y)
    updates = get_update(
        error,
        model.parameters(),
        create_graph=True,
    )
    l2l.update_module(model, updates)
    opt.step()
```

### Method
`forward(self, loss, parameters, create_graph=False, retain_graph=False, allow_unused=False, allow_nograd=False)`

### Description
Similar to torch.autograd.grad, but passes the gradients through the provided transform.

### Arguments
  * **loss** (Tensor) - The loss to differentiate.
  * **parameters** (iterable) - Parameters w.r.t. which we want to compute the update.
  * **create_graph** (bool, _optional_ , default=False) - Same as `torch.autograd.grad`.
  * **retain_graph** (bool, _optional_ , default=False) - Same as `torch.autograd.grad`.
  * **allow_unused** (bool, _optional_ , default=False) - Same as `torch.autograd.grad`.
  * **allow_nograd** (bool, _optional_ , default=False) - Properly handles parameters that do not require gradients. (Their update will be `None`.)
```

--------------------------------

### OmniglotFC

Source: http://learn2learn.net/docs/learn2learn.vision

The fully-connected network used for Omniglot experiments, as described in Santoro et al, 2016.

```APIDOC
## OmniglotFC

### Description
The fully-connected network used for Omniglot experiments, as described in Santoro et al, 2016.

### Arguments
* **input_size** (int) - The dimensionality of the input.
* **output_size** (int) - The dimensionality of the output.
* **sizes** (list, _optional_ , default=None) - A list of hidden layer sizes.

### Example
```python
net = OmniglotFC(input_size=28**2,
                 output_size=10,
                 sizes=[64, 64, 64])
```
```

--------------------------------

### ModuleTransform for Linear Layers

Source: http://learn2learn.net/docs/learn2learn.optim

Use ModuleTransform to create an optimization transform for a PyTorch Linear module. This maps gradients to updates for a given parameter.

```python
classifier = torch.nn.Linear(784, 10, bias=False)
linear_transform = ModuleTransform(torch.nn.Linear)
linear_update = linear_transform(classifier.weight)  # maps gradients to updates, both of shape (1, 7840)
loss(classifier(X), y).backward()
update = linear_update(classifier.weight.grad)
classifier.weight.data.add_(-lr, update)  # Not a differentiable update. See l2l.optim.DifferentiableSGD.
```

--------------------------------

### NWays Transform

Source: http://learn2learn.net/docs/learn2learn.data

Keeps samples from N random labels present in the task description. This is a task transformation.

```APIDOC
## NWays

### Description
Keeps samples from N random labels present in the task description.

### Arguments
* **dataset** (Dataset) - The dataset from which to load the sample.
* **n** (int, _optional_, default=2) - Number of labels to sample from the task description's labels.
```

--------------------------------

### ResNet12

Source: http://learn2learn.net/docs/learn2learn.vision

The 12-layer residual network from Mishra et al, 2017. The code is adapted from Lee et al, 2019 who share it under the Apache 2 license. Instantiate `ResNet12Backbone` if you only need the feature extractor.

```APIDOC
## ResNet12

### Description
The 12-layer residual network from Mishra et al, 2017.
The code is adapted from Lee et al, 2019 who share it under the Apache 2 license.
Instantiate `ResNet12Backbone` if you only need the feature extractor.
List of changes:
  * Rename ResNet to ResNet12.
  * Small API modifications.
  * Fix code style to be compatible with PEP8.
  * Support multiple devices in DropBlock

### Arguments
* **output_size** (int) - The dimensionality of the output (eg, number of classes).
* **hidden_size** (list, _optional_ , default=640) - Size of the embedding once features are extracted. (640 is for mini-ImageNet; used for the classifier layer)
* **avg_pool** (bool, _optional_ , default=True) - Set to False for the 16k-dim embeddings of Lee et al, 2019.
* **wider** (bool, _optional_ , default=True) - True uses (64, 160, 320, 640) filters akin to Lee et al, 2019. False uses (64, 128, 256, 512) filters, akin to Oreshkin et al, 2018.
* **embedding_dropout** (float, _optional_ , default=0.0) - Dropout rate on the flattened embedding layer.
* **dropblock_dropout** (float, _optional_ , default=0.1) - Dropout rate for the residual layers.
* **dropblock_size** (int, _optional_ , default=5) - Size of drop blocks.

### Example
```python
model = ResNet12(output_size=ways, hidden_size=1600, avg_pool=False)
```
```

--------------------------------

### Create FilteredMetaDataset from a MetaDataset

Source: http://learn2learn.net/docs/learn2learn.data

Filters a MetaDataset to include only a subset of specified labels. Labels are not remapped.

```python
train = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="train")
train = l2l.data.MetaDataset(train)
filtered = FilteredMetaDataset(train, [4, 8, 2, 1, 9])
assert len(filtered.labels) == 5
```

--------------------------------

### MetaCurvatureTransform for Parameter Gradients

Source: http://learn2learn.net/docs/learn2learn.optim

Apply MetaCurvatureTransform directly to a parameter to instantiate the transform for gradient updates. This is based on the Meta-Curvature transform by Park and Oliva, 2019.

```python
classifier = torch.nn.Linear(784, 10, bias=False)
metacurvature_update = MetaCurvatureTransform(classifier.weight)
loss(classifier(X), y).backward()
update = metacurvature_update(classifier.weight.grad)
classifier.weight.data.add_(-lr, update)  # Not a differentiable update. See l2l.optim.DifferentiableSGD.
```

--------------------------------

### update_module

Source: http://learn2learn.net/docs/learn2learn

Updates the parameters of a module in-place while preserving differentiability.

```APIDOC
## update_module(module, updates=None, memo=None)

### Description
Updates the parameters of a module in-place, in a way that preserves differentiability. The parameters of the module are swapped with their update values, according to: p←p+u, where p is the parameter, and u is its corresponding update.

### Arguments
* **module** (Module) - The module to update.
* **updates** (list, _optional_ , default=None) - A list of gradients for each parameter of the model. If None, will use the tensors in .update attributes.

### Example
```python
error = loss(model(X), y)
grads = torch.autograd.grad(
    error,
    model.parameters(),
    create_graph=True,
)
updates = [-lr * g for g in grads]
l2l.update_module(model, updates=updates)
```
```