### Python Doctest Example

Source: https://github.com/mosaicml/composer/blob/main/STYLE_GUIDE.md

Illustrates how to structure a Python function with doctests, including setup and code blocks. Use `.. testsetup::` for setup and `.. testcode::` for the actual test code.

```python
import torch
from typing import Optional

def my_function(x: Optional[torch.Tensor]) -> torch.Tensor:
    """blah function

    Args:
        input (torch.Tensor): Your guess.

    Returns:
        torch.Tensor: How good your input is.

    Raises:
        ValueError: If your input is negative.

    Example:
        .. testsetup::

            # optional setup section, not shown in docs
            import torch
            x = torch.randn(42)


        .. testcode::

            # shown in docs; runs after testsetup
            my_function(x)
    """
    ...
```

--------------------------------

### Instantiate and Use SWA with Composer Trainer

Source: https://github.com/mosaicml/composer/blob/main/composer/algorithms/swa/README.md

This example demonstrates how to instantiate the SWA algorithm with specified start and end epochs and update intervals, and then pass it to the Composer Trainer.

```APIDOC
## Instantiate and Use SWA with Composer Trainer

### Description
Instantiate the SWA algorithm with specific parameters and integrate it into the Composer Trainer. The trainer will automatically apply SWA at the designated points in the training loop.

### Method
```python
from composer.algorithms import SWA
from composer.trainer import Trainer

# Instantiate SWA with custom parameters
swa_algorithm = SWA(
    swa_start="1ep",
    swa_end="2ep",
    update_interval='5ba',
)

# Instantiate the Trainer with the SWA algorithm
trainer = Trainer(
    model=model, # Assuming 'model' is already defined
    train_dataloader=train_dataloader, # Assuming 'train_dataloader' is already defined
    eval_dataloader=eval_dataloader, # Assuming 'eval_dataloader' is already defined
    max_duration="3ep",
    algorithms=[swa_algorithm]
)

# Start training
trainer.fit()
```

### Parameters
* `swa_start` (str): Specifies when SWA begins. Defaults to '0.7dur'.
* `swa_end` (str): Specifies when SWA ends. Defaults to '0.97dur'.
* `update_interval` (str): Defines the frequency for updating the moving average weights. Example: '5ba' (every 5 batches).

### Request Example
```json
{
  "swa_start": "1ep",
  "swa_end": "2ep",
  "update_interval": "5ba"
}
```

### Response
This is an example of how to configure the SWA algorithm within the Composer Trainer. The `trainer.fit()` method initiates the training process with SWA applied.
```

--------------------------------

### Install Composer

Source: https://github.com/mosaicml/composer/blob/main/examples/checkpoint_autoresume.ipynb

Install the Composer library. Use the commented-out command to install from source instead of the latest release.

```python
%pip install mosaicml 
# To install from source instead of the last release, comment the command above and uncomment the following one.
# %pip install git+https://github.com/mosaicml/composer.git
```

--------------------------------

### Install Composer and Submitit

Source: https://github.com/mosaicml/composer/blob/main/examples/training_with_submitit.ipynb

Install the necessary libraries for distributed training with Submitit and Composer.

```python
%pip install mosaicml
%pip install submitit
```

--------------------------------

### Install Composer with All Optional Dependencies

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with all optional dependencies included.

```bash
pip install 'mosaicml[all]'
```

--------------------------------

### Install Composer with NLP Dependencies

Source: https://github.com/mosaicml/composer/blob/main/examples/finetune_huggingface.ipynb

Install Composer with the necessary NLP and TensorBoard dependencies. Use the commented-out command to install from source if needed.

```python
%pip install 'mosaicml[nlp, tensorboard]'
# To install from source instead of the last release, comment the command above and uncomment the following one.
# %pip install 'mosaicml[nlp, tensorboard] @ git+https://github.com/mosaicml/composer.git'
```

--------------------------------

### Install Composer with TensorBoard Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for TensorBoard.

```bash
pip install 'mosaicml[tensorboard]'
```

--------------------------------

### Install Composer with OCI Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for OCI.

```bash
pip install 'mosaicml[oci]'
```

--------------------------------

### Install Composer and torch_xla

Source: https://github.com/mosaicml/composer/blob/main/examples/TPU_Training_in_composer.ipynb

Installs the Composer library and necessary packages for TPU training, including torch_xla. Use the commented-out lines to install from source.

```python
%pip install mosaicml
# To install from source instead of the last release, comment the command above and uncomment the following one.
# %pip install git+https://github.com/mosaicml/composer.git

%pip install cloud-tpu-client==0.10 torch==1.12.0 https://storage.googleapis.com/tpu-pytorch/wheels/colab/torch_xla-1.12-cp37-cp37m-linux_x86_64.whl

# To install from source instead of the last release, comment the command above and uncomment the following one.
# %pip install 'mosaicml @ git+https://github.com/mosaicml/composer.git'
```

--------------------------------

### Install Composer and Matplotlib

Source: https://github.com/mosaicml/composer/blob/main/examples/custom_speedup_methods.ipynb

Installs the Composer library and Matplotlib for visualization. Use the second command to install from source.

```python
%pip install mosaicml
# To install from source instead of the last release, comment the command above and uncomment the following one.
# %pip install git+https://github.com/mosaicml/composer.git

%pip install matplotlib
```

--------------------------------

### Install Composer with Streaming Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for the streaming library.

```bash
pip install 'mosaicml[streaming]'
```

--------------------------------

### Install Composer

Source: https://github.com/mosaicml/composer/blob/main/examples/auto_microbatching.ipynb

Install Composer using pip. Uncomment the second command to install from source.

```python
%pip install mosaicml
# To install from source instead of the last release, comment the command above and uncomment the following one.
# %pip install git+https://github.com/mosaicml/composer.git
```

--------------------------------

### Build Documentation Locally

Source: https://github.com/mosaicml/composer/blob/main/CONTRIBUTING.md

Follow these steps to build the documentation locally and verify your changes. Ensure you are in the docs directory and have installed the necessary dependencies.

```bash
cd docs
pip install -e '.[docs]'
make clean && make html
make host   # open the output link in a browser.
```

--------------------------------

### Install Composer with ONNX Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for ONNX.

```bash
pip install 'mosaicml[onnx]'
```

--------------------------------

### Start Training with Trainer.fit()

Source: https://github.com/mosaicml/composer/blob/main/examples/finetune_huggingface.ipynb

Initiates the fine-tuning process using the configured Composer Trainer.

```python
# Start training
trainer.fit()
```

--------------------------------

### Install Composer with Libcloud Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for libcloud.

```bash
pip install 'mosaicml[libcloud]'
```

--------------------------------

### Combine FSDP and Tensor Parallelism (TP)

Source: https://github.com/mosaicml/composer/blob/main/docs/source/notes/distributed_training.rst

Example of initializing a Composer Trainer with both FSDP and Tensor Parallelism configurations. This setup allows for advanced model parallelism by sharding both parameters and tensors across devices.

```python
import torch.nn as nn
from torch.distributed.tensor.parallel import ColwiseParallel, RowwiseParallel
from composer import Trainer

class Block(nn.Module):
    ...

class Model(nn.Module):
    def __init__(self, n_layers):
        super().__init__()
        self.blocks = nn.ModuleList([
            Block(...) for _ in range(n_layers)
        ]),
        self.head = nn.Linear(...)

    def forward(self, inputs):
        ...

    # FSDP Wrap Function
    def fsdp_wrap_fn(self, module):
        return isinstance(module, Block)

    # Activation Checkpointing Function
    def activation_checkpointing_fn(self, module):
        return isinstance(module, Block)


class MyComposerModel(ComposerModel):

    def __init__(self, n_layers):
        super().__init__()
        self.model = Model(n_layers)
        ...

    def forward(self, batch):
        ...

    def eval_forward(self, batch, outputs=None):
        ...

    def loss(self, outputs, batch):
        ...

    ...

composer_model = MyComposerModel(n_layers=3)

fsdp_config = {
    'sharding_strategy': 'FULL_SHARD',
    'cpu_offload': False, # Not supported yet
    'mixed_precision': 'DEFAULT',
    'backward_prefetch': 'BACKWARD_POST',
    'activation_checkpointing': False,
    'activation_cpu_offload': False,
    'verbose': True
}
tp_config = {
    'tensor_parallel_degree': 2,
    layer_plan = {
        'model.0.fc1': ColwiseParallel(),
        'model.0.fc2': RowwiseParallel(),
    }
}

trainer = Trainer(
    model=composer_model,
    parallelism_config={
        'fsdp': fsdp_config,
        'tp': tp_config,
    },
    ...
)

trainer.fit()
```

--------------------------------

### Install Composer with Neptune Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for Neptune.

```bash
pip install 'mosaicml[neptune]'
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/mosaicml/composer/blob/main/CONTRIBUTING.md

Installs all dependencies required for testing and linting the Composer code.

```bash
pip install -e '.[all]'
```

--------------------------------

### Install Composer and Pre-Commit Hooks

Source: https://github.com/mosaicml/composer/blob/main/STYLE_GUIDE.md

Install Composer with development dependencies and set up pre-commit hooks to enforce style checks before each commit.

```bash
pip install '.[dev]'
pre-commit install
```

--------------------------------

### Run Profiler Demo Script (Single GPU)

Source: https://github.com/mosaicml/composer/blob/main/docs/source/trainer/performance_tutorials/profiling.md

Execute the profiler demonstration script on a single GPU. Ensure the script is in the examples directory.

```bash
python examples/profiler_demo.py
```

--------------------------------

### Install Composer with MLflow Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for MLflow.

```bash
pip install 'mosaicml[mlflow]'
```

--------------------------------

### Setup for Fine-tuning with Remote Checkpoints

Source: https://github.com/mosaicml/composer/blob/main/docs/source/notes/resumption.rst

This setup code configures a RemoteUploaderDownloader for S3 and trains a model to generate initial weights. It's a prerequisite for resuming training from object storage.

```python
from composer.loggers import RemoteUploaderDownloader
from composer.utils.object_store import S3ObjectStore

remote_uploader_downloader = RemoteUploaderDownloader(
    bucket_uri=f"s3://checkpoint-debugging_2",
)

# Train to generate and save the "pretrained_weights/model.pt",
# so we can load and resume from it
trainer = Trainer(
    ...,
    save_filename='pretrained_weights/model.pt',
    save_folder='checkpoints',
    run_name='my_cool_run',
    max_duration='1ep'
)

trainer.fit()
```

--------------------------------

### Developer Install Composer

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Clone the repository and perform a developer installation with all optional dependencies.

```bash
git clone https://github.com/mosaicml/composer.git
cd composer
pip install -e ".[all]"
```

--------------------------------

### Selective Backprop with Composer Trainer Example

Source: https://github.com/mosaicml/composer/blob/main/composer/algorithms/selective_backprop/README.md

This example demonstrates how to apply Selective Backprop when using the Composer Trainer. The algorithm is passed as an item in the 'algorithms' list.

```python
from composer.algorithms import LabelSmoothing
from composer.trainer import Trainer

trainer = Trainer(model=model,
                  train_dataloader=train_dataloader,
                  max_duration='1ep',
                  algorithms=[])

trainer.fit()
```

--------------------------------

### FSDP Integration with Composer Trainer

Source: https://github.com/mosaicml/composer/blob/main/docs/source/notes/distributed_training.rst

Example of initializing a Composer Trainer with FSDP configuration. Ensure FSDP is enabled in the parallelism_config.

```python
import torch.nn as nn
from composer import Trainer

class Block(nn.Module):
    ...

class Model(nn.Module):
    def __init__(self, n_layers):
        super().__init__()
        self.blocks = nn.ModuleList([
            Block(...) for _ in range(n_layers)
        ]),
        self.head = nn.Linear(...)

    def forward(self, inputs):
        ...

    # FSDP Wrap Function
    def fsdp_wrap_fn(self, module):
        return isinstance(module, Block)

    # Activation Checkpointing Function
    def activation_checkpointing_fn(self, module):
        return isinstance(module, Block)


class MyComposerModel(ComposerModel):

    def __init__(self, n_layers):
        super().__init__()
        self.model = Model(n_layers)
        ...

    def forward(self, batch):
        ...

    def eval_forward(self, batch, outputs=None):
        ...

    def loss(self, outputs, batch):
        ...

    ...

composer_model = MyComposerModel(n_layers=3)

fsdp_config = {
    'sharding_strategy': 'FULL_SHARD',
    'cpu_offload': False, # Not supported yet
    'mixed_precision': 'DEFAULT',
    'backward_prefetch': 'BACKWARD_POST',
    'activation_checkpointing': False,
    'activation_cpu_offload': False,
    'verbose': True
}


trainer = Trainer(
    model=composer_model,
    parallelism_config={'fsdp': fsdp_config},
    ...
)

trainer.fit()
```

--------------------------------

### Custom Algorithm Apply Method Example

Source: https://github.com/mosaicml/composer/blob/main/docs/source/trainer/algorithms.rst

An example of the apply method for a custom algorithm, demonstrating how to modify the trainer's state based on the event.

```python
def apply(self, event, state, logger):
    if event == Event.AFTER_DATALOADER:
        state.batch = process_inputs(state.batch)
    if event == Event.AFTER_FORWARD:
        state.output = process_outputs(state.outputs)
```

--------------------------------

### Install Composer with Weights & Biases Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for Weights & Biases (wandb).

```bash
pip install 'mosaicml[wandb]'
```

--------------------------------

### Load Checkpoint with LibcloudObjectStore

Source: https://github.com/mosaicml/composer/blob/main/docs/source/trainer/checkpointing.rst

Initialize a Trainer to load a checkpoint from a specified path using a LibcloudObjectStore. Ensure the Libcloud library is installed.

```python
from composer.utils import LibcloudObjectStore
from composer.trainer import Trainer

object_store = LibcloudObjectStore(
    provider="s3",  # The Apache Libcloud provider name
    container="checkpoint-debugging",  # The name of the cloud container (i.e. bucket) to use.
    provider_kwargs={
        'key': 'provider_key',  # The cloud provider key.
        'secret': '*******',  # The cloud provider secret.
    },
)

new_trainer = Trainer(
    model=model,
    train_dataloader=train_dataloader,
    max_duration="10ep",
    load_path="checkpoints/ep1.pt",
    load_object_store=object_store,
)

new_trainer.fit()
```

--------------------------------

### Inside Docker Container Setup

Source: https://github.com/mosaicml/composer/blob/main/CONTRIBUTING.md

Commands to run after entering the Composer Docker container to set up the environment and run tests.

```bash
cd /composer
pip install -e .
pytest <args> or make <args> to run the desired tests
```

--------------------------------

### Install ONNX and ONNX Runtime

Source: https://github.com/mosaicml/composer/blob/main/examples/exporting_for_inference.ipynb

Installs the necessary libraries for loading and running ONNX models. These are required for verifying the exported model.

```python
%pip install onnx
%pip install onnxruntime
```

--------------------------------

### Build and View Documentation Locally

Source: https://github.com/mosaicml/composer/blob/main/STYLE_GUIDE.md

Follow these steps to build and view the project's documentation locally. Ensure you have a development install of Composer and activate its virtual environment.

```bash
source path/to/composer_venv/bin/activate  # activate your composer virtual env
cd composer/docs  # cd to the docs folder inside your composer clone
make clean
make html
```

```bash
cd composer/docs
python3 -m http.server --directory _build/html/
```

--------------------------------

### Composer Trainer Example

Source: https://context7.com/mosaicml/composer/llms.txt

Example demonstrating the setup and usage of the Composer `Trainer` class for training a CNN on MNIST. This includes defining a model, dataloaders, optimizer, scheduler, algorithms, callbacks, and loggers.

```python
import torch
import torch.nn as nn
import torch.nn.functional as F
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

from composer import Trainer
from composer.models import ComposerClassifier
from composer.algorithms import LabelSmoothing, CutMix, ChannelsLast
from composer.callbacks import SpeedMonitor, LRMonitor, CheckpointSaver
from composer.loggers import WandBLogger
from composer.optim import DecoupledAdamW, CosineAnnealingWithWarmupScheduler

# Define a simple CNN
class MyCNN(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 16, 3, padding=1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding=1)
        self.fc = nn.Linear(32 * 7 * 7, num_classes)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2(x), 2))
        return self.fc(x.flatten(1))

# Dataloaders
transform = transforms.Compose([transforms.ToTensor()])
train_data = datasets.MNIST("data", train=True, download=True, transform=transform)
eval_data = datasets.MNIST("data", train=False, download=True, transform=transform)
train_loader = DataLoader(train_data, batch_size=256, num_workers=4)
eval_loader = DataLoader(eval_data, batch_size=256, num_workers=4)

# Model, optimizer, scheduler
model = ComposerClassifier(module=MyCNN(), num_classes=10)
optimizer = DecoupledAdamW(model.parameters(), lr=1e-3, weight_decay=1e-4)
scheduler = CosineAnnealingWithWarmupScheduler(t_warmup="5ba")

trainer = Trainer(
    model=model,
    train_dataloader=train_loader,
    eval_dataloader=eval_loader,
    optimizers=optimizer,
    schedulers=scheduler,
    max_duration="10ep",
    algorithms=[
        LabelSmoothing(smoothing=0.1),
        CutMix(alpha=1.0),
        ChannelsLast(),
    ],
    callbacks=[
        SpeedMonitor(window_size=50),
        LRMonitor(),
        CheckpointSaver(folder="checkpoints", save_interval="1ep"),
    ],
    loggers=[WandBLogger(project="my-project")],
    device="gpu",
    precision="amp_fp16",
    seed=42,
)

trainer.fit()
# To resume training from checkpoint:
# trainer.fit(reset_time=False)

```

--------------------------------

### Set up Composer Engine and Training Loop in Python

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/welcome_tour.rst

Initializes the Composer State and Engine, then runs a typical training loop, triggering various events.

```python
from composer import Time

state = State(
    model=your_model,  # ComposerModel
    max_duration=Time(10, 'epoch'),
    rank_zero_seed=0,
    dataloader=your_dataloader  # torch.utils.DataLoader,
)

engine = Engine(state=state, algorithms=[MixUp()])

engine.run_event("init")
engine.run_event("fit_start")
for epoch in range(NUM_EPOCHS):
    engine.run_event("epoch_start")
    for state.inputs, state.targets in dataloader:
        engine.run_event("after_dataloader")

        engine.run_event("batch_start")

        engine.run_event("before_forward")
        state.outputs = state.model.forward(state.inputs)
        engine.run_event("after_forward")

        engine.run_event("before_loss")
        state.loss = state.model.loss(state.outputs, state.targets)
        engine.run_event("after_loss")

        engine.run_event("before_backward")
        state.loss.backward()
        engine.run_event("after_backward")

        state.optimizers.step()
        state.schedulers.step()

        engine.run_event("batch_end")
    engine.run_event("epoch_end")
```

--------------------------------

### Initialize and Run Composer Trainer for Finetuning

Source: https://github.com/mosaicml/composer/blob/main/examples/pretrain_finetune_huggingface.ipynb

Create and run the Composer Trainer for finetuning. Key parameters include loading weights from a checkpoint, specifying optimizer and scheduler, and setting the device and precision.

```python
import torch
from composer import Trainer

# Create Trainer Object
trainer = Trainer(
    model=composer_model, # This is the model from the HuggingFaceModel wrapper class.
    train_dataloader=train_dataloader,
    eval_dataloader=eval_dataloader,
    max_duration='1ep', # Again, training for more epochs is likely to lead to higher performance
    save_folder='checkpoints/finetuning/',
    load_path=f'checkpoints/pretraining/latest-rank0.pt',
    load_weights_only=True, # We're starting a new training run, so we just the model weights
    load_strict_model_weights=False, # We're going from the original model, which is for MaskedLM, to a new model, for SequenceClassification
    optimizers=optimizer,
    schedulers=[lr_scheduler],
    device='gpu' if torch.cuda.is_available() else 'cpu',
    precision='fp32',
    seed=17,
)
# Start training
trainer.fit()
trainer.close()
```

--------------------------------

### Install Dependencies for Composer and PyTorch Lightning

Source: https://github.com/mosaicml/composer/blob/main/examples/migrate_from_ptl.ipynb

Installs the necessary libraries for using Composer and PyTorch Lightning. Uncomment the second command to install from source.

```python
%pip install pytorch-lightning
```

```python
%pip install mosaicml
# To install from source instead of the last release, comment the command above and uncomment the following one.
```

--------------------------------

### Trainer Initialization with Custom Algorithm

Source: https://github.com/mosaicml/composer/blob/main/docs/source/trainer/algorithms.rst

Demonstrates how to instantiate Composer's Trainer and include custom algorithms alongside built-in ones.

```python
from composer import Trainer
from composer.algorithms.blurpool import BlurPool
from composer.algorithms.channels_last import ChannelsLast

channels_last = ChannelsLast()
blurpool = BlurPool(replace_convs=True, replace_maxpools=True, blur_first=True)
custom_algorithm = MyAlgorithm(hparam1=1)

trainer = Trainer(model=model,
                train_dataloader=train_dataloader,
                eval_dataloader=test_dataloader,
                max_duration='90ep',
                device='gpu',
```

--------------------------------

### Instantiate Composer Trainer with BlurPool Algorithm

Source: https://github.com/mosaicml/composer/blob/main/examples/migrate_from_ptl.ipynb

Instantiate the Composer Trainer with model, optimizer, scheduler, dataloaders, and the BlurPool algorithm. Ensure correct device selection and training parameters.

```python
from composer import Trainer
from composer.algorithms import BlurPool
from composer.optim import DecoupledSGDW

model = MosaicResnet()
optimizer = DecoupledSGDW(
    model.parameters(),
    lr=0.05,
    momentum=0.9,
    weight_decay=5e-4,
)

steps_per_epoch = 45000 // 256

scheduler = OneCycleLR(
    optimizer,
    0.1,
    epochs=30,
    steps_per_epoch=steps_per_epoch,  
)

trainer = Trainer(
    model=model,
    algorithms=[
        BlurPool(
            replace_convs=True,
            replace_maxpools=True,
            blur_first=True
        ),
    ],
    train_dataloader=train_dataloader,
    device="gpu" if torch.cuda.is_available() else "cpu",
    eval_dataloader=test_dataloader,
    optimizers=optimizer,
    schedulers=scheduler,
    step_schedulers_every_batch=True,  # interval should be step                  
    max_duration='2ep',
    eval_interval=1,
    train_subset_num_batches=1,
)
trainer.fit()
```

--------------------------------

### Install Composer with COCO Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for COCO dataset.

```bash
pip install 'mosaicml[coco]'
```

--------------------------------

### Initialize Composer Trainer with Algorithms

Source: https://github.com/mosaicml/composer/blob/main/docs/source/trainer/algorithms.rst

Pass a list of algorithm classes to the Composer Trainer to automatically apply them during training. This example initializes the trainer with ChannelsLast and BlurPool algorithms.

```python
from composer import Trainer
from composer.algorithms import BlurPool, ChannelsLast

trainer = Trainer(
    model=model,
    algorithms=[ChannelsLast(), BlurPool()]
    train_dataloader=train_dataloader,
    eval_dataloader=test_dataloader,
    max_duration='10ep',
)
```

--------------------------------

### Install Composer with Comet ML Support

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for Comet ML.

```bash
pip install 'mosaicml[comet_ml]'
```

--------------------------------

### Install Composer with NLP Dependencies

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Install Composer with support for NLP models and algorithms.

```bash
pip install 'mosaicml[nlp]'
```

--------------------------------

### Running Doctests Locally

Source: https://github.com/mosaicml/composer/blob/main/STYLE_GUIDE.md

Commands to activate the virtual environment, navigate to the docs directory, clean previous builds, build HTML documentation, and run doctests.

```bash
source path/to/composer_venv/bin/activate  # activate your composer virtual env
cd composer/docs  # cd to the docs folder inside your composer clone
make clean
make html  # the html build must be completed first to ensure all doctests are identified
make doctest 2>/dev/null # For more verbosity, do not direct stderr to /dev/null
```

--------------------------------

### Import Libraries and Set Up Data Directory

Source: https://github.com/mosaicml/composer/blob/main/examples/auto_microbatching.ipynb

Imports necessary libraries and sets up the data directory and normalization constants for image processing.

```python
import torch

import composer
from torchvision import datasets, transforms


torch.manual_seed(42) # For replicability

data_directory = "./data"

# Normalization constants
mean = (0.507, 0.487, 0.441)
std = (0.267, 0.256, 0.276)

```

--------------------------------

### Configure Trainer with Algorithms

Source: https://github.com/mosaicml/composer/blob/main/examples/getting_started.ipynb

Sets up the Composer trainer, including the model, dataloaders, optimizer, learning rate scheduler, device, and loggers. Crucially, it demonstrates how to pass a list of algorithms to the trainer for enhanced training.

```python
logger_for_algorithm_run = InMemoryLogger()

optimizer = composer.optim.DecoupledSGDW(
    model.parameters(),
    lr=0.05,
    momentum=0.9,
    weight_decay=2.0e-3
)

trainer = composer.trainer.Trainer(
    model=model,
    train_dataloader=train_dataloader,
    eval_dataloader=test_dataloader,
    max_duration=train_epochs,
    optimizers=optimizer,
    schedulers=lr_scheduler,
    device=device,
    loggers=logger_for_algorithm_run,
    algorithms=algorithms # Adding algorithms this time!
)
```

--------------------------------

### Install Composer with Anaconda

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Use this command to install the core Composer library via Anaconda.

```bash
conda install -c mosaicml mosaicml
```

--------------------------------

### Python Function Example

Source: https://github.com/mosaicml/composer/blob/main/docs/source/templates/algo_card.rst

A basic Python function definition. This snippet serves as a placeholder for actual code examples.

```python
def some_function():
        msg = "Hello World"
        print(msg)
```

--------------------------------

### Run Profiler Demo Script (Multiple GPUs)

Source: https://github.com/mosaicml/composer/blob/main/docs/source/trainer/performance_tutorials/profiling.md

Execute the profiler demonstration script across multiple GPUs. Replace N_GPUS with the desired number of GPUs.

```bash
composer -n N_GPUS examples/profiler_demo.py  # set N_GPUS to the number of GPUs
```

--------------------------------

### Install Composer

Source: https://context7.com/mosaicml/composer/llms.txt

Install Composer via pip. Optional extras can be included for specific functionalities like NLP or logging.

```bash
pip install mosaicml
```

```bash
pip install "mosaicml[nlp,wandb,mlflow]"
```

--------------------------------

### Install Composer with pip

Source: https://github.com/mosaicml/composer/blob/main/README.md

Use this command to install the Composer library using pip. Ensure you have a compatible Python environment.

```bash
pip install mosaicml
```

--------------------------------

### Import and Instantiate BlurPool Algorithm

Source: https://github.com/mosaicml/composer/blob/main/examples/custom_speedup_methods.ipynb

Import the BlurPool algorithm from the Composer library and instantiate it with specific parameters. This prepares BlurPool to be composed with other algorithms.

```python
from composer.algorithms.blurpool import BlurPool

blurpool = BlurPool(
    replace_convs=True,
    replace_maxpools=True,
    blur_first=True
)
```

--------------------------------

### Example ComposerClassifier with Metrics

Source: https://github.com/mosaicml/composer/blob/main/docs/source/composer_model.rst

An example implementation of `ComposerClassifier` that includes metrics for accuracy. It defines `eval_forward`, `update_metric`, and `get_metrics` to integrate torchmetrics.

```python
import torchvision
import torchmetrics
from composer.models import ComposerModel

class ComposerClassifier(ComposerModel):

    def __init__(self):
        super().__init__()
        self.model = torchvision.models.resnet18()
        self.train_accuracy = torchmetrics.classification.MulticlassAccuracy(num_classes=1000, average='micro')
        self.val_accuracy = torchmetrics.classification.MulticlassAccuracy(num_classes=1000, average='micro')

    def eval_forward(self, batch, outputs):
        if outputs:
            return outputs
        inputs, _ = batch
        outputs = self.model(inputs)
        return outputs

    def update_metric(self, batch, outputs, metric):
        _, targets = batch
        metric.update(outputs, targets)

    def get_metrics(self, is_train=False):
        # defines which metrics to use in each phase of training
```

--------------------------------

### Install Composer with NLP Dependencies

Source: https://github.com/mosaicml/composer/blob/main/examples/pretrain_finetune_huggingface.ipynb

Install Composer with the necessary NLP dependencies to enable Hugging Face integration. Run this command in your environment.

```python
%pip install 'mosaicml[nlp]'
```

--------------------------------

### Install Pillow-SIMD

Source: https://github.com/mosaicml/composer/blob/main/docs/source/getting_started/installation.rst

Uninstall vanilla Pillow and install Pillow-SIMD for faster image data loading. Note: Pillow-SIMD is not supported on Apple M1 Macs.

```bash
pip uninstall pillow && pip install pillow-simd
```

--------------------------------

### Create and Set Permissions for Dataset Directory

Source: https://github.com/mosaicml/composer/blob/main/docs/source/tutorials/train_resnet50_on_aws.md

Create a directory for your datasets and set appropriate permissions. This ensures the training process can access and write to the dataset location.

```bash
sudo mkdir -p /datasets/ImageNet
sudo chmod -R 777 /datasets
```

--------------------------------

### Conditionally Import Optional Dependencies

Source: https://github.com/mosaicml/composer/blob/main/STYLE_GUIDE.md

Use a try-except block with MissingConditionalImportError to handle optional dependencies. This prevents ImportErrors if the dependency is not installed, allowing for a minimal Composer install.

```python
from composer import Callback
from composer.utils import MissingConditionalImportError

class SystemMetricsMonitor(Callback)
    try:
        import pynvml
    except ImportError as e:
        raise MissingConditionalImportError(extra_deps_group="system_metrics_monitor",
                                            conda_package="pynvml",
                                            conda_channel="conda-forge",) from e
```

--------------------------------

### Set Up Distributed Environment Variables

Source: https://github.com/mosaicml/composer/blob/main/examples/training_with_submitit.ipynb

Configure essential environment variables for distributed training using Submitit and SLURM. This includes RANK, WORLD_SIZE, LOCAL_RANK, and MASTER_ADDR.

```python
import os
import submitit
import subprocess
```

```python
def set_up_dist_env():
    # 1. RANK
    job_env = submitit.JobEnvironment()
    global_rank = job_env.global_rank
    
    # 2. LOCAL_RANK
    local_rank = job_env.local_rank
    
    # 3. LOCAL_WORLD_SIZE
    ngpus_per_node = torch.cuda.device_count()
    
    # 4. WORLD_SIZE
    world_size = int(os.getenv("SLURM_NNODES")) * ngpus_per_node
    
    # 5. NODE_RANK
    node_rank = int(os.getenv("SLURM_NODEID"))
    
    # 6. MASTER_ADDR
    cmd = "scontrol show hostnames " + os.getenv("SLURM_JOB_NODELIST")
    stdout = subprocess.check_output(cmd.split())
    host_name = stdout.decode().splitlines()[0]
    
    # 7. MASTER_PORT
    port = 54321
    
    # Set All the Necessary Environment Variables!
    os.environ["RANK"] = str(global_rank)
    os.environ["LOCAL_RANK"] = str(local_rank)
    os.environ["LOCAL_WORLD_SIZE"] = str(ngpus_per_node)
    os.environ["WORLD_SIZE"] = str(world_size)
    os.environ["NODE_RANK"] = str(node_rank)
    os.environ["MASTER_ADDR"] = host_name
    os.environ["MASTER_PORT"] = str(port)
    os.environ["PYTHONUNBUFFERED"] = "1"
```

--------------------------------

### Algorithm Logging and Hparams Test Example

Source: https://github.com/mosaicml/composer/blob/main/tests/algorithms/CONTRIBUTING.md

Demonstrates how to test algorithm logging and hyperparameters initialization. This example uses a mock logger to assert that hyperparameters are logged correctly.

```python
""" Results from logging and hparams initialization should also be tested.
"""


def test_myalgo_logging(state):
    """Test that the logging is as expected.

    Example:

        logger_mock = Mock()
        algorithm = AlgorithmThatLogsSomething()
        algorithm.apply(Event.INIT, state, logger=logger_mock)

        logger_mock.log_hyperparameters.assert_called_one_with({
            'some_key': some_value
        })
    """
```

--------------------------------

### Selective Backprop Functional Interface Example

Source: https://github.com/mosaicml/composer/blob/main/composer/algorithms/selective_backprop/README.md

This example shows how to integrate Selective Backprop within a custom training loop using the functional interface. Ensure necessary imports are present.

```python
# import composer.functional as cf

# def training_loop(model, train_loader):
#     opt = torch.optim.Adam(model.parameters())
#     loss_fn = F.cross_entropy
#     model.train()

#     for epoch in range(num_epochs):
#         for X, y in train_loader:
#             y_hat = model(X)
#             loss = loss_fn(y_hat, smoothed_targets)
#             loss.backward()
#             opt.step()
#             opt.zero_grad()
```