### Full Migration Example

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/create-migrations.md

A comprehensive example of a migration script, including metadata, a setup callback to move an attribute, and the main migration function. This example addresses changes related to PR 433.

```python
from anemoi.models.migrations import CkptType
from anemoi.models.migrations import MigrationContext
from anemoi.models.migrations import MigrationMetadata

metadata = MigrationMetadata(
    versions={
        "migration": "1.0.0",
        "anemoi-models": "0.9.0",
    }
)


def migrate_setup(context: MigrationContext) -> None:
    """
    Migrate setup callback to be run before loading the checkpoint.

    Parameters
    ----------
    context : MigrationContext
       A MigrationContext instance
    """
    context.move_attribute(
        "anemoi.training.schemas.data.NormalizerSchema", "anemoi.models.schemas.data_processor.NormalizerSchema"
    )


def migrate(ckpt: CkptType) -> CkptType:
    """
    Migrate the checkpoint.


    Parameters
    ----------
    ckpt : CkptType
        The checkpoint dict.

    Returns
    -------
    CkptType
        The migrated checkpoint dict.
    """
    return ckpt

```

--------------------------------

### Install Anemoi Training Package

Source: https://github.com/ecmwf/anemoi-core/blob/main/training/docs/introduction/installing.md

Install the anemoi-training package from PyPI using pip. This is the standard installation command.

```bash
python -m pip install anemoi-training
```

--------------------------------

### Example Debug Configuration

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/configuring.md

This is an example of a `debug.yaml` configuration file. Ensure correct indentation for all fields.

```yaml
defaults:
- data: zarr
- dataloader: native_grid
- diagnostics: evaluation
- system: example
- graph: multi_scale
- model: transformer # Change from default group
- task: forecaster
- training: single
- _self_

diagnostics:
log:
mlflow:
   enabled: True
   offline: True
   experiment_name: 'test'
   project_name: 'AIFS'
   run_name: 'test_anemoi_core'
   tracking_uri: 'https://mlflow-server.int'
   authentication: True
   terminal: True
```

--------------------------------

### Simple Checkpoint Pipeline Example

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/checkpoint_pipeline_configuration.md

An example of a complete checkpoint pipeline configuration including a custom source and loader.

```yaml
training:
  checkpoint_pipeline:
    stages:
      - _target_: my_module.MySource
        path: /pretrained/model.ckpt

      - _target_: my_module.MyLoader
        strict: false

    async_execution: true
```

--------------------------------

### Install Development Version from GitHub

Source: https://github.com/ecmwf/anemoi-core/blob/main/training/docs/introduction/installing.md

Install the latest development version of anemoi-training directly from its GitHub repository. This command installs the 'training' subdirectory.

```bash
python -m pip install git+https://github.com/ecmwf/anemoi-core.git#subdirectory=training
```

--------------------------------

### Create Migration Script with Setup

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/create-migrations.md

Use the `--with-setup` argument when creating a migration script to include setup callbacks for handling import errors.

```bash
anemoi-models migration create migration-name --with-setup
```

--------------------------------

### Install Profiling Dependencies

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/benchmarking.md

Install the necessary optional dependencies for the profiler. This command should be run before using the profiling features for the first time.

```bash
pip install -e anemoi-training[profile]
```

--------------------------------

### Example Full Configuration File

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/configuring.md

A comprehensive example configuration file demonstrating overrides for model, learning rate, GPU count, data paths, and graph filenames.

```yaml
defaults:
- data: zarr
- dataloader: native_grid
- diagnostics: evaluation
- system: example
- graph: multi_scale
- model: transformer # Change from default group
- task: forecaster
- training: single
- _self_

config_validation: True
data:
   resolution: n320

system:
   hardware:
      num_gpus_per_node: 1
   output:
      root: /home/username/anemoi/training/output
   input:
      dataset: datset-n320-2019-2021-6h.zarr
      graph: first_graph_n320.pt

training:
   lr:
      rate: 1e-3
```

--------------------------------

### Test Anemoi Training Installation

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/basic-set-up.md

Run this command to test your Anemoi Training installation. It uses default configurations and will intentionally fail if data paths are not specified, indicating the installation is functional.

```bash
anemoi-training train
```

--------------------------------

### WebGL Uniforms Setup

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Sets up WebGL uniforms, providing a convenient way to get and set their values. Handles various uniform types including vectors and matrices.

```javascript
function c(t,e,n){if("object"==typeof n){var u=f(n);Object.defineProperty(t,e,{get:a(u),set:l(n),enumerable:!0,configurable:!1})}else s[n]?Object.defineProperty(t,e,{get:(c=n,function(t,e,r){return t.getUniform(e.program,r[c])}),set:l(n),enumerable:!0,configurable:!1}):t[e]=function(t){switch(t){case"bool":return!1;case"int":case"sampler2D":case"samplerCube":case"float":return 0;default:var e=t.indexOf("vec");if(0<=e&&e<=1&&t.length===4+e){if((r=t.charCodeAt(t.length-1)-48)<2||r>4)throw new i("","Invalid data type");return"b"===t.charAt(0)?o(r,!1):o(r,0)}if(0===t.indexOf("mat")&&4===t.length){var r;if((r=t.charCodeAt(t.length-1)-48)<2||r>4)throw new i("","Invalid uniform dimension type for matrix "+name+": "+t);return o(r*r,0)}throw new i("","Unknown uniform data type for "+name+": "+t)}}(r[n].type);var c}
```

--------------------------------

### MigrationContext Setup Utilities

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/modules/migrations.md

Utilities available within the migration setup context for manipulating module and attribute paths.

```APIDOC
## class anemoi.models.migrations.setup_context.MigrationContext

Bases: object

A context object allowing setup callbacks to access some utilities:

* `context.move_attribute("pkg.start.MyClass", "pkg.end.MyRenamedClass")` to update paths
  : to attributes.
* `context.move_module("pkg.start", "pkg.end")` to move a full module.
* `context.delete_attribute("pkg.mod.MyClass")` to remove a class you can use “*” as
  : a wildcard for the attribute name: `context.delete_attribute("pkg.mod.*")` will remove
    all attribute from the module.

#### delete_attribute(path: str) -> None

Indicate that an attribute has been deleted. Any class referencing this module will
be replace by a `MissingAttribute` object.

* **Parameters:**
  **path** (*str*) – Path to the attribute. For example `pkg.mod.MyClass`.

#### delete_module(path: str) -> None

Mark a module for deletion.

#### move_attribute(path_start: str, path_end: str) -> None

Move and rename an attribute between modules.

* **Parameters:**
  * **path_start** (*str*) – Starting module path
  * **path_end** (*str*) – End module path

#### move_module(path_start: str, path_end: str) -> None

Move a module.

* **Parameters:**
  * **path_start** (*str*) – Starting module path
  * **path_end** (*str*) – End module path
```

--------------------------------

### Install Anemoi Packages

Source: https://github.com/ecmwf/anemoi-core/blob/main/README.md

Install the anemoi-training, anemoi-models, and anemoi-graphs packages using pip. Ensure you have Python and pip installed.

```bash
pip install anemoi-training
pip install anemoi-models
pip install anemoi-graphs
```

--------------------------------

### Install Anemoi Training with Optional Dependencies

Source: https://github.com/ecmwf/anemoi-core/blob/main/training/docs/introduction/installing.md

Install anemoi-training with additional dependencies for specific functionalities. Use the '[profile]' extra for GPU profiling or '[docs]' for documentation generation.

```bash
python -m pip install "anemoi-training[profile]"
```

```bash
python -m pip install "anemoi-training[docs]"
```

--------------------------------

### Install anemoi-datasets

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/download-era5-o96.md

Install the anemoi-datasets package with version 0.5.22 or higher to use the download command.

```bash
% pip install "anemoi-datasets>=0.5.22"
```

--------------------------------

### YAML Configuration Example

Source: https://github.com/ecmwf/anemoi-core/blob/main/models/docs/user-guide/configuring.md

Example of a YAML configuration file with a potential typo ('ofline' instead of 'offline') that can lead to validation errors.

```yaml
defaults:
- data: zarr
- dataloader: native_grid
- diagnostics: evaluation
- system: example
- graph: multi_scale
- model: transformer # Change from default group
- task: forecaster
- training: single
- _self_


diagnostics:
log:
   mlflow:
      enabled: True
      ofline: True # this is a typo - should be offline
      experiment_name: 'test'
      project_name: 'AIFS'
      run_name: 'test_anemoi_core'
      tracking_uri: 'https://mlflow-server.int'
      authentication: True
      terminal: True

```

--------------------------------

### Install Anemoi Graphs with Documentation Dependencies

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/installing.md

Install anemoi-graphs along with optional dependencies required for generating documentation.

```bash
python -m pip install "anemoi-graphs[docs]"
```

--------------------------------

### Instancing Setup

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Sets up instancing using the ANGLE_instanced_arrays extension if available.

```javascript
function P(t,e){Q&&(t.instancing=e.def(t.shared.extensions,".angle_instanced_arrays"))}
```

--------------------------------

### YAML Configuration with Union Schema Example

Source: https://github.com/ecmwf/anemoi-core/blob/main/models/docs/user-guide/configuring.md

Example of a YAML configuration file demonstrating the use of union schemas for graph node attributes.

```yaml
defaults:
- data: zarr
- dataloader: native_grid
- diagnostics: evaluation
- system: example
- graph: multi_scale
- model: transformer # Change from default group
- task: forecaster
- training: single
- _self_


graphs:
   attributes:
      nodes:
          area_weight:
            _target_: anemoi.graphs.nodes.attributes.SphericalAreaWeights # options: Area, Uniform
            norm: unit-max # options: l1, l2, unit-max, unit-sum, unit-std

```

--------------------------------

### Example Debug YAML Configuration

Source: https://github.com/ecmwf/anemoi-core/blob/main/models/docs/user-guide/configuring.md

This is an example of a `debug.yaml` configuration file. Incorrect indentation, such as for `diagnostics.log`, can cause validation errors.

```yaml
defaults:
- data: zarr
- dataloader: native_grid
- diagnostics: evaluation
- system: example
- graph: multi_scale
- model: transformer # Change from default group
- task: forecaster
- training: single
- _self_


diagnostics:
log:
mlflow:
   enabled: True
   offline: True
   experiment_name: 'test'
   project_name: 'AIFS'
   run_name: 'test_anemoi_core'
   tracking_uri: 'https://mlflow-server.int'
   authentication: True
   terminal: True
```

--------------------------------

### Install Anemoi Models with Documentation Dependencies

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/introduction/installing.md

Install anemoi-models along with optional dependencies required for generating documentation. Use this if you plan to work with or build the documentation.

```bash
python -m pip install "anemoi-models[docs]"
```

--------------------------------

### Install anemoi-training Package

Source: https://github.com/ecmwf/anemoi-core/blob/main/training/README.md

Install the anemoi-training package using pip. This command is used to add the library to your Python environment.

```bash
$ pip install anemoi-training
```

--------------------------------

### Run Local Benchmarking Tests

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/benchmarking.md

Navigate to the project directory, install necessary dependencies, and run the pytest command to execute specific benchmark tests. Ensure you have the 'tests' extra installed.

```bash
cd anemoi-core
pip install -e training[tests]
pytest -s -vvv -v training/tests/integration/ --slow --multigpu -k "test_benchmark_training_cycle"
```

--------------------------------

### Successful Validation Output

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/configuring.md

This is an example of the output you will see when your configuration is valid.

```bash
2025-01-28 09:37:23 INFO Validating configs.
2025-01-28 09:37:23 INFO Prepending Anemoi Home (/home_path/.config/anemoi/training/config) to the search path.
2025-01-28 09:37:23 INFO Prepending current user directory (/repos_path/config_anemoi_core) to the search path.
2025-01-28 09:37:23 INFO Search path is now: [provider=anemoi-cwd-searchpath-plugin, path=/repos_path/config_anemoi_core, provider=anemoi-home-searchpath-plugin, path=/home_path/.config/anemoi/training/config, provider=hydra, path=pkg://hydra.conf, provider=main, path=/repos_path/anemoi-core/training/src/anemoi/training/commands]
cfg = BaseSchema(**cfg)
2025-01-28 09:37:23 INFO Config files validated.
```

--------------------------------

### Example MLflow Sync Command

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/tracking.md

An example of how to use the anemoi-training mlflow sync command to synchronize offline logs from a specified source to a destination MLflow server, with a specific run ID and experiment name.

```bash
anemoi-training mlflow sync -s /log/path -d http://server.com -r 123-run-id -e my-experiment
```

--------------------------------

### Hydra Training Configuration Example

Source: https://context7.com/ecmwf/anemoi-core/llms.txt

A comprehensive Hydra configuration file for a single-GPU transformer model using ERA5 data. This example demonstrates default group composition and user-specific overrides.

```yaml
# my_experiment.yaml  (place in your config directory)
defaults:
  - data: zarr
  - dataloader: native_grid
  - diagnostics: evaluation
  - system: example
  - graph: multi_scale
  - model: transformer       # switch from default gnn
  - task: forecaster
  - training: single
  - _self_

config_validation: true
data:
  resolution: n320

system:
  hardware:
    num_gpus_per_node: 1
  output:
    root: /scratch/username/anemoi/output
  input:
    dataset: era5-n320-2019-2021-6h.zarr
    graph: my_graph_n320.pt

training:
  lr:
    rate: 6.25e-5
    min: 3.0e-7
    iterations: 300000
    warmup_t: 1000
  rollout:
    start: 1
    epoch_increment: 1
    max: 12
  precision: bf16-mixed
  preferred_blas_backend: cublas

diagnostics:
  log:
    mlflow:
      enabled: true
      offline: false
      experiment_name: aifs-n320
      tracking_uri: https://mlflow.example.int
```

--------------------------------

### Site and Edge Manipulation

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Functions for manipulating sites and edges within the Voronoi diagram. `Ke.prototype.prepare` filters edges, `ar.prototype.start` and `ar.prototype.end` get the start and end points of an edge.

```javascript
Ke.prototype.prepare=function(){for(var t,e=this.edges,r=e.length;r--;)(t=e[r].edge).b&&t.a||e.splice(r,1);return e.sort(Je),e.length},ar.prototype={start:function(){return this.edge.l===this.site?this.edge.a:this.edge.b},end:function(){return this.edge.l===this.site?this.edge.b:this.edge.a}}
```

--------------------------------

### Get Visible Segment of Path

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Determines the visible portion of a path within a given bounding box, considering a margin. It returns the start and end points of the visible segment and its length.

```javascript
e.getVisibleSegment=function(t,e,r){var n,i,a=e.left,o=e.right,s=e.top,l=e.bottom,u=0,c=t.getTotalLength(),f=c;function h(e){var r=t.getPointAtLength(e);0===e?n=r:e===c&&(i=r);var u=r.x<a?a-r.x:r.x>o?r.x-o:0,f=r.y<s?s-r.y:r.y>l?r.y-l:0;return Math.sqrt(u*u+f*f)}for(var p=h(u);p;){if((u+=p+r)>f)return;p=h(u)}for(p=h(f);p;){if(u>(f-=p+r))return;p=h(f)}return{min:u,max:f,len:f-u,total:c,isClosed:0===u&&f===c&&Math.abs(n.x-i.x)<.1&&Math.abs(n.y-i.y)<.1}}
```

--------------------------------

### Montgomery Reduction Setup

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Sets up parameters for Montgomery reduction, including shift, r, r^2, r_inv, and m_inv. These are used for efficient modular arithmetic.

```javascript
function k(t){T.call(this,t),this.shift=this.m.bitLength(),this.shift%26!=0&&(this.shift+=26-this.shift%26),this.r=new a(1).iushln(this.shift),this.r2=this.imod(this.r.sqr()),this.rinv=this.r._invmp(this.m),this.minv=this.rinv.mul(this.r).isubn(1).div(this.m),this.minv=this.minv.umod(this.r),this.minv=this.r.sub(this.minv)}
```

--------------------------------

### Install Anemoi Models from PyPI

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/introduction/installing.md

Install the anemoi-models package using pip. This is the standard installation method.

```bash
python -m pip install anemoi-models
```

--------------------------------

### Define migrate_setup Callback

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/create-migrations.md

Define the `migrate_setup` callback function to fix import errors before loading a checkpoint. This function receives a MigrationContext instance.

```python
from anemoi.models.migrations import MigrationContext


def migrate_setup(context: MigrationContext) -> None:
    """
    Migrate setup callback to be run before loading the checkpoint.

    Parameters
    ----------
    context : MigrationContext
       A MigrationContext instance
    """

```

--------------------------------

### GUID Validation

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Checks if a given string is a valid GUID format. Returns true for valid GUIDs, false otherwise.

```javascript
function y(t){return!!t&&/^([0-9a-f]{8}-[0-9a-f]{4}-[4][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(t)}
```

--------------------------------

### Command-line Combined Overrides

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/configuring.md

Example demonstrating how to combine multiple command-line overrides, including a config file and specific settings.

```bash
anemoi-training train --config-name=debug.yaml model=transformer diagnostics.plot.enabled=False
```

--------------------------------

### Install Development Version from GitHub

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/installing.md

Install the latest development version of anemoi-graphs directly from its GitHub repository. This installs the 'graphs' subdirectory from the anemoi-core repository.

```bash
python -m pip install git+https://github.com/ecmwf/anemoi-core.git#subdirectory=graphs
```

--------------------------------

### Processor Setup

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Initializes the processor context, including creating vertex and index buffers for various rendering tasks like tile extents, debug info, and raster bounds.

```javascript
Sn=function(t,e){this.context=new Pt(t),this.transform=e,this._tileTextures={},this.setup(),this.numSublayers=Ot.maxUnderzooming+Ot.maxOverzooming+1,this.depthEpsilon=1/Math.pow(2,16),this.crossTileSymbolIndex=new Ve,this.gpuTimers={} };Sn.prototype.resize=function(e,r){if(this.width=e*t.browser.devicePixelRatio,this.height=r*t.browser.devicePixelRatio,this.context.viewport.set([0,0,this.width,this.height]),this.style)for(var n=0,i=this.style._order;n<i.length;n+=1){var a=i[n];this.style._layers[a].resize()}},Sn.prototype.setup=function(){var e=this.context,r=new t.StructArrayLayout2i4;r.emplaceBack(0,0),r.emplaceBack(t.EXTENT,0),r.emplaceBack(0,t.EXTENT),r.emplaceBack(t.EXTENT,t.EXTENT),this.tileExtentBuffer=e.createVertexBuffer(r,Xe.members),this.tileExtentSegments=t.SegmentVector.simpleSegment(0,0,4,2);var n=new t.StructArrayLayout2i4;n.emplaceBack(0,0),n.emplaceBack(t.EXTENT,0),n.emplaceBack(0,t.EXTENT),n.emplaceBack(t.EXTENT,t.EXTENT),this.debugBuffer=e.createVertexBuffer(n,Xe.members),this.debugSegments=t.SegmentVector.simpleSegment(0,0,4,5);var i=new t.StructArrayLayout4i8;i.emplaceBack(0,0,0,0),i.emplaceBack(t.EXTENT,0,t.EXTENT,0),i.emplaceBack(0,t.EXTENT,0,t.EXTENT),i.emplaceBack(t.EXTENT,t.EXTENT,t.EXTENT,t.EXTENT),this.rasterBoundsBuffer=e.createVertexBuffer(i,O.members),this.rasterBoundsSegments=t.SegmentVector.simpleSegment(0,0,4,2);var a=new t.StructArrayLayout2i4;a.emplaceBack(0,0),a.emplaceBack(1,0),a.emplaceBack(0,1),a.emplaceBack(1,1),this.viewportBuffer=e.createVertexBuffer(a,Xe.members),this.viewportSegments=t.SegmentVector.simpleSegment(0,0,4,2);var o=new t.StructArrayLayout1ui2;o.emplaceBack(0),o.emplaceBack(1),o.emplaceBack(3),o.emplaceBack(2),o.emplaceBack(0),this.tileBorderIndexBuffer=e.createIndexBuffer(o);var s=new t.StructArrayLayout3ui6;s.emplaceBack(0,1,2),s.emplaceBack(2,1,3),this.quadTriangleIndexBuffer=e.createIndexBuffer(s),this.emptyTexture=new t.Texture(e,{width:1,height:1,data:new Uint8Array([0,0,0,0])},e.gl.RGBA);var l=this.context.gl;this.stencilClearMode=new Et({func:l.ALWAYS,mask:0},0,255,l.ZERO,l.ZERO,l.ZERO)},Sn.prototype.clearStencil=function(){var e=this.con
```

--------------------------------

### Install XArray Extras for Anemoi Graphs

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/graphs/node_coordinates/xarray_file.md

Install the 'xarray' extras for anemoi-graphs to enable reading from XArray compatible sources. This command installs the necessary dependencies.

```bash
pip install anemoi-graphs[xarray]
```

--------------------------------

### Matrix Multiplication Example

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Example of matrix multiplication, likely used for transformations in graphics.

```javascript
t.exports=function(t,e,r,n){var i,a,o,s,l,u=e[0],c=e[1],f=e[2],h=e[3],p=r[0],d=r[1],v=r[2],g=r[3];return(a=u*p+c*d+f*v+h*g)<0&&(a=-a,p=-p,d=-d,v=-
```

--------------------------------

### Verify Checkpoint Source Configuration

Source: https://github.com/ecmwf/anemoi-core/blob/main/training/docs/checkpoint_troubleshooting.md

This snippet demonstrates how to check available sources and provides an example of a correct configuration for a checkpoint source.

```python
from anemoi.training.checkpoint import ComponentCatalog
print(ComponentCatalog.list_sources())
```

```yaml
checkpoint_pipeline:
  stages:
    - _target_: my_module.MySource
      path: /path/to/checkpoint.ckpt
```

--------------------------------

### Example of a Well-Composed Checkpoint Pipeline

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/checkpoint_pipeline_configuration.md

Demonstrates a typical checkpoint pipeline configuration using YAML. It includes stages for sourcing, loading, and modifying checkpoints in the correct order.

```yaml
stages:
  # 1. Source - fetch checkpoint
  - _target_: my_module.LocalSource
    path: /checkpoint.ckpt

  # 2. Loader - apply to model
  - _target_: my_module.WeightsOnlyLoader
    strict: false

  # 3. Modifier - transform model
  - _target_: my_module.FreezingModifier
    layers: [encoder]
```

--------------------------------

### Install MLflow Export/Import Package

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/tracking.md

Install the necessary package for MLflow offline logging and syncing functionality.

```bash
pip install git+https:///github.com/mlflow/mlflow-export-import/#egg=mlflow-export-import
```

--------------------------------

### Install Anemoi Graphs from PyPI

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/installing.md

Install the stable version of the anemoi-graphs package from the Python Package Index.

```bash
python -m pip install anemoi-graphs
```

--------------------------------

### Cold Start Loading Strategy Configuration (Planned)

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/checkpoint_pipeline_configuration.md

Configures the pipeline for fresh training using pretrained weights, with options to reset specific layers. This is a planned feature for Phase 2.

```yaml
stages:
  - _target_: anemoi.training.checkpoint.loaders.ColdStartLoader
    strict: false
    reset_layers: [classifier]
```

--------------------------------

### CRPS Example Configuration

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/training-methods.md

A comprehensive configuration file for CRPS training, detailing system hardware, data resolution, training epochs, distributed strategy, and loss/validation metrics.

```yaml
defaults:
  - data: zarr
  - dataloader: native_grid
  - diagnostics: evaluation
  - system: example
  - graph: encoder_decoder_only
  - model: transformer_ens
  - task: forecaster
  - training: default
  - _self_

config_validation: True

# Changes in system
system:
  input:
    truncation: ${data.resolution}-O32-linear.mat.npz
    truncation_inv: O32-${data.resolution}-linear.mat.npz
  hardware:
    num_gpus_per_ensemble: 1
    num_gpus_per_node: 1
    num_nodes: 1
    num_gpus_per_model: 1

data:
  resolution: o96

# Changes in training
training:
  training_method: anemoi.training.train.methods.EnsembleTraining
  ensemble_size_per_device: 4
  max_epochs: 20

  # Changes in strategy
  strategy:
    _target_: anemoi.training.distributed.strategy.DDPEnsGroupStrategy
    num_gpus_per_ensemble: ${system.hardware.num_gpus_per_ensemble}
    num_gpus_per_model: ${system.hardware.num_gpus_per_model}

  # Changes in training loss
  training_loss:
    datasets:
      your_dataset_name:
        _target_: anemoi.training.losses.kcrps.AlmostFairKernelCRPS
        scalars: ["variable"]
        ignore_nans: False
        alpha: 1.0

  # Changes in validation metrics
  validation_metrics:
    datasets:
      your_dataset_name:
        kcrps:
          _target_: anemoi.training.losses.kcrps.AlmostFairKernelCRPS
          scalars: []
          ignore_nans: False
          alpha: 1.0
```

--------------------------------

### CUDA Out Of Memory Error Example

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/performance-optimisation.md

An example of a CUDA out-of-memory error message indicating insufficient GPU memory.

```text
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 8.60 GiB. GPU 0 has a total capacity of 39.56 GiB of which 4.79 GiB is free.
```

--------------------------------

### ICON Mesh Configuration Example

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/graphs/node_coordinates/icon_mesh.md

This YAML configuration demonstrates how to set up ICONCellGridNodes for data nodes and ICONMultiMeshNodes for hidden nodes, along with ICONTopologicalProcessorEdges for processor configuration. Ensure the 'grid_filename' points to a valid ICON mesh file.

```yaml
nodes:
  # Data nodes
  data:
    node_builder:
      _target_: anemoi.graphs.nodes.ICONCellGridNodes
      grid_filename: "icon_grid_0026_R03B07_G.nc"
      max_level: 3
  # Hidden nodes
  hidden:
    node_builder:
      _target_: anemoi.graphs.nodes.ICONMultiMeshNodes
      grid_filename: "icon_grid_0026_R03B07_G.nc"
      max_level: 3

edges:
  # Processor configuration
 - source_name: "hidden"
   target_name: "hidden"
   edge_builders:
   - _target_: anemoi.graphs.edges.ICONTopologicalProcessorEdges
```

--------------------------------

### Move Module Example

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/create-migrations.md

Example of a migration script that uses `context.move_module` to indicate that a module has been renamed. This is useful for refactoring.

```python
from anemoi.models.migrations import CkptType
from anemoi.models.migrations import MigrationContext
from anemoi.models.migrations import MigrationMetadata

metadata = MigrationMetadata(
    versions={
        "migration": "1.0.0",
        "anemoi-models": "0.8.1",
    }
)


def migrate_setup(context: MigrationContext) -> None:
    """
    Migrate setup callback to be run before loading the checkpoint.

    Parameters
    ----------
    context : MigrationContext
       A MigrationContext instance
    """
    context.move_module("anemoi.models.schemas.data_processor", "anemoi.models.schemas.data")


def migrate(ckpt: CkptType) -> CkptType:
    """
    Migrate the checkpoint.


    Parameters
    ----------
    ckpt : CkptType
        The checkpoint dict.

    Returns
    -------
    CkptType
        The migrated checkpoint dict.
    """
    # This is also executed. You can update the checkpoint if you need to.
    return ckpt

```

--------------------------------

### Mercator Projection Setup

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Initializes a Mercator projection with a scale and clip angle. This is a common setup for web maps.

```javascript
function be(){return(0,Ut.c)(xe).scale(250).clipAngle(142)}
```

--------------------------------

### Switch LR Scheduler via Command Line

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/modules/optimization.md

Demonstrates how to select a different LR scheduler preset using a command-line argument.

```bash
anemoi-training train training/optimization/lr_scheduler=<your_preset>
```

--------------------------------

### Start Performance Mark

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Marks the start of a performance measurement interval using the browser's Performance API.

```javascript
var rf=s.performance,nf=function(t){this._marks={start:[t.url,"start"].join("#"),end:[t.url,"end"].join("#"),measure:t.url.toString()},rf.mark(this._marks.start)};
```

--------------------------------

### Core Processor Initialization and Refresh

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Sets up the core processor, including polling and refreshing mechanisms. It handles the initial setup of the rendering context and manages the dirty state of rendering elements.

```javascript
var ut={gl:t,context:p,strings:e,next:it,current:nt,draw:h,elements:a,buffer:i,shader:f,attributes:c.state,vao:c,uniforms:u,framebuffer:l,extensions:r,timer:d,isBufferArgs:F},ct={primTypes:st,compareFuncs:Ct,blendFuncs:Lt,blendEquations:$,stencilOps:Pt,glTypes:rt,orientationType:Ot};tt&&(ct.backBuffer=[1029],ct.drawBuffer=o(n.maxDrawbuffers,(function(t){return 0===t?[0]:o(t,(function(t){return 36064+t}))})));var ft=0;return{next:it,current:nt,procs:function(){var t=_(),e=t.proc("poll"),i=t.proc("refresh"),a=t.block();e(a),i(a);var s,l=(f=t.shared).gl,u=f.next,c=f.current;a(c,".dirty=false;"),E(t,e),E(t,i,null,!0),Q&&(s=t.link(Q)),r.oes_vertex_array_object&&i(t.link(r.oes_vertex_array_object),".bindVertexArrayOES(null);");var f=i.def(f.attributes),h=i.def(0),p=t.cond(h,".buffer");p.then(l,".enableVertexAttribArray(i);",l,".bindBuffer(",34962,",",h,".buffer.buffer);",l,".vert
```

--------------------------------

### Initialize Shader Program

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Creates and returns a new shader program from vertex and fragment shader sources. Requires WebGL context and shader source code.

```javascript
t.exports=function(t,e){if(1===arguments.length&&(t=(e=t).gl),!(t.getExtension("OES_standard_derivatives")||t.getExtension("MOZ_OES_standard_derivatives")||t.getExtension("WEBKIT_OES_standard_derivatives")))throw new Error("derivatives not supported");var r=function(t){var e=n(t,g.vertex,g.fragment);
return e.attrib
```

--------------------------------

### Install Remote Download Dependencies

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/checkpoint_troubleshooting.md

If remote downloads fail with an ImportError for 'aiohttp', install the necessary dependencies using pip.

```bash
pip install anemoi-training[remote]
```

--------------------------------

### Install anemoi-graphs Package

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/README.md

Install the anemoi-graphs package using pip. This command is used to add the library to your Python environment.

```bash
$ pip install anemoi-graphs
```

--------------------------------

### Use BackwardForecaster Task via Command Line

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/user-guide/tasks.md

Example command to train using the BackwardForecaster task by overriding the default task configuration.

```bash
anemoi-training train task=backward_forecaster
```

--------------------------------

### GUID Generation

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Generates a universally unique identifier (GUID) using random numbers. Suitable for unique keys or identifiers.

```javascript
function g(){return function t(e){return e?(e^16*Math.random()>>e/4).toString(16):([1e7]+-[1e3]+-4e3+-8e3+-1e11).replace(/[018]/g,t)}}()
```

--------------------------------

### Create and Activate Python Virtual Environment (Linux)

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/installing.md

Use these commands to create a new virtual environment and activate it on Linux systems before installing packages.

```bash
python -m venv /path/to/my/venv
source /path/to/my/venv/bin/activate
```

--------------------------------

### Example Remapper Configuration

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/modules/preprocessing.md

This configuration demonstrates how to set up multiple remappers in a chain, including the normalizer. It specifies transformations for 'tp' and 'tcc' variables using different remapper types and their specific arguments.

```yaml
data:
   processors:
      normalizer:
        _target_: anemoi.models.preprocessing.normalizer.InputNormalizer
        config:
          default: "mean-std"
          max: ["tp","tcc"]
      remapper1:
        _target_: anemoi.models.preprocessing.remapper.Remapper
        config:
          power: ["tp"]
          atanh: ["tcc"]
          method_kwargs:
            power:
              lambd: 0.1
              tangent_linear_above_one: true
            atanh:
              rho: 3.0
      remapper2:
        _target_: anemoi.models.preprocessing.remapper.Remapper
        config:
          affine: ["tp"]
          displace_boundary_atoms: ["tcc"]
          method_kwargs:
            affine:
              scale: 2.0
            displace_boundary_atoms:
              lower_atom: -1.0
              lower_target: -1.5
              upper_atom: 1.0
              upper_target: 1.5
              eps: 1e-4
      remapper3:
        _target_: anemoi.models.preprocessing.remapper.Remapper
        config:
          displace_boundary_atoms: ["tp"]
          method_kwargs:
            displace_boundary_atoms:
              lower_atom: 0
              lower_target: -1
              eps: 1e-7

```

--------------------------------

### Install Anemoi Models Development Version from GitHub

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/introduction/installing.md

Install the latest development version of anemoi-models directly from its GitHub repository. This is useful for testing the most recent changes.

```bash
python -m pip install git+https://github.com/ecmwf/anemoi-core.git#subdirectory=models
```

--------------------------------

### Debug Logging Setup

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/_static/processor.html

Configures and initializes the debug logging utility. It checks for debug settings in storage or environment variables.

```javascript
function i(){var t;
try{t=e.storage.debug}catch(t){}
return!t&&void 0!==n&&"env"in n&&(t=n.env.DEBUG),t
}(e=t.exports=r(89416)).log=function(){return"object"==typeof console&&console.log&&Function.prototype.apply.call(console.log,console,arguments)}
,e.formatArgs=function(t){
var r=this.useColors;
if(t[0]=(r?"%c":"")+this.namespace+(r?" %c":" ")+t[0]+(r?"%c ":" ")+"+"+e.humanize(this.diff),
r){
var n="color: "+this.color;
t.splice(1,0,n,"color: inherit");
var i=0,a=0;
t[0].replace(/%[a-zA-Z%]/g,(function(t){"%%"!==t&&(i++," %c"===t&&(a=i))
})),
t.splice(a,0,n)
}
},e.save=function(t){try{null==t?e.storage.removeItem("debug"):e.storage.debug=t}catch(t){}}
,e.load=i,
e.useColors=function(){
return!(
```

--------------------------------

### Move Attribute Example

Source: https://github.com/ecmwf/anemoi-core/blob/main/graphs/docs/create-migrations.md

Example of a migration script that uses `context.move_attribute` to indicate that a class has been moved to a different module. This handles cases where specific classes are relocated.

```python
def migrate_setup(context: MigrationContext) -> None:
    """
    Migrate setup callback to be run before loading the checkpoint.

    Parameters
    ----------
    context : MigrationContext
       A MigrationContext instance
    """
    context.move_attribute(
        "anemoi.training.schemas.data.NormalizerSchema", "anemoi.models.schemas.data_processor.NormalizerSchema"
    )

```