Maximal Update Parametrization (μP) and Hyperparameter Transfer (μTransfer)

https://github.com/microsoft/mup

This project provides a tool that enables maximal update parametrization (μP) in PyTorch models,...

Tokens:10,314
Snippets:99
Trust Score:10
License:MIT
Update:2 days ago
Tokens:
Raw