Dolma

https://github.com/allenai/dolma

High-performance toolkit for curating large datasets for language model training with built-in...

Tokens:213,913
Snippets:413
Trust Score:8.4
License:Apache-2.0
Update:1 month ago
Tokens:
Raw