Llumnix

https://github.com/alibabapai/llumnix

Llumnix is a cross-instance request scheduling layer for LLM inference engines that optimizes...

Tokens:15,514
Snippets:174
Trust Score:8
License:Apache-2.0
Update:2 months ago
Tokens:
Raw