Candle vLLM

https://github.com/ericlbuehler/candle-vllm

An efficient, easy-to-use platform for inference and serving local LLMs, including an OpenAI...

Tokens:10,665
Snippets:121
Trust Score:9.2
License:MIT
Update:3 weeks ago
Tokens:
Raw