Tokenx

https://github.com/johannschopplich/tokenx

Fast and lightweight token count estimation for any LLM without requiring a full tokenizer,...

Tokens:29,003
Snippets:297
Trust Score:10
License:MIT
Update:1 month ago
Tokens:
Raw