Tiktoken Go

https://github.com/pkoukk/tiktoken-go

Tiktoken Go is a Go port of OpenAI's tiktoken, a fast BPE tokenizer for use with OpenAI's language...

Tokens:11,662
Snippets:73
Trust Score:7.9
License:MIT
Update:1 month ago
Tokens:
Raw