Tiktoken Go

https://github.com/tiktoken-go/tokenizer

Pure Go implementation of OpenAI's tiktoken tokenizer

Tokens:5,015
Snippets:21
Trust Score:4
License:MIT
Update:1 month ago
Tokens:
Raw