tiktoken AnuarRazii · COMPLETED
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
github.com/AnuarRazii/tiktoken · ★ 1 · Forks 0 · Size 87 KB
SUMMARY
Technologies 10
Scored 10
Observed 10
Practices 6
Evidence 12
Skips 0
COVERAGE
Analyzed 30 files · 67 commits · 0 API calls
TECHNOLOGIES & DEPTH
Rust LANGUAGE Depth 70
2 files · PRODUCTION
Markdown LANGUAGE Depth 70
2 files · PRODUCTION
TOML LANGUAGE Depth 70
2 files · PRODUCTION
YAML LANGUAGE Depth 70
1 files · PRODUCTION
Python LANGUAGE Depth 70
18 files · PRODUCTION
Cargo BUILD_TOOL Depth 80
1 files · CONFIGURATION
Poetry BUILD_TOOL Depth 80
1 files · CONFIGURATION
NumPy LIBRARY Depth 58
1 files · PRODUCTION
Requests LIBRARY Depth 66
2 files · PRODUCTION
pytest TESTING Depth 45
3 files · TEST
PRACTICES
documentation · observedautomated_tests · observedcontinuous_integration · observedcontainerization · absentlinting · observedformatting · absent
ACTIVITY & OWNERSHIP
First commit 2022-12-12
Last commit 2025-10-06
Active months 18
Commits 67