gigatoken sugatoray · COMPLETED
Language model tokenization at GB/s
github.com/sugatoray/gigatoken · ★ 0 · Forks 0 · Size 2.9 MB
SUMMARY
Technologies 16
Scored 16
Observed 16
Practices 6
Evidence 18
Skips 0
COVERAGE
Analyzed 173 files · 368 commits · 0 API calls
TECHNOLOGIES & DEPTH
Shell LANGUAGE Depth 70
3 files · PRODUCTION
Rust LANGUAGE Depth 70
61 files · PRODUCTION
JSON LANGUAGE Depth 70
4 files · PRODUCTION
TOML LANGUAGE Depth 70
4 files · PRODUCTION
Markdown LANGUAGE Depth 70
11 files · PRODUCTION
YAML LANGUAGE Depth 70
2 files · PRODUCTION
Python LANGUAGE Depth 70
65 files · PRODUCTION
uv BUILD_TOOL Depth 80
1 files · CONFIGURATION
Cargo BUILD_TOOL Depth 80
1 files · CONFIGURATION
Poetry BUILD_TOOL Depth 80
1 files · CONFIGURATION
NumPy LIBRARY Depth 85
10 files · PRODUCTION, TEST
RabbitMQ MESSAGE_BROKER Depth 61
1 files · CONFIGURATION
Redis CACHE Depth 61
1 files · CONFIGURATION
Serde LIBRARY Depth 58
1 files · PRODUCTION
TypeScript LANGUAGE Depth 58
1 files · PRODUCTION
pytest TESTING Depth 74
21 files · PRODUCTION, TEST
PRACTICES
documentation · observedautomated_tests · observedcontinuous_integration · observedcontainerization · absentlinting · observedformatting · absent
ACTIVITY & OWNERSHIP
First commit 2025-10-05
Last commit 2026-08-06
Active months 10
Commits 368