marlin realsigridjin · COMPLETED
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
github.com/realsigridjin/marlin · ★ 0 · Forks 0 · Size 776 KB
SUMMARY
Technologies 5
Scored 5
Observed 5
Practices 6
Evidence 5
Skips 0
COVERAGE
Analyzed 18 files · 13 commits · 0 API calls
TECHNOLOGIES & DEPTH
Markdown LANGUAGE Depth 70
1 files · PRODUCTION
C++ LANGUAGE Depth 70
1 files · PRODUCTION
Python LANGUAGE Depth 70
9 files · PRODUCTION
NumPy LIBRARY Depth 78
6 files · PRODUCTION
TypeScript LANGUAGE Depth 58
1 files · PRODUCTION
PRACTICES
documentation · observedautomated_tests · observedcontinuous_integration · absentcontainerization · absentlinting · absentformatting · absent
ACTIVITY & OWNERSHIP
First commit 2024-01-17
Last commit 2024-04-04
Active months 3
Commits 13