crawlee fatwang2 · COMPLETED
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
github.com/fatwang2/crawlee · ★ 0 · Forks 0 · Size 130.8 MB
SUMMARY
Technologies 18
Scored 18
Observed 18
Practices 6
Evidence 19
Skips 0
COVERAGE
Analyzed 3145 files · 4709 commits · 0 API calls
TECHNOLOGIES & DEPTH
TypeScript LANGUAGE Depth 70
1523 files · PRODUCTION
Dockerfile LANGUAGE Depth 70
50 files · PRODUCTION
XML LANGUAGE Depth 70
1 files · PRODUCTION
JavaScript LANGUAGE Depth 70
158 files · PRODUCTION
JSON LANGUAGE Depth 70
258 files · PRODUCTION
Markdown LANGUAGE Depth 70
856 files · PRODUCTION
YAML LANGUAGE Depth 70
12 files · PRODUCTION
npm BUILD_TOOL Depth 80
1 files · CONFIGURATION
Yarn BUILD_TOOL Depth 80
1 files · CONFIGURATION
ESLint QUALITY Depth 88
23 files · CONFIGURATION, TEST
Express FRAMEWORK Depth 74
6 files · PRODUCTION, TEST
Next.js FRAMEWORK Depth 81
5 files · PRODUCTION
Playwright TESTING Depth 85
35 files · PRODUCTION, TEST
Prettier QUALITY Depth 84
6 files · CONFIGURATION
React FRAMEWORK Depth 74
25 files · PRODUCTION
Vitest TESTING Depth 45
3 files · TEST
body-parser LIBRARY Depth 58
1 files · PRODUCTION
debug LIBRARY Depth 58
1 files · PRODUCTION
PRACTICES
documentation · observedautomated_tests · observedcontinuous_integration · observedcontainerization · observedlinting · observedformatting · observed
ACTIVITY & OWNERSHIP
First commit 2016-08-26
Last commit 2024-12-04
Active months 94
Commits 4709