HF Open-Model Research Crew
A CrewAI crew that scouts, benchmarks and writes up open Hub encoders.
What it is
A Python CrewAI crew that scouts the Hugging Face Hub for open embedding models, benchmarks them over the site’s own retrieval corpus, and drafts a field note. It emits ranked id lists only; the shared JS verifier re-derives recall@k, so a Python crew never becomes a second source of truth.
Take it with you
A portable spec of this system — copy or download it, code paths and all.
Where it lives
- scripts/hf-research-crew/
- src/lib/huggingface/hub.mjs
- src/lib/benchmarks/scoring.mjs
See it in action
Hugging Face field notes →FAQ
Why is the crew in Python, not Node?
It benchmarks PyTorch-only encoders (sentence-transformers) that the browser stack can’t load, and reasons over the public Hub API, the one corner the site’s Node orchestrator can’t reach cheaply.
How does it stay honest?
The crew emits ranked id lists with no score; the same JS verifier the benchmarks page uses re-derives recall@k in CI and in the browser, so the number can never disagree with the badge.