Chunking is the whole ballgame
Why embedding a whole article is a mistake
tobi · · 1 min read
A 2000-word post embedded as a single vector averages into mush and stops matching anything specific.
What actually works
Chunk on heading boundaries, then carry the heading path into the embedded text. A chunk reading "it retries three times" still knows it lives under Webhooks.
Fusing that with full-text search covers the other blind spot: vectors are confidently wrong about exact terms.