When a knowledge base answers wrong, or not at all: what to check first
· By Bee Flow · 6 min read
Microsoft put the harrier-oss-v1 embedding models on Hugging Face on 30 March. Before you swap models, work through the causes that usually come first: stale and duplicate documents, chunking, language, keyword search and reranking.

On 30 March Microsoft put harrier-oss-v1 on Hugging Face: a family of three multilingual text embedding models, with 270 million, 0.6 billion and 27 billion parameters, under the MIT licence. The model card says they "achieve state-of-the-art results on the Multilingual MTEB v2 benchmark as of the release date" (the benchmark has a public leaderboard), reports MTEB v2 scores from 66.5 for the smallest model to 74.3 for the largest, and names Dutch among the supported languages.
A release like this tends to reach an administrator as a question: the knowledge base gave a wrong answer yesterday, so should we switch to the new model? Sometimes that is the fix. More often the cause sits earlier in the chain, and swapping the model means re-embedding every document before you know whether it helped.
Below is the order we would work through, most common cause first. It assumes the usual retrieval set-up: documents are cut into chunks, each chunk gets an embedding, a question retrieves a handful of chunks, and a language model writes the answer from those.
First, make it show its sources
You cannot troubleshoot an answer when you cannot see what it was built from. Ask the system which passages it used, or set it up to list them under every answer. Then sort the failure into one of three kinds.
- The right passage came back and the model still answered wrong. That is a prompt or model problem, and retrieval is fine.
- The wrong passages came back. Work through the list below.
- Nothing came back, or nothing relevant. Also the list below, starting with whether the document is in the index at all.
That split alone saves a lot of guessing.
1. Stale and duplicate documents
A very common cause of a confidently wrong answer is a correct answer from an old document. The 2023 travel policy and the 2025 one both sit in the base, they say different things, and retrieval has no idea which one is current.
Search the base for the document's title and count the versions that come back. Remove or archive the superseded ones instead of adding the new one next to them. Byte-identical duplicates are easy to skip at ingest with a content hash. Near-duplicates, where only a date in the footer changed, are harder, and they take two of your few retrieval slots with the same text, pushing out the passage that would have answered.
For empty answers, check the dull things first. Did the upload finish? Was the file a scan without a text layer? Does the extracted text look like something you would recognise if you opened it?
2. Chunking that cuts tables and headings
The chunker decides what a passage is, and the model can only answer from passages. A fixed-size window that ignores structure produces chunks like the second half of a table without its header row, or a paragraph saying "this applies to all contracts above the threshold" while the threshold sits in the chunk before.
Open the actual chunks of the document that should have answered. A table row without its column names is just numbers to the model. A section whose heading sits in one chunk and its content in the next loses exactly the words a question is most likely to use. Splitting on headings, repeating the heading or document title inside each chunk, and keeping table rows together with their header line do more for answer quality than most model changes.
Size matters too. Tiny chunks lose their context, and huge ones bury the sentence that matters.

3. A question in Dutch, documents in English
People ask in their own language, and the manual is in English. Two things can go wrong.
The vector side only bridges languages when the embedding model was trained for it. A multilingual model places a Dutch question close to the English passage that answers it, and a model trained mostly on English may not. This is the one step where the embedding model is the likely cause, and where a release like harrier-oss-v1 is relevant: its card names Dutch among the supported languages.
The keyword side does not bridge languages at all. Postgres full text, for example, reduces words to their stems according to a text search configuration for one language, and "opzegtermijn" will never match "notice period". A mixed-language base therefore leans entirely on the vector side for questions that cross languages. Test it by asking the same question in both languages and comparing the sources.
4. Keyword search next to vector search
Vector search is good at meaning and weak at exact strings. Article numbers, error codes, a clause number, a product name that looks like an ordinary word: an embedding blurs them into "something similar", and the passage with the exact code can rank below three that are merely about the same subject.
If your system only does vector search and the failures cluster around codes and names, the fix is a keyword index next to it, with the two rankings fused. Reciprocal rank fusion is the usual method. If it already does both, check that the keyword side returns rows for those queries at all. A tokenizer that splits "AB-2041" into pieces is one reason it does not.
5. Reranking
First-stage retrieval is built for speed: it compares one question vector with many chunk vectors. A reranker, usually a cross-encoder, reads the question and each candidate passage together and scores them again. That is slower and much better at telling "mentions the topic" from "answers the question". Retrieve twenty or so candidates, rerank them, keep the top few.
Check that a reranker is configured and reachable. One that fails quietly usually falls back to the first-stage order, and nobody notices until the answers get worse. Also check how many candidates go in. If the right passage sits at place 40 and only the top 10 are reranked, the reranker never sees it.
6. Only then, the embedding model
If the sources show that the right document is in, sensibly chunked and findable by keyword, and it still does not come back for paraphrased or cross-language questions, the embedding model is a fair suspect.
A leaderboard score is an average over many tasks and datasets, and none of them are your documents. Write twenty real questions down, note the passage that should answer each, and compare models on that small set.
Some newer models expect an instruction in front of the query. The harrier-oss-v1 card is direct about it: each query needs "a one-sentence instruction that describes the task", otherwise "you will see a performance degradation", while documents need none (FAQ on the model card). A model plugged in without that prefix can look worse than the one it replaced.
Switching means re-embedding every document, because vectors from two different models cannot be compared. Plan it as a migration, with the test set run before and after.
How Bee Flow handles this
In Bee Flow every knowledge-base search runs vector and keyword retrieval side by side, fuses the two rankings with reciprocal rank fusion and reranks the top candidates with the reranker the administrator has set up, a hosted one or one you run yourself. So step 4 happens on every search without anyone having to switch it on. Answers cite numbered sources, and an agent can show them as cards under the answer, so you see which document each part came from. Content that is already in a knowledge base is skipped when the same document is uploaded again, and the administrator picks the embedding model from the providers configured on the instance. The platform page shows how knowledge bases fit together with chat and agents. You can try it without installing anything in the live demo, and Bee Flow vs Open WebUI sets the two side by side. If you want to run this checklist against your own documents, get in touch.