Question 640
A support chatbot's accuracy report counts correct final answers but never checks whether the right source was retrieved. A release test must show that a held-out retrieval report identifies what fraction of required source passages appear in the candidate set. Which decision addresses the issue with the fewest unrelated changes?
Explanation
Correct Answer
Generation correctness can mask a retrieval weakness when the model guesses without grounding. Check that a held-out retrieval report identifies what fraction of required source passages appear in the candidate set.
Incorrect Answers
A judge cannot reliably check support of a claim without the evidence it is supposed to match.
Unspecified relevance judgments make retrieval comparisons nonreproducible. But generation correctness can mask a retrieval weakness when the model guesses without grounding.
Offline benchmark gains may not translate to real user query distributions; generation correctness can mask a retrieval weakness when the model guesses without grounding.