Question 510
At Valley Utilities, a customer billing center is improving billing questions during a least-privilege verification (SF-D14-02). The administrator observes that a high-confidence answer cites an incomplete single chunk when the procedure spans several sections. Any fix must ensure that billing adjustments require verified eligibility. Which observable result would best demonstrate that the correction actually worked?
Explanation
Correct Answer
This is the direct acceptance signal for the reported failure: In a controlled Billing_Account__c acceptance test, test questions return the prerequisite and exception sections needed for a correct response; the record is retained under Valley Utilities's rollout (SF-D14-02). The chosen correction was to adjust the retriever result count and chunk design to include required related context; the measurement therefore tests the relevant source, routing or authorization outcome rather than unrelated activity.
Incorrect Answers
The proposed observation “Index status and a test query confirm that ingested records are searchable; confirm this and retain a reproducible test record” would check a different original case: A Data 360 source is ingested but no searchable index exists for its content. The correction must ensure that student records never cross program authorization boundaries. Its mechanism is: Ingestion alone does not provide retriever-ready search access. In the present case, a high-confidence answer cites an incomplete single chunk when the procedure spans several sections. The correction must ensure that billing adjustments require verified eligibility; the more direct acceptance evidence is “Test questions return the prerequisite and exception sections needed for a correct response; confirm this with production-equivalent source conditions” because retrieval window and chunk boundaries together affect grounding completeness.
The proposed observation “The template resolves expected retrieved passages during preview; confirm this and compare with approved baseline behavior” would check a different original case: A team creates a search index and expects prompt templates to query it without a retriever. The correction must ensure that billing adjustments require verified eligibility. Its mechanism is: Search indexes organize content; retrievers select relevant indexed content for a use case. In the present case, a high-confidence answer cites an incomplete single chunk when the procedure spans several sections. The correction must ensure that billing adjustments require verified eligibility; the more direct acceptance evidence is “Test questions return the prerequisite and exception sections needed for a correct response; confirm this with production-equivalent source conditions” because retrieval window and chunk boundaries together affect grounding completeness.
The proposed observation “The review output identifies the retrieved source document and passage for each sampled claim; confirm this using a least-privilege test identity” would check a different original case: Agent replies about service terms are plausible but reviewers cannot find where they came from. The correction must ensure that policy terms come from the current approved revision. Its mechanism is: Source attribution enables reviewers to challenge grounded answers. In the present case, a high-confidence answer cites an incomplete single chunk when the procedure spans several sections. The correction must ensure that billing adjustments require verified eligibility; the more direct acceptance evidence is “Test questions return the prerequisite and exception sections needed for a correct response; confirm this with production-equivalent source conditions” because retrieval window and chunk boundaries together affect grounding completeness.