Question 150
At Westmoor Sports, a consumer membership desk is improving membership renewals during a controlled sandbox pilot (SF-D03-02). The administrator observes that a private reimbursement policy is being answered with generic public web snippets. Any fix must ensure that member records cannot be exposed to other members. Which observable result would best demonstrate that the correction actually worked?
Explanation
Correct Answer
This is the direct acceptance signal for the reported failure: In a controlled Contact acceptance test, answer traces show internal approved reimbursement sections as the grounding source; the record is retained under Westmoor Sports's rollout (SF-D03-02). The chosen correction was to use an approved internal Data Library rather than ungoverned external web results; the measurement therefore tests the relevant source, routing or authorization outcome rather than unrelated activity.
Incorrect Answers
The proposed observation “An agent retrieves the relevant current policy passage in a controlled test; confirm this using a representative sandbox fixture” would check a different original case: Support agents answer policy questions from PDFs but there is no managed knowledge grounding resource. The correction must ensure that student records never cross program authorization boundaries. Its mechanism is: Data Libraries connect managed knowledge sources to grounding tools and retrieval. In the present case, a private reimbursement policy is being answered with generic public web snippets. The correction must ensure that member records cannot be exposed to other members; the more direct acceptance evidence is “Answer traces show internal approved reimbursement sections as the grounding source; confirm this using a least-privilege test identity” because source boundaries should reflect whether knowledge is internal, external, or confidential.
The proposed observation “The retriever supplies the new handbook section with correct version provenance; confirm this while testing the unauthorized case” would check a different original case: A revised employee handbook was uploaded but an agent still cites the prior edition. The correction must ensure that employee case files are accessible only to the case owner. Its mechanism is: A library answer depends on the freshness of its ingested and indexed content. In the present case, a private reimbursement policy is being answered with generic public web snippets. The correction must ensure that member records cannot be exposed to other members; the more direct acceptance evidence is “Answer traces show internal approved reimbursement sections as the grounding source; confirm this using a least-privilege test identity” because source boundaries should reflect whether knowledge is internal, external, or confidential.
The proposed observation “The template resolves expected retrieved passages during preview; confirm this and compare with approved baseline behavior” would check a different original case: A team creates a search index and expects prompt templates to query it without a retriever. The correction must ensure that billing adjustments require verified eligibility. Its mechanism is: Search indexes organize content; retrievers select relevant indexed content for a use case. In the present case, a private reimbursement policy is being answered with generic public web snippets. The correction must ensure that member records cannot be exposed to other members; the more direct acceptance evidence is “Answer traces show internal approved reimbursement sections as the grounding source; confirm this using a least-privilege test identity” because source boundaries should reflect whether knowledge is internal, external, or confidential.