Question 280
At Redfern Aviation, a supplier operations group is improving spare-part quotations during an agent routing evaluation (SF-D10-02). The administrator observes that a Data 360 source is ingested but no searchable index exists for its content. Any fix must ensure that supplier prices are never shown outside entitled teams. Which observable result would best demonstrate that the correction actually worked?
Explanation
Correct Answer
This is the direct acceptance signal for the reported failure: In a controlled Quote acceptance test, index status and a test query confirm that ingested records are searchable; the record is retained under Redfern Aviation's rollout (SF-D10-02). The chosen correction was to create the search index over the appropriate DMO content and fields; the measurement therefore tests the relevant source, routing or authorization outcome rather than unrelated activity.
Incorrect Answers
The proposed observation “Test output reflects exact account status with the right record ID; confirm this with production-equivalent source conditions” would check a different original case: An LLM response requires current account status stored in a structured DMO rather than long documents. The correction must ensure that shipment changes remain attributable and reversible. Its mechanism is: Structured record grounding differs from unstructured chunk search. In the present case, a Data 360 source is ingested but no searchable index exists for its content. The correction must ensure that supplier prices are never shown outside entitled teams; the more direct acceptance evidence is “Index status and a test query confirm that ingested records are searchable; confirm this using a representative sandbox fixture” because ingestion alone does not provide retriever-ready search access.
The proposed observation “Paraphrased questions retrieve the relevant policy paragraph; confirm this using a representative sandbox fixture” would check a different original case: Users describe a known policy concept using varied language absent from the document title. The correction must ensure that safety-critical changes need an attributable human approval. Its mechanism is: Vector retrieval matches semantic similarity rather than requiring exact words. In the present case, a Data 360 source is ingested but no searchable index exists for its content. The correction must ensure that supplier prices are never shown outside entitled teams; the more direct acceptance evidence is “Index status and a test query confirm that ingested records are searchable; confirm this using a representative sandbox fixture” because ingestion alone does not provide retriever-ready search access.
The proposed observation “The review output identifies the retrieved source document and passage for each sampled claim; confirm this using a least-privilege test identity” would check a different original case: Agent replies about service terms are plausible but reviewers cannot find where they came from. The correction must ensure that policy terms come from the current approved revision. Its mechanism is: Source attribution enables reviewers to challenge grounded answers. In the present case, a Data 360 source is ingested but no searchable index exists for its content. The correction must ensure that supplier prices are never shown outside entitled teams; the more direct acceptance evidence is “Index status and a test query confirm that ingested records are searchable; confirm this using a representative sandbox fixture” because ingestion alone does not provide retriever-ready search access.