Question 550
A CI pipeline repeatedly evaluates an unchanged model but with a new judge version that makes trends incomparable. A release review requires the engineer to demonstrate the expected behavior after the change. The sign-off condition is that the comparison manifest identifies the same judge revision across trials. Which next step fixes the reported cause?
Explanation
Correct Answer
Changing the evaluator can alter scores independent of application quality. After the change, the comparison manifest identifies the same judge revision across trials.
Incorrect Answers
Evaluation data contracts can drift independently of the model. Changing the evaluator can alter scores independent of application quality.
Auditability requires recording all inputs that influenced a scoring decision; changing the evaluator can alter scores independent of application quality.
A test executed after customer exposure cannot protect the initial rollout.