Question 470
A support assistant team changes both model and prompt together and cannot tell which change caused higher errors. A release review requires the engineer to demonstrate the expected behavior after the change. The sign-off condition is that the experiment shows separate variant scores with the same model deployment. Which configuration should the team change first?
Explanation
Correct Answer
A/B evaluation should isolate the changed prompt variable to support causal interpretation. On retest, the experiment shows separate variant scores with the same model deployment.
Incorrect Answers
Changed evidence retrieval can alter outputs independently of prompt text; a/B evaluation should isolate the changed prompt variable to support causal interpretation.
Human impression alone may not detect unsupported retrieval claims.
Prompt variant comparison is not auditable without its benchmark evidence.