Question 30
Troubleshooting report: The explanation equates one token to one full English word in all cases. Which correction is MOST directly targeted to this failure?
Explanation
Correct Answer
This correction directly addresses the reported failure. Tokenization splits text into subword or other units for model processing. The targeted change is: Demonstrate tokenization differences across words and scripts.
Incorrect Answers
A finite context window limits how much material can be processed at once. In the stated case, troubleshooting report: The explanation equates one token to one full English word in all cases; “Chunk and retrieve relevant source sections before inference” does not target that failure.
Generative models are not a substitute for auditable deterministic computation. In the stated case, troubleshooting report: The explanation equates one token to one full English word in all cases; “Implement and test fixed numeric rules in application code” does not target that failure.
Incompatible vector spaces can make nearest-neighbor rankings meaningless. In the stated case, troubleshooting report: The explanation equates one token to one full English word in all cases; “Re-embed the corpus or use a versioned compatible index” does not target that failure.