Question 330
A new JSON event contains an unexpected nested field, and an ingestion pipeline silently discards it. Which observation BEST proves that the corrective action succeeded?
Explanation
Correct Answer
A test new field is captured and promoted only after schema review. This is the most directly relevant verification because source fields outside the expected schema need explicit evolution or rescue handling. A concrete acceptance check is: A test new field is captured and promoted only after schema review. To reduce recurrence: Alert on rescued-column counts and unexpected schema changes.
Incorrect Answers
A successful run that silently omits the unfamiliar nested field repeats the original data-loss problem. Auto Loader can evolve a schema or preserve unexpected data in a rescued column; acceptance requires the field to be captured under reviewed policy and demonstrably present in the output or controlled quarantine.
Source-format readability does not imply every sink supports identical CDC writes. Here the important technical distinction is that source fields outside the expected schema need explicit evolution or rescue handling.
Format-specific decoders cannot safely infer a single stable structure from mixed files. The reported failure instead depends on this fact: source fields outside the expected schema need explicit evolution or rescue handling.