Question 30
A stateful streaming aggregation retains watermark-free state indefinitely as events arrive every second. The stream is managed with a persistent `writeStream` query and checkpoints. Which implementation choice BEST addresses the stated failure?
Explanation
Correct Answer
Apply an event-time watermark and bounded state window appropriate to tolerated lateness. This is the most directly relevant remediation because without event-time limits state can grow without bound for historical keys. A concrete acceptance check is: State metrics stabilize under long test streams while allowed late records remain included. To reduce recurrence: Monitor late-event discard and state-store size.
Incorrect Answers
Stateful aggregation memory demand can exceed the originally chosen executor resources. For the observed behavior, without event-time limits state can grow without bound for historical keys.
Parameterized task iteration reduces manual orchestration and controls parallelism. Here the important technical distinction is that without event-time limits state can grow without bound for historical keys.
Incremental maintenance capability depends on operator, source and dataset semantics. The reported failure instead depends on this fact: without event-time limits state can grow without bound for historical keys.