Question 610
A Python UDF imports a helper library only on the driver, then executors fail with ModuleNotFoundError. Which principle BEST explains why the implementation failed?
Explanation
Correct Answer
Executor processes need access to the same importable module as the driver. This is the most directly relevant technical explanation because executor processes need access to the same importable module as the driver. A concrete acceptance check is: A distributed transformation runs on multiple partitions without import failures. To reduce recurrence: Test UDFs on worker executors rather than only local driver calls.
Incorrect Answers
Serverless execution environments have their own library and runtime contracts. That consideration does not remove the current constraint: executor processes need access to the same importable module as the driver.
At-least-once event transport can replay the same logical change. In this situation, executor processes need access to the same importable module as the driver.
Importing a source file cannot provide confidentiality for embedded production passwords. For the observed behavior, executor processes need access to the same importable module as the driver.