Why must a task in a distributed data pipeline be designed to be 'idempotent' to ensure reliability during failure recovery?

In a distributed data pipeline, a task is idempotent if performing the same operation multiple times produces the same result as performing it once. Distributed systems often experience transient failures, such as network timeouts or hardware crashes, which occur after a task has successfully processed data but before the system can ackno....