Say a player assists, taking a die onto their sheet. Then the active player takes the die off their sheet in a later attempt at a task. If the player fails at the card, does the player who assisted have to be penalized (lose a health or sanity I think) even though they don't have a die on them anymore.
I can make sense of either thematically.
"If the active player is unable or unwilling to complete
the tasks on his Adventure card, any investigators who were
assisting (that is, investigators who were not the active player
but who had a die on their Investigator marker) must choose to
either lose 1 Sanity or lose 1 Stamina."
Provided the tense is intentional, it looks like even if the assist die was used to complete one Task, if the Adventure is then failed, the previous assistance is penalized. This isn't 100% clear though.
Thematically, the assisting investigator has joined in on the Adventure, not just the Task and therefore must share the hurt upon failure!