"When playing a game, the goal is to win, but it is the goal that is important, not the winning" - Dr. Reiner Knizia
Any time a response card could normally be played. That means pretty much any time! It would exclude doing it during the middle of resolving an action such as piracy, though. You could respond to the announcement of a piracy action and resolve the response before resolving piracy. But you couldn't, say, wait to see if he survives your piracy defense dice and then hit him with Storms or Scurvy to try and wipe him out before he gets his piracy dice.