When do the activated abilities of heroes have to be done (i.e. when does the defender get to use the hourglasses)?

This question came up today for my second game.

So is it accurate to say that the Defender can only use it after the six primary Phases. Can the Defender use it "in the middle" of the Assault Phase after say the Ballistas fire? Or must it be done before the Assault Phase?