The rule book says that you cannot destroy a card that has been used for its dungeon ability.
It says this under the section about using Village/Dungeon abilities. The restriction only applies to Village/Dungeon abilities used to destroy cards. (Yes, it is poor organization on the rule book's part. It took me about five plays to formulate the correct phrasing of this rule.)
Here are some things that are not Village/Dungeon abilities:
* Levelling up a Hero.
* Battle: effects.
* Aftermath: effects.
* Spoils: effects.
Here are some things that are Village/Dungeon abilities:
* Trophy: effects.
* XP effects (from Numenera or Into the Abyss).