There's no mention on the rulebook that the Treasure Chests are removed from the game.
Page 11 in the white box with the picture of the treasure chest, bold mine:
"A Hero model adjacent to a treasure chest may spend 1* to open it. This lets them draw a relic card from the Treasure Deck, which they may equip immediately or wait until the end of the round. Once the relic card has been drawn, remove the treasure chest from the dungeon map."