Yes, it is risky to collapse hidden tiles.
Yes, yes it is. In my first 4 player game (with me as Cave), the Goblins accidently collapsed 3 crystal tiles using Cave-In.
I'd been putting Crystals near the edge of the board (to make them easier to collapse later) and doing the same with Treasures (to encourage players to run around a bunch), so they often aligned, and it happened that some Treasures were on unexplored Crystal tiles.
Rather than try to collapse tiles to help himself attack the Knight, the Goblin player collapsed 3 edge tiles that had Treasures on them in an attempt to weaken me. It just so happened that the 3 he picked were Crystals (he felt a bit foolish after that).