I'm I right in thinking that once you play a ghost card you don't get it back even When a kid has been scared the ghost card is removed from the game.
Mostly correct. When a kid is scared, the Ghost gets discarded. For most purposes this is removed from the game, but some Kid abilities let you get a ghost back from your discard, so that's something to keep in mind.
But yes, most of the time when you play a ghost you won't be using that number again.