We had 2 folks go into another room for the whispering part as it seemed like it would be too chaotic otherwise. You want to be able to repeat the clues to yourself out loud for remembering purposes. Maybe it's not meant to be played that way but I think the integrity of the game was intact.
I agree - as long as the clues are only spoken and fingers/arms are not used. Bset solution: The listener should not look at the speaker.