The rules don't specify, like the rules of The Resistance: Avalon don't specify whether private communication is allowed. In both cases the omission could be because no one thought of someone doing that or because they wanted to leave it up to the players. It's certainly something you want to agree before you do it though as people will have strong opinions as whether it's fair & makes the game better.
One big problem is that the two players on either side of you could collude to determine exactly who you are.
Indeed. In a 5p game the 3rd player could tell the last player what 3 cards he handed to the 4th player. The 5th player would know for certain what the 4th player picked. Well assuming that the 3rd player gave accurate information. Of course by doing so the 2nd player would know what the 3rd player picked (assuming she could remember what 4 cards she handed him - I'm not sure I always would).
I don't see that it adds much to the game and has the potential to slow the game down. If want that kind of thing then something like The Resistance, Avalon is better for that.
The other thing is whether you are allowed to share knowledge about someone's hand gained from using the Magician or Wizard against them.