Taj Mahal, anyone?
...Why not 5, or even 6? ... I think it would definitely be playable...
Playable, sure. Enjoyable, not so sure. But why not give it a try? Lowering the number of roos seems to be a sensible thing.
I have never played Billabong with more than 4, which in my opinion is the ideal player number. It's OK with 3, didn't like it as a 2-player game.