Two players is the way to go. IMO, it's a two player game that can accommodate up to four, not the other way around.
Though I haven't played with more than 2, this makes sense to me. Also, the great part of this game is that it can be so quick, more so with two players I'd guess.
Fantastic two player game.