Zepter works completely fine with only two players. I hardly can imagine playing this with more then three or four players as downtime will grow too long. Although more players will add some fun to the auction elements in buying artefacts and sentinels.
A group can easily reduce the downtime by internalizing. The more players, the more interaction. But AP can really ruin this type of game. You need to play this with a group that is dedicated to playing it in a reasonable amount of time.