I've played it multiple times with two and never felt this was necessary. It would also slow the game down and make the promoting strategy more powerful since expending multiple resources to get only 1 point from removing a neutral troop isn't efficient. It's only worthwhile when the reward (control) pays off. With more neutral troops, this becomes more difficult.
We still want to try it. The board felt too open and easy to control some areas.