The designer reply in this thread in the Argent: Mancers of the University forum.
I am copying his answer here:
What count has Face-up Voters?
Is it only the 2 from setup (Most Influence & Most Supporter)?
Or does a Voter with a Mark from every player count as a Face-up Voter?
Only the voters that start face-up.
Also, in a 2 player game, does a Voter still require 3 Hit to be eliminated?
Yes, though feel free to reduce it to two if no assassinations are happening in your two-player games.