Even if you don't "feel" the contents of the bag, you can still count the pillars on the gameboard so you know how many are remaining.
I agree, however you have to be the starting player to pull this off first hand, otherwise someone could taken over the lead before you can end the game...
believe me this has happened! so in a 2p even a 3p game it's possible but with a 4-6P game this is far less likely.
This is not a game where you have someone take a big lead anyway.