So in order to empty your population track (and deploy all your workers), you have to growth each of the 8 rounds of the game?
There's a wonder/advisor that allows taking workers though. Take a worker means take it from your population track, towards your resource area (and make it available for deployment), right?
you're right in fact, there are other means (cards) as you said to take workers from the pop track.
example : Advisor Abraham Lincoln => Action : Take 1 Worker ; powerful but he only comes in Age 4 (II II) :-)