the last sentence in the refill rules says "The refill phase is over, once all players have 5 workers again." But when picking the 1st option - take all workers from one action location - it's possible not to have 5 workers. So does refilling definitely continuing until all players have 5 workers?
It is definitely possible, and happens often that you take all 5 workers from the same location as your first action. So that's the end of this phase for you. If you chose a location with less then 5, you take some more the next turn until you have 5, and the others also continue until they also have 5.