I guess that they have updated the rules since the ones I downloaded sometime ago.
EDIT: As mentioned in the reply to your other question I have got a pdf copy of the latest rule book after all.
There does not seem to be anything against this in the rules and the example in the book shows the same villager being moved to and then from the same stone camp/quarry.
The back of the rule book says resources are generated during the upkeep phase, which helps lead to the confusion. In a 2nd edition this should be cleaned up.