Right. I am not saying it is a problem, I am just trying to make sure that I read correctly (though it is something I will be watching for when playing). Just thinking about it, if the first player spends one action marker on the "rest" action space, it would require another player to spend (at least) two markers to get it the first player marker from them? Also, the down side to being first player twice in a row is that you need to swap out character cards before the other players need to return theirs so you may be unable to select some of the better ones you would want.
I was similarly alarmed about this until I realized that the player chooses among the remaining face down tokens. It thus seems that the Rest action space is essentially a become start player space, with consolation prizes for those who fail to get the action first. I thought the same thing. Although there is still a chance (like in Agricola) that you could benefit sitting next to a person who likes to be first, I didn't see this as a problem once I understood how it actually worked.
This mechanism seems perfectly alright to me; it essentially is the same way turn order changes in Agricola.