In two different places, the rules clearly state to remove the played cards. The rules also explicitly state "Cards are only payed once per game." The unplaced cards are recycled back into your deck. To me, it looks like your friend is incorrect. From a simulation stand point the rules make sense. If I were riding my bike and hit the wall (drew four 2's) I would slow down and catch my breath (through out the 2 after it's played). After a turn or two, I would be able to ride a little harder.
Exhaustion cards are discarded when played. Played cards are always discarded.
But to piggyback on this, for Grand Tour, only the Exhaustion cards still in the deck are used to calculate the total and remove half, correct? Exhaustion cards that a cyclist received but were played during the race stay discarded and don't count.