The slot with an arrow orders you to send the dice in question immediately to the board.
So it never placed on the slot.
The slot is only for consistency icon throughout the iconography.
This really matters if you are comparing grey dice with your opponents.
If the grey die is still on the slot, it's counted towards your supply of grey die.
So the grey die you just used to get a tribe and is now on the bank wouldn't be taken back as the gain one grey die effect, because the only places to take a die are from the initial pool of dice or another player once pool is empty right? So that die is out of play until someone takes it with rest action?