Don't forget you roll the dice on the previous turn, so your action of setting the dice is from the dice you already have at the start of your next turn.You won't find yourself rolling and then immediately setting dice.
Even if the manual didn't say "instead of rolling" I would rule it that way because when you roll then set, you are basically gaining the ability to set more than the improvement allows.
As an example, let's say I need 6 grey to finish a long line for a big point boost. If I roll my dice first, I can then safely ignore any dice that come up grey and just roll the other dice hoping for more grey. If I set my dice then roll, the rolled dice have potential to be anything.