My first game had them all being Battle actions. Took me half the game to realize I had to play that instead of a Quest action. Then the next game, I had to keep checking over and over again to remember which action was needed for which color because they were all different.
It's all starting to sink in.
There is a lot of fiddliness like that, but I think in the end it's to ensure massive replayability so that no two games will ever play out the same way.