But that would mean one character has 25% as many actions as four. I haven't played yet, so maybe I'm missing something obvious... but what am I missing?
Are you missing that Raider Cards are drawn and resolved at the end of each player's (i.e. character's) turn? So raider activity scales with number of players/characters. I played today for the first time and have only just realised I did it wrong by only drawing Raider cards after all players had completed their actions.