Not at the moment your assigning the worker. When you place the worker, your simply saying that at the end of the round after all the instant actions are taken, you plan to fight the troll/dragur.
Then when its time, you can select any of the dice you have (but not if they are on a boat fighting a bottom board monster) and they fight. You can pick as many as you want for the battle.
Just re-read the rules because I thought you sounded wrong. I have been playing it wrong this whole time! I thought you assigned your warriors the instant you make your action, which made sense to me since you assign your warriors on the boat the moment you chose that action.
I think a resumed turn layout would've made things more clear, identifying the ''Assigning warriors'' phase as being another one insted of a sub-phase.