When raiding an island site, I know that I need to remove dice for my enmity tokens or stickers on the island/garrison. My question is do I remove dice for my enmity on other island sites at this island?
Yes. All of your enmity on the island counts against you. The rulebook is really clear:
Remove dice for: ... Each of your enmity tokens and stickers the island has.