The rules state that
Price at islands: When paying an island (for example, to buy goods), add one gold to the price for each enmity of yours on the island.
(My emphasis.) I would argue that in the case you describe
Spoiler (click to reveal)
you aren't "paying the island" when you're building at a dock you control. That said, your colony is a specific site on the island, if I recall correctly; since you don't control the other sites, you would pay extra at other docks, if any.