By my reading of the rules, forts count as combat units, so yes, their die roll would be modified. If not, I'd expect to see some sort of notice to that effect somewhere. But I'm no expert!
OK, it's very clear I am no expert
It didn't seem right so I checked the rules again tonight. Lo and behold, look what I found in 2.3 Counters:
Military units represented are infantry units, cavalry units (both of these unit types are collectively know (sic) as combat units), fort units, heavy siege artillery units, and headquarters.
So I guess the correct answer is that forts do not supply a defending DRM to themselves since according to 9.3.2 they "confer a -1 DRM to all defender combat unit die rolls"