The Jolly Roger rules seem pretty ambiguous on the point, but I think it's more fun when you can screw with each player.
What do the Jolly Roger rules say? The rules I've read make it clear that the right rule is as Steve said above, all towers on the table must be unique.
I just checked my rules, cuz I'd never had a question on this point. Here they are:
in the Jolly Roger Rules it is wrote:
The first completed Clocktower in a color must be one or two Stories High. The second completed Clocktower of that color must be exactly one Story higher than the first. The third must be exactly one Story higher than the second, and so on.I took it to mean that you look at all Clocktowers in play to determine height, since it does not specifically say just look at your own towers...