Rules: "You must remove the corresponding building token and adjust the Scoring Track accordingly."

Is the "adjustment" the amount of VP for the building (if any) and/or the -VP on the Battle Card?

...we seem to be in disagreement on our end and looking for some clarification.

When the building is destroyed you lose ALL benefits of the building (as if you had never built it)

So VP and any special abilities