I could see it going either way, given that the spectral worker triggers when you place it and the ghost minion triggers when an opponent places a worker on it. Just for clarity, I'll say that the player with the spectral worker may gain reputation first.
mariottide, there you have it, from the designer himself!
I think it makes more sense this way, thematically. The bigger spectral worker causes more of rumors and reputation to be spread than your everyday ghost!