I originally read the green Taoist's power as only applying to curse die rolls that he makes, which makes sense to me. During the Yin phase, the ghosts are acting, not the player, so green's tormentors continue to torment. Also, if green's power was usable during the Yin phase, then why couldn't one spend a power token to use it as such?
I agree with you. I played it wrong a couple of times before re-reading the rules which explicitly says that the power tokens only affect the Yang phase (which implied that the green's power cannot be invoked by the power token in the Yin phase). Obviously the rule is in there just for this case (it doesn't apply to any of the other powers). It just makes playing the Green character that much more attractive.