Can we extend this reasoning to swimming, flying, and climbing?
i don't (spontaneously) see why not, with the caveat that climbing is defined as having scenario-specific effects.
My own rule of thumb (lacking official rules clarifications) is that any skill categorized as a Movement Skill changes how normal Movement is applied. Normally that also implies hindrance, but flying may be a slight exception, as you point out...
It seems that flying is affected by hindering, only in english. In french it is not.
French is the native language of the game, so if French doesn't hinder flying then that's the official rule.
Not hindering flyers' movement makes thematic sense to me.