I'd allow pulling non-airborne out. Play Aid card (back) has a sea transport costs. Departing ports outside the med to arrive in England box costs 0 transport points. Those ships would have been returning to England empty. Med ports to Med box is also free.
Are you repairing armour on the cheap in England? I used it once to transfer from Brest to England then Antwerp in the Market Garden scenario.
I have played and will continue to play this way, but I guess it's now a 'house rule'.