Thanks for the feedback. I did read that in the rules but I assumed that it referred to a situation where there are no other mills on the game board at all. I think thematically it makes sense that the Governor would not be able to travel over unsettled swampland to another mill so I will continue to remove the Governor from the game in this situation.
The situation you describe should never occur.
Placing Mill Tiles requires that you place them next to developed squares.
Placing Landscape tiles requires that you place them next to your own Mill tiles.
You should never end up with a stranded, disconnected Mill tile if you follow these rules of placement.