Is it possible for defending units to retreat from a battle into an unoccupied enemy base capturing it in the process
No; 4.3 and 4.31 say that a hex with a vacant enemy base is enemy controlled, and 5.51 says you can't retreat to an enemy-controlled hex.
If invading infantry is forced to retreat, they return to their base of origin. If that base is in enemy hands, do they initiate another invasion, or are they eliminated?
(Where do you see the bit about returning to their base of origin?
As far as I can tell, they can retreat to any hex allowed by 5.51 EDIT: I see, 4.91.) In the event that their base of origin is now enemy controlled, and they have no other place to retreat, they're eliminated (last bit of 5.5). EDIT: and that agrees with the second paragraph of 4.91.