no! your area is still the same size in a 2 or 4 player game
I disagree. The rulebook reads:
"Rules for 3-4 Players
The rules for 3 or more players are the same as the rules for 2 players with a few additions: 1. When placing Wall Cards you may place them anywhere on your team's side of the Battlefield."
So, the answer to the OP's question is yes.