Yes, you can. But as you said: You will figure out quite fast, that this doesn't make much sense ...
I'm reading the rules, getting ready to play, and wondering about the set up. And since no one else has asked the question, it's probably obvious. But what the hell, I'll ask anyway!
When you place your first 2 members after rolling the dice, can you choose to place them in the same town? Could you then place your last two members in that same town, resulting in all 4 of your family in the same town?
Perhaps there's no worthwhile reason to do this, but can you?