On the top of page 3 of the rules:
Note: This expansion provides the components needed to play 5-player games of Five Tribes. This option is only possible by adding at least one of the expansions that adds tiles to the game.
So by this, you should use the city tiles.
Of course, nobody is forcing you to do that. You could leave them out, but I imagine the game board might be too tight.
If you leave out the city tiles from the expansion, also make sure to leave out the extra meeples, or you will have too many meeples.
Also, nobody is forcing you to play with the Whim cards. But I find them a lot of fun, and add more strategy to the game.