Playing with 5 actually makes it easier. You have more characters to absorb damage, more daily and utility attacks, and the monsters attack less often since there are more turns that occur before the monster gets to be activated again.
If you want to keep it more challenging, I would stick with 2 or 3 heroes.
But on the other hand, in 5 heroes games, the treasure is split up. Will take longer to level up and you wont stick tons of recharges on a single character.