Check out my blog, Board Game Radar!!!
Stop! That tickles!
I believe that any time a 3 is rolled, you may re-roll it without that roll counting toward your total number of rolls.
I just came here with the same question. We played it the same way you say.