Once someone has bid, they have to pay that bid. The special case only applies the if everyone passes the first time around.
Just to clarify, once at least one player has made a bid, someone has to pay a bid. The only person who pays is the last bidder left. All the players who pass take back whatever they bid plus get the mice on the next card (if any).