We've tried it both ways. It's much easier to grab from your own quarry and accidentally place on the boats. We've gone back to a central quarry to resist the absent-minded urge to by-pass the sleds. YMMV.
I can totally picture myself doing this. When I played Hansa Teutonica, I had to place my extra cubes under a cup so that I didn't keep grabbing them accidentally.