I just started building and I am noticing that I spend a ridiculous amount of time effectively re-milling bearings so that they rotate freely.

Why is this the case? With cnc mills, KF should be able to spec hardware to the right tolerance within a few thousandths of a millimeter. My hand sanding is no where near as accurate as a mill and it takes forever. Am I missing something? I would save so much time if the parts were the right size in the first place.