I've been fortunate with my two bikes (01, 05 YZ 250) and have never fouled a plug, but I did put a second plug into my 01 because I used the stock plug for so long(it was still working fine after over 2 years of trail riding).
I didn't set the gap on the new plug when I put it in, but the bike runs just fine. Starts easy and still has plenty of power.
How important is it to set the gap to the mfg's specs? What are the downsides of incorrect gap size?
I have never worried about gapping the plugs. In fact the new Iridium ones say not too gap them on the box. I use standard tip plugs and don't gap them either. It's to do with consistant strong spark. If your bike runs good without gapping then I wouldn't worry about it. I don't think it can do any damage. (please someone correct me if I am wrong) I had an old kx years ago that wouldn't run properly unless i closed the gap some. But after not too long the stator died.
Not gapping your spark plugs won't cause the bike to blow up or anything. However, having the right gap will ensure that you have the proper spark and that you're getting the optimal performance by igniting all the fuel/air mixture inside your combustion chamber. There's no reason not to do it as it only takes about a minute and the tool will only set you back a dollar. Too big a gap will keep a weak stator from producing a spark as well.