When you "let off" of the gas, the front end "dives" a bit as weight is transfered forwards. This increases both the effective steering angle and the "Bite" or traction of the front tire. Any irregularity in the tire's path can cause it to steer slightly to one side. This starts a vicious cycle -- as you attempt to correct, there is windup or flex in the forks and with the increased leverage it becomes unmanageable. The usual fix is to slow down the steering by lowering the fork tubes [downwards] through the T clamps and / or lowering the rear end by increasing sag. Did the bike do this when new [my KX did]. If it's something that's just started, then the springs have probably sacked and should be re-stretched, replaced or maybe just need a bit more preload. If you're a heavier rider it's possible a stiffer spring is needed. A bit more low speed compression -either clickers or a revalve - should also help keep the front riding higher during deceleration.