The problem has to be with your disc. Spin the wheel with the calliper off (& in the absence of a dial gauge) hold a edge against the fork leg, or better yet a decent felt pen closer until it just touches (this will tell you where it is warped). Basically this problem is fairly common on dirt bikes.
The discs can also cone, when disc rotors warp the metal gets too hot & expands, either they get a buckle in them or the outside gets bigger than the inside they can ‘cone’ - the outside moves to the left or right. This will flex somewhat. Check the rotor is flat with a straight edge to be sure.
On the far less likely file: There might be a contaminant on your disc & the pads may need looking at. The calliper and brake pins should be in good order. Also check your wheel bearings & axle is straight.