I bought a new left-over '04 450F last fall after a year on a KX125 and then KTM200sx (and 30 years of bikes before that). If you have a 250F, then the 450F is just more of that. I rode several 250F and 450F before deciding on the 450F. It has much more low-end, pulls hard, and demands good throttle control, but it's not THAT brutal. Not like a TM400, at least. :) The worst thing about the 03-05 YZF's, IMHE, is getting them to turn. Several issues at work here- taller center of gravity, Yamaha geometry, and the 450 engine that's more "open-class" than the 250F. Contrary to what everyone says about sitting on the gas tank, I've found that I can turn best when I sit a little farther back and even better when I just stand all the time (like Stefan Everts! Only much slower). But, then again, I'm coming off the KX125 which was a turning machine- the 450F is about as polar opposite as you can get and still be a dirt bike.
The only thing is that I'd be a little hesitant about buying a used 4S unless you're sure of the previous owner's maintenance. On the upside, Yamaha engines have a good rep for reliability.
After riding the 450F, I can pretty much keep our TTR pinned and clank the suspension at will. The TTR is a great bike for what it is, but moving from it to the 450F is like getting off the merry-go-round and getting on a Derby contender. In fact, I'm looking at selling the TTR and getting a YZ85 for our "small" bike.