I have a 2001 DRZ400E, lots of mods done to it, I love this bike. I have ridden XR4s before, good bike too, I just like the power of the DRZ much more, and I like a taller seat.
And don't forget that ELECTIRC START!!! The XR4 is not hard to start at all, but who wouldn't like electric start?
And you can get a 2000 DRZE for around 3 grand, if you look, in pretty damn good shape too.
Personally, I would go with the DRZ, just because it is a little more advanced, has about 8hp more, close in torque too.
Whatever you get though, beleive me you will be happy, so don't worry about it much.
:)