Make sure you know what you're getting into with a used S4. My friend has one and he lives in constant fear of $7k turbo failure. Go to
www.audiworld.com and find out all of the issues on the bi-turbo V6, as there are several. A 2000.5 is the way to go, but as they get miles on them the maintenance costs rise exponentially. I think if you bought a used one and wanted to put everything right in terms of preventative maintenance upgrades (a hose here, a sensor there, etc.), you would have to stick $5-7k in it, including labor.
If AWD is a must, have you considered the new Volvo S60 R? I haven't driven it, but it seems like a hell of a value.
If AWD is not a must, stick with the bimmers. BTW, 2001 was the first year of the 330.
FWIW, if it was me, I would keep the M3. It looks like you're talking about a $30k purchase after selling the M3. If you keep the M3, which I will assume is worth $15k knowing nothing else about the car, you should still have $15k to buy something else, like an early A8, 740, convertible, etc. YMMV.
But, of course, I am biased towards the two (or more) car set-up.