I love tracking the Z4M Coupe - its been a challenge to keep pushing it just that bit more everytime. And it more rewarding and fun and the fact that its been a challenge, it has kept me engaged for almost 6 years.
I also like the uniqueness aspect of the car at tracks - call me vain
That being said, the most frustrating part about tracking it has been lack of off the shelf parts to continually and easily evolve the car as you see fit.
If you like the e46 M3, stay with it for now.