Todd,
I too have both systems, using a GH1, GF1 and E-PL1 and most of the lenses (just waiting on the 100-300mm). It was this system that caused me to sell my D700 system. I also have a pair of NEX5 with the kit lenses.
I find the NEX kit lenses to be quite good (yes, even the 16mm, which, when stopped down a bit is just stellar), but I also use my Voigtländer M-mount lenses (actually on both systems). The MF lenses range from 12mm up to 90mm and all of them perform wonderfully. The wider lenses do have some magenta corners but as much of what I do is in B&W, this doesn't bother me in the least as I generally add a vignette.
Using MF on the NEX bodies is easy to do and the results are certainly worth it. My faster lenses (f2, f1.8, f1.1), together with that lovely big sensor give amazing results. There is a certain learning curve for shooting with the NEX, especially with MF lenses, but once you get comfortable, your keep-rate will go up. With the NEX on a tripod, you've got this incredible little digital back that you can then slap any number of lenses in front of.
Why do I keep both systems? Well, I bought the NEXs based on the assumption that Sony would be bringing out a EVF for them. Unfortunately, that hasn't yet happened, but the results and the fact that (as mentioned above) the LCD is actually usable virtually all the time makes them a keeper. I end up using the NEXs with the MF lenses and use the m43 system for my AF work, especially for longer focal lengths where the 2X crop has its advantages.
Cheers,