I use the a Lexar professional workflow USB3 CF card reader and from new on an iMac it's been a complete PITA. First card of the day takes an absolute age to be recognised and transfer speeds are a joke, however if you eject the card once it's finally recognised and push straight back in it works completely fine and very quick.
Having just purchased a Sony camera, I needed an SD reader and decided to avoid Lexar and go for the Kingston HS4 media reader after reading the reviews that suggest as quick as the Lexar with better speeds over a wider range of cards and file sizes. Will let you know What I think when it arrives in the next few days.