I also find the 20/1.7 a bit too thin for easy manual focusing ... like other so-called "pancake" lenses, it's just not particularly ergonomic for that use.
In this focal length range, I prefer the Olympus ZD 25/2.8 on the DMW-MA1 adapter: it puts the focusing ring in just about the ideal location for manual focus while not being particularly large or heavy. It also puts the ring at about the same position as the adapted Konica Hexanon 40mm or Nikkor 20mm ... very hand-holdable and comfortable to focus with.
with Konica 40mm f/1.8 fitted and ZD 25mm f/2.8 aligned to show position
with Cosmicar 12.5mm f/1.4
with Olympus Pen 70mm f/2
Notice how with all of these the focusing ring is positioned to allow the classic, very stable grip "holding the camera resting on left hand and focusing with thumb and forefinger" that the 20/1.7 is too short to do well.
The 20/1.7 I tend to use more by using AF and the AEL/AFL button set to AF-only lock. I only lock and focus it manually on occasion.