Marco mentioned a trick that you can use to allow you to easily find the place in the score after you take time to look down at your left hand.
The secret is to train your brain to do this. His suggestion is to pick an item in your house that you want to remember such as a hook on your curtain rod. While watching TV, look up quickly to find that hook and then return to watching TV. Do this over an over, and eventually you should be able to find the hook easily before you return to watching TV. The more times you do it, the easier it should become.
Relating this to music playing, pick the place on the score that you need to remember, look down at your left hand when you feel it necessary, then return to the score. It should be easy to do once you learn this trick.