As the saying goes, "Draw with your eyes, not your brain." Drawing is just a loop of asking yourself whether it looks right or not. Nobody is timing you, if you have to check a reference a million times to draw a single line and then erase it angway then so be it, because the next line will maybe only take half as much, and then half of that...
Theres this wrongfull elitism that an artist has to be able to draw perfectly from memory. They don't. Fuck it. Print out an image and trace it, that practices line commitment, line width control and scale. Then you can use the traced image to practice shading and coloring. You don't put a kid on a bicycle without training wheels first, why would you do the same thing as an artist?