What is the greatest obstacle to learning to read music?

Dennis asked: No matter what instrument you play it is essential to learn to read music among other things. This poses an obstacle to learning music some people just learn by ear and others solely learn by reading music. I believe that both should happen together. What is the hardest thing you find in learning [...]