Its fine, I spent about 3-4 hours when I did on each exercise when I first did them (will probs get high E4). Some longer, some shorter so 5hrs is ok. I'd advise not spending more than like 20 minutes on a single question unless you really want to, just to save time. Take your time to soak in the concepts and methods, the logic behind things and how to write neat and tidy working out. After you learn the content learn you should focus all your energy into speed and hard problems. Skimming through the same kind of topics in others textbooks to get a better overview of things will help. Then its off to trial papers.
I'm assuming your doing the Cambridge textbook, I did a bit more than half of it, doing all questions and not skipping over anything. I did that in the new years break doing an exercise a day (inconsistently, that's why I didn't finish). Then I kind of gave up and skimmed through the rest of it. That being said I did a lot of exercises from other textbooks throughout the year so that might make up for that difference.
But most importantly don't give up! Self-depreciation like (i am just too stupid) doesn't help your mental state and will make finding motivation harder for you imo.