Perhaps the best way to think about it is that worrying is a sign that you care about how you go. It's just a matter of using that worry productively. As much as my constant worrying and doubt has annoyed me, it's better than the polar opposite, where I have no worries and am complacent. No fun.