I have a math test on Tuesday and I feel like I'm very prepared, however, in every test I've done, there are always questions I've never seen and I am not sure what to do. How can I prevent this from happening, even after I've studied heaps in advance.
Short answer, you can't.
Math teachers are quite clever, they will almost always try to add trick questions to your exams which even the most diligent students have not seen and don't know how to do. You can attempt questions in books like Cambridge/Terry Lee, but there are infinitely many trick questions teachers can work into exams.
Your teacher will never expect you or essentially any of your class to get 100% (even if they say they do, they will not believe this, as the tests need to produce a distribution with which to allocate ranks). There will occasionally be a few students who get full marks all the time, but these are the exception not the norm.
So, if you encounter a question which you don't know how to do (when you are in an exam), you should just try them using anything you think might work. If you did it right that's great, if not that's great too because you now have a new mathematical concept to learn after the test is returned.
Much of the most important learning you do in math will be from questions you get wrong, and if you are doing math at your level you will always be getting things wrong. Above all, don't worry about getting these trick questions wrong, focus instead on learning from them when you get the papers back. Learning how to do questions you initially get wrong is much more useful for higher level math or further study.