The main advantage of doing advanced maths units is that they give you a better understanding of what's going on, and go through more content in greater rigour. This 1. may interest you, and 2. will make it easier for you to take future studies in maths and do maths elsewhere (e.g. engineering).
4U is officially stated as assumed knowledge, but I think if you put in enough effort, you should be fine with the first year advanced maths units. 4U provides little to no benefit in MATH1902 Linear Algebra (though 4U students will have gone through vectors, which the lecturer will go through in the first week) and MATH1905 Statistics. MATH1921 Calculus of One Variable will spend the first 1-2 weeks going through complex numbers. Integration by parts will appear in MATH1921 and/or MATH1923. The normal stream will actually go through all of those "4U" topics as well, but somewhat slower. But honestly, those topics are pretty EZ and you can easily learn them by googling.