my man is doing the 4u Cambridge textbook for the 3u vector exam.
But for Question 10 literally do what they asked you to do, essentially work how to represent each vector as a and b, and then just plug it into either RHS or LHS and just rearrange until you get the other side of the equation.
Edit: The AB^2 and stuff are actually the lengths squared so you can apply the a . a = |a|^2 thingy