Whatever harder is subjective and would depend on your interests. IMO , mathematically and conceptually, electrical engineering is the hardest (e.g. fourier series, complex calculus, electromagnetism with vector calculus, signals, etc).
ME also has some though subjects, although not as...