Captain Nemo, it's all a bit daunting till you actually experience it and find it not that bad. Trust me, I was all scared too myself when I started. 16 year old North Shore girl who, being Asian, is small sized, and who'd only been over the Harbour Bridge maybe 5 times before starting uni? I caught the train from Central every day up until the end of second year, I was that scared. Finally, bumped into an alright classmate who was walking to Redfern, and to not look like an idiot (I was near Copy Centre then) or a North Shore princess, I decided to walk with her. And that was that. Walked from Redfern ever since then. There's always security bus too if it's after dark.
I would avoid Victoria Park though. There have been gang rapes in there. Of men or women I dunno, but if what stazi says is true (and I have no reason to doubt him), anything that moves on two legs sounds like fair game to them.