Seeing Hong Kong with a baby?

We are stopping over in Hong Kong for 4 nights, travelling from Australia to Canada..

What's the best way to get around with a 20 month old baby? Will it be too hectic trying to maneuver the crowds with a stroller, or should we bring a backpack carrier, or a front carrier?

I'm 6'4"/192cm.. I remember last time we visited I had a rough time with the trams but the subway I was mostly fine. We'll be staying on the CWB side with the occasional trip to Kowloon, the night markets, and Tsim Sha Tsui.

TIA!