How to get into the Imperial Palace?

Is there a way to freely walk around inside or is this a locked area still? I’ve heard of the back door key but I was t sure if that unlocked the ability to go in and out as I pleased