https://np.reddit.com/r/vieb/comments/oapmct/website_with_vim_keybindings/h3mhzyx/

After more researches and tests, looks like it might "partially" have something to do with an MacOS native keymapping management tool called Karabiner-Elements. I used a system wide vim keybinding via this app and it adds another layer of keypress interpretation on top of everything... :)
Anyway, that's only part of the story though. If I strip that off, nuke the datafolder, I still couldn't get the caps work. The caps just acts like caps (switch Uu cases instead of controlling the modes)

I've tried following

imap <CapsLock> <toNormalMode>
imap <CapsLock> <Esc>
imap <CapsLock> :norm <Esc><CR>

imap jk <toNormalMode>

None of the caps mapping works for me. As a backup, I mapped jk to esc and that works...