New job, already want out - stay or leave?

I just started a new job less than 3 weeks ago. It came with a pay increase, but I’m already feeling like it’s not the right fit for me. I’m a Staff Accountant and really want to get into tax, even if it means taking a pay cut.

Is it normal to realize this quickly that a new job just isn’t for you?