X-E3 Vs X-E5.

Basically I have (and LOVE) my X-E5, it's really gotten me back into photography, however, it's just that bit too big for me to pocket and take with me everywhere, and where I live I don't fancy having it on a strap everywhere I got it's just slightly too cumbersome. SO I was thinking of getting the X-E3 and using that as my daily carry, I have the 7Artisans 18mm f6.3 ll super pancake lens I was thinking of putting on it.

I know the E5 is 40mp and the E3 is 24mp, but realistically for day to day snaps, I'm thinking just lifestyle pictures for days out, evening meals etc, I'm not going to be blowing them up or dropping super tight.

I suppose what I'm getting at is this:

Am I just being daft, are they too similar to justify buying a second camera, or are they different enough I can get away with it?