That must be it, i respeced slightly and it worked properly putting me to like 4.6k

No, it's just a bug with pets not counting while the game is offline.
That's the reason why you don't regen max mana, not getting max MS with SM etc. 

Ive been using a SC/SM build at 5k MS, and ive noticed that ill only be able to afk to around 4355, (my max inactive is 4.7k) 

The build seems fine in the stats page, i had silent assassin maxed out.  Which is a +750k% to afk damage. So its saying my SC has a 249e damage and my inactive dps is something like 275e damage. But i seem to only be able to afk to around a 240e damage. Stats page says my pet does 239e so im considering this the culprit.

Any input on this? or should i respec a couple times with this sale going on lol

Is silent assassin bugged?