Server desync

I'm playing on Hermitpack 1.4.0, and lately I've been having some weird connection issues. When I join my locally hosted server(not my pc, a server in my LAN) I join and spawn in a single chunk where I logged off. I can't interact with any inventories etc, mobs are frozen. When I try to rejoin my player is somehow still ingame according to the multiplayer list.

To solve this I have to restart my server every time... Anyone know what causes this and/or has a solution?

Thanks in advance.