Invisible Blocks in Multiplayer

One of my friends has thrown up a server on his computer for this game, everything works fine and my three other friends can connect and build just fine. However we are all running into an issue where blocks will randomly be unable to be constructed, display as invisible, and we can't construct/grind any other blocks. Sometimes its fixed via reconnecting, sometimes by server reboot, we can't seem to find one core cause for the issue. We are building on a planet? Could that be the issue? All the posts on this are several years old and haven't been a lot of help. Any suggestions or is this just a server side issue?