How does a server access the internet but not be on the internet?

This may not even be the best way to ask the question, I don't think I know enough to question it correctly. My husband has his own server and has told me that is has access to the internet but is not on the internet. This makes no sense to me.

Would someone be able to explain this in an extremely simple way? Tech is definitely not my realm of knowledge.