Is it normal for Linux to use this much RAM at idle?

I got my hands on a couple of dedicated servers for a project to mostly run Postgres on. One AMD server with 64 GB of RAM and another one is Xeon with 32 GB RAM to use as a replica. Installed Debian on both of them, connected via wireguard, set basic things up etc., everything identical software so far.

Now, both servers are not running any workloads just yet. I checked htop on the AMD server out of curiosity and to my surprise it showed almost 6 GB used out of 64 (idling). The other server is showing < 400 MB, for comparison.

Does Linux use this much RAM for caching? Why the other machine does not? Can it be a hardware issue or an issue with the Linux kernel (drivers etc.). What's your take? Should I worry about it?

Here is free -h on the AMD server:

            total        used        free      shared  buff/cache   available
Mem:            62Gi       6.5Gi        55Gi       1.1Mi       834Mi        55Gi
Swap:          1.0Gi          0B       1.0Gi

And Intel, for comparison:

            total        used        free      shared  buff/cache   available
Mem:            31Gi       767Mi        29Gi       1.1Mi       734Mi        30Gi
Swap:          1.0Gi          0B       1.0Gi