Need some help figuring which quant sizes of some of the big MoEs would fit properly (with how much context + unquantized KV) on a future 256GB or 512GB dram + 48GB VRAM rig I might build later on, since I want to download and save them now (not later on when I have the rig)

I plan to download a few of them in full 16-bit safetensors, but I don't have enough storage space to just grab all the big models all in 16-bit and store all of them like that, so, I'll probably just get 1 or 2 of the biggest ones in 16-bit but get some of the others already quantized in GGUFs, in sizes that would fit potential future memory sizes of rigs I might build later on.

So, the main things I want to know about in particular are just exactly how much memory GLM5.2 uses when you run it, with unquantized KV cache, and with what sort of context size, for various quants at Q2 or Q3 or Q4. For example I am curious how much memory an IQ4_XS, or Q4_K_S or Q4_K_M of GLM5.2 uses, or some other main, popular quants that one could fit on a 512GB DRAM, 48GB VRAM rig on Linux.

Curious about the same questions for Kimi K2.x or DeepSeekV3.2, V4, Mimo, or any other giants you have run locally and know how much memory they use at different quant sizes.

This type of info would also be useful for models like Qwen 397b, MiniMax M3, MiniMax M2.x, etc

I haven't been able to run quants bigger than about 80-90GB of file size, so for the really big models, I want to make sure I don't guess wrong by like 10-20 GB and keep missing the 256GB + VRAM or 512GB + VRAM cutoffs by getting Q4_K_Ms in spots where I should be getting Q4_K_S or IQ4_XS or something.

Also, while we are on the topic, is there anything important I should know as far as which quants might be better or worse for long term safekeeping as far as "main" quants like Q4_K_M vs slightly more exotic quants like "IQ4_XS" or "IQ3_XXS", vs the really exotic ones, as far as running them on llama.cpp, or LMStudio, or vLLM, or SGLang, or Kobold, or so on? Should I stick to the _K_M types of quants, or will those IQ_XS/IQ_XXS types of quants tend to work fine on regular llama.cpp as well? I think for some I might need to get a forked version, but not as sure about some of the other ones like for Kobold or SGLang or whatever other important ones to know about.

Another thing I'm not sure whether to trust Gemini's info about, and couldn't find answers when I searched about it for a while, is how far you can go with %-of-total memory usage on a Linux rig with these big models on a rig with a huge amount of DRAM, and a bit of VRAM, before you hit kernel panic or OOM or instability or whatever problems, like, is it 90%, or 95%, or even beyond 100% if you can even go past how much DRAM you have to get it to fit into DRAM + VRAM but not DRAM alone? Does it matter which version of Linux you use, for it to work without any issues depending how much you want to max out the memory usage? I am curious just how big I can go with what type of setup with the 256GB DRAM + 48GB VRAm or 512GB DRAM + 48GB VRAM setups, with Linux. I'm a mac user (and formerly Windows before I got into local AI) so I don't know much about this yet.

I am nervous about trying to calculate or estimate how much memory the quants of various models would use, because for example I know with something like Gemma it uses memory significantly differently than some other models of the same size, so, if you tried to use the normal calculation it would come out totally wrong.

So, since I'm not sure how many different architectural quirks or settings quirks different models and quants can have, I am curious from real world use if people can post how much memory these actually use and what they would fit on without issues vs fit but with some issues vs just barely not fit at all, and with how much context size.