We know the rule of thumb… large quantized models outperform smaller less quantized models, but is there a level where that breaks down?

I ask because I’ve also heard quants below 4 bit are less effective, and that rule of thumb always seemed to compare 4bit large vs 8bit small.

As an example let’s take the large GLM 4.5 vs GLM 4.5 Air. You can have a much higher bitrate with GLM 4.5 Air… but… even with a 2bit quant made by unsloth, GLM 4.5 does quite well for me.

I haven’t figured out a great way to have complete confidence though so I thought I’d ask you all. What’s your rule of thumb when having to weigh a smaller model vs larger model at different quants?