Xeon Gold 6138, 128GB DDR4, RTX 3090 — which LLMs can I run and how do they compare?

Hey everyone,

I have a workstation with the following specs:

∙	CPU: Intel Xeon Gold 6138 (20 cores / 40 threads)

∙	RAM: 128 GB DDR4 ECC

∙	GPU: Nvidia RTX 3090 (24 GB VRAM)

I’m getting into local LLM inference and want to know:

1.	Which models can I realistically run given 24 GB VRAM?

2.	How do popular models compare on this hardware — speed, quality, use case?

3.	Is it worth adding a Tesla P40 alongside the 3090 for extra VRAM (48 GB total)?

4.	Any recommended quantization levels (Q4, Q5, Q8) for best quality/speed balance?

Mainly interested in: coding assistance, text generation, maybe some fine-tuning.

Thanks!