r/LocalLLaMA•
nad_lab

gemma 4b runs on gpu but gemma 12b runs on my cpu?

Question | Help

im trying to get gemma 12b (which is an 8gb model) to run on my gpu ONLY as my cpu (3950x is good but not as fast as my gfx card - 5060ti). I have TWICE the amount of vram needed and I only use about 4 gb of vram with all my dev apps and os running. the model fits within my vram size and it still uses the CPU TO RUN? but gemma 3:4b (which is only 3.3gb in model size runs AMAZING on my GPU. I'm not sure what I'm doing wrong?

Posted Sep 2, 2025, 3:35 PM UTC

08
Loading comments…