Anyone running GLM 4.5/4.6 @ Q8 locally?

Discussion

I love to know anyone running this, their system and ttft and tokens/sec.

Thinking about building a system to run it, thinking Epyc w/ one RTX 6000 Pro, but not sure what to expect for tokens/sec, thinking 10-15 is the best I can expect.

Posted Oct 2, 2025, 1:42 PM UTC

759
Loading comments…