Hello all
I've been playing around for 3 weeks now and there's a lot of good info here and elsewhere, but since I'm not working in IT and don't code, I'd be curious to know more about the setup of others in my situation
Currently my main use for my local llm is a general assistant to replace the well known cloud LLMs and to integrate with Home Assistant eventually (it is working, but I didn't play around much yet)
I recently upgraded to 32GB of (old) VRAM and currently have the following running: - OpenWebUI frontend - Gemma 4 26B-A3B (Q4_K_M) (running in llama.cpp) - Flux2 Klein 9B (running in diffusers with a simple python server that OpenWebUI can call, offloaded to CPU when not in use