+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: best GGUF for 24GB VRAM: early impressions (/showthread.php?tid=237)
Pages:
1
2
Re: best GGUF for 24GB VRAM: early impressions - northcraft90 - 09-13-2026
Appreciate the detail. The pattern around router policy has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: best GGUF for 24GB VRAM: early impressions - vrambandit - 09-13-2026
Thanks, that helps. The pattern around quant drift has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: best GGUF for 24GB VRAM: early impressions - southden - 09-13-2026
Tried to reproduce it just now. On M3 Max with llama.cpp, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
Re: best GGUF for 24GB VRAM: early impressions - dockridge - 09-13-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: best GGUF for 24GB VRAM: early impressions - vrambandit - 09-15-2026
I hit almost the same issue last night.