+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: best GGUF for 24GB VRAM (/showthread.php?tid=279)
best GGUF for 24GB VRAM - embershed99 - 09-14-2026
This keeps coming up in conversation so I figured I'd start a thread.
One thing I keep noticing is that people underestimate small local stacks until they actually try to run this stuff for more than one afternoon.
Curious whether that lines up with what other people are seeing.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: best GGUF for 24GB VRAM - phasescope - 09-15-2026
My first guess would be this: If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: best GGUF for 24GB VRAM - mangotinker73 - 09-18-2026
I'd start with the simpler stack.