10-18-2026, 11:09 PM
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
2x3090 • llama.cpp • too much VRAM is still not enough
|
prefix caching in llama.cpp: worth it?
|
|
« Next Oldest | Next Newest »
|
Users browsing this thread: 1 Guest(s)