Inference › Inference Engines › llama.cpp / GGUF v
prefix caching in llama.cpp on 4090

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
prefix caching in llama.cpp on 4090
vrambandit Offline
Member
***
Posts: 168
Threads: 14
Joined: Sep 2026
Reputation: 0
#4
10-15-2026, 07:28 PM
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
2x3090 • llama.cpp • too much VRAM is still not enough
Find
Reply


Messages In This Thread
Re: prefix caching in llama.cpp on 4090 - by vrambandit - 10-15-2026, 07:28 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)