Inference › Inference Engines › llama.cpp / GGUF v
prefix caching in llama.cpp

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
prefix caching in llama.cpp
vrambandit Offline
Member
***
Posts: 168
Threads: 14
Joined: Sep 2026
Reputation: 0
#2
10-11-2026, 08:41 AM
Thanks, that helps. The pattern around prefix caching has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer. Turns out 'just one more test' was a lie.
2x3090 • llama.cpp • too much VRAM is still not enough
Find
Reply


Messages In This Thread
prefix caching in llama.cpp - by vrambandit - 10-11-2026, 06:20 AM
Re: prefix caching in llama.cpp - by vrambandit - 10-11-2026, 08:41 AM
Re: prefix caching in llama.cpp - by vrambandit - 10-11-2026, 01:48 PM
Re: prefix caching in llama.cpp - by pixelcrate82 - 10-11-2026, 08:53 AM
Re: prefix caching in llama.cpp - by phasescope - 10-11-2026, 01:09 PM
Re: prefix caching in llama.cpp - by embershed99 - 10-12-2026, 02:08 AM
Re: prefix caching in llama.cpp - by visionyard23 - 10-12-2026, 10:33 AM
Re: prefix caching in llama.cpp - by edgequill - 10-12-2026, 10:53 AM
Re: prefix caching in llama.cpp - by phasescope - 10-13-2026, 01:54 AM
Re: prefix caching in llama.cpp - by embershed99 - 10-12-2026, 11:20 PM
Re: prefix caching in llama.cpp - by northcraft90 - 10-12-2026, 11:51 PM

Forum Jump:


Users browsing this thread: 1 Guest(s)