prefix caching in llama.cpp: worth it?

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

prefix caching in llama.cpp: worth it?

marbletrack Offline

Offline

Member

Posts: 61
Threads: 6
Joined: Sep 2026
Reputation: 0

#11

10-17-2026, 02:41 AM

A couple questions before I guess too hard. What model, what runtime, and roughly what context size?

Find

Reply

« Next Oldest | Next Newest »

Messages In This Thread

prefix caching in llama.cpp: worth it? - by northcraft90 - 10-14-2026, 10:53 PM

Re: prefix caching in llama.cpp: worth it? - by phasescope - 10-15-2026, 12:11 AM

Re: prefix caching in llama.cpp: worth it? - by dockridge - 10-15-2026, 12:23 PM

Re: prefix caching in llama.cpp: worth it? - by pixelcrate82 - 10-16-2026, 12:40 PM

Re: prefix caching in llama.cpp: worth it? - by latentcraft - 10-16-2026, 11:07 PM

Re: prefix caching in llama.cpp: worth it? - by latentcraft - 10-16-2026, 01:08 PM

Re: prefix caching in llama.cpp: worth it? - by mlxgarden - 10-17-2026, 12:43 AM

Re: prefix caching in llama.cpp: worth it? - by bytesupply76 - 10-16-2026, 02:49 PM

Re: prefix caching in llama.cpp: worth it? - by northcraft90 - 10-16-2026, 11:47 PM

Re: prefix caching in llama.cpp: worth it? - by embershed99 - 10-17-2026, 02:25 AM

Re: prefix caching in llama.cpp: worth it? - by marbletrack - 10-17-2026, 02:41 AM

Re: prefix caching in llama.cpp: worth it? - by dockridge - 10-18-2026, 02:51 PM

Re: prefix caching in llama.cpp: worth it? - by rocmriley - 10-18-2026, 08:50 PM

Re: prefix caching in llama.cpp: worth it? - by embershed99 - 10-18-2026, 10:54 PM

Re: prefix caching in llama.cpp: worth it? - by vrambandit - 10-18-2026, 11:09 PM

Re: prefix caching in llama.cpp: worth it? - by visionyard23 - 10-19-2026, 01:25 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)