|
prefix caching in llama.cpp on 4090
|
10-15-2026, 06:16 PM
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tool calling. Not glamorous advice, but repeatable baselines save a lot of time. Turns out 'just one more test' was a lie.
2x3090 • llama.cpp • too much VRAM is still not enough
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
prefix caching in llama.cpp on 4090 - by bytesupply76 - 10-15-2026, 05:10 PM
Re: prefix caching in llama.cpp on 4090 - by vrambandit - 10-15-2026, 06:16 PM
Re: prefix caching in llama.cpp on 4090 - by vectorport - 10-15-2026, 06:48 PM
Re: prefix caching in llama.cpp on 4090 - by vrambandit - 10-15-2026, 07:28 PM
Re: prefix caching in llama.cpp on 4090 - by bytesupply76 - 10-16-2026, 04:12 AM
Re: prefix caching in llama.cpp on 4090 - by deltastation - 10-16-2026, 04:49 AM
Re: prefix caching in llama.cpp on 4090 - by vrambandit - 10-16-2026, 09:03 AM
|
