10-15-2026, 06:48 PM
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at reranking. Not glamorous advice, but repeatable baselines save a lot of time.
|
prefix caching in llama.cpp on 4090
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
prefix caching in llama.cpp on 4090 - by bytesupply76 - 10-15-2026, 05:10 PM
Re: prefix caching in llama.cpp on 4090 - by vrambandit - 10-15-2026, 06:16 PM
Re: prefix caching in llama.cpp on 4090 - by vectorport - 10-15-2026, 06:48 PM
Re: prefix caching in llama.cpp on 4090 - by vrambandit - 10-15-2026, 07:28 PM
Re: prefix caching in llama.cpp on 4090 - by bytesupply76 - 10-16-2026, 04:12 AM
Re: prefix caching in llama.cpp on 4090 - by deltastation - 10-16-2026, 04:49 AM
Re: prefix caching in llama.cpp on 4090 - by vrambandit - 10-16-2026, 09:03 AM
|
Users browsing this thread: 1 Guest(s)