09-23-2026, 04:19 AM
Quote:Maybe, but I had the opposite result. I get why that seems right, but on 7900 XTX the bottleneck felt elsewhere.
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
|
KV cache compression worth it
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
KV cache compression worth it - by cachelab - 09-22-2026, 12:02 AM
Re: KV cache compression worth it - by mangovault67 - 09-22-2026, 01:02 AM
Re: KV cache compression worth it - by circuitwire58 - 09-22-2026, 01:43 AM
Re: KV cache compression worth it - by turboscope5 - 09-22-2026, 02:16 AM
Re: KV cache compression worth it - by fastquill74 - 09-22-2026, 02:50 AM
Re: KV cache compression worth it - by quantshed - 09-22-2026, 03:44 AM
Re: KV cache compression worth it - by kernelmike - 09-27-2026, 01:55 PM
Re: KV cache compression worth it - by circuitwire58 - 09-23-2026, 04:19 AM
Re: KV cache compression worth it - by audiorunner74 - 09-23-2026, 05:36 AM
Re: KV cache compression worth it - by edgequill - 09-23-2026, 01:46 PM
Re: KV cache compression worth it - by llm_wren - 09-24-2026, 01:52 PM
Re: KV cache compression worth it - by vramstream - 09-28-2026, 02:35 PM
|
Users browsing this thread: 1 Guest(s)