09-12-2026, 02:21 AM
What worked for me was pretty simple. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
still benchmarking everything
|
KV cache compression worth it: early impressions
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
KV cache compression worth it: early impressions - by sparrowhall - 09-12-2026, 01:43 AM
Re: KV cache compression worth it: early impressions - by mangowire - 09-12-2026, 02:21 AM
|
Users browsing this thread: 1 Guest(s)