10-18-2026, 09:51 PM
Ran a quick check on my box. On dual 3090 with vLLM, I saw basically the same behavior. The main thing that moved was latency, not quality.
measure twice, benchmark three times
|
prefix caching on repeated workflows
|
|
« Next Oldest | Next Newest »
|
Users browsing this thread: 1 Guest(s)