+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: prefix caching on repeated workflows with TensorRT-LLM (/showthread.php?tid=901)
Pages:
1
2
Re: prefix caching on repeated workflows with TensorRT-LLM - cacheline - 10-14-2026
following this because I'm in the same boat.
Re: prefix caching on repeated workflows with TensorRT-LLM - cacheline - 10-15-2026
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.