prefix caching on repeated workflows with TensorRT-LLM

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

prefix caching on repeated workflows with TensorRT-LLM

cacheline Offline

Senior Member

Posts: 341
Threads: 51
Joined: Sep 2026
Reputation: 0

10-19-2026, 03:51 AM

That matches what I was seeing. That matches what I saw on Threadripper box. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

measure twice, benchmark three times

Find

« Next Oldest | Next Newest »

Messages In This Thread

prefix caching on repeated workflows with TensorRT-LLM - by cacheline - 10-19-2026, 04:56 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by cacheline - 10-19-2026, 02:07 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by cacheline - 10-18-2026, 09:59 PM

Re: prefix caching on repeated workflows with TensorRT-LLM - by turbogrid11 - 10-19-2026, 02:19 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by sparrowhall - 10-19-2026, 04:45 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by cacheline - 10-19-2026, 03:20 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by amberplane - 10-19-2026, 05:18 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by kernelghost - 10-19-2026, 03:26 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by cacheline - 10-19-2026, 03:51 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by kernelghost - 10-19-2026, 03:58 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by cacheline - 10-19-2026, 04:59 AM

Re: prefix caching on repeated workflows with TensorRT-LLM - by turboscope - 10-19-2026, 05:57 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)

Login
Username:
Password:	Lost Password?
	Remember me

Sign in with X.com