09-07-2026, 01:18 AM
Quote:Can you fill in a bit more context? What model, what runtime, and roughly what context size?
I tested this tonight. On A6000 with vLLM, I saw basically the same behavior.
curious and caffeinated
|
TensorRT-LLM worth the setup time with TensorRT-LLM
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
TensorRT-LLM worth the setup time with TensorRT-LLM - by syshouse - 09-03-2026, 02:51 AM
Re: TensorRT-LLM worth the setup time with TensorRT-LLM - by syshouse - 09-04-2026, 03:34 PM
Re: TensorRT-LLM worth the setup time with TensorRT-LLM - by vectorport - 09-06-2026, 03:58 AM
Re: TensorRT-LLM worth the setup time with TensorRT-LLM - by rasterbench - 09-06-2026, 08:42 AM
Re: TensorRT-LLM worth the setup time with TensorRT-LLM - by syshouse - 09-06-2026, 09:19 PM
Re: TensorRT-LLM worth the setup time with TensorRT-LLM - by plainscope50 - 09-06-2026, 10:58 PM
Re: TensorRT-LLM worth the setup time with TensorRT-LLM - by batchcraft - 09-06-2026, 10:42 PM
Re: TensorRT-LLM worth the setup time with TensorRT-LLM - by vectorshed43 - 09-07-2026, 12:30 AM
Re: TensorRT-LLM worth the setup time with TensorRT-LLM - by kernelpeak - 09-07-2026, 01:18 AM
Re: TensorRT-LLM worth the setup time with TensorRT-LLM - by batchcraft - 09-07-2026, 01:30 PM
|
Users browsing this thread: 1 Guest(s)