|
TensorRT-LLM worth the setup time: early impressions
|
10-13-2026, 08:35 AM
Quote:Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt? What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals. Once you know the plain path works, then add the faster path back in piece by piece. |
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
TensorRT-LLM worth the setup time: early impressions - by kernelmike - 10-04-2026, 04:18 PM
Re: TensorRT-LLM worth the setup time: early impressions - by kernelmike - 10-06-2026, 04:40 PM
Re: TensorRT-LLM worth the setup time: early impressions - by northcraft90 - 10-06-2026, 09:24 PM
Re: TensorRT-LLM worth the setup time: early impressions - by cudapeak - 10-07-2026, 06:08 AM
Re: TensorRT-LLM worth the setup time: early impressions - by torchtrack24 - 10-08-2026, 06:19 PM
Re: TensorRT-LLM worth the setup time: early impressions - by batchsize - 10-08-2026, 07:08 PM
Re: TensorRT-LLM worth the setup time: early impressions - by kernelmike - 10-09-2026, 07:43 PM
Re: TensorRT-LLM worth the setup time: early impressions - by rasterbench - 10-10-2026, 08:18 AM
Re: TensorRT-LLM worth the setup time: early impressions - by northcraft90 - 10-13-2026, 08:35 AM
Re: TensorRT-LLM worth the setup time: early impressions - by routecrate - 10-13-2026, 09:46 AM
|
