08-29-2026, 09:40 PM
Quote:I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tokens/sec.
that fixed it for me too.
Mostly CUDA, sometimes wrong, usually measured
|
TensorRT-LLM worth the setup time
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
TensorRT-LLM worth the setup time - by kernelmike - 08-29-2026, 11:38 AM
Re: TensorRT-LLM worth the setup time - by routecrate - 08-29-2026, 12:50 PM
Re: TensorRT-LLM worth the setup time - by kernelmike - 08-29-2026, 09:40 PM
Re: TensorRT-LLM worth the setup time - by routecrate - 08-29-2026, 11:30 PM
Re: TensorRT-LLM worth the setup time - by kernelmike - 08-30-2026, 03:44 AM
Re: TensorRT-LLM worth the setup time - by audiodesk - 08-31-2026, 03:46 PM
Re: TensorRT-LLM worth the setup time - by thetapath - 09-02-2026, 03:57 AM
Re: TensorRT-LLM worth the setup time - by vectorport - 09-02-2026, 04:22 AM
Re: TensorRT-LLM worth the setup time - by torchsmith25 - 09-05-2026, 04:58 AM
Re: TensorRT-LLM worth the setup time - by tensorfield88 - 09-06-2026, 05:47 AM
Re: TensorRT-LLM worth the setup time - by pixelcrate82 - 09-06-2026, 10:28 AM
Re: TensorRT-LLM worth the setup time - by sysstation69 - 09-06-2026, 12:57 PM
|
Users browsing this thread: 1 Guest(s)