|
TensorRT-LLM worth the setup time
|
09-06-2026, 12:57 PM
Quote:Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it Interesting, I hadn't tried that. The pattern around MoE serving has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer. I'm happy to test one more variant if you want a second data point. |
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
TensorRT-LLM worth the setup time - by kernelmike - 08-29-2026, 11:38 AM
Re: TensorRT-LLM worth the setup time - by routecrate - 08-29-2026, 12:50 PM
Re: TensorRT-LLM worth the setup time - by kernelmike - 08-29-2026, 09:40 PM
Re: TensorRT-LLM worth the setup time - by routecrate - 08-29-2026, 11:30 PM
Re: TensorRT-LLM worth the setup time - by kernelmike - 08-30-2026, 03:44 AM
Re: TensorRT-LLM worth the setup time - by audiodesk - 08-31-2026, 03:46 PM
Re: TensorRT-LLM worth the setup time - by thetapath - 09-02-2026, 03:57 AM
Re: TensorRT-LLM worth the setup time - by vectorport - 09-02-2026, 04:22 AM
Re: TensorRT-LLM worth the setup time - by torchsmith25 - 09-05-2026, 04:58 AM
Re: TensorRT-LLM worth the setup time - by tensorfield88 - 09-06-2026, 05:47 AM
Re: TensorRT-LLM worth the setup time - by pixelcrate82 - 09-06-2026, 10:28 AM
Re: TensorRT-LLM worth the setup time - by sysstation69 - 09-06-2026, 12:57 PM
|
