|
dual GPU split inference: early impressions
|
10-15-2026, 11:11 AM
Quote:I'd approach it in a boring order. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once Appreciate the detail. I ended up in about the same place using TensorRT-LLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
throughput matters, but tail latency matters too
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
dual GPU split inference: early impressions - by relaynest82 - 10-13-2026, 05:21 PM
Re: dual GPU split inference: early impressions - by batchember - 10-15-2026, 05:36 AM
Re: dual GPU split inference: early impressions - by turboyard94 - 10-15-2026, 10:03 AM
Re: dual GPU split inference: early impressions - by batchsize - 10-15-2026, 11:11 AM
Re: dual GPU split inference: early impressions - by routecrate - 10-15-2026, 11:20 AM
Re: dual GPU split inference: early impressions - by embersmith - 10-15-2026, 08:09 PM
|
