|
consumer GPU serving economics
|
10-18-2026, 10:35 AM
Quote:Ran a quick check on my box. On A6000 with Ollama, I saw basically the same behavior. The main thing that moved was latency, not quality. I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup. |
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
consumer GPU serving economics - by cudapeak - 10-13-2026, 04:49 AM
Re: consumer GPU serving economics - by kernelmike - 10-13-2026, 05:25 AM
Re: consumer GPU serving economics - by batchsize - 10-13-2026, 09:41 AM
Re: consumer GPU serving economics - by ggufwire22 - 10-14-2026, 09:55 AM
Re: consumer GPU serving economics - by localpatch78 - 10-18-2026, 10:45 PM
Re: consumer GPU serving economics - by localpatch78 - 10-14-2026, 10:26 AM
Re: consumer GPU serving economics - by localpatch - 10-17-2026, 10:29 AM
Re: consumer GPU serving economics - by rocmpath - 10-18-2026, 10:35 AM
Re: consumer GPU serving economics - by sparrowshed - 10-19-2026, 02:17 AM
|
