routing between local and hosted: what are you seeing?
|
routing between local and hosted: what are you seeing?
|
09-10-2026, 03:28 AM
Quote:A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefill/decode split. Once I hit almost the same issue last night. I'm happy to test one more variant if you want a second data point.
throughput matters, but tail latency matters too
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
routing between local and hosted: what are you seeing? - by oxidegrove40 - 09-07-2026, 02:02 AM
Re: routing between local and hosted: what are you seeing? - by deepseeked - 09-07-2026, 03:03 AM
Re: routing between local and hosted: what are you seeing? - by visionforge - 09-09-2026, 03:49 PM
Re: routing between local and hosted: what are you seeing? - by tensorworks - 09-08-2026, 03:38 AM
Re: routing between local and hosted: what are you seeing? - by mintstation99 - 09-09-2026, 05:27 PM
Re: routing between local and hosted: what are you seeing? - by batchsize - 09-10-2026, 03:28 AM
Re: routing between local and hosted: what are you seeing? - by signalcraft14 - 09-09-2026, 06:41 PM
Re: routing between local and hosted: what are you seeing? - by mangoquill - 09-10-2026, 04:11 AM
Re: routing between local and hosted: what are you seeing? - by localpatch - 09-10-2026, 12:49 PM
|
