|
eval setup for local models
|
09-23-2026, 04:01 AM
Quote:Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quali I don't think that's the whole story. I get why that seems right, but on B200 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
measure twice, benchmark three times
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
eval setup for local models - by benchplease - 09-21-2026, 01:03 AM
Re: eval setup for local models - by kernelarch - 09-22-2026, 01:48 PM
Re: eval setup for local models - by cacheline - 09-22-2026, 02:11 PM
Re: eval setup for local models - by circuitworks90 - 09-22-2026, 02:50 PM
Re: eval setup for local models - by routearch - 09-23-2026, 03:39 AM
Re: eval setup for local models - by cacheline - 09-23-2026, 04:01 AM
Re: eval setup for local models - by nanobrook - 09-23-2026, 02:34 PM
Re: eval setup for local models - by benchplease - 09-23-2026, 04:40 AM
Re: eval setup for local models - by cacheline - 09-23-2026, 06:05 AM
Re: eval setup for local models - by benchplease - 09-25-2026, 03:11 AM
Re: eval setup for local models - by relayforge - 09-25-2026, 11:22 AM
|
