|
eval setup for local models
|
09-23-2026, 04:40 AM
Quote:[quote]Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, no What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at quant drift.
post logs or it didn't happen
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
eval setup for local models - by benchplease - 09-21-2026, 01:03 AM
Re: eval setup for local models - by kernelarch - 09-22-2026, 01:48 PM
Re: eval setup for local models - by cacheline - 09-22-2026, 02:11 PM
Re: eval setup for local models - by circuitworks90 - 09-22-2026, 02:50 PM
Re: eval setup for local models - by routearch - 09-23-2026, 03:39 AM
Re: eval setup for local models - by cacheline - 09-23-2026, 04:01 AM
Re: eval setup for local models - by nanobrook - 09-23-2026, 02:34 PM
Re: eval setup for local models - by benchplease - 09-23-2026, 04:40 AM
Re: eval setup for local models - by cacheline - 09-23-2026, 06:05 AM
Re: eval setup for local models - by benchplease - 09-25-2026, 03:11 AM
Re: eval setup for local models - by relayforge - 09-25-2026, 11:22 AM
|
