10-03-2026, 06:30 PM
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once quant drift becomes important.
Trying to avoid rebuilding the whole thing twice.
I care more about stability than squeezing the last possible tokens/sec out of it.
Two paths both seem viable, but they break down differently once quant drift becomes important.
Trying to avoid rebuilding the whole thing twice.
I care more about stability than squeezing the last possible tokens/sec out of it.
Mostly CUDA, sometimes wrong, usually measured