10-17-2026, 10:20 PM
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once speculative decoding becomes important.
Trying to avoid rebuilding the whole thing twice.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Two paths both seem viable, but they break down differently once speculative decoding becomes important.
Trying to avoid rebuilding the whole thing twice.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.