09-09-2026, 08:49 PM
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once prefill/decode split becomes important.
Trying to avoid rebuilding the whole thing twice.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Two paths both seem viable, but they break down differently once prefill/decode split becomes important.
Trying to avoid rebuilding the whole thing twice.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
7900 XTX / ROCm / somehow still optimistic