disaggregated inference thoughts

[papermixer4]

Appreciate the detail. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimi

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.