10-09-2026, 10:02 PM
Did a bit of testing and thought I'd post the result here.
I tried this with Mixtral using llama.cpp on 4090. The headline result was better than I expected, but VRAM headroom still ended up being the limiting factor.
Curious if anyone else landed in the same place. Turns out 'just one more test' was a lie.
I tried this with Mixtral using llama.cpp on 4090. The headline result was better than I expected, but VRAM headroom still ended up being the limiting factor.
Curious if anyone else landed in the same place. Turns out 'just one more test' was a lie.