+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Model Comparisons (https://www.datasculpting.ai/inference/forumdisplay.php?fid=67)
+--- Thread: flux pipeline local tips with llama.cpp (/showthread.php?tid=935)
flux pipeline local tips with llama.cpp - cobaltspark - 10-08-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on mini PC, I ended up with decent performance, but the setup only really made sense once I adjusted around structured output.
Happy to post more details if anyone wants them.
This is mostly for local work rather than public serving.
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Re: flux pipeline local tips with llama.cpp - routecraft - 10-09-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.