flux pipeline local tips with llama.cpp

[cobaltspark]

Did a bit of testing and thought I'd post the result here.

After a few runs on mini PC, I ended up with decent performance, but the setup only really made sense once I adjusted around structured output.

Happy to post more details if anyone wants them.

This is mostly for local work rather than public serving.

If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.

[routecraft]

That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.