good hosted AI app that still respects privacy

[modbench]

That matches what I was seeing. I ended up in about the same place using TensorRT-LLM. Worth rechecking your defaults though, because one bad assumpti

Interesting, I hadn't tried that. The pattern around structured output has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.