long context that stayed useful

[cozywire]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around LM Studio and mini PC.

Would rather learn from your scars than collect my own all at once.

[sanepeak48]

same here on my side.

[sanescope]

That matches what I was seeing. The pattern around tokens/sec has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[thetanest]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.

[stackarch]

That matches what I was seeing. I ended up in about the same place using PyTorch. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[plainember52]

Thanks, that helps. The pattern around router policy has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[bytesupply76]

That matches what I was seeing. That matches what I saw on B200. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[audiopoint18]

That matches what I was seeing. That matches what I saw on B200. I got better results once I stopped chasing one metric and just looked at whether the

same here on my side.

[fusedhall56]

following this because I'm in the same boat.

[audiopoint18]

worth checking the driver first.