+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Security, Privacy & Safety (https://www.datasculpting.ai/inference/forumdisplay.php?fid=23)
+--- Forum: Local Privacy (https://www.datasculpting.ai/inference/forumdisplay.php?fid=76)
+--- Thread: what truly stays local on B200 (/showthread.php?tid=598)
Pages:
1
2
what truly stays local on B200 - rasterbench - 09-29-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once workstation thermals becomes important.
Trying to avoid rebuilding the whole thing twice.
Re: what truly stays local on B200 - mlxgarden - 09-30-2026
I'd start with the simpler stack. The logs were somehow calmer than I was.
Re: what truly stays local on B200 - deltagrid - 09-30-2026
Good point. The pattern around tool calling has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: what truly stays local on B200 - mlxgarden - 10-01-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.
Re: what truly stays local on B200 - audioriver - 10-01-2026
Ran a quick check on my box. On A6000 with llama.cpp, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: what truly stays local on B200 - rasterbench - 10-01-2026
A couple questions before I guess too hard. Are you testing this on 4090 or something else?
Re: what truly stays local on B200 - rocmcrate26 - 10-02-2026
Appreciate the detail. That matches what I saw on 5090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: what truly stays local on B200 - deltagrid - 10-02-2026
That matches what I was seeing. That matches what I saw on Threadripper box. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: what truly stays local on B200 - flintriver50 - 10-02-2026
Can you fill in a bit more context? Are you testing this on 5090 or something else?
Re: what truly stays local on B200 - quantshed - 10-02-2026
My first guess would be this: If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious.