+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Security, Privacy & Safety (https://www.datasculpting.ai/inference/forumdisplay.php?fid=23)
+--- Forum: Local Privacy (https://www.datasculpting.ai/inference/forumdisplay.php?fid=76)
+--- Thread: privacy wins of local models: early impressions (/showthread.php?tid=1326)
privacy wins of local models: early impressions - mlxgarden - 10-16-2026
Posting this in case it saves someone else a little time.
If you were starting from scratch today for this use case, what would you choose first and what would you ignore?
Would rather learn from your scars than collect my own all at once.
Re: privacy wins of local models: early impressions - cachebench - 10-17-2026
I tested this tonight. On M3 Max with PyTorch, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: privacy wins of local models: early impressions - kernelgrid - 10-17-2026
My first guess would be this: On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: privacy wins of local models: early impressions - audioriver - 10-17-2026
A couple things I'd check first. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: privacy wins of local models: early impressions - mlxgarden - 10-17-2026
Good point. I ended up in about the same place using PyTorch.
Re: privacy wins of local models: early impressions - deltagrid - 10-17-2026
I'd approach it in a boring order. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: privacy wins of local models: early impressions - rocmriley - 10-17-2026
Can you fill in a bit more context? Are you testing this on 7900 XTX or something else?