+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Security, Privacy & Safety (https://www.datasculpting.ai/inference/forumdisplay.php?fid=23)
+--- Forum: Model / Agent Security (https://www.datasculpting.ai/inference/forumdisplay.php?fid=77)
+--- Thread: prompt injection in local agent setups (/showthread.php?tid=1087)
Pages:
1
2
prompt injection in local agent setups - pixelcrate82 - 10-13-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Gemma, Ollama, 7900 XTX, with a lot of attention on prefix caching.
If someone wants the exact settings I can clean them up and post them.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: prompt injection in local agent setups - vectorhouse71 - 10-14-2026
My first guess would be this: On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: prompt injection in local agent setups - cudacrate - 10-14-2026
Maybe, but I had the opposite result. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: prompt injection in local agent setups - routearch - 10-14-2026
I think the bottleneck is somewhere else. I've seen people blame HBM constraints when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: prompt injection in local agent setups - deepseeked - 10-14-2026
Appreciate the detail. I ended up in about the same place using MLX.
Re: prompt injection in local agent setups - mlxgarden - 10-15-2026
same here on my side.
Re: prompt injection in local agent setups - cobaltspark - 10-16-2026
that fixed it for me too.
I'm happy to test one more variant if you want a second data point.
Re: prompt injection in local agent setups - thetanest - 10-17-2026
Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. At this point the real bottleneck is my patience.
Re: prompt injection in local agent setups - kernelrunner - 10-18-2026
I don't think that's the whole story. I've seen people blame reranking when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: prompt injection in local agent setups - batchsize - 10-18-2026
Quote:My first guess would be this: On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Interesting, I hadn't tried that. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.