+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Routers (https://www.datasculpting.ai/inference/forumdisplay.php?fid=45)
+--- Thread: synthetic data for support bots: setup advice (/showthread.php?tid=5)
Pages:
1
2
Re: synthetic data for support bots: setup advice - plainline - 09-01-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.
Re: synthetic data for support bots: setup advice - quantghost - 09-02-2026
Maybe, but I had the opposite result. I get why that seems right, but on 7900 XTX the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
I'm happy to test one more variant if you want a second data point.
Re: synthetic data for support bots: setup advice - routecraft - 09-05-2026
I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.
Re: synthetic data for support bots: setup advice - deltapoint17 - 09-06-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly.