+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Harnesses, Agents & Orchestration (https://www.datasculpting.ai/inference/forumdisplay.php?fid=11)
+--- Forum: Routers (https://www.datasculpting.ai/inference/forumdisplay.php?fid=45)
+--- Thread: synthetic data for support bots on 7900 XTX (/showthread.php?tid=771)
Pages:
1
2
Re: synthetic data for support bots on 7900 XTX - phaseport36 - 10-19-2026
Quote:[quote][quote]I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and n
Thanks, that helps. The pattern around batch size has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
I'm happy to test one more variant if you want a second data point.
Re: synthetic data for support bots on 7900 XTX - amberhall - 10-19-2026
that fixed it for me too.
Re: synthetic data for support bots on 7900 XTX - routecraft - 10-19-2026
Not sure I agree there. I get why that seems right, but on A6000 the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one. Turns out 'just one more test' was a lie.
Re: synthetic data for support bots on 7900 XTX - audioforge - 10-19-2026
Quote:Not sure I agree there. I get why that seems right, but on A6000 the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting
Thanks, that helps. The pattern around continuous batching has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: synthetic data for support bots on 7900 XTX - circuitworks90 - 10-19-2026
Quote:I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Not sure I agree there. I get why that seems right, but on 7900 XTX the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: synthetic data for support bots on 7900 XTX - quantsupply - 10-19-2026
Quote:[quote]I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there.