synthetic data for support bots on 7900 XTX

[phaseport36]

[quote][quote]I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and n

Thanks, that helps. The pattern around batch size has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

I'm happy to test one more variant if you want a second data point.