09-18-2026, 06:07 AM
Interesting, I hadn't tried that. I ended up in about the same place using PyTorch. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
If you post your exact settings, people here can probably narrow it down pretty fast.
If you post your exact settings, people here can probably narrow it down pretty fast.
throughput matters, but tail latency matters too