+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: PyTorch / JAX / Other Frameworks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=34)
+--- Thread: when a plain Python stack is enough (/showthread.php?tid=996)
Pages:
1
2
Re: when a plain Python stack is enough - dockriver - 10-19-2026
that fixed it for me too.
Re: when a plain Python stack is enough - orbitgarden - 10-19-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.
Re: when a plain Python stack is enough - vectorfield - 10-19-2026
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one. Turns out 'just one more test' was a lie.
Re: when a plain Python stack is enough - dockember - 10-19-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.
Re: when a plain Python stack is enough - amberpatch25 - 10-19-2026
Interesting, I hadn't tried that. The pattern around speculative decoding has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.