what changed after the inference shift

[prompttinker44]

Posting this in case it saves someone else a little time.

Interesting, I hadn't tried that. I ended up in about the same place using PyTorch. The biggest change for me was simplifying the stack instead of adding one more optimization layer.