+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Structured Output (https://www.datasculpting.ai/inference/forumdisplay.php?fid=63)
+--- Thread: structured output failures with PyTorch (/showthread.php?tid=1413)
Pages:
1
2
structured output failures with PyTorch - cozygrid12 - 10-17-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on 5090, I ended up with decent performance, but the setup only really made sense once I adjusted around long context.
Happy to post more details if anyone wants them.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: structured output failures with PyTorch - quietsupply11 - 10-18-2026
Quote:Did a bit of testing and thought I'd post the result here.
My first guess would be this: On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.
Re: structured output failures with PyTorch - kernelmike - 10-18-2026
Quote:Quote:Did a bit of testing and thought I'd post the result here.
I hit almost the same issue last night.
Re: structured output failures with PyTorch - benchplease - 10-18-2026
worth checking the driver first.
Re: structured output failures with PyTorch - quantshed - 10-19-2026
same here on my side.
Re: structured output failures with PyTorch - oxiderunner - 10-19-2026
I'd start with the simpler stack.
Re: structured output failures with PyTorch - rocmpath - 10-19-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: structured output failures with PyTorch - ggufwire22 - 10-19-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: structured output failures with PyTorch - audiodesk - 10-19-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: structured output failures with PyTorch - localcraft17 - 10-19-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there.