+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: GPUs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=28)
+--- Thread: Used A6000 worth it now (/showthread.php?tid=966)
Used A6000 worth it now - embersmith - 10-09-2026
Did a bit of testing and thought I'd post the result here.
Short version: vLLM behaved well for me, but only after I stopped ignoring structured output.
Curious if anyone else landed in the same place.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: Used A6000 worth it now - rocmriley - 10-10-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. Not glamorous advice, but repeatable baselines save a lot of time. I keep promising myself this is the last rebuild.
Re: Used A6000 worth it now - batchember - 10-10-2026
Maybe, but I had the opposite result. I've seen people blame prefix caching when the real issue was much more mundane.
Re: Used A6000 worth it now - vrambandit - 10-13-2026
Good point. That matches what I saw on B200. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Used A6000 worth it now - turboyard94 - 10-13-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at HBM constraints. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Used A6000 worth it now - dockkite69 - 10-13-2026
That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Used A6000 worth it now - rocmriley - 10-13-2026
Quote:That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at wh
following this because I'm in the same boat.
Re: Used A6000 worth it now - amberrunner - 10-14-2026
Thanks, that helps. I ended up in about the same place using PyTorch. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Used A6000 worth it now - vrambandit - 10-17-2026
same here on my side.
Re: Used A6000 worth it now - rocmriley - 10-18-2026
Quote:same here on my side.
I'd approach it in a boring order. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.