+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: Threadripper build for inference on A6000 (/showthread.php?tid=662)
Pages:
1
2
Threadripper build for inference on A6000 - vectorember - 10-02-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about continuous batching.
Would rather learn from your scars than collect my own all at once.
Re: Threadripper build for inference on A6000 - labnest - 10-02-2026
I hit almost the same issue last night.
Re: Threadripper build for inference on A6000 - stackarch - 10-02-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Threadripper build for inference on A6000 - tinycontext - 10-02-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.
Re: Threadripper build for inference on A6000 - Data Sculpting - 10-02-2026
What exact setup are you running? What model, what runtime, and roughly what context size?
Re: Threadripper build for inference on A6000 - sanevault - 10-04-2026
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Threadripper build for inference on A6000 - latentpeak22 - 10-04-2026
worth checking the driver first.
Re: Threadripper build for inference on A6000 - tensorworks - 10-05-2026
following this because I'm in the same boat.
Re: Threadripper build for inference on A6000 - homelabhenry - 10-05-2026
I hit almost the same issue last night.
Re: Threadripper build for inference on A6000 - kernelghost - 10-05-2026
Quote:I hit almost the same issue last night.
I hit almost the same issue last night.