+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: NVIDIA / CUDA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=31)
+--- Thread: Tensor cores actually helping here with LM Studio (/showthread.php?tid=1147)
Pages:
1
2
Tensor cores actually helping here with LM Studio - edgecoast - 10-13-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of vLLM and B200 felt surprisingly usable once I tuned around HBM constraints.
If someone wants the exact settings I can clean them up and post them.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: Tensor cores actually helping here with LM Studio - pixelspark - 10-13-2026
I hit almost the same issue last night.
Re: Tensor cores actually helping here with LM Studio - syshouse - 10-15-2026
I hit almost the same issue last night.
Re: Tensor cores actually helping here with LM Studio - orbitbench6 - 10-15-2026
I tested this tonight. On 7900 XTX with llama.cpp, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
Re: Tensor cores actually helping here with LM Studio - audiorunner74 - 10-15-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: Tensor cores actually helping here with LM Studio - kernelmike - 10-15-2026
Quote:A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Thanks, that helps. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Tensor cores actually helping here with LM Studio - sparrowkite53 - 10-15-2026
Interesting, I hadn't tried that. That matches what I saw on mini PC.
Re: Tensor cores actually helping here with LM Studio - amberpatch25 - 10-16-2026
Thanks, that helps. I ended up in about the same place using TensorRT-LLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Tensor cores actually helping here with LM Studio - dockember - 10-16-2026
Quote:Thanks, that helps. I ended up in about the same place using TensorRT-LLM. Worth rechecking your defaults though, because one bad assumption can make
worth checking the driver first.
Re: Tensor cores actually helping here with LM Studio - orbitbench6 - 10-17-2026
Appreciate the detail. I ended up in about the same place using MLX. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.