+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=43)
+--- Thread: adapter stacking (/showthread.php?tid=115)
Pages:
1
2
adapter stacking - benchplease - 09-02-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about token cost.
Simple and boring is a valid answer.
Re: adapter stacking - localtinker - 09-03-2026
subscribed, curious where this lands.
Re: adapter stacking - circuitworks90 - 09-04-2026
Not sure I agree there. I get why that seems right, but on A6000 the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.
Re: adapter stacking - nanobrook - 09-04-2026
Good point. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: adapter stacking - kernelarch - 09-04-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at reranking.
Re: adapter stacking - northgrid33 - 09-04-2026
Interesting, I hadn't tried that. I ended up in about the same place using Ollama.
Re: adapter stacking - cudapeak - 09-05-2026
A couple things I'd check first. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: adapter stacking - benchplease - 09-08-2026
Quote:A couple things I'd check first. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survive
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: adapter stacking - rocmriley - 09-08-2026
A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: adapter stacking - cacheline - 09-08-2026
Maybe, but I had the opposite result. I get why that seems right, but on M3 Max the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.