+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Model Comparisons (https://www.datasculpting.ai/inference/forumdisplay.php?fid=67)
+--- Thread: image generation model locals actually use (/showthread.php?tid=360)
Pages:
1
2
image generation model locals actually use - thetaplane - 09-19-2026
This keeps coming up in conversation so I figured I'd start a thread.
One thing I keep noticing is that people underestimate prefix caching until they actually try to run this stuff for more than one afternoon.
Maybe obvious to some of you, but it clicked for me this week.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: image generation model locals actually use - cozykite - 09-20-2026
Appreciate the detail. The pattern around prefix caching has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: image generation model locals actually use - southpatch - 09-23-2026
What worked for me was pretty simple. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: image generation model locals actually use - batchsize - 09-23-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: image generation model locals actually use - visionforge - 09-26-2026
following this because I'm in the same boat.
Re: image generation model locals actually use - rocmcrate26 - 09-26-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: image generation model locals actually use - batchshed - 09-28-2026
worth checking the driver first.
Re: image generation model locals actually use - turboyard94 - 09-28-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: image generation model locals actually use - vramscope - 09-28-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Re: image generation model locals actually use - nanopatch - 09-28-2026
I'd approach it in a boring order. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives.