+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Model Comparisons (https://www.datasculpting.ai/inference/forumdisplay.php?fid=67)
+--- Thread: image generation model locals actually use with vLLM (/showthread.php?tid=558)
Pages:
1
2
image generation model locals actually use with vLLM - cachebench - 09-28-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on dual 3090, or squeeze more out of the stack and deal with the complexity.
Trying to avoid rebuilding the whole thing twice.
This is mostly for local work rather than public serving.
Re: image generation model locals actually use with vLLM - vectorshed43 - 09-29-2026
Quote:Spent some time on this today and wanted to compare notes.
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: image generation model locals actually use with vLLM - phasescope - 09-29-2026
I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere.
Re: image generation model locals actually use with vLLM - batchsize - 09-29-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at quant drift. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: image generation model locals actually use with vLLM - macrolab - 09-29-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: image generation model locals actually use with vLLM - vrampilot - 09-29-2026
Quote:Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: image generation model locals actually use with vLLM - mangopeak72 - 09-29-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly.
Re: image generation model locals actually use with vLLM - latentgarden - 09-30-2026
worth checking the driver first.
Re: image generation model locals actually use with vLLM - southtinker - 10-01-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: image generation model locals actually use with vLLM - ggufvault - 10-02-2026
Quote:I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Once you know the plain p
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.