+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Embedding / Reranking Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=53)
+--- Thread: MCP server discovery mess with vLLM (/showthread.php?tid=634)
Pages:
1
2
MCP server discovery mess with vLLM - quiettoken - 10-01-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on GB200, or squeeze more out of the stack and deal with the complexity.
What would you pick if you had to live with it for a few months?
This is mostly for local work rather than public serving.
Re: MCP server discovery mess with vLLM - mangofield33 - 10-02-2026
Quote:Spent some time on this today and wanted to compare notes.
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: MCP server discovery mess with vLLM - plainworks - 10-02-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at MoE serving.
Re: MCP server discovery mess with vLLM - pixelcrate82 - 10-02-2026
Thanks, that helps. The pattern around tail latency has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: MCP server discovery mess with vLLM - mangowire - 10-03-2026
Not sure I agree there. I've seen people blame token cost when the real issue was much more mundane.
Re: MCP server discovery mess with vLLM - mellowstream - 10-03-2026
Quote:Not sure I agree there. I've seen people blame token cost when the real issue was much more mundane.
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tail latency. Not glamorous advice, but repeatable baselines save a lot of time.
Re: MCP server discovery mess with vLLM - torchshed66 - 10-04-2026
Quote:Quote:Not sure I agree there. I've seen people blame token cost when the real issue was much more mundane.
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
Re: MCP server discovery mess with vLLM - bytepilot - 10-04-2026
I think the bottleneck is somewhere else. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: MCP server discovery mess with vLLM - labnest - 10-05-2026
Can you fill in a bit more context? Are you testing this on GB200 or something else?
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: MCP server discovery mess with vLLM - ggufwire22 - 10-05-2026
Good point. I ended up in about the same place using Ollama. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.