10-19-2026, 04:02 AM
I don't think that's the whole story. I've seen people blame continuous batching when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Inference
›
Prompting & Context Engineering
›
Structured Output
structured output failures with vLLM
structured output failures with vLLM
|
structured output failures with vLLM
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
structured output failures with vLLM - by southtinker - 10-17-2026, 06:42 PM
Re: structured output failures with vLLM - by batchpeak - 10-17-2026, 10:53 PM
Re: structured output failures with vLLM - by sparrowhall - 10-18-2026, 04:51 PM
Re: structured output failures with vLLM - by cudapoint - 10-18-2026, 05:08 PM
Re: structured output failures with vLLM - by embershed99 - 10-18-2026, 09:09 PM
Re: structured output failures with vLLM - by mangovault67 - 10-19-2026, 01:29 AM
Re: structured output failures with vLLM - by macrotinker - 10-19-2026, 01:59 AM
Re: structured output failures with vLLM - by turboarch95 - 10-19-2026, 02:29 AM
Re: structured output failures with vLLM - by batchsize - 10-19-2026, 02:30 AM
Re: structured output failures with vLLM - by vramghost - 10-19-2026, 04:02 AM
Re: structured output failures with vLLM - by vramport - 10-19-2026, 04:23 AM
Re: structured output failures with vLLM - by quantsupply - 10-19-2026, 05:25 AM
|
Users browsing this thread: 1 Guest(s)