routing between local and hosted with vLLM

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

routing between local and hosted with vLLM

relaynest82 Offline

Junior Member

Posts: 34
Threads: 6
Joined: Sep 2026
Reputation: 0

#14

10-06-2026, 09:33 AM

Thanks, that helps. The pattern around speculative decoding has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

Find

« Next Oldest | Next Newest »

Messages In This Thread

routing between local and hosted with vLLM - by cudaline - 09-28-2026, 05:06 AM

Re: routing between local and hosted with vLLM - by cacheworks - 09-29-2026, 05:42 AM

Re: routing between local and hosted with vLLM - by latentpeak22 - 09-30-2026, 06:54 AM

Re: routing between local and hosted with vLLM - by phasespark - 09-29-2026, 06:21 PM

Re: routing between local and hosted with vLLM - by bytegarden73 - 09-30-2026, 07:46 AM

Re: routing between local and hosted with vLLM - by macrotinker - 09-30-2026, 03:49 PM

Re: routing between local and hosted with vLLM - by fusedhall - 09-30-2026, 08:16 PM

Re: routing between local and hosted with vLLM - by tensorworks - 09-30-2026, 09:52 PM

Re: routing between local and hosted with vLLM - by rasterpoint - 09-30-2026, 10:37 PM

Re: routing between local and hosted with vLLM - by torchshed - 10-01-2026, 11:14 PM

Re: routing between local and hosted with vLLM - by vectorshed43 - 10-02-2026, 11:40 PM

Re: routing between local and hosted with vLLM - by torchstation - 10-05-2026, 11:47 PM

Re: routing between local and hosted with vLLM - by llmtrack - 10-06-2026, 08:21 AM

Re: routing between local and hosted with vLLM - by relaynest82 - 10-06-2026, 09:33 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)

Login
Username:
Password:	Lost Password?
	Remember me

Sign in with X.com