vector DB that stayed out of the way with vLLM

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

vector DB that stayed out of the way with vLLM

batchsize Offline

Offline

Member

Posts: 137
Threads: 16
Joined: Sep 2026
Reputation: 0

#15

10-19-2026, 01:55 AM

Interesting, I hadn't tried that. I ended up in about the same place using PyTorch. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

I'm happy to test one more variant if you want a second data point.

throughput matters, but tail latency matters too

Find

Reply

« Next Oldest | Next Newest »

Messages In This Thread

vector DB that stayed out of the way with vLLM - by dockember - 10-15-2026, 02:14 AM

Re: vector DB that stayed out of the way with vLLM - by homelabhenry - 10-17-2026, 02:30 AM

Re: vector DB that stayed out of the way with vLLM - by vramscope - 10-18-2026, 10:26 PM

Re: vector DB that stayed out of the way with vLLM - by batchsize - 10-19-2026, 01:55 AM

Re: vector DB that stayed out of the way with vLLM - by thetafield - 10-17-2026, 03:43 AM

Re: vector DB that stayed out of the way with vLLM - by orbitbench6 - 10-17-2026, 11:49 AM

Re: vector DB that stayed out of the way with vLLM - by mlxgarden - 10-17-2026, 12:57 PM

Re: vector DB that stayed out of the way with vLLM - by fastscope - 10-17-2026, 03:19 PM

Re: vector DB that stayed out of the way with vLLM - by ggufvault - 10-17-2026, 08:02 PM

Re: vector DB that stayed out of the way with vLLM - by stackpilot - 10-17-2026, 10:49 PM

Re: vector DB that stayed out of the way with vLLM - by tensorfield88 - 10-18-2026, 06:43 PM

Re: vector DB that stayed out of the way with vLLM - by sparrowkite53 - 10-18-2026, 07:40 PM

Re: vector DB that stayed out of the way with vLLM - by vramport - 10-18-2026, 11:19 PM

Re: vector DB that stayed out of the way with vLLM - by vramport - 10-18-2026, 11:40 PM

Re: vector DB that stayed out of the way with vLLM - by torchsupply99 - 10-19-2026, 12:43 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)