vector DB that stayed out of the way: anyone testing this?

[audiodesk]

I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.

I'm happy to test one more variant if you want a second data point.