Help with Vector Databases: worth it?

[rocmstack25]

My first guess would be this: On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. At this point the real bottleneck is my patience.

[southspark]

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.

[orbitbench6]

Good point. That matches what I saw on dual 3090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[southspark]

Thanks, that helps. The pattern around structured output has been pretty similar for me.

[relayforge]

Thanks, that helps. The pattern around structured output has been pretty similar for me.

Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.

[rocmstack25]

Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. I keep promising myself this is the last rebuild.