+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Model Merging (https://www.datasculpting.ai/inference/forumdisplay.php?fid=57)
+--- Thread: sqlite + embeddings surprisingly ok (/showthread.php?tid=1474)
Re: sqlite + embeddings surprisingly ok - amberpatch25 - 10-18-2026
That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: sqlite + embeddings surprisingly ok - vectorcrate98 - 10-18-2026
that fixed it for me too.
Re: sqlite + embeddings surprisingly ok - southtinker - 10-18-2026
I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: sqlite + embeddings surprisingly ok - quietarch - 10-18-2026
What worked for me was pretty simple. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: sqlite + embeddings surprisingly ok - mangohouse - 10-18-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
sqlite + embeddings surprisingly ok - turboyard94 - 10-19-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of TensorRT-LLM and M3 Max felt surprisingly usable once I tuned around tail latency.
Maybe useful as a starting point for newer folks.
I care more about stability than squeezing the last possible tokens/sec out of it.