+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Training & Customization (https://www.datasculpting.ai/inference/forumdisplay.php?fid=13)
+--- Forum: Model Merging (https://www.datasculpting.ai/inference/forumdisplay.php?fid=57)
+--- Thread: sqlite + embeddings surprisingly ok: early impressions (/showthread.php?tid=402)
sqlite + embeddings surprisingly ok: early impressions - sanescope - 09-21-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around TensorRT-LLM and mini PC.
I'm especially interested in mistakes worth avoiding.
Re: sqlite + embeddings surprisingly ok: early impressions - torchsupply99 - 09-22-2026
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: sqlite + embeddings surprisingly ok: early impressions - audiopath66 - 09-23-2026
What worked for me was pretty simple. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: sqlite + embeddings surprisingly ok: early impressions - kernelgrid - 09-23-2026
Quote:What worked for me was pretty simple. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. N
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: sqlite + embeddings surprisingly ok: early impressions - relayforge - 09-23-2026
Quote:[quote]What worked for me was pretty simple. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem surv
Appreciate the detail. The pattern around KV cache pressure has been pretty similar for me.
Re: sqlite + embeddings surprisingly ok: early impressions - edgeriver - 09-23-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.