+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: NVIDIA / CUDA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=31)
+--- Thread: Tensor cores actually helping here (/showthread.php?tid=1446)
Tensor cores actually helping here - edgecoast - 10-18-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between llama.cpp and llama.cpp for Kimi. The interesting difference for me has been embeddings.
Trying to avoid rebuilding the whole thing twice.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.