Search / Retrieval / Reranking discussion with Ollama

[syshouse]

subscribed, curious where this lands.

[cobaltworks25]

following this because I'm in the same boat.

If you post your exact settings, people here can probably narrow it down pretty fast.

[thetaarch43]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.

[cudapoint]

Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.