+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: RAG & Knowledge Systems (https://www.datasculpting.ai/inference/forumdisplay.php?fid=14)
+--- Forum: RAG (https://www.datasculpting.ai/inference/forumdisplay.php?fid=58)
+--- Thread: where reranking mattered for FLUX (/showthread.php?tid=940)
Pages:
1
2
where reranking mattered for FLUX - labpath - 10-10-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of PyTorch and 4090 felt surprisingly usable once I tuned around KV cache pressure.
Maybe useful as a starting point for newer folks.
Re: where reranking mattered for FLUX - quietcraft - 10-10-2026
What worked for me was pretty simple. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: where reranking mattered for FLUX - audionode8 - 10-11-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: where reranking mattered for FLUX - sparrowbrook - 10-12-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: where reranking mattered for FLUX - routecrate - 10-12-2026
Maybe, but I had the opposite result. I get why that seems right, but on GB200 the bottleneck felt elsewhere.
Re: where reranking mattered for FLUX - sanepeak48 - 10-12-2026
Tried to reproduce it just now. On 4090 with MLX, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
Re: where reranking mattered for FLUX - macropilot - 10-13-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: where reranking mattered for FLUX - visionforge - 10-14-2026
I hit almost the same issue last night.
Re: where reranking mattered for FLUX - tensorfield88 - 10-17-2026
Quote:I hit almost the same issue last night.
Maybe, but I had the opposite result. I've seen people blame router policy when the real issue was much more mundane.
Re: where reranking mattered for FLUX - kernelghost - 10-17-2026
Quote:Quote:I hit almost the same issue last night.
I'd start with the simpler stack.