hybrid retrieval results for FLUX

[homelabhenry]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around Ollama and GB200.

I'm especially interested in mistakes worth avoiding.

This is mostly for local work rather than public serving.

[tensorfield88]

A couple questions before I guess too hard. Is this local-only or are you routing anything externally?

[homelabhenry]

I hit almost the same issue last night.

[tensorfield88]

Can you fill in a bit more context? What model, what runtime, and roughly what context size?

[phasescope]

subscribed, curious where this lands. I keep promising myself this is the last rebuild.

[homelabhenry]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[routeforge64]

Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?

[quietsupply11]

that fixed it for me too.

I'm happy to test one more variant if you want a second data point.

[cacheworks]

What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?

[tinycontext]

worth checking the driver first.

If you post your exact settings, people here can probably narrow it down pretty fast.