Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
where reranking mattered: worth it?
#1
Spent some time on this today and wanted to compare notes.

Two paths both seem viable, but they break down differently once token cost becomes important.

Trying to avoid rebuilding the whole thing twice.

If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Reply
#2
Quote:Spent some time on this today and wanted to compare notes.

Good point. The pattern around INT4 has been pretty similar for me.
Reply
#3
I tested this tonight. On A6000 with vLLM, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)