Inference › Optimization & Performance › Quantization v
best small quant that still feels smart for Kimi

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Linear Mode
best small quant that still feels smart for Kimi
cachebench Offline
Member
***
Posts: 63
Threads: 13
Joined: Sep 2026
Reputation: 0
#1
10-12-2026, 05:23 AM
Spent some time on this today and wanted to compare notes.

I've gone back and forth between TensorRT-LLM and vLLM for Kimi. The interesting difference for me has been KV cache pressure.

Interested in tradeoffs more than fan loyalty.
Find
Reply


Messages In This Thread
best small quant that still feels smart for Kimi - by cachebench - 10-12-2026, 05:23 AM

Forum Jump:


Users browsing this thread: 1 Guest(s)