Latency in LLM Serving

llm Tag

2024

07-07

Latency in LLM Serving

03-06

How Quantization Works: From a Matrix Multiplication Perspective