MoE (Mixture of Experts) Explained with Diagrams: Routing, Mixtral Serving, Monitoring, and Kubernetes Checks
Introduction TL;DR MoE activates only a small subset of expert FFNs per token (conditional computation), scaling total capacity without proportional per-token compute. In Transformers, the mainstream pattern is replacing the dense FFN/MLP with an MoE FFN (router + experts). Production bottlenecks often come from routing imbalance, capacity overflow (drops), all-to-all communication, and memory bandwidth; serving requires observability and cluster tuning. Why it matters: MoE is a combined model + distributed-systems problem, not just a modeling trick. ...