Architecture
MoE Routing
Algorithm that selects which expert sub-networks process each input token.
Definition
Routing decides, for each token, which 1–2 of N experts to activate. Top-k softmax, expert-choice routing and load-balancing losses keep experts utilised evenly. Routing quality dominates the practical performance of MoE models like Mixtral and DeepSeek-V3.
Common use cases
- MoE training
- Expert load balancing
- Inference efficiency