Benchmarking

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
MoE-CAP introduces a comprehensive benchmark for evaluating sparse Mixture-of-Experts systems across three key dimensions: Cost, Accuracy, and Performance. The benchmark reveals fundamental trade-offs in MoE deployments and proposes sparsity-aware metrics (S-MBU and S-MFU) along with CAP Radar Diagrams to help practitioners make informed deployment decisions for large-scale MoE systems.
Yinsicheng Jiang, Yao Fu, Yeqi Huang, Ping Nie, Zhan Lu, Leyang Xue, Congjie He, Man-Kit Sit, Jilong Xue, Li Dong, Ziming Miao, Dayou Du, Tairan Xu, Kai Zou, Edoardo Ponti, Luo Mai
PDF DOI ArXiv
MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems