#memory-efficiency

EMO: How Constraining Expert Selection to Document-Level Pools Causes Semantic Modularity to Emerge During MoE Pretraining