#emergent-modularity

EMO: How Constraining Expert Selection to Document-Level Pools Causes Semantic Modularity to Emerge During MoE Pretraining