#ai-architecture

EMO: How Constraining Expert Selection to Document-Level Pools Causes Semantic Modularity to Emerge During MoE Pretraining