Mixture of Experts (MoE) is an ensemble learning technique that enables creating larger models without increasing training and inference cost.