In Musk, the entropy generally decreases in MoE, while, in EMoE, an increasing trend is observed in some cases.
also in 63,971 other papers
Quoted from the open-access full text in Europe PMC under the licence the publisher applied. The sentence is reproduced exactly as published; the emphasis is ours.