Mixture-of-Experts Token Router Visualization
Der Kolibri fliegt schnell
Route tokens
Play
Active experts:
7 of 385
Active parameters:
~3.46B of 78B
Parameter ratio:
4.4%
Expert usage heatmap (across all tokens & layers):