phimoe

Model type	phimoe
Class	Phi3MoECausalLMModel
Task	text-generation
Source	models/moe.py

Description

Phi-3 Mixture of Experts model.

Uses MoE decoder layers with sparse expert routing. Each layer replaces the standard MLP with a MoELayer containing num_local_experts expert MLPs.

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")