jetmoe¶
Model type |
|
Class |
|
Task |
|
Source |
|
Description¶
JetMoE causal language model with Mixture-of-Attention.
Differs from generic MoE models by routing Q and O projections through experts (MoA), while K and V projections are shared.
HuggingFace reference: JetMoeForCausalLM.
Usage¶
mobius build --model <MODEL_ID> output_dir/
from mobius import build
model = build("<MODEL_ID>")