qwen2_moe¶


Model type	`qwen2_moe`
Class	`Qwen2MoECausalLMModel`
Task	`text-generation`
Source	`models/moe.py`

Description¶

Qwen2-MoE causal language model.

Differs from the generic MoECausalLMModel by adding a per-layer shared expert (always active) and a sigmoid gate that scales its contribution, matching Qwen2MoeForCausalLM.

Usage¶

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")