qwen3¶
Model type |
|
Class |
|
Task |
|
Source |
|
Description¶
Qwen3 model with Q/K normalization.
Uses RMSNorm on query and key projections before attention, configured via attn_qk_norm=True in ArchitectureConfig.
Usage¶
mobius build --model <MODEL_ID> output_dir/
from mobius import build
model = build("<MODEL_ID>")