nomic_bert_moe_gguf¶


Model type	`nomic_bert_moe_gguf`
Class	`NomicBertMoEGGUFModel`
Task	`feature-extraction`
Source	`models/gguf_encoders.py`

Description¶

NomicBERT-MoE encoder with periodic routed, non-gated GELU experts.

Routed layers are selected by layer % moe_every_n_layers == 1. Router probabilities are softmaxed over every expert before top-k selection and are not renormalized after selection.

Usage¶

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")