# nomic_bert_moe_gguf

| | |
|---|---|
| **Model type** | `nomic_bert_moe_gguf` |
| **Class** | `NomicBertMoEGGUFModel` |
| **Task** | `feature-extraction` |
| **Source** | `models/gguf_encoders.py` |

## Description

NomicBERT-MoE encoder with periodic routed, non-gated GELU experts.

Routed layers are selected by ``layer % moe_every_n_layers == 1``.
Router probabilities are softmaxed over every expert before top-k
selection and are not renormalized after selection.

## Usage

```bash
mobius build --model <MODEL_ID> output_dir/
```

```python
from mobius import build

model = build("<MODEL_ID>")
```