# bailing_moe

| | |
|---|---|
| **Model type** | `bailing_moe` |
| **Class** | `Ernie45MoECausalLMModel` |
| **Task** | `text-generation` |
| **Source** | `models/moe.py` |

## Description

Ernie 4.5 Mixture of Experts causal language model.

Uses SoftmaxTopKGate routing with an always-active shared expert per layer
(no sigmoid gate on the shared contribution), matching ``Ernie4_5_MoeForCausalLM``.

Key differences from :class:`MoECausalLMModel`:
- Each MoE layer adds a ``shared_expert`` MLP summed unconditionally.
- Expert weights stored as fused 3D tensors in HF; split in ``preprocess_weights``.

## Usage

```bash
mobius build --model <MODEL_ID> output_dir/
```

```python
from mobius import build

model = build("<MODEL_ID>")
```