# pangu_embedded

| | |
|---|---|
| **Model type** | `pangu_embedded` |
| **Class** | `CausalLMModel` |
| **Task** | `text-generation` |
| **Source** | `models/base.py` |

## Description

Standard causal language model with TextModel backbone and LM head.

Compatible with Llama 2/3, Mistral, Qwen2/2.5, and other architectures
that follow the standard decoder-only transformer pattern with GQA and RoPE.

Replicates HuggingFace's ``LlamaForCausalLM``, ``MistralForCausalLM``,
``Qwen2ForCausalLM``, etc.

Inputs: input_ids, attention_mask, position_ids, past_key_values.
Outputs: logits (batch, seq_len, vocab_size), present_key_values.

## Usage

```bash
mobius build --model <MODEL_ID> output_dir/
```

```python
from mobius import build

model = build("<MODEL_ID>")
```