# vibevoice

| | |
|---|---|
| **Model type** | `vibevoice` |
| **Class** | `VibeVoiceForConditionalGeneration` |
| **Task** | `vibevoice-tts` |
| **Source** | `models/vibevoice.py` |

## Description

Eight-stage VibeVoice 1.5B continuous-token TTS pipeline.

```{mermaid}
flowchart LR
    Ref[Reference audio] --> AE[Audio encoder]
    AE --> AP[Acoustic projector]
    Text[Text tokens] --> Emb[Embedding mixer]
    AP --> Emb
    Emb --> Qwen[Qwen2 decoder]
    Qwen --> PosCond[Positive decoder condition]
    Qwen --> NegCond[Negative decoder condition]
    PosCond --> Diff[DPM-Solver diffusion head]
    NegCond --> Diff
    Qwen --> PosKV[Positive KV cache]
    Qwen --> NegKV[Negative CFG KV cache]
    PosKV --> Qwen
    NegKV --> Qwen
    Diff --> Latent[64-D acoustic latent]
    Latent --> AD[Streaming audio decoder]
    AD --> Wave[3200-sample waveform chunk]
    Wave --> SE[Semantic encoder]
    SE --> SP[Semantic projector]
    SP --> Qwen
```

The exported package contains the audio encoder, acoustic projector,
embedding mixer, Qwen2 decoder, diffusion head, audio decoder, semantic
encoder, and semantic projector. For every generated audio token, the host
runs the positive and negative CFG decoder branches, samples one 64-D
acoustic latent with DPM-Solver, emits a 3200-sample waveform chunk, and
feeds its semantic embedding into the next decoder step.

The decoder owns dual Qwen KV caches. The streaming acoustic decoder and
semantic encoder each expose their convolution histories, for 34 explicit
cache slots per tokenizer stack. The negative CFG branch resets to a valid
suffix after audio BOS; because ``GroupQueryAttention`` only represents
prefix-valid lengths through ``seqlens_k``, this decoder declares an
arbitrary-mask contract and retains standard ONNX ``Attention``. The host,
rather than ONNX Runtime GenAI, owns this multi-stage orchestration; the
package is therefore exportable but not an OGA runnable model.

## Usage

```bash
mobius build --model <MODEL_ID> output_dir/
```

```python
from mobius import build

model = build("<MODEL_ID>")
```