VibeVoiceStreamingForConditionalGenerationInference

Model type	VibeVoiceStreamingForConditionalGenerationInference
Class	VibeVoiceStreamingForConditionalGeneration
Task	vibevoice-streaming-tts
Source	models/vibevoice_streaming.py

Description

Six-stage VibeVoice Realtime TTS package with host-owned CFG orchestration.

        flowchart LR
    Text[5-token text window] --> Emb[Qwen2 embedding]
    Emb --> LM[Lower 4-layer Qwen2]
    LM --> TTS[Upper 20-layer Qwen2]
    TTS --> Pos[Positive condition]
    TTS --> Neg[Negative CFG condition]
    Pos --> Diffusion[DDPM/DPM-Solver host loop]
    Neg --> Diffusion
    Diffusion --> Latent[64-D acoustic latent]
    Latent --> Connector[Speech connector]
    Connector --> TTS
    Latent --> Decoder[Causal acoustic decoder]
    Decoder --> Audio[3200 samples per latent]
    

The lower and upper Qwen2 backbones expose independent KV caches; the acoustic decoder exposes convolution histories. Voice presets are trusted prefilled cache artifacts, while CFG, text windowing, DPM-Solver scheduling, and stopping remain host responsibilities. This package is not an ONNX Runtime GenAI runnable model.

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")