phi3small

Model type	phi3small
Class	Phi3SmallCausalLMModel
Task	text-generation
Source	models/phi.py

Description

Phi3-Small model with block-sparse attention.

Uses block-sparse attention with alternating dense/sparse layers, MuP (maximal update parameterization) scaling, and GeGELU activation with clamping.

Replicates HuggingFace’s Phi3SmallForCausalLM.

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")