granite

Model type	granite
Class	GraniteCausalLMModel
Task	text-generation
Source	models/granite.py

Description

Granite model with embedding/attention/logits/residual scaling multipliers.

Granite uses scaling factors on embeddings, attention, logits, and residual connections. These are configured via the HF config:

• embedding_multiplier: scales embeddings after lookup

• attention_multiplier: replaces 1/sqrt(head_dim) as attention scale

• logits_scaling: divides final logits

• residual_multiplier: scales attention/MLP outputs before residual add

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")