orion

Model type	orion
Class	LayerNormCausalLMModel
Task	text-generation
Source	models/base.py

Description

CausalLM variant that uses LayerNorm instead of RMSNorm.

Drop-in replacement for CausalLMModel for architectures where HuggingFace uses standard nn.LayerNorm (weight + bias) in place of the bias-free RMSNorm used by most Llama-family models.

Used by: Cohere, Cohere2, StarCoder2, StableLM.

Replicates HuggingFace’s CohereForCausalLM, Starcoder2ForCausalLM, and StableLmForCausalLM.

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")