glm

Model type	glm
Class	GlmCausalLMModel
Task	text-generation
Source	models/glm.py

Description

GLM text model (standard pre-norm + fused gate_up_proj).

GLM is architecturally identical to LLaMA except for the fused gate_up_proj in the MLP. The weight stays fused; activations are split in the MLP forward pass (see

class:

~mobius.models.base.FusedGateUpCausalLMModel).

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")