codegen

Model type	codegen
Class	CodeGenCausalLMModel
Task	text-generation
Source	models/gptj_codegen.py

Description

CodeGen causal language model.

Identical architecture to GPT-J (single-norm parallel residual) but uses a fused qkv_proj weight that is split during preprocess_weights.

CodeGen is MHA only (no GQA), so num_key_value_heads is forced to match num_attention_heads.

Replicates HuggingFace’s CodeGenForCausalLM.

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")