opt

Model type	opt
Class	OPTCausalLMModel
Task	text-generation
Source	models/opt.py

Description

OPT causal language model.

Same architecture as GPT-2 (pre-norm LayerNorm, absolute positional embeddings, standard attention with KV cache) but with different HF weight naming convention and a position embedding offset of 2.

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")