xlm

Model type	xlm
Class	XLMCausalLMModel
Task	text-generation
Source	models/xlm.py

Description

XLM causal language model (XLMWithLMHeadModel).

Uses _XLMTextModel instead of _GPT2TextModel to add the embedding LayerNorm and skip the final norm. The lm_head always has a bias (from pred_layer.proj).

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")