bloom

Model type	bloom
Class	BloomCausalLMModel
Task	text-generation
Source	models/falcon.py

Description

Bloom CausalLM with ALiBi and word_embeddings_layernorm.

Bloom applies LayerNorm to word embeddings before the transformer blocks, which standard Falcon does not.

Replicates HuggingFace’s BloomForCausalLM.

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")