falcon

Model type	falcon
Class	FalconCausalLMModel
Task	text-generation
Source	models/falcon.py

Description

Falcon CausalLM with ALiBi or RoPE positional encoding.

Supports both ALiBi (Falcon-40B, 180B) and RoPE (Falcon-7B) positional encoding, as well as parallel and sequential attention+MLP.

Outer attribute transformer matches HF Falcon weight prefix.

Replicates HuggingFace’s FalconForCausalLM.

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")