nanochat

Model type

nanochat

Class

NanoChatCausalLMModel

Task

text-generation

Source

models/nanochat.py

Description

NanoChat model: FCMLP with relu2 + parameter-free RMSNorm + QK-norm.

Usage

mobius build --model <MODEL_ID> output_dir/
from mobius import build

model = build("<MODEL_ID>")