clip_vision_model

Model type

clip_vision_model

Class

CLIPVisionModel

Task

image-classification

Source

models/clip.py

Description

CLIP vision model for standalone image feature extraction.

By default this outputs the final last_hidden_state (all encoder layers followed by post_layernorm).

Two options support multimodal feature extraction used by models such as Phi-3.5-Vision:

  • feature_layer selects an intermediate hidden_states index (HuggingFace convention, e.g. -2). When set, only the encoder layers needed to reach that hidden state are instantiated and run, and the final post_layernorm is skipped (it is only applied to the last hidden state in HuggingFace).

  • drop_class_token removes the leading CLS token from the output so that only the patch features remain (HuggingFace img_feature[:, 1:]).

Usage

mobius build --model <MODEL_ID> output_dir/

from mobius import build

model = build("<MODEL_ID>")