Would it be possible to load the model with pretrained weights as a feature extractor, so without any classification/segmentation head?

What would be the syntax to load the USFM as a feature extractor?