Optimal Adaptation Depth for Medical Image Classification: Comparing Convolutional and Transformer Architectures Across Fine-Tuning Strategies

Nicolo Pecco1,2,3, Amirali Khosravi3, Sahika Betul Yayli3, Tubo Shi3, Elham Mahmoudi3, Allison Scarbrough3, Antonella Castellano1,2, Andrea Falini1,2, Lorenzo Veronese4, Pasquale Anthony Della Rosa2, Bradley Erickson3

  1. Vita-Salute San Raffaele University, Milan, Italy
  2. Department of Neuroradiology and CERMAC IRCCS San Raffaele Hospital, Milan, Italy
  3. Department of Radiology, Mayo Clinic, Rochester, MN, USA
  4. Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy

Table: Mean test metrics (Acc, AUC, F1) across fine-tuning strategies by dataset. Values are aggregated for the selected backbone; top-performing strategies within each dataset-metric column are highlighted.

Strategy Performance

Figure: Dataset-wise trends across fine-tuning strategies for DenseNet-121 and Swin-Tiny. Points show mean test performance for each strategy, with datasets encoded by color. Backbone is encoded by mark style (DenseNet-121: circles with solid lines; Swin-Tiny: squares with dashed lines).

Model Weights

Direct download links for each dataset and backbone from the Hugging Face repository.

Dataset Backbone Weights
bloodmnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
breastmnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
chestmnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
dermamnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
octmnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
organamnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
organcmnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
organsmnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
pathmnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
pneumoniamnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
retinamnist DenseNet-121 Open Weights
Swin-Tiny Open Weights
tissuemnist DenseNet-121 Open Weights
Swin-Tiny Open Weights