Entropy-Guided Distillation for Medical Image Segmentation Under Missing Modalities

While multimodal medical image segmentation improves accuracy via complementary information, real-world constraints often result in incomplete modality inputs, posing a major challenge to robust segmentation. This work addresses the most constrained single-modality setting via a UNet-based distillation framework, which prunes skip connections in the teacher network and adaptively modulates distillation strength to guide compact, informative student network representations. First, entropy-based pruning is applied to skip connections to reduce low-level redundancy and promote semantic abstraction in the teacher network. This enhances the bottleneck and retained skip connection, yielding more informative features for effective distillation. Second, an entropy- and depth-aware temperature schedule is introduced to adaptively control distillation strength across critical semantic routes. Such modulation guides the student to focus on informative signals, enhancing representation under limited capacity. Experimental results on benchmark medical imaging datasets demonstrate that our method outperforms existing single-modality approaches.