BDFM: Foundation Model for Segmentation and Classification Tasks of Brain Diseases

Objective: The lack of high-quality annotated images and the limited transferability of task-specific models hamper the practical of AI-assisted diagnosis for brain diseases. Developing self-supervised foundation model is a promising solution to address this problem. Methods:
We establish a masked image modeling (MIM)-based foundation model of brain disease (BDFM). We construct a database named BD-15k of more than ten brain diseases for pre-training. To improve the lesion feature extraction ability of BDFM, we propose a spatial-frequency dual-domain decoder, and introduce a spatial mean masking strategy to replace the traditional masking methods. Results: Results indicate that these improvements help BDFM outperform the baseline method in reconstructing lesion details. Extensive qualitative and quantitative experiments on three downstream tasks show that BDFM generalizes well to segmentation and classification tasks based on small annotated datasets. Conclusion: BDFM outperforms task-specific models trained from scratch while avoiding complex task-specific designs. Significance: This work contributes to the advancement of medical foundation models, paving the way for more effective brain disease analysis. The source code will be made publicly available upon publication in https://github.com/zzzjjj98/BDFM.