CAF: CrossAttentionFusion for Unsupervised Brain MRI Segmentation

Unsupervised anomaly segmentation is vital for brain MRI lesion detection, enabling models trained solely on healthy data to identify tumors as deviations from normative anatomy without requiring annotated abnormal samples. Existing reconstruction-based approaches, such as autoencoders and diffusion models, often struggle with inadequate feature integration, where simple concatenation of spatial masks and frequency components fails to capture cross-domain dependencies, leading to reconstruction deviations in normal regions and high false positive rates. We propose a UNet-based diffusion framework enhanced with a Cross-Attention Fusion (CAF) module that dynamically aligns spatial mask features with frequency-derived structural priors. By leveraging multi-head attention with residual refinement, CAF adaptively integrates high-frequency edge information with spatial semantics, strengthening anomaly localization. An iterative mask refinement strategy further suppresses false positives and sharpens lesion boundaries. Extensive experiments on the BraTS 2021 dataset demonstrate that our method achieves Dice scores of 0.60-0.82, surpassing diffusion and IterMask-based baselines by 1.3 - 3.0%, while effectively reducing false positives in healthy tissues. These results highlight the value of dynamic domain fusion for unsupervised medical image analysis and suggest promising extensions to broader anomaly detection tasks.