Counterfactual Contrastive Learning for Fine Grained Image Classification

In the realm of fine-grained image classification, discerning subtle distinctions between closely related categories remains a challenge. However, these approaches typically fall short in addressing the deeper causal relationships that underlie the visible features, leading to potential biases and limited generalizability. This paper presents a fine-grained causal contrastive network (FCCN), a novel architecture that integrates causal inference with contrastive learning to address the intricacies of fine-grained visual classification. Unlike traditional approaches that predominantly rely on feature correlations, FCCN deeply studies the causal relationship between salient features and labels, which greatly enhances the model’s ability to discriminate fine-grained images. A key innovation of FCCN is the introduction of a backdoor adjustment technique for feature decoupling, effectively minimizing the impact of irrelevant context feature and purifying the feature space for more precise classification. We validate our model on CUB-200-2011, Stanford Cars, and WM-811K datasets. Both accuracy and robustness are significantly improved, which demonstrate notable improvements in accuracy and robustness.