Guided Super-Resolution for Image Fusion: A Novel Approach to Enhancing Crack Segmentation in Masonry Structures

In the field of crack segmentation, red-green-blue (RGB) and infrared (IR) images acquired with different sensors are often fused to improve segmentation accuracy. In existing studies, this fusion is often achieved by downsampling high-resolution RGB images to match the resolution of low-resolution IR images. However, this downsampling process results in coarser image detail and is likely to reduce the overall accuracy of crack segmentation. Therefore, there is potential to enhance crack segmentation accuracy by increasing the spatial resolution of IR images to that of high-resolution RGB images. This study investigates the potential enhancement of crack segmentation using super-resolution techniques, previously unexplored in the domain of crack segmentation using fused images. Our investigation starts with an exploration of traditional super-resolution techniques, including interpolation and deep learning-based methods, involving only individual IR images. Additionally, we propose a novel RGB-guided super-resolution method where a high-resolution RGB image is employed through deep learning networks to guide the reconstruction of high-frequency information in the corresponding IR image of the same scene. Both the downsampling method adopted in current practice and the super-resolution methods explored in this study are tested on the Crack900 and CrackAP400 dataset, using seven commonly used crack segmentation networks. Results indicate that super-resolution methods significantly improve crack segmentation accuracy over the downsampling approach. Our proposed RGB-guided super-resolution achieves higher segmentation accuracy across all super-resolution methods considered. A set of ablation experiments is also carried out to explore the effectiveness of each component in the RGB-guided super-resolution method, the results of which prove its superiority.