Guided Super-Resolution for Image Fusion: A Novel Approach to Enhancing Crack Segmentation in Masonry Structures

In the field of crack segmentation, red-green-blue (RGB) and infrared (IR) images acquired with different sensors are often fused to improve segmentation accuracy. In existing studies, this fusion is often achieved by downsampling high-resolution RGB images to match the resolution of low-resolution IR images. This downsampling process, however, results in coarser image detail and is likely to reduce the overall accuracy of crack segmentation; therefore, there is potential to enhance crack segmentation accuracy by increasing the spatial resolution of IR images to that of high-resolution RGB images. This study investigates the potential enhancement of crack segmentation using super-resolution techniques, previously unexplored in the domain of crack segmentation, using fused images. Our investigation starts with an exploration of traditional super-resolution techniques, including interpolation and deep-learning-based methods, involving only individual IR images. Additionally, we propose a novel RGB-guided super-resolution method where a high-resolution RGB image is employed through deep-learning networks to guide the reconstruction of high-frequency information in the corresponding IR image of the same scene. Both the downsampling method adopted in current practice and the super-resolution methods explored in this study are tested on the Crack900 and CrackAP400 datasets using seven commonly used crack segmentation networks. Results indicate that super-resolution methods significantly improve crack segmentation accuracy over the downsampling approach. Our proposed RGB-guided super-resolution achieves higher segmentation accuracy across all super-resolution methods considered. A set of ablation experiments is also carried out to explore the effectiveness of each component in the RGB-guided super-resolution method, the results of which prove its superiority.