TY - JOUR
T1 - Guided Super-Resolution for Image Fusion
T2 - A Novel Approach to Enhancing Crack Segmentation in Masonry Structures
AU - Fang, Yuan
AU - Fan, Lei
AU - Cai, Yuanzhi
N1 - Publisher Copyright:
© 2001-2012 IEEE.
PY - 2025
Y1 - 2025
N2 - In the field of crack segmentation, red-green-blue (RGB) and infrared (IR) images acquired with different sensors are often fused to improve segmentation accuracy. In existing studies, this fusion is often achieved by downsampling high-resolution RGB images to match the resolution of low-resolution IR images. This downsampling process, however, results in coarser image detail and is likely to reduce the overall accuracy of crack segmentation; therefore, there is potential to enhance crack segmentation accuracy by increasing the spatial resolution of IR images to that of high-resolution RGB images. This study investigates the potential enhancement of crack segmentation using super-resolution techniques, previously unexplored in the domain of crack segmentation, using fused images. Our investigation starts with an exploration of traditional super-resolution techniques, including interpolation and deep-learning-based methods, involving only individual IR images. Additionally, we propose a novel RGB-guided super-resolution method where a high-resolution RGB image is employed through deep-learning networks to guide the reconstruction of high-frequency information in the corresponding IR image of the same scene. Both the downsampling method adopted in current practice and the super-resolution methods explored in this study are tested on the Crack900 and CrackAP400 datasets using seven commonly used crack segmentation networks. Results indicate that super-resolution methods significantly improve crack segmentation accuracy over the downsampling approach. Our proposed RGB-guided super-resolution achieves higher segmentation accuracy across all super-resolution methods considered. A set of ablation experiments is also carried out to explore the effectiveness of each component in the RGB-guided super-resolution method, the results of which prove its superiority.
AB - In the field of crack segmentation, red-green-blue (RGB) and infrared (IR) images acquired with different sensors are often fused to improve segmentation accuracy. In existing studies, this fusion is often achieved by downsampling high-resolution RGB images to match the resolution of low-resolution IR images. This downsampling process, however, results in coarser image detail and is likely to reduce the overall accuracy of crack segmentation; therefore, there is potential to enhance crack segmentation accuracy by increasing the spatial resolution of IR images to that of high-resolution RGB images. This study investigates the potential enhancement of crack segmentation using super-resolution techniques, previously unexplored in the domain of crack segmentation, using fused images. Our investigation starts with an exploration of traditional super-resolution techniques, including interpolation and deep-learning-based methods, involving only individual IR images. Additionally, we propose a novel RGB-guided super-resolution method where a high-resolution RGB image is employed through deep-learning networks to guide the reconstruction of high-frequency information in the corresponding IR image of the same scene. Both the downsampling method adopted in current practice and the super-resolution methods explored in this study are tested on the Crack900 and CrackAP400 datasets using seven commonly used crack segmentation networks. Results indicate that super-resolution methods significantly improve crack segmentation accuracy over the downsampling approach. Our proposed RGB-guided super-resolution achieves higher segmentation accuracy across all super-resolution methods considered. A set of ablation experiments is also carried out to explore the effectiveness of each component in the RGB-guided super-resolution method, the results of which prove its superiority.
KW - Crack segmentation
KW - deep learning
KW - infrared (IR) and visible image fusion
KW - masonry structure
KW - semantic segmentation
KW - super-resolution
UR - http://www.scopus.com/inward/record.url?scp=105002679477&partnerID=8YFLogxK
U2 - 10.1109/JSEN.2025.3540119
DO - 10.1109/JSEN.2025.3540119
M3 - Article
AN - SCOPUS:105002679477
SN - 1530-437X
VL - 25
SP - 11491
EP - 11507
JO - IEEE Sensors Journal
JF - IEEE Sensors Journal
IS - 7
ER -