Abstract

Visible and infrared image fusion is one of the most crucial tasks in the field of image fusion, aiming to generate fused images with clear structural information and high-quality texture features for high-level vision tasks. However, when faced with severe illumination degradation in visible images, the fusion results of existing image fusion methods often exhibit blurry and dim visual effects, posing major challenges for autonomous driving. To this end, a Darkness-Free network is proposed to handle visible and infrared image disentanglement and fusion all at once (DFVO), which employs a cascaded multitask approach to replace the traditional two-stage cascaded training (enhancement and fusion), addressing the issue of information entropy loss caused by hierarchical data transmission. Specifically, we construct a latent-common feature extractor (LCFE) to obtain latent features for the cascaded tasks strategy. First, a details-extraction module (DEM) is devised to acquire high-frequency semantic information. Second, we design a hypercross-attention module (HCAM) to extract low-frequency information and preserve texture features from source images. Finally, a relevant loss function is designed to guide holistic network learning, thereby achieving better image fusion. Extensive experiments demonstrate that our proposed approach outperforms state-of-the-art alternatives in terms of qualitative and quantitative evaluations. Particularly, DFVO can generate clearer, more informative, and more evenly illuminated fusion results in dark environments, achieving the best performance on the LLVIP dataset with 63.258-dB PSNR and 0.724 CC, providing more effective information for high-level vision tasks. Our code is publicly accessible at https://github.com/DaVin-Qi530/DFVO

Framework

Experiment

Conclusion

In this paper, we propose a novel holistic visible and infrared image fusion network, which achieves learning of illumination enhancement and image fusion simultaneously. Through qualitative and quantitative experiments comparing with SOTA fusion methods, we validate the advantages of our approach in visual scene perception and fused image clarity. Furthermore, the performance in object detection and pedestrian recognition tasks also demonstrates the potential of our method in high-level computer vision applications.