Image Denoising Using Multi-Model Fusion Technique

Kamal Khan; Muhammad Anwar; Saifullah Khan

Kamal Khan
Muhammad Anwar
Saifullah Khan

Keywords: Image denoising, multimodal fusion, Gaussian noise, Convolutional Neural Network, PSNR, SSIM, MSE.

Abstract

Image denoising is a fundamental challenge in the field of image processing, with the primary goal of recovering high-quality images from noisy counterparts. This paper investigates the effectiveness of multimodal fusion techniques for denoising images. The study utilizes the Waterloo Exploration Database, a comprehensive collection of 4,744 pristine natural images, selecting 500 images for experimentation. Gaussian noise was artificially introduced to simulate realistic noise conditions, creating the noisy input for the denoising process. Multiple modalities—grayscale, edge, and depth images—were extracted from the noisy images to capture different aspects of the visual content. These modalities were aligned and combined using early fusion techniques, producing a single cohesive representation. A Convolutional Neural Network (CNN) was then trained using this fused data for image denoising. The evaluation focused on key metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Mean Squared Error (MSE). The results indicate that multimodal fusion significantly improves denoising performance, as evidenced by increased PSNR and reduced MSE, suggesting its potential to enhance image restoration methods.

References

[1] Buades, A., Coll, B., & Morel, J. M. (2005). A non-local algorithm for image denoising. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, 60-65.
[2] Liu, X., & Huang, L. (2014). A new nonlocal total variation regularization algorithm for image denoising. Mathematics and Computers in Simulation, 97, 224-233.
[3] Donoho, D. L. (1995). De-noising by soft-thresholding. IEEE Transactions on Information Theory, 41(3), 613-627.
[4] Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017). Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142-3155.
[5] Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., & Aila, T. (2018). Noise2Noise: Learning image restoration without clean data. Proceedings of the 35th International Conference on Machine Learning (ICML), 2018, 2965-2974.
[6] Du, J., Li, W., Lu, K., & Xiao, B. (2016). An overview of multi-modal medical image fusion. Neurocomputing, 215, 3-20.
[7] Sun, P., Shi, Y., Li, L., Wang, G., Guan, J., Qian, F., & Yu, Y. (2020, March). Multi-scale wavelet thresholding denoising algorithm of Raman spectrum. In 2019 International Conference on Optical Instruments and Technology: Optoelectronic Imaging/Spectroscopy and Signal Processing Technology (Vol. 11438, pp. 154-160). SPIE.
[8] Aharon, M., Elad, M., & Bruckstein, A. (2006). K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 54(11), 4311-4322.
[9] Zheng, H., Yong, H., & Zhang, L. (2021). Deep convolutional dictionary learning for image denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 630-641).
[10] Pires, R. G., Santos, D. F., Santos, C. F., Santana, M. C., & Papa, J. P. (2020, November). Image denoising using attention-residual convolutional neural networks. In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (pp. 101-107). Ieee.
[11] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xiaodong, S., & Warde-Farley, D. (2014). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 27.
[12] Zhong, Y., Liu, L., Zhao, D., & Li, H. (2020). A generative adversarial network for image denoising. Multimedia Tools and Applications, 79, 16517-16529.
[13] Zhuang, L., Ng, M. K., Gao, L., Michalski, J., & Wang, Z. (2023). Eigenimage2Eigenimage (E2E): A self-supervised deep learning network for hyperspectral image denoising. IEEE Transactions on Neural Networks and Learning Systems.
[14] Liu, P., Zhang, H., Lian, W., & Zuo, W. (2019). Multi-level wavelet convolutional neural networks. IEEE Access, 7, 74973-74985.
[15] Li, X., & Zhao, J. (2021). A novel multi-modal medical image fusion algorithm. Journal of Ambient Intelligence and Humanized Computing, 12(2), 1995-2002.
[16] Jebur, R. S., Zabil, M. H. B. M., Hammood, D. A., Cheng, L. K., & Al-Naji, A. (2023). Image denoising using hybrid deep learning approach and Self-Improved Orca Predation Algorithm. Technologies, 11(4), 111.