Deraining and Desnowing Algorithm on Adaptive Tolerance and Dual-tree Complex Wavelet Fusion.

Jingfeng Zang; Ningxue Xu; Rui Liu; Yuhuan Shi

doi:10.9781/ijimai.2020.11.014

Authors

Jingfeng Zang Changchun University of Science and Technology
Ningxue Xu Changchun University of Science and Technology
Rui Liu Changchun University of Science and Technology
Yuhuan Shi Changchun University of Science and Technology

DOI:

https://doi.org/10.9781/ijimai.2020.11.014

Keywords:

Wavelet, Adaptive Tolerance, Dual-tree Complex Wavelet Fusion, Deraining, Desnowing, Video Enhancement

Supporting Agencies

This work was financially supported by the Scientific and Technological Development Scheme of Jilin Province (No.20170312021ZX).

Abstract

Severe weather conditions such as rain and snow often reduce the visual perception quality of the video image system, the traditional methods of deraining and desnowing usually rarely consider adaptive parameters. In order to enhance the effect of video deraining and desnowing, this paper proposes a video deraining and desnowing algorithm based on adaptive tolerance and dual-tree complex wavelet. This algorithm can be widely used in security surveillance, military defense, biological monitoring, remote sensing and other fields. First, this paper introduces the main work of the adaptive tolerance method for the video of dynamic scenes. Second, the algorithm of dual-tree complex wavelet fusion is analyzed and introduced. Using principal component analysis fusion rules to process low-frequency sub-bands, the fusion rule of local energy matching is used to process the high-frequency sub-bands. Finally, this paper used various rain and snow videos to verify the validity and superiority of image reconstruction. Experimental results show that the algorithm has achieved good results in improving the image clarity and restoring the image details obscured by raindrops and snows.

Downloads

Download data is not yet available.

References

[1] H. Hase, K. Miyake and M. Yoneda, “Real-time snowfall noise elimination,” Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348), Kobe, vol. 2, pp. 406-409, 1999, doi:10.1109/ICIP.1999.822927.

[2] K. Garg and S. K. Nayar, “Detection and removal of rain from videos,” Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, Washington, DC, USA, pp. I-I, 2004, doi:10.1109 / CVPR.2004.1315077.

[3] K. Garg and S. K. Nayar, “Vision and Rain,” International Journal of Computer Vision 75, pp. 3–27, 2007, doi:10.1007/s11263-006-0028-6.

[4] X. Zhang, H. Li, Y. Qi, W. K. Leow and T. K. Ng, “Rain Removal in Video by Combining Temporal and Chromatic Properties,” 2006 IEEE International Conference on Multimedia and Expo, Toronto, pp. 461-464, 2006, doi:10.1109/ICME.2006.262572.

[5] N. Brewer, N. Liu, “Using the shape characteristics of rain to identify and remove rain from video,” Lecture Notes in Computer Science, vol. 5342, pp. 451-458, 2008, doi:10.1007/978-3-540-89689-0_49.

[6] X. Zhao, P. Liu, J. Liu, X. Tang, “The application of histogram on rain detection in video,” Proceedings of the 11th Joint Conference on Information Science, pp. 382-387, 2008, doi:10.2991/jcis.2008.65.

[7] P. Barnum, T. Kanade, & S. Narasimhan, “Spatio-temporal frequency analysis for removing rain and snow from videos,” The First International Workshop on Photometric Analysis for Computer Vision, 1-17 (2008).

[8] P. Barnum, S. Narasimhan, & T. Kanade, “Analysis of Rain and Snow in Frequency Space,” International Journal of Computer Vision 86, pp. 256, 2010, doi:10.1007/s11263-008-0200-2.

[9] J. Bossu, N. Hautière, & J. Tarel, “Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks,” International Journal of Computer Vision 93, pp. 348–367, 2011, doi: 10.1007/s11263-011-0421-7.

[10] L. Kang, C. Lin and Y. Fu, “Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition,” in IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 1742-1755, 2012, doi:10.1109/ TIP.2011.2179057.

[11] L. Kang, C. Lin and Y. Lin, “Self-learning-based rain streak removal for image/video,” 2012 IEEE International Symposium on Circuits and Systems (ISCAS), Seoul, pp. 1871-1874, 2012, doi:10.1109/ISCAS.2012.6271635.

[12] L. Yu, Y. Piao, X. Yan, “Video Defogging Based on Adaptive Tolerance,” TELKOMNIKA: Indonesian Journal of Electrical Engineering, vol. 10, pp. 1644-1654, 2012, doi:10.11591/telkomnika.v10i7.1556.

[13] A. E. Abdel-Hakim, “A Novel Approach for Rain Removal from Videos Using Low-Rank Recovery,” 2014 5th International Conference onIntelligent Systems, Modelling and Simulation, Langkawi, pp. 351-356, 2014, doi:10.1109 / ISMS.2014.161.

[14] J. Kim, J. Sim and C. Kim, “Video Deraining and Desnowing Using Temporal Correlation and Low-Rank Matrix Completion,” in IEEE Transactions on Image Processing, vol. 24, no. 9, pp. 2658-2670, 2015, doi:10.1109/TIP.2015.2428933.

[15] W. Ren, J. Tian, Z. Han, A. Chan and Y. Tang, “Video Desnowing and Deraining Based on Matrix Decomposition,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 2838- 2847, 2017, doi:10.1109/CVPR.2017.303.

[16] X. Zheng, Y. Liao, W. Guo, X. Fu, X. Ding, “Single-Image-Based rain and snow removal using multi-guided filter,” 20th International Conference on Neural Information Processing (ICONIP). Springer Berlin Heidelberg, vol. 8228, pp. 258-265, 2013, doi:10.1007/978-3-642-42051-1_33.

[17] Z. Wang, “Method of Removing Rain (Snow) from Video Images Based on Wavelet Fusion,” Journal of BeiHua University (Natural Science), vol. 19, no. 1, pp. 136-140, 2018.

[18] S. G. Narasimhan and S. K. Nayar, “Contrast restoration of weather degraded images,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 6, pp. 713-724, June 2003 doi:10.1109/TPAMI.2003.1201821.

[19] K. He, J. Sun, X. Tang, “Guided image filtering,” European Conference on Computer Vision 2010, Part I, Lecture Notes in Computer Science, vol. 6311, pp. 1-14, doi:10.1007/978-3-642-15549-9_1.

[20] L. Dou, D. Xu, J. Li, “Image Inpainting Based on Dual-tree Complex Wavelet Transform,” Computer Science, vol. 44, no. 6, pp. 180-191, 2017.

[21] J. Du, S. Chen, “Adaptive PCNN Image Fusion Algorithm Based on Double Tree Complex Wavelet Transform,” Infrared Technology, vol. 40, no. 10, pp. 1002-1007, 2018.

[22] W. Yang, Q. Chai, L. Wang, “Multi-focus image fusion method based on double-tree complex wavelet transform,” Electro-Optic Technology Application, vol. 24, no. 3, pp. 59-62, 2009.

[23] F. Yang, L. Zheng, “New feature extraction algorithm based on SIFT,” Applied Science and Technology, vol. 1, no. 1, pp. 1-5, 2018.

[24] Y. Xiang, H. Qin, J. Li et al., “Multi-focus image fusion using a guidedfilter-based difference image,” Applied Optics, vol. 55, no. 9, pp. 2230-2239, 2016, doi:10.1364/AO.55.002230.

[25] J. Li, Y. Yang, “An Improved MLESAC Algorithm for Estimating Fundamental Matrix,” Computer Engineering, vol. 38, no. 19, pp. 215-217, 2012.

[26] M. R. Metwalli, A. H. Nasr, O. S. Farag Allah and S. El-Rabaie, “Image fusion based on principal component analysis and high-pass filter,” 2009 International Conference on Computer Engineering & Systems, Cairo, pp. 63-70, 2009, doi:10.1109/ICCES.2009.5383308.

[27] S. Lu, L. Zou, X. Shen, W. Wu, W. Zhang, “Multi-spectral remote sensing image enhancement method based on PCA and IHS transformations,” Journal of Zhejiang University-SCIENCE A, vol. 12, pp. 453–460, 2011, doi:10.1631/jzus.A1000282.

[28] X. Wang, Y. Zhou, “Infrared and visible fusion in undecimated dual-tree complex wavelet domain,” Computer Engineering and Design, vol. 38, no.3, pp. 730-734, 2017.

[29] F. Wu, H. Li, “Image Fusion Algorithm Based on NSCT and PCA,” Aeronautical computing technique, vol. 45, no. 3, pp. 48-50, 2015, doi:10.12968/prtu.2015.45.48.

[30] J. Zang, G. Ren, J. Dong, Y. Piao, S. Jim, “Removal of rain video based on temporal intensity and chromatic constraint of raindrops,” Evolutionary Intelligence, vol. 12, pp. 349–355, 2019, doi:10.1007/s12065-018-0185-x.

[31] J. Zang, G. Ren, Y. An, Y. Piao, “Removal of rain from video based on dualtree complex wavelet fusion,” Journal of Intelligent & Fuzzy Systems, vol. 38, no. 1, pp. 105-113, 2020, doi:10.3233/JIFS-179385.