Image Motion Blur Removal Algorithm Based on Motion Offset Information Estimation

GUO Zheng; YAN Wei; WU Zhixiang; JI Yunjing; LAI Jiancheng; WANG Chunyong; LI Zhenhua

doi:10.15892/j.cnki.djzdxb.2024.01.005

Journal of Projectiles, Rockets, Missiles and Guidance >

2024 , Vol. 44 >Issue 1: 25 - 33

DOI: https://doi.org/10.15892/j.cnki.djzdxb.2024.01.005

Image Motion Blur Removal Algorithm Based on Motion Offset Information Estimation

GUO Zheng ,
YAN Wei ,
WU Zhixiang ,
JI Yunjing ,
LAI Jiancheng ,
WANG Chunyong ,
LI Zhenhua

Expand

School of Physics, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, China

Received date: 2023-09-15

Online published: 2024-12-28

Fold

Abstract

In response to the problem that existing motion blur removal algorithms for dynamic scene images are difficult to effectively restore non-uniform composite motion blur, a new motion offset estimation framework is proposed by introducing camera exposure trajectories to represent the motion information contained in blurred images. This framework is used to model pixel motion offsets of latent clear images at multiple discrete time points. A multi-scale single image deblurring network framework integrating pixel motion offset information is proposed based on the estimated motion offset information. This framework fuses motion offset information during the decoding stage through deformable convolution, giving each pixel different motion constraints. Through the multi-scale encoding and decoding structure of the network, the predicted values of each pixel on each scale are obtained, achieving end-to-end blur image restoration. The experimental results on GoPro and HIDE datasets show that this algorithm can effectively improve image quality, with an average increase of 1.9 dB in peak signal-to-noise ratio and 0.03 in structural similarity.

Key words： motion offset estimation; motion blur; image restoration; convolutional neural network

Cite this article

GUO Zheng , YAN Wei , WU Zhixiang , JI Yunjing , LAI Jiancheng , WANG Chunyong , LI Zhenhua . Image Motion Blur Removal Algorithm Based on Motion Offset Information Estimation[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2024 , 44(1) : 25 -33 . DOI: 10.15892/j.cnki.djzdxb.2024.01.005

[an error occurred while processing this directive]

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	徐振华. 基于结构先验与稀疏表示的盲图像去模糊方法研究[D]. 南京: 南京理工大学, 2023. XU Z H. Research on blind image deblurring method based on structural prior and sparse representation[D]. Nanjing: Nanjing University of Science and Technology, 2023.

[2]	CHAN T, WONG C. Total variation blind deconvolution[J]. IEEE Transactions on Image Processing, 1998, 7(3): 370-375. DOI PMID

[3]	BASCLE B, BLAKE A, ZISSERMAN A. Motion deblurring and super-resolution from an image sequence[C]// ECCV. Proceedings of the 4th European Conference on Computer Vision. Cambridge: Springer Science, 1996: 571-582.

[4]	NAYAR S K, BENEZRA M. Motion-based motion deblurring[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(6): 689-698.

[5]	NEELAMANI R, CHOI H, BARANIUK R. ForWaRD: fourier-wavelet regularized deconvolution for ill-conditioned systems[J]. IEEE Transactions on Signal Processing, 2004, 52(2): 418-433.

[6]	FERGUS R, SINGH B, HERTZMANN A, ROWEIS S, et al. Removing camera shake from a single photograph[J]. ACM Transactions on Graphics, 2006, 25(3): 787-794.

[7]	PAN J, SUN D, PFISTER H, et al. Blind image deblurring using dark channel prior[C]// IEEE. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 1628-1636.

[8]	YAN Y, REN W, GUO Y, et al. Image deblurring via extreme channels prior[C]// IEEE. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 6978-6986.

[9]	CHEN L, FANG F, WANG T, et al. Blind image deblurring with local maximum gradient prior[C]// IEEE. Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2019: 1743-1750.

[10]	SUN J, CAO W, XU Z, et al. Learning a convolutional neural network for non-uniform motion blur removal[C]// IEEE. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2015: 769-777.

[11]	GONG D, YANG J, LIU L, et al. From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur[C]// IEEE. Proceeding of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 2319-2328.

[12]	董星煜, 刘传奇, 赵健康. 融合运动信息的图像运动模糊去除算法[J]. 计算机应用研究, 2021, 38(1): 278-281. DONG X Y, LIU C Q, ZHAO J K. Image motion blur removal algorithm based on fusion of motion information[J]. Computer Application Research, 2021, 38(1): 278-281.

[13]	NAH S, KIM T, LEE K. Deep multi-scale convolutional neural network for dynamic scene deblurring[C]// IEEE. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 3883-3891.

[14]	JIN M, MEISHVILI G, FAVARO P. Learning to extract a video sequence from a single motion-blurred image[C]// IEEE. Proceedings of the 2018 IEEE/ Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018: 6334-6342.

[15]	PUROHIT K, SHAH A, RAJAGOPALAN A. Bringing alive blurred moments[C]// IEEE. Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2019: 6830-6839.

[16]	KUPYN O, MARTPNIUK T, WU J, et al. DeblurGAN-v2: deblurring (orders-of-magnitude) faster and better[C]// IEEE. Proceedings of the 2019 IEEE International Conference on Computer Vision. New York: IEEE, 2019: 8878-8887.

[17]	CHEN H, GU J, GALLO O, et al. Reblur2deblur: deblurring videos via self-supervised learning[C]// IEEE. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018: 1-9.

[18]	LIU P, JANAI J, POLLEFEYS M, et al. Self supervised linear motion deblurring[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 2475-2482.

[19]	ZHANG Y, WANG C, STEPHEN J, et al. Exposure trajectory recovery from motion blur[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(11): 7490-7504.

[20]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// IEEE. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 770-778

[21]	GAO H, TAO X, SHEN X, et al. Dynamic scene deblurring with parameter selective sharing and nested skip connections[C]// IEEE. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2019: 3843-3851.

[22]	CHEN L, LU X, ZHANG J, et al. HINet: half instance normalization network for image restoration[C]// IEEE. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE, 2021: 182-192

[23]	IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]// ICML. Proceedings of the 32nd International Conference on International Conference on Machine Learning. New York: ACM, 2015: 448-456.

[24]	TAO X, GAO H, SHEN X, et al. Scale-recurrent Network for deep image deblurring[C]// IEEE. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018: 8174-8182.

[25]	LAI W, HUANG J, AHUJA N, et al. Fast and accurate image super-resolution with deep laplacian pyramid networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11): 2599-2613.

[26]	CHO S, JI S, HONG J, et al. Rethinking coarse-to-fine approach in single image deblurring[C]// IEEE. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). New York: IEEE, 2021: 4621-4630.

[27]	SHEN Z, WANG W, LU X, et al. Human-aware motion deblurring[C]// IEEE. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). New York: IEEE, 2019: 5571-5580.

[28]	ZHANG H, DAI Y, LI H, et al. Deep stacked hierarchical multi-patch network for image deblurring[C]// IEEE. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2019: 5971-5979.

[29]	SUIN M, PUROHIT K, RAJAGOPALAN A. Spatially-attentive patch hierarchical network for adaptive motion deblurring[C]// IEEE. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2020: 3603-3612.

[30]	ZHANG K, LUO W, ZHONG Y, et al. Deblurring by realistic blurring[C]// IEEE. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 2734-2743.

Options

Outlines