基于运动偏移信息估计的图像去运动模糊算法

郭政; 严伟; 吴志祥; 纪运景; 来建成; 王春勇; 李振华

doi:10.15892/j.cnki.djzdxb.2024.01.005

弹箭与制导学报 >

2024 , Vol. 44 >Issue 1: 25 - 33

DOI: https://doi.org/10.15892/j.cnki.djzdxb.2024.01.005

基于运动偏移信息估计的图像去运动模糊算法

郭政 ,
严伟 ,
吴志祥 ,
纪运景 ,
来建成 ,
王春勇 ,
李振华

展开

南京理工大学物理学院,江苏南京 210094

李振华(1964—),男,教授,博士,研究方向:激光探测与环境感知。

郭政(1999—),男,硕士研究生,研究方向:光学图像处理。

收稿日期: 2023-09-15

网络出版日期: 2024-12-28

基金资助

国家自然科学基金项目(61971225)

收起

Image Motion Blur Removal Algorithm Based on Motion Offset Information Estimation

GUO Zheng ,
YAN Wei ,
WU Zhixiang ,
JI Yunjing ,
LAI Jiancheng ,
WANG Chunyong ,
LI Zhenhua

Expand

School of Physics, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, China

Received date: 2023-09-15

Online published: 2024-12-28

Fold

摘要

针对现有动态场景图像去运动模糊算法难以有效复原非均匀复合运动模糊的问题,通过引入相机曝光轨迹来表示模糊图像中包含的运动信息,并据此提出了一种新的运动偏移估计框架,用于对潜在清晰图像在多个离散时间点的像素运动偏移进行建模。结合估计出的运动偏移信息,提出了一种融合像素运动偏移信息的多尺度单幅图像去模糊网络框架。该框架通过可变形卷积在解码阶段对运动偏移信息进行融合,给定每一像素点不同的运动约束。经过网络的多尺度编解码结构,得到每一尺度上每一像素点的预测值,实现了端对端的模糊图像复原。在GoPro和HIDE数据集上的实验结果表明,该算法能有效改善图像质量,峰值信噪比平均提升1.9 dB,结构相似度平均提升0.03。

关键词： 运动偏移估计; 运动模糊; 图像复原; 卷积神经网络

本文引用格式

郭政 , 严伟 , 吴志祥 , 纪运景 , 来建成 , 王春勇 , 李振华 . 基于运动偏移信息估计的图像去运动模糊算法[J]. 弹箭与制导学报, 2024 , 44(1) : 25 -33 . DOI: 10.15892/j.cnki.djzdxb.2024.01.005

Abstract

In response to the problem that existing motion blur removal algorithms for dynamic scene images are difficult to effectively restore non-uniform composite motion blur, a new motion offset estimation framework is proposed by introducing camera exposure trajectories to represent the motion information contained in blurred images. This framework is used to model pixel motion offsets of latent clear images at multiple discrete time points. A multi-scale single image deblurring network framework integrating pixel motion offset information is proposed based on the estimated motion offset information. This framework fuses motion offset information during the decoding stage through deformable convolution, giving each pixel different motion constraints. Through the multi-scale encoding and decoding structure of the network, the predicted values of each pixel on each scale are obtained, achieving end-to-end blur image restoration. The experimental results on GoPro and HIDE datasets show that this algorithm can effectively improve image quality, with an average increase of 1.9 dB in peak signal-to-noise ratio and 0.03 in structural similarity.

Key words： motion offset estimation; motion blur; image restoration; convolutional neural network

[an error occurred while processing this directive]

参考文献

原文顺序 | 文献年度倒序 | 文中引用次数倒序

[1]	徐振华. 基于结构先验与稀疏表示的盲图像去模糊方法研究[D]. 南京: 南京理工大学, 2023. XU Z H. Research on blind image deblurring method based on structural prior and sparse representation[D]. Nanjing: Nanjing University of Science and Technology, 2023.

[2]	CHAN T, WONG C. Total variation blind deconvolution[J]. IEEE Transactions on Image Processing, 1998, 7(3): 370-375. DOI PMID

[3]	BASCLE B, BLAKE A, ZISSERMAN A. Motion deblurring and super-resolution from an image sequence[C]// ECCV. Proceedings of the 4th European Conference on Computer Vision. Cambridge: Springer Science, 1996: 571-582.

[4]	NAYAR S K, BENEZRA M. Motion-based motion deblurring[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(6): 689-698.

[5]	NEELAMANI R, CHOI H, BARANIUK R. ForWaRD: fourier-wavelet regularized deconvolution for ill-conditioned systems[J]. IEEE Transactions on Signal Processing, 2004, 52(2): 418-433.

[6]	FERGUS R, SINGH B, HERTZMANN A, ROWEIS S, et al. Removing camera shake from a single photograph[J]. ACM Transactions on Graphics, 2006, 25(3): 787-794.

[7]	PAN J, SUN D, PFISTER H, et al. Blind image deblurring using dark channel prior[C]// IEEE. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 1628-1636.

[8]	YAN Y, REN W, GUO Y, et al. Image deblurring via extreme channels prior[C]// IEEE. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 6978-6986.

[9]	CHEN L, FANG F, WANG T, et al. Blind image deblurring with local maximum gradient prior[C]// IEEE. Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2019: 1743-1750.

[10]	SUN J, CAO W, XU Z, et al. Learning a convolutional neural network for non-uniform motion blur removal[C]// IEEE. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2015: 769-777.

[11]	GONG D, YANG J, LIU L, et al. From motion blur to motion flow: a deep learning solution for removing heterogeneous motion blur[C]// IEEE. Proceeding of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 2319-2328.

[12]	董星煜, 刘传奇, 赵健康. 融合运动信息的图像运动模糊去除算法[J]. 计算机应用研究, 2021, 38(1): 278-281. DONG X Y, LIU C Q, ZHAO J K. Image motion blur removal algorithm based on fusion of motion information[J]. Computer Application Research, 2021, 38(1): 278-281.

[13]	NAH S, KIM T, LEE K. Deep multi-scale convolutional neural network for dynamic scene deblurring[C]// IEEE. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 3883-3891.

[14]	JIN M, MEISHVILI G, FAVARO P. Learning to extract a video sequence from a single motion-blurred image[C]// IEEE. Proceedings of the 2018 IEEE/ Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018: 6334-6342.

[15]	PUROHIT K, SHAH A, RAJAGOPALAN A. Bringing alive blurred moments[C]// IEEE. Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2019: 6830-6839.

[16]	KUPYN O, MARTPNIUK T, WU J, et al. DeblurGAN-v2: deblurring (orders-of-magnitude) faster and better[C]// IEEE. Proceedings of the 2019 IEEE International Conference on Computer Vision. New York: IEEE, 2019: 8878-8887.

[17]	CHEN H, GU J, GALLO O, et al. Reblur2deblur: deblurring videos via self-supervised learning[C]// IEEE. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018: 1-9.

[18]	LIU P, JANAI J, POLLEFEYS M, et al. Self supervised linear motion deblurring[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 2475-2482.

[19]	ZHANG Y, WANG C, STEPHEN J, et al. Exposure trajectory recovery from motion blur[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(11): 7490-7504.

[20]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// IEEE. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 770-778

[21]	GAO H, TAO X, SHEN X, et al. Dynamic scene deblurring with parameter selective sharing and nested skip connections[C]// IEEE. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2019: 3843-3851.

[22]	CHEN L, LU X, ZHANG J, et al. HINet: half instance normalization network for image restoration[C]// IEEE. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE, 2021: 182-192

[23]	IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]// ICML. Proceedings of the 32nd International Conference on International Conference on Machine Learning. New York: ACM, 2015: 448-456.

[24]	TAO X, GAO H, SHEN X, et al. Scale-recurrent Network for deep image deblurring[C]// IEEE. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018: 8174-8182.

[25]	LAI W, HUANG J, AHUJA N, et al. Fast and accurate image super-resolution with deep laplacian pyramid networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11): 2599-2613.

[26]	CHO S, JI S, HONG J, et al. Rethinking coarse-to-fine approach in single image deblurring[C]// IEEE. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). New York: IEEE, 2021: 4621-4630.

[27]	SHEN Z, WANG W, LU X, et al. Human-aware motion deblurring[C]// IEEE. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). New York: IEEE, 2019: 5571-5580.

[28]	ZHANG H, DAI Y, LI H, et al. Deep stacked hierarchical multi-patch network for image deblurring[C]// IEEE. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2019: 5971-5979.

[29]	SUIN M, PUROHIT K, RAJAGOPALAN A. Spatially-attentive patch hierarchical network for adaptive motion deblurring[C]// IEEE. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2020: 3603-3612.

[30]	ZHANG K, LUO W, ZHONG Y, et al. Deblurring by realistic blurring[C]// IEEE. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020: 2734-2743.

Options

文章导航