CN118822905B - Multi-directional deblurring method and device based on flipped deformable convolution - Google Patents
Multi-directional deblurring method and device based on flipped deformable convolution Download PDFInfo
- Publication number
- CN118822905B CN118822905B CN202411303544.5A CN202411303544A CN118822905B CN 118822905 B CN118822905 B CN 118822905B CN 202411303544 A CN202411303544 A CN 202411303544A CN 118822905 B CN118822905 B CN 118822905B
- Authority
- CN
- China
- Prior art keywords
- layer
- convolution
- image
- frequency domain
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 230000007306 turnover Effects 0.000 claims abstract description 36
- 238000012545 processing Methods 0.000 claims description 51
- 238000010606 normalization Methods 0.000 claims description 31
- 230000000295 complement effect Effects 0.000 claims description 19
- 230000007246 mechanism Effects 0.000 claims description 16
- 238000003475 lamination Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 230000006835 compression Effects 0.000 claims description 9
- 238000007906 compression Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 7
- 238000013461 design Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 11
- JEJGUIDNYBAPGN-UHFFFAOYSA-N methylenedioxydimethylamphetamine Chemical compound CN(C)C(C)CC1=CC=C2OCOC2=C1 JEJGUIDNYBAPGN-UHFFFAOYSA-N 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000013507 mapping Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of computer vision, and provides a multidirectional deblurring method and device based on turnover deformable convolution, which comprises the steps of firstly obtaining an image to be processed; and then deblurring the image to be processed by using a turnover deformation network to obtain a deblurred image of the image to be processed. The turnover deformation network adopted by the method comprises an encoder, a first decoder, a second decoder and an output module, wherein the encoder and the decoder adopt asymmetric designs, and the effect of decoupling fuzzy-clear information can be achieved. The multi-directional deformable convolution block in the first decoder is overturned to obtain the second decoder, so that the characteristic of deformable convolution can be utilized to obtain the multi-directional fuzzy characteristics, the overturned and emphasized fuzzy information in the horizontal and vertical directions can be utilized to effectively remove the motion blur in the multi-directions, and the deblurring performance of the overturned deformation network can be improved.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a multidirectional deblurring method and device based on turnover deformable convolution.
Background
Images are an important way for people to obtain information. However, during the imaging process, the obtained image is often blurred due to the influence of adverse factors such as camera shake, movement of objects in the scene, camera defocus and the like, which greatly hinders the normal use and subsequent processing of the image.
Image deblurring is a fundamental image restoration problem in image processing and computer vision, and the objective is to restore a blurred image caused by motion factors such as camera shake, movement of an object, and the like, to a clear image. Most of the existing image deblurring methods generally implement an image restoration process by inputting a blurred image into a neural network model and outputting a clear image from the neural network model.
However, conventional neural network models are often not good at fully recovering the sharp details of the image when dealing with image blurring, thereby affecting the recovery quality of the final image.
Disclosure of Invention
The invention provides a multidirectional deblurring method and device based on turnover deformable convolution, which are used for solving the defects in the prior art.
The invention provides a multidirectional defuzzification method based on turnover deformable convolution, which comprises the following steps:
acquiring an image to be processed;
deblurring the image to be processed by using a turnover deformation network to obtain a deblurred image of the image to be processed;
the turnover deformation network comprises an encoder, a first decoder, a second decoder and an output module, wherein the first decoder comprises a plurality of cascaded multidirectional deformable convolution blocks, and the second decoder is obtained by turnover of the multidirectional deformable convolution blocks in the first decoder by 90 degrees;
the encoder is used for receiving the image to be processed and encoding the image to be processed to obtain the image characteristics of the image to be processed;
The multi-direction deformable convolution block is used for processing the image characteristics based on a simple frequency domain channel attention mechanism and the multi-direction deformable convolution block to obtain multi-direction fuzzy characteristics;
The output module is used for recovering the channel number of the main shaft fuzzy residual image output by the first decoder and the channel number of the complementary residual image output by the second decoder to be the same as the channel number of the image to be processed, and superposing the main shaft fuzzy residual image, the complementary residual image and the image to be processed to obtain the deblurring image.
According to the multidirectional defuzzification method based on the overturn deformable convolution, the encoder comprises a first convolution layer and a plurality of frequency domain channel attention fuzzy processing blocks with different scales, which are sequentially connected, wherein the frequency domain channel attention fuzzy processing blocks comprise fuzzy processing blocks and simple frequency domain channel attention blocks;
the fuzzy processing block comprises a first layer normalization layer, a second convolution layer, a third convolution layer, a first multiplication layer, a Fourier transform layer, a first Hadamard lamination layer, an inverse Fourier transform layer, an activation layer and a first superposition layer which are sequentially connected;
the output end of the second convolution layer is also connected with the input end of the first multiplication layer;
the input end of the first superimposed layer is also connected with the input end of the first normalization layer, and the output end of the first superimposed layer is used for outputting the processing result of the fuzzy processing block;
the simple frequency domain channel attention block comprises a second layer normalization layer, a fourth convolution layer, a fifth convolution layer, a frequency domain channel attention module, a sixth convolution layer and a second multiplication layer which are sequentially connected, and the input end of the second multiplication layer is also connected with the input end of the second layer normalization layer.
According to the multidirectional defuzzification method based on the overturn deformable convolution, the frequency domain channel attention module comprises a channel dividing module, a frequency domain attention module, a first splicing layer, a seventh convolution layer and a third multiplication layer;
the input of the frequency domain channel attention module is subjected to the channel dividing module to obtain a plurality of channel components;
The frequency domain attention module is used for carrying out frequency domain attention calculation on the channel components to obtain a plurality of frequency components;
the plurality of frequency components are cascaded through the first splicing layer to obtain a compression vector;
And multiplying the compressed vector by the seventh convolution layer and then carrying out multiplication operation on the compressed vector and the input of the frequency domain channel attention module by the third multiplication layer to obtain the output of the frequency domain channel attention module.
According to the multidirectional defuzzification method based on the turnover deformable convolution, the multidirectional deformable convolution block comprises a fuzzy processing block, a simple frequency domain channel attention block and a multidirectional defuzzification module which are sequentially connected;
the multi-direction deblurring module comprises a first branch, a second branch and a second Hadamard layer, wherein the first branch comprises a first residual error module, three deformable convolution blocks and a second splicing layer;
the three deformable convolution blocks are connected between the first residual error module and the second splicing layer in parallel;
the second branch comprises an eighth convolution layer, a second residual error module, a third residual error module, a ninth convolution layer and a softmax layer which are sequentially connected;
the second splicing layer and the softmax layer are respectively connected with the second Hadamard laminate, and the second Hadamard laminate is used for outputting the multi-direction fuzzy characteristics.
According to the multidirectional defuzzification method based on the flip deformable convolution, the encoder comprises three frequency domain channel attention blurring processing blocks with sequentially reduced scales.
According to the multidirectional defuzzification method based on the overturn deformable convolution, the output module comprises a first output convolution layer, a second superposition layer, a second output convolution layer and a third superposition layer which are sequentially connected;
The first output convolution layer is connected with the first decoder;
the second output convolution layer is connected with the second decoder;
the second superposition layer is used for superposing the outputs of the first output convolution layer and the second output convolution layer;
The third superposition layer is used for receiving the image to be processed, and superposing the image to be processed and the output of the second superposition layer to obtain the deblurring image.
According to the multidirectional defuzzification method based on the flip deformable convolution, parameters of the first decoder and the second decoder are shared.
The invention also provides a multidirectional deblurring device based on the overturn deformable convolution, which comprises:
The image acquisition module is used for acquiring an image to be processed;
The deblurring module is used for deblurring the image to be processed by applying a turnover deformation network to obtain a deblurred image of the image to be processed;
the turnover deformation network comprises an encoder, a first decoder, a second decoder and an output module, wherein the first decoder comprises a plurality of cascaded multidirectional deformable convolution blocks, and the second decoder is obtained by turnover of the multidirectional deformable convolution blocks in the first decoder by 90 degrees;
the encoder is used for receiving the image to be processed and encoding the image to be processed to obtain the image characteristics of the image to be processed;
The multi-direction deformable convolution block is used for processing the image characteristics based on a simple frequency domain channel attention mechanism and the multi-direction deformable convolution block to obtain multi-direction fuzzy characteristics;
The output module is used for recovering the channel number of the main shaft fuzzy residual image output by the first decoder and the channel number of the complementary residual image output by the second decoder to be the same as the channel number of the image to be processed, and superposing the main shaft fuzzy residual image, the complementary residual image and the image to be processed to obtain the deblurring image.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing a multi-directional deblurring method based on a flip-deformable convolution as described in any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a multi-directional deblurring method based on a roll-over deformable convolution as described in any one of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a multi-directional deblurring method based on a roll-over deformable convolution as described in any one of the above.
Compared with the prior art, the invention has the following beneficial effects:
The multi-direction deblurring method and device based on the turnover deformable convolution firstly acquire an image to be processed, and then deblur the image to be processed by using a turnover deformable network to acquire a deblurred image of the image to be processed. The turnover deformation network adopted by the method comprises an encoder, a first decoder, a second decoder and an output module, wherein the encoder and the decoder adopt asymmetric designs, and the effect of decoupling fuzzy-clear information can be achieved. The multi-directional deformable convolution block in the first decoder is overturned to obtain the second decoder, so that the characteristic of deformable convolution can be utilized to obtain the multi-directional fuzzy characteristics, the overturned and emphasized fuzzy information in the horizontal and vertical directions can be utilized to effectively remove the motion blur in the multi-directions, and the deblurring performance of the overturned deformation network can be improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to these drawings without inventive effort.
FIG. 1 is a flow diagram of a multi-directional deblurring method based on a roll-over deformable convolution provided by the present invention;
FIG. 2 is a schematic diagram of a structure of a roll-over deformable network in a multi-directional deblurring method based on roll-over deformable convolution provided by the present invention;
FIG. 3 is a schematic diagram of an encoder structure of a roll-over deformable network in a multi-directional deblurring method based on roll-over deformable convolution provided by the present invention;
FIG. 4 is a schematic diagram of a BPB structure of a roll-over deformable network in a multi-directional deblurring method based on roll-over deformable convolution provided by the present invention;
FIG. 5 is a schematic diagram of the SFCA structure of a roll-over deformable network in a multi-directional deblurring method based on roll-over deformable convolution provided by the present invention;
FIG. 6 is a schematic diagram of a structure of a flipped network MDDM in a multi-directional deblurring method based on flipped deformable convolution provided by the present invention;
FIG. 7 is a schematic diagram of an image to be processed in a multi-directional deblurring method based on a roll-over deformable convolution provided by the present invention;
FIG. 8 is a schematic diagram of a deblurring image in a multi-directional deblurring method based on a roll-over deformable convolution provided by the present invention;
FIG. 9 is a schematic diagram of a multi-directional deblurring apparatus based on a roll-over deformable convolution provided by the present invention;
fig. 10 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a schematic flow chart of a multi-directional deblurring method based on a roll-over deformable convolution according to an embodiment of the present invention, as shown in fig. 1, where the method includes:
S1, acquiring an image to be processed;
S2, deblurring the image to be processed by using a turnover deformation network to obtain a deblurred image of the image to be processed;
the turnover deformation network comprises an encoder, a first decoder, a second decoder and an output module, wherein the first decoder comprises a plurality of cascaded multidirectional deformable convolution blocks, and the second decoder is obtained by turnover of the multidirectional deformable convolution blocks in the first decoder by 90 degrees;
the encoder is used for receiving the image to be processed and encoding the image to be processed to obtain the image characteristics of the image to be processed;
The multi-direction deformable convolution block is used for processing the image characteristics based on a simple frequency domain channel attention mechanism and the multi-direction deformable convolution block to obtain multi-direction fuzzy characteristics;
The output module is used for recovering the channel number of the main shaft fuzzy residual image output by the first decoder and the channel number of the complementary residual image output by the second decoder to be the same as the channel number of the image to be processed, and superposing the main shaft fuzzy residual image, the complementary residual image and the image to be processed to obtain the deblurring image.
Specifically, in the multi-directional deblurring method based on the flip deformable convolution provided in the embodiment of the present invention, the execution body is a multi-directional deblurring device based on the flip deformable convolution, and the device may be configured in a computer, where the computer may be a local computer or a cloud computer, and the local computer may be a computer, a tablet, or the like, and is not limited herein specifically.
Step S1 is first executed to obtain an image to be processed. The image to be processed refers to a blurred image which has motion blur and needs to be subjected to motion deblurring operation, and can be a large-size substrate image or an image of a target object such as a human face, a vehicle and the like in the monitoring field, and the image is not particularly limited.
And then executing step S2, and applying the turnover deformation network to deblur the image to be processed to obtain a deblurred image of the image to be processed. Here, the image to be processed may be input to a roll-over morphing network (Flipped Deformable Networdk, FDNet) resulting in a deblurred image output by the roll-over morphing network.
As shown in fig. 2, the roll-over deformation network may include an encoder, a first decoder, a second decoder, and an output module, where the encoder is configured to receive an image to be processed and encode the image to be processed to obtain an image feature of the image to be processed. The operation of the encoder can be expressed as:
;
Wherein, As a feature of the image it is,For the encoder, x is the image to be processed.
The first decoder includes a plurality of concatenated multi-directional deformable convolution blocks (MDDBs), each for processing the image features based on a simple frequency domain channel attention (Simple Frequency Channel Attention, SFCA) mechanism and the multi-directional deformable convolution blocks to obtain multi-directional blur features.
Compared with a frequency domain channel attention (FCA) mechanism, the simple frequency domain channel attention mechanism introduces a layer normalization operation and two convolution operations to process the input, and then processes the input through the FCA mechanism, so that the calculated amount can be reduced.
The multi-direction deformable convolution blocks are a plurality of deformable convolution blocks corresponding to different directions, and a direction parameter is added to each position of the input convolution kernel of each deformable convolution block, so that the convolution kernel of each position can freely deform and displace in the receptive field, and the convolution kernel can be further expanded to a larger range in the training process. Here, 3 Deformable convolution blocks (Deformable blocks) with different expansion ratios may be constructed, for generating three offset maps with different received fields to modulate the direction of the Deformable convolution, respectively, and thus generate 3 different directional outputs.
The second decoder is flipped 90 degrees from the multi-directional deformable convolution block in the first decoder, so the multi-directional deformable convolution block contained in the second decoder may be denoted as (I-MDDB), which is identical to the I-MDDB but is orthogonal to the content of interest.
The first decoder and the second decoder are two independent decoders, and parameters of the two decoders can be shared. According to linear span theory, a linear reconstruction consisting of the outputs of two independent decoders may contain a larger output space, as the independent regression network learns the complementary features, covering the whole output space. Ideally, the outputs of the two decoders are independent regression outputs to maximize the solution space. In other words, in the embodiment of the present invention, the output of the first decoder is a principal axis blurred residual image, that is, the degradation component with the largest variation, and the output of the second decoder is a complementary residual, so as to complete the linear reconstruction.
The principal axis blurred residual image D1 output by the first decoder may be expressed asThe complementary residual image D2 output by the second decoder can be expressed asWherein, the method comprises the steps of,AndA first decoder and a second decoder are represented respectively,A parameter representing the first decoder is indicated,Representing parameters of the second decoder.
The first decoder and the second decoder can decode the inherent information in the image features into a principal axis blurred residual image D1 along the horizontal axis and a complementary residual image D2 along the vertical axis, respectively, without applying any explicit constraint, thereby more effectively deblurring.
The output module can restore the channel number of the main shaft fuzzy residual image D1 output by the first decoder and the channel number of the complementary residual image D2 output by the second decoder to be the same as the channel number of the image to be processed, and superimpose the main shaft fuzzy residual image D1, the complementary residual image D2 and the image to be processed to obtain a deblurring image. Here, the number of channels of the image to be processed may be 3, and thus the number of channels of the principal axis blur residual image D1 and the complementary residual image D2 may be restored to 3.
The multi-direction deblurring method based on the turnover deformable convolution, provided by the embodiment of the invention, comprises the steps of firstly obtaining an image to be processed, and then deblurring the image to be processed by using a turnover deformable network to obtain a deblurred image of the image to be processed. The turnover deformation network adopted by the method comprises an encoder, a first decoder, a second decoder and an output module, wherein the encoder and the decoder adopt asymmetric designs, and the effect of decoupling fuzzy-clear information can be achieved. The multi-directional deformable convolution block in the first decoder is overturned to obtain the second decoder, so that the characteristic of deformable convolution can be utilized to obtain the multi-directional fuzzy characteristics, the overturned and emphasized fuzzy information in the horizontal and vertical directions can be utilized to effectively remove the motion blur in the multi-directions, and the deblurring performance of the overturned deformation network can be improved.
On the basis of the embodiment, the encoder comprises a first convolution layer and a plurality of frequency domain channel attention blurring processing blocks with different scales, wherein the first convolution layer and the frequency domain channel attention blurring processing blocks comprise a blurring processing block and a simple frequency domain channel attention block;
the fuzzy processing block comprises a first layer normalization layer, a second convolution layer, a third convolution layer, a first multiplication layer, a Fourier transform layer, a first Hadamard lamination layer, an inverse Fourier transform layer, an activation layer and a first superposition layer which are sequentially connected;
the output end of the second convolution layer is also connected with the input end of the first multiplication layer;
the input end of the first superimposed layer is also connected with the input end of the first normalization layer, and the output end of the first superimposed layer is used for outputting the processing result of the fuzzy processing block;
The simple frequency domain channel attention block comprises a second layer normalization layer, a fourth convolution layer, a fifth convolution layer, a frequency domain channel attention module, a sixth convolution layer and a second multiplication layer which are sequentially connected, and the input end of the second multiplication layer is also connected with the input end of the second layer normalization layer.
Specifically, as shown in fig. 3, the encoder includes a first convolution layer and a plurality of different scale frequency domain channel attention blurring processing blocks connected in sequence. In fig. 3, the encoder may include 2 frequency-domain channel attention blurring blocks, each including a blurring block (Blur Processing Block, BPB) and a simple frequency-domain channel attention (SFCA) block.
As shown in fig. 4, the BPB includes a first layer normalization (LayerNorm) layer, a second convolution (Conv) layer, a third convolution layer, a first multiplication layer, a fourier transform (FFT) layer, a first hadamard-layering layer, an inverse fourier transform (IFFT) layer, an activation layer, and a first superposition layer, which are sequentially connected. The second convolution layer and the third convolution layer can perform a convolution operation of 1×1 to expand the number of channels and obtain potential information. The activation layer may implement a nonlinear mapping using Gelu activation functions. The BPB may acquire frequency domain information through fourier transform and inverse fourier transform.
The output end of the second convolution layer is also connected with the input end of the first multiplication layer, the input end of the first superposition layer is also connected with the input end of the first normalization layer, and the output end of the first superposition layer is used for outputting the processing result of the fuzzy processing block.
The operation of the BPB may be expressed by the following formula:
;
;
;
。
Wherein Y is the input of BPB, is tensor, For the convolution operation implemented for the second convolution layer,Layer normalization operations implemented for the first layer normalization layer,For the convolution operation implemented for the third convolution layer,For the output of the second convolutional layer,For the output of the first multiplication layer,As the weight coefficient of the light-emitting diode,For the output of the first hadamard layer,Is the processing result of the BPB, namely the output of the BPB.
As shown in fig. 5, the SFCA block includes a second normalization layer, a fourth convolution layer, a fifth convolution layer, a frequency domain channel attention (FCA) module, a sixth convolution layer, and a second multiplication layer that are sequentially connected, where an input end of the second multiplication layer is further connected to an input end of the second normalization layer.
The output of the fifth convolution layer may be expressed by the following formula:
;
Wherein, For the output of the fifth convolutional layer, X is the input of the SFCA block, is the tensor,Layer normalization operations implemented for the second layer normalization layer,The convolution operation implemented for the fourth convolution layer,Convolution operations implemented for the fifth convolution layer.
The FCA module comprises a channel dividing module, a frequency domain attention module, a first splicing layer, a seventh convolution layer and a third multiplication layer, wherein the input of the FCA module is subjected to channel dividing module to obtain a plurality of channel components, the channel components are subjected to frequency domain attention calculation through the frequency domain attention module to obtain a plurality of frequency components, the frequency components are subjected to cascade connection through the first splicing layer to obtain a compression vector, and the compression vector is subjected to multiplication operation with the input of the frequency domain channel attention module through the third multiplication layer after passing through the seventh convolution layer to obtain the output of the frequency domain channel attention module.
For example, the input to the FCA module is the output of the fifth convolutional layer, i.eThe channel components obtained by the channel dividing module are that the channel number is equalN channel components of (a), i.eC isAnd is to be able to be usedAnd (5) integer division. For example, the n channel components may be represented as,,H and W are respectivelyIs a height and width of (a).
Each channel component is calculated by the frequency domain attention of the frequency domain attention module, and the frequency domain attention module comprises:
;
;
Wherein, Representation ofA component with a middle height of h and a width of w,,Respectively correspond toIs a two-dimensional (2D) index of frequency components of (a),To correspond toIs a two-dimensional DCT of (a),Is thatFrequency components calculated by the frequency domain attention of the frequency domain attention module, andAfter compression ofAnd (5) a dimension vector.Is thatCorresponding DCT weights.
The frequency components corresponding to the channel components are cascaded through the first splicing layer, namely splicing operation is performed, and a compression vector can be obtained:
;
Wherein, Is a compressed vector.
The frequency components corresponding to the channel components are cascaded through a first splicing layer to obtain a compression vectorAnd then after passing through the seventh convolution layerPerforming multiplication operation by a third multiplication layer to obtain an operation result, and taking the operation result as output of the FCA module, wherein the operation result comprises:
;
Wherein, In order to obtain the result of the operation,Convolution operations implemented for the seventh convolution layer.
It will be appreciated that the result of the operationIs a tensor with a channel number C, a height H, and a width W. In the embodiment of the invention, the convolution kernel size of the adopted seventh convolution layer can be 1 multiplied by 1, so that the calculated amount can be effectively reduced.
On the basis of the above embodiment, the MDDB includes a BPB, an SFCA block, and a multi-directional deblurring module (Multi Direction deblurring module, MDDM) connected in sequence.
As shown in fig. 6, MDDM includes a first branch including a first residual block (ResBlock), three Deformable convolution blocks (Deformable), and a second splice (Concat) layer, a second branch, and a second hadamard stack.
The three deformable convolution blocks are connected in parallel between the first residual error module and the second splicing layer.
The second branch comprises an eighth convolution layer, a second residual error module, a third residual error module, a ninth convolution layer and a softmax layer which are connected in sequence.
The second splicing layer and the softmax layer are respectively connected with a second Hadamard lamination layer, and the second Hadamard lamination layer is used for outputting multi-direction fuzzy characteristics.
In MDDM, on one hand, rough obtaining of fuzzy information is performed on the input through the first residual error module and then through the three residual error modules, and the output of the second splicing layer is obtained after the content generated by the three deformable convolution blocks is spliced along the channel dimension through the second splicing layer, on the other hand, weight mapping is obtained on the input through the eighth convolution layer, the second residual error module, the third residual error module and the ninth convolution layer in sequence and then through the Softmax layer.
And then, performing element-by-element point multiplication on the output of the second splicing layer and the weight mapping through a second Hadamard lamination to obtain MDDM output, namely the multi-direction fuzzy characteristic.
The operation of the second hadamard layering can be expressed as:
;
;
Wherein, For the output of the second hadamard layer,For the ith channel in the output of the second splice layer,For the weight map of the jth pixel point of the ith channel, h is the height of the input of MDDM and w is the width of the input of MDDM.
On the basis of the above embodiment, the parameters of the first decoder and the second decoder are shared, so that the computational complexity is not increased additionally.
Based on the above embodiment, the roll-over deformation network may be obtained by training the initial network by using the image sample with motion blur and the clear image corresponding to the image sample. The image sample can be acquired by an image acquisition device with a first shutter rate, and the clear image corresponding to the image sample can be acquired by an image acquisition device with a second shutter rate, wherein the first shutter rate is smaller than the second shutter rate. The initial network may be the same as the structure of the roll-over deformed network, except that the structural parameters of the initial network are obtained by initialization, and the structural parameters of the roll-over deformed network are obtained by training the initial network.
When the initial network is trained, the image sample can be input into the initial network to obtain an output result of the initial network, then the output result and the clear image are utilized to calculate a loss function, and the structural parameters of the initial network are iteratively updated according to the loss function until the loss function converges to obtain the turnover deformation network.
As shown in fig. 7, is an image to be processed. FIG. 8 is a deblurred image obtained by the multi-directional deblurring method based on a roll-over deformable convolution provided in an embodiment of the present invention. As can be seen from a comparison between fig. 7 and fig. 8, the multi-directional deblurring method based on the flip deformable convolution according to the embodiment of the present invention has an excellent deblurring effect.
On the basis of the above embodiment, for a roll-over morphing network, its key parameter is the number of deformable convolution blocks. The number of deformable convolution blocks represents the number of directions, and different effects on multi-directional fuzzy perception are achieved by designing different numbers of deformable convolution blocks. In Table 1, four versions of the roll-over morphing network are shown, FDNet-1, FDNet-2, FDNet-4, FDNet-8,1, 2, 4, 8, respectively, representing the number of deformable convolution blocks stacked in MDDM. Peak signal-to-noise ratio (PSNR) is one of the indicators measuring image quality. As shown in table 1, PSNR increases with the number of stacked deformable convolution blocks, and saturation occurs after the number of deformable convolution blocks exceeds four, i.e., more computing resources are required, and the effect of improvement is small. Therefore, in the embodiment of the invention, the number of the deformable convolution blocks is selected to be three, and excellent balance between performance and efficiency is achieved.
TABLE 1 correspondence of number of deformable convolutions to PSNR
In summary, in the multi-directional deblurring method based on the overturn deformable convolution provided in the embodiment of the present invention, the overturn deformation network is adopted to pay attention to the blurred information from multiple directions, so as to remove the motion blur. The turnover deformation network uses the turnover design of the first encoder and the second decoder to turn over in multiple directions, thereby realizing the study of orthogonal directions in multiple directions and achieving the characteristic attention in more directions. In addition, the multidirectional deformable convolution block better utilizes frequency domain information and plays a role in acquiring fuzzy information. By the two-decoder parameter sharing design, the computational parameters are greatly reduced, which reduces more computational consumption than other multi-decoder designs.
As shown in fig. 9, on the basis of the above embodiment, in an embodiment of the present invention, there is provided a multi-directional deblurring device based on a roll-over deformable convolution, including:
an image acquisition module 91, configured to acquire an image to be processed;
The deblurring module 92 is configured to deblur the image to be processed by applying a flipping deformation network, so as to obtain a deblurred image of the image to be processed;
the turnover deformation network comprises an encoder, a first decoder, a second decoder and an output module, wherein the first decoder comprises a plurality of cascaded multidirectional deformable convolution blocks, and the second decoder is obtained by turnover of the multidirectional deformable convolution blocks in the first decoder by 90 degrees;
the encoder is used for receiving the image to be processed and encoding the image to be processed to obtain the image characteristics of the image to be processed;
The multi-direction deformable convolution block is used for processing the image characteristics based on a simple frequency domain channel attention mechanism and the multi-direction deformable convolution block to obtain multi-direction fuzzy characteristics;
The output module is used for recovering the channel number of the main shaft fuzzy residual image output by the first decoder and the channel number of the complementary residual image output by the second decoder to be the same as the channel number of the image to be processed, and superposing the main shaft fuzzy residual image, the complementary residual image and the image to be processed to obtain the deblurring image.
On the basis of the embodiment, the encoder comprises a first convolution layer and a plurality of frequency domain channel attention blurring processing blocks with different scales, wherein the first convolution layer and the frequency domain channel attention blurring processing blocks comprise a blurring processing block and a simple frequency domain channel attention block;
the fuzzy processing block comprises a first layer normalization layer, a second convolution layer, a third convolution layer, a first multiplication layer, a Fourier transform layer, a first Hadamard lamination layer, an inverse Fourier transform layer, an activation layer and a first superposition layer which are sequentially connected;
the output end of the second convolution layer is also connected with the input end of the first multiplication layer;
the input end of the first superimposed layer is also connected with the input end of the first normalization layer, and the output end of the first superimposed layer is used for outputting the processing result of the fuzzy processing block;
the simple frequency domain channel attention block comprises a second layer normalization layer, a fourth convolution layer, a fifth convolution layer, a frequency domain channel attention module, a sixth convolution layer and a second multiplication layer which are sequentially connected, and the input end of the second multiplication layer is also connected with the input end of the second layer normalization layer.
On the basis of the above embodiment, the frequency domain channel attention module includes a channel dividing module, a frequency domain attention module, a first splicing layer, a seventh convolution layer, and a third multiplication layer;
the input of the frequency domain channel attention module is subjected to the channel dividing module to obtain a plurality of channel components;
The frequency domain attention module is used for carrying out frequency domain attention calculation on the channel components to obtain a plurality of frequency components;
the plurality of frequency components are cascaded through the first splicing layer to obtain a compression vector;
And multiplying the compressed vector by the seventh convolution layer and then carrying out multiplication operation on the compressed vector and the input of the frequency domain channel attention module by the third multiplication layer to obtain the output of the frequency domain channel attention module.
On the basis of the embodiment, the multidirectional deformable convolution block comprises a fuzzy processing block, a simple frequency domain channel attention block and a multidirectional defuzzification module which are connected in sequence;
the multi-direction deblurring module comprises a first branch, a second branch and a second Hadamard layer, wherein the first branch comprises a first residual error module, three deformable convolution blocks and a second splicing layer;
the three deformable convolution blocks are connected between the first residual error module and the second splicing layer in parallel;
the second branch comprises an eighth convolution layer, a second residual error module, a third residual error module, a ninth convolution layer and a softmax layer which are sequentially connected;
the second splicing layer and the softmax layer are respectively connected with the second Hadamard laminate, and the second Hadamard laminate is used for outputting the multi-direction fuzzy characteristics.
On the basis of the above embodiment, the encoder includes three frequency domain channel attention blurring processing blocks with sequentially decreasing scales.
On the basis of the embodiment, the output module comprises a first output convolution layer, a second superposition layer, a second output convolution layer and a third superposition layer which are sequentially connected;
The first output convolution layer is connected with the first decoder;
the second output convolution layer is connected with the second decoder;
the second superposition layer is used for superposing the outputs of the first output convolution layer and the second output convolution layer;
The third superposition layer is used for receiving the image to be processed, and superposing the image to be processed and the output of the second superposition layer to obtain the deblurring image.
On the basis of the above embodiments, the parameters of the first decoder and the second decoder are shared.
Specifically, the functions of each module in the multi-directional deblurring device based on the overturn deformable convolution provided in the embodiment of the present invention are in one-to-one correspondence with the operation flows of each step in the above method embodiment, and the achieved effects are consistent.
Fig. 10 illustrates a physical schematic diagram of an electronic device, as shown in fig. 10, which may include a Processor 110, a communication interface (Communications Interface) 120, a Memory 130, and a communication bus 140, where the Processor 110, the communication interface 120, and the Memory 130 perform communication with each other through the communication bus 140. The processor 110 may invoke logic instructions in the memory 130 to perform the multi-directional deblurring method based on the flip-deformable convolution provided in the embodiments described above.
In addition, the logic instructions in the memory 130 may be implemented in the form of software functional units and stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the multi-directional deblurring method based on a roll-over deformable convolution provided in the above embodiments.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the multi-directional deblurring method based on a roll-over deformable convolution provided in the above embodiments.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present invention.
Claims (7)
1. A multi-directional deblurring method based on a roll-over deformable convolution, comprising:
acquiring an image to be processed;
deblurring the image to be processed by using a turnover deformation network to obtain a deblurred image of the image to be processed;
the turnover deformation network comprises an encoder, a first decoder, a second decoder and an output module, wherein the first decoder comprises a plurality of cascaded multidirectional deformable convolution blocks, and the second decoder is obtained by turnover of the multidirectional deformable convolution blocks in the first decoder by 90 degrees;
the encoder is used for receiving the image to be processed and encoding the image to be processed to obtain the image characteristics of the image to be processed;
The multi-direction deformable convolution block is used for processing the image characteristics based on a simple frequency domain channel attention mechanism and a multi-direction deformable convolution block to obtain multi-direction fuzzy characteristics, wherein compared with the frequency domain channel attention mechanism, the simple frequency domain channel attention mechanism introduces layer normalization operation and two convolution operations to process input and then processes the input through the frequency domain channel attention mechanism;
The output module is used for recovering the channel number of the main shaft fuzzy residual image output by the first decoder and the channel number of the complementary residual image output by the second decoder to be the same as the channel number of the image to be processed, and superposing the main shaft fuzzy residual image, the complementary residual image and the image to be processed to obtain the deblurring image;
the multi-direction deformable convolution block comprises a fuzzy processing block, a simple frequency domain channel attention block and a multi-direction deblurring module which are connected in sequence;
The multi-direction deblurring module comprises a first branch, a second branch and a second Hadamard layer, wherein the first branch comprises a first residual error module, three deformable convolution blocks and a second splicing layer, wherein the three deformable convolution blocks are 3 deformable convolution blocks with different expansion rates and are respectively used for generating three offset graphs with different receiving fields to modulate the directions of the deformable convolution so as to generate 3 outputs with different directions;
the three deformable convolution blocks are connected between the first residual error module and the second splicing layer in parallel;
the second branch comprises an eighth convolution layer, a second residual error module, a third residual error module, a ninth convolution layer and a softmax layer which are sequentially connected;
The second splicing layer and the softmax layer are respectively connected with the second Hadamard lamination layer, and the second Hadamard lamination layer is used for outputting the multi-direction fuzzy characteristics;
The encoder comprises a first convolution layer and a plurality of frequency domain channel attention blurring processing blocks with different scales, which are sequentially connected, wherein the frequency domain channel attention blurring processing blocks comprise blurring processing blocks and simple frequency domain channel attention blocks;
the fuzzy processing block comprises a first layer normalization layer, a second convolution layer, a third convolution layer, a first multiplication layer, a Fourier transform layer, a first Hadamard lamination layer, an inverse Fourier transform layer, an activation layer and a first superposition layer which are sequentially connected;
the output end of the second convolution layer is also connected with the input end of the first multiplication layer;
the input end of the first superimposed layer is also connected with the input end of the first normalization layer, and the output end of the first superimposed layer is used for outputting the processing result of the fuzzy processing block;
The simple frequency domain channel attention block comprises a second normalization layer, a fourth convolution layer, a fifth convolution layer, a frequency domain channel attention module, a sixth convolution layer and a second multiplication layer which are sequentially connected, and the input end of the second multiplication layer is also connected with the input end of the second normalization layer;
the frequency domain channel attention module comprises a channel dividing module, a frequency domain attention module, a first splicing layer, a seventh convolution layer and a third multiplication layer;
the input of the frequency domain channel attention module is subjected to the channel dividing module to obtain a plurality of channel components;
The frequency domain attention module is used for carrying out frequency domain attention calculation on the channel components to obtain a plurality of frequency components;
the plurality of frequency components are cascaded through the first splicing layer to obtain a compression vector;
And multiplying the compressed vector by the seventh convolution layer and then carrying out multiplication operation on the compressed vector and the input of the frequency domain channel attention module by the third multiplication layer to obtain the output of the frequency domain channel attention module.
2. The multi-directional deblurring method based on a flip-deformable convolution of claim 1, wherein the encoder comprises three successively smaller scale frequency domain channel attention blurring processing blocks.
3. The multi-directional deblurring method based on a roll-over deformable convolution of any one of claims 1-2, wherein the output module comprises a first output convolution layer, a second superposition layer, a second output convolution layer, and a third superposition layer connected in sequence;
The first output convolution layer is connected with the first decoder;
the second output convolution layer is connected with the second decoder;
the second superposition layer is used for superposing the outputs of the first output convolution layer and the second output convolution layer;
The third superposition layer is used for receiving the image to be processed, and superposing the image to be processed and the output of the second superposition layer to obtain the deblurring image.
4. The multi-directional deblurring method based on a flip-deformable convolution according to any one of claims 1-2, wherein parameters of the first decoder and the second decoder are shared.
5. A multi-directional deblurring device based on a roll-over deformable convolution, comprising:
The image acquisition module is used for acquiring an image to be processed;
The deblurring module is used for deblurring the image to be processed by applying a turnover deformation network to obtain a deblurred image of the image to be processed;
the turnover deformation network comprises an encoder, a first decoder, a second decoder and an output module, wherein the first decoder comprises a plurality of cascaded multidirectional deformable convolution blocks, and the second decoder is obtained by turnover of the multidirectional deformable convolution blocks in the first decoder by 90 degrees;
the encoder is used for receiving the image to be processed and encoding the image to be processed to obtain the image characteristics of the image to be processed;
The multi-direction deformable convolution block is used for processing the image characteristics based on a simple frequency domain channel attention mechanism and a multi-direction deformable convolution block to obtain multi-direction fuzzy characteristics, wherein compared with the frequency domain channel attention mechanism, the simple frequency domain channel attention mechanism introduces layer normalization operation and two convolution operations to process input and then processes the input through the frequency domain channel attention mechanism;
The output module is used for recovering the channel number of the main shaft fuzzy residual image output by the first decoder and the channel number of the complementary residual image output by the second decoder to be the same as the channel number of the image to be processed, and superposing the main shaft fuzzy residual image, the complementary residual image and the image to be processed to obtain the deblurring image;
the multi-direction deformable convolution block comprises a fuzzy processing block, a simple frequency domain channel attention block and a multi-direction deblurring module which are connected in sequence;
The multi-direction deblurring module comprises a first branch, a second branch and a second Hadamard layer, wherein the first branch comprises a first residual error module, three deformable convolution blocks and a second splicing layer, wherein the three deformable convolution blocks are 3 deformable convolution blocks with different expansion rates and are respectively used for generating three offset graphs with different receiving fields to modulate the directions of the deformable convolution so as to generate 3 outputs with different directions;
the three deformable convolution blocks are connected between the first residual error module and the second splicing layer in parallel;
the second branch comprises an eighth convolution layer, a second residual error module, a third residual error module, a ninth convolution layer and a softmax layer which are sequentially connected;
The second splicing layer and the softmax layer are respectively connected with the second Hadamard lamination layer, and the second Hadamard lamination layer is used for outputting the multi-direction fuzzy characteristics;
The encoder comprises a first convolution layer and a plurality of frequency domain channel attention blurring processing blocks with different scales, which are sequentially connected, wherein the frequency domain channel attention blurring processing blocks comprise blurring processing blocks and simple frequency domain channel attention blocks;
the fuzzy processing block comprises a first layer normalization layer, a second convolution layer, a third convolution layer, a first multiplication layer, a Fourier transform layer, a first Hadamard lamination layer, an inverse Fourier transform layer, an activation layer and a first superposition layer which are sequentially connected;
the output end of the second convolution layer is also connected with the input end of the first multiplication layer;
the input end of the first superimposed layer is also connected with the input end of the first normalization layer, and the output end of the first superimposed layer is used for outputting the processing result of the fuzzy processing block;
The simple frequency domain channel attention block comprises a second normalization layer, a fourth convolution layer, a fifth convolution layer, a frequency domain channel attention module, a sixth convolution layer and a second multiplication layer which are sequentially connected, and the input end of the second multiplication layer is also connected with the input end of the second normalization layer;
the frequency domain channel attention module comprises a channel dividing module, a frequency domain attention module, a first splicing layer, a seventh convolution layer and a third multiplication layer;
the input of the frequency domain channel attention module is subjected to the channel dividing module to obtain a plurality of channel components;
The frequency domain attention module is used for carrying out frequency domain attention calculation on the channel components to obtain a plurality of frequency components;
the plurality of frequency components are cascaded through the first splicing layer to obtain a compression vector;
And multiplying the compressed vector by the seventh convolution layer and then carrying out multiplication operation on the compressed vector and the input of the frequency domain channel attention module by the third multiplication layer to obtain the output of the frequency domain channel attention module.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a multi-directional deblurring method based on a flip-deformable convolution as claimed in any one of claims 1-4 when the program is executed.
7. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the multi-directional deblurring method based on a flip-deformable convolution as claimed in any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411303544.5A CN118822905B (en) | 2024-09-19 | 2024-09-19 | Multi-directional deblurring method and device based on flipped deformable convolution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411303544.5A CN118822905B (en) | 2024-09-19 | 2024-09-19 | Multi-directional deblurring method and device based on flipped deformable convolution |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118822905A CN118822905A (en) | 2024-10-22 |
| CN118822905B true CN118822905B (en) | 2024-12-24 |
Family
ID=93070850
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411303544.5A Active CN118822905B (en) | 2024-09-19 | 2024-09-19 | Multi-directional deblurring method and device based on flipped deformable convolution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118822905B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112801901A (en) * | 2021-01-21 | 2021-05-14 | 北京交通大学 | Image deblurring algorithm based on block multi-scale convolution neural network |
| CN116993630A (en) * | 2023-09-27 | 2023-11-03 | 闽都创新实验室 | Method and device for removing motion blur based on attention characteristics of residual image |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20230245282A1 (en) * | 2022-01-29 | 2023-08-03 | Samsung Electronics Co., Ltd. | Method and device for depth image completion |
| KR102664276B1 (en) * | 2022-12-02 | 2024-05-10 | 고려대학교 산학협력단 | Single image deblurring method via horizontal and vertical decomposition and network system thereof |
| KR102666431B1 (en) * | 2023-02-15 | 2024-05-17 | 동국대학교 산학협력단 | Device and method for motion blur image restoration |
| CN117115040A (en) * | 2023-09-12 | 2023-11-24 | 大连大学 | Moving image deblurring model based on Fourier transform |
| CN117726544A (en) * | 2023-12-04 | 2024-03-19 | 重庆邮电大学 | An image deblurring method and system for complex motion scenes |
-
2024
- 2024-09-19 CN CN202411303544.5A patent/CN118822905B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112801901A (en) * | 2021-01-21 | 2021-05-14 | 北京交通大学 | Image deblurring algorithm based on block multi-scale convolution neural network |
| CN116993630A (en) * | 2023-09-27 | 2023-11-03 | 闽都创新实验室 | Method and device for removing motion blur based on attention characteristics of residual image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118822905A (en) | 2024-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | A unified learning framework for single image super-resolution | |
| US10360664B2 (en) | Image processing apparatus and method using machine learning | |
| Zhang et al. | Image restoration using joint statistical modeling in a space-transform domain | |
| Cao et al. | Fast image deconvolution using closed-form thresholding formulas of Lq (q= 12, 23) regularization | |
| Dong et al. | Nonlocally centralized sparse representation for image restoration | |
| KR101137753B1 (en) | Methods for fast and memory efficient implementation of transforms | |
| Yokota et al. | Simultaneous visual data completion and denoising based on tensor rank and total variation minimization and its primal-dual splitting algorithm | |
| Zuo et al. | Convolutional neural networks for image denoising and restoration | |
| Chen | Regularized generalized inverse accelerating linearized alternating minimization algorithm for frame-based Poissonian image deblurring | |
| Portilla et al. | Efficient and robust image restoration using multiple-feature L2-relaxed sparse analysis priors | |
| Dai et al. | Image super-resolution via residual block attention networks | |
| CN116993630B (en) | Method and device for removing motion blur based on attention characteristics of residual image | |
| CN110782398B (en) | Image processing method, generative countermeasure network system and electronic device | |
| CN118691503B (en) | Motion deblurring method and device based on Fourier frequency domain attention mechanism | |
| CN109102463B (en) | Super-resolution image reconstruction method and device | |
| Ono et al. | Image recovery by decomposition with component-wise regularization | |
| CN118608387A (en) | Method, device and apparatus for super-resolution reconstruction of satellite video frames | |
| CN118822905B (en) | Multi-directional deblurring method and device based on flipped deformable convolution | |
| Chen et al. | Inexact alternating direction method based on proximity projection operator for image inpainting in wavelet domain | |
| Li et al. | A new algorithm framework for image inpainting in transform domain | |
| Wang et al. | A highly scalable parallel algorithm for isotropic total variation models | |
| CN116309175A (en) | Plug-and-play image restoration method and system based on multiple regularization terms | |
| JP2011182330A (en) | Image processing method, image processing apparatus and program | |
| CN116029932A (en) | A method and system for correcting exposure errors of ultra-high resolution images | |
| Wang et al. | Deep attention-based lightweight network for aerial image deblurring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |