CN107155112A - A kind of compressed sensing method for processing video frequency for assuming prediction more - Google Patents

Abstract

The invention discloses a multi-hypothesis prediction compression sensing video processing method, the structural framework of the processing method includes an encoding end and a decoding end; at the encoding end, frames in the video are divided into key frames and non-key frames, according to the compression Perceptual theory, both key frames and non-key frames obtain measured values through the measurement matrix Φ; at the decoding end, the key frames undergo BCS-SPL reconstruction, and then perform multi-prediction assumptions and residual reconstruction respectively; the non-key frames Keyframes are residually reconstructed and decoded according to the side information generated by the keyframes. A new distributed compressed video sensing framework based on MH prediction proposed in this invention can capture and compress video at low-complexity encoder and reconstruct video efficiently at decoder, improving MH‑BCS‑SPL Frames can provide better image reconstruction quality.

Description

A compressive sensing video processing method based on multi-hypothesis prediction

technical field

The present invention relates to the technical field of compressed sensing video, in particular to a multi-hypothesis segmented video compression sensing combined with smoothing filtering to realize image reconstruction technology.

Background technique

Currently, CS (compressed sensing) has revolutionized signal sampling and processing systems by integrating compression and sensing. For the application of studying CS for video from different aspects, it is called Compressed Video Sensing (CVS). At the encoder, input video frames are grouped into picture groups consisting of a key frame and multiple non-key frames.

MPEG/H.264 encoding is used in the traditional framework to encode keyframes, CS measurement matrix to sense non-keyframes, and reconstruct keyframes using side information generated from neighboring reconstructed keyframes. The downside of this framework is that complex MPEG/H.264 encoding is still required. In subsequent refinements, CS measurements are applied to both keyframes and non-keyframes. Keyframes are reconstructed using Gradient Projection for Sparse Reconstruction (GPSR), and non-keyframes will be reconstructed using side information generated from decoded keyframes.

With the continuous advancement of technology, in order to reduce the huge computational and memory burden, L.Gan proposed a CS hypothesis (BCS) based on the independence between blocks for 2D images, Do et al. used precious decoding frames Neighboring blocks in the current frame are represented to improve the accuracy of side information, and a residual reconstruction method is developed. S. Mun extends Gan's BCS and reconstructs it in the nearest transform domain, characterized by a height-oriented decomposition. These methods are called Single Hypothesis Motion Compensation (SH-MC) schemes, which have some disadvantages. At the encoder, due to the motion estimation search, in addition to increasing the computational complexity on the encoder side, a transmission overhead of sending the block motion vector is imposed. Furthermore, SH-MC implicitly assumes that motion occurring in video frames is a uniform block translation model. Since this assumption does not always hold, block artifacts appear in recovered frames.

To address these issues, E.Taramel et al. proposed a strategy for incorporating Multi-Hypothesis Motion Compensation (MH-MC) into BCS and smoothing filtering Landweber for video reconstruction (MH-BCS-SPL), It finds more precise hypotheses by finding a linear combination of all blocks or hypotheses in the search window. The MH-MC technique improves recovery performance at the cost of more complexity at the decoder. A combined scheme based on elastic networks using MH and SH reconstruction was later proposed, which achieves acceptable performance at a more complex cost compared to Tikhonov regularized reconstruction. Then came algorithms such as hypothesis set updating and dynamic reference frame selection. Methods for sequentially deploying MH prediction in the measurement and pixel domains to develop two-level MH reconstruction schemes, and R Li et al. presenting space-time quantization and motion-aligned reconstruction for improved performance of CVS systems, among others. However, there are still some problems to be solved, because the computational burden of the encoder is released, we can get the side information (SI) through simple algorithms, but the non-keyframe reconstruction processing cannot be performed efficiently with coarse prediction.

Contents of the invention

Based on the technical problems existing in the background technology, one of the purposes of the present invention is to provide a new distributed image compression sensing video image processing framework based on multi-hypothesis prediction, in which three side information candidates are calculated to select and improve the traditional MH prediction algorithm, At the solution end, bidirectional estimation is used to calculate candidate side information, and a new algorithm is introduced to calculate correlation coefficients, key side information is selected, and non-key frames are restored.

A compressive sensing video processing method for multi-hypothesis prediction, the structural framework of the processing method includes an encoding end and a decoding end; at the encoding end, in order to improve the quality of video reconstruction and according to real-time requirements, video sequence frames are divided into The key frame and the non-key frame form a group of picture (GOP, Group Of Picture) every two frames, that is, the GOP is equal to 2. Usually odd frames are key frames and even frames are non-key frames. According to compressed sensing theory, both key frames and non-key frames obtain measurement values through the measurement matrix Φ, the difference is that the measurement rate of key frames is high, and the measurement rate of non-key frames is low; The smooth projected Landweber (BCS SPL) reconstruction algorithm is used for decoding, and then after multi-hypothesis prediction algorithm and residual reconstruction, the reconstructed key frame is obtained and stored; Information is jointly decoded together to obtain reconstructed non-keyframes. Finally, the decoded key frames and non-key frames are integrated into a video sequence according to frame order and output.

Preferably, the side information is obtained through a multi-hypothesis prediction MH algorithm based on decoded adjacent key frames.

The method steps of the multi-hypothesis prediction MH algorithm are as follows:

(1) Use bidirectional motion estimation to calculate three candidate side information SIi (i=0,1,2);

(2) Calculate the correlation coefficients between non-key frames and three candidate side information, and select the SI information with the highest correlation.

(3) During image reconstruction, in the measurement domain, use the multi-hypothesis prediction of SI information to generate a signal residual, and calculate the best linear combination of hypotheses, and use the improved multi-scale block compression sensing MS-BCS-SPL Techniques for reconstructing images.

Preferably, the assumption optimal linear combination is calculated according to the weighted regularized Tikhonov matrix.

A compressive sensing video processing method based on multi-hypothesis prediction is applied to video image processing.

Compared with the prior art, the present invention has the beneficial effects of:

A novel distributed compressed video sensing framework based on MH prediction proposed in this invention can capture and compress video at low-complexity encoder and efficiently reconstruct video at decoder. The proposed framework can estimate initial side information via MH prediction and BiME. Side information is selected according to the correlation coefficient and used to recover non-keyframes. Experimental simulation results show that the proposed framework can provide better reconstruction quality than the original MH-BCS-SPL algorithm.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the description, and are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. In the attached picture:

Fig. 1 is the block diagram of the CVS codec of a kind of multi-hypothesis prediction compressive perception video processing method that the present invention proposes;

Fig. 2 is a kind of compressive sensing video processing method of multi-hypothesis prediction based on multi-prediction assumption to generate side information block diagram;

Figure 3 shows the average PSNR obtained by using two algorithms at different sampling rates for non-key frames of the Akiyo sequence;

Figure 4 shows the average PSNR obtained by using two algorithms at different sampling rates for non-key frames of the Coastguard sequence;

Figure 5 shows the average PSNR obtained by using two algorithms at different sampling rates for non-key frames of the Foreman sequence;

Figure 6 shows the average PSNR of the non-key frames of the Stefan sequence using two algorithms at different sampling rates.

detailed description

The present invention will be further explained below in conjunction with specific embodiments.

In the traditional MH-BC-SPL structure, compressive sensing combines signal capture and dimensionality reduction by measuring the projection of a signal x of dimension N using some basis Φ of dimension M×N, where M<<N. The measurement vector y is obtained as:

y=Φx (1)

where x∈R ^N , y∈R ^M . If x is sufficiently sparse in some transformation base Ψ, x is optimally reconstructed by y as follows:

where Ψ and Φ are sufficiently irrelevant, and M is large enough.

The compression sampling rate is defined as R=M/N.

In general, Φ is a random matrix, making it incoherent to any choice of Ψ. In practical applications, most natural signals are not truly sparse in any transform basis Ψ.

Then, the reconstruction problem of x can be changed from formula (2) to the equation for the boundary:

In order to solve the problem of relaxed reconstruction in formula (3), L.Gan proposed a block-based CS (BCS) algorithm that removes the global sampling of x by dense Φ and replaces it with the block diagonal measurement matrix . When using the same ΦB for each block, Φ can take the block diagonal form as follows:

Equation (1) can be rewritten in a block-wise manner y _i =Φ _B x _i , where xi is block i of the image. Φ _B is a measurement matrix whose size is M _B ×B ² , and the measurement rate of the BCS algorithm is R _B =M _B /B ² .

In this invention, we will use the improved BCS-SPL for graph recovery reconstruction.

1. A video codec structure framework based on multi-prediction assumptions and distributed compressed sensing

As shown in Figure 1, the left side of the dotted line represents the encoding end, and the right side represents the decoding end. Split the video stream at the encoding end, and block key frames and non-key frames.

Let x ₁ denote a keyframe. The method for image reconstruction of keyframes at the decoder is:

(1) Using the block CS image reconstruction method based on smooth projection Landweber (BCS _- SPL) to perform initial image reconstruction on the key frame x1;

(2) Initially reconstruct the image through the measurement value y ₁ of the key frame of the image, and obtain the predicted frame through multiple hypothesis prediction

(3) Regarding the multi-hypothesis prediction generation x ₁ and signal residual

(4) Because the measured value y ₁ can be simply obtained by the inner product of key frame information x ₁ and its measurement matrix Φ ₁ , the measured value D ₁ is obtained by mapping the residual signal R ₁ into the measurement basis:

Using the block CS image reconstruction method based on smooth projection Landweber (BCS-SPL) algorithm to reconstruct the measured value D ₁ to obtain the initial residual signal

Keyframe x ₁ can be predicted by frame Residual signal R ₁ approximate representation with the initial residual signal

Let _x2 denote _a non-keyframe, which can be decoded by the side information produced by keyframe x1.

As shown in Figure 2, the side information SI is obtained by the multi-hypothesis prediction MH algorithm, then the relationship between the non-key frame residual signal R ₂ and the non-key frame x ₂ and the side information SI is:

D ₂ =Φ ₂ R ₂ =Φ ₂ (x ₂ -SI)=y ₂ -Φ ₂ ·SI (6)

Among them, Φ2 and y2 represent the measurement matrix and measurement value of the non _- keyframe _x2 , respectively.

Similarly, when using the BCS-SPL algorithm to reconstruct the measured value D ₂ to the reconstructed residual side information

Non-key frame x ₂ can be reconstructed from side information and residual side information:

2. Using multiple prediction assumptions to estimate side information in the measurement domain

As shown in Figure 1, the image reconstruction quality of non-keyframes strongly depends on the quality of generated side information. To make full use of the similarity between two consecutive keyframes and non-keyframes, the proposed side information generation algorithm based on MH in the measurement domain is shown in Fig. 2.

make with Two reconstructed key frames adjacent to each other in the time domain, Sn is a non-key frame, the algorithm is as follows:

(1) Obtain the initial side information SI by Bidirectional Motion Estimation (BiME).

(2) Make multiple prediction assumptions from the initial side information SI and the measured non-key frame Sn, and get the side information SI ₀

(3) Similarly, from the initial side information SI and the key frame with Side information SI ₁ and SI ₂ are obtained by making multiple prediction assumptions respectively.

(4) Calculate the similarity between the obtained three candidate side information SI _i (i=0, 1, 2) and the non-key frame Sn, and select the SI _i with the highest similarity to reconstruct the non-key frame.

(5) Use the correlation coefficient function r(y ₁ ,y ₂ ) to represent the correlation between two frames, the function is as follows:

where y1 and y2 are the measurement vectors of different blocks measured by the block, and N is the length of the measurement vector.

3. For the step of using side information SI to make multiple assumptions, predict and reconstruct non-key frames, give the following detailed answers.

(1) Let x denote the original image, Represents the predicted image. Then about x and The residual signal R between can be expressed as

(2) In the measurement domain, the residual signal R can be given by the formula Calculate.

(3) Through an image compression sensing reconstruction algorithm R(D,Φ), the formula get an approximate reconstructed image Therefore, the reconstructed image restoration quality depends heavily on the predicted image the accuracy.

The problem of predicting the image similarity that is most similar to the original image can be formulated as:

where p(X _ref ) is adjacent keyframes or side information generated by motion estimation.

Since the original image is unknown at the decoder, we use approximate to replace x, and (8) can be rewritten as

approximate image can be converted to the measurement domain and computed as:

Since the measurement y is available at the decoder, we can improve the prediction accuracy. Equation (10) can be solved by multi-hypothesis forecasting.

Each block that needs to be predicted is considered to be the side information in the key frame or the best linear combination of multiple key frame seed blocks, denoted as

where ω is the optimal linear combination coefficient, is a matrix B ² ×MB matrix composed of all candidate blocks, M is the total number of assumed prediction blocks, The middle column vector is the column representation of each hypothesized prediction block, substituting (10) into (9) and getting:

which is Penalty term, λ is the Lagrangian parameter. Γ is a diagonal matrix expressed as:

h1,...,hk yes column elements. For each block, then ω can be directly computed by the usual Tikhonov solution:

By substituting (13) into (11), the prediction block can be obtained Finally, all predicted blocks are put together with side information SI to reconstruct non-key frames.

Verification experiment: In order to evaluate the new framework and algorithm proposed by the present invention, a standard test video sequence detection experiment with QCIF format is done in the http://trace.eas.asu.edu/yuv/ website.

The sampling rate of key frames is 0.7, and the sampling rate of non-key frames is 0.1 to 0.5; in this experiment, the size of each image block B is 16×16, and the image search area corresponding to the reference frame is: the spatial window size w (image block and its surroundings) within ±15 pixels.

Using the algorithm proposed by the present invention and the original MH-BCS-SPL algorithm, the average value of the average PSNR performance is measured for the different sampling rates of the four sequences (ie Akiyo, Coastguard, Foreman and Stefan), and the results are shown in Table 1.

Table 1. Non-keyframe reconstruction quality at different sampling rates described by average PSNR (dB)

(Unit: dB).

Conclusions: The data in Table 1 describe the non-keyframe reconstruction quality of different videos at different sampling rates. Compared with the MH-BCS-SPL algorithm, the algorithm proposed by the invention has 0.3-1dB improvement in reconstruction quality. For the Akiyo sequence with gentle motion and the Coastguard sequence with less violent motion, the algorithm proposed by the present invention improves about 1dB; for the Foreman and Stefan sequences with severe motion, the algorithm proposed by the present invention improves about 0.3dB.

From Fig. 3-6, we can see that our improved MH-BCS-SPL framework provides better image reconstruction quality in the whole test range. For sequences with fast or complex motions, such as Coastguard and Foreman sequences, our proposed method shows significant performance gain; for Akiyo sequences with low motion, the performance is also improved.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, any person familiar with the technical field within the technical scope disclosed in the present invention, according to the technical solution of the present invention Any equivalent replacement or change of the inventive concepts thereof shall fall within the protection scope of the present invention.

Claims (5)

1. a kind of compressed sensing method for processing video frequency for assuming prediction more, the structural framing of the processing method include coding side and Decoding end, it is characterised in that：In the coding side, video sequence frame is divided into key frame and non-key frame, according to compressed sensing Theory, key frame and non-key frame obtain measured value by calculation matrix Φ；In the decoding end, key frame, which passes through, is based on block The Landweber algorithm for reconstructing smoothly projected is decoded, and after then being rebuild through excessive hypothesis prediction algorithm and residual error, obtains weight Key frame and storage after building；Non-key frame is carried out after residual error reconstruction, combines solution together with the side information produced according to key frame Code, the non-key frame after being rebuild.Finally, decoded key frame and non-key frame are integrated into video sequence according to frame sequential Arrange and export.

2. a kind of compressed sensing method for processing video frequency for assuming prediction according to claim 1, it is characterised in that described more Side information assumes that prediction MH algorithms are tried to achieve according to decoded adjacent key frame through excessive.

3. a kind of compressed sensing method for processing video frequency for assuming prediction according to claim 1, it is characterised in that described more The method and steps for assuming prediction MH algorithms as follows more：

(1) three candidate side information SIi (i=0,1,2) are calculated with bidirectional-movement estimation；

(2) coefficient correlation of non-key frame and three candidate side information is calculated respectively, chooses correlation highest SI information；

(3) in Image Reconstruction, in measurement field, a kind of signal residual error is generated using many hypothesis prediction of SI information, and calculate vacation If optimum linear combination, perceive MS-BCS-SPL technology reengineering images with improved multiple dimensioned splits' positions.

4. a kind of compressed sensing method for processing video frequency for assuming prediction according to claim 3 more, it is characterised in that according to Weight the optimum linear combination that regularization Tikhonov matrix computations are assumed.

5. a kind of compressed sensing method for processing video frequency for assuming prediction according to claim 1-4 are applied to video image more Processing.