CN114519668A - Estimation method for just noticeable distortion threshold of top-down natural image - Google Patents

Abstract

The invention discloses a top-down natural image perceptible distortion threshold estimation method, which divides and quantizes a grayscale image of a source image to obtain a vectorized matrix; obtains a covariance matrix and a covariance matrix of the vectorized matrix. The eigenvalues and eigenvectors of the variance matrix, arrange the eigenvectors in descending order of eigenvalues to obtain the KLT kernel; calculate the KLT coefficient matrix, the KLT coefficient energy, the normalized KLT coefficient energy, the cumulative normalized KLT coefficient energy, and Calculate the perceptual distortion-free critical point according to the derived perceptual distortion-free critical point calculation equation; construct the perceptual distortion-free coefficient reconstruction matrix, and reconstruct the perceptual distortion-free coefficient matrix; convert each dimension vector in the perceptual distortion-free coefficient matrix into an image block and recreate it By splicing together, the perceptual distortion-free critical image is obtained, and then the perceptible distortion threshold map is obtained; the advantage is that it can well reflect the visual masking characteristics of the human visual system, and can well describe the visual perception redundancy of natural images.

Description

A top-down method for estimating just-perceptible distortion thresholds in natural images

technical field

The invention relates to a natural image just noticeable distortion (Just Noticeable Distortion, JND) threshold estimation technology, in particular to a top-down natural image just noticeable distortion threshold estimation method, which is based on a top-down design The idea is to use the KLT (Karhunen-Loéve Transform) transform technology to realize the estimation of the perceptible distortion threshold of natural images.

Background technique

Just Noticeable Distortion (JND) refers to the maximum change amplitude of the visual signal that cannot be perceived by the Human Visual System (HVS). It reflects the sensitivity of the human visual system (HVS) to changes in visual information and the potential perceptual redundancy in visual signals. This makes it widely used in many image/video perceptual processing tasks, including image/video compression, image/video enhancement, information hiding, and image/video evaluation, etc. Just because of its wide application, the Just Noticeable Distortion (JND) threshold estimation of natural images has received extensive attention and research.

Existing just perceptible distortion (JND) threshold estimation models can be divided into two categories: just perceptible distortion (JND) threshold estimation models based on pixel domain and just perceptible distortion (JND) threshold estimation models based on transform domain. The just-noticeable distortion (JND) threshold estimation model based on pixel domain mainly considers factors such as Luminance Adaption (LA), Contrast Masking (CM) and Pattern Complexity (PC). Transform Domain-Based Just Noticeable Distortion (JND) Threshold Estimation Model transforms the image to a specific transform domain and estimates the Just Noticeable Distortion (JND) threshold for each subband, which mainly considers the contrast sensitivity function (Contrast SensitivityFunction, CSF) and other factors. From the design point of view, the existing pixel domain-based just perceptible distortion (JND) threshold estimation model is roughly the same as the transform domain-based just perceptible distortion (JND) threshold estimation model. Visual masking effects with different influences (such as LA, CM, PC, CSF) are modeled, and then the different visual masking effect models are fused to obtain the final Just Noticeable Distortion (JND) threshold estimation model. Such a design idea can be regarded as a bottom-up strategy, that is, starting from several influencing factors that contribute to the bottom, and deriving the final Just Noticeable Distortion (JND) threshold estimation model. However, such a design approach has some inherent limitations: first, it is difficult to take into account all potentially relevant influencing factors due to the lack of a deep and comprehensive understanding of the properties of the human visual system (HVS); second, it is considered The incoming influencing factors are often difficult to be accurately characterized by simple mathematical models; third, the interrelationships between different influencing factors are also difficult to model. Therefore, the existing Just Noticeable Distortion (JND) threshold estimation models are often difficult to achieve satisfactory results, although one can explore more influencing factors and the corresponding visual masking effects through experiments, and at the same time use more accurate mathematical Models model them, but such work is endless. Therefore, how to overcome these shortcomings at the same time and design a more advanced just-detectable distortion (JND) threshold estimation model is of great significance.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a top-down natural image perceptible distortion threshold estimation method, which can well reflect the visual masking characteristics of the human visual system and can well describe the visual appearance of natural images. Perceptual redundancy, which in turn can provide effective guidance for various visual signal perceptual processing tasks.

The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a top-down natural image just perceptible distortion threshold estimation method, which is characterized by comprising the following steps:

Step 1: take a natural image to be processed as the source image; then convert the source image into a grayscale image, denoted as I _Y ; wherein, the source image is an RGB color image, and the widths of the source image and I _Y are both W and height is H;

的图像块；然后对I_Y中的每个图像块进行向量化处理，得到I_Y中的每个图像块对应的列向量，将I_Y中的第n个图像块对应的列向量记为x_n；再将I_Y中的所有图像块对应的列向量拼接构成一个向量化矩阵，记为X，X＝[x₁,x₂,…,x_n,…,x_Num]；其中，

设定W和H均能够被

整除，K的取值为4²或5²或6²或7²或8²或9²或10²，1≤n≤Num，x₁表示I_Y中的第1个图像块对应的列向量，x₂表示I_Y中的第2个图像块对应的列向量，x_Num表示I_Y中的第Num个图像块对应的列向量，x₁、x₂、x_n、x_Num的维数均为K×1，X的维数为K×Num，符号“[]”为向量或矩阵的表示形式；Step 2: Divide I _Y into Num non-overlapping sizes.

Then, vectorize each image block in I _Y to obtain the column vector corresponding to each image block in I _Y , and denote the column vector corresponding to the nth image block in I _Y as x _n ; splicing the column vectors corresponding to all image blocks in I _Y to form a vectorized matrix, denoted as X, X=[x ₁ , x ₂ ,...,x _n ,...,x _Num ]; wherein,

Setting both W and H can be

Divisible, the value of K is 4 ² or 5 ² or 6 ² or 7 ² or 8 ² or 9 ² or 10 ² , 1≤n≤Num, x ₁ represents the column vector corresponding to the first image block in I _Y , x ₂ represents the column vector corresponding to the second image block in I _Y , x _Num represents the column vector corresponding to the Num th image block in I _Y , and the dimensions of x ₁ , x ₂ , x _n , and x _Num are all is K×1, the dimension of X is K×Num, and the symbol “[]” is the representation of a vector or matrix;

Step 3: Calculate the covariance matrix of X, denoted as C; then use the eigenvalue decomposition technology to process C to obtain the K eigenvalues of C and the corresponding K eigenvectors; then the K eigenvectors of C according to the corresponding The K eigenvalues of C are sorted in descending order from large to small, and the matrix formed by the K eigenvectors of C according to their sorting results is used as the prior information extracted from X; among them, the dimension of C is K×K , the feature vector is a column vector, the dimension of the feature vector is K×1, each column of the prior information extracted from X is a feature vector of C, and the dimension of the prior information extracted from X is K×K;

Step 4: The prior information extracted from X is used as the KLT kernel of I _Y , denoted as P; then according to P and X, the KLT coefficient matrix of I _Y is calculated, denoted as Q, Q=(P) ^T X; Then Q is expressed as Q=[q ₁ , q ₂ ,...,q _k ,...,q _K ] ^T ; wherein, the dimension of P is K×K, the dimension of Q is K×Num, 1≤k≤ K, q ₁ represents the 1st dimension KLT spectral component in Q, q ₂ represents the 2nd dimension KLT spectral component in Q, q _k represents the kth dimension KLT spectral component in Q, q _K represents the Kth dimension in Q For KLT spectral components, the dimensions of q ₁ , q ₂ , q _k , and q _K are all Num×1;

然后计算Q中的每一维KLT谱分量的归一化KLT系数能量，将q_k的归一化KLT系数能量记为

再计算Q中的每一维KLT谱分量的累积归一化KLT系数能量，将q_k的累积归一化KLT系数能量记为

最后将Q中的所有KLT谱分量的累积归一化KLT系数能量组成累积归一化KLT系数能量向量，记为E^cum，

其中，q_k(n)表示q_k中的第n个元素的值，1≤ζ≤K，E_ζ表示Q中的第ζ维KLT谱分量q_ζ的KLT系数能量，

表示q₁的归一化KLT系数能量，

表示q₂的归一化KLT系数能量，E^cum的维数为1×K，

表示q₁的累积归一化KLT系数能量，

表示q₂的累积归一化KLT系数能量，

表示q_K的累积归一化KLT系数能量；Step 5: Calculate the KLT coefficient energy of each dimension KLT spectral component in Q, and denote the KLT coefficient energy of q _k as E _k ,

Then calculate the normalized KLT coefficient energy of each dimension KLT spectral component in Q, and denote the normalized KLT coefficient energy of q _k as

Then calculate the cumulative normalized KLT coefficient energy of each dimension KLT spectral component in Q, and denote the cumulative normalized KLT coefficient energy of q _k as

Finally, the cumulative normalized KLT coefficient energy of all KLT spectral components in Q is composed of cumulative normalized KLT coefficient energy vector, denoted as E ^cum ,

Among them, q _k (n) represents the value of the nth element in q _k , 1≤ζ≤K, E _ζ denotes the KLT coefficient energy of the ζth dimension KLT spectral component q _ζ in Q,

represents the normalized KLT coefficient energy of _q1 ,

represents the normalized KLT coefficient energy of q ₂ , E ^cum has a dimension of 1 × K,

represents the cumulative normalized KLT coefficient energy of _q1 ,

represents the cumulative normalized KLT coefficient energy of _q2 ,

represents the cumulative normalized KLT coefficient energy of q _K ;

接着采用

重建得到感知无失真系数矩阵，记为

再将

表示为

其中，L为正整数，1≤L≤K，

的维数为K×Num，

中的“＝”为赋值符号，q_L表示Q中的第L维KLT谱分量，q_L的维数为Num×1，

至

均为全0向量，

的维数均为Num×1，

的维数为K×Num，

表示

中的第1维感知无失真系数向量，

表示

中的第2维感知无失真系数向量，

表示

中的第n维感知无失真系数向量，

表示

中的第Num维感知无失真系数向量，

的维数均为K×1；Step 6: Substitute E ^cum as the input into the perceptual distortion-free critical point calculation model, and calculate the perceptual distortion-free critical point of I _Y , denoted as L; then construct the perceptual distortion-free coefficient reconstruction matrix according to L, denoted as

Then use

Reconstruction to get the perceptual distortion-free coefficient matrix, denoted as

again

Expressed as

Among them, L is a positive integer, 1≤L≤K,

The dimension of is K × Num,

The "=" in is the assignment symbol, q _L represents the L-th dimension KLT spectral component in Q, and the dimension of q _L is Num×1,

to

are all 0 vectors,

The dimensions of are all Num × 1,

The dimension of is K × Num,

express

The 1st-dimensional perceptual distortion-free coefficient vector in ,

express

The 2nd-dimensional perceptual distortion-free coefficient vector in ,

express

The nth-dimensional perceptual distortion-free coefficient vector in ,

express

The Num-th dimension perceptually undistorted coefficient vector in ,

The dimensions of are K × 1;

中的每一维感知无失真系数向量转换成尺寸大小为

的图像块，将

转换成的图像块作为第n个图像块；然后将

中的所有感知无失真系数向量转换成的图像块拼接成图像作为感知无失真临界图像，记为I_L；再根据I_Y和I_L，计算恰可察觉失真阈值图，记为M，将M中坐标位置为(a,b)的像素点的像素值记为M(a,b)，M(a,b)＝|I_Y(a,b)-I_L(a,b)|；其中，1≤a≤W,1≤b≤H，I_Y(a,b)表示I_Y中坐标位置为(a,b)的像素点的像素值，I_L(a,b)表示I_L中坐标位置为(a,b)的像素点的像素值，符号“| |”为取绝对值符号。Step 7: According to the inverse operation of the vectorization process in step 2, the

Each dimension in the vector of perceptually undistorted coefficients is converted to size as

image block, will

The converted image block is taken as the nth image block; then the

All the image blocks converted into perceptual distortion-free coefficient vectors in the image are spliced into an image as the perceptual distortion-free critical image, denoted as _IL ; and then according to I _Y and _IL , calculate the perceptible distortion threshold map, denoted as M, and M The pixel value of the pixel point whose middle coordinate position is (a,b) is denoted as M(a,b), M(a,b)=|I _Y (a, b)-I _L (a, b)|; where , 1≤a≤W, 1≤b≤H, I _Y (a, b) represents the pixel value of the pixel whose coordinate position is (a, b) in I _Y , and I _L (a, b) represents the pixel value in I _L The pixel value of the pixel point whose coordinate position is (a, b), the symbol "| |" is the symbol of taking the absolute value.

In the step 2, the acquisition process of x _n is: arranging the pixel values of all pixel points in the n-th image block in I _Y into a column in a zigzag scanning manner to form x _n .

其中，上标“T”表示向量或矩阵的转置，

表示对X按行取均值得到的均值向量，

的维数为K×1。In the described step 3, the calculation formula of C is:

where the superscript "T" represents the transpose of a vector or matrix,

represents the mean vector obtained by taking the row-wise mean of X,

The dimension is K × 1.

In the step 4, P=[p ₁ , p ₂ ,...,p _K ], where p ₁ indicates that the K eigenvectors of C are sorted in descending order of the corresponding K eigenvalues After the first eigenvector, p ₂ represents the second eigenvector after sorting the K eigenvectors of C in descending order of the corresponding K eigenvalues, p _K represents the K eigenvectors of C The eigenvectors are the Kth eigenvector after the corresponding K eigenvalues are sorted in descending order, and the dimensions of p ₁ , p ₂ , and p _K are all K×1.

In the described step 6, the acquisition process of the perceptual distortion-free critical point calculation model is as follows:

整除，高清图像的灰度图像分割的图像块的尺寸大小为

1≤i≤S，Q'_i的维数为K×Num'，Num'表示高清图像的灰度图像分割的图像块的总个数，

q'_i,1表示Q'_i中的第1维KLT谱分量，q'_i,2表示Q'_i中的第2维KLT谱分量，q'_i,k表示Q'_i中的第k维KLT谱分量，q'_i,K表示Q'_i中的第K维KLT谱分量，q'_i,1、q'_i,2、q'_i,k、q'_i,K的维数均为Num'×1；Step 6_1: Select S high-definition images, and convert each high-definition image into a grayscale image; then follow the process from steps 2 to 4 to obtain the KLT coefficient matrix of the grayscale image of each high-definition image in the same way, and convert the first The KLT coefficient matrix of the grayscale image of i high-definition images is denoted as Q' _i , and Q' _i is represented as Q' _i =[q' _i,1 ,q' _i,2 ,...,q' _i,k ,... ,q'i _,K ] ^T ; wherein, S≥100, the high-definition image is an RGB color image, the width of the high-definition image is W' and the height is H', and both W' and H' can be

Divide evenly, the size of the image block divided by the grayscale image of the high-definition image is

1≤i≤S, the dimension of Q' _i is K×Num', and Num' represents the total number of image blocks divided by the grayscale image of the high-definition image,

q' _i,1 represents the first dimension KLT spectral component in Q' _i , q' _i,2 represents the second dimension KLT spectral component in Q' _i , q' _i,k represents the kth dimension in Q' _i KLT spectral components, q' _i,K represents the K-th dimension KLT spectral components in Q' _i , the dimensions of q' _i,1 , q' _i,2 , q' _i,k , q' _i,K are all Num'×1;

然后重建每幅高清图像的灰度图像对应的K个感知系数矩阵，将第i幅高清图像的灰度图像对应的第k个感知系数矩阵记为

将

表示为

其中，

的维数为K×Num'，

中的“＝”为赋值符号，

至

均为全0向量，

的维数均为Num'×1，

的维数为K×Num'，P'_i表示第i幅高清图像的灰度图像的KLT核，P'_i的维数为K×K，1≤n'≤Num'，n'为正整数，

表示

中的第1维感知系数向量，

表示

中的第2维感知系数向量，

表示

中的第n'维感知系数向量，

表示

中的第Num'维感知系数向量，

的维数均为K×1；Step 6_2: Construct K perceptual coefficient reconstruction matrices corresponding to the grayscale image of each high-definition image, and record the kth perceptual coefficient reconstruction matrix corresponding to the grayscale image of the ith high-definition image as

Then the K perceptual coefficient matrices corresponding to the grayscale image of each high-definition image are reconstructed, and the kth perceptual coefficient matrix corresponding to the grayscale image of the ith high-definition image is recorded as

Will

Expressed as

in,

The dimension of is K × Num',

The "=" in it is an assignment symbol,

to

are all 0 vectors,

The dimensions of are all Num'×1,

The dimension of P'i is K×Num', P'i represents the KLT kernel of the grayscale image of the _ith high-definition image, the dimension of P'i is K×K, 1≤n'≤Num', _n ' is a positive integer ,

express

The 1st-dimensional perceptual coefficient vector in ,

express

The 2nd-dimensional perceptual coefficient vector in ,

express

The n'-dimensional perceptual coefficient vector in ,

express

The Num'-dimensional perceptual coefficient vector in ,

The dimensions of are K × 1;

的图像块；然后将每幅高清图像的灰度图像对应的每个感知系数矩阵中的所有感知系数向量转换成的图像块拼接成一幅图像，将第i幅高清图像的灰度图像对应的第k个感知系数矩阵

中的所有感知系数向量转换成的图像块拼接成的图像作为第i幅高清图像的灰度图像对应的第k幅重建图像，记为

其中，

的宽度为W'且高度为H'；Step 6_3: According to the inverse operation of the vectorization process in step 2, convert each dimension of the perceptual coefficient vector in each perceptual coefficient matrix corresponding to the grayscale image of each high-definition image into a size of

Then splicing the image blocks converted from all perceptual coefficient vectors in each perceptual coefficient matrix corresponding to the grayscale image of each high-definition image into one image, and splicing the first image corresponding to the grayscale image of the ith high-definition image. k perceptual coefficient matrices

The image spliced into the image blocks converted into all perceptual coefficient vectors in the ith high-definition image is the kth reconstructed image corresponding to the grayscale image of the ith high-definition image, denoted as

in,

has a width of W' and a height of H';

那么将

作为第d位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界图像，同时将数值k作为第d位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点，记为

然后将所有志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点构成的向量记为J_i，

其中，D＞1，1≤d≤D，

中的“＝”为赋值符号，J_i的维数为1×D，

表示第1位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点，

表示第2位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点，

表示第D位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点；Step 6_4: Summon D volunteers, each volunteer compares the grayscale image of each high-definition image and its corresponding reconstructed images in turn by visual observation, and each volunteer corresponds to the grayscale image of each high-definition image. Among the K reconstructed images, a reconstructed image is determined as the perceptual distortion-free critical image corresponding to the grayscale image, and the sequence number of the determined reconstructed image is taken as the perceptual distortion-free critical point corresponding to the grayscale image; and the grayscale image of the i-th high-definition image, the d-th volunteer compares the gray-scale image of the i-th high-definition image with its corresponding first reconstructed image, the second reconstructed image, ... , The K-th reconstructed image, once the d-th volunteer cannot distinguish the grayscale image of the i-th high-definition image from one of the corresponding reconstructed images, the comparison process is stopped, assuming that the reconstructed image is the gray-scale of the i-th high-definition image. The kth reconstructed image corresponding to the degree image

then will

As the perceptual distortion-free critical image corresponding to the grayscale image of the i-th high-definition image observed by the d-th volunteer, and taking the value k as the perceptually undistorted image corresponding to the gray-scale image of the i-th high-definition image observed by the d-th volunteer Distortion critical point, denoted as

Then the vector formed by the perceptual distortion-free critical points corresponding to the grayscale image of the ith high-definition image observed by all volunteers is denoted as J _i ,

Among them, D>1, 1≤d≤D,

The "=" in is the assignment symbol, and the dimension of J _i is 1×D,

represents the perceptual distortion-free critical point corresponding to the grayscale image of the i-th high-definition image observed by the first volunteer,

represents the perceptual distortion-free critical point corresponding to the grayscale image of the i-th high-definition image observed by the second volunteer,

represents the perceptual distortion-free critical point corresponding to the grayscale image of the i-th high-definition image observed by the D-th volunteer;

和

然后在所有志愿者观察下每幅高清图像的灰度图像对应的感知无失真临界点构成的向量中剔除离群值，对于J_i，若

不满足

则判定

为离群值，将

从J_i中剔除，将剔除离群值后得到的向量记为

Step 6_5: Calculate the mean and standard deviation of all perceptual distortion-free critical points in the vector formed by the perceptual distortion-free critical points corresponding to the grayscale images of each high-definition image observed by all volunteers, and calculate all perceptual distortion-free critical points in J _i . The mean and standard deviation of the critical point are recorded as

and

Then, outliers are eliminated from the vector formed by the perceptual distortion-free critical points corresponding to the grayscale images of each high-definition image observed by all volunteers. For J _i , if

not satisfied

then judge

is an outlier, the

Remove from J _i , and record the vector obtained after removing outliers as

中的所有感知无失真临界点的均值记为

然后获取每幅高清图像的灰度图像对应的感知无失真临界点，将第i幅高清图像的灰度图像对应的感知无失真临界点记为J_i，

其中，符号

为向上取整运算符号；Step 6_6: Calculate the average value of all perceptual distortion-free critical points after removing outliers from the vector of the perceptual distortion-free critical points corresponding to the grayscale images of each high-definition image observed by all volunteers, and set the

The mean of all perceptually distortion-free critical points in

Then, the perceptual distortion-free critical point corresponding to the grayscale image of each high-definition image is obtained, and the perceptual distortion-free critical point corresponding to the grayscale image of the ith high-definition image is recorded as J _i ,

Among them, the symbol

to round up the operator symbol;

然后计算每幅高清图像的灰度图像的KLT系数矩阵中的每一维KLT谱分量的归一化KLT系数能量，将Q'_i中的q'_i,k的归一化KLT系数能量记为

再计算每幅高清图像的灰度图像对应的感知无失真临界点处的累积归一化KLT系数能量，将第i幅高清图像的灰度图像对应的感知无失真临界点J_i处的累积归一化KLT系数能量记为

最后将所有高清图像的灰度图像对应的感知无失真临界点处的累积归一化KLT系数能量构成一个向量，记为U^cum，

其中，q'_i,k(n')表示q'_i,k中的第n'个元素的值，1≤ζ≤K，U_i,ζ表示Q'_i中的第ζ维KLT谱分量q'_i,ζ的KLT系数能量，

表示Q'_i中的q'_i,1的归一化KLT系数能量，

表示Q'_i中的q'_i,2的归一化KLT系数能量，

表示Q'_i中的

的归一化KLT系数能量，

表示Q'_i中的第J_i维KLT谱分量，U^cum的维数为1×S，

表示第1幅高清图像的灰度图像对应的感知无失真临界点J₁处的累积归一化KLT系数能量，

表示第2幅高清图像的灰度图像对应的感知无失真临界点J₂处的累积归一化KLT系数能量，

表示第S幅高清图像的灰度图像对应的感知无失真临界点J_S处的累积归一化KLT系数能量；Step 6-7: Calculate the KLT coefficient energy of each dimension KLT spectral component in the KLT coefficient matrix of the grayscale image of each high-definition image, and denote the KLT coefficient energy of q' _i,k in Q' _i as U _i,k ,

Then calculate the normalized KLT coefficient energy of each dimension KLT spectral component in the KLT coefficient matrix of the grayscale image of each high-definition image, and denote the normalized KLT coefficient energy of q' _{i, k} in Q' _i as

Then calculate the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point corresponding to the grayscale image of each high-definition image, and calculate the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point J _i corresponding to the grayscale image of the ith high-definition image. The energy of the normalized KLT coefficient is recorded as

Finally, the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point corresponding to the grayscale images of all high-definition images is formed into a vector, denoted as U ^cum ,

Among them, q' _i,k (n') represents the value of the n'th element in q' _i,k , 1≤ζ≤K, U _{i ,ζ} denotes the ζth dimension KLT spectral component q in Q'i ' _{i, the KLT coefficient energy of ζ} ,

represents the normalized KLT coefficient energy of q' _i,1 in Q' _i ,

represents the normalized KLT coefficient energy of q' _i,2 in Q' _i ,

means that in Q' _i

The normalized KLT coefficient energy of ,

represents the J _i -th dimension KLT spectral component in Q' _i , the dimension of U ^cum is 1×S,

represents the cumulative normalized KLT coefficient energy at the perceptual distortion _- free critical point J1 corresponding to the grayscale image of the first high-definition image,

represents the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point J ₂ corresponding to the grayscale image of the second high-definition image,

represents the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point J _S corresponding to the grayscale image of the S-th high-definition image;

和

然后根据

和

得到感知无失真临界点计算模型，描述为：

其中，L表示I_Y的感知无失真临界点。Step 6_8: Calculate the mean and standard deviation of all cumulative normalized KLT coefficient energies in U ^cum , corresponding to

and

then according to

and

The perceptual distortion-free critical point calculation model is obtained, which is described as:

where L represents the perceptual distortion-free critical point of I _Y.

In the described step _{6_1} , the acquisition process of Q'i is:

的图像块；然后对I'_Y,i中的每个图像块进行向量化处理，得到I'_Y,i中的每个图像块对应的列向量，将I'_Y,i中的第n'个图像块对应的列向量记为x'_i,n'；再将I'_Y,i中的所有图像块对应的列向量拼接构成一个向量化矩阵，记为X'_i，X'_i＝[x'_i,1,x'_i,2,…,x'_i,n',…,x'_i,Num']；其中，

K的取值为4²或5²或6²或7²或8²或9²或10²，1≤n'≤Num'，x'_i,1表示I'_Y,i中的第1个图像块对应的列向量，x'_i,2表示I'_Y,i中的第2个图像块对应的列向量，x'_i,Num'表示I'_Y,i中的第Num'个图像块对应的列向量，x'_i,1、x'_i,2、x'_i,n'、x'_i,Num'的维数均为K×1，X'_i的维数为K×Num'；Step 6_1a: mark the grayscale image of the i-th high-definition image as I' _{Y, i} ; then divide I' _{Y, i} into Num' non-overlapping sizes as

Then, vectorize each image block in I' _{Y, i to} obtain the column vector corresponding to each image block in I' _{Y, i} , and convert the The column vectors corresponding to the image blocks are denoted as x'_i,n'; then the column vectors corresponding to all the image blocks in I' _{Y, i} are spliced to form a vectorized matrix, denoted as X' _i , X' _i =[ x' _i,1 ,x' _i,2 ,…,x'_i,n',…,x'_i,Num']; where,

The value of K is 4 ² or 5 ² or 6 ² or 7 ² or 8 ² or 9 ² or 10 ² , 1≤n'≤Num', x' _i,1 means the first one in I' _Y,i The column vector corresponding to the image block, x' _i,2 represents the column vector corresponding to the second image block in I' _Y,i , x'_i,Num' represents the Num'th image block in I' _Y,i Corresponding column vector, the dimensions of x' _i,1 , x' _i,2 , x'_i,n' , x'_i,Num' are all K×1, and the dimension of X' _i is K×Num';

Step 6-1b: calculate the covariance matrix of _X'i , denoted as _C'i ; then utilize eigenvalue decomposition technology to process _C'i , obtain K eigenvalues of _C'i and corresponding K eigenvectors; The K eigenvectors of _C'i are sorted in descending order of the corresponding K eigenvalues, and the matrix formed by the sorting results of the K eigenvectors of _C'i is used as the matrix extracted from _X'i Prior information; wherein, the dimension of C' _i is K×K, the feature vector is a column vector, the dimension of the feature vector is K×1, and each column of the prior information extracted from X' _i is C 1 feature vector of ' _i , the dimension of the prior information extracted from X' _i is K×K;

Step 6_1c: The prior information extracted from X' _i is used as the KLT kernel of I' _Y,i , denoted as P'_i; then according to P' _i and X' _i , calculate the KLT coefficient of I' _Y,i Matrix Q' _i , Q' _i =(P' _i ) ^T X' _i , wherein the dimension of P' _i is K×K, and the dimension of Q' _i is K×Num'.

Compared with the prior art, the advantages of the present invention are:

1) The method of the present invention starts from the definition of just perceptible distortion, and converts the problem of threshold estimation of perceptible distortion into the estimation problem of perceptual distortion-free critical image, and the perceptual distortion-free critical image refers to a distorted image in which the human visual system just cannot detect the distortion. , the perceptual distortion-free critical image is still distorted compared with the source image, but its distortion cannot be perceived by the human eye from the perceptual level. Therefore, the source image can be subtracted from the perceptual distortion-free critical image to obtain the corresponding source image. This method can effectively avoid the errors caused by the complex and inaccurate HVS visual masking factor modeling and fusion of the traditional just-observable distortion threshold estimation model, so that it can more accurately reflect human The visual masking properties of the visual system and the visual perceptual redundancy that characterize natural images can provide effective guidance for various visual signal perceptual processing tasks. Using the threshold map of perceptible distortion estimated by the method of the present invention to guide image noise addition and JPEG compression, on the premise that the subjective perception quality is almost completely consistent, the method of the present invention can hide more noise and save more bits Rate.

2) The method of the present invention uses the KLT transform technology to estimate the perceptual distortion-free critical image of the source image, does not depend on a specific image processing task, and has better universality. Firstly, KLT transform is performed on the source image, and the perceptual distortion-free critical point of the source image is found according to the convergence characteristics of the accumulated normalized KLT coefficient energy. Perceptual distortion-free critical images, the perceptual distortion-free critical images obtained in this way are universal and can be well applied to different visual information perceptual processing tasks.

Description of drawings

Fig. 1 is the overall flow chart of the method of the present invention;

Figure 2a is the first source image img1;

Figure 2b is the second source image img2;

Figure 2c is the third source image img3;

Figure 2d is the fourth source image img4;

Fig. 2e is the respective cumulative normalized KLT coefficient energy curves corresponding to Fig. 2a, Fig. 2b, Fig. 2c, Fig. 2d;

Figure 3a is the source image I03;

Figure 3b is a perceptible distortion threshold figure obtained by processing the source image shown in Figure 3a by the method of the present invention;

Figure 3c is a noise-added image generated using the guidance of Figure 3b under the condition of PSNR=26dB;

Figure 3d is an enlarged view of the portion inside the frame in Figure 3c;

Fig. 4a is the result that the source image I01 directly adopts JPEG compression;

Fig. 4b is the JPEG compression result that source image I01 adopts the method of the present invention to obtain just the JPEG compression result guided by the perceptible distortion threshold value map;

Figure 4c is the JPEG compression result guided by the perceptible distortion threshold map obtained by the Wu2017 method for the source image I01;

FIG. 5 is the variation curve of the average gain value of the method of the present invention and the method of Wu2017 with the quality factor QP.

Detailed ways

The present invention will be further described in detail below with reference to the embodiments of the accompanying drawings.

A top-down natural image perceptible distortion threshold estimation method proposed by the present invention, the overall flow diagram of which is shown in Figure 1, which includes the following steps:

Step 1: take a natural image to be processed as the source image; then convert the source image into a grayscale image, denoted as I _Y ; wherein, the source image is an RGB color image, and the widths of the source image and I _Y are both W and The height is H, and the calculation formula of the pixel value I _Y (a, b) of the pixel point whose coordinate position is (a, b) in I _Y is: I _Y (a, b)=0.299I _R (a, b) +0.587I _G (a,b)+0.114I _B (a,b), 1≤a≤W, _1≤b≤H , IR (a,b) indicates that the coordinate position in the red channel of the source image is (a , b), I _G (a, b) represents the pixel value of the pixel whose coordinate position is (a, b) in the green channel of the source image, I _B (a, b) represents the source image The pixel value of the pixel whose coordinate position is (a,b) in the blue channel.

的图像块；然后对I_Y中的每个图像块进行向量化处理，得到I_Y中的每个图像块对应的列向量，将I_Y中的第n个图像块对应的列向量记为x_n；再将I_Y中的所有图像块对应的列向量拼接构成一个向量化矩阵，记为X，X＝[x₁,x₂,…,x_n,…,x_Num]；其中，

设定W和H均能够被

整除，K的取值为4²或5²或6²或7²或8²或9²或10²，一般情况下取8²，1≤n≤Num，x₁表示I_Y中的第1个图像块对应的列向量，x₂表示I_Y中的第2个图像块对应的列向量，x_Num表示I_Y中的第Num个图像块对应的列向量，x₁、x₂、x_n、x_Num的维数均为K×1，X的维数为K×Num，符号“[]”为向量或矩阵的表示形式。Step 2: Divide I _Y into Num non-overlapping sizes.

Then, vectorize each image block in I _Y to obtain the column vector corresponding to each image block in I _Y , and denote the column vector corresponding to the nth image block in I _Y as x _n ; splicing the column vectors corresponding to all image blocks in I _Y to form a vectorized matrix, denoted as X, X=[x ₁ , x ₂ ,...,x _n ,...,x _Num ]; wherein,

Setting both W and H can be

Divisible, the value of K is 4 ² or 5 ² or 6 ² or 7 ² or 8 ² or 9 ² or 10 ² , in general, it is 8 ² , 1≤n≤Num, x ₁ represents the first in I _Y Column vector corresponding to each image block, x ₂ represents the column vector corresponding to the second image block in I _Y , x _Num represents the column vector corresponding to the Num th image block in I _Y , x ₁ , x ₂ , x _n The dimensions of , x _Num are both K×1, the dimension of X is K×Num, and the symbol “[]” is the representation of a vector or matrix.

In this embodiment, in step 2, the acquisition process of x _n is: arranging the pixel values of all pixel points in the n-th image block in I _Y into a column in a zigzag scanning manner to form x _n .

In the field of image processing, it is a conventional technical means to perform vectorization processing on image blocks, that is, the pixel values of all pixels in the image block are in a certain order (for example, in the order of line scanning, first scan the first line, and then scan the first line. Two rows, and so on, that is, zigzag scanning mode) are arranged to form a column vector; when multiple column vectors are spliced into a vectorized matrix, they can be spliced in the order of the image blocks, such as the first column of the vectorized matrix. The column vector corresponding to the first image block, and the last column of the vectorized matrix is the column vector corresponding to the Num-th image block.

Step 3: Calculate the covariance matrix of X, denoted as C; then use the existing eigenvalue decomposition technology to process C to obtain K eigenvalues and corresponding K eigenvectors of C; The vectors are sorted in descending order of the corresponding K eigenvalues, and the matrix formed by the K eigenvectors of C according to their sorting results is used as the prior information extracted from X; among them, the dimension of C is K×K, the feature vector is a column vector, the dimension of the feature vector is K×1, each column of the prior information extracted from X is a feature vector of C, the prior information extracted from X The dimension of is K×K.

其中，上标“T”表示向量或矩阵的转置，

表示对X按行取均值得到的均值向量，

的维数为K×1。In this embodiment, in step 3, the calculation formula of C is:

where the superscript "T" represents the transpose of a vector or matrix,

represents the mean vector obtained by taking the row-wise mean of X,

The dimension is K × 1.

Step 4: Use the prior information extracted from X as the KLT (Karhunen-Loéve Transform) kernel of I _Y , denoted as P; then calculate the KLT coefficient matrix of I _Y according to P and X, denoted as Q, Q= (P) ^T X; then express Q as Q=[q ₁ , q ₂ ,...,q _k ,...,q _K ] ^T ; wherein, the dimension of P is K×K, and the dimension of Q is K× Num, 1≤k≤K, q ₁ represents the 1st dimension KLT spectral component in Q, q ₂ represents the 2nd dimension KLT spectral component in Q, q _k represents the kth dimension KLT spectral component in Q, q _K represents For the Kth dimension KLT spectral components in Q, the dimensions of q ₁ , q ₂ , q _k , and q _K are all Num×1.

In this embodiment, in step ₄ , P ₌ [p ₁ , p ₂ , . The first eigenvector after sorting by the way, p ₂ represents the second eigenvector after sorting the K eigenvectors of C in descending order of the corresponding K eigenvalues, p _K represents the The K eigenvectors of k are sorted in descending order of the corresponding K eigenvalues in descending order, and the dimensions of p ₁ , p ₂ , and p _K are all K×1.

然后计算Q中的每一维KLT谱分量的归一化KLT系数能量，将q_k的归一化KLT系数能量记为

再计算Q中的每一维KLT谱分量的累积归一化KLT系数能量，将q_k的累积归一化KLT系数能量记为

最后将Q中的所有KLT谱分量的累积归一化KLT系数能量组成累积归一化KLT系数能量向量，记为E^cum，

其中，q_k(n)表示q_k中的第n个元素的值，1≤ζ≤K，E_ζ表示Q中的第ζ维KLT谱分量q_ζ的KLT系数能量，

表示q₁的归一化KLT系数能量，

表示q₂的归一化KLT系数能量，E^cum的维数为1×K，

表示q₁的累积归一化KLT系数能量，

表示q₂的累积归一化KLT系数能量，

表示q_K的累积归一化KLT系数能量；图2a给出了第1幅源图像img1，图2b给出了第2幅源图像img2，图2c给出了第3幅源图像img3，图2d给出了第4幅源图像img4，图2e给出了图2a、图2b、图2c、图2d各自对应的累积归一化KLT系数能量曲线。从图2e中可以看见，第1维KLT谱分量的累积归一化KLT系数能量是最大的，并且累积归一化KLT系数能量随着KLT谱分量索引的增加而增加。随着KLT谱分量索引的增加，累积归一化KLT系数能量逐渐趋向于饱和，最终变成1。不同图像对应的累积归一化KLT系数能量曲线虽然有相同的趋势但并不完全一致。Step 5: Calculate the KLT coefficient energy of each dimension KLT spectral component in Q, and denote the KLT coefficient energy of q _k as E _k ,

Then calculate the normalized KLT coefficient energy of each dimension KLT spectral component in Q, and denote the normalized KLT coefficient energy of q _k as

Then calculate the cumulative normalized KLT coefficient energy of each dimension KLT spectral component in Q, and denote the cumulative normalized KLT coefficient energy of q _k as

Finally, the cumulative normalized KLT coefficient energy of all KLT spectral components in Q is composed of cumulative normalized KLT coefficient energy vector, denoted as E ^cum ,

Among them, q _k (n) represents the value of the nth element in q _k , 1≤ζ≤K, E _ζ denotes the KLT coefficient energy of the ζth dimension KLT spectral component q _ζ in Q,

represents the normalized KLT coefficient energy of _q1 ,

represents the normalized KLT coefficient energy of q ₂ , E ^cum has a dimension of 1 × K,

represents the cumulative normalized KLT coefficient energy of _q1 ,

represents the cumulative normalized KLT coefficient energy of _q2 ,

Represents the cumulative normalized KLT coefficient energy of q _K ; Fig. 2a shows the 1st source image img1, Fig. 2b shows the 2nd source image img2, Fig. 2c shows the 3rd source image img3, Fig. 2d The fourth source image img4 is given, and Fig. 2e shows the corresponding cumulative normalized KLT coefficient energy curves of Fig. 2a, Fig. 2b, Fig. 2c, and Fig. 2d. It can be seen from Fig. 2e that the cumulative normalized KLT coefficient energy of the 1st dimension KLT spectral components is the largest, and the cumulative normalized KLT coefficient energy increases as the index of the KLT spectral components increases. As the KLT spectral component index increases, the cumulative normalized KLT coefficient energy gradually tends to saturate, eventually becoming 1. Although the cumulative normalized KLT coefficient energy curves corresponding to different images have the same trend, they are not completely consistent.

接着采用

重建得到感知无失真系数矩阵，记为

再将

表示为

其中，L为正整数，1≤L≤K，

的维数为K×Num，

中的“＝”为赋值符号，q_L表示Q中的第L维KLT谱分量，q_L的维数为Num×1，

至

均为全0向量，

的维数均为Num×1，

的维数为K×Num，

表示

中的第1维感知无失真系数向量，

表示

中的第2维感知无失真系数向量，

表示

中的第n维感知无失真系数向量，

表示

中的第Num维感知无失真系数向量，

的维数均为K×1。Step 6: Substitute E ^cum as the input into the perceptual distortion-free critical point calculation model, and calculate the perceptual distortion-free critical point of I _Y , denoted as L; then construct the perceptual distortion-free coefficient reconstruction matrix according to L, denoted as

Then use

Reconstruction to get the perceptual distortion-free coefficient matrix, denoted as

again

Expressed as

Among them, L is a positive integer, 1≤L≤K,

The dimension of is K × Num,

The "=" in is the assignment symbol, q _L represents the L-th dimension KLT spectral component in Q, and the dimension of q _L is Num×1,

to

are all 0 vectors,

The dimensions of are all Num×1,

The dimension of is K × Num,

express

The 1st-dimensional perceptual distortion-free coefficient vector in ,

express

The 2nd-dimensional perceptual distortion-free coefficient vector in ,

express

The nth-dimensional perceptual distortion-free coefficient vector in ,

express

The Num-th dimension perceptually undistorted coefficient vector in ,

The dimensions are all K × 1.

In this embodiment, in step 6, the acquisition process of the perceptual distortion-free critical point calculation model is as follows:

整除，高清图像的灰度图像分割的图像块的尺寸大小为

1≤i≤S，Q'_i的维数为K×Num'，Num'表示高清图像的灰度图像分割的图像块的总个数，

q'_i,1表示Q'_i中的第1维KLT谱分量，q'_i,2表示Q'_i中的第2维KLT谱分量，q'_i,k表示Q'_i中的第k维KLT谱分量，q'_i,K表示Q'_i中的第K维KLT谱分量，q'_i,1、q'_i,2、q'_i,k、q'_i,K的维数均为Num'×1。Step 6_1: Select S high-definition images, and convert each high-definition image into a grayscale image; then follow the process from steps 2 to 4 to obtain the KLT coefficient matrix of the grayscale image of each high-definition image in the same way, and convert the first The KLT coefficient matrix of the grayscale image of i high-definition images is denoted as Q' _i , and Q' _i is represented as Q' _i =[q' _i,1 ,q' _i,2 ,...,q' _i,k ,... ,q'i _,K ] ^T ; wherein, S≥100, in the experiment, S=500 is desirable, the high-definition image is an RGB color image, the width of the high-definition image is W' and the height is H', and both W' and H' can be quilt

Divide evenly, the size of the image block divided by the grayscale image of the high-definition image is

1≤i≤S, the dimension of Q' _i is K×Num', and Num' represents the total number of image blocks divided by the grayscale image of the high-definition image,

q' _i,1 represents the first dimension KLT spectral component in Q' _i , q' _i,2 represents the second dimension KLT spectral component in Q' _i , q' _i,k represents the kth dimension in Q' _i KLT spectral components, q' _i,K represents the K-th dimension KLT spectral components in Q' _i , the dimensions of q' _i,1 , q' _i,2 , q' _i,k , q' _i,K are all Num'×1.

In this embodiment, in step 6_1, the acquisition process of Q' _i is:

的图像块；然后对I'_Y,i中的每个图像块进行向量化处理，得到I'_Y,i中的每个图像块对应的列向量，将I'_Y,i中的第n'个图像块对应的列向量记为x'_i,n'；再将I'_Y,i中的所有图像块对应的列向量拼接构成一个向量化矩阵，记为X'_i，X'_i＝[x'_i,1,x'_i,2,…,x'_i,n',…,x'_i,Num']；其中，

K的取值为4²或5²或6²或7²或8²或9²或10²，一般情况下取8²，1≤n'≤Num'，x'_i,1表示I'_Y,i中的第1个图像块对应的列向量，x'_i,2表示I'_Y,i中的第2个图像块对应的列向量，x'_i,Num'表示I'_Y,i中的第Num'个图像块对应的列向量，x'_i,1、x'_i,2、x'_i,n'、x'_i,Num'的维数均为K×1，X'_i的维数为K×Num'。Step 6_1a: mark the grayscale image of the i-th high-definition image as I' _{Y, i} ; then divide I' _{Y, i} into Num' non-overlapping sizes as

Then, vectorize each image block in I' _{Y, i to} obtain the column vector corresponding to each image block in I' _{Y, i} , and convert the The column vectors corresponding to the image blocks are denoted as x'_i,n'; then the column vectors corresponding to all the image blocks in I' _{Y, i} are spliced to form a vectorized matrix, denoted as X' _i , X' _i =[ x' _i,1 ,x' _i,2 ,…,x'_i,n',…,x'_i,Num']; where,

The value of K is 4 ² or 5 ² or 6 ² or 7 ² or 8 ² or 9 ² or 10 ² , generally 8 ² , 1≤n'≤Num', x' _i,1 means I' _{Y ,} the column vector corresponding to the first image block in i, x' _{i, 2} represents the column vector corresponding to the second image block in I' _{Y, i} , x' _{i, Num'} represents I' _{Y, i} The column vector corresponding to the Num'th image block, the dimensions of x' _i,1 , x' _i,2 , x'_i,n' , x'_i,Num' are all K×1, and the dimensions of X' _i The dimension is K × Num'.

Step _{6_1b} : Calculate the covariance matrix of X'i, denoted as _C'i ; then utilize the existing eigenvalue decomposition technology to process _C'i , and obtain K eigenvalues and corresponding K eigenvectors of _C'i ; Then sort the K eigenvectors of _C'i in descending order of the corresponding K eigenvalues, and take the matrix formed by the K eigenvectors of _C'i according to their sorting results as the matrix from _X'i The extracted prior information; among them, the dimension of _C'i is K×K, the feature vector is a column vector, and the dimension of the feature vector is K×1, and each of the prior information extracted from _X'i is One column is a feature vector of C' _i , and the dimension of the prior information extracted from X' _i is K×K.

Step 6_1c: The prior information extracted from X' _i is used as the KLT kernel of I' _Y,i , denoted as P'_i; then according to P' _i and X' _i , calculate the KLT coefficient of I' _Y,i Matrix Q' _i , Q' _i =(P' _i ) ^T X' _i , wherein the dimension of P' _i is K×K, and the dimension of Q' _i is K×Num'.

然后重建每幅高清图像的灰度图像对应的K个感知系数矩阵，将第i幅高清图像的灰度图像对应的第k个感知系数矩阵记为

将

表示为

其中，

的维数为K×Num'，

中的“＝”为赋值符号，

至

均为全0向量，

的维数均为Num'×1，

的维数为K×Num'，P'_i表示第i幅高清图像的灰度图像的KLT核，P'_i的维数为K×K，1≤n'≤Num'，n'为正整数，

表示

中的第1维感知系数向量，

表示

中的第2维感知系数向量，

表示

中的第n'维感知系数向量，

表示

中的第Num'维感知系数向量，

的维数均为K×1。Step 6_2: Construct K perceptual coefficient reconstruction matrices corresponding to the grayscale image of each high-definition image, and record the kth perceptual coefficient reconstruction matrix corresponding to the grayscale image of the ith high-definition image as

Then the K perceptual coefficient matrices corresponding to the grayscale image of each high-definition image are reconstructed, and the kth perceptual coefficient matrix corresponding to the grayscale image of the ith high-definition image is recorded as

Will

Expressed as

in,

The dimension of is K × Num',

The "=" in it is an assignment symbol,

to

are all 0 vectors,

The dimensions of are all Num'×1,

The dimension of P'i is K×Num', P'i represents the KLT kernel of the grayscale image of the _ith high-definition image, the dimension of P'i is K×K, 1≤n'≤Num', _n ' is a positive integer ,

express

The 1st-dimensional perceptual coefficient vector in ,

express

The 2nd-dimensional perceptual coefficient vector in ,

express

The n'-dimensional perceptual coefficient vector in ,

express

The Num'-dimensional perceptual coefficient vector in ,

The dimensions are all K × 1.

的图像块；然后按步骤2中图像块分割时的先后顺序将每幅高清图像的灰度图像对应的每个感知系数矩阵中的所有感知系数向量转换成的图像块拼接成一幅图像，将第i幅高清图像的灰度图像对应的第k个感知系数矩阵

中的所有感知系数向量转换成的图像块拼接成的图像作为第i幅高清图像的灰度图像对应的第k幅重建图像，记为

其中，

的宽度为W'且高度为H'。Step 6_3: According to the inverse operation of the vectorization process in step 2, convert each dimension of the perceptual coefficient vector in each perceptual coefficient matrix corresponding to the grayscale image of each high-definition image into a size of

Then, in the order of image block segmentation in step 2, the image blocks converted from all perceptual coefficient vectors in each perceptual coefficient matrix corresponding to the grayscale image of each high-definition image are spliced into one image. The kth perceptual coefficient matrix corresponding to the grayscale image of i high-definition images

The image spliced into the image blocks converted into all perceptual coefficient vectors in the ith high-definition image is the kth reconstructed image corresponding to the grayscale image of the ith high-definition image, denoted as

in,

has a width of W' and a height of H'.

那么将

作为第d位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界图像，同时将数值k作为第d位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点，记为

然后将所有志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点构成的向量记为J_i，

其中，D＞1，在实验中可取D＝30，1≤d≤D，

中的“＝”为赋值符号，J_i的维数为1×D，

表示第1位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点，

表示第2位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点，

表示第D位志愿者观察下第i幅高清图像的灰度图像对应的感知无失真临界点。Step 6_4: Summon D volunteers, each volunteer compares the grayscale image of each high-definition image and its corresponding reconstructed images in turn by visual observation, and each volunteer corresponds to the grayscale image of each high-definition image. Among the K reconstructed images, a reconstructed image is determined as the perceptual distortion-free critical image corresponding to the grayscale image, and the sequence number of the determined reconstructed image is taken as the perceptual distortion-free critical point corresponding to the grayscale image; and the grayscale image of the i-th high-definition image, the d-th volunteer compares the gray-scale image of the i-th high-definition image with its corresponding first reconstructed image, the second reconstructed image, ... , The K-th reconstructed image, once the d-th volunteer cannot distinguish the grayscale image of the i-th high-definition image from one of the corresponding reconstructed images, the comparison process is stopped, assuming that the reconstructed image is the gray-scale of the i-th high-definition image. The kth reconstructed image corresponding to the degree image

then will

As the perceptual distortion-free critical image corresponding to the grayscale image of the i-th high-definition image observed by the d-th volunteer, and taking the value k as the perceptually undistorted image corresponding to the gray-scale image of the i-th high-definition image observed by the d-th volunteer Distortion critical point, denoted as

Then the vector formed by the perceptual distortion-free critical points corresponding to the grayscale image of the ith high-definition image observed by all volunteers is denoted as J _i ,

Among them, D>1, in the experiment, D=30, 1≤d≤D,

The "=" in is the assignment symbol, and the dimension of J _i is 1×D,

represents the perceptual distortion-free critical point corresponding to the grayscale image of the i-th high-definition image observed by the first volunteer,

represents the perceptual distortion-free critical point corresponding to the grayscale image of the i-th high-definition image observed by the second volunteer,

Indicates the perceptual distortion-free critical point corresponding to the grayscale image of the i-th high-definition image observed by the D-th volunteer.

和

然后在所有志愿者观察下每幅高清图像的灰度图像对应的感知无失真临界点构成的向量中剔除离群值，对于J_i，若

不满足

则判定

为离群值，将

从J_i中剔除，将剔除离群值后得到的向量记为

Step 6_5: Calculate the mean and standard deviation of all perceptual distortion-free critical points in the vector formed by the perceptual distortion-free critical points corresponding to the grayscale images of each high-definition image observed by all volunteers, and calculate all perceptual distortion-free critical points in J _i . The mean and standard deviation of the critical point are recorded as

and

Then, outliers are eliminated from the vector formed by the perceptual distortion-free critical points corresponding to the grayscale images of each high-definition image observed by all volunteers. For J _i , if

not satisfied

then judge

is an outlier, the

Remove from J _i , and record the vector obtained after removing outliers as

中的所有感知无失真临界点的均值记为

然后获取每幅高清图像的灰度图像对应的感知无失真临界点，将第i幅高清图像的灰度图像对应的感知无失真临界点记为J_i，

其中，符号

为向上取整运算符号。Step 6_6: Calculate the average value of all perceptual distortion-free critical points after removing outliers from the vector of the perceptual distortion-free critical points corresponding to the grayscale images of each high-definition image observed by all volunteers, and set the

The mean of all perceptually distortion-free critical points in

Then, the perceptual distortion-free critical point corresponding to the grayscale image of each high-definition image is obtained, and the perceptual distortion-free critical point corresponding to the grayscale image of the ith high-definition image is recorded as J _i ,

Among them, the symbol

Operator symbol for round up.

然后计算每幅高清图像的灰度图像的KLT系数矩阵中的每一维KLT谱分量的归一化KLT系数能量，将Q'_i中的q'_i,k的归一化KLT系数能量记为

再计算每幅高清图像的灰度图像对应的感知无失真临界点处的累积归一化KLT系数能量，将第i幅高清图像的灰度图像对应的感知无失真临界点J_i处的累积归一化KLT系数能量记为

最后将所有高清图像的灰度图像对应的感知无失真临界点处的累积归一化KLT系数能量构成一个向量，记为U^cum，

其中，q'_i,k(n')表示q'_i,k中的第n'个元素的值，1≤ζ≤K，U_i,ζ表示Q'_i中的第ζ维KLT谱分量q'_i,ζ的KLT系数能量，

表示Q'_i中的q'_i,1的归一化KLT系数能量，

表示Q'_i中的q'_i,2的归一化KLT系数能量，

表示Q'_i中的

的归一化KLT系数能量，

表示Q'_i中的第J_i维KLT谱分量，U^cum的维数为1×S，

表示第1幅高清图像的灰度图像对应的感知无失真临界点J₁处的累积归一化KLT系数能量，

表示第2幅高清图像的灰度图像对应的感知无失真临界点J₂处的累积归一化KLT系数能量，

表示第S幅高清图像的灰度图像对应的感知无失真临界点J_S处的累积归一化KLT系数能量；Step 6-7: Calculate the KLT coefficient energy of each dimension KLT spectral component in the KLT coefficient matrix of the grayscale image of each high-definition image, and denote the KLT coefficient energy of q' _i,k in Q' _i as U _i,k ,

Then calculate the normalized KLT coefficient energy of each dimension KLT spectral component in the KLT coefficient matrix of the grayscale image of each high-definition image, and denote the normalized KLT coefficient energy of q' _{i, k} in Q' _i as

Then calculate the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point corresponding to the grayscale image of each high-definition image, and calculate the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point J _i corresponding to the grayscale image of the ith high-definition image. The energy of the normalized KLT coefficient is recorded as

Finally, the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point corresponding to the grayscale images of all high-definition images is formed into a vector, denoted as U ^cum ,

Among them, q' _i,k (n') represents the value of the n'th element in q' _i,k , 1≤ζ≤K, U _{i ,ζ} denotes the ζth dimension KLT spectral component q in Q'i ' _{i, the KLT coefficient energy of ζ} ,

represents the normalized KLT coefficient energy of q' _i,1 in Q' _i ,

represents the normalized KLT coefficient energy of q' _i,2 in Q' _i ,

means that in Q' _i

The normalized KLT coefficient energy of ,

represents the J _i -th dimension KLT spectral component in Q' _i , the dimension of U ^cum is 1×S,

represents the cumulative normalized KLT coefficient energy at the perceptual distortion _- free critical point J1 corresponding to the grayscale image of the first high-definition image,

represents the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point J ₂ corresponding to the grayscale image of the second high-definition image,

represents the cumulative normalized KLT coefficient energy at the perceptual distortion-free critical point J _S corresponding to the grayscale image of the S-th high-definition image;

和

然后根据

和

得到感知无失真临界点计算模型，描述为：

其中，L表示I_Y的感知无失真临界点。Step 6_8: Calculate the mean and standard deviation of all cumulative normalized KLT coefficient energies in U ^cum , corresponding to

and

then according to

and

The perceptual distortion-free critical point calculation model is obtained, which is described as:

where L represents the perceptual distortion-free critical point of I _Y.

中的每一维感知无失真系数向量转换成尺寸大小为

的图像块，将

转换成的图像块作为第n个图像块；然后按步骤2中图像块分割时的先后顺序将

中的所有感知无失真系数向量转换成的图像块拼接成图像作为感知无失真临界图像，记为I_L；再根据I_Y和I_L，计算恰可察觉失真阈值图，记为M，将M中坐标位置为(a,b)的像素点的像素值记为M(a,b)，M(a,b)＝|I_Y(a,b)-I_L(a,b)|；其中，1≤a≤W,1≤b≤H，I_Y(a,b)表示I_Y中坐标位置为(a,b)的像素点的像素值，I_L(a,b)表示I_L中坐标位置为(a,b)的像素点的像素值，符号“||”为取绝对值符号。Step 7: According to the inverse operation of the vectorization process in step 2, the

Each dimension in the vector of perceptually undistorted coefficients is converted to size as

image block, will

The converted image block is taken as the nth image block;

All the image blocks converted into perceptual distortion-free coefficient vectors in the image are spliced into an image as the perceptual distortion-free critical image, denoted as _IL ; and then according to I _Y and _IL , calculate the perceptible distortion threshold map, denoted as M, and M The pixel value of the pixel point whose middle coordinate position is (a,b) is denoted as M(a,b), M(a,b)=|I _Y (a, b)-I _L (a, b)|; where , 1≤a≤W, 1≤b≤H, I _Y (a, b) represents the pixel value of the pixel whose coordinate position is (a, b) in I _Y , and I _L (a, b) represents the pixel value in I _L The pixel value of the pixel point whose coordinate position is (a, b), the symbol "||" is the symbol of taking the absolute value.

In order to further illustrate the feasibility and effectiveness of the method of the present invention, experiments were carried out on the method of the present invention.

将

中坐标位置为(a,b)的像素点的像素值记为

将源图像的绿色通道对应的加噪图像记为

将

中坐标位置为(a,b)的像素点的像素值记为

将源图像的蓝色通道对应的加噪图像记为

将

中坐标位置为(a,b)的像素点的像素值记为

最终可以得到源图像的加噪图像；其中，1≤a≤W,1≤b≤H，I_R(a,b)表示源图像的红色通道中坐标位置为(a,b)的像素点的像素值，M(a,b)表示恰可察觉失真阈值图中坐标位置为(a,b)的像素点的像素值，N(a,b)表示维数为W×H的随机二值矩阵中下标为(a,b)处的元素的值，N(a,b)为+1或-1，θ为噪声大小的调节参数，通过改变θ的值可以控制所注入的噪声量，I_G(a,b)表示源图像的绿色通道中坐标位置为(a,b)的像素点的像素值，I_B(a,b)表示源图像的蓝色通道中坐标位置为(a,b)的像素点的像素值。The first part of the experiment is the noised image generation experiment guided by the threshold estimation model of just perceptible distortion. Input a source image, use the method of the present invention to obtain a corresponding threshold value map of perceptible distortion, and conduct noise-guided noise addition to the red channel, green channel, and blue channel of the source image respectively, and add the red channel of the source image to the red channel, green channel, and blue channel of the source image. The noised image corresponding to the channel is denoted as

Will

The pixel value of the pixel with the middle coordinate position (a, b) is recorded as

Denote the noised image corresponding to the green channel of the source image as

Will

The pixel value of the pixel with the middle coordinate position (a, b) is recorded as

Denote the noised image corresponding to the blue channel of the source image as

Will

The pixel value of the pixel with the middle coordinate position (a, b) is recorded as

Finally, the noise-added image of the source image can be obtained; among them, 1≤a≤W, _1≤b≤H , IR (a, b) represents the pixel point whose coordinate position is (a, b) in the red channel of the source image. Pixel value, M(a,b) represents the pixel value of the pixel whose coordinate position is (a,b) in the threshold map of perceptible distortion, and N(a,b) represents a random binary matrix with dimension W×H The subscript in the middle is the value of the element at (a,b), N(a,b) is +1 or -1, θ is the adjustment parameter of the noise size, the amount of injected noise can be controlled by changing the value of θ, I _G (a, b) represents the pixel value of the pixel whose coordinate position is (a, b) in the green channel of the source image, and I _B (a, b) represents the coordinate position (a, b) in the blue channel of the source image ) of the pixel value of the pixel point.

Here, S=500, D=60, and K=64 are set, and the PSNR of the noise-added image of the obtained source image is controlled to be about 26dB. PSNR, or peak signal-to-noise ratio, is an existing image quality evaluation index. 20 source images were selected for the experiment. The six existing JND threshold map generation methods are used for comparative research. The six JND threshold map generation methods are: the first, Yang2005 (X.Yang, W.Lin, Z.Lu, E.Ong, and S. Yao, "Motion-compensated residue pre-processing in video coding based on just-noticeable-distortion profile," IEEE Transactions on Circuits and Systems for Video Technology, vol.15, no.6, pp.742–752, 2005. (Based on Video coding motion compensation residual preprocessing for just-noticeable distortion model)); the second, Zhang2005 (X. Zhang, W. Lin, and P. Xue, "Improved estimation for just-noticeable visual distortion," Signal Processing, vol .85, no.4, pp.795–808, 2005. (Improved Estimation of Just Perceptible Visual Distortion); No. 3, Wu2013 (J.Wu,G.Shi,W.Lin,A.Liu,and F.Qi, "Just noticeabledifference estimation for images with free-energy principle," IEEE Transactions on Multimedia, vol.15, no.7, pp.1705–1710, 2013. ); No. 4, Wu2017 (J.Wu, L.Li, W.Dong, G.Shi, W.Lin, and C.-C.J.Kuo, "Enhanced just noticeable difference model for images with patterncomplexity," IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2682–2693, 2017. (Image Enhancement Perceptual Distortion Model Based on Pattern Complexity)); No. 5, Jakhetiya2018 (V.Jakhetiya,W.Lin,S .Jaiswal, K.Gu, and S.C. Guntuku, "Just noticeable difference for natur al images using rms contrast and feed-back mechanism,”Neurocomputing,vol.275,pp.366–376,2018.(Natural images according to rms contrast and feed-back mechanism are just perceptible distortion)); No. 6, Chen2020(Z .Chen and W.Wu, “Asymmetric foveated just-noticeable-difference model for images with visual field inhomogeneities,” IEEE Transactions on Circuits and Systems for Video Technology, vol.30, no.11, pp.4064–4074, 2020. (Model of just perceptible distortion of image asymmetric funnel in terms of field inhomogeneity)). For each source image, the method of the present invention and the existing 6 JND threshold map generation methods are used to obtain a threshold map of perceptible distortion; then the noise-added image of each source image is obtained, that is, the peak value can be obtained for each source image Seven noisy images with approximately the same signal-to-noise ratio.

Table 1 Experimental comparison of the noise-added images obtained by using the method of the present invention and the existing 6 JND threshold map generation methods

30 volunteers were called to conduct subjective experiments. Each volunteer was asked to compare each source image and its corresponding 7 noised images, and rated the noised image between 0 and -1. If the score was 0, it indicated the visual difference between the noised image and the source image. The quality is very close; a score of -1 indicates that the visual quality of the noised image is very poor compared to the source image. Each noise-added image can receive 30 scores. After removing outliers, the average value is obtained to obtain the subjective score of the noise-added image, which is recorded as the MOS value. The higher the MOS value, the better the visual quality of the noise-added image. . In addition, using the existing image quality objective evaluation algorithm MS-SSIM (Z. Wang, E.P. Simoncelli, and A.C. Bovik, "Multiscale structural similarity for image quality assessment," in The Thrity-Seventh Asilomar Conference on Signals, Systems Computers, 2003, vol .2, 2003, pp.1398–1402Vol.2. (Multi-scale structural similarity is used for image quality evaluation)) to evaluate the quality of the noised image, the higher the MS-SSIM value, the better the quality of the noised image. The MOS and MS-SSIM values for each noised image are listed in Table 1.

In Table 1, I01 represents the first source image, and a total of 20 source images participated in the experiment, so they are marked as I01 to I20 in turn. Fig. 3a shows the source image I03, Fig. 3b shows the perceptible distortion threshold map obtained by processing the source image shown in Fig. 3a using the method of the present invention, and Fig. 3c shows the PSNR generated using the guidance of Fig. 3b The noise-added image under the condition of = 26dB, Fig. 3d shows an enlarged view of the part inside the box in Fig. 3c, it can be seen from Fig. 3d that the noise is almost invisible, and the visual quality is good. As shown in Table 1, the noised image generated by the perceptible distortion threshold map obtained by the method of the present invention exceeds the MS-SSIM value of a single noised image or the MS-SSIM average value of other noised images. A threshold map of just perceptible distortion obtained by the method guides the generated noised image. On the noised image corresponding to the source image I12, the MOS value of the noised image generated by the just perceptible distortion threshold map obtained by the method of the present invention is only lower than that of the just perceptible distortion threshold map generated by the Chen2020 method. , but higher than the noised images guided by the threshold map of just perceptible distortion obtained by other methods. In addition, on the noised images corresponding to other source images, the MOS value and the MOS average value of the noised image generated under the guidance of the perceptible distortion threshold map obtained by the method of the present invention are higher than those of other methods. This experiment shows that, on the premise of keeping the amount of added noise consistent, the noise-added image generated by the perceptible distortion threshold map obtained by the method of the present invention has better visual effect, and can guide the injection of noise into the visually imperceptible image. place. Therefore, the method of the present invention can more accurately reflect the visual masking characteristics of HVS, and describe its visual perception redundancy.

The second part of the experiment is the JPEG compression experiment guided by the threshold of perceptible distortion.

个互不重叠的尺寸大小为8×8的图像块。According to the requirements of the JPEG compression model, the red channel, green channel, blue channel of the source image and the corresponding perceptible distortion threshold map obtained by the method of the present invention are divided into image blocks in the same order, and the red channel, The green channel, the blue channel and the corresponding perceptible distortion threshold map obtained by the method of the present invention are respectively divided into

non-overlapping image blocks of size 8×8.

The red channel, green channel, and blue channel of the source image are respectively subjected to the smoothing process guided by the perceptible distortion threshold map in the unit of image block.

以图像块为单位，对源图像的绿色通道中的第j个图像块进行恰可察觉失真阈值图中的第j个图像块引导的平滑处理的计算公式为：

以图像块为单位，对源图像的蓝色通道中的第j个图像块进行恰可察觉失真阈值图中的第j个图像块引导的平滑处理的计算公式为：

其中，

表示源图像的红色通道中的第j个图像块经平滑处理后得到的图像块中坐标位置为(a',b')的像素点的像素值，1≤a'≤8,1≤b'≤8，I_R,j(a',b')表示源图像的红色通道中的第j个图像块中坐标位置为(a',b')的像素点的像素值，M_j(a',b')表示恰可察觉失真阈值图中的第j个图像块中坐标位置为(a',b')的像素点的像素值，

表示源图像的红色通道中的第j个图像块中的所有像素点的像素值的平均值，

表示源图像的绿色通道中的第j个图像块经平滑处理后得到的图像块中坐标位置为(a',b')的像素点的像素值，I_G,j(a',b')表示源图像的绿色通道中的第j个图像块中坐标位置为(a',b')的像素点的像素值，

表示源图像的绿色通道中的第j个图像块中的所有像素点的像素值的平均值，

表示源图像的蓝色通道中的第j个图像块经平滑处理后得到的图像块中坐标位置为(a',b')的像素点的像素值，I_B,j(a',b')表示源图像的蓝色通道中的第j个图像块中坐标位置为(a',b')的像素点的像素值，

表示源图像的蓝色通道中的第j个图像块中的所有像素点的像素值的平均值。平滑处理完成后，将图像块按照图像块分割时的顺序重新拼接回去得到平滑处理完的红色通道、绿色通道、蓝色通道，最终得到经过平滑处理后的平滑图像。Taking the image block as the unit, the calculation formula of the jth image block in the red channel of the source image for the smoothing process guided by the jth image block in the perceptible distortion threshold map is:

Taking the image block as the unit, the calculation formula of the smoothing process guided by the jth image block in the perceptible distortion threshold map for the jth image block in the green channel of the source image is:

Taking the image block as the unit, the calculation formula of the smoothing process guided by the jth image block in the perceptible distortion threshold map for the jth image block in the blue channel of the source image is:

in,

Represents the pixel value of the pixel whose coordinate position is (a', b') in the image block obtained by smoothing the jth image block in the red channel of the source image, 1≤a'≤8,1≤b' ≤8, IR _,j (a',b') represents the pixel value of the pixel at the coordinate position (a',b') in the jth image block in the red channel of the source image, M _j (a',b') represents the pixel value of the pixel whose coordinate position is (a', b') in the j-th image block in the perceptible distortion threshold map,

represents the average value of the pixel values of all pixels in the jth image block in the red channel of the source image,

Indicates the pixel value of the pixel whose coordinate position is (a',b') in the image block obtained after smoothing the jth image block in the green channel of the source image, _IG,j (a',b') Represents the pixel value of the pixel whose coordinate position is (a', b') in the jth image block in the green channel of the source image,

represents the average value of the pixel values of all pixels in the jth image block in the green channel of the source image,

Represents the pixel value of the pixel whose coordinate position is (a',b') in the image block obtained by smoothing the jth image block in the blue channel of the source image, I _B,j (a',b' ) represents the pixel value of the pixel whose coordinate position is (a', b') in the jth image block in the blue channel of the source image,

Represents the average of the pixel values of all pixels in the jth image block in the blue channel of the source image. After the smoothing process is completed, the image blocks are spliced back together in the order in which the image blocks are divided to obtain the smoothed red channel, green channel, and blue channel, and finally a smoothed image after smoothing is obtained.

然后计算增益，记为Gain，

ΔBitrate越大代表节省比特率越多，ΔPSNR越小代表PSNR损失越少。若增益Gain越大则代表损失尽可能少的PSNR，换取尽可能大的压缩比特率节省，代表恰可察觉失真阈值图引导的图像压缩性能越好。Set the quality factor (coding quantization parameter) QP=1, and perform JPEG compression. Perform JPEG compression on the input source image, obtain the JPEG compressed image of the source image, calculate the compressed bit rate and PSNR value of the JPEG compressed image of the source image, and denote Bitrate ₁ and PSNR ₁ correspondingly; perform JPEG compression on the smooth image of the source image , obtain the JPEG compressed image of the smoothed image of the source image, and calculate the compressed bit rate and PSNR value of the JPEG compressed image of the smoothed image of the source image, which are correspondingly recorded as Bitrate ₂ and PSNR ₂ . Calculate the bit rate saving percentage and PSNR loss percentage, corresponding to ΔBitrate and ΔPSNR,

Then calculate the gain, denoted as Gain,

A larger ΔBitrate represents more bit rate savings, and a smaller ΔPSNR represents less PSNR loss. If the gain is larger, it means that the loss of PSNR is as small as possible, in exchange for saving the compression bit rate as much as possible, which means that the image compression performance guided by the perceptible distortion threshold map is better.

Table 2 The experimental comparison between the JPEG compression guided by the just perceptible distortion threshold map obtained by the method of the present invention and the JPEG compression guided by the just perceptible distortion threshold map obtained by the Wu2017 method

Here S=500, D=60, K=64 are set. The 20 source images used in the first part of the experiment were selected for the experiment, and the Wu2017 method was selected as the comparison method. Fig. 4a shows the result of JPEG compression of the source image I01 directly, Fig. 4b shows the JPEG compression result of the source image I01 obtained by the method of the present invention, and Fig. 4c shows the JPEG compression result of the source image I01 obtained by the method of the invention Figure-guided JPEG compression results obtained by Wu2017's method with just the perceptible distortion threshold. The boxes on the right in each of Figures 4a, 4b, and 4c are partial enlargements of the boxes on the left. Comparing the boxes on the right in Fig. 4a, Fig. 4b, Fig. 4c, we can see that the visual effect of the JPEG compression result guided by the perceptible distortion threshold map obtained by the method of the present invention and the source image I01 directly adopt the result of JPEG compression. Similar, the visual quality is almost not reduced, but the JPEG compression result of the source image I01 using the Wu2017 method to obtain just the perceptible distortion threshold map has a very obvious blur, and its visual quality is compared with the source image I01 directly uses JPEG compression. The results are quite different. The experimental data for each source image are shown in Table 2. It can be seen that the perceptible distortion threshold map-guided JPEG compression obtained by the method of the present invention is lower than the Wu2017 method in terms of bit rate saving, but is less than the Wu2017 method in terms of PSNR loss. And except on the I11 source image, the JPEG compression gain Gain guided by the perceptible distortion threshold map obtained by the method of the present invention is slightly weaker than that of the Wu2017 method, and the gain Gain on all other source images and the average value of the gain Gain are stronger. Method in Wu2017. Further, change the quality factor QP value, and test the average gain of the method of the present invention and the Wu2017 method on 20 source images, draw a line graph, as shown in Figure 5, it can be seen that on each QP value, the present invention The average gains of the methods are all higher than those of the Wu2017 method. This experiment shows that the JPEG compression guided by the perceptible distortion threshold map obtained by the method of the present invention can ensure the visual quality of the compressed image as much as possible under the premise of reducing the compression bit rate as much as possible. Therefore, the method of the present invention can more accurately reflect the visual masking characteristics of HVS and describe its visual perception redundancy.

Claims (6)

1. A top-down natural image just noticeable distortion threshold estimation method, comprising the steps of:

step 1: taking a natural image to be processed as a source image; then converting the source image into a gray image, marked as I_Y(ii) a Wherein, the source image is RGB color image, source image and I_YAll width of (A) and all height of (B) are W and H;

step 2: will I_YIs divided into Num non-overlapping sizes of

The image block of (1); then to I_YEach image block in the image processing system is vectorized to obtain I_YThe column vector corresponding to each image block in (1), and (3)_YThe column vector corresponding to the nth image block is marked as x_n(ii) a Then mix I_YThe column vectors corresponding to all image blocks in the image block are spliced to form a vectorization matrix, which is marked as X, X ═ X₁,x₂,…,x_n,…,x_Num](ii) a Wherein,

setting both W and H can be

Integer division, K is 4²Or 5²Or 6²Or 7²Or 8²Or 9²Or 10²，1≤n≤Num，x₁Is represented by_Y1 st image block of (2) and the corresponding column vector, x₂Is represented by_YThe column vector, x, corresponding to the 2 nd image block in (1)_NumIs represented by I_YColumn vector, x, corresponding to the Num image block in (2)₁、x₂、x_n、x_NumIs K × 1, the dimension of X is K × Num, the symbol "[ alpha ]]"is a representation of a vector or matrix;

and step 3: calculating the covariance matrix of X, and recording as C; then processing C by using a characteristic value decomposition technology to obtain K characteristic values of C and K corresponding characteristic vectors; sorting the K eigenvectors of the C in a descending manner from large to small according to the corresponding K eigenvalues, and taking a matrix formed by the K eigenvectors of the C according to sorting results as prior information extracted from the X; the dimension of C is KxK, the dimension of the characteristic vector is a column vector, the dimension of the characteristic vector is Kx1, each column of the prior information extracted from X is 1 characteristic vector of C, and the dimension of the prior information extracted from X is KxK;

and 4, step 4: using the prior information extracted from X as I_YKLT nucleus of (1), denoted as P; then root ofFrom P and X, calculate I_YKLT coefficient matrix (KLT) of (KLT), denoted as Q, Q ═ P)^TX; and Q is represented by Q ═ Q₁,q₂,…,q_k,…,q_K]^T(ii) a Wherein the dimension of P is KxK, the dimension of Q is KxNum, K is more than or equal to 1 and less than or equal to K, Q is₁Representing the 1 st-dimensional KLT spectral component in Q, Q₂Representing the 2 nd-dimensional KLT spectral component in Q, Q_kRepresenting the k-dimensional KLT spectral component in Q, Q_KRepresenting the K-th dimensional KLT spectral component of Q, Q₁、q₂、q_k、q_KThe dimensions of (A) are Num multiplied by 1;

and 5: calculating KLT coefficient energy of KLT spectral components of each dimension in Q, and calculating_kKLT coefficient energy of (E)_k，

Then, the normalized KLT coefficient energy of each dimension KLT spectral component in Q is calculated, and Q is calculated_kNormalized KLT coefficient energy of

Then, the accumulated normalized KLT coefficient energy of each dimension KLT spectral component in the Q is calculated, and the Q is converted into a linear motion_kThe cumulative normalized KLT coefficient energy of (D) is recorded as

And finally, forming the accumulated normalized KLT coefficient energy of all the KLT spectral components in the Q into an accumulated normalized KLT coefficient energy vector, and recording the energy vector as E^cum，

Wherein q is_k(n) represents q_kThe value of the nth element in (1) ≦ ζ ≦ K, E_ζRepresents the second in QZeta dimension KLT spectral component q_ζThe energy of the KLT coefficient of (c),

denotes q₁The energy of the normalized KLT coefficient of (c),

denotes q₂Normalized KLT coefficient energy of E^cumHas a dimension of 1 x K (x K),

denotes q₁The accumulated normalized KLT coefficient energy of (a),

denotes q₂The accumulated normalized KLT coefficient energy of (a),

denotes q_KThe accumulated normalized KLT coefficient energy of (1);

step 6: will E^cumSubstituting the input into a perceptual undistorted critical point calculation model to obtain I_YThe perceptual undistorted critical point of (1) is marked as L; then, a perceptual undistorted coefficient reconstruction matrix is constructed according to L and is recorded as

Followed by using

Reconstructing to obtain a perceptual undistorted coefficient matrix, and recording as

Then will be

Is shown as

Wherein L is a positive integer, L is more than or equal to 1 and less than or equal to K,

has the dimension of K multiplied by Num,

wherein, q is an assigned symbol_LRepresenting the L-th dimensional KLT spectral component in Q, Q_LHas the dimension of Num x 1,

to

Are all the vectors of 0, and all the vectors,

the dimensions of (a) are each Num x 1,

has the dimension of K multiplied by Num,

to represent

The 1 st-dimensional perceptual undistorted coefficient vector in (a),

to represent

The 2 nd-dimensional perceptual undistorted coefficient vector in (1),

to represent

The nth-dimension perceptual undistorted coefficient vector of (1),

to represent

The Num-th dimension of (1) perceives a distortion-free coefficient vector,

the dimensions of (A) are Kx 1;

and 7: in the reverse operation of the vectorization process in step 2, will

The vector of each dimension of the perceptual distortion-free coefficients in (1) is converted into a size of

Image block of

The converted image block is used as the nth image block; then will be

The image blocks converted from all the perceptual undistorted coefficient vectors in the image are spliced into an image as a perceptual undistorted critical image, which is marked as I_L(ii) a Then according to I_YAnd I_LCalculating a just noticeable distortion threshold value map, recording the just noticeable distortion threshold value map as M, and recording the pixel value of the pixel point with the coordinate position (a, b) in MIs M (a, b), M (a, b) ═ I_Y(a,b)-I_L(a, b) |; wherein a is more than or equal to 1 and less than or equal to W, b is more than or equal to 1 and less than or equal to H, I_Y(a, b) represents I_YThe pixel value of the pixel point with the middle coordinate position (a, b), I_L(a, b) represents I_LThe middle coordinate position is the pixel value of the pixel point of (a, b), and the symbol "|" is the absolute value symbol.

2. The method as claimed in claim 1, wherein in step 2, x is_nThe acquisition process comprises the following steps: scanning I in a Z-shaped manner_YThe pixel values of all the pixel points in the nth image block are arranged into a column to form x_n。

3. The method according to claim 1 or 2, wherein in step 3, C is calculated as:

wherein the superscript "T" denotes the transpose of a vector or matrix,

representing the mean vector obtained by averaging X by row,

has a dimension of K × 1.

4. A method as claimed in claim 3, wherein in step 4, P ═ P is used to estimate the threshold value of distortion just noticeable in the natural image from top to bottom₁,p₂,…,p_K]Wherein p is₁Representing that the K eigenvectors of C are sorted in a descending way from the big to the small of the corresponding K eigenvaluesThe last 1 st feature vector, p₂Representing the 2 nd eigenvector, p, sorted by the descending order of the corresponding K eigenvalues of C_KRepresenting the K-th eigenvector, p, sorted by the descending order of the corresponding K eigenvalues₁、p₂、p_KThe dimensions of (A) are each K × 1.

5. The method as claimed in claim 4, wherein in step 6, the process of obtaining the perceptual undistorted critical point calculation model is as follows:

step 6_ 1: selecting S high-definition images, and converting each high-definition image into a gray image; then, according to the process from the step 2 to the step 4, acquiring the KLT coefficient matrix of the gray level image of each high-definition image in the same way, and recording the KLT coefficient matrix of the gray level image of the ith high-definition image as Q'_iPrepared from Q'_iIs represented by Q'_i＝[q'_i,1,q'_i,2,…,q'_i,k,…,q'_i,K]^T(ii) a Wherein S is more than or equal to 100, the high-definition image is an RGB color image, the width of the high-definition image is W 'and the height of the high-definition image is H', and both W 'and H' can be set

The size of the image block divided by the gray scale image of the integer division and high definition image is

1≤i≤S，Q'_iHas dimensions of K multiplied by Num ', Num' represents the total number of image blocks of the grey scale image segmentation of the high definition image,

q'_i,1represents Q'_i1-dimensional KLT spectral component of (1) 'q'_i,2Represents Q'_i2-dimensional KLT spectral component of (1), q'_i,kRepresents Q'_iK-th dimensional KLT spectral component of (1), q'_i,KRepresents Q'_iK-th dimension KLT spectral component of (1), q'_i,1、q'_i,2、q'_i,k、q'_i,KThe dimensions of (A) are Num' x 1;

step 6_ 2: constructing K perception coefficient reconstruction matrixes corresponding to the gray level images of each high-definition image, and recording the kth perception coefficient reconstruction matrix corresponding to the gray level image of the ith high-definition image as

Then, K perception coefficient matrixes corresponding to the gray level images of each high-definition image are reconstructed, and the K perception coefficient matrix corresponding to the gray level image of the ith high-definition image is recorded as

Will be provided with

Is shown as

Wherein,

has the dimension of K multiplied by Num',

wherein "═ is an assigned symbol,

to

Are all the vectors of 0, and all the vectors,

the dimensions of (a) are Num' x 1,

is K x Num ', P'_iKLT kernel, P 'representing a grayscale image of the ith high-definition image'_iThe dimension of the integer is K multiplied by K, n ' is more than or equal to 1 and less than or equal to Num ', n ' is a positive integer,

to represent

The 1 st-dimensional perceptual coefficient vector in (b),

to represent

The 2 nd-dimensional perceptual coefficient vector of (1),

to represent

The nth' dimensional perceptual coefficient vector of (a),

to represent

The Num' th dimension of the perceptual coefficient vector,

the dimensions of (A) are Kx 1;

step 6_ 3: converting each dimension of the perception coefficient vector in each perception coefficient matrix corresponding to the gray level image of each high-definition image into the size of the perception coefficient vector according to the inverse operation of the vectorization processing in the step 2

The image block of (1); then, all the perception coefficient vectors in each perception coefficient matrix corresponding to the gray level image of each high-definition image are converted into image blocks which are spliced into an image, and the kth perception coefficient matrix corresponding to the gray level image of the ith high-definition image is spliced into an image

The image spliced by the image blocks converted from all the perception coefficient vectors is used as the k-th reconstructed image corresponding to the gray level image of the ith high-definition image and is marked as

Wherein,

has a width of W 'and a height of H';

step 6_ 4: d volunteers are summoned, each volunteer sequentially compares the gray level image of each high-definition image with each corresponding reconstructed image in a visual observation mode, each volunteer determines one reconstructed image from K reconstructed images corresponding to the gray level image of each high-definition image as a perception undistorted critical image corresponding to the gray level image, and simultaneously, the sequence number of the determined reconstructed image is used as a perception undistorted critical point corresponding to the gray level image; for the d-th volunteer and the gray level image of the ith high-definition image, sequentially comparing the gray level image of the ith high-definition image with the corresponding gray level image by the d-th volunteer in a visual observation modeOnce the d-th volunteer cannot distinguish the gray image of the ith high-definition image from one of the corresponding reconstructed images, the contrast process is stopped, and the reconstructed image is assumed to be the K-th reconstructed image corresponding to the gray image of the ith high-definition image

Then will be

The value k is used as a perception undistorted critical point corresponding to the gray image of the ith high-definition image observed by the d-th volunteer, and is recorded as a perception undistorted critical point corresponding to the gray image of the ith high-definition image observed by the d-th volunteer

Then recording a vector formed by sensing undistorted critical points corresponding to the gray level image of the ith high-definition image observed by all volunteers as J_i，

Wherein D is more than 1, D is more than or equal to 1 and less than or equal to D,

wherein ═ is an assignment symbol, J_iHas a dimension of 1 x D and,

representing the perception undistorted critical point corresponding to the gray image of the ith high-definition image observed by the 1 st volunteer,

represents the 2 nd bitThe volunteer observes the perception undistorted critical point corresponding to the gray level image of the ith high-definition image,

expressing a perception undistorted critical point corresponding to the gray level image of the ith high-definition image observed by the D-th volunteer;

step 6_ 5: calculating the mean value and standard deviation of all the sensing undistorted critical points in the vector formed by the sensing undistorted critical points corresponding to the gray level image of each high-definition image observed by all the volunteers, and calculating J_iThe mean and standard deviation correspondence of all perceptual undistorted critical points in (1) is recorded as

And

then, outliers are removed from vectors formed by perception undistorted critical points corresponding to the gray level images of each high-definition image under the observation of all volunteers, and J is considered_iIf, if

Not meet the requirements of

Then it is decided

For outliers, will

From J_iAnd (4) medium elimination, recording the vectors obtained after the outliers are eliminated as the vectors

Step 6_ 6: calculating the perception undistorted image corresponding to the gray image of each high-definition image observed by all volunteersAfter outliers are removed from vectors formed by the boundary points, the mean values of all perception distortion-free critical points are obtained

The mean of all perceptual distortion-free critical points in (1) is recorded as

Then obtaining a perception undistorted critical point corresponding to the gray level image of each high-definition image, and recording the perception undistorted critical point corresponding to the gray level image of the ith high-definition image as J_i，

Wherein, the symbol

Is a sign of an upward rounding operation;

step 6_ 7: calculating KLT coefficient energy of KLT spectral component of each dimension in KLT coefficient matrix of gray level image of each high definition image, and calculating Q'_iQ 'of'_i,kKLT coefficient energy of (1) is recorded as U_i,k，

Then calculating the normalized KLT coefficient energy of each dimension KLT spectral component in the KLT coefficient matrix of the gray-scale image of each high-definition image, and Q'_iQ 'of'_i,kNormalized KLT coefficient energy of

Then, the accumulated normalized KLT coefficient energy at the perception undistorted critical point corresponding to the gray level image of each high-definition image is calculated, and the perception undistorted critical point J corresponding to the gray level image of the ith high-definition image is compared_iThe cumulative normalized KLT coefficient energy of (A) is recorded as

And finally, forming a vector by accumulated normalized KLT coefficient energy at a perception undistorted critical point corresponding to the gray level images of all the high-definition images, and recording the vector as U^cum，

Wherein, q'_i,k(n ') represents q'_i,kThe value of the n' th element in (1) ≦ ζ ≦ K, U_i,ζRepresents Q'_iζ -th dimensional KLT spectral component q 'of (1)'_i,ζThe energy of the KLT coefficient of (c),

represents Q'_iQ 'of'_i,1The normalized KLT coefficient energy of (a),

represents Q'_iQ 'of'_i,2The energy of the normalized KLT coefficient of (c),

represents Q'_iIn

The energy of the normalized KLT coefficient of (c),

represents Q'_iJ in (1)_iDimension KLT spectral component, U^cumHas a dimension of 1 x S and has a dimension of,

perception undistorted critical point J corresponding to gray scale image representing 1 st high-definition image₁Cumulative return of pointsThe energy of the KLT coefficient is normalized,

perception undistorted critical point J corresponding to gray scale image representing 2 nd high-definition image₂The accumulated normalized KLT coefficient energy of (a),

perception undistorted critical point J corresponding to gray level image representing S-th high-definition image_SThe accumulated normalized KLT coefficient energy of (a);

step 6_ 8: calculate U^cumThe mean and standard deviation of all accumulated normalized KLT coefficient energies in (1) are correspondingly noted as

And

then according to

And

obtaining a perceptual undistorted critical point calculation model, which is described as:

wherein L represents I_YIs free of distortion critical points.

6. The method of claim 5, wherein Q 'in step 6_ 1'_iThe acquisition process comprises the following steps:

step 6_1 a: recording the gray level image of the ith high-definition image as I'_Y,i(ii) a Followed by mixing of l'_Y,iDivided into Num' non-overlapping sizes of

The image block of (1); then to I'_Y,iPerforming vectorization on each image block to obtain I'_Y,iIs 'to each image block in the image'_Y,iThe column vector corresponding to the n 'th image block in (b) is denoted as x'_i,n'(ii) a Then is prepared from'_Y,iThe column vectors corresponding to all the image blocks in (1) are spliced to form a vectorization matrix, which is recorded as X'_i，X'_i＝[x'_i,1,x'_i,2,…,x'_i,n',…,x'_i,Num'](ii) a Wherein,

k has a value of 4²Or 5²Or 6²Or 7²Or 8²Or 9²Or 10²，1≤n'≤Num'，x'_i,1Is represented by l'_Y,iColumn vector, x 'corresponding to the 1 st image block in (1)'_i,2Is represented by l'_Y,iColumn vector, x 'corresponding to the 2 nd image block in (1)'_i,Num'Is represented by l'_Y,iColumn vector, x ' corresponding to the Num ' image block of (1) '_i,1、x'_i,2、x'_i,n'、x'_i,Num'Are all Kx 1, X'_iHas a dimension of K × Num';

step 6_1 b: calculating X'_iOf (C)'_i(ii) a Then C 'is decomposed by characteristic value decomposition technology'_iIs processed to obtain C'_iK eigenvalues and corresponding K eigenvectors; then to C'_iThe K eigenvectors are sorted in descending order of the corresponding K eigenvalues, and C'_iIs taken as a matrix consisting of K eigenvectors in the order of X'_iExtracting prior information; wherein, C'_iIs K X K, the feature vector is a column vector, the feature vector has a dimension of K X1, from X'_iIs C 'for each column in the extracted prior information'_i1 feature vector of, from X'_iThe dimensionality of the extracted prior information is K multiplied by K;

step 6_1 c: will be from X'_iThe extracted prior information is taken as I'_Y,iKLT nucleus of (E), noted as P'_i(ii) a Then according to P'_iAnd X'_iCalculating l'_Y,iKLT coefficient matrix Q'_i，Q'_i＝(P'_i)^TX'_i(ii) a Wherein, P'_iHas a dimensionality of K × K, Q'_iHas the dimension of K by Num'.

Images