CN109657273B - Bayesian parameter estimation method based on noise enhancement - Google Patents

Abstract

The invention discloses a Bayesian parameter estimation method based on noise enhancement. Belonging to the field of signal processing. The method is a method for improving parameter estimation performance under the Bayesian rule by utilizing noise enhancement characteristics. Adding independent additive noise to a nonlinear system input signal, establishing a noise enhancement Bayesian parameter estimation model, carrying out Bayesian estimation on the input parameter by utilizing the nonlinear system output signal, finally solving the optimal additive noise under the model, and obtaining the optimal noise enhancement Bayesian estimation performance. The invention improves the Bayesian parameter estimation performance by utilizing noise enhancement, and can further reduce the corresponding mean square error when carrying out Bayesian estimation on the input parameters based on the nonlinear system output signals by adding proper noise to the nonlinear system input signals.

Description

A Bayesian Parameter Estimation Method Based on Noise Enhancement

technical field

The invention belongs to the field of signal processing, in particular to noise enhancement and Bayesian parameter estimation.

Background technique

In traditional signal processing, noise is generally regarded as an unwanted signal that is not only of no benefit but a disturbance to the system. In fact, noise always exists with useful signals at the same time. Too much noise in the system usually reduces the capacity of the transmission channel, which not only reduces the accuracy of signal detection, but also deteriorates the performance of parameter estimation, which seriously affects the system work. To improve system performance, noise is usually removed or separated from useful signals as much as possible. However, the effects of noise on a system are not all negative. Under certain conditions, noise can positively enhance the signal through a nonlinear system, and this phenomenon is called noise enhancement. It has been shown that adding noise to the input of some nonlinear systems or adjusting the background noise level can significantly improve the detection and/or estimation performance of the system. The estimate that minimizes the average cost is the Bayesian estimate, and the resulting estimator is the Bayesian estimator. The squared error between the parameter estimate and the true value is one of the most common cost functions for Bayesian estimation. When using the output signal of the nonlinear system to make Bayesian estimation of the unknown parameters in the input signal, combined with the theory of noise enhancement, it can be known that under certain circumstances, the corresponding optimal Bayesian estimation mean square error when noise is added to the input signal of the nonlinear system It may be smaller than the corresponding original optimal Bayesian estimation mean square error when no noise is added to the input signal.

Contents of the invention

The purpose of the present invention is to propose a Bayesian parameter estimation method based on noise enhancement based on the Bayesian estimation with the square error between the parameter estimated value and the true value as the cost function and in combination with the principle of noise enhancement. After adding appropriate noise to the input signal of the nonlinear system, the mean square error between the estimated value and the true value of the parameter can be further reduced when the output signal of the nonlinear system is used to estimate the input parameters.

The present invention specifically comprises the following steps:

1) Build a noise-enhanced nonlinear system:

Described nonlinear system comprises three parts: nonlinear system input signal, nonlinear system and nonlinear system output signal; nonlinear system input signal x is closely related to parameter θ, and the value of θ is determined by its probability density function p _θ ( θ) is determined; add an independent additive noise n to the input signal x of the nonlinear system, and after passing through the nonlinear system, obtain the output signal y=T(x+n) of the noise-corrected nonlinear system, where T( ) represents The transfer function of the nonlinear system;

2) Establish a noise-enhanced Bayesian parameter estimation model:

之间均方误差最小的贝叶斯估计为The input parameter θ is estimated by using the output signal y of the nonlinear system; when the value of y is constant, the input parameter θ and its estimator

The Bayesian estimate with the smallest mean square error between

The corresponding mean square error is

ε _MMSE (y)＝E(θ ² |y)-E ² (θ|y) (2) Formula

也是使得平均均方误差最小的估计，对应的最小平均均方误差为Where E(θ|y) and E(θ ² |y) respectively represent the expectation of θ and θ ² when the value of y is certain; further,

It is also an estimate that minimizes the average mean square error, and the corresponding minimum average mean square error is

where p _y (y) is the probability density function of the output signal y of the nonlinear system; p _y (y), E(θ|y) and E(θ ² |y) are calculated as:

p_n(n)为加性噪声n的概率密度函数，而p_x(x|θ)表示θ的值一定时非线性系统输入信号x的条件概率密度函数；in

p _n (n) is the probability density function of additive noise n, and p _x (x|θ) represents the conditional probability density function of the nonlinear system input signal x when the value of θ is certain;

3) Solve for the additive noise needed to minimize the Bayesian cost:

Substituting equations (4), (5) and (6) into equation (3), it can be seen that when adding additive noise n with probability density function p _n (n) to the input signal x of the nonlinear system, the noise is used to correct the nonlinear When the output signal y of the linear system is Bayesian estimated for the input parameter θ, the corresponding average mean square error is

In order to obtain the optimal additive noise corresponding to the minimum MMSE(p _n (n)) in (7), the following noise enhancement optimization problem is constructed:

的特性，可知一定有/>

成立，从而可将(8)式模型中多元函数求极值问题等价为(9)式中关于参数n的一元函数求极值问题：combine

The characteristics of , it can be seen that there must be />

established, so that the problem of finding the extreme value of the multivariate function in the model (8) can be equivalent to the problem of finding the extreme value of the one-variable function with respect to the parameter n in the model of (9):

表示当所加噪声n为常向量时，对应的平均最小均方误差；获得上述一元函数的优化解后，即可获得使均方误差最小时所需的加性噪声n^opt；in

Indicates the corresponding average minimum mean square error when the added noise n is a constant vector; after obtaining the optimal solution of the above-mentioned unary function, the additive noise n ^opt required to minimize the mean square error can be obtained;

4) Optimal noise-enhanced Bayesian estimation:

When using the noise-enhanced nonlinear system output signal y=T(x+n ^opt ) to estimate the input parameter θ, the Bayesian estimate that minimizes the mean square error is:

之间的均方误差为：Input parameter θ and its Bayesian estimator

The mean square error between is:

The present invention combines the noise enhancement and the Bayesian estimation method. After adding noise to the input signal of the nonlinear system, and using the output signal of the nonlinear system to perform Bayesian estimation on the input parameters, the difference between the estimated value of the parameter and the real value can be made The minimum mean square error between is further reduced.

The present invention mainly adopts the method of simulation experiment for verification, and all steps and conclusions are verified correctly on MATLAB R2016a.

Description of drawings

Fig. 1 is the workflow block diagram of the present invention.

Fig. 2 is the noise enhancement corresponding to different t values in the simulation of the present invention and the original optimal Bayesian estimation mean square error.

Fig. 3 is the noise enhancement corresponding to different A values in the simulation of the present invention and the mean square error of the original optimal Bayesian estimation.

Fig. 4 is the corresponding noise enhancement and original optimal Bayesian estimation mean square error of different μ _θ values in the simulation of the present invention.

Fig. 5 is the noise enhancement corresponding to different σ _θ values in the simulation of the present invention and the original optimal Bayesian estimation mean square error.

Detailed ways

The present invention will be further described below in conjunction with the examples, but it should not be understood that the scope of the subject of the present invention is limited to the following examples. Without departing from the above-mentioned technical ideas of the present invention, various replacements and changes made according to common technical knowledge and conventional means in this field shall be included in the protection scope of the present invention.

This embodiment discloses a Bayesian parameter estimation method based on noise enhancement, comprising the following steps:

1) Build a noise-enhanced nonlinear system:

Described nonlinear system comprises three parts: nonlinear system input signal, nonlinear system and nonlinear system output signal; nonlinear system input signal x is closely related to parameter θ, and the value of θ is determined by its probability density function p _θ ( θ) is determined; add an independent additive noise n to the input signal x of the nonlinear system, and after passing through the nonlinear system, obtain the output signal y=T(x+n) of the noise-corrected nonlinear system, where T( ) represents The transfer function of the nonlinear system;

2) Establish a noise-enhanced Bayesian parameter estimation model:

之间的均方误差ε(y)为The input parameter θ is estimated using the nonlinear system output signal y. When the value of y is constant, the input parameter θ and its estimator

The mean square error ε(y) between

和/>

分别表示在非线性系统输出信号y一定的情况下输入参数θ的条件均值和条件方差。此外，p(θ|y)表示输入参数θ的后验概率，由(13)式可得in

and />

Respectively represent the conditional mean and conditional variance of the input parameter θ when the output signal y of the nonlinear system is constant. In addition, p(θ|y) represents the posterior probability of the input parameter θ, which can be obtained from formula (13)

Where p _y (y) represents the probability density function of the output signal y of the nonlinear system, and p _y (y|θ) represents the conditional probability density function of the output signal y of the nonlinear system when the value of parameter θ is constant.

无关，所以当y值一定时使得均方误差ε(y)最小的贝叶斯估计为According to formula (12), since var(θ|y) is non-negative and is related to the estimator

irrelevant, so when the value of y is constant, the Bayesian estimate that minimizes the mean square error ε(y) is

That is to say, when the value of y is constant, the minimum mean square error ε _MMSE (y) corresponding to Bayesian estimation is the conditional variance of the input parameter θ:

表示在非线性系统输出信号y一定的情况下θ²的期望。进一步地，对任意非线性系统输出信号y而言，使得均方误差最小的贝叶斯估计均为(14)式中的/>

因此，使得所有可能的非线性系统输出信号y的平均均方误差最小的估计同样为/>

即/>

是使得∫ε(y)p_y(y)dy最小的估计，对应的平均均方误差为in

Indicates the expectation of θ ² when the output signal y of the nonlinear system is constant. Furthermore, for any nonlinear system output signal y, the Bayesian estimate that minimizes the mean square error is the formula (14)

Therefore, the estimate that minimizes the average mean square error of the output signal y of all possible nonlinear systems is also />

i.e. />

is the estimate that makes ∫ε(y)p _y (y)dy the smallest, and the corresponding average mean square error is

When the additive noise n with probability density function p _n (n) is added to the input signal x of the nonlinear system, the conditional probability density function of the output signal y of the nonlinear system can be calculated as

Further, the probability density function of the output signal y of the nonlinear system is calculated as follows

in

It can be regarded as the probability density function of the output signal y of the nonlinear system when the added noise n is a constant vector, and the definition of the probability density function shows that 0≤R _y (n)≤1.

Further, E(θ|y) and E(θ ² |y) are calculated as:

in

3) Solve for the additive noise needed to minimize the Bayesian cost:

Substituting equations (18), (20) and (21) into equation (16), it can be seen that when adding additive noise n with probability density function p _n (n) to the input signal x of the nonlinear system, the non-linear When the output signal y of the linear system is Bayesian estimated for the input parameter θ, the corresponding minimum mean square error is ε _MMSE = MMSE(p _n (n)), where

(24) where E _n represents the expectation based on p _n (n).

In order to obtain the optimal additive noise corresponding to the noise-enhanced minimum mean square error MMSE(p _n (n)) in (24), the following noise-enhanced optimization model is constructed:

当z₂≥0时，函数F(z)的黑塞矩是半正定的，从而可得F(z)是凸函数。反之，当z₂≥0时，函数/>

为凹函数，则有(26)式成立In order to solve the optimization problem in (25), first introduce the function

When z ₂ ≥ 0, the Hessian moment of the function F(z) is positive semi-definite, so it can be obtained that F(z) is a convex function. Conversely, when z ₂ ≥ 0, the function />

is a concave function, then formula (26) holds

Let z ₁ =G _y (n), z ₂ =R _y (n) and z ₃ =J _y (n). Because R _y (n)≥0, so for any nonlinear system output signal y and any possible noise probability density function p _n (n), it is known that

The integral on both sides of the inequality (27) has the following results

表示当所加噪声n为常向量时，对应的平均最小均方误差。由于/>

从而一定有/>

进一步地，in

Indicates the corresponding average minimum mean square error when the added noise n is a constant vector. due to />

so there must be />

further,

To sum up, the problem of finding the extreme value of the multivariate function in formula (25) can be equivalent to the problem of finding the extreme value of the one-variable function with respect to the parameter n in formula (29):

After obtaining the optimal solution of the above-mentioned unary function, the additive noise n ^opt that minimizes the mean square error can be obtained;

4) Optimal noise-enhanced Bayesian estimation:

The input parameter θ is estimated by using the noise-enhanced nonlinear system output signal y=T(x+n ^opt ), so that the Bayesian estimate with the smallest mean square error is:

之间的均方误差为：Input parameter θ and its Bayesian estimator

The mean square error between is:

Effect of the present invention can be further illustrated by following simulation experiments:

即θ服从均值为μ_θ、方差为/>

的高斯分布。此外，v为零均值的非对称高斯混合背景噪声，其概率密度函数表示为p_v(v)＝tγ(v；(1-t)μ_b,σ_b ²)+(1-t)γ(v；-tμ_b,σ_b ²)，其中0＜t＜1。当θ值一定时，非线性系统输入信号x的条件概率密度函数为p_x(x|θ)＝p_v(x-θ)。假设非线性系统为限幅系统，给非线性系统输入信号x加入常量n作为噪声时，对应的非线性系统输出信号y为：Assuming that the input signal of the nonlinear system is x=θ+v, where θ is an unknown parameter, the corresponding probability density function is

That is, θ obeys the mean value μ _θ and the variance is />

Gaussian distribution. In addition, v is an asymmetric Gaussian mixture background noise with zero mean, and its probability density function is expressed as p _v (v)=tγ(v; (1-t)μ _b ,σ _b ² )+(1-t)γ( v; -tμ _b ,σ _b ² ), where 0<t<1. When the θ value is constant, the conditional probability density function of the nonlinear system input signal x is p _x (x|θ) = p _v (x-θ). Assuming that the nonlinear system is a limiting system, when adding a constant n as noise to the input signal x of the nonlinear system, the corresponding output signal y of the nonlinear system is:

的估计和LMMSE(n^opt)的获取。以t＝0.75、A＝3、μ_θ＝3、σ_θ＝1、μ_b＝3和σ_b＝1为例，在没有给非线性系统输入信号x加任何噪声时，对应的原贝叶斯估计的最小均方误差为0.8067，通过给非线性系统输入信号x加入常量n^opt＝-2.75可使得贝叶斯估计的最小均方误差降至0.7003，与原贝叶斯估计相比性能提升了13.2％。Using MATLAB language programming to realize the solution to n ^opt ,

Estimation of and acquisition of LMMSE(n ^opt ). Taking t=0.75, A=3, μ _θ =3, σ _θ =1, μ _b =3 and σ _b =1 as an example, when no noise is added to the input signal x of the nonlinear system, the corresponding original Bayes The minimum mean square error of the Bayesian estimate is 0.8067, and the minimum mean square error of the Bayesian estimate can be reduced to 0.7003 by adding a constant n ^opt = -2.75 to the input signal x of the nonlinear system, which improves performance compared with the original Bayesian estimate up 13.2%.

Table 1 shows that when A=3, μ _θ =3, σ _θ =1, μ _b =3 and σ _b =1, the input signals to the nonlinear system are respectively in the case of t values of 0.075, 0.75 and 0.9 When the constant n ^opt is added, the minimum mean square error corresponding to Bayesian estimation before and after adding noise.

Table 1 The minimum mean square error corresponding to Bayesian estimation before and after adding noise

It can be seen from Table 1 that under certain circumstances, adding appropriate constants to the input signal of the nonlinear system can further improve the performance of the optimal Bayesian estimation.

In order to further study the estimation performance achieved by adding different noises to the input signal of the nonlinear system, the background noise parameter t, the nonlinear system threshold A, the mean value μ _θ and the standard deviation σ _θ of the input parameter θ are successively changed to compare different The mean square error of the optimal Bayesian estimation before and after adding noise under the condition is as follows:

Keep A=3, μ _θ =3, σ _θ =1, μ _b =3 and σ _b =1, and increase t from 0 to 1 at an interval of 0.05. For each value of t, add the corresponding optimal additive noise n ^opt to the input signal x of the nonlinear system, and obtain the minimum mean square error of the noise-enhanced Bayesian estimation corresponding to the added noise, and compare with the original noise without adding noise The minimum mean square error of Yeesian estimation is compared, and the results are shown in Figure 2. As the value of t increases, the minimum mean square error corresponding to the noise-enhanced Bayesian estimation and the original Bayesian estimation first increases and then decreases, and the former is symmetrical about t=0.5. In addition, for any possible value of t, the minimum mean square error corresponding to the noise-enhanced Bayesian estimation is smaller than that corresponding to the original Bayesian estimation.

Keep t = 0.75, μ _θ = 3, σ _θ = 1, μ _b = 3 and σ _b = 1, increase A from 0 to 10 at an interval of 0.5, and solve the corresponding noise enhancement maximum value for each value of A The mean square error of Bayesian estimation is compared with that without noise, and the results are shown in Figure 3. When the value of A gradually increases from zero, the minimum mean square error of the original Bayesian estimation gradually decreases from 0.9358, and when the value of A is greater than 5.75, it remains constant at 0.6895. When 0<A<7.5, the minimum mean square error of the noise-enhanced Bayesian estimation is smaller than that of the original Bayesian estimation, and when A>7.5, no matter what noise is added, the corresponding mean square error of the Bayesian estimation cannot be made The square error is reduced.

Keep t = 0.75, A = 3, σ _θ = 1 μ _b = 3 and σ _b = 1, increase μ _θ from 0 to 5 at an interval of 0.25, and solve the corresponding optimal noise enhancement for each value of μ _θ The mean square error of Bayesian estimation is compared with that without noise, and the results are shown in Figure 4. The minimum mean square error corresponding to the original Bayesian estimation increases with the increase of μ _θ , while the minimum mean square error corresponding to the optimal noise-enhanced Bayesian estimation remains constant at 0.7003. This shows that the performance of the noise-enhanced Bayesian estimation for μ _θ > 0 is the same as that of the original Bayesian estimation for μ _θ = 0. In addition, the degree of improvement of the minimum mean square error corresponding to Bayesian estimation increases as the value of μ _θ increases.

Keeping t = 0.75, A = 3, μ _θ = 3, μ _b = 3, and σ _b = 1, increase σ _θ from 0 to 2 at intervals of 0.1, and solve for each value of σ _θ the corresponding noise enhancement The mean square error of the optimal Bayesian estimation is compared with that without noise, and the results are shown in Figure 5. The minimum mean square error corresponding to the noise-enhanced Bayesian estimation and the original Bayesian estimation is a monotonically increasing function of σ _θ . When the value of σ _θ approaches zero, it is impossible to improve the performance of Bayesian estimation no matter what noise is added. When the value of σ _θ increases to a certain extent, the degree of improvement of the minimum mean square error corresponding to Bayesian estimation through adding noise increases with the increase of σ _θ .

Claims (1)

1. a Bayesian parameter estimation method based on noise enhancement, is characterized in that, comprises the following steps: 1) Build a noise-enhanced nonlinear system: The nonlinear system includes three parts: the nonlinear system input signal, the nonlinear system and the nonlinear system output signal; the nonlinear system input signal x is closely related to the input parameter θ, and the value of θ is determined by its probability density function p _θ (θ) is determined; add an independent additive noise n to the input signal x of the nonlinear system, and after passing through the nonlinear system, obtain the output signal y=T(x+n) of the noise-corrected nonlinear system, where T( ) Represents the transfer function of the nonlinear system; 2) Establish a noise-enhanced Bayesian parameter estimation model: The input parameter θ is estimated by using the output signal y of the nonlinear system; when the value of y is constant, the input parameter θ and its estimator

The Bayesian estimate with the smallest mean square error between

The mean square error corresponding to the formula is ε _MMSE (y)＝E(θ ² |y)-E ² (θ|y) (2) Formula Where E(θ|y) and E(θ ² |y) respectively represent the expectation of θ and θ ² when the value of y is certain; further,

It is also an estimate that minimizes the average mean square error, and the corresponding minimum average mean square error is

Mode where p _y (y) is the probability density function of the output signal y of the nonlinear system; p _y (y), E(θ|y) and E(θ ² |y) are calculated as:

in

p _n (n) is the probability density function of additive noise n, and p _x (x|θ) represents the conditional probability density function of the nonlinear system input signal x when the value of θ is certain; 3) Solve for the additive noise needed to minimize the Bayesian cost: Substituting equations (4), (5) and (6) into equation (3), it can be seen that when adding additive noise n with probability density function p _n (n) to the input signal x of the nonlinear system, the noise is used to correct the nonlinear When the output signal y of the linear system is Bayesian estimated for the input parameter θ, the corresponding average mean square error is

In order to obtain the optimal additive noise corresponding to the minimum MMSE(p _n (n)) in (7), the following noise enhancement optimization problem is constructed:

combine

The characteristics of , it can be seen that there must be />

established, so that the problem of finding the extreme value of the multivariate function in the model (8) can be equivalent to the problem of finding the extreme value of the one-variable function with respect to the parameter n in the model of (9):

in

Indicates the corresponding average minimum mean square error when the additive noise n is a constant vector; after obtaining the optimal solution of the above-mentioned unary function, the additive noise n ^opt required to minimize the mean square error can be obtained; 4) Optimal noise-enhanced Bayesian estimation: When using the noise-enhanced nonlinear system output signal y=T(x+n ^opt ) to estimate the input parameter θ, the Bayesian estimate that minimizes the mean square error is:

Input parameter θ and its Bayesian estimator

The mean square error between is:

Images