Posterior Contraction in Bayesian Inverse Problems Under Gaussian Priors

We study Bayesian inference in statistical linear inverse problems with Gaussian noise and priors in a separable Hilbert space setting. We focus our interest on the posterior contraction rate in the small noise limit, under the frequentist assumption that there exists a fixed data-generating value of the unknown. In this Gaussian-conjugate setting, it is convenient to work with the concept of squared posterior contraction (SPC), which is known to upper bound the posterior contraction rate. We use abstract tools from regularization theory, which enable a unified approach to bounding SPC. We review and re-derive several existing results, and establish minimax contraction rates in cases which have not been considered until now. Existing results suffer from a certain saturation phenomenon, when the data-generating element is too smooth compared to the smoothness inherent in the prior. We show how to overcome this saturation in an empirical Bayesian framework by using a non-centered data-dependent prior.

1. 1.

   When considering the SPC uniformly over some class of inputs x ^∗ then if follows from (3) that the best (uniform) contraction rate cannot be better than the corresponding minimax rate for statistical estimation.

Authors and Affiliations

1. Department of Mathematics and Statistics, University of Cyprus, Nicosia, Cyprus

   Sergios Agapiou

2. Weierstraß Institute for Applied Analysis and Stochastics, Berlin, Germany

   Peter Mathé

Corresponding author

Correspondence to Peter Mathé .

Editors and Affiliations

1. Fakultät für Mathematik, Technische Universität Chemnitz, Chemnitz, Germany

   Bernd Hofmann

2. Department of Mathematics, Federal University of Santa Catarin, Florianopolis, Brazil

   Antonio Leitão

3. Pura e Aplicada, Instituto Nacional de Matematica, Rio de Janeiro, Rio de Janeiro, Brazil

   Jorge P. Zubelli

Appendix

Proof (of Lemma 1)

We first express the element \(x^\delta _\alpha \) in terms of z ^δ.

We notice that

The expectation of the posterior mean with respect to the distribution generating z ^δ when x ^∗ is given, is thus

For the next calculations we shall use that

Therefore we rewrite

which proves the first assertion. The variance is \(\mathbb E^{x^{\ast }} \left \| x^\delta _\alpha - \mathbb E^{x^{\ast }}x^\delta _\alpha \right \|{ }_{ }^{2}\), and this can be written as in (8), by using similar reasoning as for the bias term.

Proof (of Proposition 1)

We notice that \(\left \| I + \alpha g_{\alpha }(B^{\ast }B) \right \|{ }_{}\leq 1 + \gamma _{\ast }\), which gives

Since \(\left \| \left ( \alpha + B^{\ast }B \right )^{-1} B^{\ast }B \right \|{ }_{}\leq 1\) we see that

and the proof is complete.

Proof (of Lemma 2)

Since C ₀ has finite trace, it is compact, and we use the eigenbasis (arranged by decreasing eigenvalues) u _j , j = 1, 2, … Under Assumption 1 this is also the eigenbasis for T ^∗ T. If t _j , j = 1, 2, … denote the eigenvalues then we see that

Correspondingly, \(C_0 = \sum _{j=1}^{\infty } \left ( \psi ^{2} \right )^{-1}(\tau _{j}) u_{j}\otimes u_{j}\), which gives the first assertion. Moreover, the latter representation yields that

such that

and the proof is complete.

Proof (of Proposition 2)

For the first item (1), we notice that \(\varphi \prec \varTheta _{\psi }^{2}\) if and only if φ(f ²(t)) ≺ t. The linear function t↦t is a qualification of Tikhonov regularization with constant γ = 1. Thus, by Lemma 3 we have

which completes the proof for this case. For item (2), we have that

For any 0 < α ≤ 1, we have α + t ≤ 1 + t, hence

We conclude that there exists a constant \(c_{1}=c_{1}(x^{\ast },\left \| B^{\ast }B \right \|{ }_{})\), such that for small α it holds

On the other hand, since t ≺ φ(f ²(t)), there exists a constant c ₂ > 0 which depends only on the index functions φ, f and on \(\left \| B^{\ast }B \right \|{ }_{}\), such that

For item (3), we have that

and the proof is complete.

Proof (of Lemma 4)

The continuity is clear. For the monotonicity we use the representation (15) to get

The trace on the right hand side is positive. Indeed, if (s _j , u _j , u _j ) denotes the singular value decomposition of B ^∗ B then this trace can be written as

where the right hand side is positive since the operator C ₀ is positive definite. Thus, if α < α ^′ then \(S_{T,C_0}(\alpha ) - S_{T,C_0}(\alpha ^{\prime })\) is positive, which proves the first assertion.

The proof of the second assertion is simple, and hence omitted. To prove the last assertion we use the partial ordering of self-adjoint operators in Hilbert space, that is, we write A ≤ B if 〈Ax, x〉≤〈Bx, x〉, x ∈ X, for two self-adjoint operators A and B. Plainly, with \(a:= \left \| T^{\ast }T \right \|{ }_{}\), we have that T ^∗ T ≤ aI. Multiplying from the left and right by \(C_0^{1/2}\) this yields B ^∗ B ≤ aC ₀, and thus for any α > 0 that αI + B ^∗ B ≤ αI + aC ₀. The function t↦ − 1/t, t > 0 is operator monotone, which gives \(\left ( \alpha I + aC_0 \right )^{-1} \leq \left ( \alpha I + B^{\ast }B \right )^{-1}\). Multiplying from the left and right by \(C_0^{1/2}\) again, we arrive at

This in turn extends to the traces and gives that

Now, let us denote by \(t_j,\ j\in \mathbb N\), the singular numbers of C ₀, then we can bound

If \(S_{T,C_0}(\alpha )\) were uniformly bounded from above, then there would exist a finite natural number, say N, such that \(t_{N} \geq \frac \alpha a > t_{N+1}\), for α > 0 small enough. But this would imply that t _N+1 = 0, which contradicts the assumption that C ₀ is positive definite.

Lemma 5

For t > 0 let \(\varTheta ^2_{\psi }(t)=t\exp (-2qt^{-\frac {b}{1+2a}})\) , for some q, b, a > 0. Then for small s we have \((\varTheta ^2_{\varPsi })^{-1}(s)\sim (\log s^{-\frac 1{2q}})^{-\frac {1+2a}{b}}\).

Proof

Let

and observe that t is small if and only if s is small. Applying [3, Lem 4.5] for x = t ⁻¹ we get the result.

Proof (of Proposition 6)

In this example the explicit solution of Eq. (16) in Theorem 1 is more difficult. However, as discussed in Sect. 3.4, it suffices to asymptotically balance the squared bias and the posterior spread using an appropriate parameter choice α = α(δ). Indeed, under the stated choice of α the squared bias is of order

while the posterior spread term is of order

Proof (of Proposition 8)

According to the considerations in Remark 10, it is straightforward to check that without preconditioning the best SPC rate that can be established is \(\delta ^{\frac {4+8a+8p}{3+4a+6p}}\) which proves item (1). In the preconditioned case, the explicit solution of Eq. (16) in Theorem 1, which in this case has the form

is again difficult. However, as discussed in Sect. 3.4, it suffices to asymptotically balance the squared bias and the posterior spread using an appropriate parameter choice α = α(δ). Indeed, using [3, Lem 4.5] we have that the solution to the above equation behaves asymptotically as the stated choice of α, and substitution gives the claimed rate.

Proof (of Proposition 10)

We begin with items (1) and (3). The explicit solution of Eq. (16) in Theorem 1, which in this case has the form

is difficult. As discussed in Sect. 3.4, it suffices to asymptotically balance the squared bias and the posterior spread using an appropriate parameter choice α = α(δ). Indeed, under the stated choice of α both quantities are bounded from above by \(\delta ^{\frac {2\beta }{\beta +q}}\). For item (2), according to the considerations in Remark 10, it is straightforward to check that without preconditioning the best SPC rate that can be established is \(\delta ^{\frac {4q}{\beta +q}}\).

Reprints and permissions

© 2018 Springer International Publishing AG

Policies and ethics