Shrinkage estimation of varying covariate effects based on quantile regression

Varying covariate effects often manifest meaningful heterogeneity in covariate-response associations. In this paper, we adopt a quantile regression model that assumes linearity at a continuous range of quantile levels as a tool to explore such data dynamics. The consideration of potential non-constancy of covariate effects necessitates a new perspective for variable selection, which, under the assumed quantile regression model, is to retain variables that have effects on all quantiles of interest as well as those that influence only part of quantiles considered. Current work on l ₁-penalized quantile regression either does not concern varying covariate effects or may not produce consistent variable selection in the presence of covariates with partial effects, a practical scenario of interest. In this work, we propose a shrinkage approach by adopting a novel uniform adaptive LASSO penalty. The new approach enjoys easy implementation without requiring smoothing. Moreover, it can consistently identify the true model (uniformly across quantiles) and achieve the oracle estimation efficiency. We further extend the proposed shrinkage method to the case where responses are subject to random right censoring. Numerical studies confirm the theoretical results and support the utility of our proposals.

The authors are grateful to the editor, associate editor, and the two referees for many helpful comments. This research has been supported by the National Heart, Lung, And Blood Institute of the National Institute of Health under Award Number R01HL 113548 (the first author). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Health.

Authors and Affiliations

1. Department of Biostatistics and Bioinformatics, Emory University, 1518 Clifton Rd. NE, Atlanta, USA

   Limin Peng

2. Department of Statistics and Applied Probability, National University of Singapore, Singapore, 117546, Singapore

   Jinfeng Xu

3. Department of Rehabilitation Medicine, Emory University, Atlanta, 30322, USA

   Nancy Kutner

Corresponding author

Correspondence to Limin Peng.

Appendix A: Proof of Theorem 1

Define B _n,τ (C)={β: β=β ₀(τ)+(n ^−1/2 ℓ ₂ n)u,∥u∥≤C}, and let ∂B _n,τ (C) denote the boundary set of B _n,τ (C). Since \(W_{n, \lambda_{n}}({\boldsymbol {\beta }}; \tau)\) is convex in β for all τ∈Δ, it is sufficient to show that for any ϵ>0, there exists C ₀>0 and N ₀>0 such that for n≥N ₀,

To show (2), first note that

Write and u(τ;β)=β−β ₀(τ), and let \(\mathcal {D}\) and \(\mathcal {D}_{i}\) respectively denote operators such that \(\mathcal {D}(U)=U-E(U)\) and \(\mathcal {D}_{i}(U)=U-E(U|\boldsymbol {Z}_{i})\) for a random variable U. We can further decomposed the term I as I=III+IV, where

and

For the term IV ₁, we can show that the function class, , is a Donsker class (page 81 in Van der Vaart and Wellner (2000)), where \(\mathcal{A}(C_{0})=\{\boldsymbol {b}\in R^{p}:\ \inf_{\tau\in \varDelta }\|\boldsymbol {b}-{\boldsymbol {\beta }}_{0}(\tau)\|\leq C_{0}\}\). Given the uniform boundedness of the functional class \(\mathcal{F}\) and since \(\mathcal{A}(C_{0})\) covers \(\mathcal{B}_{n,\tau}(C_{0})\) for all τ∈Δ, we can apply the functional law of the iterated logarithm (LIL) (Goodman et al. 1981) to \(n^{-1}\sum_{i=1}^{n}\) , and get

Similarly, we can show that, for j=2,…,8,

Therefore, we have

For the term III, note that, under condition C2 (i), inf _τ∈Δ,z f _τ (0|z)>0. By the definition of B _n,τ (C ₀), there exists N ₁>0 such that for n≥N ₁,

This, coupled with condition C2 (ii), implies that for n>N ₁, f _τ (x|z)>inf _τ∈Δ,z f _τ (0|z)/2 for any with β∈∂B _n,τ (C ₀). Let δ ₂≡inf _τ,z f _τ (0|z)/2 and , where \({\rm eig}\min(\cdot)\) denotes the minimum eigenvalue of a matrix. Let E _Z (⋅) denote the expectation with regard to Z. We get, for any τ∈Δ and β∈∂B _n,τ (C ₀),

Since a similar result can be shown for the second term in III, it follows that

For the term II, it is easy to see that

By the uniform consistency of \(\tilde{\beta}_{n}(\tau)\), for 2≤j≤s, \(\sup_{\tau\in \varDelta } |w_{n,j}(\tau)|=(\sup_{\tau\in \varDelta }|\tilde{\beta}_{n}^{(j)}(\tau )|)^{-1}=O_{p}(1)\). Therefore, with (n ^1/2 ℓ ₂ n)⁻¹ λ _n =O(1),

Based on equations (3), (4), and (5), it follows that

Therefore, (2) holds if we choose C ₀ large enough. This completes the proof of Theorem 1.

Appendix B: Proof of Theorem 2

Define \(\operatorname {sgn}(x)=I(x>0)-I(x<0)\), \(U_{n, j}({\boldsymbol {\beta }}; \tau)= \frac {\partial W_{n, \lambda_{n}}({\boldsymbol {\beta }}; \tau)}{\partial\beta^{(j)}}\), and let . Define , , and .

First, by Theorem 1, for j=2,…,s, we have

Next, we note that given the uniform consistency of \(\widehat{{\boldsymbol {\beta }}}_{n, \lambda_{n}}^{\mathrm {US}}(\tau)\) implied by Theorem 1, it can be shown by following the lines of Lemma 1 in Peng and Huang (2008) that

(7)

This implies, for j=s+1,…,p,

By the definition of \(\widehat{{\boldsymbol {\beta }}}_{n, \lambda_{n}}^{\mathrm {US}}(\tau)\),

Applying Functional LIL to n ⁻¹ U _n,j {β ₀(τ);τ} gives

An application of Taylor expansion to \(\mu_{j}\{\widehat{{\boldsymbol {\beta }}}_{n, \lambda_{n}}^{\mathrm {US}}(\tau)\}-\mu_{j}\{{\boldsymbol {\beta }}_{0}(\tau)\}\), coupled with Theorem 1 and the uniform boundedness of A _j (β), shows that

In addition, the proof of Theorem 1 can be used to justify that \(\sup_{\tau\in \varDelta }|\tilde{\beta}_{n}^{(j)}(\tau )|=O_{p}(n^{-1/2}\ell_{2} n)\) and thus

By (9), (10), (11), and (12), with a fixed M>0,

This, coupled with (8), implies that, for j=s+1,…,p,

The proof of Theorem 2(i) is completed based on (6) and (13).

Let \({\buildrel a\over =}\) denote asymptotic equivalence in the sense that the difference converges to zero in probability uniformly in τ∈Δ. Define . By the result in Theorem 2(i), we have

and hence \(n^{-1/2}\boldsymbol {U}_{n}(\bar{{\boldsymbol {\beta }}}_{n}; \tau){\buildrel a\over =}0\). Using the result in (7) and applying Taylor expansion to \(\mu_{j}\{\bar{{\boldsymbol {\beta }}}_{n}(\tau)\}-\mu_{j}\{{\boldsymbol {\beta }}_{0}(\tau)\}\), we get

(14)

where \(\check{{\boldsymbol {\beta }}}(\tau)\) is on the line segment between β ₀(τ) and \(\bar{{\boldsymbol {\beta }}}(\tau)\), and .

Since lim _n→∞ n ^−1/2 λ _n =0, sup _τ∈Δ |w _n,j (τ)|=O _p (1) for j=1,…,s, and \(A\{\check{{\boldsymbol {\beta }}}(\tau)\}{\buildrel a\over =}A\{{\boldsymbol {\beta }}_{0}(\tau)\}\) given the uniform convergence of \(\widehat{{\boldsymbol {\beta }}}_{n,\lambda_{n}}^{\mathrm {US}}(\tau)\) to β ₀(τ), it follows from (14) that

(15)

where A ₁₁(⋅) stands for the submatrix of A(⋅) formed by the first s rows and columns. An application of the Donsker theorem based on (15) thus shows that \(n^{1/2}\{\widehat{{\boldsymbol {\beta }}}_{n,\lambda_{n}}^{\mathrm {US}(1:s)}(\tau)-{\boldsymbol {\beta }}_{0}^{(1:s)}(\tau)\}\) converges weakly to a mean zero Gaussian process with the covariance matrix,

Using similar steps, we can show that (15) still holds when \(\widehat{{\boldsymbol {\beta }}}_{\rm oracle}(\tau)\) is in place of \(\widehat{{\boldsymbol {\beta }}}_{n,\lambda_{n}}^{\mathrm {US}(1:s)}(\tau)\). This completes the proof of Theorem 2.

Reprints and permissions