A Bayesian Predictive Discriminant Analysis with Screened Data

The topic of analyzing multivariate screened data has received a great deal of attention over the last few decades. In the standard multivariate problem, an analysis of data generated from a p-dimensional screened random vector $x\stackrel{d}{=}[v|v_0\in {\mathbf{C}}_q)]$ is our issue of interest, where the $p\times 1$ random vector v and the $q\times 1$ random vector $v_0$ (called the screening vector) are jointly distributed with the correlation matrix $Corr(v,v_0^{\top })\ne \mathbf{0}.$ Thus, we observe x only when the unobservable screening vector $v_0$ belongs to a known subset ${\mathbf{C}}_q$ of its space ${\mathbb{R}}^q$, such that $0\le P(v_0\in {\mathbf{C}}_q)\le 1.$ That is, x is subject to the screening scheme or hidden truncation (or simply truncation if $v=v_0$). Model parameters underlying the joint distribution of v and $v_0$ are then estimated from the screened data (i.e., observations of x) using the conditional density $f\left(x\right|v_0\in {\mathbf{C}}_q).$

The screening of sample (or sample selection) arises as open in practice as a result of controlling observability of the outcome of interest in the study. For example, the dataset consists of the Otis IQ test scores (the values of x) of freshmen of a college. These students had been screened in the college admission process, which examines whether their prior school grade point average (GPA) and the Scholastic Aptitude Test (SAT) scores (i.e., the screening values denoted by $v_0$) are satisfactory. What the true vale of screening vector variable $v_0$ of each student is may not be available due to a college regulation. The observations available are the IQ values of x, the screened data. For the application with real screened data, one can refer to that with the student aid grants data given by [1] and that with the U.S. labor market data given by [2], as well. A variety of methods have been suggested for analyzing such screened data. See, e.g., [3,4,5,6], for various distributions for modeling and analyzing screened data; see, [7,8] for the estimative classification analysis with screened data; and see, e.g., [1,2,9,10], for the regression analysis with screened response data.

The majority of existing methods rely on the fact that v and $v_0$ are jointly multivariate normal, and the screened observation vector x is subject to a univariate screening scheme defined by an open interval ${\mathbf{C}}_q$ with $q=1.$ In many practical situations, however, the screened data are generated from a non-normal joint distribution of v and $v_0$, having a multivariate screening scheme defined by a q-dimensional ($q>1$) rectangle region ${\mathbf{C}}_q$ of $v_0.$ In this case, a difficulty in applications with the screened data is that the empirical distribution of the screened data is skewed; its parametric model involves a complex density; and hence, standard methods of analysis cannot be used. See [4,6] for the conditional densities, $f\left(x\right|v_0\in {\mathbf{C}}_q)$, useful for fitting the rectangle screened data generated from a non-normal joint distribution of v and $v_0.$ In this article, we develop yet another multivariate technique applicable for analyzing the rectangle screened data: we are interested in constructing a Bayesian predictive discrimination procedure for the data. More precisely, we consider a Bayesian multivariate technique for sorting, grouping and prediction of multivariate data generated from K rectangle screened populations. In the standard problem, a training sample $\mathcal{D}=\{(z_i,x_i),i=1,\dots ,n\}$ is available, where, for each $i=1,\dots ,n$, $x_i$ is a $p\times 1$ rectangle screened observation vector coming from one of K populations and taking values in ${\mathbb{R}}^p$, and $z_i$ is a categorical response variable representing the population membership, so that $z_i=k$ implies that the predictor $x_i$ belongs to the k-th rectangle screened population (denoted by ${\pi }_k$), $k=1,\dots ,K.$ Using the training sample $\mathcal{D}$, the goal of the predictive discriminant analysis is to predict population membership of a new screened observation x based on the posterior probability of x belonging to ${\pi }_k.$ The posterior probability is given by: where z is the the population membership of x, $p(z=k|\mathcal{D})$ is the prior probability of ${\pi }_k$ updated by the training sample $\mathcal{D}$ and:

$p\left(x\right|{\Theta }_k)=p\left(x\right|v_0\in {\mathbf{C}}_q,z=k)$ and $p\left({\Theta }_k\right|\mathcal{D},z=k)$, respectively, denote the predictive density, the probability density of x and the posterior density of parameters ${\Theta }_k$ associated with ${\pi }_k.$ One of the first and most applied predictive approaches by [11] is the case of unscreened and normally-distributed populations ${\pi }_k$ with unknown parameters ${\Theta }_k=\{{\mathrm{\mu }}_k,{\Sigma }_k\}$, namely ${\pi }_k:N_p({\mu }_k,{\Sigma }_k)$ for $k=1,\dots ,K.$ This is called a Bayesian predictive discriminant analysis with normal populations ($BPD{A}_N$) in which a multivariate Student t distribution is obtained for Equation (2).

A practical example where the predictive discriminant analysis with the rectangle screened populations (${\pi }_k$’s) is applicable is in the discrimination between passed and failed pairs of applicants in a college admission process (the second screening process). Consider the case where college admission officers wish to set up an objective criterion (with a predictor vector x) for admitting students for matriculation; however, the admission officers must first ensure that a student with observation x has passed the first screening process. The first screening scheme may be defined by the q-dimensional region ${\mathbf{C}}_q$ of the random vector $v_0$ (consisting of SAT scores, high-school GPA, and so on), so that only the students who satisfy $v_0\in {\mathbf{C}}_q$ can proceed to the admission process. In this case, we encounter a crucial problem for applying the normal classification by [11]; given the screening scheme $v_0\in {\mathbf{C}}_q$, the assumption of the multivariate normal population distribution for $\left[x\right|z=k]\stackrel{d}{=}\left[x\right|{\pi }_k]$, $k=1,2,\dots ,K$ is not valid. The work in [7,12] found that the normal classification shows a lack of robustness to the departure from the normality of the population distribution, and hence, the performance of the normal classification can be very misleading, if used with the continuous, but non-normal or screened normal input vector x.

Thus, the predictive density in Equation (2) has two specific features to be considered for Bayesian predictive discrimination with the rectangle screened populations, one about the prior distribution of the parameters ${\Theta }_k$ and the other about the distributional assumption of the population model with density $p\left(x\right|{\Theta }_k).$ For the unscreened populations case, there have been a variety of studies that are concerned with the two considerations. See, for example, [11,13,14] for the choice of the prior distributions of ${\Theta }_k$, and see [15,16] for copious references to the literature on the predictive discriminant analysis with non-normal population models. Meanwhile, for deriving Equation (2) of the rectangle screened observation x, we need to develop a population model with density $p\left(x\right|{\Theta }_k)$ that uses the screened sample information in order to maintain consistency with the underlying theory associated with the populations ${\pi }_k$ generating the screened sample. Then, we propose a Bayesian hierarchical approach to flexibly incorporate the prior knowledge about ${\Theta }_k$ with the non-normal sample information, which is the main contribution of this paper to the literature on Bayesian predictive discriminant analysis.

The rest of this paper is organized as follows. Section 2 considers a class of screened scale mixture of normal (SSMN) population models, which well accounts for the screening scheme conducted through a q-dimensional rectangle region ${\mathbf{C}}_q$ of an external scale mixture of normal vector, $v_0.$ Section 3 proposes a hierarchical screened scale mixture of normal (HSSMN) model to derive the predictive density Equation (2) and proposes an optimal rule for Bayesian predictive discriminant analysis (BPDA) with the SSMN populations (abbreviated as $BPD{A}_{SSMN}$). Approximation of the rule is studied in Section 4 by using an MCMC method applied to the HSSMN model. In Section 5, a simulation study is done to check the convergence of the MCMC method and the performance of the $BPD{A}_{SSMN}$ by making a comparison between the $BPD{A}_{SSMN}$ and the $BPD{A}_N.$ Finally, concluding remarks are given in Section 6.

Assume that the joint distribution of respective $q\times 1$ and $p\times 1$ vector variables $v_0$ and v, associated with ${\pi }_k$, is $F\in \mathcal{F}$, where:

$k=1,\dots ,K$, $s=(q+p)$, η is a mixing variable with the pdf $g\left(\mathrm{\eta }\right)$, $\kappa \left(\mathrm{\eta }\right)$ is a suitably-chosen weight function and ${\mathrm{\mu }}_k^{*}$ and ${\Sigma }_k^{*}$ are partitioned corresponding to the orders of $v_0$ and v:

Notice that $\mathcal{F}$ defined by Equation (3) denotes a class of scale mixture of multivariate normal (SMN) distributions (see, e.g., [17,18] for details), equivalently denoted as $SM{N}_s({\mathrm{\mu }}_k^{*},{\Sigma }_k^{*},\kappa \left(\mathrm{\eta }\right),G)$ in the remainder of the paper, where $G=G\left(\mathrm{\eta }\right)$ denote the cdf of $\mathrm{\eta }.$

Given the joint distribution $\left[v^{*}\right|{\pi }_k]\sim SM{N}_s({\mathrm{\mu }}_k^{*},{\Sigma }_k^{*},\kappa \left(\mathrm{\eta }\right),G)$, the SSMN distribution is defined by the following screening scheme: where ${\mathbf{C}}_q(\alpha ,\beta )=\{v_0\in {\mathbb{R}}^q|\alpha \le v_0\le \beta \}$ is a q-dimensional rectangle screening region in the space of $v_0\in {\mathbb{R}}^q.$ Here, $\alpha ={({\alpha }_1,\cdots ,{\alpha }_q)}^T$, $\beta ={({\beta }_1,\cdots ,{\beta }_q)}^T$, and ${\alpha }_j<{\beta }_j$ for $j=1,\cdots ,q$. This region contains the cases of ${\mathbf{C}}_q(\alpha ,\infty )$ and ${\mathbf{C}}_q(-\infty ,\beta )$ as special cases.

The pdf of x is given by: where:

${\mathrm{\mu }}_{v_{0k}|x}={\mathrm{\mu }}_{0k}+{\Delta }_k^{\top }{\Sigma }_k^{-1}(x-{\mathrm{\mu }}_k)$ and ${\Sigma }_{v_{0k}|x}={\Sigma }_{0k}-{\Delta }_k^{\top }{\Sigma }_k^{-1}{\Delta }_k.$ Here, ${\varphi }_q(·\mathrm{\mu },\Sigma )$ and ${\overline{\Phi }}_q({\mathbf{C}}_q(\alpha ,\beta );\mathrm{\mu },\Sigma )$, respectively, denote the pdf and the probability of the rectangle region of a random vector $w\sim N_q(\mathrm{\mu },\Sigma )$. The latter is equivalent to $Pr(w\in {\mathbf{C}}_q(\alpha ,\beta )).$

One particular member of the class of SSMN distributions is the rectangle-screened normal (RSN) distribution defined by Equation (5) and Equation (6), for which $G\left(\mathrm{\eta }\right)$ is degenerate with $\kappa \left(\mathrm{\eta }\right)=1$. The work in [4,8] studied properties of the distribution and denoted it as the $RS{N}_p({\mathbf{C}}_q(\alpha ,\beta );{\mathrm{\mu }}_k^{*},{\Sigma }_k^{*})$ distribution. Another member of the class is the rectangle-screened p-variate Student t distributions ($RS{t}_p$) considered by [8]. Its pdf is given by: where $t_p(·|\mathbf{a},B,c)$ and ${\overline{T}}_p({\mathbf{C}}_p;\mathbf{a},B,c)$ are the respective pdf and probability of a rectangle region ${\mathbf{C}}_p$ of the p-variate Student t distribution with the location vector $\mathbf{a}$, the scale matrix B, the degrees of freedom c and:

Similar to the RSN distributions, the density Equation (7) of $\left[x\right|{\pi }_k]\sim RS{t}_p({\mathbf{C}}_q(\alpha ,\beta );{\mathrm{\mu }}_k^{*},{\Sigma }_k^{*},\mathrm{\nu })$ is obtained by taking $\kappa \left(\mathrm{\eta }\right)=1/\mathrm{\eta }$ and $\mathrm{\eta }\sim Gamma(\mathrm{\nu }/2,\mathrm{\nu }/2)$, i.e.,

The stochastic representations of the RSN and $RS{t}_p$ distributions are immediately obtained by applying the following lemma, for which detailed proof can be found in [4].

Lemma 1. Suppose $\left[x\right|{\pi }_k]\sim SSM{N}_p({\mathbf{C}}_q(\alpha ,\beta );{\mathrm{\mu }}_k^{*},{\Sigma }_k^{*},\kappa \left(\mathrm{\eta }\right),G)$. Then, it has the following stochastic representation in a hierarchical fashion,where ${\mathbf{Z}}_p\sim N_p(\mathbf{0},\kappa \left(\mathrm{\eta }\right)I_p)$ and ${\mathbf{Z}}_{{\mathbf{C}}_q-{\mathrm{\mu }}_{0k}}\stackrel{d}{=}\left[{\mathbf{Z}}_q\right|{\mathbf{Z}}_q\in {\mathbf{C}}_q(a_k,b_k)]$ are conditionally independent and ${\mathbf{Z}}_q\sim N_q(\mathbf{0},\kappa \left(\mathrm{\eta }\right){\Sigma }_{0k})$. Here, $a_k=\alpha -{\mathrm{\mu }}_{0k}$ and $b_k=\beta -{\mathrm{\mu }}_{0k}.$

Lemma 1 provides the following: (i) an intrinsic structure of the SSMN population distributions, which reveals a type of departure from the SMN law because the distribution of $\left[x\right|{\pi }_k]$ reduces to the SMN distribution if ${\Delta }_k=\mathbf{0}$ (i.e., $Cov(v_0,v|{\pi }_k)=\mathbf{0}$); (ii) the representation provides a convenient device for random number generation; (iii) it leads to a simple and direct construction of a HSSMN model for the BPDA with the SSMN populations, i.e., $\left[x\right|{\pi }_k]\sim SSM{N}_p({\mathbf{C}}_q(\alpha ,\beta );{\mathrm{\mu }}_k^{*},{\Sigma }_k^{*},\kappa \left(\mathrm{\eta }\right),G)$.