PRINCIPAL COMPONENTS AND THE MAXIMUM LIKELIHOOD METHODS AS TOOLS TO ANALYZE

LARGE DATA WITH A PSYCHOLOGICAL TESTING EXAMPLE

Abstract:
Based on the study of correlations between large numbers of quantitative variables, the method factor analysis
(FA) aims at finding structural anomalies of a commonality composed of p-variables and many data (large sample
size). It reduces the number of original (observed) variables by calculating a smaller number of new variables,
which are called factors. This paper overviews the factor analysis and its application. Here, the method of principal
components analysis (PCA) to calculate factors with Varimax rotation is applied. The method of maximum
likelihood with Quartimax rotation is used for comparison purposes involving the statistic package SPSS. The
results report the usefulness of multivariate statistical analysis (factor analysis). The application is done by a set of
data from psychological testing.
1. Introduction:
Factor analysis is a class of multivariate statistical methods whose primary purpose is data reduction and
summarization. It addresses the problem of analyzing the interrelationship among many variables and then
explaining these variables in terms of their common, underlying factors.
This method is a summary of the principal component Analysis (PCA) method since the results and the
interpretations of methods are similar, but the mathematical models are different. The FA method relates to the
correlations between many quantitative variables. It reduces the number of primary variables by calculating a
smaller number of new variables, which are called factors. This reduction is achieved by grouping variables into
factors which means each variable within each factor is closely correlated and variables that belong to different
factors are less correlated.
In the area of factors calculation, THE PRINCIPAL COMPONENTS ANALYSIS (PCA), and the MAXIMUM
LIKELIHOOD METHOD (ML), are two of the most applied techniques.
If PCA is used, then the minimum average partial method can be used whereas if ML is used, then fit indices
can be used as described by (Fabrigar et al. 1999, Browne & Cudeck 1992) For a more comprehensive review of
options we can see. There are also other methods named in general as factor rotation, which make an orthogonal
transformation of the loading’s matrix. Here, the loadings of variables of an extracted factor are maximized and the
loadings of variables in the other factors are minimized. In this case, the variables are expected to be independent.
Consequently, one of the orthogonal rotations is Varimax, which attempts to maximize the variance of squared
loadings on a factor.
The scores of factors are calculated utilizing the regress technique. Their score values are saved in new variables
in the data file and can be used later for statistical analysis. In this way, each factor now embodies a linear
combination of the primary variables. We calculate the numerical characteristics for these variables from which the
new variables are standardized with mean 0 and variance 1. Achievable by means of the correlation matrix, the new
variables must be uncorrelated among them. The present paper overviews factor analysis and related factors
extracting methods. In addition, it reports the usefulness of factors analysis, and the new variables (factors) are
exemplified. In this paper, the use of PCA when calculating factors with Varimax rotation is reported. The
maximum likelihood method with Quartimax rotation is applied for calculation purposes. The correlations matrix
is applied and, the factor number is chosen by the eigenvalues that are greater than 1.
2. Factors rotation and interpretation
 In the area of FA implementation, usually, the factors found by methods previously described cannot be easily
interpreted. Determining exactly the variables belonging to factors one, two, and so on, up to factor q is difficult, as
some variables tend to load on some factors. Consequently, many factors can be interpreted by one variable. On the
contrary, the present paper aims at finding one factor to interpret one or more variables.
There are several techniques, named together as factor rotation, which make an orthogonal transformation of the
matrix L (factor loadings matrix) so that the interpretation of factors can be simplified. Reporting of the percentage
of variables is explained by each axe (factor), the rotation affects loadings (big loadings become bigger and small
loadings become smaller) even in individual values. However, the sum of individual values remains unchanged.
Once the factors are extracted, these techniques could be used bringing different results for the same communality
of data. Nevertheless, all analysts aim at building simple structures where each variable loads only in one factor.
Consequently, one variable could be interpreted only by one factor. Here in this paper, both Varimax and
Quartimax rotations are reported.
3. Choosing the number of factors
Choosing the number of common factors is very important. We draw a graphic of pairs (j, λ j), the “scree plot”,
and we observe the position in which this graphic begins to become “flat”. Another criterion to address the number
of factor problems is the Kaiser criterion. With this approach, a factor j is important when the eigenvalue is λ j> 1.
If the number of factors found by the Kaiser Test is equivalent to the number of factors, which have resulted
even from the “scree plot”, then we can continue with the other procedures, or otherwise, we must choose one of
the results already obtained. If one of the obtained results from the “scree plot” graphic is chosen, the procedures
and arrangement of the best number of factors must be repeated. The results change as the number of factors
changes. Available options include Kaiser’s “eigenvalues greater than one” rule, the scree plot, a priory theory, and
retaining the number of factors that give a high proportion of variance accounted for or that give the most
interpretable solution. The application of these two techniques is demonstrated in the paragraphs below.
4. Calculation of the Factor Scores
The score for everyone chosen in connections with factors is of great importance for FA methods when applied
for the calculation of factor loadings. Consequently, being of great importance, the new variables that correspond
to the factors could be added to our selection. SPPS builds a column for each factor extracted and then places the
scores of these factors for each subject inside this column. Once placed in the column, these scores can be used for
statistical analysis or simply to identify groups of subjects. After the evaluation of dispersion errors, the model ()
can calculate scores (the factor values). The Statistical package SPSS contains three different methods of
evaluation: Regression, Bartlett (Johnson and Wichern, 1982) and Anderson-Rubin.

5. Data Application of Factorial Analysis (FA) with PCA method

 The set of data psychological testing helps report the usefulness of the techniques. The set of data in
psychological testing (Revelle, 2010) provides
information for n=1000 individuals. It contains
three dependent variables: Prelim, GPA, and
MA, and five predictor variables. The variables
of the set are in Table 1 reported. Determining a
smaller number of uncorrelated variables to
describe the data is of great interest.

Initially the PCA method was used to

calculate factors with Varimax rotation method. Correlation
matrix reports that the eigenvalues values are greater than 1,
which is a means to address the choice of the number of factors.
Table-2 shows couples of variables (e.g. GREV, GREQ and
GREA) which are better correlated with each other.
regarding the Meyer-Olkin (KMO) statistical value which in
this case is 0.657, see Table 3, we see the Kaiser- certain
correlation
of data (Hair et al., 2010) is reported.

Table 4 shows the proportion of variability which is firstly

explained with all factors together and then only with the
factors before and after rotation. The result shows the two
first common factors, which explain 81% of the total
variance, a quite good percentage. After the rotation
method, this percentage does not change, but it changes to
the percentage that explains each factor. Specifically, these
percentages are transformed to reduce the differences
between them after rotation.
Another analysis (scree plot, fig 1) confirms the
conclusions: the graphic representation of couples is in the
same order as reported in Table 4. Separation processes of
the two eigenvalues greater than 1 in conjunction with
those remaining.