RESEARCH PAPER

COMPETENCY STANDARDS FOR CRITICAL CARE NURSES:

DO THEY MEASURE UP?
Murray J. Fisher, RN, ICCert, DipAppSc(Nurs), BHSc(Nurs), ACKNOWLEDGEMENTS
MHPEd, MRCNA, Lecturer, Faculty of Nursing and Midwifery, The authors wish to acknowledge the NSW Nurses’ Registration Board and the
University of Sydney, Australia Australian College of Critical Care Nurses for their funding contributions to this
mfisher@nursing.usyd.edu.au study. We would like to thank Dr Leanne Cowin for her expertise and advice on
statistical analysis and initial development of this manuscript. Many thanks to
Andrea P. Marshall, RN, ICCert, BN, MN(Research), GradCertEd Professor Doug Elliott, Associate Professor Sandra West and Dr Martha Curley
Stud(Higher Ed), Senior Research Fellow, Critical Care Nursing for their critique and constructive comments.
Professorial Unit, Royal North Shore Hospital and University of
Technology, Sydney, PhD candidate, Faculty of Nursing and
Midwifery, University of Sydney, Australia

Tina S. Kendrick, RN, PaedICCert, MN, BN(Hon), Clinical Nurse

Specialist, Sydney Children’s Hospital, Honorary Research
Fellow, University of Technology, Sydney, Australia

Accepted for publication March 2004

Key words: competencies, competency standards, construct validity, critical care

ABSTRACT Results:
Statistically there was no support for the current
Objective: structure for the ACCCN competencies because the
To determine the construct validity of the elements did not fit uniquely to a single competency,
Australian College of Critical Care Nurses (ACCCN) but were multidimensional and loaded across several
competency standards as a tool for assessing the competencies. Competency statements also loaded
clinical practice of specialist level critical care nurses across several domains. Modification of the current
in Australia. model resulted in the identification of a four-factor
competency model, which demonstrated reasonable
Design: model fit.
A comparative descriptive design was used to
examine the relationship between the domains, Conclusion:
competencies and elements of the ACCCN competency Several issues are highlighted, resulting in concerns
standards. Participants were sent a questionnaire and regarding the validity of the ACCCN Competency
asked to describe on a 7-point Likert scale how closely Elements and Standards as a tool with which to assess
each competency statement and related elements the practice of critical care nurses.
reflected their level of critical care nursing practice.

Subjects:
INTRODUCTION
A systematic sampling method was used to In 1996 ACCCN developed competency standards for
randomly select 1000 critical care nurses from a Australian specialist level critical care nurses (ACCCN
prelisting of members of ACCCN. A total of 532 competency standards 1996) from a multicentre
completed questionnaires were returned. observational study (Confederation of Australian Critical
Care Nurses 1996). This development was in keeping
Main outcome measure: with a move toward competency-based standards for
The purpose of this study was to determine the industry and professions in addition to growing debate
construct validity of the ACCCN competency that beginning level competencies did not adequately
standards by examining two structural models. The capture more advanced nursing practice (McMillan et al
1997; Nursing Competencies Assessment Project 1990).
first examined how well the descriptive elements fit
with their respective competency standard. The second The structure of the ACCCN competency standards is
model examined how well the competency standards three-tiered and includes elements, competencies and
group together under specific domains. domains. ACCCN defines elements as related aspects of

Australian Journal of Advanced Nursing 32 2005 Volume 22 Number 4

 RESEARCH PAPER

performance that collectively provide evidence for a competence of specialist level critical care nurses must at
specific competency. All elements of a competency must best be viewed as problematic.
be considered before inferences about the competency of
The purpose of this study was to determine the
an individual can be made. The competencies are
construct validity of the ACCCN competency standards as
attributes of a specialist nurse who functions at a high
a tool for assessing the clinical practice of specialist level
level of performance. Competency statements are
critical care nurses in Australia. It was hypothesised that a
grouped according to related facets of specialist practice
structural model using Confirmatory Factor Analysis
known as domains. These domains include enabling,
should represent the theoretical construct of the ACCCN
clinical problem solving, professional practice, reflective
competency standards. This testing of the structural
practice, teamwork and leadership (see figure 1) (ACCCN
model of the ACCCN Competency Standards will in turn
2002).
inform further development and refinement of tools for
The content validity of these standards has recently assessing the clinical competence of critical care nurses.
been examined (Greenwood et al 2001), however the
construct validity has not been determined. Construct
validity examines how well the conceptual theoretical
METHOD
definition, or in this case the structure of the competency A comparative descriptive design was used to examine
standards, fits with the operational definition of measured relationships between the domains, competencies and
variables. That is, do the elements and competency elements of the ACCCN competency standards. This
statements adequately measure the construct we call study examined two structural models. Model one
competence? While the ACCCN competency standards examined the theoretical construct of the elements within
were not developed as a tool to measure clinical practice the ACCCN competency standards by determining the
directly, many hospitals (Liverpool Health Service 2003), degree of fit these elements have with their respective
universities (University of Sydney 2001) and professional competency factor within the sample. Model two
bodies (Underwood et al 1999) use them as a framework examined the degree of fit of the ACCCN competency
for the assessment of clinical performance (Fisher and standards with their respective domains. The Human
Parolin 2000). To date, little research has been undertaken Ethics Committee of the University of Sydney granted
to examine the suitability of competency standards for use ethics approval.
in the assessment of clinical practice (Williams et al 2001;
Fisher and Parolin 2000) despite an articulated need for Participants
this to occur (Kendrick et al 2000). Without determining The sampling procedure was designed to establish a
the construct validity the claim that the ACCCN representative sample of 1000 Australian critical care
competency standards can be used to measure clinical nurses. Thus a systematic sampling technique was used to

Figure 1: The relationship between elements, competencies and domains of the ACCCN competency standards

Domain
Clinical problem solving

C8 - Integrated comprehensive patient C9 - Evaluates and responds effectively to C10 - Develops and manages a plan of
assessment and interpretive skills to changing situations care to achieve desired outcomes
Competencies achieve optimal patient outcome

E9.1 Initiates pre-emptive E10.1 Formulates and

E8.1 Gathers, analyses and interventions in anticipation of implements a plan of care
integrates data from a variety of potential patient complications incorporating specialised
Elements sources and determines the knowledge, to achieve desired
significance of findings outcomes
E9.2 Analyses alterations in
physiological parameters and
intervenes appropriately E10.2 Assesses effectiveness
of nursing management in
achieving desired outcomes
E9.3 Effectively anticipates and and reviews plan in
manages emergency situations accordance with evaluated
data

E10.3 Effectively plans

continuity of care

Australian Journal of Advanced Nursing 33 2005 Volume 22 Number 4

 RESEARCH PAPER

identify participants from the ACCCN membership RESULTS

database. Coded questionnaires were then mailed to the
identified sample. A follow-up letter and replacement Model 1: The Elements Model
questionnaire was sent to non-respondents to maximise Model one examined the theoretical construct of the 58
the response rate. This procedure achieved a response rate elements by determining the degree of fit these elements
of 54% (n=540). Direct comparisons between the have with the 20 competency factors.
respondents and all Australian critical care nurses could
Descriptive statistics at item and factor level for
not be established as no central repository of
the elements model
demographic data for Australian critical care nurses is
held outside the ACCCN membership database. The results of a descriptive analysis at the element and
Consequently, the representativeness of the sample was competency level for the element’s model revealed a
unable to be determined, however the use of a random narrow dispersion range suggesting non-normality in the
sampling technique coupled with a response rate of over data. Internal consistency (reliability) analyses
50% is considered sufficient to achieve adequate demonstrated good scores (α0.6) for all factors that
representation of the ACCCN membership. contained more than one item.

Instrument Correlation analysis at the element item and factor

level
The initial section of the questionnaire asked subjects
for demographic information. The second section listed A Pearson’s correlation coefficient generated for each
item and proposed factor revealed that a number of
58 elements of ACCCN competencies and 20 competency
elements correlated more with other competencies than
statements. Participants were asked to describe on a 7-
their own. Due to the propensity of elements to correlate
point Likert scale (where 1 = never or almost never true
more with non-theoretically determined factors, it was
and 7 = always or almost always true) how closely each
impossible to determine any factor structure from the
competency statement and the related elements reflected
results of the correlation matrix. Therefore, both
their view of their level of critical care nursing practice.
exploratory and confirmatory factor analyses were
Data analysis undertaken.

Both exploratory factor analyses (EFA) and Exploratory factor analysis of the elements model
confirmatory factor analyses (CFA) were performed to An exploratory factor analysis using PCA with a
test the two independent ‘a priori’ models. These analyses Varimax rotation, revealed 10 factors with eigenvalues
were conducted firstly, to determine if the elements of over one. The 10-factor model accounted for 64% of the
each competency fit their proposed competency factor variance. The exploratory factor analysis did not support
and secondly, the degree to which the competency the 20 theoretically proposed competency factors for the
statements load to particular domains and thus the elements model. All element items loaded onto the first
construct of competence. Model fits were determined by factor (>0.3) and were split across the other nine factors
utilising the Tucker Lewis Index (TLI), the Relative Non- in a random pattern. No factor structure was discernable.
Centrality Index (RNI) and the Root Mean Square Error
of Approximation (RMSEA). An acceptable model fit Confirmatory Factor Analysis (CFA) of the elements
utilising the TLI and RNI as fit indices is supported by model
results >0.90. The RMSEA is accepted as reasonable if A CFA of the elements was conducted by allowing
<0.07 and good if <0.05 (Holmes-Smith 2002). By each element to load only onto the hypothesised latent
examining correlation coefficients and modification factor (competency) (table 1). Results based on the CFA
indices the researchers are able to make recommendations demonstrate that the factor loadings were generally
for model (competency) re-specifications. greater than 0.6. Six elements demonstrated target
loadings less than 0.6, indicating that a significant portion
Principal Component Analysis (PCA) with Varimax of the variance of these elements is not accounted for by
rotation procedure and Principal Axis Factoring (PAF) their respective competency factor.
with an Oblimin rotation method were used to search for
groups of items that have variance in common. To The confirmatory factor analysis revealed high
determine internal consistency, Cronbach’s coefficient correlations between competency factors (Table 2). A
alphas were also calculated at the item and scale levels. third of competency factors were correlated greater than
0.70 with other competency factors, suggesting there is
Exploratory factor analyses and reliability of the scales little difference between the competency factors. For
were examined using SPSS version 10 (SPSS Inc 2000). example, C1 was highly correlated with C2 (>0.90)
LISREL 8.0, (Jöreskog and Sörbom 1993) and AMOS suggesting that there is no statistical difference between
version 4.0 (Arbuckle 1997) were used to conduct the C1 and C2. Confirmatory factor analysis testing of the
confirmatory factor analyses. ACCCN element model in this sample showed

Australian Journal of Advanced Nursing 34 2005 Volume 22 Number 4

 RESEARCH PAPER

unacceptable fit (χ2=4436.02, df=1405; TLI=0.81; 0.67). Cronbach’s alpha was examined at competency
RNI=0.84; RMSEA=0.06). item and domain levels. Only those domains with three or
more competencies produced item estimations. The
Model 2: The competency model results at competency level reveal consistent results
Model two (figure 2) examined the degree of fit of the within the proposed domain. Those domains with only
twenty ACCCN competency standards with the six two competency items demonstrated the lowest alpha
domains: enabling, clinical problem solving, professional scores. The domain ‘Reflective Practice’ (C15 and C16)
practice, reflective practice, teamwork and leadership revealed a low alpha score (α=0.34) demonstrating
(Australian College of Critical Care Nurses 2002). unacceptable internal consistency for this factor.

Descriptive statistics at item and factor level for the Results of a correlation analysis at competency item
competency model and factor level
In a similar manner to the Elements Model, the item Pearson’s coefficients generally demonstrated
(competency) means fell within a narrow range (6.12 to acceptable correlation for each competency and domain:
6.77). Standard deviations were at a low and narrow range Enabling - 0.31 to 0.56; Clinical Problem Solving - 0.52
for both competencies (0.49 to 1.0) and domains (0.49 to to 0.61; Professional Practice - 0.38 to 0.49; Teamwork -

Table 1: Results of the Confirmatory Factor Analysis for the ACCCN competency element subscales (n=532)

ACCCN Competency element subscale factor loadings

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20

E1.1 0.58 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E1.2 0.69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E1.3 0.74 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E1.4 0.65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E2.1 0 0.63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E2.2 0 0.69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E2.3 0 0.73 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E2.4 0 0.77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E3.1 0 0 0.91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E4.1 0 0 0 0.91 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E5.1 0 0 0 0 0.59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E5.2 0 0 0 0 0.81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E5.3 0 0 0 0 0.83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E5.4 0 0 0 0 0.80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E6.1 0 0 0 0 0 0.68 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E6.2 0 0 0 0 0 0.76 0 0 0 0 0 0 0 0 0 0 0 0 0 0
E7.1 0 0 0 0 0 0 0.75 0 0 0 0 0 0 0 0 0 0 0 0 0
E7.2 0 0 0 0 0 0 0.81 0 0 0 0 0 0 0 0 0 0 0 0 0
E8.1 0 0 0 0 0 0 0 0.90 0 0 0 0 0 0 0 0 0 0 0 0
E9.1 0 0 0 0 0 0 0 0 0.79 0 0 0 0 0 0 0 0 0 0 0
E9.2 0 0 0 0 0 0 0 0 0.80 0 0 0 0 0 0 0 0 0 0 0
E9.3 0 0 0 0 0 0 0 0 0.77 0 0 0 0 0 0 0 0 0 0 0
E10.1 0 0 0 0 0 0 0 0 0 0.79 0 0 0 0 0 0 0 0 0 0
E10.2 0 0 0 0 0 0 0 0 0 0.77 0 0 0 0 0 0 0 0 0 0
E10.3 0 0 0 0 0 0 0 0 0 0.73 0 0 0 0 0 0 0 0 0 0
E11.1 0 0 0 0 0 0 0 0 0 0 0.72 0 0 0 0 0 0 0 0 0
E11.2 0 0 0 0 0 0 0 0 0 0 0.76 0 0 0 0 0 0 0 0 0
E11.3 0 0 0 0 0 0 0 0 0 0 0.69 0 0 0 0 0 0 0 0 0
E11.4 0 0 0 0 0 0 0 0 0 0 0.69 0 0 0 0 0 0 0 0 0
E11.5 0 0 0 0 0 0 0 0 0 0 0.66 0 0 0 0 0 0 0 0 0
E12.1 0 0 0 0 0 0 0 0 0 0 0 0.76 0 0 0 0 0 0 0 0
E12.2 0 0 0 0 0 0 0 0 0 0 0 0.79 0 0 0 0 0 0 0 0
E13.1 0 0 0 0 0 0 0 0 0 0 0 0 0.64 0 0 0 0 0 0 0
E13.2 0 0 0 0 0 0 0 0 0 0 0 0 0.64 0 0 0 0 0 0 0
E13.3 0 0 0 0 0 0 0 0 0 0 0 0 0.69 0 0 0 0 0 0 0
E14.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0.69 0 0 0 0 0 0
E14.2 0 0 0 0 0 0 0 0 0 0 0 0 0 0.75 0 0 0 0 0 0
E14.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0.66 0 0 0 0 0 0
E14.4 0 0 0 0 0 0 0 0 0 0 0 0 0 0.59 0 0 0 0 0 0
E15.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.68 0 0 0 0 0
E15.2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.72 0 0 0 0 0
E16.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.78 0 0 0 0
E16.2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.87 0 0 0 0
E16.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.78 0 0 0 0
E16.4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.78 0 0 0 0
E17.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.78 0 0 0
E17.2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.75 0 0 0
E17.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.59 0 0 0
E17.4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.78 0 0 0
E18.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.91 0 0
E19.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.77 0
E19.2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.75 0
E19.3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.68 0
E19.4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.77 0
E19.5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.36 0
E19.6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.63 0
E20.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.78
E20.2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.54

Australian Journal of Advanced Nursing 35 2005 Volume 22 Number 4

 RESEARCH PAPER

0.51 and Leadership - 0.48. However, the two results determined that the theoretical factors are not
competencies that theoretically constitute the Reflective supported in this analysis. Target loadings were modest to
Practice domain were only weakly correlated at 0.25. A substantial ranging from 0.32 to 0.93; however, the
number of competencies had higher correlations with competency standards did not load into pre-defined
domains other than their own, specifically those factors representing their respective domains. Cross
competencies in Reflective Practice, Teamwork and loadings occurred for eight of the 20 competency
Leadership. standards. The PAF results do not support the theoretical
structure of the ACCCN competency standards and their
All domains demonstrated significant correlations
domains. The ACCCN competency standards were
(<0.001). Overall, all the competency standards correlated
therefore further examined using confirmatory factor
more with their current assigned domain than with any
analysis.
other domain. The Enabling and Clinical Problem
Solving domains revealed the highest correlation value of
Figure 2: Competency construct components
0.78 whilst other correlations ranged from 0.49 to 0.72.
In a similar manner to the elements model, there is a
propensity of competencies to correlate with non- Measurement component Structural component
(Competency standards) (Domains)
theoretically determined factors, causing some difficulty
in determining an overall item to factor structure for the
competency model.

Results of an exploratory factor analysis of the

competency model
The competency items were entered into a factor
analysis (PCA with Varimax rotation) to assess any
theoretically derived factor structure. The factor analysis
revealed three factors with eigenvalues >1.0 accounting
for 56% of the variance. The competency model, which
is constructed from the elements model, theoretically
consists of six factors (domains); however these are
not supported in this analysis. Item factor loadings
greater than 0.3 were distributed evenly throughout the
three factors, however a number of item loadings
(competencies 4, 8, 14, 15 and 17) were split across all
three factors. No clear factor structure was discernable.
A further factor analysis was undertaken in an attempt
to clarify the factor structure. By utilising Principal Axis
Factoring (PAF) with an Oblimin rotation method and by
fixing items to their theoretically designated factors, the Note: C = Competency standards

Table 2: Confirmatory Factor Analysis Phi Index of competency factors (φ)

Factor correlations (φ)

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20

C1 1
C2 0.91 1
C3 0.66 0.68 1
C4 0.64 0.50 0.49 1
C5 0.56 0.56 0.50 0.58 1
C6 0.71 0.73 0.66 0.68 0.81 1
C7 0.63 0.67 0.59 0.50 0.65 0.85 1
C8 0.44 0.49 0.47 0.42 0.48 0.62 0.79 1
C9 0.63 0.64 0.58 0.65 0.71 0.89 0.85 0.69 1
C10 0.70 0.76 0.60 0.61 0.73 0.91 0.86 0.69 0.82 1
C11 0.61 0.67 0.56 0.54 0.54 0.72 0.70 0.50 0.66 0.74 1
C12 0.68 0.76 0.60 0.54 0.54 0.75 0.66 0.45 0.68 0.78 0.87 1
C13 0.53 0.52 0.62 0.62 0.59 0.76 0.70 0.52 0.76 0.72 0.73 0.79 1
C14 0.65 0.76 0.58 0.51 0.54 0.67 0.65 0.49 0.60 0.81 0.91 0.87 0.71 1
C15 0.65 0.64 0.60 0.52 0.56 0.73 0.72 0.53 0.69 0.73 0.72 0.71 0.80 0.76 1
C16 0.43 0.41 0.37 0.37 0.43 0.42 0.58 0.46 0.44 0.53 0.59 0.53 0.46 0.67 0.61 1
C17 0.66 0.68 0.56 0.63 0.64 0.73 0.69 0.54 0.71 0.79 0.67 0.77 0.79 0.74 0.76 0.59 1
C18 0.45 0.50 0.40 0.38 0.57 0.62 0.58 0.43 0.52 0.62 0.63 0.61 0.53 0.61 0.62 0.45 0.64 1
C19 0.64 0.66 0.55 0.55 0.66 0.77 0.75 0.55 0.69 0.82 0.78 0.75 0.75 0.81 0.88 0.70 0.81 0.75 1
C20 0.50 0.54 0.46 0.56 0.61 0.71 0.68 0.56 0.71 0.71 0.70 0.67 0.73 0.76 0.71 0.70 0.74 0.59 0.93 1

Note: All coefficients are presented in standardised format. All factor correlations greater than 0.2 are statistically significant (p<0.01)

Australian Journal of Advanced Nursing 36 2005 Volume 22 Number 4

 RESEARCH PAPER

Confirmatory factor analysis of the competency model Re-specification of the competency model
The results of the CFA demonstrated target loadings After careful assessment of the previous competency
greater than 0.5 for most competency standards (table 3). model and specifically taking into account correlation
The target loadings are highest for the domain of Clinical results at item and factor level as well as modification
Problem Solving (0.74 to 0.75) although no proposed indices in Structural Equation Modelling (SEM), a
factor revealed consistently high loadings of >0.75. The decision was made to collapse the six-factor model into
domain of Reflective Practice revealed low target four domains. The domains of Reflective Practice and
loadings ranging from 0.43 to 0.58. Team Work were collapsed, with competency items 15
and 17 moving to the factor of Clinical Problem Solving,
The correlations among the domains as seen in the Phi and competency items 16 and 18 moving to Leadership.
Index (table 4), represent a concerning array of results. The following results assess the viability of the re-
All domains appear correlated >0.79 with several over specified model using internal consistency scores, factor
0.90, suggesting that there is no statistical differences analyses and model fit indices. At this point, in order to
between these factors. Again, the results have not test the re-specified model a new sample would be
supported the proposed factor structure of the ACCCN valuable.
competency factors.
Internal consistencies for the four-factor model
The domain of Reflective Practice was highly
After collapsing the model, reliability analysis was
problematic, revealing correlations to other domains
performed on all competencies and domains. The results
greater than one. Given that a factor cannot correlate
reveal an improvement in overall reliabilities for the new
greater than one, the Phi Index results represent an
domains. Alpha scores were reasonable and demonstrated
improper solution. It is possible that the two-competency
good internal consistency at both the competency
domains are problematic, although they may not
(ranging from 0.62 to 0.82) and domain (ranging from
necessarily result in improper solutions. In order to 0.76 for Leadership to 0.84 for Enabling and Clinical
counteract this difficulty, the factor loadings for two- Problem Solving) levels.
competency domains can be constrained to be equal in the
initial analysis or the problematic domains may be Factor analyses
collapsed into larger domains. An EFA (PAF with Oblimin rotation) revealed that
Confirmatory Factor Analysis testing of the ACCCN most competencies still show a tendency to load onto the
Competency Model in this sample showed borderline first factor rather than into the four proposed factors. The
model fit (χ2=567.31, df=155; TLI=0.89; RNI=0.91; four-factor model accounted for 50% of the variance,
RMSEA=0.071). The effects of an improper solution which is a slight drop from the 56% of the current six-
from the Phi Index for the factor of Reflective Practice factor model.
may or may not have influenced the results of model fit The results of the CFA revealed reasonable target
that is below accepted standards. There is now strong loadings (>0.5) for all of the competency standards (table
evidence for the attempt of model respecification based 5). Two competency standards (1 and 16) demonstrated
on these results. factor loadings <0.6. The factor loading for item 15 has

Table 3: Confirmatory Factor Analysis for the ACCCN Competency Standards

Factor Loadings
Enabling Clinical Professional Practice Reflective Practice Teamwork Leadership
Problem Solving

C1 0.56 0 0 0 0 0
C2 0.61 0 0 0 0 0
C3 0.66 0 0 0 0 0
C4 0.69 0 0 0 0 0
C5 0.64 0 0 0 0 0
C6 0.67 0 0 0 0 0
C7 0.73 0 0 0 0 0
C8 0 0.75 0 0 0 0
C9 0 0.75 0 0 0 0
C10 0 0.74 0 0 0 0
C11 0 0 0.63 0 0 0
C12 0 0 0.70 0 0 0
C13 0 0 0.68 0 0 0
C14 0 0 0.68 0 0 0
C15 0 0 0 0.58 0 0
C16 0 0 0 0.43 0 0
C17 0 0 0 0 0.72 0
C18 0 0 0 0 0.71 0
C19 0 0 0 0 0 0.74
C20 0 0 0 0 0 0.66

Note: All coefficients are presented in standardised format. All factor correlations greater than 0.2 are statistically significant (p<0.01)

Australian Journal of Advanced Nursing 37 2005 Volume 22 Number 4

 RESEARCH PAPER

risen from the previous Reflective Practice loading of ACCCN membership database for the sampling frame has
0.58 to the current 0.64. The other Reflective Practice led to high item scores and low item variance. This
competency (16) added to the Leadership factor reflects the high level of experience (mean = 11.54 years;
demonstrated an improved factor loading from a previous SD=6.05) and critical care qualifications (92.3%) of the
score of 0.43 to 0.50 in the current model. sample. As the purpose of data analyses was to examine
the statistical model, non-normal data has a minimal
The results of factor correlations from the Phi Index of
effect on these results. In light of this, it is recommended
the CFA for the re-specified four-factor model still that another study using a more diverse sample be
demonstrated high correlations, most factors reveal scores conducted to determine if the re-specified model can be
>0.80 (table 6). substantiated.
While the revised model has determined improved Exploratory and confirmatory factor analyses for the
correlations to the previous six-factor model where an elements model revealed no discernable pattern between
improper solution was revealed, the high scores elements at the competency level. The elements are not
demonstrate that statistically there is little difference discrete and linear where an element fits uniquely to one
between domains. competency but are multidimensional and load across
Model fit indices for the re-specified model revealed a several competencies. These results are of considerable
TLI score of 0.91, a RNI score of 0.92, a RMSEA of concern as they provide strong statistical evidence that
0.068 and a χ2 of 564.46 with 164 df. The new model is a there is no match with the proposed theoretical structure.
substantial improvement from the previous six-factor An assessment of the competency model results has
model based on these results. Overall, the results of SEM revealed a number of difficulties relating to the ‘a priori’
have provided good evidence for the re-specified model. model. Specifically, the two item domains (Reflective
Practice and Team Work) have proven to be problematic
in exploratory and confirmatory factor analyses. The
CONCLUSION
factor of Reflective Practice performed poorly in all
The sampling strategy used in this study has created analyses. Firstly, correlations between the items and item
the effect of non-normal data distribution. The use of the to factor were low. CFA factor loadings for the

Table 4: Competency factor correlations in CFA Phi Index

Phi Index
Enabling Clinical Professional Practice Reflective Practice Teamwork Leadership
Problem Solving

EN 1
CPS 0.96 1
PP 0.90 0.86 1
RP 0.95 1.01 1.01 1
TW 0.88 0.89 0.90 1.09 1
Lead 0.79 0.84 0.82 1.14 0.96 1

Note: EN=Enabling; CPS =Clinical Problem Solving; PP= Professional Practice; RP= Reflective Practice; TW= Teamwork; Lead= Leadership.

Table 5: Confirmatory factor analysis for the four-factor model

ACCCN four factors

Enabling Clinical Professional Practice Leadership
Problem Solving

C1 0.56 0 0 0
C2 0.61 0 0 0
C3 0.66 0 0 0
C4 0.69 0 0 0
C5 0.64 0 0 0
C6 0.67 0 0 0
C7 0.73 0 0 0
C8 0 0.74 0 0
C9 0 0.73 0 0
C10 0 0.73 0 0
C15 0 0.64 0 0
C17 0 0.71 0 0
C11 0 0 0.63 0
C12 0 0 0.70 0
C13 0 0 0.68 0
C14 0 0 0.69 0
C16 0 0 0 0.50
C18 0 0 0 0.74
C19 0 0 0 0.76
C20 0 0 0 0.67

Australian Journal of Advanced Nursing 38 2005 Volume 22 Number 4

 RESEARCH PAPER

Table 6: Competency factor correlations Phi Index for the re-specified model

Phi Index
Enabling Clinical Professional Practice Leadership
Problem Solving

Enabling 1
Clinical Problem Solving 0.96 1
Professional Practice 0.90 0.90 1
Leadership 0.78 0.85 0.81 1

competencies within Reflective Practice were also low competency standards should not be static, but should be
(<0.6). Lastly, factor correlations between Reflective in a constant state of development and refinement.
Practice and the other domains led to an improper
The competencies do not appear to lend themselves
solution with correlations >1.00. These results are of readily to statistical assessment and any changes to the
concern, as they provided no statistical support for the competency factor structure based on construct validity
model. and reliability analyses present a danger of being
Problematic statistical issues have improved somewhat conducted without theoretical substantiation. Similarly, it
with the re-specified model. However, the issue of high would be unwise to continue with the use of these and
correlations between proposed domains continue, albeit similar competency standards to measure clinical
less than the theoretical model of six domains. Another performance without the exploration of their construct
important issue that should not become subsumed by the validity. It is strongly recommended that all future work
results of the statistical analyses is the fact that as yet in developing competency standards for nurses include
there is no theoretical support for a four-domain model. SEM prior to being used to assess clinical practice.
In the original study (CACCN 1996) the domains were
configured based on version 1 of the National REFERENCES
Competency Standards for the Registered Nurse (Nursing Arbuckle, J.A. 1997. AMOS users’ guide: Version 4. Chicago: Smallwaters.
Competencies Assessment Project 1990). None of the Australian College of Critical Care Nurses Inc. (ACCCN) 2002. Competency
competency standards developed for nurses in Australia standards for specialist critical care nurses. 2nd edn. Carlton: ACCCN.
have had their construct validity established so it may be Confederation of Australian Critical Care Nurses (CACCN). 1996. Competency
that the problems highlighted in the current study are standards for specialist critical care nurses. Subiaco: Inkpress.

present in all. Fisher, M. and Parolin, M. 2000. The reliability of measuring nursing clinical
performance using a competency based assessment tool: A pilot Study.
Collegian. 7(3):21-27.
The competency model, be it six or four domains, as a
Greenwood, M., Kendrick, T., Gill, F., Hale, N., Birkett, K. and Porter, J.
higher order model remains dependent on the model Sydney. 2001 World Federation of Societies of Intensive and Critical Care
performance at the elements level. Given that construct Medicine. 8th World Congress of Intensive and Critical Care Medicine.
Competency standards for specialist critical care nurses validation project.
validity support at the statistical level was poor for the
elements model, it is not surprising that difficulties Holmes-Smith, P. 2002. Introduction to structural equation modelling using
Lisrel. Canberra: Australian Consortium for Social and Political Research Inc.
continue to arise with the current educational reliance on
Jöreskog, K. and Sörbom, D. 1993. New features in LISREL 8. Chicago:
the competency model as a framework for the purposes of Scientific Software.
assessment. Having said this, the analyses for the Kendrick, T., Greenwood, M., Grech, C., Gill, F., Birkett, K. and Porter, J. 2000.
respecified model with four domains do represent an Credentialing Australian critical care nurses: The pilot study. Australian Critical
Care. 13(3):113-116.
improvement at the statistical level.
Liverpool Health Service. 2003. Graduate certificate in specialty nursing core
The elements model and competency model have been curriculum. Sydney: Liverpool Health Service.

examined for internal consistency, item and factor Marsh, H.W. 1987. The hierarchical structure of self-concept: An application of
hierarchical confirmatory factor analysis. Journal of Educational Measurement.
correlations, factor structure and model fit with the data. 24:17-39.
Several issues have been highlighted, resulting in McMillan, M., Keatinge, D., Little, P., Madjar, I., Bujack, E., Andrews, J. and
concerns regarding the validity of the ACCCN Johnson, R. 1997. Competency Standards for the Advanced Nurse. Melbourne:
Australian Nursing Federation.
Competency Elements and Standards as a tool with which
Nursing Competencies Assessment Project. 1990. Report to the Australian
to assess nurses' work skills and knowledge. Marsh points nurse registering authorities conference. Adelaide: Nurses Board of South
out that ‘theory building and instrument construction are Australia.
inexorably intertwined and that each will suffer if the two SPSS Inc 1999. SPSS version 10.0.5. Chicago: SPSS Inc.
are separated’ (1987, p.19). Marsh’s warnings are The University of Sydney. 2001. Clinical assessment tools workbook. Sydney:
University of Sydney.
specifically applicable here. It is acknowledged that the
Underwood, M., Elliott, D., Aitken, L., Austen, D., Currey, J., Field, T., Grech,
development of these competencies standards is based on C., Kiegaldie, D., Marshall, A., Pittard, M. and Worral-Carter, L. 1999.
the direct observation of clinical practice. While this is Australian College of Critical Care Nurses Ltd.: Position statement on
postgraduate critical care nursing education. October 1999. Australian Critical
important for their development, empirical research such Care. 12(4):160-164.
as described here should be included in the development Williams, A., Wellard S.J. and Bethune E. 2001. Assessing Australian
process. Furthermore the content and construct validity of undergraduate clinical learning. Collegian. 8(4):9-13.

Australian Journal of Advanced Nursing 39 2005 Volume 22 Number 4